JP7405528B2 - Media discrimination device, medium discrimination system, and medium discrimination method - Google Patents

Description

The present invention relates to a medium discrimination device, a medium discrimination system, and a medium discrimination method.

2. Description of the Related Art Conventionally, a technique is known for determining the type of medium, such as a form, on which characters are written based on an image of the medium.

For example, in Patent Document 1, features of vertical ruled lines and horizontal ruled lines are extracted from a form image, a plurality of promising candidate forms are selected by comparing the extracted ruled line features, and printing is performed at a predetermined position of the selected promising candidate forms. A method is disclosed that outputs a form type determination result based on the degree of matching between a character and a character recognized in a corresponding area on a form image. Further, Patent Document 1 describes that the inclination of the document image is corrected so that the vertical ruled lines and horizontal ruled lines of the imported document image are parallel to the reference direction (for example, see paragraphs [0021] and [0032]). , if there is a positional shift between the likely candidate form and the form image, the area where the printed characters of the likely candidate form are printed should be corrected in consideration of this positional deviation (for example, paragraphs [0030], [0067] -0070]), normalization by compressing the histogram based on the edges of ruled lines to make it easier to compare with the standard histogram (see, for example, paragraphs [0038], [0046], and [0066]), etc. has been done.

JP2014-16921A

However, in the method described in Patent Document 1, if the rotation angle of the form image is too large, it is difficult to accurately determine the type of the form even if the above-described tilt correction and positional deviation correction are performed. In addition, when capturing a form image with a camera rather than a scanner, if the edge of the form cannot be detected, such as when the edge of the form protrudes from the image, it may not be possible to determine the type of the form, even if the above-mentioned normalization is performed. There is. Furthermore, when capturing a form image with a camera rather than a scanner, the type of the form may not be determined because the size (resolution) of the form image does not match depending on the shooting situation. In the first place, the method described in Patent Document 1 cannot determine the type of medium without ruled lines.

The present invention has been made in view of the above-mentioned current situation, and provides a medium discriminating device, a medium discriminating system, and a medium discriminating method that are capable of discriminating the types of various media with high precision regardless of the orientation and size in which images are taken. The purpose is to provide the following.

In order to solve the above-mentioned problems and achieve the objects, the present invention is a medium discrimination device, which includes a feature detection unit that detects a plurality of characteristic parts of the medium based on an image of the medium, and a feature detection unit that detects a plurality of characteristic parts of the medium based on an image of the medium. The present invention is characterized by comprising a type determination unit that determines the type of the medium based on the characteristic portion.

Further, in the present invention, in the above invention, the medium discrimination device specifies a character recognition area to be subjected to character recognition based on the relative positions between the plurality of characteristic parts and the type of the medium. The present invention is characterized in that it further includes a recognition area specifying section and a character recognition section that recognizes characters within the character recognition area.

Further, in the present invention, in the above invention, the medium discrimination device further includes a character recognition orientation correction unit that corrects the orientation of the character recognition area based on the relative position between the plurality of characteristic parts, The character recognition unit is characterized in that it recognizes characters within the character recognition area whose orientation has been corrected.

Further, in the above invention, the present invention provides that the medium discrimination device displays the image displayed on the display unit based on the relative position between the display unit that displays the image of the medium and the plurality of characteristic parts. The display device further includes a display orientation correction unit that corrects the orientation of the display.

Further, the present invention is characterized in that, in the above invention, the feature detection unit detects the plurality of feature parts using a machine-learned inference model.

Further, in the above invention, the present invention provides that the medium determining device determines the size of the medium based on the plurality of characteristic parts and the type of the medium, and that the entire medium is in the image. The device is characterized by further comprising a determination unit that determines whether or not an image is being captured inside.

Further, the present invention is a medium discrimination system characterized by comprising the medium discrimination device and an imaging device that captures an image of the medium.

The present invention also provides a medium discrimination method, which includes a feature detection step of detecting a plurality of characteristic portions of the medium based on an image of the medium, and a step of determining the type of the medium based on the plurality of characteristic portions. a recognition area specifying step of specifying a character recognition area to be subjected to character recognition based on the relative positions between the plurality of characteristic parts and the type of the medium; A character recognition step of recognizing characters within.

According to the medium discriminating device, medium discriminating system, and medium discriminating method of the present invention, the types of various media can be discriminated with high precision regardless of the orientation and size of images of the media.

It is a schematic diagram for explaining an overview of a medium discrimination method in Embodiment 1, and shows a rectangular bounding box set for multiple characteristic parts of a medium in the machine learning stage and a character recognition target for character recognition. area. 2 is a schematic diagram for explaining an overview of a medium discrimination method in Embodiment 1, showing a plurality of characteristic parts detected in the medium discrimination stage and a character recognition area specified based on the plurality of detected characteristic parts. FIG. shows. 1 is a diagram illustrating the overall configuration of a medium discrimination system according to a first embodiment; FIG. 1 is a block diagram illustrating the configuration of a medium discrimination device according to a first embodiment. FIG. 2 is a diagram for explaining a method for specifying a character recognition area in Embodiment 1, in which (a) shows a plurality of characteristic parts and reference positions of the character recognition area, and (b) shows positions detected by a feature detection unit; FIG. The positions of the plurality of characteristic parts and the position of the character recognition area specified by the recognition area specifying section are shown. 3 is a flowchart illustrating an example of a procedure of a medium discrimination process performed by the medium discrimination system according to the first embodiment. It is a figure explaining the whole structure of a medium discrimination system concerning a modification.

DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of a medium discrimination device, a medium discrimination system, and a medium discrimination method according to the present invention will be described below with reference to the drawings. In the present invention, the specific types of media that are determined by the media discriminating device, the media discriminating system, and the media discriminating method are not particularly limited. Examples include paper media such as receipts issued at distribution stores, prescriptions, admission tickets to theme parks, etc., gift certificates, and gift certificates.

<Overview of media discrimination method>
First, an overview of the medium discrimination method in the first embodiment will be explained. In this embodiment, the type of medium is determined from the image of the medium to be determined by registering and machine learning in advance multiple characteristic parts necessary for determining the type of medium, and furthermore, the type of medium is determined from the image of the medium to be determined. Recognizes characters in the part you want to read. Also, at this time, the orientation of the medium is determined based on the recognized positional relationship of the plurality of characteristic parts, and the image of the medium is rotated and corrected in an orientation in which the characters can be read. A more detailed explanation will be given below using FIGS. 1 and 2.

First, in the machine learning stage, as shown in FIG. 1, various media M are photographed in advance, and rectangular bounding boxes are set for each medium in a plurality of feature parts Ma used for determining keywords, logos, etc. Teacher data is created based on the settings, and each feature Ma is learned using a machine learning algorithm called Single Shot Multibox Detector (SSD). In addition, as shown in FIG. 1, for each medium, the relative positional relationship between each characteristic part Ma and the character recognition area Mb that is the target of character recognition is specified and registered in the standard information of the medium. I'll keep it.

Next, in the medium discrimination stage, as shown in FIG. The medium M is determined based on the combination of the plurality of characteristic parts Ma. Next, a character recognition area Mb is specified based on the relative positional relationship between the plurality of detected feature parts Ma and the reference information. Finally, the characters within the specified character recognition area Mb are recognized. Note that the number of character recognition areas Mb to be specified and recognized may be one or more.

According to the present embodiment, since the characteristic portion is detected using the inference program (trained model), the characteristic portion can be detected regardless of conditions such as the orientation and size (image size) of the medium. Furthermore, since the medium is discriminated only from the characteristic parts, it is less susceptible to the influence of parts other than the characteristic parts, such as handwritten parts and stamp parts, which vary depending on the medium.

<Overall configuration of media discrimination system>
Next, the overall configuration of the medium discrimination system of this embodiment will be described using FIG. 3. As shown in FIG. 3, the medium discrimination system 1 of this embodiment includes a camera 2 as an imaging device that captures an image of the medium M, and a medium discrimination device 3. The camera 2 is communicably connected to the medium discrimination device 3. In this embodiment, as shown in FIG. 3, the camera 1 images the medium M placed on the top surface of the reading table 6, which has a flat top surface.

The camera 2 is fixed above the reading table 6 and captures an image of a predetermined area including the upper surface of the reading table 6, in this case a moving image (video). The photographed video (moving image) is output to the medium discrimination device 3. The camera 2 may acquire RGB color images or monochrome images.

The medium discrimination device 3 is capable of communicating with a monitor (display device) 4 serving as a display section for displaying photographed images, etc., and an input device 5 (for example, a keyboard, a mouse, etc.) through which an operator performs various input operations. It is connected. Note that the monitor 4 and the input device 5 may be configured from a display device with an input function such as a touch panel display.

The medium discrimination device 3 is configured so that the operator can view the video imaged by the camera 2 on the monitor 4 in real time.

<Configuration of medium discrimination device>
Next, the configuration of the medium discrimination device 3 will be further explained using FIG. 4. The medium discrimination device 3 is composed of an information processing device having functions equivalent to a general personal computer, and includes a control section 10 and a storage section 20, as shown in FIG.

The control unit 10 includes a video input unit 11, a feature detection unit 12, a type determination unit 13, a recognition area identification unit 14, a character recognition orientation correction unit 15, a character recognition unit 16, and a display orientation correction unit. 17 and a determination unit 18. The control unit 10 includes, for example, a software program for implementing various processes, a CPU (Central Processing Unit) that executes the software program, and various hardware controlled by the CPU. In order to speed up processing during machine learning and when executing a trained model, the control unit 10 may include hardware such as a GPU (Graphics Processing Unit) in addition to the CPU. Software programs and data necessary for the operation of the control unit 10 are stored in the storage unit 20.

The storage unit 20 is composed of a storage device such as a hard disk device or a non-volatile memory, and stores an inference model 21 and reference information 22.

The video input unit 11 performs a process of acquiring video from the camera 2 and outputs the video to the feature detection unit 12.

The feature detection unit 12 detects a plurality of feature parts of the medium based on the video (image) of the medium output from the video input unit 11 . That is, at least two portions that characterize the medium, such as specific character strings, are detected in a video of the medium. The feature detection unit 12 is realized using an object detection method that detects the position and category (class) of a specific object from an image. In other words, the feature detection unit 12 does not determine whether or not a feature exists at a predetermined position on the medium, but searches the entire medium and detects a pre-registered feature at an arbitrary position on the medium. It is something that can be found in a location and recognizes its type. Further, the feature detection unit 12 finds the feature part registered in advance on the medium regardless of its rotation angle and size (resolution), that is, regardless of the rotation angle and size (resolution) of the medium, and Recognize its type.

In this embodiment, the feature detection unit 12 detects (estimates) a plurality of feature parts using the inference model 21 stored in the storage unit 20.

Here, the inference model 21 will be explained. The inference model 21 is created by supervised machine learning of a data set (teacher data) to which label information (correct data) is attached. More specifically, the inference model 21 uses an image of a medium (a two-dimensional still image) as input data, uses information such as the position and class (type) given to each feature of the medium as a label, and performs convolution. It is created by performing deep learning using a learning program using a neural network (CNN: Convolutional Neural Network). In this embodiment, an SSD is used as the convolutional neural network.

The inference model 21 created by supervised machine learning functions as an inference program (learned model) in which learned parameters are incorporated. Note that the learned parameters are parameters (coefficients) obtained as a result of learning using a data set. Also. The inference program applies the learned parameters obtained as a result of learning to the medium video (each still image that makes up the video) given as input, and calculates the results for the video (specifically, the above-mentioned This is a program that defines a series of calculation procedures for outputting the position, class, etc. of each characteristic part.

A dataset used for machine learning can be generated, for example, as follows. First, images (two-dimensional still images) of various media are acquired. At this time, images of the medium in various states are obtained by appropriately changing the rotation angle, shape (wrinkles or creases), presence or absence of handwriting or stamps, etc. Then, in each acquired image, by setting a rectangular bounding box for each characteristic part of each medium, the coordinates of the upper left and lower right of the bounding box are designated as the position of each characteristic part. Furthermore, a class (type) is set for each characteristic part. At this time, the positions and classes of at least two characteristic parts are registered for one medium. Further, at this time, an invalid area may be designated as an area that is not used for machine learning in each acquired image. As a result, if there are parts of the same type that vary depending on the medium, such as handwritten parts or stamp parts, such parts can be designated as invalid areas, making it possible to more effectively identify characteristic parts. can be learned. As a result of the above, a data set is generated for each image, in which the position and class of each feature are attached as label information. As a result, images of each characteristic part in various media are associated with the position and class of the characteristic part.

Note that the created inference model 21 may be additionally trained thereafter. That is, by applying a different data set to the inference model 21 and performing further learning, newly learned parameters are generated, and an inference program incorporating the newly learned parameters is used as the inference model 21. Good too.

Furthermore, although machine learning may be executed by a processor included in the medium discrimination device 3, it is preferably executed by a computer with excellent arithmetic processing ability, such as a dedicated server or a server on a cloud.

The type determining unit 13 determines the type of medium based on the plurality of features detected by the feature detecting unit 12.

More specifically, the reference information 22 includes, for each type of medium, combination information indicating a combination of a plurality of characteristic parts that the medium has, and the type discrimination unit 13 includes combination information indicating a combination of a plurality of characteristic parts that the medium has. The type of medium is determined by comparing the combination of characteristic parts with the combination information.

Note that although the combination of classes of a plurality of feature parts is set so as not to match between different types of media, some classes of feature parts may overlap between different types of media. Specifically, for example, there are four classes of feature parts, a to d, and the combination of feature parts of medium A is (a, b), and the combination of feature parts of medium B is (c, d). , and the combination of the characteristic parts of medium C may be (a, c). Of course, the classes of feature parts do not need to overlap at all between different types of media. That is, for example, there are six classes of feature parts a to f, the combination of feature parts of medium A is (a, b), the combination of feature parts of medium B is (c, d), The combination of features of medium C may be (e, f).

Additionally, the number of combinations of features may vary between different types of media. For example, for some media, the type of the medium is determined based on a combination of two feature classes; for other media, the type of the media is determined based on a combination of three or more feature classes. May be determined.

In this way, a feature of a medium may be any feature that can identify the type of medium by a combination of multiple features, such as a specific character string such as a keyword or title, a design, a logo, etc. It will be done. Each characteristic portion may be determined artificially in advance, or may be determined mechanically in advance. In the latter case, for example, image processing may be performed to extract a portion with small variations from images of multiple media of the same type, and that portion may be determined as the characteristic portion. As a result, when there are portions of the same type that vary depending on the medium, such as a handwritten portion or a stamp portion, it is possible to easily determine a characteristic portion that is more effective for detection.

The recognition area specifying unit 14 recognizes a character as a character recognition target based on the relative position between the plurality of feature parts detected by the feature detecting unit 12 and the type of medium determined by the type determining unit 13. Identify the area.

More specifically, the reference information 22 includes position information indicating the respective positions of each characteristic portion and character recognition area serving as a reference for each type of medium, and the recognition area specifying unit 14 The position of the character recognition area is calculated based on the relative positional relationship between the characters and the positional information corresponding to the type of the medium.

To explain more specifically using FIG. 5, for example, the position information of the reference information 22 includes the reference positions (coordinates) of each of the two characteristic parts A and B, as shown in FIG. 5(a), The recognition area specifying unit 14 determines the coordinates of the feature parts A' and B' detected by the feature detection unit 12 (see FIG. 5(b)). , the features A' and B' are translated in parallel while maintaining the relative positional relationship so that the position of one feature A' matches the reference position of the corresponding feature A (Fig. 5(a) )reference). Then, as shown in FIG. 5(a), the rotation angle θ in the direction from feature A'(A) to feature B' with respect to the direction from feature A'(A) to feature B, which has been matched, is determined. calculate. This rotation angle θ corresponds to the rotation angle of the imaged medium with respect to the reference direction. Further, the ratio R of the length L from the feature part A to the feature part B and the length L' from the feature part A' to the feature part B' is calculated (L'=R×L). This length ratio R corresponds to the magnification of the imaged medium relative to the reference image. Thereafter, as shown in FIG. 5B, the recognition area specifying unit 14 determines the relative position of the character recognition area C with respect to the feature part A (from the feature part A to the character recognition area C) based on the position information of the reference information 22. The position of the temporary character recognition area C'' with respect to the feature A' detected by the feature detection unit 12 is calculated from the direction and length ratio R), and the character recognition area C'' is calculated with the feature A' as the center. The position of the target character recognition area C' is calculated by rotating the position by the rotation angle θ.

Note that the medium discriminating device 3 may be configured to be able to display output results from the feature detecting section 12 and the recognition area specifying section 14 on the monitor 4. For example, as shown in FIG. 2, a bounding box surrounding each characteristic portion Ma and each character recognition area Mb may be overlaid displayed on the image of the medium M on the monitor.

The character recognition orientation correction unit 15 corrects the orientation of the character recognition area specified by the recognition area identification unit 14 based on the relative positions between the plurality of feature parts detected by the feature detection unit 12. Thereby, it is possible to reduce the rate at which the character recognition unit 16 incorrectly recognizes characters within the character recognition area. For example, if the character string to be recognized is "069", if the medium is placed in the opposite direction, the character string will be mistakenly recognized as "690". By correcting the orientation of the character recognition area by the recognition orientation correction unit 15, it is possible to prevent such erroneous recognition.

More specifically, the character recognition orientation correction unit 15 calculates a rotation angle in the opposite direction to the rotation angle θ calculated by the recognition area identification unit 14 based on the relative positions between the plurality of features as described above. Affine transformation is performed on the character recognition area so that it is rotated by (-θ).

The character recognition unit 16 recognizes characters (optical character recognition: OCR) within the character recognition area whose orientation has been corrected by the character recognition orientation correction unit 15.

More specifically, the storage unit 20 stores all types of characters that can be used in the character recognition area of the medium to be determined as character images, and the character recognition unit 16 stores the character recognition area whose orientation has been corrected. Each constituent character of the character string within is compared with a character image to identify each constituent character, and finally, by combining these identified characters, the character string within the character recognition area is recognized.

The display orientation correction unit 17 corrects the orientation of the image of the medium displayed on the monitor 4 based on the relative positions between the plurality of feature parts detected by the feature detection unit 12. This allows the image of the medium to be displayed on the monitor 4 in an orientation that makes it easy for the operator to visually recognize the characters within the character recognition area.

More specifically, the display orientation correction unit 17 calculates a rotation angle ( The entire image on the medium is affine transformed so that it is rotated by -θ).

The determining unit 18 determines the size of the medium based on the plurality of characteristic parts detected by the characteristic detecting unit 12 and the type of medium determined by the type determining unit 13, and determines whether the entire medium is It is determined whether the image is captured in the video. This makes it possible to notify the operator of an error message when a portion of the medium, for example, a character recognition area, is not imaged.

More specifically, the reference information 22 includes, for each type of medium, first medium area information indicating a medium area that defines the external shape of the medium, and relative information of the medium area with respect to each feature corresponding to the medium area. Based on the first medium area information corresponding to the type of medium determined by the type determining unit 13, the determining unit 18 determines the medium of the medium based on the first medium area information corresponding to the type of medium determined by the type determining unit 13. The size of the medium is determined by specifying the area. Further, the determining unit 18 determines the position of each characteristic portion detected by the characteristic detecting unit 12 and the second medium area information corresponding to the type of medium determined by the type determining unit 13. The position of the medium area in the video is identified, and it is determined whether the entire medium is captured in the video.

The determining unit 18 determines the character recognition area based on the plurality of characteristic parts detected by the characteristic detecting unit 12, the type of medium determined by the type determining unit 13, and the character recognition area specified by the recognition area specifying unit 14. It may be determined whether the entire character recognition area is captured in the video. Thereby, when at least a part of the character recognition area is not imaged, it is possible to notify the operator of an error message to notify this fact.

In this case, the determining unit 18 determines whether the entire character recognition area is captured in the video based on the position of the character recognition area specified by the recognition area specifying unit 14.

<Procedure for media discrimination processing>
Next, the procedure of the medium discrimination process performed by the medium discrimination system 1 will be explained using FIG.

As shown in FIG. 6, first, a video of a medium is input from the camera 2 to the video input unit 11 (video input step S11).

Next, the feature detection unit 12 uses the inference model 21 to detect a plurality of feature parts of the medium from the video of the medium input in the video input step S11 (feature detection step S12). At this time, if only one characteristic part or no characteristic part is detected, the medium is treated as an unidentifiable medium. For example, processing is performed to notify the operator that the medium is an unidentifiable medium.

Next, the type determination unit 13 determines the type of medium based on the plurality of characteristic parts detected in the characteristic detection step S12 (type determination step S13). At this time, if the combination of all the detected feature parts matches the combination information of any registered medium type, the medium is determined to be of that type. On the other hand, if the combinations of all detected characteristic parts do not match the combination information of any registered medium type, the medium is treated as an unidentifiable medium. For example, processing is performed to notify the operator that the medium is an unidentifiable medium.

Next, the recognition area specifying unit 14 determines the character recognition target based on the relative positions between the plurality of feature parts detected in the feature detection step S12 and the type of medium determined in the type discrimination step S13. A character recognition area is specified (recognition area specification step S14).

Next, the determination unit 18 determines the size of the medium based on the plurality of characteristic parts detected in the characteristic detection step S12 and the type of medium determined in the type determination step S13, and determines the size of the medium. It is determined whether the entire area is captured in the video (determination step S15).

In the determination step S15, the determination unit 18 determines the plurality of characteristic parts detected in the feature detection step S12, the type of medium determined in the type determination step S13, and the character recognition area specified in the recognition area determination step S14. Based on this, it may be determined whether the entire character recognition area is captured in the video.

Next, the character recognition orientation correction unit 15 corrects the orientation of the character recognition area specified in the recognition area identification step S14 based on the relative positions between the plurality of feature parts detected in the feature detection step S12. (Character recognition orientation correction step S16).

Next, the character recognition unit 16 recognizes the characters within the character recognition area whose orientation has been corrected in the recognition area correction step S16 (character recognition step S17).

Next, the display orientation correction unit 17 corrects the orientation of the image of the medium to be displayed on the monitor 4 based on the relative positions between the plurality of features detected in the feature detection step S12 (display orientation correction Step S18).

Then, the monitor 4 displays the image of the medium whose orientation has been corrected in the display orientation correction step S18 (display step S19), and the medium discrimination process ends.

Note that the processing execution timing of the display orientation correction step S18 and the display step S19 is not particularly limited as long as it is after the feature detection step S12, and can be changed as appropriate.

As described above, in the present embodiment, the feature detection unit 12 detects a plurality of feature parts of the medium based on the video (image) of the medium, and the type discrimination unit 13 detects the plurality of features detected by the feature detection unit 12. Since the type of medium is determined based on a plurality of characteristic parts, the characteristic detecting section 12 finds a plurality of pre-registered characteristic parts at any location on the medium regardless of their rotation angle, and In addition, the type of the medium can be recognized, and the type determination unit 13 can determine the type of the medium from a combination of the plurality of detected characteristic parts. Therefore, the types of various media can be determined regardless of the orientation in which images are taken. Additionally, since the type of media is determined based on multiple features, the type of media can be determined with higher accuracy than when determining the type of media based on a single feature. .

Further, in the present embodiment, the recognition area specifying unit 14 determines whether the character Since the character recognition area to be recognized is specified and the character recognition unit 16 recognizes the characters within the character recognition area specified by the recognition area identification unit 14, the area to be recognized can be accurately specified. In addition, character recognition can be performed within the area. Therefore, the characters in the target area can be accurately recognized regardless of the orientation of the medium or the image size.

Note that in the above embodiment, a case has been described in which the type of medium is determined based on a moving image (video) of the medium, but the image used may be a still image.

Further, in the above embodiment, a case has been described in which the inference model 21 is constructed by deep learning using a convolutional neural network, but the inference model 21 is not particularly limited as long as it is created by machine learning, and The model 21 may be created by machine learning other than deep learning. However, since the detection processing of media features can be applied to various targets, deep detection methods using convolutional neural networks, such as the SSD mentioned above, You Only Look Once (YOLO), and Regions with Convolutional Neural Networks (R-CNN), Preferably, it is constructed by learning. In particular, SSD is preferred.

Furthermore, in the above embodiment, a case has been described in which the feature detection unit 12 detects a plurality of feature parts using the inference model 21 created by machine learning, but the feature detection method by the feature detection unit 12 is Any object detection method that can detect an object regardless of its rotation angle and size (resolution) may be used, and is not particularly limited to a detection method that uses machine learning. For example, an object detection method using local features such as Scale-Invariant Feature Transform (SIFT) or Speeded-Up Robust Features (SURF) may be used.

Further, in the above embodiment, a case has been described in which the medium discriminating device 3 is configured as one device, but each function of the medium discriminating device 3 may be realized by a distributed processing system that is appropriately distributed among a plurality of devices.

Specifically, as shown in FIG. 7, for example, a camera 102, a cloud server 103 communicably connected to the camera 102, and a personal computer 104 equipped with a monitor communicably connected to the cloud server 103. A media discrimination system may be configured. The camera 102 is provided with the function of the feature detection unit 12 described above, and the cloud server 103 includes the type determination unit 13, the recognition area identification unit 14, the character recognition orientation correction unit 15, the character recognition unit 16, and the display orientation. It has the functions of the correction unit 17 and the determination unit 18, and uses the monitor of the personal computer 104 as a display unit to display the image of the medium and the output results from the feature detection unit 12 and the recognition area identification unit 14 on the monitor. Good too.

Although the embodiments of the present invention have been described above with reference to the drawings, the present invention is not limited to the above embodiments. Furthermore, the configurations of each embodiment may be combined or modified as appropriate without departing from the gist of the present invention.

As described above, the present invention is a technique useful for determining the types of various media.

1: Medium discrimination system 2, 102: Camera 3: Medium discrimination device 4: Monitor 5: Input device 6: Reading table 10: Control unit 11: Video input unit 12: Feature detection unit 13: Type determination unit 14: Recognition area identification Unit 15: Character recognition orientation correction unit 16: Character recognition unit 17: Display orientation correction unit 18: Determination unit 20: Storage unit 21: Inference model 22: Reference information 103: Cloud server 104: Personal computer M: Medium Ma: Characteristic part Mb: Character recognition area

Claims (8)

a feature detection unit that detects a plurality of characteristic parts of the medium based on an image of the medium;
a type determination unit that determines the type of the medium based on the plurality of characteristic parts;
a recognition area specifying unit that specifies a character recognition area that is a target of character recognition based on the relative positions between the plurality of characteristic parts and the type of the medium;
a character recognition unit that recognizes characters within the character recognition area;
Equipped with
Each of the plurality of feature parts includes at least one of a specific character string and a logo,
The recognition area specifying unit selects the position information of the medium determined by the type determining unit from among the position information set for each type of medium and indicating the respective positions of each characteristic part and character recognition area. The position of the character recognition area is calculated based on the position information corresponding to the type and the relative positional relationship between the plurality of feature parts detected by the feature detection unit.
A medium discrimination device characterized by: Further comprising a character recognition orientation correction unit that corrects the orientation of the character recognition area based on the relative position between the plurality of feature parts,
2. The medium discrimination device according to claim 1 , wherein the character recognition unit recognizes characters within the character recognition area whose orientation has been corrected. a display unit that displays the image of the medium;
The medium according to claim 1 or 2, further comprising a display orientation correction unit that corrects the orientation of the image displayed on the display unit based on relative positions between the plurality of characteristic parts. Discrimination device. 4. The medium discrimination device according to claim 1, wherein the feature detection unit detects the plurality of feature parts using a machine-learned inference model. The feature detection unit detects the plurality of feature parts using an inference model machine-learned using a data set in which positions and classes of at least two feature parts are attached as label information for the medium. The medium discrimination device according to any one of claims 1 to 4 . The method further includes a determining unit that determines the size of the medium based on the plurality of characteristic parts and the type of the medium, and determines whether the entire medium is captured in the image. The medium discrimination device according to any one of claims 1 to 5 . A medium discrimination device according to any one of claims 1 to 6 ,
an imaging device that captures an image of a medium;
A medium discrimination system comprising: a feature detection step of detecting a plurality of features of the medium based on an image of the medium;
a type determination step of determining the type of the medium based on the plurality of characteristic parts;
a recognition area specifying step of specifying a character recognition area to be subjected to character recognition based on the relative positions between the plurality of feature parts and the type of the medium;
a character recognition step of recognizing characters within the character recognition area;
including;
Each of the plurality of feature parts includes at least one of a specific character string and a logo,
The recognition area specifying step includes position information set for each type of medium, which indicates the respective positions of each characteristic part and character recognition area, of the medium determined in the type determination step. The position of the character recognition area is calculated based on the position information corresponding to the type and the relative positional relationship between the plurality of features detected by the feature detection step.
A medium discrimination method characterized by the following.

Images