JP7375497B2 - Number recognition device and method - Google Patents

Description

The present invention relates to the field of information technology, and in particular to a number recognition device and method.

With the advancement of information technology, number recognition has become widely applied. For example, by recognizing numbers on objects waiting to be detected, such as signboards, athletes, workers, and students taking entrance exams, various applications can be achieved.

For example, at a basketball game, number detection has many applications in video surveillance. For example, by detecting and recognizing the number on a player's uniform, the corresponding player can be determined based on the recognition result. In this way, video of the match can be used to track and describe the trajectory of each player to provide technical assistance. In conventional number recognition methods, a classifier classifies all possible numbers. For example, for a basketball player, the possible numbers are 0 to 99, and in this case, there are 100 classifications of the classifier.

It should be noted that the above background art introduction is intended to clearly and completely explain the technical solution of the present invention, and to help those skilled in the art easily understand it. Since these technical solutions are described as part of the background art of the present invention, they should not be interpreted as being well known to those skilled in the art.

The inventor discovered the following. In other words, in the conventional method using a classifier, it is necessary to collect and train a large amount of training data for each classification, which is time-consuming and labor-intensive. Sometimes it is difficult. For example, in the case of recognizing the numbers of basketball players, it is necessary to collect 100 types of training data, but for some numbers, there are very few players who use the numbers, so training data for such numbers is not collected. It is quite difficult to collect and conduct training.

An object of the present invention is to provide a number recognition device and method. These number recognition devices and methods need to detect only one digit number, so they can be easily and easily implemented by collecting only 10 types of training samples from 0 to 9 and training the classifier. The training of the classifier can be completed quickly, and since detection and merging are performed for only one-digit numbers, the recognition accuracy is relatively high.

According to a first aspect of an embodiment of the present invention, there is provided a number recognition device, the device comprising: a first detection unit for detecting a detection object from an input image; a first detection unit for detecting a detection object from an input image; a second detection unit for detecting a one-digit number from the area where the object to be detected is located; It includes a merging unit for obtaining a number in the area where the object waiting to be detected is located.

According to a second aspect of an embodiment of the invention, there is provided an electronic device, said electronic device comprising a number recognition device according to the first aspect of the embodiment of the invention.

According to a third aspect of the embodiment of the present invention, there is provided a number recognition method, the method comprising: detecting an object to be detected from an input image; Detecting a one-digit number from; and merging the one-digit numbers detected from the area where the object waiting to be detected is located to obtain a number in the area where the object waiting to be detected is located. including.

The beneficial effects of the present invention are as follows. That is, first, each object waiting to be detected is detected, then a one-digit number within the area where each object waiting to be detected is located, and finally, these one-digit numbers are merged to detect each object waiting to be detected. You can get the number corresponding to the object. Since it is necessary to detect only one digit number, we can complete the training of the classifier easily and quickly by collecting only 10 types of training samples from 0 to 9 and training the classifier. Furthermore, since detection and merging are performed for only one-digit numbers, the recognition accuracy is relatively high.

Note that terms such as "comprising/having" as used herein refer to the presence of a feature, element, step, or assembly, but also include one or more other features, elements, steps, It also refers to not excluding the existence or addition of an assembly.

1 is a diagram showing a number recognition device in Example 1 of the present invention. FIG. FIG. 3 is a diagram showing an input image in Example 1 of the present invention. FIG. 3 is a diagram showing a detection frame in which a detection-waiting object is located in FIG. 2 and a detection result of its one-digit number. FIG. 3 is a diagram illustrating detecting a one-digit number from an area where one detection-waiting object is located in the first embodiment of the present invention. FIG. 7 is another diagram showing detecting a one-digit number from a region where one detection-waiting object is located in the first embodiment of the present invention. FIG. 3 is a diagram showing an electronic device in Example 2 of the present invention. FIG. 3 is a diagram showing a system configuration of an electronic device in Example 2 of the present invention. FIG. 7 is a diagram showing a number recognition method in Example 3 of the present invention.

Hereinafter, preferred embodiments for carrying out the present invention will be described in detail with reference to the accompanying drawings.

Embodiments of the invention provide a number recognition device. FIG. 1 is a diagram showing a number recognition device in Embodiment 1 of the present invention. As shown in FIG. 1, the number recognition device 100 includes the following.

First detection unit 101: detects an object waiting to be detected from an input image;
Second detection unit 102: detects a one-digit number from the area where the detected object waiting to be detected is located; and Merging unit 103: merges the one-digit number detected from the area where the object waiting to be detected is located. By doing this, the number in the area where the object waiting to be detected is located is obtained.

As you can see from the example above, it is necessary to detect only one digit number, so it is easy to collect only 10 types of training samples from 0 to 9 and train the classifier. Moreover, the training of the classifier can be completed quickly, and since detection and merging are performed for only one-digit numbers, the recognition accuracy is relatively high.

In this embodiment, the object waiting to be detected may be any object whose number needs to be recognized. For example, the objects waiting to be detected include players in a match, signboards including numbers, staff members participating in an event, students participating in a test, players participating in a match, and the like.

For example, for a basketball or soccer player participating in a game, the possible numbers are 0-99, and for a track and field athlete participating in the game, the possible numbers are 0000-9999.

In this embodiment, the input image is an image that may include an object waiting to be detected. For example, the object awaiting detection is a basketball player and the input image is at least one image in a video of a basketball game.

In this embodiment, the detection of the object waiting to be detected by the first detection unit 101 and the detection of the one-digit number by the second detection unit 102 may be based on various detection methods. The detection unit 101 and the second detection unit 102 use a convolutional neural network (CNN) to detect the object to be detected in the input image and the one-digit number in the area where the object to be detected is located, respectively. Detection may also be performed for.

In this embodiment, the prior art can be referred to for the specific configuration of the convolutional neural network. For example, the convolutional neural network is Faster R-CNN, FPN (Feature Pyramid Networks for object Detection) or YOLO (You Only Look Once: Unified, Real-Time Object Detection).

In this embodiment, the prior art can also be referred to for the training method for the convolutional neural network, so a detailed explanation thereof will be omitted here.

Since the convolutional neural network has high target recognition ability and can also simplify complex factors in the real environment, the detection efficiency and detection accuracy can be further improved.

In this embodiment, numbers on the uniforms of players during a basketball game will be exemplified.

FIG. 2 is a diagram showing an input image in Example 1 of the present invention. As shown in FIG. 2, the input image is one image in a basketball game video, which includes multiple players as objects waiting to be detected. The first detection unit 101 detects each player in the input image and obtains the area where each player is located, and such area is indicated by each detection frame.

FIG. 3 is a diagram showing the detection frame in which the detection-waiting object in FIG. 2 is located, and the detection result of its one-digit number. As shown in Figure 3, the area where each detected player is located is shown in each detection frame, and the detection results of the one-digit number in each detection frame are sequentially “No result”, “3 and 0”, “2”, “no result”, “2 and 3”, “0”, and “8”.

In this example, after detecting a single digit number from the area where each object waiting to be detected is located, and before merging the single digit numbers in each area, some preprocessing is performed. Also good. For example, as shown in FIG. 1, the device 100 may further include the following.

First removal unit 104: for a detected one-digit number, when the score output by the convolutional neural network is lower than a first threshold, remove the one-digit number from the detection result.

In this example, when a convolutional neural network is used to detect a single digit number, the output results include the coordinate position of the detected single digit number and the coordinate position of the detected single digit number. The score may include a probability that the detected single-digit number belongs to the class of the single-digit number.

In this embodiment, the first threshold value may be set according to actual needs, for example, the first threshold value is 0.5.

By removing single-digit numbers with relatively low scores in this way, some false positive results can be effectively removed, thereby further improving the recognition accuracy of number recognition.

As shown in FIG. 1, the device 100 may further include the following.

Second removal unit 105: When the distance between the at least two adjacent one-digit numbers detected in the area where the detection-waiting object is located is greater than the second threshold, the at least two adjacent one-digit numbers are removed. Among the digits, one digit whose score output by the convolutional neural network is lower than a third threshold is removed.

In this embodiment, the second threshold value and the third threshold value may be determined according to actual needs. For example, the second threshold is the sum of the widths of detection frames of two adjacent one-digit numbers, and the third threshold is 0.9.

FIG. 4 is a diagram showing a one-digit number detected from an area where one detection-waiting object is located in the first embodiment of the present invention. As shown in Figure 4, two adjacent single-digit numbers are detected within the area where one detection-waiting object is located, 2 and 3, respectively, and the center of the two single-digit numbers is 2 and 3. The distance is d, and the scores output by the convolutional neural network for these two single-digit numbers are 0.68 and 0.99, respectively.

For example, if the distance d is larger than the sum of the widths of the detection frames of single-digit numbers 2 and 3, remove the single-digit number with a score lower than 0.9, that is, the single-digit number 2 with a score of 0.68. can do.

In this way, by removing a single digit number with a relatively low score among adjacent single digit numbers with a relatively large distance, it is possible to ensure that the result of the merger is the number of the same object waiting to be detected. This makes it possible to further improve the recognition accuracy of number recognition.

As shown in FIG. 1, the device 100 may further include the following.

Determining unit 106: A score output by the convolutional neural network when a one-digit number is detected from each of at least two overlapping detection areas in the area where the object waiting for detection is located. The highest single digit number is used for the merge process.

FIG. 5 is another diagram showing a one-digit number detected from an area where one detection-waiting object is located in the first embodiment of the present invention. As shown in Figure 5, two superimposed detection areas (detection frames) are detected for the area where one object waiting for detection is located, and the detection results are one-digit numbers 7 and 1, respectively. , in this case, remove the single digit number 1, which has a relatively low score.

In this way, erroneously detected one-digit numbers can be further removed, and therefore the recognition accuracy of number recognition can be further improved.

In this embodiment, the first removal unit 104, the second removal unit 105, and the determination unit 106 may be optional.

In this embodiment, after a one-digit number is detected from the area where each object waiting to be detected is located, at least one preprocessing described above may be performed, and then the one-digit numbers may be merged. One-digit numbers may be directly merged without performing these pre-processing steps.

In this embodiment, the merging unit 103 obtains the number in the area where the object waiting to be detected is located by merging the one-digit numbers detected in the area where the object waiting to be detected is located. I can do it. In other words, when a plurality of objects waiting to be detected are detected from the input image, the merging unit 103 merges one-digit numbers one by one for the area where each object waiting to be detected is located. A number corresponding to each object waiting to be detected can be obtained. Note that when only one one-digit number is detected in the area where one detection-waiting object is located, it is not necessary to perform the merging process.

A specific method by which the merging unit 103 performs merging on one-digit numbers will be exemplified below.

In this embodiment, the merging unit 103 merges each one-digit number in order from left to right based on the center position of the one-digit number detected in the area where the object waiting for detection is located. .

As shown in FIG. 3, the merging unit 103 merges one-digit numbers one by one for the area where each object waiting to be detected is located, and obtains a number recognition result corresponding to each object waiting to be detected. , respectively, are "no result", "30", "2", "no result", "23", "0" and "8".

In this embodiment, when the number of one-digit numbers detected in the area where the object waiting for detection is located is greater than the maximum number of digits of the number, the score output by the convolutional neural network is calculated based on the score output by the convolutional neural network. , the same number of single-digit numbers as the maximum number of digits may be merged from among the detected single-digit numbers.

For example, for a basketball player, when his number is 2 digits and three or more 1-digit numbers are detected in the area where one detection waiting object is located, the convolutional neural network for each 1-digit number is Based on the score output by the network, two single-digit numbers may be selected and merged. For example, after sorting three or more single-digit numbers according to the order of their scores from high to low, the previous two single-digit numbers may be selected and merged.

In this way, it is possible to avoid outputting incorrect number recognition results.

As can be seen from the above example, first, each object to be detected is detected, then a one-digit number within the area where each object to be detected is located, and finally, these one-digit numbers are merged. In this way, a number corresponding to each object waiting to be detected can be obtained. Since it is necessary to detect only one digit number, we can complete the training of the classifier easily and quickly by collecting only 10 types of training samples from 0 to 9 and training the classifier. Furthermore, since detection and merging are performed for only one-digit numbers, the recognition accuracy is relatively high.

The embodiment of the present invention further provides an electronic device, and FIG. 6 is a diagram showing the electronic device in a second embodiment of the present invention. As shown in FIG. 6, the electronic device 600 includes a number recognition device 601, and the configuration and function of the number recognition device 601 are the same as those described in the first embodiment, so a detailed explanation thereof will be omitted here. .

FIG. 7 is a block diagram showing the system configuration of an electronic device in Example 2 of the present invention. As shown in FIG. 7, electronic device 700 includes a central processing unit 701 and a memory 702, and the memory 702 is connected to the central processing unit 701. Note that the figure is merely an example, and the telecommunication function or other functions may be realized by supplementing or replacing the structure with other types of structures.

As shown in FIG. 7, the electronic device 700 may further include an input unit 703, a display 704, a power source 705, and the like.

In one implementation, the functionality of the number recognition device described in Example 1 may be integrated into the central processing unit 701. The central processing unit 701 may be configured as follows, that is, detects an object waiting to be detected from an input image, and calculates a one-digit number from the area where the detected object waiting to be detected is located. The one-digit number detected in the area where the object waiting to be detected is located is merged to obtain the number in the area where the object waiting to be detected is located.

For example, the object to be detected and the one-digit number are respectively detected by a convolutional neural network.

For example, when the score output by the convolutional neural network for a detected one-digit number is lower than a first threshold, the one-digit number is removed from the detection results.

For example, when the distance between at least two adjacent one-digit numbers detected in the area where the detection-waiting object is located is greater than a second threshold, the distance between the at least two adjacent one-digit numbers is Among them, one-digit numbers whose scores output by the convolutional neural network are lower than the third threshold are removed.

For example, when detecting one-digit numbers in each of at least two overlapping detection areas in the area where the detection-waiting object is located, the one-digit number output by the convolutional neural network with the highest score is merged. used for

For example, merging the one-digit numbers detected from the area where the object waiting to be detected is located means that the center position of the one-digit number detected from the area where the object waiting to be detected is located. involves merging each single-digit number according to left-to-right order based on .

For example, merging one-digit numbers detected from the area where the object waiting to be detected is located means that the number of one-digit numbers detected from the area where the object waiting to be detected is located is Based on the score output by the convolutional neural network when the number is greater than the maximum number of digits, among the detected one-digit numbers, for the same number of one-digit numbers as the maximum number of digits. Including undertaking a merger.

In another implementation, the number recognition device described in Example 1 may be arranged separately from the central processing unit 701, for example, the number recognition device can be placed on a chip connected to the central processing unit 701. (chip), and the functions of the number recognition device may be realized under the control of the central processing unit 701.

In this embodiment, the electronic device 700 does not need to include all the components shown in FIG.

As shown in FIG. 7, the central processing unit 701 may be referred to as a controller or operating controller, and may include a microprocessor, other processor devices, and/or logic devices; can receive input and control the operation of each component of the electronic device 700.

The storage 702 may be, for example, one or more of a buffer, fresh memory, HDD, movable media, volatile storage, non-volatile storage, or other suitable device. Further, the central processing unit 701 can store and process information by executing a program stored in the storage device 702. Note that other functions are similar to those of the prior art, so detailed explanation thereof will be omitted here. Furthermore, each component of the electronic device 700 may be realized by dedicated hardware, firmware, software, or a combination thereof, all of which fall within the scope of the present invention.

As you can see from the example above, since it is necessary to detect only one digit number, we can easily and quickly collect only 10 types of training samples from 0 to 9 and train the classifier. It is possible to complete the training of the classifier in a short period of time, and since detection and merging are performed for only one digit number, the recognition accuracy is relatively high.

The embodiment of the present invention further provides a number recognition method, which corresponds to the number recognition device in the first embodiment. FIG. 8 is a diagram showing a number recognition method in Example 3 of the present invention. As shown in FIG. 8, the method includes the following steps.

Step 801: Detect the object waiting to be detected from the input image;
Step 802: Detecting a one-digit number from the area where the detected object waiting to be detected is located; and Step 803: Merging the one-digit numbers detected from the area where the object waiting to be detected is located; The number in the area where the object waiting to be detected is located is acquired.

In this embodiment, the description in Embodiment 1 can be referred to for the specific implementation method of each of the above-mentioned steps, and redundant explanation thereof will be omitted here.

As you can see from the example above, since it is necessary to detect only one digit number, we can easily and quickly collect only 10 types of training samples from 0 to 9 and train the classifier. It is possible to complete the training of the classifier in a short period of time, and since detection and merging are performed for only one digit number, the recognition accuracy is relatively high.

The embodiment of the present invention further provides a computer readable program, wherein when the program is executed in a number recognition device or an electronic device, the program causes the computer to read the number recognition device or the electronic device. Inside, the number recognition method described in Example 3 is executed.

The embodiment of the present invention further provides a storage medium storing a computer-readable program, wherein the computer-readable program causes a computer to perform the number recognition method according to embodiment 3 in a number recognition device or an electronic device. can be executed.

Further, the apparatuses, methods, etc. according to the embodiments of the present invention may be implemented by software, hardware, or a combination of hardware and software. The invention also relates to such a computer readable program, i.e. said program, when executed by a logic component, is capable of causing said logic component to implement the above-mentioned device or component; A logic component may implement the method or steps thereof described above. Furthermore, the present invention also relates to a storage medium storing the above program, such as a hard disk, a magnetic disk, an optical disk, a DVD, and a fresh memory.

Further, regarding the above-mentioned embodiments, the following additional notes are also disclosed.

(Additional note 1)
A number recognition method,
Detect the object waiting to be detected from the input image;
detecting a one-digit number from the area where the detected object waiting to be detected is located; and merging the one-digit numbers detected from the area where the object waiting to be detected is located; A method comprising obtaining a number in a region.

(Additional note 2)
The method described in Appendix 1,
A method for detecting the object to be detected and the one-digit number, respectively, by a convolutional neural network.

(Appendix 3)
The method described in Appendix 2, further comprising:
A method comprising removing a detected single-digit number from a detection result when a score output by the convolutional neural network for the detected single-digit number is lower than a first threshold.

(Appendix 4)
The method described in Appendix 2, further comprising:
When the distance between at least two adjacent one-digit numbers detected from the area where the detection-waiting object is located is greater than a second threshold, the convolution A method comprising removing single digit numbers for which the score output by the neural network is lower than a third threshold.

(Appendix 5)
The method described in Appendix 2, further comprising:
When detecting one-digit numbers from each of at least two overlapping detection areas in the area where the detection-waiting object is located, the number with the highest score output by the convolutional neural network is used for the merging. ,Method.

(Appendix 6)
The method described in Appendix 1,
Merging the one-digit numbers detected from the area where the object waiting for detection is located includes:
The method includes merging each single-digit number in order from left to right based on the center position of the single-digit number detected from the area where the detection-waiting object is located.

(Appendix 7)
The method described in Appendix 2,
Merging the one-digit numbers detected from the area where the object waiting for detection is located includes:
A single-digit number detected based on a score output by the convolutional neural network when the number of single-digit numbers detected from the area where the object awaiting detection is located is greater than the maximum number of digits of the number. A method comprising selecting the same number of one-digit numbers as the maximum number of digits from among the numbers and performing a merger.

Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and any changes to the present invention fall within the technical scope of the present invention unless they depart from the spirit of the present invention.

Claims (7)

A number recognition device,
a first detection unit for detecting an object to be detected from an input image;
a second detection unit for detecting a one-digit number from the area where the detected object waiting to be detected is located; and merging the one-digit numbers detected from the area where the object waiting to be detected is located; comprising a merging unit for obtaining a number in an area where the object waiting to be detected is located;
The first detection unit and the second detection unit detect the object waiting to be detected and the one-digit number, respectively, by a convolutional neural network;
The number recognition device further includes:
When the distance between at least two adjacent one-digit numbers detected from the area where the detection-waiting object is located is greater than a second threshold, the convolution A number recognition device, comprising a second removal unit for removing single-digit numbers whose score output by the neural network is lower than a third threshold. The number recognition device according to claim 1 , further comprising:
Number recognition , comprising a first removal unit for removing the detected one-digit number from the detection result when the score output by the convolutional neural network is lower than a first threshold. Device. The number recognition device according to claim 1 , further comprising:
When detecting one-digit numbers from at least two overlapping detection areas in the area where the detection-waiting object is located, the one-digit number with the highest score output by the convolutional neural network is used in the combination. Number recognition device, including a confirmation unit for use. The number recognition device according to claim 1,
The merging unit is a number recognition device that merges each one-digit number in order from left to right based on the center position of the one-digit number detected from the area where the detection-waiting object is located. The number recognition device according to claim 1 ,
The merging unit performs detection based on the score output by the convolutional neural network when the number of one-digit numbers detected from the area where the detection -waiting object is located is greater than the maximum number of digits of the number. A number recognition device that selects the same number of one-digit numbers as the maximum number of digits from among the one-digit numbers and merges them. An electronic device comprising the number recognition device according to any one of claims 1 to 5 . A number recognition method,
Detect the object waiting to be detected from the input image;
Detecting a one-digit number from the area where the detected object waiting to be detected is located; and merging the one-digit numbers detected from the area where the object waiting to be detected is located, and determining the location of the object waiting to be detected. including obtaining a number in an area where
respectively detecting the object waiting to be detected and the one-digit number by a convolutional neural network;
The number recognition method further includes:
When the distance between at least two adjacent one-digit numbers detected from the area where the detection-waiting object is located is greater than a second threshold, the convolution A number recognition method comprising removing single digit numbers whose score output by a neural network is lower than a third threshold .

Images