JP7058053B2 - Computer system, information code reading method and program - Google Patents

Description

The present invention relates to a computer system for reading an information code, an information code reading method and a program.

In recent years, information codes (for example, one-dimensional code, two-dimensional code, AR marker) have been utilized in various uses such as goods management, work management, entrance / exit management, membership card, and payment. Utilization of such an information code is performed by reading the displayed information code by an application, a reading device, or the like.

As an example of a technique for utilizing such an information code, a user-desired information code is specified from an image containing a plurality of information codes, and the specified information code is enlarged and photographed. A system for reading an information code desired by this user is disclosed (see Patent Document 1).

Japanese Unexamined Patent Publication No. 2011-84384

However, in the system of Patent Document 1, since it is necessary to specify the information code to be read by the user, there is a possibility that the user cannot determine which information code should be specified.

The present invention provides a computer system, an information code reading method, and a program capable of reading an information code even when the size of the information code contained in the image is small or when a plurality of information codes are present. With the goal.

The present invention provides the following solutions.

The present invention is a computer system that reads an information code.
An acquisition means for acquiring an image in which the information code is taken, and
An analysis means for analyzing the acquired image and
A detection means for detecting the information code based on the analysis result of the image and the learned feature amount of the information code.
If the detected information code cannot be recognized, an expansion means for expanding the information code and an expansion means.
A recognition means for recognizing the expanded information code and
A calculation means for calculating the coordinates of the recognized information code in the image, and
An update means for updating the feature amount of the learned information code by synthesizing the acquired image with the calculated coordinates.
Equipped with
When a plurality of information codes are present in the detected image, the enlarging means expands any one of the plurality of information codes based on a predetermined condition .
The coordinates provide a computer system characterized by being the center point of the information code .

According to the present invention, the computer system that reads the information code acquires the image in which the information code is taken, analyzes the acquired image, and the analysis result of the image and the learned feature amount of the information code. When the information code is detected and the detected information code cannot be recognized, the information code is enlarged, the enlarged information code is recognized, and the coordinates in the image of the recognized information code are recognized. Is calculated, the acquired image is combined with the calculated coordinates, the feature amount of the learned information code is updated, and when a plurality of information codes are present in the detected image, a predetermined condition is obtained. The information code of any of the plurality of information codes is expanded based on the above, and the coordinates are the center points of the information codes .

Although the present invention is in the category of systems, the same actions and effects are exhibited in other categories such as methods and programs according to the categories.

INDUSTRIAL APPLICABILITY According to the present invention, a computer system, an information code reading method, and a program capable of reading an information code even when the size of the information code contained in the image is small or when a plurality of information codes exist are provided. It becomes possible to do.

FIG. 1 is a diagram showing an outline of the information code reading system 1. FIG. 2 is an overall configuration diagram of the information code reading system 1. FIG. 3 is a diagram showing a flowchart of a learning process executed by the computer 10. FIG. 4 is a diagram showing a flowchart of the information code reading process executed by the computer 10. FIG. 5 is a diagram showing a flowchart of the relearning process executed by the computer 10. FIG. 6 is a diagram schematically showing an example of an image including an information code. FIG. 7 is a diagram schematically showing an enlarged state of the information code.

Hereinafter, the best mode for carrying out the present invention will be described with reference to the drawings. It should be noted that this is just an example, and the technical scope of the present invention is not limited to this.

[Overview of information code reading system 1]
An outline of a preferred embodiment of the present invention will be described with reference to FIG. FIG. 1 is a diagram for explaining an outline of an information code reading system 1 which is a preferred embodiment of the present invention. The information code reading system 1 is a computer system composed of a computer 10 and reading an information code.

The information code reading system 1 includes a photographing device such as a camera, a terminal device having a photographing function (for example, a mobile terminal such as a smartphone or a tablet terminal, a head-mounted display such as a smart glass, or a wearable terminal such as a smart watch). Other terminals, devices, and the like may be included. In this case, the information code reading system 1 is executed by any or a plurality of combinations of the computer 10 and the included terminals, devices, and the like.

Further, the information code reading system 1 may be realized by one computer such as a computer 10, or may be realized by a plurality of computers such as a cloud computer.

Further, the information code reading system 1 may be configured by the terminal device described above instead of the computer 10.

The computer 10 is connected to the above-mentioned photographing device, terminal device, other terminals, devices, etc. via a public network or the like so as to be capable of data communication, and executes necessary data and information transmission / reception.

The computer 10 acquires an image in which the information code is taken. For example, a photographing device or a terminal device photographs an information code and transmits the captured image to the computer 10. By receiving this image, the computer 10 acquires the image in which the information code is taken.

When the information code reading system 1 is composed of a terminal device, the terminal device captures the information code to acquire an image in which the information code is captured.

The computer 10 analyzes the acquired image. For example, the computer 10 analyzes the acquired image and extracts the feature amount in this image. The feature amount is, for example, a statistical numerical value such as a shape, an outline, a hue, an average of pixel values, a variance, or a histogram. For example, in the case of a two-dimensional code, the computer 10 extracts unique features such as a finder pattern, an alignment pattern, a quiet zone, and a timing pattern existing in this image as feature quantities. Similarly, even if it is another one such as a one-dimensional code or an AR marker, a peculiar one is extracted as a feature quantity.

The computer 10 detects the information code based on the analysis result and the feature amount of the learned information code. The computer 10 machine-learns the feature amount of the information code in advance, and creates a trained model for information code detection based on the learning result. The computer 10 detects the information code existing in the acquired image based on the feature amount extracted this time and the learned model for information code detection created in advance.

If the computer 10 cannot recognize the detected information code, the computer 10 expands this information code. For example, the computer 10 attempts to recognize the detected information code. When the computer 10 cannot recognize the detected information code, for example, when the detected information code is not large enough (that is, smaller than a predetermined size), there are a plurality of detected information codes. (That is, one information code cannot be specified). If the computer 10 cannot recognize such a detected information code, the computer 10 expands the detected information code to a recognizable size.

The computer 10 recognizes the expanded information code. For example, the computer 10 recognizes the expanded information code and reads the information and the like stored in this information code.

The computer 10 may be configured to calculate the coordinates of the image of the information code recognized this time and update the feature amount of the learned information code based on the acquired image and the coordinates. For example, the computer 10 calculates the coordinates in the image of the information code recognized this time. The computer 10 creates a new learning data set based on the calculated coordinates and the image, and the information is based on the created training data set and the information code detection trained model created in advance. Update the trained model for code detection. The computer 10 will use the updated information code detection trained model when detecting the information code from the next time onward.

Next, an outline of the processing executed by the information code reading system 1 will be described.

First, the computer 10 acquires an image in which the information code is taken (step S01). The photographing device or the terminal device photographs an information code and transmits the captured image to the computer 10. The computer 10 receives this image. As a result, the computer 10 acquires the image in which the information code is taken.

The computer 10 analyzes the acquired image (step S02). The computer 10 analyzes the acquired image and extracts the above-mentioned feature amount in this image.

The computer 10 detects the information code based on the analysis result and the feature amount of the learned information code (step S03). The computer 10 detects the information code existing in the acquired image based on the learned model for information code detection created by machine learning the feature amount of the information code and the feature amount extracted this time.

If the computer 10 cannot recognize the detected information code, the computer 10 expands this information code (step S04). The computer 10 tries to recognize this information code. If the computer 10 cannot recognize the information code as a result of the recognition attempt, the computer 10 expands the information code to a recognizable size. When the computer 10 detects only one information code in the image, the computer 10 enlarges the one information code. Further, when the computer 10 detects a plurality of information codes in the image, the computer 10 enlarges any one of the plurality of information codes based on a predetermined condition . When the computer 10 detects a plurality of information codes in an image, the computer 10 may be configured to sequentially expand the plurality of information codes based on predetermined conditions .

The computer 10 recognizes the expanded information code (step S05). The computer 10 recognizes the enlarged information code and reads the information and the like stored in the information code.

The above is an outline of the processing executed by the information code reading system 1.

[System configuration of information code reading system 1]
Based on FIG. 2, the system configuration of the information code reading system 1 which is a preferred embodiment of the present invention will be described. FIG. 2 is a diagram showing a system configuration of an information code reading system 1 which is a preferred embodiment of the present invention. In FIG. 2, the information code reading system 1 is a computer system composed of a computer 10 and reading an information code such as a one-dimensional code, a two-dimensional code, and an AR marker.

The computer 10 is connected to a photographing device, a terminal device, other terminals, devices, etc. via a public network or the like so as to be capable of data communication, and executes necessary data and information transmission / reception.

The information code reading system 1 may include a photographing device, a terminal device, other terminals, devices, and the like. In this case, the information code reading system 1 executes each process described later by any or a plurality of combinations of the computer 10 and the terminals and devices included in the computer 10.

Further, the information code reading system 1 may be realized by one computer such as a computer 10, or may be realized by a plurality of computers such as a cloud computer.

Further, the information code reading system 1 may be configured by the terminal device described above instead of the computer 10.

The computer 10 includes a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), and the like as a control unit, and has other terminals and devices as a communication unit. It includes a device for enabling communication, for example, a Wi-Fi (Wi-Filess-Fidelity) compatible device conforming to IEEE802.11. Further, the computer 10 includes a data storage unit such as a hard disk, a semiconductor memory, a recording medium, and a memory card as a recording unit. Further, the computer 10 includes various devices and the like that execute various processes as a processing unit.

In the computer 10, the control unit reads a predetermined program to realize the learning data set acquisition module 20 and the image acquisition module 21 in cooperation with the communication unit. Further, in the computer 10, the control unit reads a predetermined program and cooperates with the recording unit to realize the recording module 30. Further, in the computer 10, when the control unit reads a predetermined program, the analysis module 40, the learning module 41, the model creation module 42, the detection module 43, the recognition module 44, and the recognition determination module 45 cooperate with the processing unit. , Expansion module 46, calculation module 47, synthesis module 48, and model update module 49 are realized.

[Learning process]
The learning process executed by the information code reading system 1 will be described with reference to FIG. FIG. 3 is a diagram showing a flowchart of a learning process executed by the computer 10. The process executed by each of the above-mentioned modules will be described together with this process.

The learning data set acquisition module 20 acquires an information code learning data set (step S10). In step S10, the learning data set acquisition module 20 acquires this learning data set from another computer, an external server, or the like. This training data set is, for example, an image in which one or more information codes are taken.

The analysis module 40 analyzes the acquired learning data set (step S11). In step S11, the analysis module 40 performs image analysis on this learning data set and extracts the feature amount of the information code. As described above, in the case of a two-dimensional code, the feature amount is peculiar to the two-dimensional code such as a finder pattern, an alignment pattern, a quiet zone, and a timing pattern, and is another one such as a one-dimensional code or an AR marker. In some cases, it is unique to each of these.

The learning module 41 executes machine learning based on the analysis result (step S12). In step S12, the learning module 41 executes supervised learning using the extracted features as a teacher as machine learning.

The model creation module 42 creates a trained model for information code detection based on the learning result (step S13). In step S13, the model creation module 42 creates a trained model for information code detection incorporating the feature amount of the trained information code.

The recording module 30 records the created trained model for detecting an information code in itself (step S14). The computer 10 will use the recorded information code detection trained model in the processing described later.

The above is the learning process.

[Information code reading process]
The information code reading process executed by the information code reading system 1 will be described with reference to FIG. FIG. 4 is a diagram showing a flowchart of the information code reading process executed by the computer 10. The process executed by each of the above-mentioned modules will be described together with this process.

The image acquisition module 21 acquires an image in which the information code is captured (step S20). In step S20, the image acquisition module 21 acquires an image in which the photographing device or the terminal device has captured this information code. At this time, the photographing device or the terminal device transmits this image, and the image acquisition module 21 receives this image, so that the image acquisition module 21 acquires this image.

When the information code reading system 1 is composed of a terminal device, the terminal device captures the information code to acquire an image in which the information code is captured. The terminal device is configured to execute the processing described later using the captured image.

The analysis module 40 analyzes the acquired image (step S21). In step S21, the analysis module 40 extracts the feature amount of the acquired image. As described above, the analysis module 40 extracts statistical values such as shape, contour, hue, average of pixel values, variance, and histogram as feature quantities, and is unique in one-dimensional code, two-dimensional code, AR marker, and the like. Extract things.

The detection module 43 detects the information code based on the analysis result and the feature amount of the learned information code (step S22). In step S22, the detection module 43 executes this process based on the feature amount extracted this time and the learned model for information code detection created by the above-mentioned learning process. The detection module 43 compares the feature amount extracted this time with the feature amount in the trained model for information code detection, determines whether or not they match or approximate, and matches or approximates the information code. Is detected.

If the computer 10 cannot detect the information code, the computer 10 ends this process.

The recognition module 44 recognizes the detected information code (step S23). In step S23, the recognition module 44 attempts to read the information stored in this information code.

The recognition determination module 45 determines whether or not the information code can be recognized (step S24). In step S24, the recognition determination module 45 determines that the information code could not be recognized when the detected information code is not large enough or when there are a plurality of detected information codes. When the recognition determination module 45 determines that the information code can be recognized (YES in step S24), the recognition module 44 reads the information stored in this information code (step 25). The computer 10 will perform necessary processing based on the read information.

On the other hand, when the recognition determination module 45 determines in step S24 that the information code could not be recognized (step S24 NO), the expansion module 46 expands the detected information code (step S26). In step S26, the expansion module 46 increases the size of the detected information code. The computer 10 determines that the reason why it could not be recognized is the size of the detected information code. Specifically, this is to determine that the information code present in the image is not large enough to be recognized.

If there is only one information code that the recognition module 44 could not recognize, the expansion module 46 expands this one information code.

Further, when there are a plurality of information codes that cannot be recognized by the recognition module 44, the expansion module 46 specifies one information code to be expanded based on a predetermined condition. The predetermined conditions are, for example, the coordinates of the information code (for example, close to the center of the screen, far from the center of the screen, close to the edge of the screen, far from the edge of the screen), the size of the information code (for example, maximum and minimum), and the information code. The presence or absence of descriptive text. The expansion module 46 expands this one identified information code.

When the computer 10 detects a plurality of information codes in an image, the computer 10 may be configured to sequentially expand the plurality of information codes based on predetermined conditions .

Further, if the cause that the computer 10 could not recognize is other than the size, for example, damage, stain, distortion, etc., the computer 10 determines that the information code cannot be recognized instead of enlarging it, and this process is performed. To finish. That is, the computer 10 ends this process without enlarging the information code that cannot be recognized even if it is enlarged.

The recognition module 44 recognizes the expanded information code (step S27). In step S27, the recognition module 44 recognizes the expanded information code and reads the information stored in the information code. The computer 10 will perform necessary processing based on the read information.

The above-mentioned information code reading process will be described with reference to FIGS. 6 and 7. FIG. 6 is a diagram schematically showing an example of an image in which an information code is taken. In FIG. 6, the image 200 has a plurality of information codes 210. As a result of the analysis of the image 200, the detection module 43 detects nine information codes 210-218. As a result of the trial of the recognition module 44, it is assumed that the recognition determination module 45 cannot recognize the nine information codes 210-218. The expansion module 46 identifies one information code 214 based on a predetermined condition (for example, the information code 214 located at the center of the screen) among the nine information codes 210-218, and the specified one information. Enlarge code 214.

FIG. 7 schematically shows an example of the expanded information code. In FIG. 7, the enlarged information code 300 is an enlarged version of the specified information code 214 to a recognizable size. The recognition module 44 reads the information stored in the information code 214 by recognizing the expanded information code 300.

The above is the information code reading process.

[Re-learning process]
The re-learning process executed by the information code reading system 1 will be described with reference to FIG. FIG. 5 is a diagram showing a flowchart of the relearning process executed by the computer 10. The process executed by each of the above-mentioned modules will be described together with this process.

The calculation module 47 calculates the coordinates in the image of the information code recognized by the above-mentioned information code reading process (step S30). In step S30, the calculation module 47 calculates the coordinates of the information code in this image by XY coordinates as the coordinates in this image. At this time, the center of the information code is calculated as these coordinates.

The composition module 48 combines the calculated coordinates with the image (step S31). In step S31, the synthesis module 48 synthesizes the features of the coordinates and the information code with the image. The synthesis module 48 synthesizes the feature amount of the information code with the coordinates corresponding to this information code in the image. The computer 10 uses this synthesis result as a data set for re-learning in a process described later.

The learning module 41 relearns the feature amount of the information code based on the learned model for information code detection created by the above-mentioned learning process and the synthesis result (step S32). In step S32, the learning module 41 relearns the coordinates in the image of the information code and the feature amount of the information code based on the trained model for detecting the information code and the data set for relearning.

The model update module 49 updates the trained model for detecting the information code recorded by the recording module 30 based on the result of the re-learning (step S33). In step S33, the model update module 49 updates the trained model for information code detection by incorporating the coordinates in the image of the retrained information code and the feature amount of the information code into the trained model for information code detection.

As a result, the model update module 49 updates the feature amount of the learned information code based on the acquired image and the calculated coordinates.

The recording module 30 records the trained model for detecting the updated information code (step S34). When the computer 10 again executes the above-mentioned information code reading process, the computer 10 uses the updated information code detection learned model recorded by this process.

The above is the re-learning re-processing.

The above-mentioned means and functions are realized by a computer (including a CPU, an information processing device, and various terminals) reading and executing a predetermined program. The program is provided, for example, in the form of being provided from a computer via a network (Software as a Service). Further, the program is provided in a form recorded on a computer-readable recording medium such as a flexible disk, a CD (CD-ROM or the like), or a DVD (DVD-ROM, DVD-RAM or the like). In this case, the computer reads the program from the recording medium, transfers it to an internal recording device or an external recording device, records the program, and executes the program. Further, the program may be recorded in advance on a recording device (recording medium) such as a magnetic disk, an optical disk, or a magneto-optical disk, and the program may be provided from the recording device to a computer via a communication line.

Although the embodiments of the present invention have been described above, the present invention is not limited to these embodiments described above. Further, the effects described in the embodiments of the present invention merely list the most suitable effects arising from the present invention, and the effects according to the present invention are limited to those described in the embodiments of the present invention. is not.

1 Information code reading system 10 Computer

Claims (5)

A computer system that reads information codes
An acquisition means for acquiring an image in which the information code is taken, and
An analysis means for analyzing the acquired image and
A detection means for detecting the information code based on the analysis result of the image and the learned feature amount of the information code.
If the detected information code cannot be recognized, an expansion means for expanding the information code and an expansion means.
A recognition means for recognizing the expanded information code and
A calculation means for calculating the coordinates of the recognized information code in the image, and
An update means for updating the feature amount of the learned information code by synthesizing the acquired image with the calculated coordinates.
Equipped with
When a plurality of information codes are present in the detected image, the enlarging means expands any one of the plurality of information codes based on a predetermined condition .
A computer system characterized in that the coordinates are the center points of the information code . The predetermined condition is the presence or absence of the coordinates, size, or explanatory text of the information code.
The computer system according to claim 1. When a plurality of information codes are present in the detected image, the magnifying means sequentially magnifies the image based on the predetermined conditions .
The computer system according to claim 1. Reading an information code This is an information code reading method executed by a computer system.
The step of acquiring the image in which the information code was taken and
Steps to analyze the acquired image and
A step of detecting the information code based on the analysis result of the image and the learned feature amount of the information code.
If the detected information code cannot be recognized, the step of expanding the information code and
The step of recognizing the expanded information code and
The step of calculating the coordinates in the image of the recognized information code and
A step of synthesizing the acquired image with the calculated coordinates and updating the feature amount of the learned information code.
Equipped with
In the expanding step, when a plurality of information codes are present in the detected image, one of the plurality of information codes is expanded based on a predetermined condition .
An information code reading method characterized in that the coordinates are the center points of the information code. To the computer that reads the information code
Steps to acquire the image in which the information code was taken,
Steps to analyze the acquired image,
A step of detecting the information code based on the analysis result of the image and the learned feature amount of the information code.
If the detected information code cannot be recognized, the step of expanding the information code,
Steps to recognize the expanded information code,
Step of calculating the coordinates in the image of the recognized information code,
A step of synthesizing the acquired image with the calculated coordinates and updating the feature amount of the learned information code.
To execute,
In the expanding step, when a plurality of information codes are present in the detected image, one of the plurality of information codes is expanded based on a predetermined condition .
A computer-readable program for the coordinates to be the center point of the information code .

Images