CN117083616A - Information reading device - Google Patents

Abstract

The invention provides a technology for coping with a situation that information in a two-dimensional code cannot be read. The information reading device (2) has first and second code processing execution units (30) and an adjustment unit (230). A first code processing execution unit (30) detects first relationship information, which is information defining a two-dimensional area of a first information code (for example, position coordinates (L1 to L4) of points at four corners) from an image of the first information code, and reads information recorded in the first information code based on the detected first relationship information. An adjustment unit (230) adjusts parameters of the learning model based on training data including a plurality of successful cases of the reading process using the first relationship information. A second code processing execution unit (30) inputs an image of a second information code captured by a camera (20) to a learning model, acquires second relationship information, which is an estimated value of information defining a two-dimensional area of the second information code, from the learning model, reads information recorded in the second information code based on the acquired second relationship information, and outputs a read result.

Description

Information reading device

Technical Field

The technology disclosed in the present specification relates to a technology for reading information recorded in a two-dimensional information code.

Background

Patent document 1 discloses a reading system for adjusting reading conditions (brightness, exposure time, presence or absence of a filter) for reading a two-dimensional code and reading information in the two-dimensional code. When the reading system fails to read information in the two-dimensional code due to the influence of deterioration of the density of the two-dimensional code, external light, or the like, the reading system reads information in the two-dimensional code in accordance with the reading condition indicated by the extended library instead of the reading condition indicated by the reference library (bank). The extended library is generated by adjusting the reference library according to a predetermined adjustment rule.

Prior art literature

Patent literature

Patent document 1: japanese patent laid-open No. 2018-045735

Disclosure of Invention

The invention aims to solve the technical problems

The above-described technique merely adjusts the reading conditions. Therefore, for example, even if the reading condition is adjusted, the information in the two-dimensional code cannot be read unless the information for reading the information in the two-dimensional code from the image of the two-dimensional code (for example, the information indicating the position of the two-dimensional code) is detected.

In the present specification, a technique for coping with a situation where information in a two-dimensional code cannot be read is provided.

Technical means for solving the technical problems

The information reading device disclosed in the present specification includes:

A camera which can shoot two-dimensional information codes;

a first code process execution unit that executes a first code process including: a detection process of detecting first relation information, which is information defining a two-dimensional code area of the first information code, from an image of the first information code, which is the information code, captured by the camera, a reading process of reading information recorded in the first information code based on the detected first relation information, and an output process of outputting a result of the reading process;

an adjustment unit that adjusts parameters of a learning model, which are second relation information that are estimated values of information defining a two-dimensional code region of the information code, from an image output of the information code captured by the camera, based on training data including at least a plurality of successful cases of the reading process read from the first relation information; and

and a second code processing execution unit that performs a second code processing including: an acquisition process of inputting an image of a second information code, which is the information code captured by the camera, to the learning model and acquiring the second relationship information from the learning model, a reading process of reading information recorded in the second information code based on the acquired second relationship information, and an output process of outputting a result of the reading process.

According to this configuration, the information reading apparatus adjusts the parameters of the learning model that outputs the relationship information on the information code from the image of the information code, using training data including at least a plurality of successful cases of the reading process using the first relationship information detected by the detection process (i.e., based on the training data). After the parameter is adjusted, the information reading apparatus executes an acquisition process using the learning model, acquires the second relationship information from the second information code, and executes a reading process using the acquired second relationship information. For example, in a case where the detection process fails even if the reading condition of the reading process is adjusted, the second relationship information can be acquired from the second information code by the acquisition process using the learning model. It is possible to cope with a situation in which information in the two-dimensional code cannot be read because the reading condition detection process fails even if the reading process is adjusted.

The first relationship information may be information indicating position coordinates of points (positions) at four corners of the information code.

According to this configuration, the position coordinates of the points (positions) at the four corners of the information code can be acquired with reference to the learning model (i.e., based on the learning model).

The first relationship information may be information indicating a black-and-white pattern of a plurality of cells constituting a two-dimensional code area of the information code.

According to this configuration, a black-and-white pattern of a plurality of cells of the information code can be acquired with reference to the learning model (that is, based on the learning model).

In the first relation information, information indicating a position of each of the plurality of cells may be associated with a value indicating one of black and white.

For example, consider a comparative example in which data representing a pattern of black and white of a plurality of cells (hereinafter, referred to as "pattern data") represents a plurality of pixels of an information code. In this comparative example, the information amount of the pattern information increases together with an increase in the number of pixels. In contrast, according to the above configuration, since one value is associated with one cell, the information amount of the pattern data does not increase even if the number of pixels increases as long as the number of cells of the information code is constant. The increase in the information amount of the pattern data can be suppressed as compared with the above-described comparative example.

The present invention may further include: one or more code reading devices including the camera, the first code processing executing unit, and the second code processing executing unit; and a learning device including the adjustment unit independent of the one or more code reading devices, the one or more code reading devices acquiring the adjusted parameters from the learning device, the second code processing execution unit executing the second code processing with reference to the acquired adjusted parameters (i.e., according to parameters).

For example, consider a comparative example in which one device includes a camera, a first code process execution unit, a second code process execution unit, and an adjustment unit. In this comparative example, the one apparatus performs not only reading of the information code but also adjustment of the parameters of the learning model. Therefore, the processing capacity of the one apparatus needs to be improved as compared with the conventional configuration in which adjustment of the parameters of the learning model is not performed. In contrast, according to the above-described configuration, the adjustment of the parameters of the learning model is performed by the learning device independent of the code reading device. Therefore, the adjustment of the parameters of the learning model can be performed even if the processing capacity of the code reading apparatus is not improved as compared with the conventional one.

The learning device may be provided on the internet, and further include an intermediary device that intermediates communication between the one or more code reading devices and the learning device, and is connected to the internet.

According to this structure, the code reading apparatus can be restrained from directly receiving an access from the internet.

The present invention may further include an output device capable of outputting a specific information code in which the adjusted parameter is recorded, wherein the one or more code reading devices each read the adjusted parameter from the specific information code outputted by the output device, and acquire all or a part of the adjusted parameter from the learning device.

According to this configuration, the code reading device can obtain the parameter by reading the specific information code.

The one or more code reading apparatuses may include a first code reading apparatus, and a second code reading apparatus different from the first code reading apparatus, wherein the learning apparatus acquires the training data from the first code reading apparatus, adjusts the parameter with reference to the acquired training data, and the second code reading apparatus acquires the parameter adjusted by using the training data of the first code reading apparatus from the learning apparatus, and executes the second code processing with reference to the acquired parameter.

According to this configuration, the second code reading device can use the success case in the first code reading device different from the second code reading device.

The present invention may further include a specific memory including a first area storing a program for executing the first code process and the second code process, and a second area different from the first area, the second area being an area for storing a plurality of pieces of learning information, each of the plurality of pieces of learning information including the learning model and the adjusted parameter.

According to this configuration, a plurality of pieces of learning information corresponding to various conditions of reading the information code can be stored. The various conditions can be dealt with.

The adjustment unit may be configured to start adjustment of the parameter of the learning model after the number of the plurality of successful cases included in the training data exceeds a predetermined number.

The information reading apparatus may further include: a classification unit configured to classify an information code of an object for which the reading process is successful in the first code process into a specific pattern among a plurality of patterns related to a type of degradation of the information code; and a judging unit configured to judge whether or not a successful case of the information code of the object is adopted as the training data, based on the classified specific pattern.

In order to ensure the reliability of the parameters of the learning model, more than a prescribed number of successful cases are required. However, when successful cases are unconditionally used as the training data, the deviation of successful cases in the training data may deviate from cases in a specific deteriorated pattern (for example, a specific stain at a specific site) at a time when the number of successful cases as the training data exceeds a predetermined number. According to the above configuration, the information reading apparatus classifies successful cases of the target information code by the pattern of deterioration of the information code, and determines whether to adopt the successful cases of the target information code as the training data based on the classification result. This suppresses the deviation of successful cases in the training data from cases in a specific degraded pattern. In addition, "degradation" in the present specification includes not only aged degradation of the information code but also deformation of the information code due to degradation of a printing device that prints the information code and unstable support of a medium on which the information code is printed.

The classifying unit may compare an image of a repair code repaired in error correction performed on the information code of the object with an image obtained by binarizing an actual image of the information code of the object, specify a deteriorated portion of the information code of the object, and classify the information code of the object into the specific pattern based on the deteriorated portion specified by the comparison of the repair code with the actual image.

According to this configuration, the information code of the object can be classified according to the position of degradation of the information code of the object in the actual image.

The classification unit may classify the information code of the object into the specific pattern based on at least one of a contrast of an image of the information code of the object and a distortion of the information code of the object.

According to this configuration, the information code of the object can be classified according to at least one of the contrast of the information code of the object in the actual image and the actual deformation of the information code of the object.

The information reading apparatus may further include: a classification unit configured to classify an information code of an object for which the reading process is successful in the first code process as a specific item among a plurality of items indicating intrinsic factors of the information reading apparatus; and a judging unit configured to judge whether or not a successful case of the information code of the object is adopted as the training data, based on the classified specific items. Here, the plurality of items may include at least one of two or more items related to image processing performed on the image of the information code by the information reading apparatus, two or more items related to a shooting condition under which the information code is shot by the information reading apparatus, and two or more items related to a processing time for the information reading apparatus to read the information recorded in the information code.

According to this configuration, it is possible to suppress a bias of successful cases in training data to cases in patterns corresponding to specific intrinsic factors.

The adjustment unit may be configured to start adjustment of the parameter of the learning model when the number of two or more successful cases, each of which indicates that the possibility of the information code failing to be read although the information code is successfully read, exceeds a predetermined number smaller than the predetermined number, even before the number of the plurality of successful cases included in the training data exceeds the predetermined number.

According to this configuration, even when the possibility of failure in reading the information code increases, learning can be started promptly without waiting for the number of successful cases to reach a predetermined number.

The present invention may further include: a first memory; and a first storage control unit configured to store a successful instance of the information code of the object in the first memory as the training data when the successful instance of the information code of the object is determined to be employed as the training data based on the classified specific pattern, and not to store a successful instance of the information code of the object in the specific memory when the successful instance of the information code of the object is determined not to be employed as the training data based on the classified specific pattern, wherein the adjustment unit refers to the training data in the first memory, and adjusts the parameter.

According to this structure, the amount of use of the first memory storing training data can be reduced.

The image processing device may further include an addition control unit that performs image processing on a code image that is an image of the information code captured by the camera, generates virtual cases of the reading processing, and increases the number of successful cases in the training data. Here, the image processing may include at least one of a process of adjusting a contrast of the code image, a process of adding a predetermined image to the code image, a process of rotating the code image, and a process of processing each cell of the information code represented by the code image.

In order to ensure the reliability of the parameters of the learning model, more than a prescribed number of successful cases are required. According to the above configuration, the number of successful cases can be increased when the number of successful cases does not exceed the predetermined number. Even if the number of actual successful cases does not exceed a predetermined number, the reliability of the parameters of the learning model can be ensured.

The image processing apparatus may further include a second memory that stores, for each of the plurality of information codes, case information indicating an instance of the reading process performed on the information code, and the addition control unit may select one or more image processes from among the plurality of types of image processes based on a tendency of the instance of the reading process indicated by the plurality of pieces of case information stored in the second memory, and may execute the selected one or more image processes.

For example, a comparative example is assumed in which image processing predetermined by a manager or the like is performed and a virtual case is generated. In this comparative example, the predetermined image processing may correspond to an instance different from the instance of the actual reading processing. According to the above configuration, an appropriate image processing can be performed in consideration of the tendency of the actual reading processing, and a virtual case can be generated.

The information processing apparatus may further include a second storage control unit that, when the reading process using the first relation information is performed, stores the case information indicating an instance of the reading process performed on the first information code in the second memory.

According to this configuration, it is possible to store and accumulate case information for generating virtual cases each time the reading process is performed.

The second code processing may include a cutting processing of cutting an image of the second information code from an image captured by the camera with reference to a position of an instruction mark irradiated to the second information code.

According to this configuration, the code image including the two-dimensional code can be simply cut from the captured image based on the set indication mark.

The control method of the information reading apparatus, the computer program for the information reading apparatus, and the storage medium storing the computer program are novel and useful.

Drawings

In the drawings:

fig. 1 is a conceptual diagram of an information reading system.

Fig. 2 is a block diagram of an information reading system.

Fig. 3 is a flowchart showing a process of the code reading apparatus.

Fig. 4 is a flowchart showing a normal reading process.

Fig. 5 is a flowchart showing a process of the learning apparatus.

Fig. 6 is a flowchart showing learning read processing.

Fig. 7 is a flowchart showing the processing of the reading apparatus of the second embodiment.

Fig. 8 is a flowchart showing the degradation classification processing.

Fig. 9 is a flowchart showing the pattern determination process.

Fig. 10 is a flowchart showing the processing of the code reading apparatus of the third and fourth embodiments.

Fig. 11 is a flowchart showing the processing of the learning device of the third and fourth embodiments.

Fig. 12 is a diagram showing a learning object of the fifth embodiment.

Fig. 13 is a flowchart showing learning read processing of the fifth embodiment.

Fig. 14 is a flowchart showing a process of the learning apparatus of the sixth embodiment.

Fig. 15 is a block diagram of an information reading system of the seventh embodiment.

Fig. 16 is a diagram showing classification based on the coarsening/refinement phenomenon.

Fig. 17 is a diagram showing classification based on distortion phenomenon.

Fig. 18 is a diagram showing classification based on the pitch misalignment phenomenon.

Fig. 19 is a flowchart of an intrinsic factor classification process of the ninth embodiment.

Fig. 20 is a flowchart of an intrinsic factor classification process of the tenth and eleventh embodiments.

Fig. 21 is a flowchart showing a process of the code reading apparatus of the twelfth embodiment.

Fig. 22 is a diagram showing specific example 1 of the shearing process.

Fig. 23 is a diagram showing specific example 2 of the shearing process.

Fig. 24 is a diagram showing specific example 3 of the shearing process.

Detailed Description

(first embodiment)

(information reading System; FIGS. 1 and 2)

The information reading system 2 of the present embodiment is a system for reading information recorded in a two-dimensional code area CR from a two-dimensional code CD having the two-dimensional code area CR and recording the information in the code area CR. For example, in factories, outdoor workplaces, and the like, the two-dimensional code CD is displayed on a specific medium (for example, metal, substrate, resin, paper medium, and the like). For example, as illustrated in fig. 1, the code region CR has a rectangular shape (square or rectangular shape), and black cells (dark cells) BC and white cells (light cells) WC are mapped according to a specific rule and the coded information (information to be displayed).

In addition, the two-dimensional code CR may be used for a long time. Therefore, the two-dimensional code CR may deteriorate over time. For example, with the lapse of time, a portion or whole of the two-dimensional code CR becomes lighter in black (a portion of black cells (dark cells) BR), and the contrast of the black portion of the two-dimensional code CR with respect to the white portion (a portion of white cells (light cells) WR) decreases. For example, a part of the two-dimensional code CR is defective and stains adhere to a part of the two-dimensional code CR.

As shown in fig. 2, the information reading system 2 includes two code reading apparatuses 10, a learning apparatus 200, and a printer 500. Each of the devices 10, 200, 500 is connected to the LAN4, and can communicate via the LAN 4. LAN4 is a wired LAN or a wireless LAN. The information reading system 2 includes two code reading apparatuses 10, but this is only an example. For example, the information reading system 2 may include only one code reading device 10, or may include three or more code reading devices 10.

(Structure of code reading device 10; FIGS. 1 and 2)

The code reading device 10 is a portable device for reading information recorded in the two-dimensional code CR. The appearance of the code reading device 10 shown in fig. 1 is merely an example, and the code reading device 10 may have the same appearance as a smart phone, for example.

The code reading device 10 includes an operation unit 12, a display unit 14, a camera 20, a communication interface 22, and a control unit 30. Hereinafter, the interface will be referred to as "I/F".

The operation unit 12 includes a plurality of keys. The user can input various instructions to the code reading device 10 by operating the operation unit 12. The display unit 14 is a display for displaying various information. The display unit 14 may also function as a touch panel (i.e., the operation unit 12) that can receive a user operation. The camera 20 includes a light emitting source such as an LED lamp and a CCD image sensor. The communication I/F22 is an I/F for performing communication via the LAN 4. The communication I/F22 is connected to the LAN 4.

The control unit 30 includes a CPU32 and a memory 34 (functioning as a non-transitory computer scale storage medium (non-transit computer readable recording medium)) including a nonvolatile memory or the like. The CPU32 executes various processes in accordance with the program 40 stored in the memory 34. In addition, the memory 34 further stores learning information 50 related to machine learning using the multi-layer neural network. Here, the multi-layer neural network is a function including an input layer, an intermediate layer, and an output layer, and data input to the input layer is processed by the intermediate layer and output from the output layer. For example, the multi-layer neural network is a convolutional neural network, a fully-coupled neural network, or the like. In addition, the mechanical learning is not limited to the multi-layer neural network, and for example, a support vector machine may be used. A multilayer neural network and the like are known techniques, and a detailed description thereof is omitted here.

The learning information 50 includes a learning model 52 and model parameters 54. The learning model 52 is a model (i.e., a mathematical formula) of a multi-layer neural network. Model parameters 54 are parameters of learning model 52, and specifically, various weight values in the middle layer of learning model 52. For example, the learning model 52 is installed from a server (not shown) provided by a vendor (vendor) of the information reading system 2. In the modification, the learning model 52 may be stored in the memory 34 in advance in the shipment stage of the code reading apparatus 10. On the other hand, model parameters 54 are generated by learning device 200 and stored in memory 34.

Learning device 200 is a device that adjusts model parameters 254 of learning model 252. The learning device 200 is, for example, a server. The learning device 200 includes a communication I/F222 and a control unit 230. Communication I/F222 is connected to LAN 4.

The control unit 230 includes a CPU232 and a memory 234. The CPU32 executes various processes according to a program 240 stored in the memory 234. In addition, the memory 234 further stores a plurality of training data 242 and learning information 250. The learning information 250 includes the learning model 52 similar to the code reading device 10, model parameters 254 similar to the code reading device 10, and initial parameters 256. Initial parameters 256 are initial values of model parameters 254 (i.e., initial values of various weights in the middle layer). The initial parameters 256 are predetermined, for example, by the vendor of the information reading system 2.

Training data 242 is information utilized for adjustment of model parameters 254. For example, model parameters 254 adjust each of the plurality of training data 242 such that errors in the output of learning model 252 when the training data 242 is input, and the correct values represented by the training data 242, are minimized.

(processing of the code reading device 10; FIG. 3)

With reference to fig. 3, a process executed by the CPU32 of the code reading apparatus 10 according to the program 40 will be described. The code reading device 10 receives an instruction to start reading the two-dimensional code through the operation unit 12 as a trigger, and starts the process of fig. 3.

When the CPU32 receives an instruction to shoot through the operation unit 12 in S2, it controls the camera 20 to shoot the two-dimensional code displayed on the specific medium (for example, metal or the like). Thus, the CPU32 acquires captured image data representing a captured image that is an image that has been captured by the camera 20.

In the next S4, the CPU32 executes a normal reading process for reading information recorded in the two-dimensional code from the captured image represented by the captured image data that has been acquired. The details of the general reading process will be described in fig. 4.

In the next S6, the CPU32 determines whether or not the reading of the information in the normal reading process is successful. If it is determined that the reading of the information in the normal reading process is successful (yes in S6), the CPU32 proceeds to S40. On the other hand, when determining that the reading of the information in the normal reading process has failed (no in S6), the CPU32 proceeds to S10.

In S10, the CPU32 determines whether the number of times of information read failure in the normal read processing corresponding to the instruction of shooting in S2 is greater than a prescribed threshold (for example, three times). When determining that the number of failures is equal to or less than the threshold value (no in S10), the CPU32 proceeds to S12.

In S12, the CPU32 changes the conditions (for example, sensitivity, exposure time, presence or absence of a light source, intensity of the light source, and the like) of the photographing of the camera 20, and photographs the two-dimensional code again. At the end of S12, the CPU32 returns to S4.

If the CPU32 determines that the number of failures is greater than the threshold value (yes in S10), the process proceeds to S20. In S20, the CPU32 determines whether the model parameters 54 in the memory 34 have been updated. Model parameters 54 are updated by receiving adjusted model parameters 254 from learning device 200 and storing adjusted model parameters 254 as model parameters 54 in memory 34. If the CPU32 determines that the model parameter 54 has been updated (yes in S20), the routine proceeds to S24.

In S24, the CPU32 executes learning read processing for reading information recorded in the two-dimensional code from a captured image represented by captured image data acquired by the camera 20. The learning reading process is different from the normal reading process of S4, and is a process using learning information 50. Details of the learning read process will be described later with fig. 6.

In the next S26, the CPU32 determines whether the reading of the information in the learning-reading process is successful. If it is determined that the information in the learning-reading process is successfully read (yes in S26), the CPU32 proceeds to S40.

In S40, the CPU32 outputs either one of the read result of the normal read process of S4 and the read result of the learning read process of S26. For example, the CPU32 causes the display unit 14 to display an image indicating the reading result. In addition, for example, the CPU32 transmits data indicating the read result to an external device (for example, a PC or the like).

In S42, when the two-dimensional code is successfully read in either the normal reading process in S4 or the learning reading process in S26, the CPU32 transmits the successful instance of the process of either one as the training data 242 to the learning device 200. Details of the training data 242 (i.e., successful instances) will be described later with respect to fig. 5.

In next S44, the CPU32 determines whether or not to accept an instruction to end reading of the two-dimensional code by the operation unit 12. When determining that the instruction to end the reading of the two-dimensional code is accepted (yes in S44), the CPU32 ends the processing of fig. 3. On the other hand, when determining that the instruction to end the reading of the two-dimensional code is not received (no in S44), the CPU32 returns to S2 to capture the two-dimensional code again.

In addition, if it is determined that the model parameter 54 has not been updated (no in S20) or if it is determined that the reading of the information in the learning-reading process has failed (no in S26), the process proceeds to S30. In S30, the CPU32 causes the display unit 14 to display a failure notification indicating that information reading from the two-dimensional code failed. When the process of S30 ends, the process of fig. 3 ends.

(general reading process; FIG. 4)

The normal reading process in S4 of fig. 3 will be described with reference to fig. 4. In S50, the CPU32 executes a clipping process of clipping a code image, which is an image representing a two-dimensional code, from the captured image by a predetermined number of pixels. Specific examples of the shearing process will be described later with reference to fig. 22 to 24.

In S52, the CPU32 specifies symbol marks (also referred to as viewfinder patterns) arranged at three of the four corners of the two-dimensional code from the code image cut out in S50. The CPU32 calculates position coordinates L1, L2, L4 of points representing corners (angles) of three out of four corners of the two-dimensional code from the specified three symbol marks. Further, the CPU32 calculates the remaining position coordinates L3 from the three calculated position coordinates L1, L2, L4. Thereby, position coordinates L1 to L4 of points representing four corners (corners) of the two-dimensional code (hereinafter, simply referred to as position coordinates L1 to L4 of the four corners) are calculated. In the drawings, the coordinates are simply referred to as coordinates L1 to L4 at four corners.

In S54, the CPU32 specifies the position coordinates of each cell constituting the two-dimensional code based on the position coordinates L1 to L4 of the four corners calculated in S52.

In S56, the CPU32 executes binarization processing for the code image. Thus, each cell of the two-dimensional code is identified as either white or black.

In S58, the CPU32 executes code processing for decoding the two-dimensional code based on the position coordinates of each cell specified in S54 and the black-and-white value of each cell specified in the binarization processing in S56. Here, the decoding process may include an error correction process. The error correction process is a process of repairing the original two-dimensional code based on the position coordinates of each cell and the black and white value of each cell when there is a difference between the original two-dimensional code and the two-dimensional code in the captured image due to the stain adhering to a part of the two-dimensional code or the like. Although the error correction process is performed a plurality of times in the decoding process, in the case where the decoding of the two-dimensional code also fails, the reading of the two-dimensional code based on the process of fig. 4 fails. On the other hand, in other cases, the reading of the two-dimensional code based on the processing of fig. 4 is successful. When the process of S58 ends, the process of fig. 4 ends.

(processing of learning device 200; FIG. 5)

Referring to fig. 5, the processing performed by CPU232 of learning device 200 is described in terms of program 240. The process shown in fig. 5 starts with the reception of training data 242 from the code reading apparatus 10 as a trigger. Here, the training data 242 is received from one of the two code reading apparatuses 10. In the modification, the training data 242 may be received from both the two code reading apparatuses 10.

In S60, the CPU232 stores the training data 242 received from the code reading apparatus 10 in the memory 234. Here, as a successful example of the normal reading processing of fig. 4, the training data 242 in the present embodiment includes code image data indicating the code image cut out in S50 and the position coordinates L1 to L4 of the four corners calculated in S52. The training data 242 may include a successful case of the learning read process of fig. 6, which will be described later.

In S62, the CPU232 determines whether the number of training data 242 in the memory 234 is equal to or greater than a target number (for example, 100). When determining that the number of training data 242 is equal to or greater than the target number (yes in S62), the CPU232 proceeds to S64. On the other hand, when it is determined that the number of training data 242 is smaller than the target number (no in S62), CPU232 ends the processing of fig. 5.

In S64, the CPU232 performs a learning process for adjusting the model parameters 254 of the learning model 252 using the plurality of training data 242 (i.e., with reference to the training data 242) in the memory 234. In the present embodiment, the learning model 252 is a model that inputs code image data to the input layer and outputs estimated values of position coordinates of four corners (position coordinates of points representing the four corners) of the two-dimensional code from the output layer. In the learning process of S64, the CPU232 selects one training data 242 from the plurality of training data 242, and inputs code image data within the selected one training data 242 to the input layer of the learning model 252. Thereby, the estimated values of the position coordinates of the four corners of the two-dimensional code are output from the output layer of the learning model 252. The CPU232 executes adjustment processing of adjusting the model parameters 254 of the middle layer of the learning model 252 so that the difference between the position coordinates L1 to L4 of the four corners within the one training data 242 that has been selected and the estimated value output from the learning model 252 is minimized. The CPU232 performs this adjustment process for all of the plurality of training data 242. Further, the plurality of training data 242 utilized in the learning process are deleted from the memory 234. Further, when the training data 242 of the target number or more is stored in the memory 234 again, the learning process is performed again. Thereby, the model parameters 254 are repeatedly adjusted. The frequency of execution of the learning process is not limited to the above example, and for example, the learning process may be executed every time training data 242 is received from the code reading device 10.

In S66, the CPU232 executes verification processing for detecting the accuracy of the model parameters 254 updated in S64. Specifically, the CPU232 inputs code image data representing the similar code image to the learning model 252, and obtains estimated values of position coordinates of four corners of the similar code image. The CPU232 uses the acquired estimation value (i.e., based on the estimation value), performs the same processing as S54 to S58 of fig. 4 on the similar code image, and performs reading of the two-dimensional code represented by the similar code image. The CPU232 counts the number of times of error correction processing performed in reading of the similar two-dimensional code. When the counted number of times is equal to or less than a predetermined number of times (for example, twice), the CPU232 determines that the verification result "pass" indicating that the accuracy of the model parameter 254 is good. On the other hand, when the counted number of times is larger than the predetermined number of times, the CPU232 determines that the verification result "reject" indicating that the accuracy of the model parameter 254 is poor. In the modification, the verification process is not limited to the verification based on the number of times of the error correction process, and may be, for example, verification based on a difference between the estimated values of the position coordinates of the four corners of the learning model 252 and the actual position coordinates of the four corners of the similar code image.

In S68, the CPU232 determines whether the verification result of S66 indicates "pass". If the CPU232 determines that the verification result indicates "acceptable" (yes in S68), it proceeds to S70. On the other hand, when the CPU232 determines that the verification result indicates "failure" (no in S68), the process of S70 is skipped and the process of fig. 5 is terminated.

In S70, the CPU232 transmits the model parameters 254 adjusted in S64 to the code reading apparatus 10. Here, the adjusted model parameters 254 are transmitted not only to the first code reading device 10 that is the transmission source of the training data 242, but also to the second code reading device 10 that is not the transmission source of the two code reading devices 10, out of which the adjusted model parameters 254 are transmitted. With this configuration, the second code reading device 10 can perform learning and reading processing using the success cases of the other code reading devices 10 (see fig. 6). In the modification, the adjusted model parameters 254 may not be transmitted to the second code reading apparatus 10. When S70 ends, the process of fig. 5 ends.

(learning read process; FIG. 6)

The learning read processing in S24 of fig. 3 will be described with reference to fig. 6. S80 is the same as S50 of fig. 4. In S82, the CPU32 of the code reading apparatus 10 inputs code image data representing the code image clipped in S80 to the learning model 52 in the memory 34, and obtains estimated values of position coordinates of four corners of the code image from the learning model 52. S84 to S88 are the same as S54 to S58 of fig. 4 except that the estimated value obtained in S82 is used. When S88 ends, the process of fig. 6 ends.

(effects of the present embodiment)

According to the configuration of the present embodiment, learning device 200 refers to training data 242 including a plurality of successful cases of decoding processing using position coordinates L1 to L4 of four corners of the two-dimensional code calculated in S52 of the normal reading processing of fig. 4, and adjusts model parameters 254 of learning model 252 that outputs estimated values of the position coordinates of the four corners of the two-dimensional code from the image of the two-dimensional code (S64 of fig. 5). When the reading in the normal reading process fails although the imaging condition is changed (no in S10 after S12 in fig. 3), the code reading device 10 executes the learning reading process (S24). In the learning-reading processing, instead of the calculation based on S52 of the normal-reading processing, processing of outputting estimated values of position coordinates of four corners with reference to the learning model 52 is performed (S82 of fig. 6). For example, it is assumed that the calculation of the position coordinates L1 to L4 of the four corners of the two-dimensional code fails due to the aged deterioration of the two-dimensional code, and the reading by the normal reading process fails. In the present embodiment, even in the case where the calculation of the position coordinates L1 to L4 of the four corners of the two-dimensional code fails due to the aged deterioration of the two-dimensional code, the information within the two-dimensional code can be read by performing the learning reading process.

For example, a comparative example is assumed in which the code reading apparatus 10 executes the processing of fig. 6 without the learning apparatus 200 in the information reading system 2. In this comparative example, the code reading apparatus 10 performs not only the reading of the two-dimensional code but also the adjustment of the model parameters 54 of the learning model 52. Therefore, the processing capacity of the CPU32 of the code reading apparatus 10 needs to be improved as compared with a conventional apparatus that does not perform adjustment of the model parameters 54 of the learning model 52. In contrast, according to the structure of the present embodiment, the adjustment of the model parameters 254 of the learning model 252 is performed by the learning device 200 independent of the code reading device 10. Therefore, the adjustment of the model parameters 254 can be performed without increasing the processing capability of the CPU32 of the code reading apparatus 10. In the modification, the structure of the comparative example may be adopted.

(correspondence relation)

The information reading system 2 is an example of "information reading apparatus". The two code reading apparatuses 10, one of the code reading apparatuses 10, and the other code reading apparatus 10 in fig. 2 are examples of "one or more code reading apparatuses", "first code reading apparatus", and "second code reading apparatus", respectively. The camera 20 is an example of a "camera". The learning device 200 is an example of a "learning device". The two-dimensional code is an example of "information code (first information code and second information code)". The position coordinates L1 to L4 of the four corners of the two-dimensional code are an example of "first relation information". The estimated values of the position coordinates of the four corners of the two-dimensional code are one example of "relationship information (and second relationship information)". Training data 242, learning model 252, and model parameters 254 are examples of "training data", "learning model", and "parameters", respectively. The process of fig. 4 is an example of "first code process" in S40 of fig. 3. S52, S58 in fig. 4 and S40 in fig. 3 are examples of "detection processing", "reading processing" and "output processing", respectively. The process of fig. 6 and S40 of fig. 3 are examples of "second code processing". S82 in fig. 6 is an example of "acquisition processing".

The control unit 30 of the code reading apparatus 10 is an example of a "first code processing executing unit" and a "second code processing executing unit". The control unit 230 of the learning device 200 is an example of an "adjustment unit".

(second embodiment)

The present embodiment is the same as the first embodiment except that a part of the contents of the processing of the code reading apparatus 10 is different.

(processing of the code reading device 10; FIG. 7)

The processing of the code reading apparatus 10 of the present embodiment is the same as the processing of fig. 3, which is the processing of the code reading apparatus 10 of the first embodiment, except that the processing of S100 to S102 is added and the processing of S104 is executed instead of S42. The same reference numerals are given to the same processes as those of the first embodiment, and the same applies to the embodiments described later.

As shown in fig. 7, when the CPU32 outputs the read result in S40, in S100, the degradation classification process is executed. The degradation classification process classifies the two-dimensional code to be read according to the degraded pattern, and stores the successful instance corresponding to the read result of the two-dimensional code as the training data 242 in the training data table 60. The pattern of deterioration is a pattern based on the presence or absence of adhesion of stains, the presence or absence of defects, deteriorated sites, and the like. The degradation classification process will be described later with fig. 8.

The training data table 60 is stored in the memory 34 of the code reading device 10. As shown in fig. 7, the training data table 60 is a table in which, for each of a plurality of degraded patterns, a pattern number (for example, "p 001"), a data group (for example, "TD1, TD 2"), and an upper limit value (for example, "20") are stored in association with each other. The pattern number is a number identifying the corresponding degraded pattern. The data set is one or more training data 242 classified as a corresponding degraded pattern. The upper limit value is an upper limit value of the number of training data 242 that can be stored in the corresponding degraded pattern.

In the next S102, the CPU32 determines whether the number of training data 242 stored in the training data table 60 is equal to or greater than the target number. Here, the target number is the same as that of S62 in fig. 5, for example. When determining that the number of training data 242 is equal to or greater than the target number (yes in S102), the CPU32 proceeds to S104. On the other hand, when determining that the number of training data 242 is smaller than the target number (no in S102), the CPU32 skips the processing in S104 and proceeds to S44.

In S104, CPU32 transmits all training data 242 in training data table 60 to learning device 200. When S104 ends, the CPU32 proceeds to S44.

(degradation classification process; FIG. 8)

The degradation classification process of S100 of fig. 7 will be described with reference to fig. 8. In S110, the CPU32 determines whether or not the error correction process is performed in the normal reading process of S4 or the learning reading process of S24 of fig. 7. When determining that the error correction process has been performed (yes in S110), the CPU32 proceeds to S112. On the other hand, when the CPU32 determines that the error correction process is not executed (no in S110), the process in S112 is skipped, and the process proceeds to S114.

In S112, the CPU32 compares the image of the repair code, which is the two-dimensional code repaired in the error correction process, with the binarized image of the actual code, which is the two-dimensional code actually photographed, and executes a pattern determination process of determining the pattern of degradation of the actual code. The image of the repair code is represented by black and white binary values. The binarized image of the actual code is an image obtained by binarizing an actual image of the actual code. The pattern determination process will be described later with fig. 9. When the process of S112 ends, the CPU32 proceeds to S114.

In S114, the CPU32 determines whether or not the stored number of the data group of the object in the training data table 60 reaches the upper limit number corresponding to the data group of the object. Here, when the error correction process is not performed (no in S110), the target data set is a data set associated with the pattern number identifying the pattern without degradation. In addition, when the error correction process is performed on the target data set (yes in S110), the data set is associated with the pattern number identifying the pattern determined by the pattern determination process. If the CPU32 determines that the number of stored data sets to be stored has not reached the upper limit number (no in S114), the routine proceeds to S116. On the other hand, when the CPU32 determines that the number of stored data sets to be stored has reached the upper limit number (yes in S114), the process of S116 is skipped and the process of fig. 8 is terminated.

In S116, the CPU32 stores the success instance corresponding to the read result of S40 of fig. 7 as the training data 242 in the training data table 60. When S116 ends, the process of fig. 8 ends.

For example, it is conceivable that the judgment of S114 is not performed, but all successful cases are stored as comparative examples in the training data table 60 as the training data 242. According to the configuration of the present embodiment, the number of stored training data 242 in the training data table 60 can be limited by the judgment of S114. The amount of memory 34 storing training data table 60 can be reduced as compared with the comparative example. In addition, in the modification, the structure of the comparative example may be adopted.

The pattern determination process of S112 of fig. 8 will be described with reference to fig. 9. In the present embodiment, in order to classify the deteriorated portions, the two-dimensional code is equally divided into 9 areas. Degradation values, which are values indicating types of degradation, are associated with 9 areas, respectively. In S130, the CPU32 determines one of the 9 areas as the object area. The number of divided regions is not limited to 9, and may be, for example, 4 or 12. In addition, the division of the region is not limited to the division, and for example, the sizes of the regions may be different from each other.

In S132, the CPU32 determines whether there is a difference between the target area of the image of the repair code and the target area of the binarized image of the actual code. The difference between the target area of the image of the repair code and the target area of the binarized image of the actual code means that the target area of the photographed two-dimensional code is degraded. If it is determined that there is a difference between the target areas of the two codes (yes in S132), the CPU32 proceeds to S134. On the other hand, when determining that there is no difference between the target areas of the two codes (no in S132), the CPU32 skips the processing in S134 to S138, and proceeds to S140.

In S134, the CPU32 determines whether the difference between the target areas of the two codes is a difference corresponding to black degradation or a difference corresponding to white degradation. Here, the black degradation is degradation in which white cells of the two-dimensional code are blackened. Black degradation is, for example, a black ink adhering to stains in the two-dimensional code. In addition, the white degradation is degradation in which black cells of the two-dimensional code become white. The white deterioration is, for example, a stain of a white ink adhering to the two-dimensional code, a defect of a black cell of the two-dimensional code, or the like.

If the CPU32 determines that the difference between the target areas of the two codes is a difference corresponding to black degradation (yes in S134), the process proceeds to S136. In S136, the CPU32 determines the degradation value corresponding to the target area as a value "black" representing black degradation. On the other hand, when determining that the difference between the target areas of the two codes is a difference corresponding to white degradation (no in S134), the CPU32 proceeds to S138. In S138, the CPU32 determines the degradation value corresponding to the target area as a value "white" indicating white degradation. When one of S136 and S138 ends, the CPU32 proceeds to S140.

In S140, the CPU32 determines whether there are unselected areas among the 9 areas as the target area. If the CPU32 determines that there is an unselected area as the target area (yes in S140), it returns to S130. On the other hand, when there is no unselected area as the target area, that is, when the processing of S132 to S138 is performed for all 9 areas (no in S140), the CPU32 ends the processing of fig. 9.

(effects of the present embodiment)

To ensure reliability of model parameters 254, successful instances exceeding a target number (e.g., 100) are required. However, when the successful cases are unconditionally used as the training data 242, at the time when the number of training data 242 exceeds the target number, the deviation of the successful cases may be biased to, for example, cases in a specific deteriorated pattern (for example, black deterioration in a specific area). According to the configuration of the present embodiment, the code reading apparatus 10 classifies successful cases of the two-dimensional code to be read by the pattern of degradation of the two-dimensional code (S112 of fig. 8). Based on the classification result, the code reading apparatus 10 determines whether or not to store the successful instance of the two-dimensional code to be read as the training data 242 in the training data table 60 (S114). This can suppress cases in patterns in which the successful cases of the training data 242 are biased to specific degradations.

In the present embodiment, in the classification of the degraded pattern, the presence or absence of the adhesion of the stain, the presence or absence of the defect, and the degraded portion are used. Here, in the modification, the contrast may be used for classifying the degraded pattern. For example, the degraded pattern may be classified based on a difference between an upper limit value and a lower limit value of the luminance value of the code image in the cutting process in S50 of fig. 4 or S80 of fig. 6. For example, if the difference is a value in a first value range, a successful instance of the two-dimensional code to be read may be classified as a first pattern, and if the difference is a value in a second value range different from the first value range, a successful instance of the two-dimensional code to be read may be classified as a second pattern. In this modification, deterioration due to the adhesion of stains or defects, which cannot be classified, can be classified based on contrast. In another modification, the degradation of the two-dimensional code may be classified by combining the classification based on the adhesion of stains and defects with the classification based on the contrast.

In another modification, the modification of the information code may be used for classifying the degraded pattern. The distortion of the information code is, for example, a roughening/thinning phenomenon of fig. 16, a distortion phenomenon of fig. 17, a pitch misalignment phenomenon of fig. 18, or the like. For example, the deformation of the information code occurs due to degradation of a printing device that prints the two-dimensional code or unstable support of a medium that prints the information code at the time of printing.

(classification based on coarsening/refinement phenomenon; FIG. 16)

The coarsening/thinning phenomenon includes a coarsening phenomenon and a thinning phenomenon. The coarsening phenomenon is a phenomenon in which the actual width of the black cell is thicker than the ideal width. On the other hand, the thinning phenomenon is a phenomenon in which the actual width of the black cell is smaller than the ideal width.

For example, a time-series pattern (timing pattern) of a two-dimensional code is analyzed to calculate a roughening/thinning ratio indicating the degree of roughening/thinning phenomenon. The timing pattern is utilized for the specification of the position coordinates of the symbol mark. The timing pattern is a pattern in which white cells and black cells are alternately arranged. For example, the roughening/thinning ratio in the lateral direction is calculated as a ratio of a difference between the sum of the widths of the black cells of the lateral timing pattern and the sum of the widths of the white cells of the lateral timing pattern to the entire length of the lateral timing pattern. The "0%" of the coarsening/thinning ratio in the lateral direction means that no coarsening/thinning phenomenon occurs, the coarsening/thinning ratio in the lateral direction means that the coarsening phenomenon occurs in the lateral direction, and the coarsening/thinning ratio in the lateral direction means that the thinning phenomenon occurs in the lateral direction. Further, using the time series pattern in the vertical direction (i.e., based on the time series pattern), the ratio of roughening/thinning in the vertical direction is calculated in the same manner as the ratio of roughening/thinning in the horizontal direction. For example, the ratio of coarsening/thinning of the entire two-dimensional code is calculated as an average value of the ratio of coarsening/thinning in the lateral direction and the ratio of coarsening/thinning in the longitudinal direction. In the modification, the overall ratio of coarsening/thinning may be a larger value of the ratio of coarsening/thinning in the lateral direction and the ratio of coarsening/thinning in the longitudinal direction, or may be a larger value of the ratio of coarsening/thinning in the lateral direction and the ratio of coarsening/thinning in the longitudinal direction.

In this modification, the CPU32 classifies successful cases of the two-dimensional code to be read based on the overall roughening/thinning ratio, and stores the classified cases in the training data table 60 (see S116 in fig. 8). For example, as shown in fig. 16, successful cases of the two-dimensional code to be read are classified according to five value ranges of the overall roughening/thinning ratio. Note that the five ranges in fig. 16 are only examples.

(classification based on distortion phenomenon; FIG. 17)

In general, the ratio of the longitudinal length to the transverse length (hereinafter, the "aspect ratio") of the two-dimensional code is 1:1. The distortion phenomenon is a phenomenon in which the transverse length of the two-dimensional code is distorted with respect to the longitudinal length of the two-dimensional code, and the aspect ratio is changed from 1:1. For example, the aspect ratio is calculated as a ratio of a distance between centers of cells adjacent in the longitudinal direction to a distance between centers of cells adjacent in the transverse direction.

In this modification, the CPU32 classifies successful cases of the two-dimensional code to be read based on the aspect ratio, and stores the classified cases in the training data table 60 (see S116 in fig. 8). For example, as shown in fig. 17, successful cases of the two-dimensional code to be read are classified according to six patterns of aspect ratios. Note that the six patterns in fig. 17 are only examples.

(classification based on pitch misalignment phenomenon; FIG. 18)

In general, cells constituting a two-dimensional code are arranged at a constant interval (i.e., pitch) in the longitudinal direction and the transverse direction. The pitch shift phenomenon is a phenomenon in which the pitch of adjacent cells is shifted from the ideal pitch as the constant pitch. For example, the pitch shift ratio indicating the degree of pitch shift is calculated as the ratio of the maximum amount of pitch shift to the ideal pitch. The amount of pitch misalignment is calculated as the absolute value of the difference between the ideal pitch and the distance between the centers of adjacent cells. In the modification, the pitch shift ratio may be a ratio of an average value of the amounts of pitch shift to an ideal pitch, for example.

In the present modification, the CPU32 classifies successful cases of the two-dimensional code to be read based on the pitch shift ratio, and stores the classified cases in the training data table 60 (see S116 in fig. 8). For example, as shown in fig. 18, successful cases of the two-dimensional code to be read are classified according to four value ranges of the pitch misalignment ratio. Note that the four ranges in fig. 18 are only examples.

(correspondence relation)

The target number in S102 in fig. 7 is an example of "predetermined number". The memory 34 of the code reading apparatus 10 is an example of "first memory". The control unit 30 of the code reading apparatus 10 that executes the processing of S100 of fig. 7 is an example of a "classification unit" and a "determination unit".

(third embodiment)

The present embodiment is the same as the first embodiment except that the content of the processing of the code reading apparatus 10 and a part of the content of the processing of the learning apparatus 200 are different.

(processing of the code reading device 10; FIG. 10)

The processing of the code reading apparatus 10 of the present embodiment is the same as the processing of fig. 3, which is the processing of the code reading apparatus 10 of the first embodiment, except for the processing of adding S200.

As shown in fig. 10, when the normal reading process is executed in S4, the CPU32 transmits case information about the case of the process executed in the normal reading process to the learning device 200 in S200. Thus, the case information is stored in the case table 270 (see fig. 11) in the memory 234 of the learning device 200. According to this configuration, each time the normal reading process is executed, the case information can be stored in the case table 270, and the case information can be stored.

As shown in fig. 11, the case table 270 is a table in which processing information indicating the content of the processing, cause information indicating the cause of the processing, and the number of times of occurrence are stored in association with each other for each of the processing performed in the normal reading processing. The processing information indicates, for example, "symbol" indicating a specific failure of at least one symbol mark in the normal reading processing, "error correction" indicating that the error correction processing has been performed in the normal reading processing, and the like. The cause information is information indicating the cause of the corresponding process, and indicates, for example, the type of degradation such as "black degradation", "white degradation" (see fig. 9). The number of occurrences indicates the number of occurrences of processing indicated by a combination of the corresponding processing information and the corresponding cause information. The number of occurrences is incremented each time a combination of corresponding processing information and corresponding cause information (i.e., case information) is received from the code reading apparatus 10.

For example, in a case where black degradation of the two-dimensional code is likely to occur in a predetermined area (for example, a predetermined factory) using the code reading device 10, the number of occurrences corresponding to the cause information "black degradation" in the case table 270 is relatively large. The tendency of deterioration in the condition of using the code reading apparatus 10 can be known by the information in the case table 270.

(processing of learning device 200; FIG. 11)

The process of the learning apparatus 200 of the present embodiment is the same as the process of fig. 5, which is the process of the learning apparatus 200 of the first embodiment, except that the process of S205 is performed instead of S62 of fig. 5, and the processes of S210 and S220 are added.

S205 is the same as S64 of fig. 5 except that the judgment is made based on the number of sub-targets (for example, 50) smaller than the number of targets. When determining that the number of training data 242 in the memory 234 is equal to or greater than the sub-target number (yes in S62), the CPU232 proceeds to S210.

In S210, the CPU232 determines one or more processing processes to be executed in S220 described later. The processing is a process of generating new training data 242 by processing the training data 242 in the memory 234. In the processing, information in the case table 270 is used. For example, the CPU32 specifies case information (i.e., processing information and cause information) stored in association with the most number of occurrences from the case table 270. Then, the CPU32 determines the processing to be performed in S220 according to the specified case information. Further, the CPU32 specifies case information stored in association with the second largest number of occurrences from the case table 270, and specifies other processing according to the specified case information. In addition, for example, the CPU32 may randomly specify case information from the case table 270. The CPU32 may determine one processing or two or more processing.

The processing of the training data 242 is various. For example, in the processing, when the specified processing information indicates "symbol" and the specified cause information indicates "white deterioration", the processing is performed to perform image processing for whitening a part of the symbol mark with respect to the code image indicated by the training data 242. The image processing is, for example, processing of adding a white image to a part of the symbol mark, processing of reducing or enlarging a part of a plurality of cells constituting the symbol mark, and the like. For example, when the specified processing information indicates "error correction", the image processing may be performed on cells other than the cells constituting the symbol mark.

Further, for example, the CPU32 may execute various image processing including processing for adjusting the contrast of the code image, processing for rotating the code image, and the like, with respect to the code image represented by the training data 242, in addition to the image processing according to the cause information (for example, "white degradation"). This increases the variation in the processed training data 242. In the case of performing the process of rotating the code image, for example, the values of the position coordinates L1 to L4 of the four corners indicated by the training data 242 may be converted into values rotated at the same rotation angle as the code image.

For example, the CPU32 may execute processing such as processing for adjusting the contrast of the code image and processing for rotating the code image, instead of executing image processing according to the cause information.

In S220 following S210, the CPU32 executes each of the one or more processing determined in S210. The CPU32 adjusts, for example, a portion to which a white image is added, a rotation angle of the rotation code image, and the like, and executes one processing process a plurality of times. The CPU32 executes each of the one or more processing steps a plurality of times until the number of training data 242 in the memory 234 reaches a predetermined target number (for example, 100). Upon completion of S220, the CPU232 advances to S64.

(effects of the present embodiment)

To ensure reliability of model parameters 254, successful instances exceeding a target number (e.g., 100) are required. According to the configuration of the present embodiment, when the number of successful cases does not exceed the target number, processing can be performed to generate virtual successful cases, that is, virtual training data 242 (S220 in fig. 11). This can increase the number of successful cases, that is, the number of training data 242. Even if the number of successful cases does not exceed the target number, reliability of the model parameters 254 can be ensured. In addition, when the number of actual successful cases does not reach the target number, the model parameters 254 with relatively high reliability can be quickly generated.

For example, a comparative example is assumed in which the case table 270 is not provided, and for example, a comparative example is assumed in which processing using image processing predetermined by a manager or the like of the learning apparatus 200 is performed. In this comparative example, the processing may correspond to a case different from the case of the actual reading processing. According to the configuration of the present embodiment, the case table 270 stores the actual normal reading processing cases. Then, learning device 200 determines one or more processing processes from example table 270 (S210 in fig. 11). An appropriate processing process can be performed in consideration of the tendency of the case of the actual reading process.

(correspondence relation)

The control unit 230 of the learning device 200 that executes the process of S220 of fig. 11 is an example of an "increase control unit". The memory 234 of the learning device 200 is an example of "second memory". The control unit 230 that stores the case information transmitted in S200 of fig. 10 in the memory 234 is an example of "second storage control unit".

(fourth embodiment)

( A process of the code reading apparatus 10 and the learning apparatus 200; FIGS. 10 and 11 )

The present embodiment is the same as the third embodiment except that a plurality of case tables 270 (see fig. 11) are used, and the content of the case information transmitted in S200 of fig. 10 is different from a part of the content of the training data 242 transmitted in S42.

The two-dimensional codes to be read are various. For example, two-dimensional codes are displayed on various media (e.g., substrates, metals, paper media, etc.). In addition, for example, the two-dimensional code is displayed on the medium by various display methods (for example, printing, cutting, etc.). For example, the two-dimensional code is generated according to a specific specification among various specifications (for example, specifications of size, encryption, and the like). Each of the plurality of case tables 270 corresponds to each of a plurality of categories of the two-dimensional code to be read.

In S200 of fig. 10, the case information transmitted to the learning device 200 includes, in addition to the information of the third embodiment (i.e., the process information and the cause information), specific information for specifying the type of the two-dimensional code to be read in the normal reading process. The specific information is information indicating a medium, a display method, a specification, and the like of the reading target. For example, specific information is input to the code reading device 10 by a user of the code reading device 10.

Upon receiving the case information including the specific information from the code reading apparatus 10, the learning apparatus 200 specifies one case table 270 to store the received case information from among the plurality of case tables 270 in the memory 234, based on the specific information in the case information. Then, learning device 200 stores information in the received case information in a specific one of case tables 270.

In S42 of fig. 10, the code reading device 10 transmits the specific information to the learning device 200 together with the training data 242.

Upon receiving the training data 242 and the specific information from the code reading apparatus 10, the learning apparatus 200 stores the received training data 242 in the memory 234 as the training data 242 corresponding to the category specified by the received specific information in S60 of fig. 11. When determining that the number of training data 242 corresponding to the received specific information is equal to or greater than the sub-target value (yes in S205), learning device 200 proceeds to the processing at S210 and thereafter. Specifically, the learning device 200 specifies one of the plurality of case tables 270 in the memory 234 to be used in the learning process of S64 in fig. 11, based on the specifying information in the training data 242. Then, learning device 200 refers to the specified one instance table 270, and executes the processing after S210.

According to the configuration of the present embodiment, case information is accumulated for each of various types of two-dimensional codes to be read. For example, it is possible to read a specific kind of reading object not only on the latest day of the use of the code reading device 10 but also on the past day of the use. In addition, a specific type of reading target may be read in a region different from a predetermined region using the code reading apparatus 10. According to the configuration of the present embodiment, it is possible to perform processing with reference to not only the case of a specific type of two-dimensional code read in a predetermined area on the latest day but also the case of a specific type of two-dimensional code read in a different area or the like at the past date and time (S220 of fig. 11).

(fifth embodiment)

This embodiment is the same as the first embodiment except that the content of the training data 242 and the value output by the learning model 252 are different, and a part of the learning read process is different.

(training data and learning model; FIG. 12)

The learning model 252 of the present embodiment is a model in which code image data is input to an input layer, and an estimated value of a black-and-white pattern of cells of a two-dimensional code is output from an output layer. The data of the black-and-white pattern of the cells of the two-dimensional code (hereinafter referred to as "pattern data") is generated, for example, from the position coordinates of each cell specified by the normal reading process (for example, refer to S54 of fig. 4) and the black-and-white value of each cell specified by the binarization process of the process. For example, the position coordinates of the cells are defined as (N, 0), (0, N), (N, N) by using the upper left cell as the origin (0, 0), and the lower left cell as the lower right cell. Here, "N" is the number of cells on one side of the two-dimensional code, for example, 24. Further, the position coordinates of the cells are given continuous numbers in the order of the upper left cell, the upper right cell, the lower left cell, and the lower right cell. The pattern data is data in which the numbers of the cells are associated with each other, and the black and white values of the cells indicated by the numbers are associated with each other. Here, the values of black and white, for example, "white" means "1", and "black" means "0". In the case where the error correction process is performed in S58 of the normal reading process of fig. 4, the value of black and white in the pattern data may be a value repaired by the error correction process.

The training data 242 of the present embodiment includes code image data representing the code image cut out in S50 of the normal reading process of fig. 4 and the pattern data described above. The learning device 200 refers to the training data 242, and executes learning processing for adjusting the model parameters 254 (refer to S64 in fig. 5). The training data 242 may include code image data representing the code image cut out in S80 of the learning and reading process of fig. 13 described later in the present embodiment and pattern data generated in accordance with the learning and reading process of fig. 1 3. In this case, the pattern data may be generated based on the position coordinates of each cell specified in S84 of fig. 13 and the black-and-white value of each cell estimated in S300 of the process.

(learning read process; FIG. 1 3)

The learning-reading process of the present embodiment is the same as the process of fig. 6 of the first embodiment except that the process of S300 is performed instead of the binarization process of S86.

In S300, the CPU232 inputs code image data representing the code image cut out in S80 to the learning model 52 in the memory 34, and obtains an estimated value of the black-and-white pattern of the two-dimensional code represented by the code image from the learning model 52. In S88 following S300, the CPU32 refers to the position coordinates of each cell specified in S84 and the estimated value of the black-and-white pattern acquired in S300, and determines the black-and-white value of each cell of the two-dimensional code. Further, the CPU32 executes code processing based on the determined black-and-white value of each cell.

In the learning-reading process of the present embodiment, instead of the binarization process of S86 based on the normal-reading process, the learning model 52 is referred to determine the black-and-white value of each cell of the two-dimensional code (S300 of fig. 13). For example, it is assumed that the color of the two-dimensional code is mostly changed in black and white due to aged deterioration of the two-dimensional code, and thus a reading failure by a normal reading process is caused. In the present embodiment, even when the value of black and white of each cell determined by binarization processing of the two-dimensional code due to aged deterioration of the two-dimensional code is greatly different from the value of black and white of each cell of the two-dimensional code before deterioration, information in the two-dimensional code can be read by executing learning reading processing.

Further, it is assumed that the pattern data is a comparative example representing a plurality of pixels of the two-dimensional code. In this comparative example, the information amount of the pattern data increases together with an increase in the number of pixels. In contrast, according to the configuration of the present embodiment, since one value is associated with one cell, the information amount of the pattern data does not increase even if the number of pixels increases as long as the number of cells of the two-dimensional code is constant. The increase in the information amount of the pattern data can be suppressed as compared with the above-described comparative example. In the modification, the structure of the comparative example may be adopted.

In this embodiment, the black-and-white pattern of the cells of the two-dimensional code includes the black-and-white value of each cell for all the cells. For example, the two-dimensional code includes cell groups arranged in a predetermined pattern such as a symbol mark (i.e., a viewfinder pattern), an alignment pattern, and a time-series pattern. In the modification, the black-and-white pattern of the cells of the two-dimensional code may include the black-and-white value of the plurality of cells excluding the cell group arranged in the above-described predetermined pattern from all the cells, and may not include the black-and-white value of the cell group arranged in the above-described predetermined pattern.

In this embodiment, as shown in fig. 12, when the two-dimensional code is a code having the size of the number of cells (24, 24), continuous numbers in the black-and-white pattern of each cell are given 1 to 624. 624 is the same value as the number of cells of the two-dimensional code. In the modification, 1 to M (M is a value larger than 624) may be given to the continuous number in the black-and-white pattern of each cell. Here, for example, M may be the number of cells in the maximum size of the two-dimensional code, for example, (177 ) = 31329. In this case, 624 consecutive numbers may be used among consecutive numbers of 1 to 31329, and the remaining consecutive numbers may not be used. For example, all values corresponding to the remaining consecutive numbers may be set to "0". In this modification, when there are a plurality of two-dimensional codes, one learning model 252 can be shared by learning the two-dimensional codes of the plurality of sizes.

(correspondence relation)

The pattern data is an example of "first relation information". The estimated value of the pattern data is one example of "relationship information (and second relationship information)".

(sixth embodiment)

The present embodiment is the same as the first embodiment except for a part of the processing of the learning device 200.

(processing of learning device 200; FIG. 14)

The process of the learning device 200 of the present embodiment is the same as the process of fig. 5 of the first embodiment except that the process of S400 is performed instead of the process of S70 of fig. 5. In the present embodiment, in the learning process in S64, all of the model parameters 254 of the learning model 252 may be adjusted, or a part of the model parameters 254 may be adjusted. In the case of adjusting a part of the model parameters 254, for example, the parameters on the input layer side of the multi-layer neural network may be fixed to predetermined values, and the parameters on the output layer side of the multi-layer neural network may be adjusted in the learning process.

If the CPU232 determines that the verification result indicates "acceptable" (yes in S68), it proceeds to S400. In S400, the CPU232 generates a two-dimensional code that records the model parameters 254 adjusted in S64. Further, the CPU232 transmits print data indicating the generated two-dimensional code to the printer 500. Thereby, the two-dimensional code is printed.

The code reading device 10 reads the printed two-dimensional code. Thus, the code reading device 10 obtains the model parameters 254 in the two-dimensional code and stores them in the memory 34.

According to this configuration, the code reading apparatus 10 can obtain the model parameter 254 by reading the two-dimensional code. For example, it is assumed that there are a first code reading device 10 that is a transmission source of the training data 242 and a second code reading device 10 that is not the transmission source, and that the second code reading device 10 is not connected to the LAN 4. The second code reading device 10 can obtain the model parameters 254 by reading the printed two-dimensional code even if communication with the learning device 200 is not performed.

The information recorded in the two-dimensional code is not limited to the plain text of the model parameter 254, and may be, for example, data obtained by reducing the model parameter 254 in the plain text. The CPU232 may divide the model parameter 254 into a plurality of pieces of data, and generate a two-dimensional code in which the data is recorded for each of the divided pieces of data. According to this configuration, even when the information amount of the model parameter 254 exceeds the information amount storable in one two-dimensional code, the two-dimensional code in which the model parameter 254 is recorded can be created.

The information recorded in the two-dimensional code is not limited to all of the model parameters 254, and may be a part of the model parameters 254. For example, in the case where the value of a part of the model parameters 254 is fixed, only a part of the parameters adjusted in the learning process of S64 may be recorded in the two-dimensional code. For example, when all the model parameters 254 are adjusted, only the parameters that are changed from the model parameters 254 before adjustment among the adjusted model parameters 254 may be recorded in the two-dimensional code. The model parameters 254 before adjustment may be, for example, the model parameters 254 in the shipment stage or the model parameters 254 after adjustment in the previous learning process. The amount of information recorded in the two-dimensional code can be reduced as compared with a configuration in which all of the model parameters 254 are recorded.

(correspondence relation)

The printer is an example of the "output device". In the modification, the learning device 200 may display the two-dimensional code in which the model parameter 254 is recorded on a display (not shown) that can communicate with the learning device 200. In this modification, the display is an example of the "output device".

(seventh embodiment)

In the present embodiment, the code reading apparatus 10 stores a plurality of learning information 50 in the memory 34. For example, the code reading device 10 is utilized in various situations. For example, a situation is envisaged in which the code reading device 10 is used not only in a factory but also in the field. For example, it is assumed that the code reading device 10 reads not only a two-dimensional code displayed on a first medium (for example, a paper medium) but also a two-dimensional code displayed on a second medium (for example, a metal) different from the first medium. According to the configuration of the present embodiment, two pieces of unique learning information 50 can be stored for each situation. This can cope with various situations.

The memory 34 is divided into a plurality of memory areas including memory areas a1, b2. The program 40 is stored in the storage area a1, the first learning information 50 is stored in the storage area b1, and the second learning information 50 is stored in the storage area b2. For example, a comparative example is assumed in which a plurality of learning information 50 are stored in one storage area. In this comparative example, a process of retrieving the learning information 50 to be updated from among the plurality of learning information 50 is required. In contrast, according to the configuration of the present embodiment, since the plurality of pieces of learning information 50 are stored in the different storage areas, the processing of retrieving the learning information 50 to be updated from the plurality of pieces of learning information 50 is not required, and only the storage area in which the learning information 50 to be updated is stored may be accessed.

(correspondence relation)

The memory 34 is an example of "specific memory". The storage area a1 is an example of a "first area". The storage areas b1 and b2 are examples of "second areas".

(eighth embodiment)

(information reading System 2; FIG. 15)

The present embodiment is the same as the first embodiment except that the information reading system 2 includes the mediating apparatus 700 and the learning apparatus 200 provided on the internet 6.

The mediation device 700 is, for example, a server. The mediation device 700 is connected to the LAN 4. Thus, the mediation device 700 can communicate with the code reading device 10 via the LAN 4. The mediation device 700 is connected to the internet 6. Thus, the mediation device 700 can communicate with the learning device 200 via the internet 6.

The code reading device 10 can directly communicate with the mediation device 700, but cannot directly communicate with the learning device 200. For example, the code reading device 10 transmits training data 242 to the mediation device 700. Thereby, the mediation device 700 transmits the training data 242 to the learning device 200. In addition, for example, learning device 200 transmits adjusted model parameters 254 to mediation device 700. Thus, the mediation device 700 transmits the adjusted model parameters 254 to the code reading device 10. That is, the mediation device 700 mediates communication between the code reading device 10 and the learning device 200.

According to the configuration of the present embodiment, the code reading device 10 can be disconnected from the internet 6 by the mediation device 700. The code reading device 10 can be restrained from directly receiving an access from the internet 6. The mediation device 700 is an example of "mediation device".

(ninth embodiment)

(intrinsic factor classification process: FIG. 19)

In the present embodiment, the same as the second embodiment is performed except that, instead of the degradation classification processing of S100 of fig. 7, an intrinsic factor classification processing of classifying according to intrinsic factors as the intrinsic processing of the code reading apparatus 10, the setting of the code reading apparatus 10, and the like is performed.

The code reading apparatus 10 may perform image processing for a code image for the purpose of, for example, improving the reading accuracy. The image processing is a process of converting a code image using at least one filter. The filter is, for example, a smoothing filter, a black expansion filter, a black contraction filter, or the like. The smoothing filter smoothes the luminance value of the image. The black dilation filter is a filter that dilates black spots (blocks) in an image. The black shrink filter is a filter that shrinks black spots (blocks) within an image. These image processes are used, for example, to print a code image of an unclear two-dimensional code. The specific example of the filter described above is only an example, and the code reading device 10 may use a filter other than the specific example described above.

The intrinsic factor classification processing of the present embodiment is processing of classifying the two-dimensional codes to be read according to intrinsic factors as the use modes of the filters of the code reading apparatus 10. Here, the presence or absence of the use of the filter may be set by the user, or may be automatically selected by the code reading apparatus 10. In addition, the filter may be used a plurality of times, and the size of the filter is also various.

The process of fig. 19 is the same as the process of fig. 8 except that the processes of S510 and S512 are performed instead of S110 and S112 of fig. 8. In S510, the CPU32 determines whether at least one filter is utilized for the code image. If it is determined that one filter is not used for the code image (no in S510), the CPU32 skips the processing of S512 described later, and proceeds to S114.

In S512, the CPU32 classifies the code images according to the use of the filters. As shown in the training data table 60 of fig. 19, the items to be classified are, for example, the number of times of use of the filter, the type of the filter, the size of the filter, and the like. For example, as shown in fig. 19, in the case where only the smoothing filter of the one-time size "3×3" is used for the code image, the successful instance corresponding to the code image is stored in the training data table 60 as training data TD1 corresponding to the items "size" 3×3", the smoothing filter, and the one-time" as well (S116). If one filter is not used for the code image (no in S510), the successful instance corresponding to the code image is stored in the training data table 60 as training data in association with the item of the number of times of use of "0" (S116). The items shown in fig. 19 are only examples, and for example, when two types of filters are used, a combination of these items may be used as the items.

In the present embodiment, the same effects as those of the second embodiment can be obtained. That is, compared to a configuration in which a successful case is unconditionally used as the training data 242, it is possible to suppress a deviation of the successful case from being biased to a case in use of a specific filter (for example, use of a smoothing filter).

(tenth embodiment)

(intrinsic factor classification process: FIG. 20)

The intrinsic factor classification processing of the present embodiment is processing of classifying the two-dimensional code of the reading object according to intrinsic factors that are shooting conditions of the code reading apparatus 10. The shooting conditions are exposure time, distance from the shooting subject, on/off of illumination (e.g., flash), and the like. The shooting conditions may be set by the user or may be automatically selected by the code reading device 1 0. For example, the distance to the subject may be calculated from the focal length, or the distance to the subject may be calculated stereoscopically with reference to the indication mark irradiated to the reading subject.

The process of fig. 20 is the same as the process of fig. 8 except that the process of S612 is performed instead of S110, S112 of fig. 8. In S612, the CPU32 classifies the code images according to shooting conditions currently used for shooting of the code images. As shown in the training data table 60 of fig. 20, items to be classified are, for example, exposure time, distance, illumination, and the like. For example, as shown in fig. 20, when the exposure time is equal to or less than 0.5ms, the distance from the subject is equal to or less than 100mm, and the illumination is on, the successful instance corresponding to the code image is stored in the training data table 60 as training data TD1 corresponding to the items "0.5 mm 4 or less, 100mm or less, and on" (S116). The items in fig. 20 are only examples, and other imaging conditions may be adopted.

In the present embodiment, the same effects as those of the second embodiment can be obtained. That is, compared with a configuration in which a successful case is unconditionally used as the training data 242, it is possible to suppress deviation of the successful case from being biased to a case in a specific shooting condition.

(eleventh embodiment)

(intrinsic factor classification process: FIG. 20)

The intrinsic factor classification processing of the present embodiment is processing of classifying the two-dimensional code to be read according to intrinsic factors that are processing time of the reading processing of the code reading apparatus 10. The processing time is, for example, a time from the start to the end of the normal reading processing (see fig. 4) or a time from the start to the end of the learning reading processing (see fig. 6). In another modification, the processing time may be a part of the normal reading processing, for example, a time from the start to the end of the decoding processing.

In S612 of the present embodiment, the CPU32 classifies the code images according to the processing time. As shown in the training data table 60 of fig. 20, the items to be classified are, for example, a processing time of "100ms or less", a processing time of "150ms or less", and the like. For example, when the processing time is greater than 100ms and equal to or less than 150ms, the successful instance corresponding to the code image is stored in the training data table 60 as training data TD3 corresponding to the item "150ms or less" (S116). The items in fig. 20 are only examples.

The reason for the long processing time is, for example, degradation of the two-dimensional code, and the length of the processing time may have a correlation with the degree of degradation. By classifying the cases based on the processing time, it is possible to suppress cases in which the deviation of successful cases is biased to a certain degree of degradation. In the present embodiment, the same effects as those of the second embodiment can be obtained.

(twelfth embodiment)

(processing of the code reading device 10; FIG. 21)

The present embodiment is the same as the second embodiment except that the judgment of S700 is added. If the CPU32 determines that the number of training data 242 is smaller than the target number (no in S102), it proceeds to S700. In S700, the CPU32 determines whether or not the combined count of the data sets of one or more items having a high influence among the plurality of items in the training data table 60 is equal to or greater than a predetermined number. The predetermined number is a value smaller than the target number of S102. The influence level is set according to the possibility of the reading failure of the two-dimensional code. The item with high influence is, for example, a pattern number indicating degradation of the symbol mark in the pattern determination process (fig. 9). The area in which data is recorded in the two-dimensional code can be repaired by error correction even if it is deteriorated. In contrast, the symbol mark cannot be repaired by error correction, and if the symbol mark is degraded, the position of the two-dimensional code cannot be specified, and the possibility of failure in reading the two-dimensional code is high. In the present embodiment, an item indicating degradation of a symbol mark having a high possibility of failure in reading of a two-dimensional code is used as an item having a higher influence than other items.

When determining that the combined count of the data sets of one or more items having a high influence level is equal to or greater than the predetermined number (yes in S700), the CPU32 proceeds to S104. On the other hand, when determining that the combined count of the data sets of one or more items having a high influence level is smaller than the predetermined number (no in S700), the CPU32 skips S104 and proceeds to S44.

According to the configuration of the present embodiment, even if the number of training data 242 does not reach the target number (no in S102), when it is determined that the combined count of the data sets of one or more items having a high degree of influence is 5 or more (yes in S700), the code reading apparatus 10 transmits all the training data 242 to the learning apparatus 200 (S104). A successful case corresponding to a data set of one or more items having a high degree of influence is a case indicating that although the reading of the two-dimensional code is successful, the possibility of the two-dimensional code being failed due to degradation of the symbol mark or the like is high. When the probability of failure in reading the two-dimensional code is high, learning can be started promptly without waiting for the number of training data 242 to reach the target number.

In addition, the judgment of S700 of fig. 21 may be employed not only in the classification based on the deterioration of the information code, but also in any one of the classification based on the contrast of the information code (modification of the second embodiment), the classification based on the deformation of the information code (other modifications of the second embodiment (fig. 16 to 18)), and the classification based on the intrinsic factor of the code reading apparatus (ninth to eleventh embodiments (fig. 19, 20)). The determination of S700 of fig. 21 may be employed in classification based on a combination of at least two of the above-described degradation, contrast, distortion, and intrinsic factors.

(specific example 1 of shearing treatment; FIG. 22)

Specific example 1 of the shearing process performed in S50 of fig. 4 and S80 of fig. 6 and 13 will be described. The code reading device 10 can irradiate an indication mark indicating the two-dimensional code to be read. The indication mark has a predetermined shape such as a cross, a circle, or a quadrangle. In specific example 1, the CPU32 specifies the position of the indication mark from the captured image of the camera 20. For example, the CPU32 extracts a prescribed shape of the instruction mark from the captured image, and specifies the position of the prescribed shape as the position of the instruction mark. In the modification, the CPU32 may specify the center of the captured image as the position of the instruction mark in the case where the light source of the instruction mark is located on the center line of the angle of view of the camera 20.

Next, the CPU32 sets a clipping range with a predetermined number of pixels (horizontal and vertical) = (N1, N2) centering on the position of the instruction mark, and clips the captured image with the clipping range. The predetermined number of pixels (N1, N2) is set in advance, for example, in accordance with the size of the two-dimensional code to be read. N1 and N2 are positive integers, and N2 may be the same as or different from N1.

According to this configuration, a code image including a two-dimensional code can be simply cut out from the captured image based on the instruction mark.

(specific example 2 of shearing treatment; FIG. 23)

Other specific examples 2 of the shearing treatment will be described. The two-dimensional code is composed of a black-and-white pattern. Therefore, as shown in fig. 23, the luminance of the pixels indicating the two-dimensional code varies greatly, and the luminance of the pixels indicating the background other than the two-dimensional code hardly varies. In this specific example, the CPU32 analyzes the brightness of the pixels along the lateral direction from the position of the instruction mark, and estimates pixels whose brightness variation is equal to or greater than a predetermined value as the boundary line of the two-dimensional code in the lateral direction. Similarly, the CPU32 performs the same analysis in the vertical direction from the position of the instruction mark, and estimates pixels whose brightness variation is equal to or greater than a predetermined value as the boundary line of the two-dimensional code in the vertical direction. The CPU32 determines a line of a predetermined number of pixels N3 from the boundary line in the lateral direction as a line in the longitudinal direction of the clipping range, and determines a line of a predetermined number of pixels N4 from the boundary line in the longitudinal direction as a line in the lateral direction of the clipping range. The predetermined number of pixels N3, N4 is set in advance, for example, in accordance with the size of the two-dimensional code to be read. N3 and N4 are positive integers, and N3 may be the same as or different from N4. By making the clipping range smaller than the boundary line width estimated by the above analysis by the predetermined number of pixels N3 and N4, even if the cells of the outline of the two-dimensional code become shallow due to aged deterioration of the two-dimensional code to be read, the two-dimensional code can be included in the clipping range.

(specific example 3 of shearing treatment; FIG. 24)

Other specific examples 3 of the shearing treatment will be described. In this specific example, the CPU32 calculates the position coordinates of the same four corners as in S52 of fig. 4 for at least a part of the code image data in the plurality of training data 242 stored in S60 of fig. 5. Further, the CPU32 sets a range including all four corners calculated for at least a part of the code image data in the plurality of training data 242 as a clipping range.

With this configuration, even the code reading device 10 that does not use the indication mark can set the shearing range. Such a code reading device 10 is a fixed code reading device fixed to a factory production line, a POS register, or the like, for example.

Specific examples of the technology disclosed in the present specification have been described above, but these are merely examples and do not limit the scope of the claims. The technology described in the claims includes various modifications and changes to the specific examples described above. For example, the following modifications may be employed.

The "information code" in modification 1 is not limited to the two-dimensional code, and may be, for example, a bar code, a multi-level bar code, or the like.

(modification 2-1) the learning object of the learning device 200 is not limited to the position coordinates of the four corners of the two-dimensional code (refer to the first embodiment), and the black-and-white pattern of each cell of the two-dimensional code (refer to the fifth embodiment). For example, the learning object may be code position information indicating the position of the code image within the captured image. For example, it is assumed that the black portion of the two-dimensional code becomes shallow, and it is difficult to find the two-dimensional code from the captured image. By learning the code position information, even in the above-described situation, the two-dimensional code can be found from the captured image, and the cutting of the code image can be performed in the cutting process of fig. 4. In this modification, the code position information is an example of "relationship information (first relationship information and second relationship information)".

Further, in modification 2-2, the learning object may be a range to be clipped in the clipping process. Further, the learning object may be a position coordinate of each of four corners of the three symbol marks. Further, the learning object may be a position of a time-series pattern of the two-dimensional code. In general, the learning object may be information (such as a position and a size) indicating a predetermined pattern (for example, a symbol mark and a time series pattern) in the two-dimensional code. In these modifications, information on the shearing range, the position coordinates of the four corners of the symbol mark, or the specific pattern in the two-dimensional code is an example of "relationship information (first relationship information and second relationship information)".

The "relationship information (first relationship information and second relationship information)" of (modification 2-3) is not limited to one type of information (for example, the position coordinates of the four corners of the two-dimensional code), and may be a plurality of types of information (for example, the position coordinates of the four corners of the symbol mark and the position coordinates of the four corners of the two-dimensional code).

In each of the above embodiments, the training data 242 may include not only a successful case of the normal reading process but also a successful case of the learning reading process. In a modification, the training data 242 may include a successful case of the normal reading process, and may not include a successful case of the learning reading process. In general, the "training data" may include at least a plurality of successful cases of the reading process using the first relationship information.

The "classification unit" and the "judgment unit" in (modification 4) are not limited to the control unit 30 of the code reading apparatus 10, and may be, for example, the control unit 230 of the learning apparatus 200. In this case, the processing of fig. 8 may also be executed by the control section 230 of the learning device 200.

In the second embodiment, the training data table 60 is not limited to being stored in the memory 34 of the code reading apparatus 10, but may be stored in the memory 234 of the learning apparatus 200, for example. In the present modification, the memory 234 is an example of "first memory".

The technical elements described in the present specification or the drawings are useful for the technology alone or in various combinations, and are not limited to the combinations described in the claims at the time of application. In addition, the technology illustrated in the present specification or the drawings achieves a plurality of objects at the same time, and achieving one of the objects itself has technical usefulness.

Description of the reference numerals

2: information reading system

4：LAN

6: internet network

10: code reading device

12: operation part

14: display unit

20: camera with camera body

22: communication I/F

30: control unit

32：CPU

34: memory device

40: program

50: learning information

52: learning model

54: model parameters

60: training data table

200: learning device

222: communication I/F

230: control unit

232：CPU

234: memory device

240: program

242: training data

250: learning information

252: learning model

254: model parameters

256: initial parameters

270: case list

500: printer with a printer body

700: intermediary device

L1 to L4: coordinates (position coordinates)

a1, b1, b2: storage area

Claims (20)

1. An information reading apparatus includes:

a camera which can shoot two-dimensional information codes;

a first code process execution unit that executes a first code process including: a detection process of detecting first relation information, which is information defining a two-dimensional code area of the first information code, from an image of the first information code, which is the information code, captured by the camera, a reading process of reading information recorded in the first information code based on the detected first relation information, and an output process of outputting a result of the reading process;

An adjustment unit that adjusts parameters of a learning model, which are estimation values of information defining a two-dimensional code region of the information code, from an image output of the information code captured by the camera, based on training data including at least a plurality of successful cases of the reading process read from the first relationship information; and

and a second code processing execution unit that performs a second code processing including: an acquisition process of inputting an image of a second information code, which is the information code captured by the camera, to the learning model and acquiring the second relationship information from the learning model, a reading process of reading information recorded in the second information code based on the acquired second relationship information, and an output process of outputting a result of the reading process.

2. The information reading apparatus according to claim 1, wherein,

the first relation information is information indicating position coordinates of points at four corners of the information code.

3. The information reading apparatus according to claim 1, wherein,

the first relationship information is information indicating a black-and-white pattern of a plurality of cells constituting the two-dimensional code region of the information code.

4. An information reading apparatus according to claim 3, wherein,

in the first relation information, information indicating a position of each of the plurality of cells is associated with a value indicating one of black and white.

5. The information reading apparatus according to any one of claims 1 to 4, comprising:

one or more code reading apparatuses including the camera, the first code processing execution unit, and the second code processing execution unit; and

a learning device including the adjustment unit independent of the one or more code reading devices,

the one or more code reading devices acquire the adjusted parameters from the learning device,

the second code processing execution unit executes the second code processing based on the acquired adjusted parameter.

6. The information reading apparatus according to claim 5, wherein,

the learning device is built on the internet,

the device further comprises an intermediary device which intermediates communication between the one or more code reading devices and the learning device, and is connected to the internet.

7. The information reading apparatus according to claim 5 or 6, wherein,

further comprising an output device capable of outputting a specific information code recording the adjusted parameter,

the one or more code reading devices each acquire all or a part of the adjusted parameters from the learning device by reading the adjusted parameters from the specific information code output from the output device.

8. The information reading apparatus according to any one of claims 5 to 7, wherein,

the one or more code reading devices include a first code reading device, and a second code reading device different from the first code reading device,

the learning device obtains the training data from the first code reading device and adjusts the parameters according to the obtained training data,

the second code reading device acquires the parameter adjusted according to the training data of the first code reading device from the learning device, and executes the second code processing according to the acquired parameter.

9. The information reading apparatus according to any one of claims 1 to 8, wherein,

further provided with a specific memory device, and the memory device,

The specific memory includes a first area storing a program for executing the first code process and the second code process, and a second area different from the first area,

the second area is an area for storing a plurality of learning information,

the plurality of learning information each includes the learning model and the adjusted parameter.

10. The information reading apparatus according to any one of claims 1 to 9, wherein,

the adjustment unit is configured to start adjustment of the parameter of the learning model after the number of the plurality of successful cases included in the training data exceeds a predetermined number.

11. The information reading apparatus according to claim 10, wherein,

the information reading device further includes:

a classification unit configured to classify an information code of an object for which the reading process is successful in the first code process into a specific pattern among a plurality of patterns related to a type of degradation of the information code; and

and a judging unit configured to judge whether or not a successful case of the target information code is adopted as the training data, based on the classified specific pattern.

12. The information reading apparatus according to claim 11, wherein,

The classifying unit compares an image of a repair code repaired in error correction performed on the information code of the object with an image obtained by binarizing an actual image of the information code of the object, specifies a deteriorated portion of the information code of the object,

the information code of the object is classified into the specific pattern according to the deteriorated portion specified by the comparison of the repair code and the actual image.

13. The information reading apparatus according to claim 11 or 12, wherein,

the classification unit classifies the information code of the object into the specific pattern based on at least one of a contrast of an image of the information code of the object and a distortion of the information code of the object.

14. The information reading apparatus according to claim 11, wherein,

the information reading device further includes:

a classification unit configured to classify an information code of an object for which the reading process is successful in the first code process as a specific item among a plurality of items indicating an intrinsic factor of the information reading apparatus; and

a judging unit configured to judge whether or not a successful instance of the information code of the object is adopted as the training data based on the classified specific item,

The plurality of items includes at least one of:

more than two items related to image processing performed by the information reading device on the image of the information code;

more than two items related to shooting conditions under which the information code is shot by the information reading device; and

more than two items related to processing time of the information reading device for reading the information recorded in the information code.

15. The information reading apparatus according to any one of claims 10 to 14, wherein,

the adjustment unit is configured to start adjustment of the parameter of the learning model even when the number of two or more successful cases out of the number of successful cases exceeds a predetermined number smaller than the predetermined number before the number of the plurality of successful cases included in the training data exceeds the predetermined number,

the two or more successful cases are cases each showing an increased possibility that the reading of the information code is failed although the reading of the information code is successful.

16. The information reading apparatus according to any one of claims 9 to 15, further comprising:

A first memory; and

a first storage control unit that stores the successful instance of the information code of the object as the training data in the first memory when the successful instance of the information code of the object is determined to be employed as the training data based on the classified specific pattern, and does not store the successful instance of the information code of the object in the specific memory when the successful instance of the information code of the object is determined not to be employed as the training data based on the classified specific pattern,

the adjustment unit adjusts the parameter according to the training data in the first memory.

17. The information reading apparatus according to any one of claims 1 to 16, wherein,

the image processing device further includes an addition control unit that performs image processing on a code image that is an image of the information code captured by the camera, generates virtual cases of the reading processing, and increases the number of successful cases in the training data.

18. The information reading apparatus according to claim 17, wherein,

the image processing includes at least one of:

A process of adjusting the contrast of the code image;

a process of adding a predetermined image to the code image;

a process of rotating the code image; and

and processing each cell of the information code represented by the code image.

19. The information reading apparatus according to claim 18, wherein,

further comprising a second memory for storing, for each of the plurality of information codes, case information indicating an instance of the reading process performed on the information code,

the addition control unit selects one or more image processes from among the plurality of types of image processes based on the tendency of the case of the reading process indicated by the plurality of pieces of case information stored in the second memory, and executes the selected one or more image processes.

20. The information reading apparatus according to claim 19, wherein,

further, the information processing apparatus includes a second storage control unit that, when the reading process using the first relation information is performed, stores the case information indicating an instance of the reading process performed on the first information code in the second memory.