JP7459730B2 - Reading device, program, and reading method - Google Patents

Description

The present invention relates to a reading device , a program , and a reading method .

Handheld terminals (reading devices) have been used by delivery companies as terminal devices for reading code information (e.g., bar codes, two-dimensional codes, etc.) attached to slips. Such handheld terminals are equipped with imaging sensors, and can obtain identification information for the slip by capturing an image of the code information attached to the slip using the imaging sensor and decoding the code information based on the image.

A method for obtaining identification information from code information attached to a slip such as the one described above is known in which the surface image of the slip is obtained and analyzed to detect the position coordinates of the barcode, and the pixel data of the slip image is extracted based on the detected barcode position coordinates to obtain the identification information (Patent Document 1).

Japanese Patent Application Publication No. 2000-251010

This method of acquiring identification information from code information can be implemented by a simple operation of capturing an image of a slip. However, for example, when a user stands up and reads a slip attached to a baggage placed on the floor, the distance from the handheld terminal to the slip becomes long, and the number of pixels in the code information part of the slip in the captured image increases. There is a problem in that the acquisition of identification information fails because the number of identification information decreases.

An object of the present invention is to provide a reading device , a program, and a reading method that can reduce the burden of distance adjustment during imaging.

In order to solve the above problems, the reading device of the present invention is characterized in that it comprises an acquisition unit that acquires identification information related to the slip from code information based on an image including the slip to which the code information is attached; a first judgment unit that, in a first mode, judges whether the acquisition of the identification information by the acquisition unit is successful or not; and a search unit that, when it is determined by the first judgment unit that the acquisition of the identification information has failed, searches for search information that corresponds a comparison image stored in a memory unit with the identification information, and acquires the identification information associated with the comparison image that matches the image including the slip .
The program according to the present invention includes a computer including: an acquisition unit that acquires identification information related to a slip from code information based on an image including the slip to which code information is attached;
In this mode, the code is characterized in that it functions as a first judgment unit that judges whether the acquisition unit has been successful in acquiring the identification information, and a search unit that, when the first judgment unit determines that the acquisition of the identification information has failed, searches for search information that associates a comparison image stored in a memory unit with the identification information, and acquires the identification information associated with the comparison image that matches an image including the slip.
Furthermore, the reading method of the present invention is a reading method executed by a reading device, and is characterized in that it includes an acquisition process for acquiring identification information related to the slip from code information based on an image including the slip to which code information is attached, a first judgment process for judging whether acquisition of the identification information by the acquisition process has been successful, and a search process for searching for search information in which a comparison image stored in a memory unit is associated with the identification information when it is judged by the first judgment process that acquisition of the identification information has failed, and acquiring the identification information associated with the comparison image that matches the image including the slip.

According to the present invention, the burden of distance adjustment during imaging can be reduced.

FIG. 2 is an external view showing the front side of the handheld terminal. FIG. 3 is an external view showing the back of the handy terminal. FIG. 1 is a block diagram showing the overall configuration of a handy terminal. 10 is a flowchart showing the flow of a reading process executed by the handheld terminal. It is a figure which shows the example of the image which photographed a slip from a short distance. 10 is a flowchart showing the flow of an image registration process executed by the handheld terminal. FIG. 4 is a diagram illustrating an example of an image taken with the user standing on the slip while placing the package on the floor. 3 is a flowchart showing the flow of image search processing executed by the handy terminal. 10 is a flowchart showing the flow of a check decode process executed by the handheld terminal. FIG. 7 is a diagram illustrating an example of an image including a slip representing the number of pixels in the width of code information and the number of pixels in the width of the image including the slip. FIG. 7 is a diagram illustrating an example of an image including a slip representing the number of pixels in the width of code information and the number of pixels in the width of the image including the slip.

The following describes an embodiment of the present invention with reference to the drawings. However, the scope of the present invention is not limited to the following embodiment or those described in the drawings.

First, an overview of the handheld terminal 10 as a reading device according to this embodiment will be described. The handheld terminal 10 is a terminal device used by delivery companies to read code information (e.g., barcodes, two-dimensional codes, etc.) attached to slips. Figure 1 is an external view showing the front of the handheld terminal 10. Figure 2 is an external view showing the rear of the handheld terminal 10.

An imaging unit 18 is provided on the back of the handheld terminal 10. The handheld terminal 10 also includes a trigger key 151 that accepts an input operation to cause the imaging unit 18 to start an imaging operation, a general input key 152 that accepts an input operation for inputting various information by the user, and a display unit 16 that displays various information.

One trigger key 151 is provided on the side of the handy terminal 10. Note that a plurality of trigger keys 151 may be provided.
The general input keys 152 are character input keys for inputting numbers, letters, etc., function keys for calling and executing a predetermined function, and objects for moving a cursor displayed in the display unit 16 and inputting information. Contains cursor keys for transitioning between areas. These general input keys 152 and trigger keys 151 constitute the operation section 15 (see FIG. 3).

The display unit 16 is located on the front of the handheld terminal 10 and is composed of an LCD (Liquid Crystal Display) or an organic EL (Electro Luminescence) display.

The imaging unit 18 includes a small camera using an imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor). A camera equipped with a focusing (focal length adjustment) mechanism that can read from short distances to long distances is expensive, takes time to focus, and cannot perform speedy scanning. Therefore, a single-focus type camera is used as a small camera provided in the imaging section 18.
Further, when an input operation to the trigger key 151 (operation of pressing the trigger key 151) is performed, the imaging unit 18 performs photography with the camera in accordance with the input operation, and transfers image data of the photographed image to the CPU 11 (Central Processing Unit) (see Figure 3).

FIG. 3 is a block diagram showing the functional configuration of the handheld terminal 10. As shown in FIG.
The handheld terminal 10 includes a CPU 11 (computer), a RAM 12 (Random Access Memory), a ROM 13 (Read Only Memory), a storage unit 14, an operation unit 15, a display unit 16, a communication unit 17, an imaging unit 18, and a bus 19. Each unit of the handheld terminal 10 is connected to the bus 19. In the following, a description of the configuration that has already been described will be omitted.

The CPU 11 is a processor that controls each part of the handheld terminal 10 by performing various processes according to a program 141 and a business application 142 stored in the storage unit 14 .
The CPU 11 executes the program 141 to perform a reading process, which will be described later, to obtain identification information relating to the slip from the code information based on an image including the slip to which the code information is attached. Here, the CPU 11 functions as an obtaining unit.
Moreover, the CPU 11 executes the program 141 to perform a reading process described later, thereby determining whether or not the acquisition unit has successfully acquired the identification information in the delivery mode (first mode). Here, the CPU 11 functions as a first determination unit.
Furthermore, the CPU 11 executes the program 141 to perform an image search process described later, and when it is determined that acquisition of the identification information has failed, it searches for search information 143 in which the matching image and the identification information stored in the storage unit 14 are associated with each other, and acquires the identification information associated with the matching image that matches the image including the slip. Here, the CPU 11 functions as a search unit.
Moreover, the CPU 11 executes the program 141 to perform a reading process described later, thereby determining whether or not the acquisition unit has successfully acquired the identification information in the registration mode (second mode). Here, the CPU 11 functions as a second determination unit.
Furthermore, the CPU 11 executes the program 141 to perform an image registration process described later, and when the second determination unit has succeeded in acquiring identification information, the CPU 11 associates the identification information with an image including the slip from which the identification information has been successfully acquired, and stores the associated information in the storage unit 14 as search information 143. Here, the CPU 11 functions as a storage control unit.

RAM 12 provides working memory space for CPU 11 and stores temporary data. RAM 12 may include non-volatile memory.

ROM 13 is a non-volatile memory unit from which information can be read, and stores various setting data, etc.

The storage unit 14 includes a storage device capable of writing and reading data, such as a hard disk drive (HDD), a solid state drive (SSD), or a flash memory, and stores a program 141, a business application 142, and various data.
The storage unit 14 also stores search information 143 that associates the identification information of a slip with image (image for comparison) data that includes code information written on the slip. The search information 143 includes the number of registrations of each combination of identification information and image data.

The communication unit 17 is a communication module having an antenna, a modulation/demodulation circuit, a signal processing circuit, etc., and performs communication control in accordance with a communication standard related to wireless communication using a wireless LAN.

Next, a description will be given of the operation of the handy terminal 10 according to this embodiment. The operation of the handy terminal 10 has two modes: a registration mode (second mode) and a delivery mode (first mode).
In the registration mode, the CPU 11 performs an image registration process, which will be described later, to associate the identification information with an image including the slip and store the associated information in the memory unit 14 as search information 143 .
In addition, when the CPU 11 fails to obtain identification information in the delivery mode, it performs an image search process described below to search the search information 143 stored in the memory unit 14 and obtains the identification information associated with the image including the slip.
4 is a flowchart showing the flow of the reading process executed by the handheld terminal 10. Here, the operation mode of the handheld terminal 10 has been selected in advance by the user's operation of the business application 142 or the like.
When the user performs an input operation on the trigger key 151 (depressing the trigger key 151), the CPU 11 starts the reading process.

First, the CPU 11 controls the imaging unit 18 to capture an image including the code information of the slip (step S1). FIG. 5 shows an example of an image of a slip taken from a close distance.
Next, the CPU 11 determines whether the current operation mode of the handy terminal 10 is the registration mode (step S2). If the operation mode is the registration mode (step S2; YES), the CPU 11 detects code information from the image data of the photograph taken by the imaging unit 18 in step S1 (step S3).
Next, the CPU 11 determines whether code information has been detected from the image data of the photograph taken by the imaging unit 18 (step S4). If the code information is detected (step S4; YES), the CPU 11 decodes the detected code information to obtain identification information (step S5). When registering the identification information of a slip before delivery, the image containing the code information is photographed from a close distance, so the code information part of the photographed image has a sufficient number of pixels, so that the code information can be decoded and the identification information can be stored. can be obtained.
Next, the CPU 11 determines whether the decoding in step S5 was successful (step S6). If the decoding is successful (step S6; YES), the CPU 11 executes image registration processing (step S7).

FIG. 6 is a flowchart showing the flow of the image registration process executed by the handy terminal 10. As shown in FIG.
In the image registration process, the CPU 11 associates the image data of the image (matching image) captured in step S1 of the reading process with the identification information acquired in step S5, and stores them in the memory unit 14 as search information 143 (step S71).
Next, the CPU 11 increments the number of registrations, which is the number of combinations of image data and identification information stored in the storage unit 14, by +1 (step S72), ends the image registration process, and returns to the flowchart of the reading process shown in FIG.

Next, the CPU 11 transmits the identification information acquired in step S5 to the outside via the communication unit 17 (step S8), and ends the process.
Furthermore, if the code information cannot be detected from the captured image data (step S4; NO), or if decoding of the code information fails (step S6; NO), the CPU 11 displays an error message on the display unit 16 (step S9) and terminates the processing.

Also, in step S2, if the operating mode is the delivery mode (step S2; NO), the CPU 11 performs steps S10 to S13 similar to steps S3 to S6 above, but may place the package on the floor and photograph the code information written on the slip while the user is standing. Figure 7 shows an example of an image of a slip photographed while the user is standing. In this case, the distance from the handheld terminal 10 to the slip becomes greater, and the number of pixels in the code information portion of the slip in the photographed image becomes smaller, so that the code information cannot be detected (step S11; NO) or the code information may fail to be decoded (step S13; NO). In this case, the CPU 11 executes an image search process (step S14).

FIG. 8 is a flowchart showing the flow of the image search process executed by the handheld terminal 10. As shown in FIG.
In the image search process, the CPU 11 determines whether at least one set of search information 143, which is a combination of image (matching image) data including code information written on the invoice and the corresponding identification information of the invoice, is registered in the memory unit 14, i.e., whether there is any registered matching image data (step S141).
If there is comparison image data (step S141; YES), the CPU 11 extracts one comparison image data and compares it with the image data (captured image data) of the image captured in step S1 of the reading process shown in Figure 4 by image recognition (step S142).
Next, the CPU 11 judges whether or not the comparison result of step S142 indicates that the comparison image data has been detected in the photographic image data (step S143). If the example of the image shown in Fig. 5 is the comparison image data D1 and the example of the image shown in Fig. 7 is the photographic image data D2, the comparison image data D1 can be detected from the photographic image data D2.
If the comparison image data is detected in the captured image data (step S143; YES), the CPU 11 acquires the identification information stored in association with the comparison image data (step S144), and ends the image search process.

Further, if the verification image data cannot be detected in the photographed image data (step S143; NO), the CPU 11 determines whether there is a combination of verification image data and identification information that is registered next. (Step S145).
Next, if there is a registered combination of verification image data and identification information (step S145; YES), the CPU 11 moves to step S142. Then, the CPU 11 continues from step S142 until the comparison image data is detected in the photographed image data (step S143; YES) or until all the comparison image data and photographed image data are compared (step S145; NO). Step S145 is repeated.
If the combination of verification image data and identification information is not registered (step S141; NO), or if all verification image data is not detected in the captured image data (step S145; NO), the CPU 11 It is assumed that the identification information corresponding to the photographed image data is not stored (step S146), the image search process is ended, and the process returns to the flowchart of the reading process shown in FIG.

Next, the CPU 11 determines whether or not identification information corresponding to the photographed image data has been acquired in the image search process of step S14 (step S15).
If the identification information has been acquired (step S15; YES), the CPU 11 proceeds to step S8. If the identification information has not been acquired (step S15; NO), the CPU 11 ends the reading process.

As described above, the reading device (handy terminal 10) according to the above embodiment includes an acquisition unit (CPU 11) that acquires identification information related to a slip from code information based on an image including a slip to which code information is attached; mode (delivery mode), a first determination unit (CPU 11) that determines whether or not the acquisition of identification information by the acquisition unit was successful; , searches the search information 143 in which the matching image and identification information stored in the storage unit (storage unit 14) are associated with each other, and searches the identification information associated with the matching image that matches the image containing the slip. It includes a search unit (CPU 11) for obtaining information.
Therefore, the burden of distance adjustment during imaging can be reduced.
The reading device (handy terminal 10) according to the embodiment described above also includes a second determination unit (which determines whether or not the acquisition unit (CPU 11) has successfully acquired identification information in the second mode (registration mode)). When the acquisition of the identification information is successful by the second determination unit, the CPU 11) associates the identification information with the image including the slip from which the identification information has been successfully acquired, and stores it as search information 143 in the storage unit (storage unit 14). The identification information can be acquired based on the image including the slip.

(Modification)
In this modified example, a check decode process is performed instead of step S5 in the flowchart of the reading process shown in Fig. 4 of the above embodiment. In the reading process including the check decode process, if the ratio of the number of pixels in the width of the image including the slip to the width of the code information in the image including the slip is less than a predetermined value, the CPU 11 causes the display unit 16 to display an error message to notify the user. Here, the CPU 11 functions as a notification control unit, and the display unit 16 functions as a notification unit.
Therefore, in an image including a slip, if the ratio of the number of pixels in the width of the image including the slip to the width of the code information is less than a specified value, an error message can be displayed to the user, and the user can be prompted to re-take the image including the slip so that the ratio is equal to or greater than the specified value.
If this ratio is equal to or greater than a specified value in an image captured in registration mode, information about the area surrounding the slip, such as the exterior design of the luggage, will also be included in the captured image, increasing the amount of information available for image recognition in the image search process, thereby improving the detection rate of images containing slips.
FIG. 9 is a flowchart showing the flow of the check decode process executed by the handy terminal 10.

First, the CPU 11 calculates the ratio of the number of pixels in the width of the image including the slip captured in step S1 to the width of the code information (step S51).
10 and 11 show examples of images including a slip. If the number of pixels in the width of the code information is represented as a and the number of pixels in the width of the image including the slip is represented as b, then the ratio of the number of pixels in the width of the image including the slip to the width of the code information is b/a.
Next, the CPU 11 determines whether the ratio (b/a) is equal to or greater than a predetermined value (for example, 2.3) (step S52).
If the ratio is equal to or greater than 2.3 (step S52; YES), the CPU 11 decodes the code information to obtain the identification information (step S53).
Next, the CPU 11 judges whether the decoding in step S53 is successful (step S54). If the decoding is successful (step S54; YES), the CPU 11 holds "decoding successful" (step S55) and ends the check decoding process.

Further, if the ratio is less than 2.3 (step S52; NO) or if decoding has failed (step S54; NO), the CPU 11 retains "decoding failure" (step S56) and performs check decoding processing. finish. In this case, the CPU 11 displays an error in step S9 of the reading process flowchart shown in FIG.

As described above, the reading device (handheld terminal 10) of the above embodiment is equipped with a notification control unit (CPU 11) that causes the notification unit (display unit 16) to issue a notification when, in the second mode (registration mode), the ratio of the width of the image including the slip to the width of the code information in the image including the slip is less than a predetermined value, thereby urging the user to re-take the image including the slip.
In addition, since the search unit (CPU 11) compares the image containing the slip with the matching image based on information about the surroundings of the slip contained in the image, the amount of information available for image recognition in the image search process increases, thereby improving the detection rate of images containing slips and making it possible to easily obtain identification information.

As described above, the present invention is not limited to the above-described embodiment, and it goes without saying that appropriate modifications can be made without departing from the spirit of the present invention.
For example, in the above embodiment and modified example, the memory unit 14 has the search information 143 which is a combination of image data including code information written on the slip and the corresponding identification information of the slip, but this is not limited to the above. The memory unit having the search information 143 may be provided in an external device. In this case, the CPU 11 transmits the combination of image data (image for comparison) including code information written on the slip and the corresponding identification information of the slip to a specified memory unit of the external device via the communication unit 17 and registers it.

Furthermore, in the above embodiments and modifications, when registering the identification information of a slip, the combination of image data including code information written on the slip and the corresponding identification information of the slip is registered in the storage unit, but this is not limited to this. do not have. Instead of the image data including the code information written on the slip, a path to the image data or a file name of the image data may be registered in association with the identification information of the slip.

In addition, in the modified example of the above embodiment, the CPU 11 displays an error message to notify the user when the ratio of the number of pixels in the width of the image including the slip to the width of the code information in the image including the slip is less than 2.3, but this is not limited to the above. The ratio is not limited to 2.3, and may be any number that improves the detection rate in the image search process.

Although the embodiments of the present invention have been described, the scope of the present invention is not limited to the above-described embodiments, but includes the scope of the invention described in the claims and equivalent ranges thereof.
Below, the invention described in the claims first attached to the application of this application will be added. The claim numbers listed in the supplementary notes are as in the claims originally attached to the request for this application.
[Additional note]
<Claim 1>
an acquisition unit that acquires identification information related to the slip from the code information based on an image including the slip to which code information is attached;
In a first mode, a first determination unit that determines whether the acquisition of the identification information by the acquisition unit is successful;
If the first determination unit determines that the acquisition of the identification information has failed, search the search information in which the verification image stored in the storage unit and the identification information are associated with each other, and retrieve the slip. a search unit that acquires the identification information associated with the matching image that matches the containing image;
A reading device comprising:
<Claim 2>
In a second mode, a second determination unit that determines whether the acquisition of the identification information by the acquisition unit is successful;
If the acquisition of the identification information is successful by the second determination unit, the storage control unit associates the identification information with an image including the slip from which the identification information has been successfully acquired and stores it in the storage unit as the search information. The reading device according to claim 1, comprising:
<Claim 3>
In the second mode, the invention further comprises a notification control unit that causes a notification unit to perform notification when a ratio of the width of the image including the slip to the width of the code information in the image including the slip is less than a predetermined value. The reading device according to item 2.
<Claim 4>
The reading device according to any one of claims 1 to 3, wherein the search unit compares the verification image with the verification image based on information surrounding the slip included in the image including the slip.
<Claim 5>
computer,
an acquisition unit that acquires identification information related to the slip from the code information based on an image including the slip to which code information is attached;
In a first mode, a first determination unit that determines whether the acquisition of the identification information by the acquisition unit is successful;
If the first determination unit determines that the acquisition of the identification information has failed, search the search information in which the verification image stored in the storage unit and the identification information are associated with each other, and retrieve the slip. a search unit that acquires the identification information associated with the matching image that matches the containing image;
A program that functions as

10 Handy terminal (reading device)
11 CPU (acquisition unit, first judgment unit, second judgment unit, search unit, storage control unit, notification control unit, computer)
12 RAM
13 ROM
14 Storage section 141 Program 142 Business application 15 Operation section 151 Trigger key 152 General input key 16 Display section (notification section)
17 Communication section 18 Imaging section

Claims (6)

an acquisition unit that acquires identification information related to a slip from code information based on an image including the slip to which code information is attached;
a first determination unit that determines whether or not the acquisition unit has successfully acquired the identification information in a first mode;
a search unit that searches for search information in which a matching image and the identification information stored in a storage unit are associated with each other when the first judgment unit judges that acquisition of the identification information has failed, and acquires the identification information associated with the matching image that matches an image including the slip;
A reading device comprising: a second determination unit that determines whether or not the acquisition unit has successfully acquired the identification information in the second mode;
The reading device according to claim 1, further comprising a memory control unit that, when the second judgment unit is successful in obtaining the identification information, associates the identification information with an image including the slip for which the identification information has been successfully obtained and stores the associated information in the memory unit as the search information. The reading device according to claim 2, further comprising a notification control unit that causes a notification unit to issue a notification when, in the second mode, the ratio of the width of the image including the slip to the width of the code information in the image including the slip is less than a predetermined value. The reading device according to any one of claims 1 to 3, wherein the search unit compares the image for verification based on information about the surroundings of the slip contained in an image including the slip. computer,
an acquisition unit that acquires identification information related to the slip from the code information based on an image including the slip to which code information is attached;
In a first mode, a first determination unit that determines whether the acquisition of the identification information by the acquisition unit is successful;
If the first determination unit determines that the acquisition of the identification information has failed, search the search information in which the verification image stored in the storage unit and the identification information are associated with each other, and retrieve the slip. a search unit that acquires the identification information associated with the matching image that matches the containing image;
A program that functions as A reading method performed by a reading device, the reading method comprising:
an acquisition process for acquiring identification information relating to the slip from the code information based on an image including the slip to which the code information is attached;
In a first mode, a first determination process that determines whether or not the acquisition of the identification information by the acquisition process was successful;
If it is determined that the acquisition of the identification information has failed in the first determination process, search the search information in which the verification image stored in the storage unit and the identification information are associated with each other, and retrieve the slip. a search process for acquiring the identification information associated with the matching image that matches the containing image;
A reading method including:

Images