JP2020027317A - Portable terminal, system, character information acquisition method and program - Google Patents

Abstract

To provide a portable terminal capable of improving reliability of acquiring character information related to a code image.SOLUTION: A portable terminal includes a scanner unit that generates a scan image, a position detection unit that detects an own position, an acquisition unit that acquires a code image corresponding to the own position, an addition unit that adds the code image to the scan image, and a decode unit that decodes character image represented by the code image from the scan image with code image added.SELECTED DRAWING: Figure 5

Description

  The present invention relates to a mobile terminal, a system, a character information obtaining method, and a program.

  Barcodes are widely used in supermarkets, convenience stores, distribution warehouses, and the like. The barcode is printed on a label, affixed to a packaging material, a display shelf, a warehouse shelf, or the like, and read by a barcode scanner.

  By the way, in the related art, there is a problem that if the surface of the label on which the barcode is printed becomes dirty, erroneous reading occurs.

  The present invention has been made in view of the above problems, and a main object of the present invention is to provide a portable terminal, a system, a character information obtaining method, and a mobile terminal capable of improving the reliability of obtaining character information related to a code image. To provide a program.

  In order to solve the above problem, a portable terminal according to one embodiment of the present invention acquires a scanner unit that generates a scan image, a position detection unit that detects a position of the own device, and a code image corresponding to the position of the own device. An acquisition unit that adds the code image to the scan image, and a decoding unit that decodes character information represented by the code image from the scan image to which the code image is added.

  According to another aspect of the present invention, there is provided a mobile terminal including a scanner unit configured to generate a scan image, a position detection unit configured to detect a position of the mobile terminal, and a decode unit configured to decode character information represented by a code image included in the scan image. And an acquisition unit that acquires character information corresponding to the position of the own device when character information is not decoded from the scan image.

  A system according to another aspect of the present invention includes a portable terminal, and a server that can access a storage unit that stores a position and a code image in association with each other, wherein the portable terminal includes a scanner that generates a scan image. Unit, a position detecting unit that detects the position of the own device, an obtaining unit that obtains a code image corresponding to the position of the own device from the server, an adding unit that adds the code image to the scanned image, and the code A decoding unit that decodes character information represented by the code image from the scan image to which the image has been added.

  A system according to another aspect of the present invention includes a portable terminal, and a server that can access a storage unit that stores a position and character information in association with each other, wherein the portable terminal includes a scanner that generates a scanned image. Unit, a position detection unit that detects the position of the own device, a decoding unit that decodes character information represented by a code image included in the scan image, and a position of the own device when character information is not decoded from the scan image. And an acquisition unit that acquires character information corresponding to the above from the server.

  Further, according to another aspect of the present invention, there is provided a character information obtaining method, wherein a scan image is generated, a code image corresponding to a position of a mobile terminal is obtained, the code image is added to the scan image, and the code image is added. The character information represented by the code image is decoded from the scanned image.

  A character information obtaining method according to another aspect of the present invention generates a scan image, decodes character information represented by a code image included in the scan image, and automatically decodes the character information when the character information is not decoded from the scan image. Get character information corresponding to the machine position.

  According to another aspect of the present invention, there is provided a program that stores a computer of a portable terminal including a scanner unit that generates a scan image and a position detection unit that detects a position of the own device, into a code image corresponding to the position of the own device. , An adding unit that adds the code image to the scanned image, and a decoding unit that decodes character information represented by the code image from the scanned image to which the code image is added.

  A program according to another aspect of the present invention includes a computer of a portable terminal including a scanner unit that generates a scan image and a position detection unit that detects a position of the own device. The decoding unit functions as a decoding unit that decodes character information to be represented and an acquisition unit that acquires character information corresponding to the position of the own device when the character information is not decoded from the scanned image.

  ADVANTAGE OF THE INVENTION According to this invention, it becomes possible to aim at improvement of the acquisition reliability of the character information regarding a code image.

FIG. 1 is a diagram illustrating a configuration example of a system according to an embodiment. It is a front view showing the example of appearance of the personal digital assistant concerning an embodiment. It is a side view which shows the example of an external appearance of a portable terminal. FIG. 3 is a block diagram illustrating a configuration example of a mobile terminal. It is a figure showing the example of contents of a code database. FIG. 3 is a block diagram illustrating a functional configuration example of a mobile terminal. It is a flowchart which shows the example of a procedure of the character information acquisition method which concerns on embodiment. FIG. 4 is a diagram illustrating an example of a scan image to which a code image is added. FIG. 4 is a diagram illustrating an example of a scan image to which a code image is added. FIG. 4 is a diagram illustrating an example of a scanned image to which a code image is not added. FIG. 4 is a diagram illustrating an example of an image displayed on a display unit. FIG. 3 is a diagram illustrating an example of a virtual three-dimensional space. It is a figure showing an example of a priority list. It is a figure showing an example of arrangement of a virtual camera. It is a figure showing the example of addition of a code picture. It is a figure showing an example of arrangement of a virtual camera. It is a figure showing the example of addition of a code picture. It is a block diagram which shows the other example of a function structure of a portable terminal. It is a flowchart which shows the other example of a procedure of the character information acquisition method.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings. Each of the embodiments described below exemplifies a method and an apparatus for embodying the technical idea of the present invention, and the technical idea of the present invention is not limited to the following. Absent. Various changes can be made to the technical idea of the present invention within the technical scope described in the claims.

  FIG. 1 is a diagram illustrating a configuration example of a system 100. The system 100 includes a mobile terminal 1 and a server 2. The mobile terminal 1 and the server 2 can communicate with each other via a network NW. The server 2 can access the database 3. The database 3 may be provided inside the server 2 or may be provided outside the server 2.

  The mobile terminal 1 is a handy terminal that can read a code such as a barcode or a QR code (registered trademark). Not limited to this, the mobile terminal 1 may be, for example, a smartphone or a tablet computer.

  The mobile terminal 1 is used for managing an article B arranged on a shelf S such as a warehouse or a store. A label L on which a code corresponding to the article B is printed is attached to the shelf S, and when the user brings the mobile terminal 1 close to the label L, the code is read by the mobile terminal 1.

  The database 3 includes an inventory database (DB) 31 for managing inventory of the article B, and a code database (DB) 32 for managing code information. Details of the code DB 32 will be described later.

  By the way, when the label L on which the code is printed as described above is used, there is a possibility that the surface of the label L becomes dirty and erroneous reading of the code occurs. Therefore, in the present embodiment, code information can be obtained by an AR (augmented reality) method described below.

  FIG. 2A and FIG. 2B are a front view and a side view showing an example of the appearance of the portable terminal 1. FIG. 3 is a block diagram illustrating a configuration example of the mobile terminal 1. The mobile terminal 1 includes a control unit 10, a scanner unit 11, an input unit 12, a communication I / F 13, a position detection unit 14, a posture detection unit 15, and a display unit 16.

  The control unit 10 is a computer including a CPU, a RAM, a ROM, a nonvolatile memory, an input / output interface, and the like. The CPU executes information processing according to a program loaded from the ROM or the nonvolatile memory into the RAM. The program may be supplied via an information storage medium such as an optical disk or a memory card, or may be supplied via a communication network such as the Internet.

  The scanner unit 11 includes an image sensor such as a CMOS image sensor, for example, captures a subject, generates a scan image, and outputs the scan image to the control unit 10. The scanner unit 11 is provided at an upper portion on the back surface of the mobile terminal 1 (see FIG. 2B).

  The input unit 12 includes, for example, a plurality of key buttons. The input unit 12 is provided at a lower part on the front of the mobile terminal 1 (see FIG. 2A). Not limited to this, the input unit 12 may be, for example, a touch panel or the like.

  The communication I / F 13 is a communication interface that realizes wireless communication such as a wireless LAN. Not limited to this, the communication I / F 13 may be a communication interface that realizes wired communication.

  The position detection unit 14 detects the position of the own device and outputs the position to the control unit 10. It is preferable that the position detection unit 14 can detect the position of the mobile terminal 1 in the actual three-dimensional space (including not only the two-dimensional plane position but also the height). The position detection unit 14 includes, for example, a receiver that receives a beacon using BLE (Bluetooth Low Energy) transmitted from the transmitter A (see FIG. 1). However, the position detector 14 may be a GNSS receiver or the like.

  The posture detection unit 15 detects the posture of the own device and outputs the detected posture to the control unit 10. The posture detection unit 15 is composed of, for example, a gyro sensor.

  The display unit 16 is, for example, a liquid crystal display panel or an organic EL display panel. The display unit 16 is provided on the front upper part of the mobile terminal 1 and is located on the opposite side of the scanner unit 11 (see FIGS. 2A and 2B).

  FIG. 4 is a diagram illustrating an example of the contents of the code DB 32. The code DB 32 has fields of “ID”, “type”, “code image”, “character information”, “position”, “direction”, and “size”.

  “ID” represents the identification number of the code information. “Type” indicates the type of the code. The type of code is, for example, the type of a barcode (one-dimensional code) such as JAN, CODE39, CODE128, or NW-7, or the type of two-dimensional code such as QR or DataMatrix.

  “Code image” and “character information” represent code information. "Code image" represents an image indicating a code such as a barcode. In the “code image”, a storage location and a file name of the code image are described. “Character information” represents character information corresponding to the code image, that is, character information obtained by decoding the code image. “Character information” is, for example, ASCII data.

  “Position”, “direction”, and “size” represent the position, direction, and size of the code image in the virtual three-dimensional space. In the “position”, the position of the code image in the virtual three-dimensional space is described by three-dimensional coordinates. "Direction" indicates the direction of the surface on which the code of the code image is displayed.

[First Embodiment]
FIG. 5 is a block diagram illustrating a functional configuration example of the mobile terminal 1. The control unit 10 of the mobile terminal 1 includes an acquisition unit 101, an addition unit 102, and a decoding unit 103. These functional units are realized by the CPU of the control unit 10 executing information processing according to a program loaded from the ROM or the nonvolatile memory into the RAM.

  The acquiring unit 101 acquires the position of the own device from the position detecting unit 14, acquires the posture of the own device from the posture detecting unit 15, acquires the code image corresponding to the position and the posture of the own device from the server 2, The acquired code image is output to the adding unit 102. In addition, the attitude of the own device may be omitted.

  The adding unit 102 acquires a scanned image from the scanner unit 11, acquires a code image from the acquiring unit 101, adds a code image to the scanned image, and outputs the scanned image with the code image added to the decoding unit 103. .

  The decoding unit 103 acquires the scan image to which the code image is added from the addition unit 102, decodes the character information represented by the code image from the scan image to which the code image is added, and displays the decoded character information on the display unit 16 Output to

  FIG. 6 is a flowchart illustrating a procedure example of the character information acquisition method. The control unit 10 of the mobile terminal 1 functions as the acquisition unit 101, the addition unit 102, and the decoding unit 103 by executing the information processing illustrated in FIG.

  First, the control unit 10 acquires a scanned image from the scanner unit 11 (S11).

  Next, the control unit 10 acquires the position of the own device from the position detecting unit 14, acquires the posture of the own device from the posture detecting unit 15 (S12), and stores the code image corresponding to the position and the posture of the own device in the server. 2 (S13, processing as the acquisition unit 101). Details of acquiring the code image will be described later.

  Next, the control unit 10 adds a code image to the scanned image (S14, processing as the adding unit 102).

  As in the example shown in FIG. 7, the code image 5 is superimposed on the scan image 4 and covers a part of the scan image 4. Not limited to this, the code image 5 may cover the entirety of the scanned image 4. The code image 5 is arranged at a predetermined position such as the center of the scan image 4. Not limited to this, the code image 5 may be arranged at a position in the scan image 4 corresponding to a position in the virtual three-dimensional space.

  Returning to the description of FIG. 6, next, the control unit 10 decodes the character information represented by the code image from the scan image to which the code image has been added (S15, processing as the decoding unit 103). By decoding the scan image to which the code image is added, character information represented by the code image is output as, for example, ASCII data.

  As shown in the example of FIG. 8, even when the scanned image 4 includes an actual code image (hereinafter, referred to as an actual code image 41) by capturing the label L, the code acquired from the server 2 is used. By adding the image 5 to the scanned image 4, a part or the whole of the actual code image 41 is covered by the code image 5, and the code image 5 becomes easier to be decoded.

  On the other hand, as in the example shown in FIG. 9, the actual image code 41 is included in the scan image 4 by photographing the label L, and the code image 5 is not acquired from the server 2; If not added, the actual code image 41 included in the scanned image 4 is to be decoded. That is, it operates similarly to the conventional barcode scanner.

  Returning to the description of FIG. 6, next, the control unit 10 outputs an image including the decoded character information to the display unit 16 (S16). The output destination of the character information is not limited to the display unit 16, but may be an application execution unit such as a messaging application realized in the control unit 10 by an application program.

  As in the example shown in FIG. 10, the image 7 displayed on the display section 16 includes a character string 71 representing decoded character information. The character string 71 represents, for example, the name of the article. Further, the image 7 includes a stock confirmation button 72 for acquiring stock information of the article corresponding to the character string 71 and a quantity input button 73 for inputting the quantity of the article corresponding to the character string 71. ing.

  When the stock confirmation button 72 is selected, the control unit 10 accesses the stock DB 31 (see FIG. 1), and generates and outputs a stock confirmation screen (not shown) representing stock information of the article corresponding to the character string 71. . When the quantity input button 73 is selected, the control unit 10 generates and outputs a quantity input screen (not shown) for inputting the incoming quantity or outgoing quantity of the article corresponding to the character string 71.

  According to the first embodiment described above, by decoding the character information represented by the code image from the scan image to which the code image has been added, it is possible to eliminate the problem of misreading due to stain on the label. Become.

  Further, according to the first embodiment, when the code image acquired from the server 2 is not added to the scanned image, it can be used in the same manner as a conventional barcode scanner. In other words, the configuration of the conventional barcode scanner can be used other than the configuration for obtaining and adding the code image (the obtaining unit 101 and the adding unit 102).

  Further, according to the first embodiment, it is possible to omit the time for aiming by the scanner unit 11. Further, according to the first embodiment, by updating the contents of the code DB 32, the code information can be dynamically changed.

[Acquisition of code image]
Hereinafter, a specific method of acquiring a code image will be described. In the code DB 32, the code information (code image and character information) is associated with the position, direction, and size in the virtual three-dimensional space (see FIG. 4), that is, mapped in the virtual three-dimensional space.

  As in the example shown in FIG. 11, a plurality of code images 5 are arranged in the virtual three-dimensional space 61. Each code image 5 has a surface for displaying a code. The arrow in the figure indicates the direction of the surface of the code image 5. In the example shown in the figure, a plurality of code images 5 are horizontally arranged in a line, and the directions of the surfaces are alternately switched.

  A virtual camera 62 is arranged in the virtual three-dimensional space 61. A field of view 63 is formed in the shooting direction of the virtual camera 62. The position of the virtual camera 62 in the virtual three-dimensional space 61 corresponds to the position of the mobile terminal 1 in the real three-dimensional space. The shooting direction of the virtual camera 62 corresponds to the attitude of the mobile terminal 1, specifically, the shooting direction of the scanner unit 11.

  Of the plurality of code images 5 arranged in the virtual three-dimensional space 61, the code images 5 included in the visual field 63 of the virtual camera 62 are candidates for acquisition. At this time, the code image 5 whose surface direction faces the shooting direction of the virtual camera 62 has a higher priority, and the code image 5 whose surface direction does not face the shooting direction of the virtual camera 62 has a lower priority or has a lower priority. Come off. Further, among the code images 5 included in the visual field 63 of the virtual camera 62, the code image 5 closer to the center of the visual field 63 has higher priority, and the code image 5 farther from the center of the visual field 63 has lower priority. Good.

  In this way, the code images 5 to be acquired and their priorities are determined based on the visual field 63 of the virtual camera 62, and a priority list indicating the priorities of the code images 5 as shown in FIG. 12 is created. .

  The control unit 10 acquires the code image 5 included in the priority list from the server 2 and adds the code image 5 to the scan image. Here, only one code image 5 having the highest priority may be obtained, or a plurality of code images 5 having a predetermined or higher priority may be obtained.

  When a plurality of code images 5 are acquired, the position or size of the code image 5 to be added to the scan image 4 may be changed according to the priority. For example, a code image 5 with a higher priority may be arranged closer to the center of the scanned image 4, a code image 5 with a higher priority may be arranged so as to be larger, or a higher priority is used. The layer may be higher (to the viewer side) as the code image 5 is. As a result, the code image 5 having a higher priority is more easily decoded.

  13A to 14B, the position or the size of the code image 5 to be added to the scan image 4 may be changed according to the distance between the virtual camera 62 and the code image 5. 13A and 13B show a case where the distance between the virtual camera 62 and the code image 5 is relatively large, and FIGS. 14A and 14B show a case where the distance between the virtual camera 62 and the code image 5 is relatively small. I have.

  In the examples shown in these figures, the smaller the distance between the virtual camera 62 and the code image 5 is, the larger the code image 5 added to the scan image 4 is. Further, for example, as the distance between the virtual camera 62 and the code image 5 becomes smaller, the code image 5 may be arranged closer to the center of the scan image 4 or the layer of the code image 5 may be made higher. . As a result, the code image 5 closer to the virtual camera 62 is more easily decoded.

  Note that the above-described calculation performed between the detection of the position and the direction of the mobile terminal 1 and the acquisition of the code image 5, that is, the code based on the visual field 63 of the virtual camera 62 in the virtual three-dimensional space 61. The calculation for determining the priority of the image 5 may be performed by the mobile terminal 1, may be performed by the server 2, or may be performed by the mobile terminal 1 and the server 2 in a distributed manner. .

[Second embodiment]
Hereinafter, the second embodiment will be described. The same components as those in the first embodiment are denoted by the same reference numerals, and detailed description is omitted.

  FIG. 15 is a block diagram illustrating another example of a functional configuration of the mobile terminal 1. The control unit 10 of the mobile terminal 1 includes an acquisition unit 101 and a decoding unit 103. These functional units are realized by the CPU of the control unit 10 executing information processing according to a program loaded from the ROM or the nonvolatile memory into the RAM.

  The decoding unit 103 decodes character information from the scanned image acquired from the scanner unit 11. When the character information is decoded from the scanned image, the decoding unit 103 outputs the decoded character information to the display unit 16. On the other hand, when the character information is not decoded from the scanned image, the decoding unit 103 notifies the obtaining unit 101 of the fact.

  Upon receiving the notification that the character information is not decoded from the scanned image, the obtaining unit 101 obtains the position of the own device from the position detecting unit 14, obtains the posture of the own device from the position detecting unit 15, and Character information corresponding to the position and orientation is obtained from the server 2 and the obtained character information is output to the display unit 16.

  FIG. 16 is a flowchart illustrating another example of the procedure of the character information acquiring method according to the embodiment. The control unit 10 of the mobile terminal 1 functions as the acquisition unit 101 and the decoding unit 103 by executing the information processing illustrated in FIG.

  First, the control unit 10 acquires a scanned image from the scanner unit 11 (S21).

  Next, the control unit 10 decodes the character information from the scanned image (S22, processing as the decoding unit 103). Here, if the scanned image contains the actual code image and it is readable, the character information is decoded. If the scanned image contains the actual code image but it is not readable, If the actual code image is not included in the scanned image, the character information is not decoded.

  When the character information is decoded from the scanned image (S23: YES), the control unit 10 outputs an image including the decoded character information to the display unit 16 (S26).

  On the other hand, if the character information is not decoded from the scanned image (S23: NO), the control unit 10 acquires the position of the own device from the position detecting unit 14 and acquires the posture of the own device from the posture detecting unit 15 ( (S24) Character information corresponding to the position and orientation of the own device is acquired from the server 2 (S25, processing as the acquisition unit 101), and an image including the acquired character information is output to the display unit 16 (S26).

  The acquisition of the character information is performed by the same method as that of the above-described acquisition of the code image, and thus the description is omitted.

  According to the second embodiment described above, it is possible to acquire character information even when character information is not decoded from a scanned image.

  Further, according to the second embodiment, as long as character information is decoded from a scanned image, it can be used in the same manner as a conventional barcode scanner. In particular, the second embodiment can be easily applied to a barcode scanner in which a configuration for generating and decoding a scanned image (the scanner unit 11 and the decoding unit 103) is integrally provided.

Reference Signs List 1 mobile terminal, 2 server, 3 database (DB), 31 inventory DB, 32 code DB, 10 control unit, 11 scanner unit, 12 input unit, 13 communication I / F, 14 position detection unit, 15 posture detection unit, 16 Display part, 4 scan images, 41 real code images, 5 code images, 61 virtual three-dimensional space, 62 virtual cameras, 63 fields of view, 7 images, 71 character strings, 72 stock confirmation buttons, 73 quantity input buttons, 100 systems, 101 Acquisition unit, 102 addition unit, 103 decoding unit, NW network, A transmitter, B article, S shelf, L label

Claims (13)

A scanner unit for generating a scanned image;
A position detection unit for detecting the position of the own device,
An acquisition unit that acquires a code image corresponding to the position of the own device,
An adding unit that adds the code image to the scanned image,
From the scan image to which the code image is added, a decoding unit that decodes character information represented by the code image,
A mobile terminal comprising: It further includes an attitude detection unit that detects the attitude of the own machine,
The acquisition unit acquires a code image corresponding to the position and orientation of the own device,
The mobile terminal according to claim 1. The acquisition unit is configured to, based on a position in the virtual three-dimensional space corresponding to the position of the own device in the real three-dimensional space, obtain a partial code image from a plurality of code images associated in the virtual three-dimensional space Select the
The mobile terminal according to claim 1. It further includes an attitude detection unit that detects the attitude of the own machine,
The acquiring unit selects a part of the code images from a plurality of code images having directions in the virtual three-dimensional space based on a direction in the virtual three-dimensional space corresponding to a posture of the own device.
The mobile terminal according to claim 3. The adding unit sets a position or a size of the code image to be added to the scan image according to a distance between the position in the virtual three-dimensional space and the code image.
The mobile terminal according to claim 3. A scanner unit for generating a scanned image;
A position detection unit for detecting the position of the own device,
A decoding unit for decoding character information represented by a code image included in the scan image;
When character information is not decoded from the scan image, an acquisition unit that acquires character information corresponding to the position of the own device,
A mobile terminal comprising: It further includes an attitude detection unit that detects the attitude of the own machine,
The acquisition unit acquires character information corresponding to the position and orientation of the own device,
The mobile terminal according to claim 6. Mobile devices,
A server that can access a storage unit that stores the position and the code image in association with each other;
With
The mobile terminal,
A scanner unit for generating a scanned image;
A position detection unit for detecting the position of the own device,
An acquisition unit that acquires a code image corresponding to the position of the own device from the server,
An adding unit that adds the code image to the scanned image,
From the scan image to which the code image is added, a decoding unit that decodes character information represented by the code image,
A system comprising: Mobile devices,
A server that can access a storage unit that stores the position and the character information in association with each other;
With
The mobile terminal,
A scanner unit for generating a scanned image;
A position detection unit for detecting the position of the own device,
A decoding unit for decoding character information represented by a code image included in the scan image;
When character information is not decoded from the scan image, an acquisition unit that acquires character information corresponding to the position of the own device from the server,
A system comprising: Generate a scanned image,
Get the code image corresponding to the location of the mobile device,
Adding the code image to the scanned image,
Decoding the character information represented by the code image from the scan image to which the code image is added,
Character information acquisition method. Generate a scanned image,
Decoding the character information represented by the code image included in the scan image,
When character information is not decoded from the scanned image, character information corresponding to the position of the own device is obtained.
Character information acquisition method. A computer of a mobile terminal including a scanner unit that generates a scan image and a position detection unit that detects the position of the own device,
An acquisition unit that acquires a code image corresponding to the position of the own device,
An adding unit that adds the code image to the scanned image; and
From the scan image to which the code image is added, a decoding unit that decodes character information represented by the code image,
A program to function as A computer of a mobile terminal including a scanner unit that generates a scan image and a position detection unit that detects the position of the own device,
A decoding unit that decodes character information represented by a code image included in the scan image, and
When character information is not decoded from the scanned image, an acquiring unit that acquires character information corresponding to the position of the own device,
Program to function as

Images