JP6339542B2 - Information processing apparatus and program - Google Patents

Description

  Embodiments described herein relate generally to an information processing apparatus and a program.

  2. Description of the Related Art Conventionally, there are object recognition scanners that read a code symbol (barcode or the like) attached to an article from a captured image of the article or perform an object recognition process on the article. A POS (Point-of-Sale) system performs code symbol reading processing using this object recognition scanner, and performs object recognition processing when code symbol reading information is not obtained. The object recognition process is started after the code symbol reading process based on a switching signal generated by pressing the switching button by the cashier. However, this delay causes a loss in the time from when the cashier starts to pick up the article to the reading window until the result of the object recognition process is obtained. Because of this loss, more time is required until the result of the object recognition process is obtained compared to the time until the code symbol is read, and therefore improvement is desired.

  The problem to be solved by the present invention is to provide an information processing apparatus and program capable of reducing the acquisition time of the result of object recognition processing per article.

The information processing apparatus according to the embodiment includes an acquisition unit, a detection unit, a first extraction unit, a second extraction unit, an instruction unit, and a notification control unit, and the acquisition unit includes an imaging unit. but write-free take the image of the goods picked up. The detection means detects an article . The first extraction means, based on the image captured by the above capturing means, intends line extraction processing for extracting the code symbol is added to the article in which the detection unit detects. It said second extraction means, based on the image captured by the above capturing means, intends line extraction processing for extracting information indicating the article which the detection unit detects. Said instructing means, said the detecting means detects an article, the only extraction process by first extracting means indicates the start of the extraction process only on the first extraction means as carried out, the first extraction means The second extraction means so that the extraction process by the first extraction means and the extraction process by the second extraction means are performed in parallel at a first timing after a predetermined time has elapsed from the start of the extraction process by Is instructed to start the extraction process . The notification control means, the period from the start of the extraction process by first extracting means to the second timing after the first timing, the case where by Rico Doshinboru is extracted in the first extracting means the extracted code symbol notifies the commodity registration destination, to the time when the code symbol was not been extracted, in a period from the second extraction means according to the first timing until the second timing Information indicating the product extracted by the extraction process is notified to the product registration destination after the second timing .

FIG. 1 is an external perspective view of a checkout system according to an embodiment. FIG. 2 is a block diagram illustrating an example of a hardware configuration of the POS terminal and the commodity reading apparatus. FIG. 3 is a diagram illustrating an example of functional block configurations of the POS terminal and the commodity reading apparatus. FIG. 4 is a flowchart showing an example of product registration and checkout processing of the checkout system. FIG. 5 is a flowchart showing an example of the commodity reading process. FIG. 6 is a flowchart illustrating an example of a merchandise reading process according to a modification. FIG. 7 is an external perspective view of the self-POS. FIG. 8 is a block diagram illustrating an example of a hardware configuration of the self-POS.

  Hereinafter, embodiments of an information processing apparatus and a program will be described in detail with reference to the accompanying drawings. In the present embodiment, as an embodiment of the information processing apparatus and the program, an application example of a checkout system provided in a store such as a supermarket to a commodity reading apparatus is shown.

  FIG. 1 is an external perspective view of an example of a checkout system 1. As shown in FIG. 1, the checkout system 1 registers a commodity reading device 101 that reads commodity identification information of an article, and the commodity from which the commodity identification information is read as a customer's purchased commodity, and settles it in one transaction break. And a POS terminal 11 for processing. In the following, the product identification information is assumed to be code reading information or information indicating candidate products (candidate product information). The code reading information is identification information based on a product code or a product code obtained by reading a product code of an article. The candidate product information is identification information such as an identification ID of a candidate product obtained by an object recognition process described later.

  In the example of FIG. 1, the POS terminal 11 is provided in a state where it is placed on the drawer 21 on the cash register 41. The POS terminal 11 has a keyboard 22 on the upper surface of which operation pressing keys are arranged. Further, the POS terminal 11 includes a first display device 23 used for input and the like by a user (as an example, a cashier in the following) and a second display device 24 used by a customer for confirmation above the keyboard 22. Have. The first display device 23 has a display panel 23a such as a liquid crystal, and is a touch input type in which a touch panel 26 is laminated on the display panel 23a. The second display device 24 includes a display panel 24a such as a liquid crystal. The second display device 24 is erected on the upper portion of the POS terminal 11 so as to be rotatable. The drawer 21 stores banknotes, coins, and the like inside and opens the inside by control from the POS terminal 11.

  Next to the cash register table 41, a soccer table 151 that forms an L shape together with the cash register table 41 is provided. On the upper surface of the soccer table 151, a cargo receiving surface 152 for placing a shopping basket 153 or the like is formed. In this example, as the shopping basket 153, a first shopping basket 153a and a second shopping basket 153b are shown. The first shopping basket 153a is a shopping basket brought in by the customer. That is, the first shopping basket 153a is a shopping basket containing an article (for example, an article G) before the merchandise identification information is read. The second shopping basket 153b is a shopping basket for storing an article (for example, the article G of the first shopping basket 153a) whose commodity identification information has been read by the commodity reading apparatus 101. The cashier picks up, for example, the item G before reading from the first shopping basket 153a, causes the item reading apparatus 101 to read the item identification information of the item G, and then transfers the item G to the second shopping basket 153b. .

  The commodity reading apparatus 101 is an information processing apparatus that is erected at the center of the cargo receiving surface 152 of the soccer stand 151 and connected to the POS terminal 11 so that data can be transmitted and received by wire or wirelessly. The commodity reading apparatus 101 has a reading unit 110 (see FIG. 2) arranged so that the reading window 103 is exposed on the front surface of the housing 102. The reading window 103 is made of a transparent glass plate or the like. A display / operation unit 104 is provided above the housing 102. The display / operation unit 104 has a third display device 106 such as a liquid crystal mainly used by the cashier for input. A touch panel 105 is stacked on the screen of the third display device 106 to enable touch input on the screen. In addition, a keyboard 107 is provided on the right side of the third display device 106 to enable keyboard input by a cashier. In addition, a fourth display device 109 that is mainly used by a customer for confirmation is provided facing the back side of the display / operation unit 104.

  The reading unit 110 includes an imaging unit 164 (see FIG. 2) for imaging an object such as a hand, an article, or a fixed structure in the back of the reading window 103. The cashier picks up, for example, an article G before reading from the first shopping basket 153a and moves the article G to the second shopping basket 153b, and then causes the reading window 103 to pick up the article G and read the article. The imaging unit 164 captures the article G trapped in the reading window 103 and generates a frame image. The reading unit 110 detects the article G based on the frame image. Further, the reading unit 110 reads a code symbol such as a barcode or QR code (registered trademark) attached to the article G for the frame image, and an object recognition process by a general object recognition method to be described later. I do. Then, the reading unit 110 determines the product from one of the processing results. When the reading unit 110 determines the product, the reading unit 110 outputs the product information of the confirmed product (the product corresponding to the product code or the candidate product) to the display / operation unit 104. Thereafter, the reading unit 110 notifies the POS terminal 11 of product identification information of the product in order to register the confirmed product as a purchased product.

  The display / operation unit 104 displays, for example, a preset screen on which a product key for selecting a product, a numeric keypad for inputting the number, and the like are arranged on the third display device 106 so as to be inputable. Further, it is assumed that the product of the article imaged in the reading unit 110 is determined based on the result of the code reading process or the object recognition process. In that case, the display / operation unit 104 displays the product information of the confirmed product on the third display device 106.

  In the POS terminal 11, based on the product identification information notified from the product reading device 101, product information such as product classification, product name, unit price, discount, etc. of the product corresponding to the product identification information is registered in a registration file (not shown). Then, register the sales by performing the checkout process for the registered product for one transaction.

  FIG. 2 is a block diagram illustrating a hardware configuration of the POS terminal 11 and the commodity reading apparatus 101. The POS terminal 11 includes a CPU (Central Processing Unit) 61, a ROM (Read Only Memory) 62, a RAM (Random Access Memory) 63, a first display device 23, a touch panel 26, a second display device 24, a keyboard 22, and an HDD 64. , Drawer 21, printer 66, communication interface 25, connection interface 65, and the like. The CPU 61, the ROM 62, and the RAM 63 are connected by a bus. The first display device 23, the touch panel 26, the second display device 24, the keyboard 22, the HDD 64, the drawer 21, the printer 66, the communication interface 25, and the connection interface 65 all have various input / output circuits (all of which are shown in the figure). (Not shown) to the bus.

  The CPU 61 is a central processing unit that controls the entire POS terminal 11. The ROM 62 is a nonvolatile memory that stores fixed programs and the like. The RAM 63 is a volatile memory that the CPU 61 uses as a work area or the like.

  The HDD 64 stores various programs and various files. The various programs include a registration / settlement processing program PR for merchandise sales data processing. The various files are a PLU file F1 and the like distributed and stored from the store computer SC.

  In the PLU file F1, merchandise information of each article to be displayed and sold in a store and a feature amount of the merchandise are set in association with each other. The product information is product identification information such as a product code, product classification such as “vegetables”, a product name such as “carrot”, and other information such as unit price and discount. The feature amount is a feature amount indicating the feature of the article extracted in advance from the sample image of the article by the same predetermined algorithm as the object recognition process.

  The communication interface 25 is a network card or the like for performing data communication with the store computer SC. The store computer SC is installed in a store backyard or the like. A PLU file F1 distributed to the POS terminal 11 is stored in the HDD (not shown) of the store computer SC.

  The connection interface 65 is an interface that enables communication with the commodity reading apparatus 101. Communication is performed by wire or wireless. The printer 66 is a printing device that prints transaction contents on a receipt or the like and discharges the receipt.

  The commodity reading apparatus 101 includes a CPU 161, a ROM 162, a RAM 163, an imaging unit 164, an audio output unit 165, a first timer 166-1, a second timer 166-2, a connection interface 175, a connection interface 176, a touch panel 105, a third Display device 106, keyboard 107, fourth display device 109, and the like. The CPU 161, the ROM 162, and the RAM 163 are connected by a bus. In addition, the imaging unit 164, the audio output unit 165, the first timer 166-1, the second timer 166-2, and the connection interface 175 are all connected via the various input / output circuits (all not shown). Connected to the bus. In addition, the connection interface 175 and the connection interface 176 are connected so as to be able to communicate by wire or wirelessly. The touch panel 105, the third display device 106, the keyboard 107, and the fourth display device 109 are all connected to the connection interface 176 via various input / output circuits (all not shown).

  The CPU 161 is a central processing unit that controls the entire product reading apparatus 101. The ROM 162 is a non-volatile memory that stores a control program and the like. The control program includes a program for starting a code reading process and an object recognition process at a predetermined timing, and deferring output of a result obtained by the object recognition process. The RAM 163 is a volatile memory that the CPU 161 uses as a work area. A part of the storage area of the RAM 163 is an area for setting status flags of “first product detection flag”, “second product detection flag”, and “object recognition flag” which will be described later in the operation flow. Use as

  The imaging unit 164 is a color image sensor having an imaging element such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). The imaging unit 164 starts imaging in response to an imaging ON signal from the CPU 161, converts light (reflected light of an object) that has entered through the reading window 103 into an electrical signal at a predetermined frame rate (for example, 30 fps), and Output images sequentially.

  The audio output unit 165 includes a reproduction circuit, a speaker, and the like for reproducing a preset reading sound (“beep”), a warning sound, audio, and the like. The sound output unit 165 reproduces a reading sound, a warning sound, a sound, and the like by a control signal from the CPU 161 and notifies the reading state and the like by sound.

  The first timer 166-1 and the second timer 166-2 start timing by the control signal of the CPU 161, and output a signal to the CPU 161 when the set time is reached. In this example, information indicating a time interval from the start of the code reading process to the start of the object recognition process is set in the first timer 166-1. Also, information indicating a time interval (priority period to be described later) from the start of the code reading process to the end of the period for prioritizing the processing result of the code reading process is set in the second timer 166-2.

  Further, a connection interface 175 is connected to the CPU 161. The connection interface 175 is communicatively connected to the connection interface 65 of the POS terminal 11 to enable data transmission / reception between the commodity reading apparatus 101 and the POS terminal 11. The connection interface 175 is connected to the connection interface 176 to perform data transmission / reception with the display / operation unit 104.

  The connection interface 175 is an interface that enables communication between the POS terminal 11 and the display / operation unit 104 of the commodity reading apparatus 101. The connection interface 176 is an interface that enables communication with the POS terminal 11 and the reading unit 110 of the commodity reading apparatus 101.

Next, the functional block configuration of the POS terminal 11 and the commodity reading apparatus 101 will be described.
FIG. 3 is a diagram illustrating an example of the configuration of the functional blocks of the POS terminal 11 and the commodity reading apparatus 101. In the commodity reading apparatus 101, the CPU 161 appropriately executes a program stored in the ROM 162, and the image capturing unit 51, the commodity detecting unit 52, the code reading unit 53, the object recognizing unit 54, the reading control unit 55, the display control unit 56, and the input Functional units such as the reception unit 57 and the determination unit 58 are realized. In the POS terminal 11, the CPU 61 appropriately executes programs stored in the ROM 62 and the HDD 64 to realize functional units such as the sales registration unit 611.

  The image capturing unit 51 outputs an imaging on signal to the imaging unit 164 and causes the imaging unit 164 to start an imaging operation. The image capturing unit 51 captures frame images output from the image capturing unit 164 after the start of the image capturing operation, and sequentially stores them in the RAM 163.

  When the frame image is stored in the RAM 163 by the image capturing unit 51, the product detection unit 52 detects that an article has been trapped in the reading window 103 based on a plurality of continuous frame images. Specifically, when a frame image is output from the imaging unit 164, the product detection unit 52 reads the output frame image from the RAM 163 and compares the luminance with the most recently read frame image. In the luminance comparison, the commodity detection unit 52 determines that the frame image has some reflection of a hand or an article when there is a difference in luminance gradation. Further, the product detection unit 52 binarizes the frame image continuous with the reflected frame image and extracts the outline of the article. Then, when there is a change in the position of the contour line of the continuous frame image, the product detection unit 52 specifies the image within the contour line as an article image indicating the article that is trapped in the reading window 103. In this way, the commodity detection unit 52 detects that an article has been trapped in the reading window 103.

  The method for specifying the article image is not limited to the above, and other methods may be used. For example, you may make it identify with the presence or absence of a skin color area instead of the outline of the articles | goods in a frame image. If there is a skin color area, it is expected that there will be reflection of the cashier's hand. Therefore, a contour line is extracted from the frame image, and an image within the contour line of the contour line indicating the shape of the hand at the position where the article is gripped by the hand is specified as the article image.

  The code reading unit 53 performs code reading processing (reading processing) for reading a code symbol such as a bar code or a two-dimensional code (for example, QR code (registered trademark)) attached to an article, and is encoded. Output product code. Specifically, the code reading unit 53 reads a frame image including an article image from the RAM 163, and executes a code symbol decoding process on the frame image. When the code reading unit 53 obtains a product code in a predetermined format by the decoding process, the code reading unit 53 inquires the product code, for example, in the PLU file F1. Then, when the same product code exists, the code reading unit 53 outputs the product code to the reading control unit 55 as code reading information.

  The object recognition unit 54 performs object recognition processing (an example of extraction processing) on the article image in the frame image, and extracts and outputs candidate goods for the article (candidate merchandise information indicating the candidate goods). Specifically, the object recognizing unit 54 reads a frame image including an article image from the RAM 163, and reads a surface state such as the color of the article and surface irregularities from the article image as a feature amount of the article. Then, the object recognizing unit 54 collates the read feature quantity with the feature quantity of each product registered in the PLU file F1, and extracts candidate products (candidate product information) having a high degree of similarity (matching degree). Thereafter, the object recognition unit 54 outputs the extracted candidate product (candidate product information) to the reading control unit 55.

  In the present embodiment, the object recognizing unit 54 outputs to the reading control unit 55 candidate product information that has the highest similarity and is the highest among the extracted candidate product information. If all the similarities of the extracted candidate product information are less than the threshold value, the object recognition unit 54 assumes that the object cannot be recognized and does not output to the reading control unit 55. In this case, the object recognizing unit 54 reads out the frame image that has been subsequently captured from the RAM 163 and performs object recognition processing on the article image.

  The similarity indicates, for example, how similar the feature amount of the article image is when the feature amount of each product in the PLU file F1 is 100% = “similarity: 1.0”. The result of similarity may be calculated by absolute evaluation or may be calculated by relative evaluation.

  For example, assume that the similarity is calculated by absolute evaluation. In this case, the feature amount of the article image and the feature amount of each registered product in the PLU file F1 are compared on a one-to-one basis, and the similarity calculated as a result of this comparison is used as it is. Here, it is assumed that the similarity is 95% as a threshold value. In that case, when there is a registered product with a similarity of 95% or more, the object recognition unit 54 outputs the product with the highest similarity to the reading control unit 55. On the other hand, the object recognition unit 54 does not output to the reading control unit 55 when all the registered products have a similarity of less than 95%. That is, it is assumed that the object cannot be recognized, and the similarity of the product is recalculated for the captured frame image.

  Assume that the similarity is calculated by relative evaluation. In that case, calculation is performed so that the sum of the similarities with each registered product in the PLU file F1 is 1.0 (100%). For example, it is assumed that feature amounts of five registered products (first product, second product, third product, fourth product, and fifth product) are registered in the PLU file F1. In this case, the similarity is, for example, the similarity “0.6” for the first product, the similarity “0.1” for the second product, the similarity “0.1” for the third product, The similarity to the product is “0.1”, the similarity to the fifth product is “0.1”, etc., and the sum of the similarities is always 1.0. Here, it is assumed that the degree of similarity is 0.5 as a threshold value. In that case, the object recognition unit 54 outputs the registered product to the reading control unit 55 when there is a registered product having a similarity of 0.5 or more. On the other hand, the object recognition unit 54 does not output to the reading control unit 55 when all the registered products have a similarity of less than 0.5. That is, it is assumed that the object cannot be recognized, and the similarity of the product is recalculated for the captured frame image.

Note that object recognition for recognizing an object included in an image is also called generic object recognition. Various recognition techniques for general object recognition have been explained as in the following document, so please refer to them for details.
Keiji Yanai, “Current Status and Future of General Object Recognition”, IPSJ Journal, Vol. 48, no. SIG16 [Search August 10, 2010], Internet <URL: http://mm.cs.uec.ac.jp/IPSJ-TCVIM-Yanai.pdf>

  Jamie Shotton et al., “Semantic Texton Forests for Image Categorization and Segmentation”, [searched August 10, 2010], Internet <URL: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1. 1.145.3036 & rep = rep1 & type = pdf>

  After starting the code reading process, the reading control unit 55 starts the object recognition process at a first timing after a predetermined time has elapsed. Then, the reading control unit 55 determines the product code read by the code reading process during the period from the start of the code reading process to the second timing after the lapse of the first timing (priority period). 58 is preferentially output. When the result of the object recognition process is obtained within the priority period, the reading control unit 55 temporarily holds the output of the result. If the product code is not obtained by the code reading process within the priority period, the reading control unit 55 sends the result of the object recognition process obtained during the priority period to the determination unit 58 when the priority period ends. Output. If the result of the object recognition process is not obtained within the priority period, the reading control unit 55 determines the information obtained earlier by the code reading process and the object recognition process after the priority period ends. 58.

  Here, an example of a setting method of the first timing and the second timing will be described. First, the code reading process of the article with the barcode is tried for each point, and the elapsed time until the barcode reading is completed is obtained for each point. Then, the reading control unit 55 sets the elapsed time (or the latest time) until the reading of the barcode that matches a predetermined number of articles or all the articles among the tried articles is the second timing. Is set in the second timer 166-2. Further, the first timer 166-1 for detecting the first timing by the reading control unit 55 is different from the elapsed time set in the second timer 166-2, and is within the elapsed time other than 0. Set the value. More specifically, when the imaging speed of the imaging unit 164 is 30 fps, a value of, for example, about 300 msec to 500 msec is set as the elapsed time in the second timer 166-2. For example, a value of about 200 msec is set in the first timer 166-1. These set values are stored in advance in the ROM 162 or the like. Set values are set for the first timer 166-1 and the second timer 166-2 when the timer is started.

  The number of frames is stored in the ROM 162 or the like as information for detecting the first timing and the second timing so that the reading control unit 55 can detect the first timing and the second timing based on the number of frames. It may be deformed. For example, when the priority period is “500 msec”, the number of frames corresponding to 500 msec is stored.

  The reading control unit 55 starts the first timer 166-1 and the second timer 166-2 together with the start of the code reading process. When a signal is output from the first timer 166-1, it is determined that the object recognition processing start time is reached, and when a signal is output from the second timer 166-2, it is determined that the priority period is ended.

  The display control unit 56 displays various screens on the third display device 106. The display control unit 56 displays, as various screens, preset screens on which, for example, product keys for selecting products, numeric keys for inputting the number, and the like are arranged. In addition, the display control unit 56 displays a screen showing the result of the object recognition unit 54, a screen showing information about the product determined based on the results of the code reading unit 53 and the object recognition unit 54, and the like.

  The input receiving unit 57 receives various input operations via the touch panel 105 or the keyboard 107. For example, when an input screen is displayed on the third display device 106, a touch operation on the touch panel 105 or a press operation on the keyboard 107 is accepted as operation information of a corresponding operator on the input screen.

  The confirmation unit 58 confirms the product code or candidate product output by the reading control unit 55 as a purchased product, and notifies the sales registration unit 611 of the POS terminal 11 of the product identification information of the product. The notification to the sales registration unit 611 is performed by the confirmation unit 58 instructing the connection interface 175 to notify the sales registration unit 611. Thereby, the product identification information is transmitted from the connection interface 175 to the connection interface 65, and the product identification information is notified to the sales registration unit 611. The product identification information may be information that can uniquely identify the product with the POS terminal 11. For example, a product code or a product name can be used as the product identification information.

  The sales registration unit 611 receives the product identification information transmitted from the confirmation unit 58, and registers the product indicated by the product identification information as a purchased product. And the sales registration part 611 performs the adjustment process of the goods for one transaction registered so far, for example based on the closing input operation from the touch panel 26 etc., and sells registered goods.

Next, the operation of the checkout system 1 will be described.
FIG. 4 is a flowchart showing an example of processing for one transaction for the product registration and checkout processing of the checkout system 1. Hereinafter, the operation of the checkout system 1 will be described with reference to the flowchart shown in FIG.

  First, the POS terminal 11 instructs the product reading device 101 to start a product reading process (S1). For example, the cashier operates predetermined keys on the keyboard 22 and the touch panel 26 to declare the start of the product registration process. Based on the declaration, the POS terminal 11 transmits a reading start signal for instructing the merchandise reading apparatus 101 to start the merchandise reading process. Further, the POS terminal 11 waits at the sales registration unit 611 for product identification information notified from the product reading device 101 as a result of the product reading process or the like. The sales registration unit 611 is realized by the CPU 61 executing a program related to sales registration stored in the ROM 62, the HDD 64, or the like.

  Upon receiving the reading start signal transmitted from the product reading apparatus 101 in step S1, the product reading apparatus 101 performs a predetermined product reading process (S2). Specifically, the product reading apparatus 101 starts imaging of the imaging unit 164 (see FIG. 2) and captures a frame image in the imaging range including the reading window 103. Further, when the commodity reading apparatus 101 detects an article placed on the reading window 103 from the frame image, the commodity reading apparatus 101 executes a code reading process. Then, the object recognition processing is run in parallel at the first timing after a certain time has elapsed. When the product reading apparatus 101 obtains the result of the object recognition process within the priority period, the product reading apparatus 101 temporarily holds the result. Then, when the result of the code reading process is not obtained within the priority period, the commodity reading apparatus 101 determines the object recognition process result obtained within the priority period as a purchased commodity after the priority period ends. The series of product reading processes will be described later again using a flowchart (FIG. 5) shown as an example of the processes.

  After the process of step S2, the product reading apparatus 101 notifies the product registration information of the product confirmed in step S2 to the sales registration unit 611 of the POS terminal 11 (S3-1).

  Here, the product reading apparatus 101 accepts input of product identification information from the product key by the cashier in parallel with the series of product reading processes. Specifically, the display control unit 56 displays a preset screen on the third display device 106, and the input receiving unit 57 receives an operation input of a product key from the touch panel 105 or the keyboard 107 by the cashier. When the cashier operates the product key, the product reading device 101 notifies the sales registration unit 611 of the POS terminal 11 of product identification information corresponding to the product key.

  Subsequently, the POS terminal 11 receives the notification of the product identification information from the product reading device 101, and registers the product as a purchased product (S4-1). Specifically, the sales registration unit 611 receives the notification. Then, the sales registration unit 611 extracts various product information (price, discount, etc.) corresponding to the notified product identification information from the PLU file F1. Further, the sales registration unit 611 registers the extracted information set as a purchased product in a registration file (not shown) or the like.

  Steps S3-N and S4-N are repeated until all purchased items are read. However, it is set to an integer indicating N = 2, 3,.

  Subsequently, the POS terminal 11 receives a reading end signal indicating the end of the product reading process from the product reading device 101 (S5), and performs a settlement process for the registered product for one transaction (S6). For example, the cashier presses a tightening key on the touch panel 105 or the keyboard 107 of the commodity reading apparatus 101. Then, a reading end signal is transmitted from the commodity reading apparatus 101, and the POS terminal 11 receives this signal (S5). Upon receipt of the reading end signal, the POS terminal 11 extracts products for one transaction in the registration file and performs a settlement process such as calculating the total amount and change amount (S6).

  Next, a specific example of a series of product reading processes shown in steps S2 and S3 of FIG. 4 will be shown. In a series of product reading processes, the CPU 161 executes a program stored in the ROM 162, and the image capturing unit 51, the product detection unit 52, the code reading unit 53, the object recognition unit 54, the reading control unit 55, the input receiving unit 57, and the confirmation unit. This is done by realizing 58.

  FIG. 5 is a flowchart of a specific example of a series of product reading processing. In the following description, it is assumed that the above-described units are realized. Further, in this example, the reading control unit 55 secures flag areas of the “first product detection flag”, the “second product detection flag”, and the “object recognition flag” in the RAM 163, and performs processing according to the flag setting. An example of controlling is shown.

  First, the reading control unit 55 sets a default value “0” to the “first product detection flag”, the “second product detection flag”, and the “object recognition flag” in the RAM 163 (S21).

  Subsequently, the image capturing unit 51 outputs an imaging on signal to the image capturing unit 164 to capture a frame image from the image capturing unit 164, and the product detection unit 52 sorts the frame image including the image of the article with the frame image as a target ( S22). As this sorting process, the product detection unit 52 obtains a difference in luminance gradation from, for example, a frame image previously captured by the image capture unit 51. When there is a difference in luminance gradation, there is a possibility that an image of the article is included in each frame image. Therefore, in that case, the product detection unit 52 further extracts the contour lines of the objects from the respective frame images, and compares the positions of the contour lines of the respective objects. If there is a change in the position of the contour line by this comparison, there is a high possibility that the contour line indicates an article. In that case, the product detection unit 52 sorts the frame image captured by the image capturing unit 51 as a frame image to be read.

  Subsequently, the product detection unit 52 determines from the frame image whether or not the article is being trapped in the reading window 103 (S23). Here, the commodity detection unit 52 determines that the article is being trapped in the reading window 103 when the frame image to be read is obtained by the sorting process in step S22 (step S23: Yes determination). On the other hand, the merchandise detection unit 52 determines that the article is not trapped in the reading window 103 when the frame image to be read is not obtained in step S22 (step S23: No determination).

  Assume that step S23 is Yes. Then, the reading control unit 55 reads the value of the “first product detection flag” in the RAM 163 and determines whether it is set to “1” (S24). In the first round, since the “first product detection flag” is “0”, step S24 is No. From the second round onward, the “first product detection flag” is set to “1” in step S27 of the first round, so that the “first product detection flag” is returned to “0” in step S36. In step S24, a Yes determination is made.

  Assume that step S24 is No. Then, the reading control unit 55 instructs the code reading unit 53 to start up and starts the code reading process (S25). Thereby, the code reading unit 53 executes the code reading process on the frame image classified as the reading target in step S22.

  Further, the reading control unit 55 starts the first timer 166-1 and the second timer 166-2 (S26).

  Then, the reading control unit 55 changes the setting of the “first product detection flag” in the RAM 163 to “1” (S27), and proceeds to the process of step S32.

  On the other hand, suppose that step S24 was Yes determination. Then, the reading control unit 55 reads the value of the “second product detection flag” in the RAM 163 and determines whether it is set to “1” (S28). The “second product detection flag” is set to “0” in step S21. For this reason, step S28 is No determination in the 2nd round in which determination of step S24 is performed for the first time. From the third round onward, the “second product detection flag” is set to “1” in step S31 of the second round, so that the “second product detection flag” is returned to “0” in step S36. In step S28, a Yes determination is made. If step S28 is YES, the process proceeds to step S32.

  Assume that step S28 is No. Then, the reading control unit 55 determines whether the start time (first timing) of the object recognition process has come (S29). In this example, the reading control unit 55 determines that the first timing is reached when a signal indicating time-up is output from the first timer 166-1 (step S29: Yes determination). When the signal indicating the time up is not output from the first timer 166-1, the reading control unit 55 determines that the first timing is not reached (step S29: Yes determination). If step S29 is No, the process proceeds to step S32.

  Assume that step S29 is Yes. Then, the reading control unit 55 instructs the object recognition unit 54 to start up and starts the object recognition process (S30). Accordingly, the object recognition processing unit 54 executes the object recognition processing in parallel with the code reading processing of the code reading unit 53 for the frame image classified as the reading target in step S23.

  Then, the reading control unit 55 changes the setting of the “second product detection flag” in the RAM 163 to “1” (S31), and proceeds to the process of step S32.

  Subsequently, the reading control unit 55 determines whether the product code is read by the code reading unit 53 (S32). When the result of the code reading process is returned from the code reading unit 53, the reading control unit 55 determines that the product code has been read by the code reading unit 53 (step S32: Yes determination). If the result of the code reading process is not returned from the code reading unit 53, it is determined that the product code is not read by the code reading unit 53 (step S32: No determination). The case of No determination is a case where the code reading process by the code reading unit 53 is still being tried. For example, since the barcode attached to the article is not included in the frame image, the code reading unit 53 cannot read the product code yet. Further, since the article is not attached with a barcode, the code reading process itself is impossible.

  Assume that step S32 is Yes. Then, the determination unit 58 notifies the POS terminal 11 (sales registration unit 611) of product identification information corresponding to the read product code (S33). Specifically, when a product code is returned from the code reading unit 53, the reading control unit 55 outputs the product code to the determination unit 58. The confirmation unit 58 notifies the sales registration unit 611 of the product code or other product identification information corresponding to the product code.

  After the notification in step S33, the reading control unit 55 initializes the code reading process and the object recognition process (S34). For example, the reading control unit 55 instructs the code reading unit 53 and the object recognition unit 54 to initialize the code reading process and the object recognition process to the state before the start of the process. Further, the reading control unit 55 clears the frame image stored in the RAM 163, the display information of the candidate products used for display in step S39, the candidate product information stored in the RAM 163 in step 40, and the like. Steps S39 and S40 will be described later. Further, the reading control unit 55 stops the first timer 166-1 and the second timer 166-2 started in step S26 and resets them to the initial state (S35). Further, the reading control unit 55 returns the settings of the “first product detection flag”, the “second product detection flag”, and the “object recognition flag” in the RAM 163 to the default value “0” (S36). Then, when the process of step S36 ends, the process proceeds to step S22. On the other hand, if step S32 is No, the reading control unit 55 performs the following process.

  First, the reading control unit 55 determines whether a result is obtained from the object recognition unit 54 (S37). For example, it is assumed that the highest candidate product information whose similarity exceeds a predetermined threshold is returned from the object recognition unit 54 to the reading control unit 55. In that case, the reading control unit 55 determines that the result is obtained from the object recognition unit 54 (step S37: Yes determination). On the other hand, when no object recognition processing result is returned from the object recognition unit 54, the reading control unit 55 determines that no result is obtained from the object recognition unit 54 (S37: No determination). The case of No determination here is a case where the object recognition processing by the object recognition unit 54 is still being tried. For example, the object recognition process cannot be performed because the number of frame images to be read is insufficient. In addition, there is a case where a product whose similarity exceeds a predetermined threshold value cannot be obtained because the orientation of the article is bad.

  When the determination in step S37 is Yes, the reading control unit 55 reads the value of the “object recognition flag” in the RAM 163 and determines whether the “object recognition flag” is set to “1” (S38). Since the “object recognition flag” is “0” for the first time, step S38 is No.

  Here, it is assumed that step S38 is No. Then, the reading control unit 55 causes the third display device 106 to display the image of the candidate product information returned from the object recognition unit 54, product information, and the like (S39). Further, the reading control unit 55 performs control to temporarily hold the output of the candidate product information returned from the object recognition unit 54 to the determination unit 58, that is, the determination as a purchased product (S40). Specifically, the reading control unit 55 stores candidate product information returned from the object recognition unit 54 in the RAM 163. Then, the reading control unit 55 outputs the candidate product information to the determination unit 58 until the priority period ends, that is, until the second timer 166-2 outputs a signal indicating time-up in this example. Hold output. Then, the reading control unit 55 changes the setting of the object recognition flag in the RAM 163 from “0” to “1” (S41), and proceeds to the process of step S42. Note that the display information in step S39 and the hold information in step S40 are cleared from the RAM 163 by the process in step S34.

  On the other hand, if step S38 is Yes, step S39 to step S41 are skipped and the process proceeds to step S42.

  Subsequently, the reading control unit 55 determines whether or not the end time (second timing) of the priority period has come (S42). In this example, the reading control unit 55 determines that the priority period has ended when a signal indicating time-up is output from the second timer 166-2 (step S42: Yes determination). If the second timer 166-2 does not output a signal indicating time-up, the reading control unit 55 determines that it is within the priority period (step S42: No determination). If step S42 is No, the process returns to step S22.

  On the other hand, when the determination in step S42 is Yes, the reading control unit 55 outputs the candidate product information obtained by the object recognition process to the determination unit 58 and notifies the sales registration unit 611 of the product identification information (S43). For example, the reading control unit 55 reads candidate product information temporarily suspended from being output to the determination unit 58 from the RAM 163, and outputs it to the determination unit 58. In addition, in the processing after the priority period when the candidate product is not obtained within the priority period, the reading control unit 55 outputs the candidate product information obtained at the time of the latest step S37 to the determination unit 58. To do. The confirmation unit 58 automatically confirms the candidate product output from the reading control unit 55 as a purchased product, and notifies the sales registration unit 611 of the product identification information of the candidate product. Then, the process proceeds to step S34.

  When step S37 is No, the reading control unit 55 determines whether the end time of the priority period has been reached (step S44), as in step S42. Furthermore, when step S44 is No determination, the reading control part 55 determines whether the goods key was pressed down (S45). If the product key is pressed (step S45: Yes determination), the confirmation unit 58 notifies the product registration information corresponding to the product key to the sales registration unit 611 (S46). After step S46, the process proceeds to step S34. On the other hand, when the product key is not pressed (step S45: No determination), the process proceeds to step S22.

  If the determination in step S44 is Yes, the reading control unit 55 reads the value of the “object recognition flag” in the RAM 163 and determines whether the “object recognition flag” is set to “1” as in step S38. Determine (S47). If the determination at least once in step S37 after the start of the object recognition process in step S30, the “object recognition flag” becomes “1”. Therefore, in such a case, step S47 becomes Yes determination and it transfers to the process of step S43. That is, the reading control unit 55 temporarily outputs candidate product information whose output to the determination unit 58 has been temporarily suspended to the determination unit 58 and notifies the sales registration unit 611 of the product identification information. On the other hand, if the determination in step S37 has never been Yes, the “object recognition flag” remains “0”. In that case, step S47 becomes No determination and it transfers to the process of step S45.

  If step S23 is No, the reading control unit 55 determines whether the second timer 166-2 is being activated (S48). When the 2nd timer 166-2 is starting (step S48: Yes determination), it transfers to the process of step S44. When the second timer 166-2 is stopped (step S48: No determination), the process proceeds to step S22.

  In the above processing, if the product code is obtained from the code reading unit 53 within the priority period, the determination in step S32 is Yes and step S33 is executed. Thereby, the product reading apparatus 101 transmits product identification information based on the product code to the POS terminal 11.

  Further, it is assumed that the result is obtained from the object recognition unit 54 before the code reading unit 53 within the priority period. In that case, step S37 becomes Yes determination, and the output of the result is temporarily suspended by step S40. Thereafter, when the product code is obtained from the code reading unit 53 within the priority period, Step S32 is Yes and Step S33 is executed. That is, the product reading apparatus 101 transmits product identification information based on the product code to the POS terminal 11. In that case, the result of temporarily holding the output in step S40 is discarded in step S34.

  Further, it is assumed that the product code is not obtained from the code reading unit 53 within the priority period. In this case, if a result is obtained from the object recognition unit 54 within the priority period, the determination process of the first step S42 or step S44 performed after the end of the priority period is Yes, and step S43 is executed. That is, the merchandise reading apparatus 101 transmits to the POS terminal 11 the merchandise identification information as a result of the output temporarily suspended in step S40.

  Further, it is assumed that no result is obtained from the code reading unit 53 or the object recognition unit 54 within the priority period. In that case, it becomes No determination by determination of the first step S47 performed after completion | finish of a priority period. And step S45 (No determination), step S22, S23 (Yes determination or No determination), S24 (Yes determination), S28 (Yes determination), S32 (No determination), S37 (No determination), S44 (Yes determination) The loop processing of S47 (No determination) is repeated until the determination result of any one of step S32, step S37, or step S45 is Yes.

  As described above, in this embodiment, the commodity reading apparatus 101 as the information processing apparatus includes the imaging unit 164 as the imaging unit, the image capturing unit 51 as the capturing unit, the commodity detecting unit 52 as the detecting unit, A code reading unit 53 as a first extraction unit, an object recognition unit 54 as a second extraction unit, a reading control unit 55 as an instruction unit and a notification control unit, and a determination unit 58.

  With this configuration, the commodity reading apparatus 101 starts the object recognition process after a predetermined time of the code reading process. During the priority period, even if the result of the object recognition process is obtained before the result of the code reading process, the result of the object recognition process is suspended, and the result of the code reading process obtained later is stored in the article. Applicable as product identification information. When the result of the code reading process cannot be obtained within the priority period, the result of the object recognition process obtained and held during the priority period is applied as product identification information indicating the article.

  As a result, only the code reading process is performed until the first timing elapses from the start of the code reading process, and the CPU load during this period can be reduced. In addition, since the object recognition process is performed in parallel with the code reading process from the first timing, there is a high possibility that the result of the object recognition process will be output by the second timing. If the result of the object recognition process has been obtained by the second timing, the result of the object recognition process can be displayed immediately or confirmed as a purchased product at the second timing. Even if the result is not obtained by the second timing, the object recognition process has already started before the second timing, so the result of the object recognition process at an early stage after the elapse of the second timing. Can be displayed or confirmed. Therefore, it is possible to shorten the acquisition time of the result of the object recognition process per article.

(Modification)
In the embodiment, the object recognizing unit 54 outputs the candidate product (candidate product information) having the highest similarity among the extracted candidate products to the reading control unit 55, and the confirming unit 58 purchases the product. It showed about the thing of the aspect determined automatically as goods. Here, the object recognizing unit 54 outputs the candidate product (candidate product information) of the upper group among the extracted candidate products to the reading control unit 55, and the confirming unit 58 inputs one candidate product from the group to the cashier. A modification example to be determined according to is shown. Below, the description which overlaps with embodiment is abbreviate | omitted suitably, and demonstrates the main characteristics of a modification.

  In the checkout system of the modification, when the object recognition unit 54 (see FIG. 3) extracts candidate products (candidate product information) by the object recognition process, the upper group (for example, the top six candidate product information) is extracted. Is output to the reading control unit 55 (see FIG. 3).

  When the reading control unit 55 acquires the candidate product information of the upper group, the determination unit 58 (see FIG. 3) until the information of one candidate product selected by the cashier is acquired from the input receiving unit 57 (see FIG. 3). Suspend the output of candidate product information to. Therefore, during this time, the reading control unit 55 stores the candidate product information of the upper group in the RAM. Further, when the reading control unit 55 acquires the candidate product information of the upper group, the reading control unit 55 instructs the display control unit 56 (see FIG. 3) to display the candidate product of the upper group. Further, when the reading control unit 55 acquires information indicating one candidate product selected by the cashier from the input receiving unit 57, the reading control unit 55 outputs candidate product information in the RAM corresponding to the information to the determination unit 58.

  When the display control unit 55 is instructed to display the upper group candidate product, the display control unit 56 generates display information of the upper group candidate product and outputs it to the third display device 106 (see FIG. 2). .

  When the upper group candidate products are displayed on the third display device 106, the input receiving unit 57 receives a selection input by the cashier of one of the candidate products.

  When one candidate product is selected by the cashier, the determination unit 58 determines the candidate product as a purchased product.

Next, a reading control flow of the modified product reading apparatus will be described.
FIG. 6 is a flowchart of reading control of the merchandise reading apparatus according to the modification. FIG. 6 is obtained by adding candidate product display processing, determination processing, and the like to the reading control flowchart (see FIG. 5) of the product reading apparatus 101 of the embodiment. Below, a different part from FIG. 5 is demonstrated.

  First, the processing from step S39 is shown. In step S <b> 39, the reading control unit 55 causes the third display device 106 to display the selection screen for candidate products of the upper group returned from the object recognition unit 54. Specifically, the reading control unit 55 instructs the display control unit 56 to display a selection screen for candidate products in the upper group. Then, the display control unit 56 generates display screen information including a selection key for the candidate product of the upper group, and outputs the display screen information to the third display device 106. As a result, a selection screen for candidate products for the upper group is displayed on the third display device 106. Further, in step S <b> 40, the reading control unit 55 temporarily stores the candidate product information of the upper group returned from the object recognition unit 54 in the RAM 163 and suspends output to the determination unit 58. In step S41, the reading control unit 55 changes the setting of the object recognition flag in the RAM 163 from “0” to “1”.

  Subsequently, the reading control unit 55 determines whether one candidate product is selected from the candidate products of the upper group (S50). Specifically, it is assumed that one candidate product is selected from the upper group candidate products displayed on the third display device 106 by, for example, touching a selection key corresponding to the cashier. Then, the input receiving unit 57 receives the input of the selection key. The reading control unit 55 obtains input information from the input receiving unit 57, and determines whether any one of the candidate product information of the upper group candidate product information stored in the RAM 163 is included in the input information. The above determination is made by making a comparison.

  Here, it is assumed that the cashier has selected one candidate product (step S50: Yes determination). In that case, the process proceeds to step S43, and the reading control unit 55 outputs the candidate product information obtained by the object recognition process to the determination unit 58. However, the candidate product information output to the determination unit 58 is only one candidate product information selected by the cashier.

  On the other hand, when the cashier has not selected a candidate product (step S50: No determination), the reading control unit 55 performs a process of determining the end of the priority period in step S42. When it is within the priority period (step S42: No determination), the process proceeds to step S22. When the priority period has expired (step S42: Yes determination), the reading control unit 55 displays information indicating that on the selection screen of the third display device 106 (S51). Specifically, the reading control unit 55 instructs the display control unit 56 to display a screen that prompts the cashier to input a selection key. Then, the display control unit 56 generates display screen information of a screen in which information for prompting the input of the selection key is set on the selection screen, and outputs the display screen information to the third display device 106. Information prompting the selection key to input is, for example, a message such as “Please press the selection key”, a change to a predetermined color such as the selection key or background screen, or a blinking screen. What can be done may be set appropriately.

  When the process of step S51 ends, the process proceeds to step S50, and it is determined again whether one candidate product has been selected from the candidate products of the higher group. The loop processing in the order of steps S50 (No determination), S42 (Yes determination), S51, and S50 is repeated until the cashier selects one candidate product and a Yes determination is made in step S50.

  Next, the process from step S37 (No determination) is shown. In step S37, it is assumed that the reading control unit 55 determines that no result is obtained from the object recognition unit 54 (step S37: No determination). Then, as in step S50, the reading control unit 55 determines whether one candidate product is selected from the candidate products of the upper group (S52). If the determination in step S37 is once even after the start of the object recognition process in step S30, the candidate product in the upper group is displayed on the third display device 106. Therefore, if the cashier touches the corresponding selection key, step S52 is Yes. On the other hand, if the determination in step S37 has never been Yes, the candidate product of the upper group is not displayed on the third display device 106, and therefore the determination in step S52 is No.

  When step S52 is Yes, the process proceeds to step S43, and the reading control unit 55 outputs the candidate product information obtained by the object recognition process to the determination unit 58. However, the candidate product information output to the determination unit 58 is only one candidate product information selected by the cashier.

  When step S52 is No, the reading control unit 55 performs a process of determining whether the end time of the priority period is reached in step S44. If it is within the priority period (step S44: No determination), the process proceeds to step S45. When the priority period has expired (step S44: Yes determination), the reading control unit 55 reads the value of the “object recognition flag” in the RAM 163 and determines whether the “object recognition flag” is set to “1”. (S47).

  Here, it is assumed that step S47 is Yes determination. In that case, the reading control unit 55 displays information prompting the cashier to input a selection key on the selection screen of the third display device 106, similarly to step S51 (S53). When the process of step S53 ends, the process proceeds to the process of step S52, and the reading control unit 55 determines again whether one candidate product is selected from the candidate products of the upper group. Thereafter, the loop processing in the order of steps S52 (No determination), S44 (Yes determination), S47 (Yes determination), S53, and S52 is repeated until the cashier selects one candidate product and the determination in Step S52 is Yes. . On the other hand, if Step S47 is No, the process proceeds to Step S45.

  Even in the case of step S48 (Yes determination), the determination process of step S52 is performed, and the above-described procedure is performed.

  In the modified example, when the priority period ends, information that prompts the user to select a higher candidate product extracted by the object recognition process using a selection key on the display screen is visually displayed on the display screen. However, the present invention is not limited to this, and the selection may be prompted by an alarm sound or voice. For example, when the priority period ends, a sound such as a predetermined alarm sound (for example, “Piece”) or “Please select a candidate product from the selection key” may be reproduced in the sound output unit 165.

  As described above, in the modification, when the result is returned from the object recognition unit 54, the reading control unit 55 displays the selection key for the candidate product of the upper group on the screen. It is known that a cashier is an article that is not pre-barcoded when an article such as cabbage is placed on the reading window 103. Accordingly, in such a case, even within the priority period, the cashier can select the selection key to determine the article as one candidate product among the upper candidate products. After the priority period is over, the screen is switched to a screen prompting the cashier to input a selection key. For this reason, the cashier can visually determine that the priority period has ended and can promptly input the selection key.

  As described above, in the modification, it is possible to finish the determination of the product from the candidate products obtained by the object recognition process within the priority period.

  In the embodiment and the modification, the product detection unit 52 is provided as a detection unit, and the product detection unit 52 detects an article image from the frame image, thereby detecting that the article has been tampered with the reading window 103. . However, it may be detected by other detection means that the article has been trapped in the reading window 103. For example, it may be detected by an infrared sensor that an article has been trapped in the reading window 103.

  In addition to the code reading process and the object recognition process, other reading processes may be applied.

  Further, the value of about 300 msec to 500 msec shown as the priority period is an example, and the setting may be appropriately adjusted according to the processing capability of the cashier, the specifications of the apparatus, the speed of the object recognition processing, and the like. For example, in the case of a cashier that reads a bar code quickly, the priority period may be further shortened. In addition, in the case of a cashier that reads barcodes slowly, the priority period may be further extended.

  Moreover, you may provide the change means for changing a priority period. For example, a setting screen for setting a priority period is provided as changing means, and the setting screen is displayed on the display screen in response to a call signal. When the priority period is set by a cashier or the like on the displayed setting screen, the setting information is reflected in the priority period of the program.

  In the present embodiment and the modification, an example in which the information processing apparatus and the program are applied to a separate checkout system in which a POS terminal that performs settlement and product registration and a reading device that reads product identification information are applied. But this is not the case. Such an information processing apparatus and program may be applied to, for example, a self POS that performs settlement, product registration, and product identification information reading by an integrated device.

Here, a configuration example of the self-POS is shown.
FIG. 7 is an external perspective view showing the configuration of the self-POS 200.
FIG. 8 is a block diagram showing a hardware configuration of the self-POS 200.
In addition, below, the same code | symbol is attached | subjected and shown about the same structure shown by FIG. 1 and FIG. 2, and the duplicate description is abbreviate | omitted.

  As shown in FIGS. 7 and 8, the main body 202 of the self-POS 200 includes a third display device 106 having a touch panel 105 disposed on the surface, a reading unit 110 for reading a product image, and the like. The third display device 106 displays a settlement screen in addition to the preset screen and the candidate product selection screen shown in the embodiment. The main body 202 includes a product mounting table 203 for placing a shopping cart 153a (see FIG. 1) containing unsettled products on the right side, and a product mounting table 204 for placing settled products on the left side. . The product placing table 204 includes a bag hook 205 for hanging a bag for putting a settled product, and a temporary placing table 206 for temporarily placing the settled product before putting it in the bag. The product placement tables 203 and 204 also have weighing devices 207 and 208, respectively, and have a function for confirming that the product weight is the same before and after the settlement. Further, the main body 202 includes a changer 201 on the front surface.

  In the self-POS 200, the CPU 161 appropriately executes the program stored in the ROM 162, and the image capturing unit 51, the product detection unit 52, the code reading unit 53, the object recognition unit 54, the reading control unit 55, and the display control unit 56 shown in FIG. , Function units such as the input reception unit 57 and the determination unit 58 are realized. Further, the CPU 61 appropriately executes a storage program in the ROM 62 or the HDD 64 to realize a functional unit such as the sales registration unit 611 shown in FIG. When there is an input declaring the start of the product registration process from the touch panel 105, the input signal is input to the CPU 61 via the connection interface 176 and the connection interface 65. Based on the input signal, the CPU 61 outputs a reading start signal to the reading unit 110 via the connection interface 65 and the connection interface 175. Other operations of the self-POS 200 can be described in substantially the same manner as described with reference to FIGS. The description of the operation here will be omitted because it will be repeated.

  A predetermined program is stored in the ROM 62 or the HDD 64, and the CPU 61 in addition to the sales registration unit 611, an image capture unit 51, a product detection unit 52, a code reading unit 53, an object recognition unit 54, a reading control unit 55, a display You may make it implement | achieve a part or all of functional parts, such as the control part 56, the input reception part 57, and the determination part 58. FIG. When the CPU 61 realizes the reading control unit 55, the determination unit 58 is omitted, and the reading control unit 55 directly outputs the product code and the product identification information of the candidate product to the sales registration unit 611 to register the product.

  Various programs used in the information processing apparatus of the present embodiment and the modification are files in an installable or executable format, such as a CD-ROM, a flexible disk (FD), a CD-R, a DVD (Digital Versatile Disk), and the like. The program may be provided by being recorded on a computer-readable recording medium, and may be executed by being read into an HDD or a flash ROM of an information processing apparatus.

  Further, the program may be provided by being stored on a computer connected to a network such as the Internet and downloaded via the network.

  In the above embodiment and modification, the configuration of the information processing apparatus and the program has been described. However, this embodiment and modification are presented as examples, and are not intended to limit the scope of the invention. . The novel embodiments and modifications can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

11 POS terminal 51 Image capture unit
52 Product detection unit 53 Code reading unit 54 Object recognition unit 55 Reading control unit 56 Display control unit 57 Input reception unit 58 Determination unit 61, 161 CPU
64 HDD
101 Product Reading Device 611 Sales Registration Department F1 PLU File

JP 2013-178722 A

Claims (8)

Capture means for capturing an image of the article imaged by the imaging means;
Detection means for detecting the article;
First extraction means for performing an extraction process for extracting a code symbol added to an article detected by the detection means, based on an image captured by the capture means;
A second extraction unit that performs an extraction process for extracting information indicating the article detected by the detection unit based on the image captured by the capture unit;
When the detection means detects an article, only the extraction process by the first extracting means the start of the extraction process only on the first extraction means is instructed to be performed, the start of the extraction process by the first extraction means The extraction process is started in the second extraction unit so that the extraction process by the first extraction unit and the extraction process by the second extraction unit are performed in parallel at a first timing after a predetermined time has elapsed. An instruction means for instructing
During the period from the start of the extraction process by the first extraction means to the second timing after the first timing, if I Rico Doshinboru is extracted in the first extraction means, extracted code symbol the notify commodity registration destination, if the code symbol was not extracted into the period, extracted by the extraction process in the period until the second timing from the first timing by said second extraction means Notification control means for notifying the commodity registration destination of information indicating an article after the second timing ;
An information processing apparatus comprising: If the code symbol is not extracted by the first extraction unit by the second timing , the notification control unit has the second control unit within a period from the first timing to the second timing . Information indicating the article stored in the storage means after the second timing is temporarily stored in the storage means on condition that the information indicating the article is extracted by the extraction means. To the product registration destination,
The information processing apparatus according to claim 1. The notification control means, when the information indicating the code symbol or the article until the second timing is not together extracted, after said second timing, previous among the information indicating the code symbol or the article To notify the product registration destination of the information extracted in
The information processing apparatus according to claim 1 or 2. The notification control means obtains, from the second extraction means, information indicating an article having a matching level that is equal to or higher than a predetermined level as information indicating the article to be notified to the commodity registration destination.
The information processing apparatus according to claim 1, wherein the information processing apparatus is an information processing apparatus. The notification control unit displays a selection screen for selecting one piece of information from the plurality of information on the display unit when the plurality of pieces of information indicating the article are extracted by the second extraction unit, and the selection When one piece of information is selected on the screen, the one piece of information is notified to the product registration destination after the second timing .
The information processing apparatus according to claim 1, wherein the information processing apparatus is an information processing apparatus. Capture means for capturing an image of the article imaged by the imaging means;
Detection means for detecting the article;
First extraction means for performing an extraction process for extracting a code symbol added to an article detected by the detection means, based on an image captured by the capture means;
A second extraction unit that performs an extraction process for extracting information indicating the article detected by the detection unit based on the image captured by the capture unit;
An instruction means for instructing the first extraction means to start extraction processing when the detection means detects an article, and for instructing the second extraction means to start extraction processing at a first timing after the start;
When a code symbol is extracted by the first extraction unit during a period from the start of the extraction process by the first extraction unit to a second timing after the first timing, the extracted code symbol is When the product registration destination is notified and the code symbol is not extracted during the period, the article extracted by the extraction process within the period from the first timing to the second timing by the second extracting means If the information indicating the code symbol or the article is not extracted by the second timing, the information to be shown is notified to the product registration destination after the second timing, and after the second timing, Notification control means for notifying the product registration destination of information extracted earlier in the code symbol or information indicating the article;
An information processing apparatus comprising: Computer
Capture means for capturing an image of the article imaged by the imaging means;
Detection means for detecting the article;
First extraction means for performing an extraction process for extracting a code symbol added to an article detected by the detection means, based on an image captured by the capture means;
A second extraction unit that performs an extraction process for extracting information indicating the article detected by the detection unit based on the image captured by the capture unit;
When the detection means detects an article, only the extraction process by the first extracting means the start of the extraction process only on the first extraction means is instructed to be performed, the start of the extraction process by the first extraction means The extraction process is started in the second extraction unit so that the extraction process by the first extraction unit and the extraction process by the second extraction unit are performed in parallel at a first timing after a predetermined time has elapsed. An instruction means for instructing
During the period from the start of the extraction process by the first extraction means to the second timing after the first timing, if I Rico Doshinboru is extracted in the first extraction means, extracted code symbol the notify commodity registration destination, if the code symbol was not extracted into the period, extracted by the extraction process in the period until the second timing from the first timing by said second extraction means Notification control means for notifying the commodity registration destination of information indicating an article after the second timing ;
Program to work with. Computer
Capture means for capturing an image of the article imaged by the imaging means;
Detection means for detecting the article;
First extraction means for performing an extraction process for extracting a code symbol added to an article detected by the detection means, based on an image captured by the capture means;
A second extraction unit that performs an extraction process for extracting information indicating the article detected by the detection unit based on the image captured by the capture unit;
An instruction means for instructing the first extraction means to start extraction processing when the detection means detects an article, and for instructing the second extraction means to start extraction processing at a first timing after the start;
When a code symbol is extracted by the first extraction unit during a period from the start of the extraction process by the first extraction unit to a second timing after the first timing, the extracted code symbol is When the product registration destination is notified and the code symbol is not extracted during the period, the article extracted by the extraction process within the period from the first timing to the second timing by the second extracting means If the information indicating the code symbol or the article is not extracted by the second timing, the information to be shown is notified to the product registration destination after the second timing, and after the second timing, Notification control means for notifying the product registration destination of information extracted earlier in the code symbol or information indicating the article;
Program to make it work.

Images