US20160358150A1 - Information processing apparatus and commodity recognition method by the same - Google Patents
Information processing apparatus and commodity recognition method by the same Download PDFInfo
- Publication number
- US20160358150A1 US20160358150A1 US15/242,787 US201615242787A US2016358150A1 US 20160358150 A1 US20160358150 A1 US 20160358150A1 US 201615242787 A US201615242787 A US 201615242787A US 2016358150 A1 US2016358150 A1 US 2016358150A1
- Authority
- US
- United States
- Prior art keywords
- commodity
- standard
- standard commodity
- candidate
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
- G07G1/00—Cash registers
- G07G1/12—Cash registers electronically operated
- G07G1/14—Systems including one or more distant stations co-operating with a central processing unit
- G07G1/145—PLU-management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q20/00—Payment architectures, schemes or protocols
- G06Q20/08—Payment architectures
- G06Q20/20—Point-of-sale [POS] network systems
- G06Q20/208—Input by product or record sensing, e.g. weighing or scanner processing
-
- G06K9/4604—
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/183—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a single remote source
Definitions
- Embodiments described herein relate to an information processing apparatus and a commodity recognition method executed by the information processing apparatus.
- the feature amount (e.g. shape) of a commodity is acquired from an image data of the commodity photographed and a candidate of the recognition commodity is determined based on the acquired feature amount.
- FIG. 1 is a perspective view of the external construction of a checkout system according to an embodiment
- FIG. 2 is a block diagram illustrating the hardware configuration of a POS terminal and a commodity reading apparatus shown in FIG. 1 ;
- FIG. 3 is a schematic diagram illustrating an example of the data configuration of a PLU file shown in FIG. 2 ;
- FIG. 4 is a block diagram illustrating the functional configuration of the POS terminal and the commodity reading apparatus shown in FIG. 1 ;
- FIG. 5 is a diagram illustrating an example of a commodity candidate displayed on the display device of the commodity reading apparatus
- FIG. 6 is a diagram illustrating an example of a commodity candidate displayed on the display device of the commodity reading apparatus
- FIG. 7 is a flow chart illustrating the procedure of a commodity recognition processing carried out by the commodity reading apparatus
- FIG. 8 is a flow chart illustrating the procedure of a sales registration processing carried out by the POS terminal
- FIG. 9 is a perspective view illustrating the configuration of a self-checkout POS according to an embodiment
- FIG. 10 is a block diagram illustrating the hardware configuration of the self-checkout POS terminal shown in FIG. 9 .
- an information processing apparatus comprising a processor configured to execute a program to implement elements, comprising an acquisition module configured to acquire an image photographed by an image capturing module; an extraction module configured to extract, from the photographed image, a feature amount representing an external feature of a commodity photographed by the image capturing module; a recognition module configured to recognize a standard commodity that is a candidate for the commodity by comparing the feature amount extracted by the extraction module with a feature amount of each standard commodity; a determination module configured to determine, for the standard commodity recognized by the recognition module, whether or not there is any related standard commodity belonging to a same commodity item as the standard commodity recognized and having an external feature different from an external feature of the standard commodity recognized; and a display module configured to display the standard commodity recognized by the recognition module as the candidate for the commodity contained in the photographed image, and display the related standard commodity as a candidate for the commodity if the determination module determines that there is any related standard commodity.
- Embodiments of the information processing apparatus and method thereof are described below with reference to the accompanying drawings.
- the embodiments are only illustrative of but are not to be construed as limiting the configuration and specification and the like of the information processing apparatus.
- the embodiments described herein are application examples of a checkout system, including a POS terminal and a commodity reading apparatus, which is installed in a supermarket or store.
- the POS terminal executes a registration and settlement of a commodity in one transaction and the commodity reading apparatus reads information relating to the commodity.
- FIG. 1 is a perspective view of the external construction of a checkout system 1 .
- the checkout system 1 comprises a POS terminal 11 and a commodity reading apparatus 101 serving as an information processing apparatus.
- the POS terminal 11 is placed on the upper surface of a drawer 21 arranged on a checkout counter 51 .
- the drawer 21 is opened under the control of the POS terminal 11 .
- a keyboard 22 which is operated by an operator (shop clerk) is arranged on the upper surface of the POS terminal 11 .
- a display for operator 23 for displaying information to the operator is arranged at a back side through the keyboard 22 from the operator who operates the keyboard 22 .
- the display for operator 23 displays information on the display surface 23 a thereof.
- a touch panel 26 is laminated on the display surface 23 a.
- a display for customer 24 is arranged in a free rotatable manner at a further back side to the display for operator 23 .
- the display for customer 24 displays information on the display surface 24 a thereof.
- the display surface 24 a of the display for customer 24 shown in FIG. 1 faces the operator, but it is rotated so that the display surface 24 a faces a customer to display information to a customer.
- a horizontally long table-shaped counter 151 is arranged such that it is located in a right angle to the checkout counter 51 on which the POS terminal 11 is placed.
- a receiving surface 152 is formed on the upper surface of the counter 151 .
- a shopping basket 153 for accommodating a commodity G is placed on the receiving surface 152 .
- the shopping basket 153 includes a first shopping basket 153 a, placed on the counter at a side, which is held by a customer and a second shopping basket 153 b which is placed on the counter 51 at a side opposite to the first shopping basket 153 a through the commodity reading apparatus 101 .
- the commodity reading apparatus 101 connected with the POS terminal 11 to perform transmission of data is arranged on the receiving surface 152 of the counter 151 .
- the commodity reading apparatus 101 has a thin rectangular shaped housing 102 .
- a reading window 103 is arranged at the front side of the housing 102 .
- a display operation section 104 is arranged on the upper part of the housing 102 .
- a display device 106 having a touch panel 105 laminated on the surface thereof is arranged on the display operation section 104 .
- a keyboard 107 is arranged at the right-hand side of the display device 106 .
- the card reading slot 108 of a card reader (not shown) is arranged at the right-hand side of the keyboard 107 .
- a display for customer 109 is arranged at the left-hand side of the display operation section 104 with respect to the operator to provide information to a customer.
- Commodities G in one translation are housed in the first shopping basket 153 a carried by the customer.
- the commodities G in the first shopping basket 153 a are moved one by one to the second shopping basket 153 b by the operator who operates the commodity reading apparatus 101 .
- the commodity G is exposed to the reading window 103 of the commodity reading apparatus 101 .
- the image capturing module 164 (refer to FIG. 2 ) arranged inside the reading window 103 photographs the commodity G.
- FIG. 2 is a block diagram illustrating the hardware configuration of the POS terminal and the commodity reading apparatus 101 .
- the POS terminal 11 is provided with a microcomputer 60 functioning as an information processing unit for carrying out an information processing.
- the microcomputer 60 includes a CPU (Central Processing Unit) 61 for carrying out various operations to control each section, a ROM (Read Only Memory) 62 and a RAM (Random Access Memory) 63 connected via a bus line.
- CPU Central Processing Unit
- ROM Read Only Memory
- RAM Random Access Memory
- the drawer 21 , the keyboard 22 , the display device 23 , the display for customer 24 , the communication interface 25 and the touch panel 26 are connected with the CPU 61 of the POS terminal 11 via various input/output circuits (not shown).
- the keyboard 22 includes a numeric keypad 22 d on which numeric characters such as ‘1’, ‘2’, ‘3’ and a multiplying operator such as ‘*’ are indicated, a temporary closing key 22 e and a closing key 22 F.
- the CPU 61 of the POS terminal 11 is connected with an HDD (Hard Disk Drive) 64 , in which various programs and files are stored. Programs and files stored in the HDD 64 are copied in all or in part onto the area of the RAM 63 and are executed by the CPU 61 when the POS terminal 11 is activated.
- HDD Hard Disk Drive
- the HDD 64 stores a data file such as a PLU file F 1 and the like.
- the PLU file F 1 is readable by the commodity reading apparatus 101 through a connection interface 65 .
- the PLU file F 1 is a data file in which a commodity G sold in a store is associated with information concerning the sales registration of the commodity G.
- FIG. 3 is a schematic diagram illustrating an example of the data configuration of the PLU file F 1 .
- a unique commodity ID is registered in association with information relating to a commodity (such as a commodity category to which a commodity belongs, a commodity name, a unit price, etc.), an image of the commodity photographed and a disparate sales information showing whether or not there is other commodity sold in a different form for the same commodity.
- the feature amount (feature amount data of a standard commodity) of each commodity G is registered (stored) in advance in the PLU file F 1 in an associated manner.
- the commodity image is a standard image (reference image) obtained by photographing each commodity G as a standard commodity.
- the commodity image is showed as a commodity candidate described later when the commodity candidate is presented. It is preferable to show, as a commodity image, an image illustrating the sales form of the commodity, for example, for a commodity sold in a form in which it is cut, and the commodity image that the cut surface of the commodity is photographed is used.
- a commodity G is sold in a plurality of sales forms in the same commodity G
- the commodity ID of the commodity G sold in a different sales form to the same commodity G sold in a standard sales form is added to the commodity G sold in the standard sales form as disparate sales information.
- the commodity ID “XXXXXXX3’ of the commodity ‘chinese cabbage’ sold in a form in which it is cut in half is registered as a disparate sales information to the commodity ‘chinese cabbage’ of the commodity ID ‘XXXXXX2’ sold in a whole unit.
- the commodity registered in a PLU file F 1 is hereinafter referred to as a registered commodity.
- a registered commodity represented by disparate sales information is referred to as a disparate sales commodity (commodity in disparate sales form).
- the feature amount of a commodity G previously extracted from the photographed image (e.g. commodity image) of each commodity G is registered in association with a corresponding commodity ID.
- the feature amount refers to information representing the feature of the commodity G, such as an external tint, a pattern, a concave-convex condition and a shape.
- the commodity ID of the commodity G sold in a sales form different from a standard sales form thereof registered is added to the same commodity sold in the standard sales form as disparate sales information.
- the present embodiment is not limited to this, and the commodity IDs of a registered commodity sold in one sales form (disparate sales commodity) and a registered same commodity sold in another sales form may be mutually added to one and another commodities as disparate sales information.
- the commodity ID of a commodity sold in a disparate sales form is used as disparate sales information.
- the present embodiment is not limited to this.
- a commodity name and the file name of a commodity image capable of specifying the commodity ID of a commodity sold in a disparate sales form may be used.
- the storage location (storage address in the PLU file F 1 ) of the commodity ID may also be used as disparate sales information.
- the commodity image of each registered commodity is stored in the PLU file F 1 .
- the commodity image of each registered commodity may also be stored in a data file different from the PLU file F 1 in association with a corresponding commodity ID.
- the feature amount of each commodity G is previously registered in the PLU file F 1 .
- the present embodiment is not limited to this, and the feature amount of each commodity may also be extracted from the image of the commodity by a feature amount extraction module 1613 described later.
- An image for presentation may be registered instead of a commodity image.
- the CPU 61 of the POS terminal 11 is connected with a communication interface 25 , which transmits data with the store computer SC, via an input-output circuit (not shown).
- the store computer SC is installed in the backyard of a store.
- the HDD (not shown) of the store computer SC stores the PLU file F 1 , which is distributed to the POS terminal 11 .
- connection interface 65 enables transmission and reception of data with the commodity reading apparatus 101 .
- the connection interface 65 is connected with the commodity reading apparatus 101 .
- a printer 66 performs printing on a receipt and the like.
- the POS terminal 11 enables the printer 66 to print content of one transaction on a receipt under the control of the CPU 61 .
- the commodity reading apparatus 101 comprises a commodity reading section 110 and a display-operation section 104 .
- the commodity reading section 110 is provided with a microcomputer 160 , which is formed with a CPU 161 and, a ROM 162 and a RAM 163 connected to the CPU 161 via a bus line.
- the ROM 162 stores programs executed by the CPU 161 .
- the CPU 161 is connected with the image capturing module 164 , a sound output module 165 and a connection interface 175 via various input/output circuits (not shown).
- the image capturing module 164 , the sound output module 165 and the connection interface 175 are controlled by the CPU 161 .
- the image capturing module 164 includes a color CCD sensor or a color CMOS sensor, and performs the image capturing through the reading window 103 , as an image capturing element. For example, the image capturing module 164 captures dynamic images at 30 fps. The frame images (captured images) sequentially captured by the image capturing module 164 at a given frame rate are stored in the RAM 163 .
- the sound output module 165 includes a voice circuit and loudspeaker that generate a preset warning sounds. Under the control of the CPU 161 , the sound output module 165 gives a notice by the warning sound.
- the display-operation section 104 comprises a touch panel 105 , a display device 106 , a keyboard 107 , a display for customer 109 and a connection interface 176 .
- the connection interface 175 of the commodity reading section 110 is connected with the connection interface 65 of the POS terminal to perform transmission/reception of data with the POS terminal 11 .
- the connection interface 175 is connected with the display-operation section 104 via a connection interface 176 , and the CPU 161 performs transmission/reception of data with the commodity reading section 110 and the display-operation section 104 via the connection interface 175 .
- FIG. 4 is a block diagram illustrating the functional structure of the POS terminal 11 and the commodity reading apparatus 101 .
- the CPU 161 of the commodity reading apparatus 101 functions as an image acquisition section 1611 , a commodity detection section 1612 , a feature amount extraction module 1613 , a similarity determination module 1614 , a commodity candidate presentation section 1615 , an input reception section 1616 and an information output module 1617 .
- the image acquisition section 1611 outputs an image capturing-on signal to the image capturing module 164 to start an image capturing operation.
- the image acquisition section 1611 sequentially acquires images that are captured by the image capturing module 164 and stored in the RAM 163 after the image capturing operation is started.
- the image acquisition section 1611 acquires the captured images in the order in which the images are stored in the RAM 163 .
- the commodity detection section 1612 detects whole or part of the outline of a commodity contained in the captured image acquired by the image acquisition section 1611 , using a well-known pattern matching technology. Next, the outline extracted from the captured image (frame image) of the last time is compared with that extracted from the frame image of this time to detect a changed part, that is, an area where a commodity G facing the reading window 103 is imaged.
- a skin color area is detected in the captured image. If the skin color area is detected, that is, the hand of a shop clerk imaged is detected, it is tried to extract the outline of a commodity G which may be held by the shop clerk by performing the above-described outline detection at an area nearby the color skin area. At this time, if a hand-shaped outline and the outline of another object nearby the hand-shaped outline are detected, the area of the commodity G imaged is detected according to the outline of the object.
- the feature amount extraction module 1613 extracts the state (external tint, pattern, concave-convex condition and shape) of the surface of the area, in which the commodity G is imaged, detected by the commodity detection section 1612 within the captured image acquired by the image acquisition section 1611 as feature amount of the commodity G (extraction module).
- the similarity determination module 1614 compares the feature amount extracted by the feature amount extraction module 1613 with the feature amount of each commodity registered in the PLU file F 1 of the POS terminal 11 to calculate the similarity degree indicating the relationship between the two feature amounts.
- the similarity determination module 1614 recognizes a registered commodity (commodity ID) having a similarity degree calculated greater than a given threshold as a candidate for the commodity G photographed by the image capturing module 164 (recognition module).
- the similarity degree is at least a value indicating how much similar the feature amount of the commodity G and the feature amount of each commodity registered in the PLU file F 1 are to one the other by comparing the two feature amounts.
- Concept of the similarity degree is not limited to the example above, and it may also be a value indicating the degree of the consistency of the two feature amounts, or a value indicating how much the feature amount of the commodity G and the feature amount of each commodity registered in the PLU file F 1 are mutually correlated.
- the recognition of an object contained in an image is referred to as a general object recognition, which is carried out by various recognition methods that are respectively illustrated in the following documents:
- the similarity degree may be calculated as an absolute evaluation or a relative evaluation. If the similarity degree is calculated as an absolute evaluation, the image of the photographed commodity G and each registered commodity are compared one to one, and the similarity degree derived from the result of this comparison can be directly adopted as it is. If the similarity degree is calculated as a relative evaluation, the similarity degree may be calculated so that the sum of the similarity degrees between the photographed commodity G and each registered commodity is 1.0 (100%)
- the commodity candidate presentation section 1615 displays, on the display device 106 , the information concerning the registered commodity recognized by the similarity determination module 1614 as a candidate of recognition commodity.
- the commodity candidate presentation section 1615 determines whether or not disparate sales information is added to a registered commodity recognized as a candidate (determination module). If disparate sales information is added to the registered commodity, the records of the commodity registered as disparate sales information and the registered commodity recognized as a candidate are read from the PLU file F 1 of the POS terminal 11 and displayed on the display device 106 . On the other hand, if no disparate sales information is registered on the registered commodity recognized as a candidate, the record of the registered commodity recognized is read from the PLU file F 1 of the POS terminal 11 and displayed on the display device 106 .
- FIG. 5 is a diagram illustrating an example of the display of a commodity candidate.
- a commodity candidate presentation area A 11 on the display screen of the display device 106 , the commodity images G 1 and G 2 and the commodity names those of contained in the records of the commodity candidates are displayed in descending order of similarity degrees of registered commodities.
- the commodity images G 1 and G 2 can be selected according to the touch-operation on the touch panel 105 .
- a selection button B 11 is arranged at the lower part of the commodity candidate presentation area A 11 to select a commodity from a commodity list, and the commodity selected from the commodity list is processed as a determined commodity described later.
- an area A 12 the image captured by the image capturing module 164 is displayed.
- disparate sales information is added to a registered commodity recognized by the similarity determination module 1614 as a candidate, that is, if there is a commodity sold in a disparate sales form in the same commodity as the candidate commodity, records of the candidate commodity and the commodity sold in a disparate sales form are separately displayed as commodity candidates.
- the operator can select the commodity G as a correct candidate from the candidate commodity and commodities sold in disparate sales forms, resulting in an improvement in convenience.
- Examples of the display of the commodity candidate are not limited to that in FIG. 5 .
- a registered commodity recognized as a candidate has disparate sales information
- information that prompts an operator to photograph a part of the commodity (e.g. cut surface) reflecting the sales form of the disparate sales commodity to correctly recognize the commodity may be displayed on the display device 106 (refer to M 1 shown in FIG. 6 ), as shown in FIG. 6 .
- the operator is prompted to photograph the cut surface, and thus the recognition rate of the commodity G can be improved.
- the feature amount extracted by the feature amount extraction module 1613 may be used to determine whether or not there is a commodity sold in a disparate sales form.
- the input reception section 1616 receives various input operations corresponding to the display of the display device 106 through the touch panel 105 or keyboard 107 .
- the input reception section 1616 receives the selection operation of any one of the commodity candidates displayed on the display device 106 .
- the input reception section 1616 receives the commodity candidate selected as a commodity corresponding to the commodity G (determined commodity) photographed by the image capturing module 164 . If a plurality of commodities G are detected by the commodity detection section 1612 , the input reception section 1616 may receive the selection operation of plural commodity candidates from the plurality of commodity candidates.
- the information output module 1617 outputs, information (e.g. commodity ID and commodity name) representing the commodity determined as described above to the POS terminal 11 through the connection interface 175 (output module).
- information e.g. commodity ID and commodity name
- the information output module 1617 may further output the number of sales amounts separately input from the touch panel 105 or keyboard 107 to the POS terminal 11 .
- a commodity ID read by the information output module 1617 from the PLU file F 1 may be notified, directly, or a commodity name capable of specifying a commodity ID and the file name of a commodity image may also be notified, or the storage location (storage address in the PLU file F 1 ) of the commodity ID may still be notified.
- the CPU 61 of the POS terminal 11 functions as a sales registration section 611 .
- the sales registration section 611 registers the sales of a commodity based on the commodity ID and the number of sales amounts of the commodity output from the information output module 1617 of the commodity reading apparatus 101 . Specifically, referring to the PLU file F 1 , the sales registration section 611 performs the sales registration by recording the notified commodity ID, and the commodity category, the commodity name, and the unit price corresponding to the commodity ID, together with the number of sales amounts in a sales master file.
- FIG. 7 is a flow chart illustrating the procedure of a commodity recognition processing carried out by the commodity reading apparatus 101 .
- the image acquisition section 1611 outputs an image capturing-on signal to the image capturing module 164 to start the image capturing of the image capturing module 164 (ACT S 11 ) when the processing is started in response to the start of the commodity registration by the POS terminal 11 .
- the image acquisition section 1611 sequentially acquires frame images that are captured by the image capturing module 164 and are stored in the RAM 163 (ACT S 12 ).
- the commodity detection section 1612 detects whole or part of a commodity G from the captured image acquired in the ACT S 12 (ACT S 13 ).
- the feature amount extraction module 1613 extracts the feature amount of the commodity G detected in the ACT S 13 from the captured image acquired in the ACT S 12 (ACT S 14 ).
- the similarity determination module 1614 calculates the similarity degree between the feature amount extracted in the ACT S 14 and the feature amount of each registered commodity in the PLU file F 1 of the POS terminal 11 (ACT S 15 ). The similarity determination module 1614 determines whether or not there is a registered commodity having the feature amount extracted in the ACT ST 14 higher than a threshold within the registered commodities the similarity degree of which is calculated in the ACT S 15 (ACT S 16 ).
- the feature amount extraction module 1613 recognizes the registered commodity as a candidate for the commodity G photographed by the image capturing module 164 , and then, Otherwise, the ACT S 12 is taken again.
- the commodity candidate presentation section 1615 determines, based on whether or not disparate sales information is added to the registered commodity recognized as a candidate in the ACT S 16 , whether or not a commodity sold in a disparate sales form different from the sales form of the registered commodity exists in the registered commodity (ACT S 17 ). If the feature amount extracted by the feature amount extraction module 1613 or the comparison result of the similarity determination module 1614 is used to determine whether or not there is a disparate sales commodity, the commodity candidate presentation section 1615 performs the determination operation in the ACT S 17 .
- the commodity candidate presentation section 1615 reads the records corresponding to the registered commodity recognized as a candidate in the ACT S 16 and corresponding to a registered commodity indicated by the disparate sales information added to the registration information of the registered commodity from the PLU file F 1 of the POS terminal 11 and makes the display device 106 display them as a commodity candidate (ACT S 18 ).
- the commodity candidate presentation section 1615 reads the record of the registered commodity recognized as a candidate in the ACT S 16 from the PLU file F 1 of the POS terminal 11 and makes the display device 106 display the registered commodity as a commodity candidate (ACT S 19 ). It should be noted that if there is no disparate sales commodity (NO in ACT S 17 ), the ACT S 19 is skipped over, and one commodity candidate having the highest similarity degree may be output.
- the input reception section 1616 determines whether or not the selection on a commodity candidate is received through the touch panel 105 or keyboard 107 (ACT S 20 ). If the selection operation is received (YES in ACT S 20 ), the input reception section 1616 receives the selected commodity candidate as a determined commodity corresponding to the commodity G photographed by the image capturing module 164 , and then, the ACT S 21 is taken. On the other hand, if no selection operation is received (NO in ACT S 20 ), the ACT S 12 is taken again.
- the information output module 1617 outputs information (commodity ID and the like) representing the determined commodity which is selected in the ACT S 23 to the POS terminal 11 through the connection interface 175 (S 21 ), and then the ACT S 22 is taken.
- the number of sales amounts is separately input through the touch panel 105 or the keyboard 107 , the number of sales amounts is also output to the POS terminal 11 along with the information indicating the determined commodity in the ACT S 21 .
- ‘1’ one may be output as a default value of the number of sales amounts.
- the CPU 161 determines whether or not there is a termination of service, e.g., notification for termination of the commodity registration, sent from the POS terminal 11 (ACT S 22 ). If the service is to be continued (NO in ACT S 22 ), the CPU 161 returns to execute the ACT S 12 to continue the processing. On the other hand, if the service is to be ended (YES in ACT S 22 ), the image acquisition section 1611 outputs an image capturing-off signal to the image capturing module 164 to end the image capturing of the image capturing module 164 (ACT S 23 ), then the processing is ended.
- a termination of service e.g., notification for termination of the commodity registration
- FIG. 8 is a flow chart illustrating the procedure of a sales registration processing carried out by the POS terminal 11 .
- the CPU 61 receives the commodity ID and the number of sales amounts of the determined commodity output in the ACT S 21 shown in FIG. 7 (ACT S 31 ). Then, the sales registration section 611 reads the commodity category and the unit price from the PLU file F 1 based on the commodity ID and the number of sales amounts received in the ACT S 31 , and registers the sales of the commodity G read by the commodity reading apparatus 101 in the master file (ACT S 32 ).
- the CPU 61 determines whether or not there is a termination of service, e.g., termination of sales registration, instructed through the keyboard (ACT S 33 ). If the service is to be continued (NO in ACT S 33 ), the CPU 61 returns to the ACT S 31 to continue the processing. If the service is to be ended (YES in ACT S 33 ), the CPU 61 ends the processing.
- a termination of service e.g., termination of sales registration
- a commodity G cut into halves is described as an example of processed commodity, however, the commodity G may also be cut into quarters (1 ⁇ 4).
- the POS terminal 11 is equipped with the PLU file F 1 .
- the commodity reading apparatus 101 may be equipped with the PLU file F 1 in a whole or a part thereof, instead.
- a commodity candidate is recognized by the commodity reading apparatus 101 .
- the POS terminal 11 is provided with the functional section of the commodity reading apparatus 101 and thus may perform the recognition processing.
- the commodity reading apparatus 101 functions as an image capturing device, and the POS terminal 11 recognizes and determines a commodity candidate based on a captured image (frame image) sent from the commodity reading apparatus 101 .
- a scanner device of a stationary type (commodity reading apparatus 101 ) is used.
- a so-called handy type scanner may be connected with the POS terminal 11 .
- the present invention is applied to the commodity reading apparatus 101 in a checkout system 1 consisting the POS terminal 11 and the commodity reading apparatus 101 .
- the present invention may also be applied to a single apparatus equipped with functions of the POS terminal 11 and the commodity reading apparatus 101 in one housing, or a checkout system, for example, in which the commodity reading apparatus 101 and the POS terminal 11 shown in FIG. 1 are connected to one another in a wired or wireless manner.
- a self-checkout apparatus hereinafter referred to as a self-checkout POS
- a self-checkout POS installed in a store or supermarket can be given.
- FIG. 9 is a perspective view of the external constitution of a self-checkout POS 200
- FIG. 10 is a block diagram illustrating the hardware constitution of the self-checkout POS 200 .
- same numerals are applied to constitutions similar to those shown in FIG. 1 and FIG. 2 and therefore explanations those of are not repeated.
- the main body 202 of the self-checkout POS 200 includes a display device 106 having a touch panel 105 on the surface thereof and a commodity reading unit 110 which reads a commodity image to recognize (detect) the category of the commodity.
- a liquid crystal display is used as the display device 106 , for example.
- the display device 106 displays a guidance screen for providing a guidance of the operation of the self-checkout POS 200 to the customer, various input screens, a registration screen for registering the commodity information read by the commodity reading unit 110 and a settlement screen for selecting method of payment on which the total amount of commodities, a deposit amount (provisional payment) and a change amount are displayed.
- the commodity reading unit 110 reads a commodity image, using an image capturing module 164 , which captures image of the commodity when the customer holds a code symbol affixed on the commodity over the reading window 103 of the commodity reading unit 110 .
- a first commodity receiving table 203 for placing unsettled commodities housed in a basket is provided at the right-hand side of the main body 202 .
- a second commodity receiving table 204 for placing settled commodities is provided at left-hand side of the main body 202 .
- a bag hook 205 for holding a bag into which the settled commodities are taken is arranged above the second commodity receiving table 204 and a temporary receiving table 206 for temporarily placing the settled commodities before taking the settled commodities into the bag is arranged at the left-hand side of the main body 202 .
- Weighing scales 207 and 208 are respectively provided to the first and second commodity receiving tables 203 and 204 to confirm weight of a commodity before and after the settlement operation whether or not the weight of the commodity is unchanged after the commodity is settled.
- a change device 201 for depositing and dispensing bill for settlement is arranged in the main body 202 of the self-checkout POS 200 .
- the self-checkout POS 200 functions as an information processing apparatus.
- a single apparatus having functions of the POS terminal 11 and the commodity reading apparatus 101 is not limited to the self-checkout POS 200 described above and may be constructed without the weighing scales 207 and 208 , which are installed in the self-checkout POS 200 .
- the programs executed by each apparatus are pre-compiled in the storage medium (ROM or memory unit) of the apparatus, however, the present invention is not limited to this, the programs may be stored as in a computer-readable recording medium such as CD-ROM, flexible disk (FD), CD-R, DVD (digital versatile disk) as an installable or executable file.
- the storage medium which is not limited to a medium independent from a computer or an assembled system, may include a storage medium which downloads the programs that are transmitted over an LAN or the Internet and then stored or temporarily stored.
- each apparatus may be stored in a computer connected with a network such as the
- Internet to be provided after being downloaded from the network or provided or issued by a network such as the Internet.
- the programs involved in the embodiment above may also be installed in a portable information terminal with a communication function, such as a mobile telephone, a smart phone, a PDA (Personal Digital Assistant).
- a communication function such as a mobile telephone, a smart phone, a PDA (Personal Digital Assistant).
Landscapes
- Business, Economics & Management (AREA)
- Engineering & Computer Science (AREA)
- Accounting & Taxation (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Strategic Management (AREA)
- General Business, Economics & Management (AREA)
- Finance (AREA)
- Theoretical Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Cash Registers Or Receiving Machines (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
An information processing apparatus includes: an acquisition module configured to acquire an image photographed by an image capturing module; an extraction module configured to extract, from the image, a feature amount representing an external feature of a commodity photographed; a recognition module configured to recognize a standard commodity that is a candidate for the commodity by comparing the feature amount extracted with a feature amount of each standard commodity; a determination module configured to determine whether there is any related standard commodity belonging to a same commodity item as the standard commodity recognized and having an external feature different from an external feature of the standard commodity recognized; and a display module configured to display the standard commodity recognized as the candidate for the commodity contained in the image, and display the related standard commodity as a candidate for the commodity if there is any related standard commodity.
Description
- This application is a Continuation of application Ser. No. 13/868,232 filed on Apr. 23, 2013, the entire contents of which are incorporated herein by reference.
- This application is based upon and claims the benefit of priority from the prior Japanese Patent Application No. 2012-196425, filed on Sep. 6, 2012, the entire contents of which are incorporated herein by reference.
- Embodiments described herein relate to an information processing apparatus and a commodity recognition method executed by the information processing apparatus.
- In the past, a technology is proposed in which the feature amount (e.g. shape) of a commodity is acquired from an image data of the commodity photographed and a candidate of the recognition commodity is determined based on the acquired feature amount.
- However, in a store such as a supermarket and the like, there is a case that a commodity, e.g., fresh vegetables or fruits, is sometimes sold in different sales forms, for example, such that the commodity is prepared in one bunch or block unit or a half thereof even those are the same commodities.
- In this case, however, if the commodity is photographed from a direction that the state of sales thereof is hardly distinguished such that an uncut surface of the commodity is photographed, there may be a possibility that the photographed commodity is not recognized as a commodity being cut in half, for example. The technology conventionally proposed remains to be improved.
-
FIG. 1 is a perspective view of the external construction of a checkout system according to an embodiment; -
FIG. 2 is a block diagram illustrating the hardware configuration of a POS terminal and a commodity reading apparatus shown inFIG. 1 ; -
FIG. 3 is a schematic diagram illustrating an example of the data configuration of a PLU file shown inFIG. 2 ; -
FIG. 4 is a block diagram illustrating the functional configuration of the POS terminal and the commodity reading apparatus shown inFIG. 1 ; -
FIG. 5 is a diagram illustrating an example of a commodity candidate displayed on the display device of the commodity reading apparatus; -
FIG. 6 is a diagram illustrating an example of a commodity candidate displayed on the display device of the commodity reading apparatus; -
FIG. 7 is a flow chart illustrating the procedure of a commodity recognition processing carried out by the commodity reading apparatus; -
FIG. 8 is a flow chart illustrating the procedure of a sales registration processing carried out by the POS terminal; -
FIG. 9 is a perspective view illustrating the configuration of a self-checkout POS according to an embodiment; -
FIG. 10 is a block diagram illustrating the hardware configuration of the self-checkout POS terminal shown inFIG. 9 . - In general, according to one embodiment, an information processing apparatus, comprising a processor configured to execute a program to implement elements, comprising an acquisition module configured to acquire an image photographed by an image capturing module; an extraction module configured to extract, from the photographed image, a feature amount representing an external feature of a commodity photographed by the image capturing module; a recognition module configured to recognize a standard commodity that is a candidate for the commodity by comparing the feature amount extracted by the extraction module with a feature amount of each standard commodity; a determination module configured to determine, for the standard commodity recognized by the recognition module, whether or not there is any related standard commodity belonging to a same commodity item as the standard commodity recognized and having an external feature different from an external feature of the standard commodity recognized; and a display module configured to display the standard commodity recognized by the recognition module as the candidate for the commodity contained in the photographed image, and display the related standard commodity as a candidate for the commodity if the determination module determines that there is any related standard commodity.
- Embodiments of the information processing apparatus and method thereof are described below with reference to the accompanying drawings. The embodiments are only illustrative of but are not to be construed as limiting the configuration and specification and the like of the information processing apparatus. The embodiments described herein are application examples of a checkout system, including a POS terminal and a commodity reading apparatus, which is installed in a supermarket or store. The POS terminal executes a registration and settlement of a commodity in one transaction and the commodity reading apparatus reads information relating to the commodity.
-
FIG. 1 is a perspective view of the external construction of a checkout system 1. As shown inFIG. 1 , the checkout system 1 comprises aPOS terminal 11 and acommodity reading apparatus 101 serving as an information processing apparatus. - The
POS terminal 11 is placed on the upper surface of adrawer 21 arranged on acheckout counter 51. Thedrawer 21 is opened under the control of thePOS terminal 11. Akeyboard 22 which is operated by an operator (shop clerk) is arranged on the upper surface of thePOS terminal 11. A display foroperator 23 for displaying information to the operator is arranged at a back side through thekeyboard 22 from the operator who operates thekeyboard 22. The display foroperator 23 displays information on thedisplay surface 23 a thereof. Atouch panel 26 is laminated on thedisplay surface 23 a. - A display for
customer 24 is arranged in a free rotatable manner at a further back side to the display foroperator 23. The display forcustomer 24 displays information on thedisplay surface 24 a thereof. - The
display surface 24 a of the display forcustomer 24 shown inFIG. 1 faces the operator, but it is rotated so that thedisplay surface 24 a faces a customer to display information to a customer. - A horizontally long table-
shaped counter 151 is arranged such that it is located in a right angle to thecheckout counter 51 on which thePOS terminal 11 is placed. Areceiving surface 152 is formed on the upper surface of thecounter 151. Ashopping basket 153 for accommodating a commodity G is placed on thereceiving surface 152. Theshopping basket 153 includes afirst shopping basket 153 a, placed on the counter at a side, which is held by a customer and asecond shopping basket 153 b which is placed on thecounter 51 at a side opposite to thefirst shopping basket 153 a through thecommodity reading apparatus 101. - The
commodity reading apparatus 101 connected with thePOS terminal 11 to perform transmission of data is arranged on thereceiving surface 152 of thecounter 151. Thecommodity reading apparatus 101 has a thin rectangularshaped housing 102. - A
reading window 103 is arranged at the front side of thehousing 102. Adisplay operation section 104 is arranged on the upper part of thehousing 102. Adisplay device 106 having atouch panel 105 laminated on the surface thereof is arranged on thedisplay operation section 104. Akeyboard 107 is arranged at the right-hand side of thedisplay device 106. Thecard reading slot 108 of a card reader (not shown) is arranged at the right-hand side of thekeyboard 107. A display forcustomer 109 is arranged at the left-hand side of thedisplay operation section 104 with respect to the operator to provide information to a customer. - Commodities G in one translation are housed in the
first shopping basket 153 a carried by the customer. The commodities G in thefirst shopping basket 153 a are moved one by one to thesecond shopping basket 153 b by the operator who operates thecommodity reading apparatus 101. While moving the commodity from thefirst shopping basket 153 a, the commodity G is exposed to thereading window 103 of thecommodity reading apparatus 101. At this time, the image capturing module 164 (refer toFIG. 2 ) arranged inside thereading window 103 photographs the commodity G. -
FIG. 2 is a block diagram illustrating the hardware configuration of the POS terminal and thecommodity reading apparatus 101. - The
POS terminal 11 is provided with amicrocomputer 60 functioning as an information processing unit for carrying out an information processing. Themicrocomputer 60 includes a CPU (Central Processing Unit) 61 for carrying out various operations to control each section, a ROM (Read Only Memory) 62 and a RAM (Random Access Memory) 63 connected via a bus line. - The
drawer 21, thekeyboard 22, thedisplay device 23, the display forcustomer 24, thecommunication interface 25 and thetouch panel 26 are connected with theCPU 61 of thePOS terminal 11 via various input/output circuits (not shown). - The
keyboard 22 includes anumeric keypad 22 d on which numeric characters such as ‘1’, ‘2’, ‘3’ and a multiplying operator such as ‘*’ are indicated, atemporary closing key 22 e and a closing key 22F. TheCPU 61 of thePOS terminal 11 is connected with an HDD (Hard Disk Drive) 64, in which various programs and files are stored. Programs and files stored in theHDD 64 are copied in all or in part onto the area of theRAM 63 and are executed by theCPU 61 when thePOS terminal 11 is activated. - The
HDD 64 stores a data file such as a PLU file F1 and the like. The PLU file F1 is readable by thecommodity reading apparatus 101 through aconnection interface 65. - The PLU file F1 is a data file in which a commodity G sold in a store is associated with information concerning the sales registration of the commodity G.
-
FIG. 3 is a schematic diagram illustrating an example of the data configuration of the PLU file F1. As shown inFIG. 3 , for each commodity G, a unique commodity ID is registered in association with information relating to a commodity (such as a commodity category to which a commodity belongs, a commodity name, a unit price, etc.), an image of the commodity photographed and a disparate sales information showing whether or not there is other commodity sold in a different form for the same commodity. In addition, the feature amount (feature amount data of a standard commodity) of each commodity G is registered (stored) in advance in the PLU file F1 in an associated manner. - The commodity image is a standard image (reference image) obtained by photographing each commodity G as a standard commodity. The commodity image is showed as a commodity candidate described later when the commodity candidate is presented. It is preferable to show, as a commodity image, an image illustrating the sales form of the commodity, for example, for a commodity sold in a form in which it is cut, and the commodity image that the cut surface of the commodity is photographed is used.
- If a commodity G is sold in a plurality of sales forms in the same commodity G, the commodity ID of the commodity G sold in a different sales form to the same commodity G sold in a standard sales form is added to the commodity G sold in the standard sales form as disparate sales information. For example, in
FIG. 3 , for a commodity ‘chinese cabbage’, the commodity ID “XXXXXXX3’ of the commodity ‘chinese cabbage’ sold in a form in which it is cut in half is registered as a disparate sales information to the commodity ‘chinese cabbage’ of the commodity ID ‘XXXXXXX2’ sold in a whole unit. The commodity registered in a PLU file F1 is hereinafter referred to as a registered commodity. A registered commodity represented by disparate sales information is referred to as a disparate sales commodity (commodity in disparate sales form). - Further, the feature amount of a commodity G previously extracted from the photographed image (e.g. commodity image) of each commodity G is registered in association with a corresponding commodity ID. The feature amount refers to information representing the feature of the commodity G, such as an external tint, a pattern, a concave-convex condition and a shape.
- According to the present embodiment, in a commodity G sold in a plurality of different forms, the commodity ID of the commodity G sold in a sales form different from a standard sales form thereof registered is added to the same commodity sold in the standard sales form as disparate sales information. However, the present embodiment is not limited to this, and the commodity IDs of a registered commodity sold in one sales form (disparate sales commodity) and a registered same commodity sold in another sales form may be mutually added to one and another commodities as disparate sales information.
- Further, in the present embodiment, the commodity ID of a commodity sold in a disparate sales form is used as disparate sales information. However, the present embodiment is not limited to this. A commodity name and the file name of a commodity image capable of specifying the commodity ID of a commodity sold in a disparate sales form may be used. In addition to this, the storage location (storage address in the PLU file F1) of the commodity ID may also be used as disparate sales information.
- Still further, in the present embodiment, the commodity image of each registered commodity is stored in the PLU file F1. However, the present embodiment is not limited to this. The commodity image of each registered commodity may also be stored in a data file different from the PLU file F1 in association with a corresponding commodity ID. Further, in the present embodiment, the feature amount of each commodity G is previously registered in the PLU file F1. However, the present embodiment is not limited to this, and the feature amount of each commodity may also be extracted from the image of the commodity by a feature
amount extraction module 1613 described later. An image for presentation may be registered instead of a commodity image. - Return to
FIG. 2 , theCPU 61 of thePOS terminal 11 is connected with acommunication interface 25, which transmits data with the store computer SC, via an input-output circuit (not shown). The store computer SC is installed in the backyard of a store. The HDD (not shown) of the store computer SC stores the PLU file F1, which is distributed to thePOS terminal 11. - The
connection interface 65 enables transmission and reception of data with thecommodity reading apparatus 101. Theconnection interface 65 is connected with thecommodity reading apparatus 101. Aprinter 66 performs printing on a receipt and the like. ThePOS terminal 11 enables theprinter 66 to print content of one transaction on a receipt under the control of theCPU 61. - The
commodity reading apparatus 101 comprises acommodity reading section 110 and a display-operation section 104. Thecommodity reading section 110 is provided with amicrocomputer 160, which is formed with aCPU 161 and, aROM 162 and aRAM 163 connected to theCPU 161 via a bus line. TheROM 162 stores programs executed by theCPU 161. - The
CPU 161 is connected with theimage capturing module 164, asound output module 165 and aconnection interface 175 via various input/output circuits (not shown). Theimage capturing module 164, thesound output module 165 and theconnection interface 175 are controlled by theCPU 161. - The
image capturing module 164 includes a color CCD sensor or a color CMOS sensor, and performs the image capturing through the readingwindow 103, as an image capturing element. For example, theimage capturing module 164 captures dynamic images at 30 fps. The frame images (captured images) sequentially captured by theimage capturing module 164 at a given frame rate are stored in theRAM 163. Thesound output module 165 includes a voice circuit and loudspeaker that generate a preset warning sounds. Under the control of theCPU 161, thesound output module 165 gives a notice by the warning sound. - The display-
operation section 104 comprises atouch panel 105, adisplay device 106, akeyboard 107, a display forcustomer 109 and aconnection interface 176. Theconnection interface 175 of thecommodity reading section 110 is connected with theconnection interface 65 of the POS terminal to perform transmission/reception of data with thePOS terminal 11. Theconnection interface 175 is connected with the display-operation section 104 via aconnection interface 176, and theCPU 161 performs transmission/reception of data with thecommodity reading section 110 and the display-operation section 104 via theconnection interface 175. - Next, the functional constitution of the
CPU 161 and theCPU 61 achieved with the program executed by theCPU 161 and theCPU 61 are described below with reference toFIG. 4 . -
FIG. 4 is a block diagram illustrating the functional structure of thePOS terminal 11 and thecommodity reading apparatus 101. As shown inFIG. 4 , by executing programs sequentially, theCPU 161 of thecommodity reading apparatus 101 functions as animage acquisition section 1611, acommodity detection section 1612, a featureamount extraction module 1613, asimilarity determination module 1614, a commoditycandidate presentation section 1615, aninput reception section 1616 and aninformation output module 1617. - The
image acquisition section 1611 outputs an image capturing-on signal to theimage capturing module 164 to start an image capturing operation. Theimage acquisition section 1611 sequentially acquires images that are captured by theimage capturing module 164 and stored in theRAM 163 after the image capturing operation is started. Theimage acquisition section 1611 acquires the captured images in the order in which the images are stored in theRAM 163. - The
commodity detection section 1612 detects whole or part of the outline of a commodity contained in the captured image acquired by theimage acquisition section 1611, using a well-known pattern matching technology. Next, the outline extracted from the captured image (frame image) of the last time is compared with that extracted from the frame image of this time to detect a changed part, that is, an area where a commodity G facing the readingwindow 103 is imaged. - Following is one of other methods for detecting a commodity G. It is detected whether or not a skin color area is detected in the captured image. If the skin color area is detected, that is, the hand of a shop clerk imaged is detected, it is tried to extract the outline of a commodity G which may be held by the shop clerk by performing the above-described outline detection at an area nearby the color skin area. At this time, if a hand-shaped outline and the outline of another object nearby the hand-shaped outline are detected, the area of the commodity G imaged is detected according to the outline of the object.
- The feature
amount extraction module 1613 extracts the state (external tint, pattern, concave-convex condition and shape) of the surface of the area, in which the commodity G is imaged, detected by thecommodity detection section 1612 within the captured image acquired by theimage acquisition section 1611 as feature amount of the commodity G (extraction module). - The
similarity determination module 1614 compares the feature amount extracted by the featureamount extraction module 1613 with the feature amount of each commodity registered in the PLU file F1 of thePOS terminal 11 to calculate the similarity degree indicating the relationship between the two feature amounts. Thesimilarity determination module 1614 recognizes a registered commodity (commodity ID) having a similarity degree calculated greater than a given threshold as a candidate for the commodity G photographed by the image capturing module 164 (recognition module). The similarity degree is at least a value indicating how much similar the feature amount of the commodity G and the feature amount of each commodity registered in the PLU file F1 are to one the other by comparing the two feature amounts. Concept of the similarity degree is not limited to the example above, and it may also be a value indicating the degree of the consistency of the two feature amounts, or a value indicating how much the feature amount of the commodity G and the feature amount of each commodity registered in the PLU file F1 are mutually correlated. - The recognition of an object contained in an image is referred to as a general object recognition, which is carried out by various recognition methods that are respectively illustrated in the following documents:
- YANAI Keiji, ‘The current state and further directions on General Object Recognition’, in Proceedings of Information Processing Society of Japan, Vol. 48, No SIG 16, In URL:http://mm.cs.uec.ac.jp/IPSJ-TCVIM-Yanai.pdf [retrieved on Jul. 26, 2012].
- In addition, the technology for recognizing a general object by partitioning an image into areas for each article (object) is described in the following document:
- Jamie Shotton: ‘Semantic Texton Forests for Image Categorization and Segmentation, In URL:http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.1 45.3036&rep=rep1&type=pdf (retrieved on Jul. 26, 2012).
- It does not matter what type of calculation method of the similarity degree. For example, the similarity degree may be calculated as an absolute evaluation or a relative evaluation. If the similarity degree is calculated as an absolute evaluation, the image of the photographed commodity G and each registered commodity are compared one to one, and the similarity degree derived from the result of this comparison can be directly adopted as it is. If the similarity degree is calculated as a relative evaluation, the similarity degree may be calculated so that the sum of the similarity degrees between the photographed commodity G and each registered commodity is 1.0 (100%)
- The commodity
candidate presentation section 1615 displays, on thedisplay device 106, the information concerning the registered commodity recognized by thesimilarity determination module 1614 as a candidate of recognition commodity. - Specifically, the commodity
candidate presentation section 1615 determines whether or not disparate sales information is added to a registered commodity recognized as a candidate (determination module). If disparate sales information is added to the registered commodity, the records of the commodity registered as disparate sales information and the registered commodity recognized as a candidate are read from the PLU file F1 of thePOS terminal 11 and displayed on thedisplay device 106. On the other hand, if no disparate sales information is registered on the registered commodity recognized as a candidate, the record of the registered commodity recognized is read from the PLU file F1 of thePOS terminal 11 and displayed on thedisplay device 106. -
FIG. 5 is a diagram illustrating an example of the display of a commodity candidate. As shown inFIG. 5 , in a commodity candidate presentation area A11 on the display screen of thedisplay device 106, the commodity images G1 and G2 and the commodity names those of contained in the records of the commodity candidates are displayed in descending order of similarity degrees of registered commodities. The commodity images G1 and G2 can be selected according to the touch-operation on thetouch panel 105. A selection button B11 is arranged at the lower part of the commodity candidate presentation area A11 to select a commodity from a commodity list, and the commodity selected from the commodity list is processed as a determined commodity described later. In an area A12, the image captured by theimage capturing module 164 is displayed. - Depending on the photographing direction of the
image capturing module 164 to a commodity G, there may be a case in which it is difficult to recognize the sales form of the commodity G according to the captured image. For example, as shown inFIG. 5 , if the leaf surface (uncut surface) of a chinese cabbage is photographed, it is difficult to determine whether or not the commodity is sold in the whole unit or in a cut unit (for example, cut into halves) according to the captured image. In this case, even if a commodity sold in a half unit faces theimage capturing module 164, the commodity is still recognized as being sold in the whole unit. Therefore, there is a problem that the recognition rate decreases. - According to the present embodiment, if disparate sales information is added to a registered commodity recognized by the
similarity determination module 1614 as a candidate, that is, if there is a commodity sold in a disparate sales form in the same commodity as the candidate commodity, records of the candidate commodity and the commodity sold in a disparate sales form are separately displayed as commodity candidates. As a result, even if a commodity G is photographed in a direction from which the sales form of the commodity G is difficult to recognize, the operator can select the commodity G as a correct candidate from the candidate commodity and commodities sold in disparate sales forms, resulting in an improvement in convenience. - Examples of the display of the commodity candidate are not limited to that in
FIG. 5 . For example, if a registered commodity recognized as a candidate has disparate sales information, information that prompts an operator to photograph a part of the commodity (e.g. cut surface) reflecting the sales form of the disparate sales commodity to correctly recognize the commodity may be displayed on the display device 106 (refer to M1 shown inFIG. 6 ), as shown inFIG. 6 . As a result, even if an uncut surface of the commodity is photographed in the commodity sold in a half unit, the operator is prompted to photograph the cut surface, and thus the recognition rate of the commodity G can be improved. - In the present embodiment, according to the presence of disparate sales information, it is determined whether or not a disparate sales commodity exists in the registered commodity recognized as a candidate. However, it is not limited to this. For example, the feature amount extracted by the feature
amount extraction module 1613 may be used to determine whether or not there is a commodity sold in a disparate sales form. - For example, it is a common way that a commodity (vegetable or fruit) cut into halves is wrapped with a plastic film. Thus, if an information or data indicating the existence of the wrapping material added over the commodity G as a feature amount of the commodity G (e.g. a reverse portion of image caused by the light reflected from a wrapping material) is obtained, then it is determined that a disparate sales commodity exists in the registered commodity which becomes a candidate for the commodity G, and the registered commodity (candidate of the commodity G) can be presented together with the same commodity sold in a disparate sales form indicated with a disparate sales information. Further, if no feature amount representing the existence of the wrapping material is acquired, even if the registered commodity that becomes a candidate is added with disparate sales information, the disparate sales commodity represented by the disparate sales information is not presented.
- Returning to
FIG. 4 , theinput reception section 1616 receives various input operations corresponding to the display of thedisplay device 106 through thetouch panel 105 orkeyboard 107. For example, theinput reception section 1616 receives the selection operation of any one of the commodity candidates displayed on thedisplay device 106. Theinput reception section 1616 receives the commodity candidate selected as a commodity corresponding to the commodity G (determined commodity) photographed by theimage capturing module 164. If a plurality of commodities G are detected by thecommodity detection section 1612, theinput reception section 1616 may receive the selection operation of plural commodity candidates from the plurality of commodity candidates. - The
information output module 1617 outputs, information (e.g. commodity ID and commodity name) representing the commodity determined as described above to thePOS terminal 11 through the connection interface 175 (output module). - The
information output module 1617 may further output the number of sales amounts separately input from thetouch panel 105 orkeyboard 107 to thePOS terminal 11. As information output from theinformation output module 1617 to thePOS terminal 11, a commodity ID read by theinformation output module 1617 from the PLU file F1 may be notified, directly, or a commodity name capable of specifying a commodity ID and the file name of a commodity image may also be notified, or the storage location (storage address in the PLU file F1) of the commodity ID may still be notified. - On the other hand, by executing a program, the
CPU 61 of thePOS terminal 11 functions as asales registration section 611. Thesales registration section 611 registers the sales of a commodity based on the commodity ID and the number of sales amounts of the commodity output from theinformation output module 1617 of thecommodity reading apparatus 101. Specifically, referring to the PLU file F1, thesales registration section 611 performs the sales registration by recording the notified commodity ID, and the commodity category, the commodity name, and the unit price corresponding to the commodity ID, together with the number of sales amounts in a sales master file. - Next, operation of the checkout system 1 is described. The operation of the
commodity reading apparatus 101 is described first.FIG. 7 is a flow chart illustrating the procedure of a commodity recognition processing carried out by thecommodity reading apparatus 101. - As shown in
FIG. 7 , theimage acquisition section 1611 outputs an image capturing-on signal to theimage capturing module 164 to start the image capturing of the image capturing module 164 (ACT S11) when the processing is started in response to the start of the commodity registration by thePOS terminal 11. - The
image acquisition section 1611 sequentially acquires frame images that are captured by theimage capturing module 164 and are stored in the RAM 163 (ACT S12). Next, thecommodity detection section 1612 detects whole or part of a commodity G from the captured image acquired in the ACT S12 (ACT S13). The featureamount extraction module 1613 extracts the feature amount of the commodity G detected in the ACT S13 from the captured image acquired in the ACT S12 (ACT S14). - The
similarity determination module 1614 calculates the similarity degree between the feature amount extracted in the ACT S14 and the feature amount of each registered commodity in the PLU file F1 of the POS terminal 11 (ACT S15). Thesimilarity determination module 1614 determines whether or not there is a registered commodity having the feature amount extracted in the ACT ST14 higher than a threshold within the registered commodities the similarity degree of which is calculated in the ACT S15 (ACT S16). - If it is determined that there is a registered commodity the similarity degree of which is higher than the threshold (YES in ACT S16), the feature
amount extraction module 1613 recognizes the registered commodity as a candidate for the commodity G photographed by theimage capturing module 164, and then, Otherwise, the ACT S12 is taken again. - Sequentially, in the ACT S17, the commodity
candidate presentation section 1615 determines, based on whether or not disparate sales information is added to the registered commodity recognized as a candidate in the ACT S16, whether or not a commodity sold in a disparate sales form different from the sales form of the registered commodity exists in the registered commodity (ACT S17). If the feature amount extracted by the featureamount extraction module 1613 or the comparison result of thesimilarity determination module 1614 is used to determine whether or not there is a disparate sales commodity, the commoditycandidate presentation section 1615 performs the determination operation in the ACT S17. - If it is determined that there is a disparate sales commodity (YES in ACT S17), the commodity
candidate presentation section 1615 reads the records corresponding to the registered commodity recognized as a candidate in the ACT S16 and corresponding to a registered commodity indicated by the disparate sales information added to the registration information of the registered commodity from the PLU file F1 of thePOS terminal 11 and makes thedisplay device 106 display them as a commodity candidate (ACT S18). On the other hand, if it is determined that there is no disparate sales commodity (NO in ACT S17), the commoditycandidate presentation section 1615 reads the record of the registered commodity recognized as a candidate in the ACT S16 from the PLU file F1 of thePOS terminal 11 and makes thedisplay device 106 display the registered commodity as a commodity candidate (ACT S19). It should be noted that if there is no disparate sales commodity (NO in ACT S17), the ACT S19 is skipped over, and one commodity candidate having the highest similarity degree may be output. - Then, the
input reception section 1616 determines whether or not the selection on a commodity candidate is received through thetouch panel 105 or keyboard 107 (ACT S20). If the selection operation is received (YES in ACT S20), theinput reception section 1616 receives the selected commodity candidate as a determined commodity corresponding to the commodity G photographed by theimage capturing module 164, and then, the ACT S21 is taken. On the other hand, if no selection operation is received (NO in ACT S20), the ACT S12 is taken again. - Next, the
information output module 1617 outputs information (commodity ID and the like) representing the determined commodity which is selected in the ACT S23 to thePOS terminal 11 through the connection interface 175 (S21), and then the ACT S22 is taken. - If the number of sales amounts is separately input through the
touch panel 105 or thekeyboard 107, the number of sales amounts is also output to thePOS terminal 11 along with the information indicating the determined commodity in the ACT S21. In addition, if the number of sales amounts is not input, ‘1’ (one) may be output as a default value of the number of sales amounts. - Then, the
CPU 161 determines whether or not there is a termination of service, e.g., notification for termination of the commodity registration, sent from the POS terminal 11 (ACT S22). If the service is to be continued (NO in ACT S22), theCPU 161 returns to execute the ACT S12 to continue the processing. On the other hand, if the service is to be ended (YES in ACT S22), theimage acquisition section 1611 outputs an image capturing-off signal to theimage capturing module 164 to end the image capturing of the image capturing module 164 (ACT S23), then the processing is ended. - Next, the processing operation carried out by the
POS terminal 11 is described.FIG. 8 is a flow chart illustrating the procedure of a sales registration processing carried out by thePOS terminal 11. - First, when a commodity registration processing is started according to an operation instruction from the
keyboard 22, theCPU 61 receives the commodity ID and the number of sales amounts of the determined commodity output in the ACT S21 shown inFIG. 7 (ACT S31). Then, thesales registration section 611 reads the commodity category and the unit price from the PLU file F1 based on the commodity ID and the number of sales amounts received in the ACT S31, and registers the sales of the commodity G read by thecommodity reading apparatus 101 in the master file (ACT S32). - Sequentially, the
CPU 61 determines whether or not there is a termination of service, e.g., termination of sales registration, instructed through the keyboard (ACT S33). If the service is to be continued (NO in ACT S33), theCPU 61 returns to the ACT S31 to continue the processing. If the service is to be ended (YES in ACT S33), theCPU 61 ends the processing. - As stated above, in accordance with the present embodiment, if there is a commodity sold in a disparate sales form different from the sales form of the registered commodity that is determined as a candidate of the commodity G, information concerning the disparate sales commodity and information concerning the registered commodity are presented together as a selectable commodity candidate. Thus, even if the commodity G is sold in a whole unit and in a cut state (e.g. cut into halves), the selection can also be made on the commodity G cut into halves as a commodity candidate when the commodity is recognized as a commodity sold in a whole unit. Therefore, operability of the user can be enhanced and recognition on a disparate sales commodity can also be effectively executed. In the present embodiment, a commodity G cut into halves is described as an example of processed commodity, however, the commodity G may also be cut into quarters (¼).
- While certain embodiments have been described, these embodiments have been presented by way of example only, and are not intended to limit the scope of the present invention. Indeed, the novel embodiments may be embodied in a variety of other forms; furthermore, various omissions, substitutions and changes in the form of the embodiments may be made without departing from the spirit of the present invention. The accompanying claims and their equivalents are intended to cover such forms or modifications as would fall within the scope and spirit of the present invention.
- For example, in the embodiments described herein, the
POS terminal 11 is equipped with the PLU file F1. However, it is not limited to this, and thecommodity reading apparatus 101 may be equipped with the PLU file F1 in a whole or a part thereof, instead. - Further, in the embodiments described herein, a commodity candidate is recognized by the
commodity reading apparatus 101. However, thePOS terminal 11 is provided with the functional section of thecommodity reading apparatus 101 and thus may perform the recognition processing. In this case, thecommodity reading apparatus 101 functions as an image capturing device, and thePOS terminal 11 recognizes and determines a commodity candidate based on a captured image (frame image) sent from thecommodity reading apparatus 101. - Further, in the embodiments described herein, a scanner device of a stationary type (commodity reading apparatus 101) is used. However, it is not limited to this, and a so-called handy type scanner may be connected with the
POS terminal 11. - Further, in the embodiments described above, the present invention is applied to the
commodity reading apparatus 101 in a checkout system 1 consisting thePOS terminal 11 and thecommodity reading apparatus 101. However, it is not limited to this, and the present invention may also be applied to a single apparatus equipped with functions of thePOS terminal 11 and thecommodity reading apparatus 101 in one housing, or a checkout system, for example, in which thecommodity reading apparatus 101 and thePOS terminal 11 shown inFIG. 1 are connected to one another in a wired or wireless manner. As a single apparatus equipped with functions of thePOS terminal 11 and thecommodity reading apparatus 101, a self-checkout apparatus (hereinafter referred to as a self-checkout POS) installed in a store or supermarket can be given. -
FIG. 9 is a perspective view of the external constitution of a self-checkout POS 200, andFIG. 10 is a block diagram illustrating the hardware constitution of the self-checkout POS 200. InFIGS. 9 and 10 , same numerals are applied to constitutions similar to those shown inFIG. 1 andFIG. 2 and therefore explanations those of are not repeated. - As shown in
FIG. 9 andFIG. 10 , themain body 202 of the self-checkout POS 200 includes adisplay device 106 having atouch panel 105 on the surface thereof and acommodity reading unit 110 which reads a commodity image to recognize (detect) the category of the commodity. - A liquid crystal display is used as the
display device 106, for example. Thedisplay device 106 displays a guidance screen for providing a guidance of the operation of the self-checkout POS 200 to the customer, various input screens, a registration screen for registering the commodity information read by thecommodity reading unit 110 and a settlement screen for selecting method of payment on which the total amount of commodities, a deposit amount (provisional payment) and a change amount are displayed. - The
commodity reading unit 110 reads a commodity image, using animage capturing module 164, which captures image of the commodity when the customer holds a code symbol affixed on the commodity over the readingwindow 103 of thecommodity reading unit 110. - A first commodity receiving table 203 for placing unsettled commodities housed in a basket is provided at the right-hand side of the
main body 202. A second commodity receiving table 204 for placing settled commodities is provided at left-hand side of themain body 202. Abag hook 205 for holding a bag into which the settled commodities are taken is arranged above the second commodity receiving table 204 and a temporary receiving table 206 for temporarily placing the settled commodities before taking the settled commodities into the bag is arranged at the left-hand side of themain body 202. Weighingscales - A
change device 201 for depositing and dispensing bill for settlement is arranged in themain body 202 of the self-checkout POS 200. - In the case in which the present invention is applied to the self-
checkout POS 200 having the constitution described above, the self-checkout POS 200 functions as an information processing apparatus. A single apparatus having functions of thePOS terminal 11 and thecommodity reading apparatus 101 is not limited to the self-checkout POS 200 described above and may be constructed without the weighingscales checkout POS 200. - Further, in the embodiments above, the programs executed by each apparatus are pre-compiled in the storage medium (ROM or memory unit) of the apparatus, however, the present invention is not limited to this, the programs may be stored as in a computer-readable recording medium such as CD-ROM, flexible disk (FD), CD-R, DVD (digital versatile disk) as an installable or executable file. Moreover, the storage medium, which is not limited to a medium independent from a computer or an assembled system, may include a storage medium which downloads the programs that are transmitted over an LAN or the Internet and then stored or temporarily stored.
- In addition, the programs executed by each apparatus may be stored in a computer connected with a network such as the
- Internet to be provided after being downloaded from the network or provided or issued by a network such as the Internet.
- Further, the programs involved in the embodiment above may also be installed in a portable information terminal with a communication function, such as a mobile telephone, a smart phone, a PDA (Personal Digital Assistant).
- While certain embodiments have been described, these embodiments have been presented by way of example only, and are not intended to limit the scope of the invention. Indeed, the novel methods and systems described herein may be embodied in a variety of other forms; furthermore, various omissions, substitutions and changes in the form of the methods and systems described herein may be made without departing from the spirit of the invention. The accompanying claims and their equivalents are intended to cover such forms or modifications as would fall within the scope and spirit of the invention.
Claims (8)
1. An information processing apparatus, comprising:
a processor configured to execute a program to implement elements, comprising:
an acquisition module configured to acquire an image photographed by an image capturing module;
an extraction module configured to extract, from the photographed image, a feature amount representing an external feature of a commodity photographed by the image capturing module;
a recognition module configured to recognize a standard commodity that is a candidate for the commodity by comparing the feature amount extracted by the extraction module with a feature amount of each standard commodity;
a determination module configured to determine, for the standard commodity recognized by the recognition module, whether or not there is any related standard commodity belonging to a same commodity item as the standard commodity recognized and having an external feature different from an external feature of the standard commodity recognized; and
a display module configured to display the standard commodity recognized by the recognition module as the candidate for the commodity contained in the photographed image, and display the related standard commodity as a candidate for the commodity if the determination module determines that there is any related standard commodity.
2. The information processing apparatus according to claim 1 , wherein
the determination module determines that there is any related standard commodity if, based on data concerning a plurality of standard commodities belonging to a same commodity item in which information for recognizing one standard commodity having a standard external feature is associated with information for recognizing another standard commodity having an external feature different from the standard external feature, the recognition module recognizes the one standard commodity having the standard external feature, and
the display module displays the standard commodity recognized by the recognition module and the related standard commodity that is associated with the standard commodity recognized as candidates for the commodity if the determination module determines that there is any related standard commodity.
3. The information processing apparatus according to claim 1 , wherein the related standard commodity is a part of the standard commodity recognized, which belongs to the same commodity item as the related standard commodity, cut therefrom.
4. The information processing apparatus according to claim 1 , wherein the determination module determines that there is any related standard commodity if the feature amount extracted by the extraction module includes information showing existence of a specific addition added to the commodity.
5. The information processing apparatus according to claim 1 , wherein the display module displays information which prompts a person to photograph a part of the commodity capable of recognizing a sales form of the related standard commodity if the determination module determines that there is any related standard commodity.
6. The information processing apparatus according to claim 1 , wherein the display module displays the image photographed by the image capturing module together with the candidate for the commodity.
7. The information processing apparatus according to claim 1 , comprising a reception module configured to receive a selection of any one candidate of the commodity displayed by the display module.
8. An information processing method, comprising:
acquiring, by a processor, an image photographed by an image capturing module;
extracting, by the processor, from the photographed image, a feature amount representing an external feature of a commodity photographed by the image capturing module;
recognizing, by the processor, a standard commodity that is a candidate for the commodity by comparing the feature amount extracted with a feature amount of each standard commodity;
determining, by the processor, for the standard commodity recognized, whether or not there is any related standard commodity belonging to a same commodity item as the standard commodity recognized and having an external feature different from an external feature of the standard commodity recognized; and
displaying, by the processor, the standard commodity recognized as the candidate for the commodity contained in the photographed image, and displaying the related standard commodity as a candidate for the commodity if it is determined that there is any related standard commodity.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/242,787 US20160358150A1 (en) | 2012-09-06 | 2016-08-22 | Information processing apparatus and commodity recognition method by the same |
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2012-196425 | 2012-09-06 | ||
JP2012196425A JP5612645B2 (en) | 2012-09-06 | 2012-09-06 | Information processing apparatus and program |
US13/868,232 US20140067573A1 (en) | 2012-09-06 | 2013-04-23 | Information processing apparatus and commodity recognition method by the same |
US15/242,787 US20160358150A1 (en) | 2012-09-06 | 2016-08-22 | Information processing apparatus and commodity recognition method by the same |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/868,232 Continuation US20140067573A1 (en) | 2012-09-06 | 2013-04-23 | Information processing apparatus and commodity recognition method by the same |
Publications (1)
Publication Number | Publication Date |
---|---|
US20160358150A1 true US20160358150A1 (en) | 2016-12-08 |
Family
ID=50188786
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/868,232 Abandoned US20140067573A1 (en) | 2012-09-06 | 2013-04-23 | Information processing apparatus and commodity recognition method by the same |
US15/242,787 Abandoned US20160358150A1 (en) | 2012-09-06 | 2016-08-22 | Information processing apparatus and commodity recognition method by the same |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/868,232 Abandoned US20140067573A1 (en) | 2012-09-06 | 2013-04-23 | Information processing apparatus and commodity recognition method by the same |
Country Status (3)
Country | Link |
---|---|
US (2) | US20140067573A1 (en) |
JP (1) | JP5612645B2 (en) |
CN (2) | CN106373298B (en) |
Families Citing this family (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPWO2015145977A1 (en) * | 2014-03-27 | 2017-04-13 | 日本電気株式会社 | Information processing apparatus, image processing method and program, and POS terminal apparatus |
JP6197952B2 (en) * | 2014-05-12 | 2017-09-20 | 富士通株式会社 | Product information output method, product information output program and control device |
JP6443858B2 (en) * | 2014-11-20 | 2018-12-26 | インターナショナル・ビジネス・マシーンズ・コーポレーションInternational Business Machines Corporation | Calculation device, calculation method, learning device, learning method, and program |
JP6295944B2 (en) * | 2014-12-18 | 2018-03-20 | カシオ計算機株式会社 | Product identification device and product recognition navigation method |
WO2016147612A1 (en) * | 2015-03-16 | 2016-09-22 | 日本電気株式会社 | Image recognition device, system, image recognition method, and recording medium |
JP6675915B2 (en) * | 2016-04-15 | 2020-04-08 | 東芝テック株式会社 | Information processing device and program |
JP2017211880A (en) * | 2016-05-26 | 2017-11-30 | 東芝テック株式会社 | Information processing apparatus and program |
JP2018032332A (en) * | 2016-08-26 | 2018-03-01 | 東芝テック株式会社 | Information processor and program |
JP6886906B2 (en) * | 2017-10-10 | 2021-06-16 | 東芝テック株式会社 | Readers and programs |
JP6547856B2 (en) * | 2018-01-09 | 2019-07-24 | カシオ計算機株式会社 | Information display device, guidance display method and program |
JP2019208482A (en) * | 2018-06-08 | 2019-12-12 | シャープ株式会社 | Controller, animal toilet, information processing unit, information processing terminal, control program and control method |
Citations (18)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6434257B1 (en) * | 1995-02-17 | 2002-08-13 | International Business Machines Corporation | Size recognition system with method for determining price of a commodity |
US20020123932A1 (en) * | 2001-02-28 | 2002-09-05 | Elliot Brenhouse | Self-checkout system with anti-theft deactivation device |
JP2002312651A (en) * | 2001-04-09 | 2002-10-25 | Fuji Photo Film Co Ltd | Inventory management system |
US20030128876A1 (en) * | 2001-12-13 | 2003-07-10 | Kabushiki Kaisha Toshiba | Pattern recognition apparatus and method therefor |
US6621921B1 (en) * | 1995-12-19 | 2003-09-16 | Canon Kabushiki Kaisha | Image processing apparatus |
US6693539B2 (en) * | 1997-07-24 | 2004-02-17 | Checkpoint Systems, Inc. | Inventory system using articles with RFID tags |
US20060210143A1 (en) * | 2005-03-16 | 2006-09-21 | Atsushi Miyamoto | Method and apparatus for measuring shape of a specimen |
US20070080228A1 (en) * | 2000-11-24 | 2007-04-12 | Knowles C H | Compact bar code symbol reading system employing a complex of coplanar illumination and imaging stations for omni-directional imaging of objects within a 3D imaging volume |
US20080110992A1 (en) * | 2000-11-24 | 2008-05-15 | Knowles C H | Method of illuminating objects at a point of sale (POS) station by adaptively controlling the spectral composition of the wide-area illumination beam produced from an illumination subsystem within an automatic digital image capture and processing system |
US20080205769A1 (en) * | 2007-01-29 | 2008-08-28 | Junichi Taguchi | Apparatus, method and program product for matching with a template |
US20100217678A1 (en) * | 2009-02-09 | 2010-08-26 | Goncalves Luis F | Automatic learning in a merchandise checkout system with visual recognition |
US20100296131A1 (en) * | 2009-05-20 | 2010-11-25 | Dacuda Ag | Real-time display of images acquired by a handheld scanner |
US20110019921A1 (en) * | 2008-04-24 | 2011-01-27 | Nec Corporation | Image matching device, image matching method and image matching program |
US20110091116A1 (en) * | 2009-10-19 | 2011-04-21 | Canon Kabushiki Kaisha | Object recognition apparatus and object recognition method |
US20110286628A1 (en) * | 2010-05-14 | 2011-11-24 | Goncalves Luis F | Systems and methods for object recognition using a large database |
US20120020558A1 (en) * | 2010-07-24 | 2012-01-26 | Canon Kabushiki Kaisha | Method for estimating attribute of object, apparatus thereof, and storage medium |
US8191780B2 (en) * | 2005-04-07 | 2012-06-05 | Freedom Shopping, Inc. | Self checkout kiosk and retail security system |
US20120308152A1 (en) * | 2010-01-28 | 2012-12-06 | Hitach High Technologies Corporation | Apparatus for Forming Image for Pattern Matching |
Family Cites Families (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7337960B2 (en) * | 2004-02-27 | 2008-03-04 | Evolution Robotics, Inc. | Systems and methods for merchandise automatic checkout |
US7246745B2 (en) * | 2004-02-27 | 2007-07-24 | Evolution Robotics Retail, Inc. | Method of merchandising for checkout lanes |
US7657126B2 (en) * | 2005-05-09 | 2010-02-02 | Like.Com | System and method for search portions of objects in images and features thereof |
JP4825594B2 (en) * | 2005-07-07 | 2011-11-30 | 株式会社リコー | Parts selection device |
US7809522B2 (en) * | 2007-03-06 | 2010-10-05 | Kenneth Wargon | Apparatus and method for determining and numerically displaying a volume dependent characteristic of any unseparated part of an item |
JP2009163331A (en) * | 2007-12-28 | 2009-07-23 | Toshiba Tec Corp | Merchandise sales data processor and computer program |
US8306872B2 (en) * | 2008-08-08 | 2012-11-06 | Nikon Corporation | Search supporting system, search supporting method and search supporting program |
JP5261312B2 (en) * | 2009-07-30 | 2013-08-14 | 富士フイルム株式会社 | Image analysis apparatus, image analysis method, and program |
JP2011113549A (en) * | 2009-11-30 | 2011-06-09 | Toshiba Tec Corp | Shopping support device and shopping support program |
CN201965631U (en) * | 2010-05-27 | 2011-09-07 | 王键辉 | Management system utilizing particle information to recognize products |
JP5194149B2 (en) * | 2010-08-23 | 2013-05-08 | 東芝テック株式会社 | Store system and program |
CN102063616A (en) * | 2010-12-30 | 2011-05-18 | 上海电机学院 | Automatic identification system and method for commodities based on image feature matching |
-
2012
- 2012-09-06 JP JP2012196425A patent/JP5612645B2/en active Active
-
2013
- 2013-03-29 CN CN201610730245.9A patent/CN106373298B/en active Active
- 2013-03-29 CN CN201310106181.1A patent/CN103679977B/en active Active
- 2013-04-23 US US13/868,232 patent/US20140067573A1/en not_active Abandoned
-
2016
- 2016-08-22 US US15/242,787 patent/US20160358150A1/en not_active Abandoned
Patent Citations (18)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6434257B1 (en) * | 1995-02-17 | 2002-08-13 | International Business Machines Corporation | Size recognition system with method for determining price of a commodity |
US6621921B1 (en) * | 1995-12-19 | 2003-09-16 | Canon Kabushiki Kaisha | Image processing apparatus |
US6693539B2 (en) * | 1997-07-24 | 2004-02-17 | Checkpoint Systems, Inc. | Inventory system using articles with RFID tags |
US20080110992A1 (en) * | 2000-11-24 | 2008-05-15 | Knowles C H | Method of illuminating objects at a point of sale (POS) station by adaptively controlling the spectral composition of the wide-area illumination beam produced from an illumination subsystem within an automatic digital image capture and processing system |
US20070080228A1 (en) * | 2000-11-24 | 2007-04-12 | Knowles C H | Compact bar code symbol reading system employing a complex of coplanar illumination and imaging stations for omni-directional imaging of objects within a 3D imaging volume |
US20020123932A1 (en) * | 2001-02-28 | 2002-09-05 | Elliot Brenhouse | Self-checkout system with anti-theft deactivation device |
JP2002312651A (en) * | 2001-04-09 | 2002-10-25 | Fuji Photo Film Co Ltd | Inventory management system |
US20030128876A1 (en) * | 2001-12-13 | 2003-07-10 | Kabushiki Kaisha Toshiba | Pattern recognition apparatus and method therefor |
US20060210143A1 (en) * | 2005-03-16 | 2006-09-21 | Atsushi Miyamoto | Method and apparatus for measuring shape of a specimen |
US8191780B2 (en) * | 2005-04-07 | 2012-06-05 | Freedom Shopping, Inc. | Self checkout kiosk and retail security system |
US20080205769A1 (en) * | 2007-01-29 | 2008-08-28 | Junichi Taguchi | Apparatus, method and program product for matching with a template |
US20110019921A1 (en) * | 2008-04-24 | 2011-01-27 | Nec Corporation | Image matching device, image matching method and image matching program |
US20100217678A1 (en) * | 2009-02-09 | 2010-08-26 | Goncalves Luis F | Automatic learning in a merchandise checkout system with visual recognition |
US20100296131A1 (en) * | 2009-05-20 | 2010-11-25 | Dacuda Ag | Real-time display of images acquired by a handheld scanner |
US20110091116A1 (en) * | 2009-10-19 | 2011-04-21 | Canon Kabushiki Kaisha | Object recognition apparatus and object recognition method |
US20120308152A1 (en) * | 2010-01-28 | 2012-12-06 | Hitach High Technologies Corporation | Apparatus for Forming Image for Pattern Matching |
US20110286628A1 (en) * | 2010-05-14 | 2011-11-24 | Goncalves Luis F | Systems and methods for object recognition using a large database |
US20120020558A1 (en) * | 2010-07-24 | 2012-01-26 | Canon Kabushiki Kaisha | Method for estimating attribute of object, apparatus thereof, and storage medium |
Also Published As
Publication number | Publication date |
---|---|
JP2014052810A (en) | 2014-03-20 |
CN103679977A (en) | 2014-03-26 |
US20140067573A1 (en) | 2014-03-06 |
CN103679977B (en) | 2016-10-05 |
CN106373298B (en) | 2019-07-30 |
CN106373298A (en) | 2017-02-01 |
JP5612645B2 (en) | 2014-10-22 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20160358150A1 (en) | Information processing apparatus and commodity recognition method by the same | |
US9189782B2 (en) | Information processing apparatus and information display method by the same | |
US9292748B2 (en) | Information processing apparatus and information processing method | |
US20160140534A1 (en) | Information processing apparatus, store system and method | |
US20180225746A1 (en) | Information processing apparatus and information processing method | |
US20140067574A1 (en) | Information processing apparatus and information processing method | |
US20160371769A1 (en) | Information processing apparatus and information processing method | |
EP3002739A2 (en) | Information processing apparatus and information processing method by the same | |
US20130100295A1 (en) | Information processing apparatus and method | |
US20150023548A1 (en) | Information processing device and program | |
JP2018055716A (en) | Information processing apparatus and program | |
EP2980729B1 (en) | Information processing apparatus and method for recognizing object by the same | |
US9524433B2 (en) | Information processing apparatus and information processing method | |
US20170344851A1 (en) | Information processing apparatus and method for ensuring selection operation | |
US20140064570A1 (en) | Information processing apparatus and information processing method | |
JP5770899B2 (en) | Information processing apparatus and program | |
US20140222602A1 (en) | Information processing apparatus and method for detecting stain on iamge capturing surface thereof | |
JP6138060B2 (en) | Information processing apparatus and program | |
JP2014052811A (en) | Information processing apparatus and program | |
EP2960831A1 (en) | Information processing apparatus and information processing method | |
JP2014052799A (en) | Information processing apparatus and program | |
JP6306776B2 (en) | Information processing apparatus and program | |
JP6392927B2 (en) | Information processing apparatus and program |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: TOSHIBA TEC KABUSHIKI KAISHA, JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:NAITO, HIDEHIRO;SUGASAWA, HIROSHI;REEL/FRAME:039495/0061 Effective date: 20130419 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |