JP2018085094A - Information processing system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information processing system capable of capturing a high quality image of an object in a manner that enhances recognition accuracy of the object placed in a predetermined area.SOLUTION: An information processing system comprises: an image acquisition section 234 to acquire a captured image which can be obtained as a result of capturing a predetermined area where an object can be placed; an existence recognition section 235 which recognizes existence of the object in the predetermined area on the basis of the captured image; and an illuminating section 208 which illuminates the object in a manner that eliminates or lighten a shadow cast by the object in the captured image.SELECTED DRAWING: Figure 10

Description

  The present invention relates to an information processing system.

  Traditionally, in stores such as convenience stores, supermarkets, shopping centers, and various mass retailers, buyers pick up products from the product shelves, put them in shopping carts and shopping baskets, and pay for the products near the store exits. To the cash register terminal for paying out the goods and paying the price of the goods using a self cash register or the like (for example, see Patent Document 1).

However, when the cash register terminal is used to pay for the product in the shopping cart or shopping basket, even if the self-cash register terminal is used for payment, the barcode for each product is used to identify one of the products. Will be read. For this reason, it is not possible to eliminate the queue of purchasers waiting for settlement, and the purchaser waits for payment for a long time by the cash register terminal.
In addition, when a purchaser purchases a product that he / she wants to purchase at a store, if there are many other purchasers, the purchaser does not want to wait for the cash register terminal to settle for a long time, and as a result, the purchase itself May give up.
In consideration of the above circumstances, when a purchaser purchases a product displayed in a store, it is possible to automate the settlement of the price of the product and shorten the time required for the settlement of the price of the product. The system is requested.
Therefore, when a product is placed in a predetermined area such as a cash register, the applicant images the predetermined area with a sensing device such as a camera, recognizes the object of the product candidate from the captured image obtained as a result, A patent application has already been filed for a next-generation cash register system that can identify a product that is an object (product candidate) and can execute a series of processing until payment is made for the specified product (Japanese Patent Application) 2014-080624).

JP 2001-76261 A

  However, in the conventional self-registration system including Patent Document 1 described above, when a product placed in a predetermined area (predetermined area) of the cash register is imaged with a sensing device, a shadow is generated in the predetermined area depending on the product. May appear in the captured image. If it does so, the information processing system using a picked-up image may misidentify goods, or cannot recognize.

  An object of the present invention is to provide an information processing system capable of capturing an object with high quality in order to improve the recognition accuracy of an object placed in a predetermined area.

In order to achieve the above object, an information processing system according to one embodiment of the present invention includes:
An acquisition means for acquiring a captured image obtained as a result of imaging a predetermined area where an object can be placed at an arbitrary position;
Recognition means for recognizing the presence of the object in the predetermined area based on the acquired captured image;
Illuminating means for uniformly illuminating the entire predetermined area including the object so that the shadow of the object in the captured image does not occur or decreases;
Enclosing means for surrounding the predetermined area so that outside light does not enter the predetermined area;
With
The predetermined area is a plate surface of a plate installed inside the surrounding means.

  Further, a plurality of the illuminating means may be provided, and the illuminating means may be arranged so as to remove at least a part of the shadow generated in the predetermined area by the one illuminating means by another illuminating means.

  Further, the illumination means can emit light that can be changed to two or more colors.

  The illumination means may be a display device that emits light over the entire flat surface and illuminates the predetermined area.

  Further, at least a part of the predetermined area can be formed so as to prevent rolling of an object on the predetermined area.

Further, the information processing system includes a surrounding unit surrounding the predetermined area,
The surrounding means has a light shielding means that can be switched between a transparent state that becomes transparent so that the predetermined area can be visually recognized and an opaque state that becomes opaque.
The light shielding means may further comprise a light shielding control means for switching between the transparent state and the opaque state.

  The information processing system may further include a presentation unit that presents one or more states related to the information processing system.

  According to the present invention, it is possible to provide an information processing system capable of capturing an image of an object placed in a predetermined area with high accuracy.

It is a block diagram which shows the structure of the goods recognition system as one Embodiment of the information processing system of this invention. It is a block diagram which shows the hardware constitutions of the server which comprises the goods recognition system of FIG. It is a block diagram which shows the hardware constitutions of the cash register terminal which comprises the goods recognition system of FIG. It is a schematic perspective view which shows an external appearance when the surrounding part with which the cash register terminal of FIG. 3 was equipped is an opaque state. It is a perspective view which shows the general view when the surrounding part with which the cash register terminal of FIG. 3 was equipped is a transparent state. It is the perspective view which shows the inside which looked at the surrounding part of the cash register terminal of FIG. 3 from the front which faces a purchaser. It is a schematic front view which shows the general | schematic structure by which the predetermined area of the cash register terminal of FIG. 3 was formed in the waveform. It is a perspective view which shows an external view when the surrounding part of other embodiment of the cash register terminal of FIG. 3 is a transparent state. It is a schematic front view which shows the part containing the predetermined area of other embodiment of the cash register terminal of FIG. It is a functional block diagram which shows an example of a functional structure with the server of FIG. 2, and the cash register terminal of FIG.

  First, an outline of an embodiment of an information processing system to which the present invention is applied will be described.

The product recognition system (next-generation cash register system) according to an embodiment of the information processing system of the present invention has at least the following functions.
That is, the product recognition system of this embodiment includes at least an acquisition function, a recognition function, and a specific function.

  In the product recognition system of the present embodiment, as a premise that these functions are exhibited, when an object is placed in a predetermined area, a sensing device such as a camera images the predetermined area and data of the captured image obtained as a result Is output.

Here, there is one or more sensing devices that image an object. Hereinafter, an example in which a camera is employed as an example of the sensing device will be described. At least one camera is fixed above a predetermined area. The camera may image an object from a plurality of directions by moving one camera or the like, or the plurality of cameras may image an object from a plurality of directions.
“Imaging” means that a predetermined area is divided into a plurality of areas, and each pixel corresponding to each of the plurality of divided areas has a pixel value.

Under such a premise, the “acquisition function” is a function for acquiring captured image data output from the camera.
Here, a captured image including an object placed in a predetermined area as a subject is hereinafter appropriately referred to as an “object captured image”.
In addition, when the product recognition system performs various types of image processing on an object captured image, it is actually handled in the form of data, but in the following description, the data will be omitted for convenience of explanation.

The “recognition function” is a function for recognizing the presence of an object based on the acquired object captured image.
In the present specification, what is not specified as a product is called an “object”. In the present specification, a product in which “object” is specified as a product is referred to as a “product”.
Here, “recognizing the presence of an object” means performing region definition by an image processing method such as background difference. In the image processing method such as background difference, the captured image before the “object” is placed in the predetermined area is compared with the captured image after the “object” is placed in the predetermined area.

The “specific function” is a function for specifying what kind of product is the object whose presence is recognized by demonstrating the recognition function.
Note that the product recognition system of the present embodiment also has a function of performing payment (settlement) for the specified product.

The product recognition system having such a function has the following problems (a) to (c) if no measures are taken.
(A) When the camera captures an object placed in a predetermined area of the cash register terminal, the shadow of the object is reflected in the object captured image. In the recognition function, the shadow is part of the object and the shadow and the object The region where the two are combined is defined as the object region, or the shadow region is mistakenly defined as the object region as a single object of the shadow, thereby degrading the accuracy of object recognition. As a result, the product may not be specified in the specific function.
(B) Since the predetermined area on which the object is placed is flat, for example, a cylindrical object is in a state of slightly rolling when placed on the predetermined area in a lying posture. When the camera captures an object in such a state, the camera cannot focus on the object and cannot accurately capture the object, or the object stops and becomes ready for imaging. It takes time.
(C) The predetermined area is covered with a cover (hereinafter also referred to as “go part” as appropriate) so that the external light does not enter the predetermined area in order to keep the recognition accuracy by the recognition unit and the specific accuracy by the specifying unit above a certain level. There is a case. In such a case, the predetermined area is always covered with the surrounding portion regardless of whether the recognition function or the specific function is exhibited.
For this reason, for example, in a predetermined area, the visibility in the surrounding portion cannot be ensured. Accordingly, not only is the purchaser difficult to place the product in the predetermined area, but also the store clerk cannot grasp the fraud or the like made in the Go section.
(D) Regardless of the presence or absence of the Go part, the clerk will indicate whether an object placed in a predetermined area is being settled, whether it is being settled, whether an abnormality has occurred, etc. Or the buyer who waits for payment cannot grasp.

Therefore, the product recognition system of the present embodiment has the following features (A) to (D) in order to solve each of the problems (a) to (d) described above.
(A) The product recognition system includes an illumination unit that illuminates an object placed in a predetermined area so that the shadow of the object in the object captured image does not occur or decreases (so that the shadow does not appear in the object captured image). The feature of preparing. With this feature, the shadow is not defined as the object region, and the recognition accuracy of the object can be improved.
(B) The product recognition system is characterized in that at least a part of the predetermined area is formed in a wave shape. Due to this feature, concave portions and convex portions are repeatedly formed in the predetermined area. As a result, at least a part of the object is sandwiched and held between the two protrusions, so that the object is prevented from rolling on the predetermined area.
(C) The product recognition system includes a surrounding portion surrounding a predetermined area, and the surrounding portion is transparent so that a store clerk or a purchaser waiting for payment can visually recognize the predetermined area, and an opaque state that becomes opaque The feature that it can be switched to. With this feature, it is possible to improve the image quality of an object captured image as an opaque state when taking an image, and it is easy for a purchaser to place a product in a predetermined area as a transparent state when no image is taken, and the store clerk grasps fraud in the surrounding area. can do.
(D) The product recognition system is characterized by including a presentation unit that presents one or more states related to the product recognition system. With this feature, the store clerk or the purchaser who waits for payment can grasp the state of the product placed in the predetermined area.

Next, a product recognition system as an embodiment of the information processing system of the present invention will be described with reference to the drawings.
FIG. 1 is a configuration diagram showing a configuration of a product recognition system as an embodiment of an information processing system of the present invention.
The product recognition system according to the present embodiment has a configuration as shown in FIG. 1 in order to accurately capture an image of an object placed in a predetermined area.

That is, the product recognition system has a server 1 and n (n is an arbitrary integer) cash register terminals 2-1 to 2-n.
The server 1 and the cashier terminals 2-1 to 2-n are connected to each other via a network N such as the Internet line.
For convenience of explanation, only one server 1 in FIG. 1 is drawn, but there may actually be a plurality of servers.
Hereinafter, when it is not necessary to distinguish the cashier terminals 2-1 to 2-n individually, these are collectively referred to as “the cashier terminal 2”.

The server 1 executes each process in order to manage each operation of the cash register terminal 2.
FIG. 2 is a block diagram illustrating a hardware configuration of the server 1.

  The server 1 includes a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, a bus 104, an input / output interface 105, an output unit 106, an input unit 107, A storage unit 108, a communication unit 109, and a drive 110.

The CPU 101 executes various processes according to a program stored in the ROM 102 or a program loaded from the storage unit 108 to the RAM 103.
The RAM 103 also stores data necessary for the CPU 101 to execute various processes as appropriate.

  The CPU 101, ROM 102, and RAM 103 are connected to each other via a bus 104. An input / output interface 105 is also connected to the bus 104. An output unit 106, an input unit 107, a storage unit 108, a communication unit 109, and a drive 110 are connected to the input / output interface 105.

The output unit 106 includes a display, a speaker, and the like, and outputs various types of information as images and sounds.
The input unit 107 includes a keyboard, a mouse, a touch panel, and the like, and inputs various information.

The storage unit 108 includes a hard disk, a DRAM (Dynamic Random Access Memory), and the like, and stores various data.
The communication unit 109 communicates with other devices (the cashier terminals 2-1 to 2-n in the example of FIG. 1) via the network N (FIG. 1) including the Internet.

A removable medium 120 made of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is appropriately attached to the drive 110. The program read from the removable medium 120 by the drive 110 is installed in the storage unit 108 as necessary.
The removable medium 120 can also store various data stored in the storage unit 108 in the same manner as the storage unit 108.

  The cash register terminal 2 connected to the server 1 via the network N includes a product recognition area on which a product to be recognized is placed, and is configured by various hardware configurations.

FIG. 3 is a block diagram showing a hardware configuration of the cash register terminal 2.
The cash register terminal 2 includes a CPU 201, a ROM 202, a RAM 203, a bus 204, an input / output interface 205, an output unit 206, an input unit 207, a lighting unit 208, a light shielding unit 209, a presentation unit 210, and an imaging unit. Unit 211, storage unit 212, communication unit 213, and drive 214.
As shown in FIGS. 4 to 6, the cash register terminal 2 includes a surrounding portion 310 that surrounds a predetermined area. The surrounding portion 310 is switched between a transparent state that is transparent so that the predetermined area can be visually recognized and an opaque state that is opaque so that the predetermined area cannot be visually recognized.

  The CPU 201, the ROM 202, the RAM 203, the bus 204, the input / output interface 205, the storage unit 212, the communication unit 213, the drive 214, and the removable medium 220 of the cash register terminal 2 are configured in the same manner as those of the server 1.

The output unit 206 includes a display unit D that displays information related to the product score, information related to payment, and a receipt output unit R that outputs a receipt.
The input unit 207 includes a touch panel (not shown) and a card reader unit C.

The illumination unit 208 illuminates the predetermined area so that a shadow is not generated or reduced in the object captured image obtained by capturing an object placed in the predetermined area (so as not to appear in the object captured image).
The light shielding portion 209 is a component of the surrounding portion 310, and is configured of an instantaneous light control sheet so that the surrounding portion 310 can be switched between a transparent state in which the surrounding portion 310 is transparent and an opaque state in which the surrounding portion 310 is opaque.
Instantaneous light control sheet is a transparent state when a liquid crystal is sandwiched between two transparent electrode films and a voltage is applied to both transparent electrode films, and can be instantaneously changed to an opaque state when not applied. It is a sheet. When the light shielding unit 209 is configured by the instantaneous light control sheet, the inside of the surrounding unit 310 may or may not be visible.
The presentation unit 210 is provided on one or both end surfaces of the surrounding unit 310 surrounding the predetermined area, or is provided separately from the surrounding unit 310, and emits one or more states related to the product recognition system in different colors. Present by doing.

  The imaging unit 211 includes a single camera that captures an object placed in a predetermined area. The imaging unit 211 is fixed to the top plate portion of the go unit 310 and images a predetermined area. The imaging unit 211 captures an object placed in a predetermined area, and acquires a plurality of captured images obtained as a result as an object captured image.

Here, a specific example of the external configuration of the cash register terminal 2 will be described with reference to FIGS.
FIG. 4 is a schematic perspective view showing a general configuration of the cash register terminal 2 when the surrounding part 310 is in an opaque state.
FIG. 5 is a schematic perspective view showing a general configuration of the cash register terminal 2 when the surrounding part 310 is in a transparent state.
FIG. 6 is a perspective view showing the inside of the Go unit 310 as seen from the front facing the purchaser.
FIG. 7 is a schematic front view showing an overview configuration in which the predetermined area A in the surrounding portion 310 is formed in a wave shape.

As shown in FIGS. 4 to 6, the cash register terminal 2 includes a surrounding portion 310 that surrounds a predetermined area A on which the product G (FIG. 7) is placed. The surrounding portion 310 includes a top plate portion 311, a bottom plate portion 312, a pair of side plate portions 313, and a back plate portion 314.
A housing part 320 is installed on the bottom plate part 312. The housing unit 320 includes the CPU 201, the ROM 202, the RAM 203, and the like of the cash register terminal 2, and the output unit 206 and the input unit 207 are provided on the front surface (a front surface facing the purchaser).
A plate 330 is installed on the housing part 320. The board surface of the plate 330 is defined as a predetermined area A. A space is provided between the predetermined area A and the top plate portion 311.

Illuminating units 208D and 208U (FIG. 5) are provided below the board surface of the predetermined area A and in the top panel 311. The illuminating unit 208 in FIG. 3 includes illuminating units 208D and 208U that emit light not only in white but also in various colors such as blue and red. The light emission colors of the illumination units 208D and 208U are switched by control on the system. An object placed in the predetermined area A is illuminated from above and below by the illumination units 208D and 208U so that the shadow of the object does not occur or decreases in the predetermined area A.
In addition, in order to illuminate the object also from the side, the pair of side plate portions 313 and the back plate portion 314 may include the illumination unit 208. When such an illumination unit 208 is not provided, the inner surface of the side plate portion 313 and the inner surface of the back plate portion 314 are used as reflection surfaces for light emitted from the illumination units 208D and 208U.

The plate 330 is formed in a cloudy color. When the illumination unit 208 (illumination units 208D and 208U in the example of FIG. 5) is switched to white or various colors to emit light, the predetermined area A is switched to white or various colors because the plate 330 is cloudy. Is illuminated. Merchandise is generally colored or patterned. Therefore, when the predetermined area A is illuminated with white or various colors, the shadow of the object is not generated or reduced on the predetermined area A. Therefore, the product recognition system can easily perform the recognition function.
That is, the CPU 201 of the cash register terminal 2 performs a recognition function, compares the images before and after the presence recognition unit 235 (FIG. 10) is placed in the predetermined area A, and performs white difference by background difference processing. It is possible to specify the region of the object placed in the predetermined area A with various colors as backgrounds.

  Whether the predetermined area A is illuminated in white or colored, a texture may be formed in at least a part of the predetermined area A. For example, a white object is assimilated with white illumination, and it becomes difficult to specify the region of the object. However, when the texture is formed, the texture is hidden by the object placed in the predetermined area A, and the region of the object is specified. Therefore, due to the texture, the white object is not assimilated with the predetermined area A illuminated in white, and the recognition function is exhibited.

Specifically, the texture is a pattern in which a predetermined pattern such as a dot pattern, a stripe pattern, or a lattice pattern is repeated. The texture may be uniform throughout the predetermined area A or may be non-uniform. For example, when the texture is formed of a large number of dots, the size of the dots may be increased at a certain area of the predetermined area A and decreased at other areas.
Further, when the texture is formed of dots or stripes, the pitch may be shortened at a part where the predetermined area A is present and the pitch may be lengthened at another part. Or you may make it a different pattern by a site | part with the predetermined area A, and another site | part.

The front side end face of the surrounding part 310 facing the purchaser constitutes a presentation part 210 that emits light in various colors according to the state of the cash register terminal 2. The presentation unit 210 presents the state of the cash register terminal 2. For example, when the presentation unit 210 emits blue light, it indicates that it is in normal standby. Normal standby means a state in which no one is operating or a state after completion of payment.
When the presentation unit 210 emits green light, it indicates that the payment is being made. The settlement includes a case where the object is recognized and the product is specified, and the result is displayed, and then the payment is in a waiting state.
Further, when the presenting unit 210 emits yellow light, it indicates that the store clerk is operating. The store clerk operation includes a state of waiting for age confirmation and a state of waiting for a clerk call.
Further, when the presenting unit 210 emits red light, it indicates that it is in an abnormal state. The abnormal state includes an error state that seems to have stopped, an improper purchaser, and the like.

  The presentation unit 210 may be provided not only on the front side end face of the surrounding portion 310 but also on the top plate portion 311, the side plate portion 313, and the back plate portion 314 of the surrounding portion 310. Further, the presentation unit 210 provided in the top plate unit 311, the side plate unit 313, and the back plate unit 314 may not only emit light in different colors but also display characters and the like.

  An imaging unit 211 is attached to the inner surface of the top plate part 311 of the surrounding part 310. Naturally, the imaging unit 211 is attached with the lens facing downward so as to image the predetermined area A. The imaging unit 211 is provided so as not to obstruct illumination by the illumination unit 208U on the top plate unit 311 side.

  Further, the predetermined area A is formed in a wave shape as shown in FIG. The predetermined area A shown in FIG. 7 is provided on a board surface in which the surface of the plate 330 is formed into a wave shape. The wave shape may be not only a sinusoidal wave shape as shown, but also a rectangular wave shape, and the pitch and amplitude may be not only equal but also uneven. In any case, the predetermined area A is formed in a wave shape in which a concave portion and a convex portion are repeatedly formed, so that at least a part of a cylindrical or spherical object is sandwiched between the convex portion and the convex portion and does not roll. Can be.

A plurality of illumination units 208 </ b> D are embedded in the plate 330. The plate 330 has, for example, a cloudy color that transmits light from the illumination unit 208D. When the light from the illumination unit 208D is emitted from the surface of the plate 330, the predetermined area A is illuminated in white. Even if a plurality of illumination units 208D are arranged at intervals, only the portion of the illumination unit 208 is not brightened, and the entire predetermined area A is illuminated uniformly.
Regardless of the color of the plate 330, a texture is formed in the predetermined area A by printing a pattern such as a dot on the surface of the plate 330 or by sticking a seal.

  The illumination unit 208 in FIG. 3 can also be configured by a thin display device 208P as shown in FIGS. The display device 208P is installed on the upper surface of the housing unit 320. The display device 208P emits light on the entire flat surface (excluding the frame portion) facing upward, and illuminates the predetermined area A with an arbitrary color. Furthermore, the thin display device 208P can display not only illumination but also an image including a texture. A corrugated plate 331 is installed on the upper surface of the display device 208P. The plate 331 is a transparent member that transmits light from the display device 208P.

FIG. 10 is a functional block diagram showing an example of a functional configuration of the server 1 of FIG. 2 and the cash register terminal 2 of FIG.
In the CPU 101 of the server 1 (FIG. 2), the DB management unit 401 functions as shown in FIG.
A product DB 402 is provided in one area of the storage unit 108 (FIG. 2) of the server 1. The product DB 402 stores information related to products.

  The DB management unit 401 manages a product image stored in advance in the product DB 402 of the storage unit 108 as product information in order to specify a product. In the cash register terminal 2, the same product registration as that performed by the DB management unit 401 of the server 1 may be performed.

  In the CPU 201 of the cash register terminal 2, as shown in FIG. 10, a light shielding control unit 230, a light emission control unit 231, a presentation control unit 232, an image acquisition unit 234, a presence recognition unit 235, a product identification unit 236, The settlement unit 237 and the display control unit 238 function.

The light blocking control unit 230 performs control to switch between a transparent state at a timing when the light blocking unit 209 provided in the surrounding unit 310 captures an object placed in the predetermined area A and an opaque state at a timing when no image is captured.
The light emission control unit 231 switches between a state in which the illumination unit 208 emits light at a timing when an object is imaged and a state in which light is not emitted at a timing when the object is not imaged, a situation where an object placed in the predetermined area A is recognized Control to switch the emission color.
The presentation control unit 232 controls the emission color to change so that the presentation unit 210 presents the state of the cash register terminal 2. Since the state of the cash register terminal 2 is constantly changing, the color of the presentation unit 210 changes each time the state changes.

The image acquisition unit 234 acquires data of an object image captured by the imaging unit 211 with the object placed in the predetermined area A.
The presence recognition unit 235 recognizes the presence of an object placed in the predetermined area A using the predetermined image recognition method described above.

The product specifying unit 236 uses an image recognition technique such as specific object recognition or general object recognition based on an object captured image and image information of a product stored in advance for an object whose existence is recognized, It is specified which product the object placed in the area A is. For example, the product specifying unit 236 matches image information obtained from an object whose existence is recognized with image information held by the DB information holding unit 241. That is, if the two pieces of image information match, the object whose presence is recognized by the presence recognition unit 235 is a product held in the DB information holding unit 241, so the product specifying unit 236 holds the DB information holding unit 241. The product information such as the product number, the product name, and the price is specified.
The settlement unit 237 settles the product placed in the predetermined area A. For example, the settlement unit 237 reads the price of the product placed in the predetermined area A from the DB information holding unit 241, adds it, and displays it on the display unit D to perform settlement.

  The display control unit 238 executes control for sequentially displaying at least a part of the acquired object captured image and an image processed or created based on the object captured image to the purchaser and the store clerk. . Specifically, the display control unit 238 sequentially outputs at least a part of the acquired object captured image and an image processed or created based on the object captured image, the output unit 206 (display unit D). Execute the control to be displayed on the screen.

In the product recognition system of the present embodiment, the following operation (processing) is executed.
When the presentation unit 210 emits blue light, it can be recognized from the outside that the cash register terminal 2 is on standby.
When an object is placed in the predetermined area A of the waiting cash register terminal 2, the imaging unit 211 images the object. Further, since the light shielding portion 209 of the surrounding portion 310 is in a transparent state, it is easy for the purchaser to place the product (object) in the predetermined area A.

Prior to acquiring the captured image of the object, first, if the light shielding unit 209 of the surrounding unit 310 is in a transparent state, the light shielding control unit 230 switches the light shielding unit 209 to an opaque state, and blocks external light hitting the object. By blocking external light, it is possible to acquire an object captured image in which excess light is not reflected.
Next, when the illumination unit 208 is not emitting light, the light emission control unit 231 switches the illumination unit 208 to emit light so as not to cause a shadow in the predetermined area A.
When the purchaser presses a recognition button (not shown), the imaging unit 211 images a predetermined position (for example, before the predetermined area A) with respect to the predetermined area A. At this time, since the illumination unit 208 is illuminated, a shadow of an object placed in the predetermined area A does not occur in the predetermined area A.
Then, the image acquisition unit 234 acquires an object captured image obtained as a result of imaging an object on the predetermined area A.

  The plurality of object captured images captured by the imaging unit 211 are input to the image acquisition unit 234 as having high accuracy. Then, the image acquisition unit 234 outputs the object captured image to the display control unit 238. The display control unit 234 executes control to display at least a part of the processed or created image sequentially to the purchaser and the store clerk based on the object captured image output from the image acquisition unit 234. To do. Specifically, the display control unit 234 executes control to sequentially display at least a part of the acquired object captured image and the processed or created image on the output unit 206 (display unit D). .

  Then, the data of the plurality of object captured images acquired by the image acquisition unit 234 is output to the presence recognition unit 235. The presence recognition unit 235 recognizes the presence of an object from an object captured image obtained by imaging the predetermined area A. The recognized object data is output to the product specifying unit 236.

  The product specifying unit 236 specifies which product the object placed in the predetermined area A is. The product specifying unit 236 matches, for example, information obtained from an object whose existence is recognized with product information held by the DB information holding unit 241. Thereby, the product specifying unit 236 specifies which product of the matched product is the product information held in the DB information holding unit 241. Data such as the product number, product name, and price for this product is output to the settlement unit 237 and the display control unit 238.

The settlement unit 237 settles the product placed in the predetermined area A. Specifically, the checkout unit 237 pays out all the products placed in the predetermined area A by individually adjusting the specified products placed in the predetermined area A. That the payment is in progress is switched by the presentation control unit 232 so that the presentation unit 210 emits light with a predetermined color, and the surroundings are informed that the payment is in progress.
In addition, the display control unit 238 outputs the product name, price, and the like to the output unit 206. The output unit 206 outputs the product name, price, and the like to the display unit D.
In such a cash register terminal 2, when the presentation unit 210 emits green light, it is recognized that the payment is being made from the outside. Further, when the presentation unit 210 emits red light, it is recognized from the outside that an abnormal state has occurred.

  When the payment is completed or when the imaging is completed, the light shielding unit 209 is switched to a transparent state by the light shielding control unit 230 so that the inside of the surrounding unit 310 can be visually recognized, and the illumination control unit 231 illuminates the illumination unit 208. Is cut off.

  Although one embodiment of the present invention has been described above, the present invention is not limited to the above-described embodiment, and modifications and the like within a scope that can achieve the object of the present invention are included in the present invention. is there.

For example, the above-described image processing by the CPU 201 may be processing by a GPU (Graphics Processing Unit).
2 and 3 are merely examples for achieving the object of the present invention, and the present invention is not limited to this.

  Moreover, the external configuration of the cash register terminal 2 shown in FIGS. 4 to 9 is an example, and the present invention is not limited to this external appearance. At least the number of the imaging units 211 is not limited to the above embodiment, and if the present invention can be implemented, for example, a support member that rotates around the predetermined area A is provided, and this support member includes one imaging unit 211. You may make it support.

  Further, the functional block diagram shown in FIG. 10 is merely an example and is not particularly limited. That is, it is sufficient that the information processing system has a function capable of executing the above-described series of processes as a whole, and what functional block is used to realize this function is not particularly limited to the example of FIG.

For example, the location where the functional block exists is not limited to FIG. 10 and may be arbitrary.
In addition, one functional block may be configured by hardware alone, may be configured by software alone, or may be configured as a combination thereof.
For example, the recognition function of the cash register terminal 2 can be transferred to the server 1. More specifically, for example, at least one of the presence recognizing unit 235 and the product specifying unit 236 can be transferred to the server 1 and only the result of the processing by the server 1 can be transmitted to the cash register terminal 2. Conversely, at least some functions such as the DB management unit 401 of the server 1 can be transferred to the cash register terminal 2.
When the processing of each functional block is executed by software, a program constituting the software is installed on a computer or the like from a network or a recording medium.
The computer may be a computer incorporated in dedicated hardware. The computer may be a general-purpose smartphone or personal computer in addition to a computer capable of executing various functions by installing various programs, for example, a server.

  A recording medium including such a program is not only constituted by a removable medium distributed separately from the apparatus main body in order to provide the program to each user, but also provided to each user in a state of being incorporated in the apparatus main body in advance. The recording medium may be provided.

Furthermore, in the above-described embodiment, a plurality of illumination units are provided. However, the illumination unit may be provided only in one place as long as an object shadow does not occur or is reduced in a predetermined area.
Further, in the above-described embodiment, when the illumination unit is a display device, the color can be changed to two or more colors. However, light may be emitted in one color such as only white or only blue.

In the above-described embodiment, the texture is formed in at least a part of the predetermined area. However, if the illumination unit can prevent or reduce the shadow of the object in the object captured image, the texture needs to be formed. There is no.
In the above-described embodiment, at least a part of the predetermined area is formed in a wave shape. However, the predetermined area is viscous or fluffy, and a cylindrical object is placed. In the case where there is not, it does not necessarily have to be formed in a wave shape.

In the above-described embodiment, the surrounding part 310 that can be switched between the transparent state and the opaque state is provided. However, the surrounding part 310 may be only in the opaque state and may not be switched to the transparent state. In this case, the light shielding control unit is not provided.
Further, the information processing system may not include the go part in the first place. In this case, the imaging unit 211 is separately held by a holding member such as a support.
In the above-described embodiment, one or more states related to the information processing system are presented by the presentation unit 210. However, when it is not necessary to present the state, it is not necessary to provide the presentation unit 210. In this case, the presentation control unit 232 is not provided.

In summary, the information processing system to which the present invention is applied only needs to have the following configuration, and can take various embodiments.
That is, the information processing system to which the present invention is applied (for example, the product recognition system in FIG. 1)
Acquisition means (for example, the image acquisition unit 234 of FIG. 10) for acquiring a captured image obtained as a result of imaging a predetermined area (for example, the predetermined area A of FIG. 4) where an object can be placed at an arbitrary position;
Recognition means (for example, presence recognition unit 235 in FIG. 10) for recognizing the presence of the object in the predetermined area based on the acquired captured image;
Illuminating means (for example, the illumination unit 208 in FIG. 10) that uniformly illuminates the entire predetermined area including the object so that the shadow of the object in the captured image does not occur or decreases;
Enclosing means for enclosing the predetermined area so as to prevent outside light from entering the predetermined area (for example, the enclosing portion 310 in FIG. 4);
With
The predetermined area is a board surface of a plate (for example, plate 330 in FIG. 4) installed inside the surrounding means.
Thereby, it is possible to illuminate the predetermined area so that the shadow of the object does not occur or to decrease, and to improve the accuracy of object recognition.

Further, a plurality of the illumination means (for example, the illumination unit 208 in FIG. 10) are provided, and are arranged so that at least a part of the shadow generated in the predetermined area by one illumination means is removed by another illumination means. Can be.
Thereby, one illumination unit and another illumination unit cooperate to prevent the shadow of an object in the captured image from occurring.

Further, the illumination means (for example, the illumination unit 208 in FIG. 10) can emit light so that the color can be changed to two or more colors.
Thereby, according to the color of the object placed in the predetermined area, the object is not assimilated with the illumination color, and the image area of the object can be accurately specified.

Further, the illumination means (for example, the illumination unit 208 in FIG. 10) may be a display device (for example, the illumination unit 208P in FIG. 8) that emits light over the entire flat surface and illuminates a predetermined area.
This makes it easy to emit light in various colors, to emit light in different colors between a certain area and another part, or to make a texture appear.

Further, at least a part of the predetermined area (for example, the predetermined area A in FIG. 4) can be formed so as to prevent rolling of an object on the predetermined area.
Thereby, even if the object placed in the predetermined area is cylindrical, the object can be prevented from rolling in the predetermined area.

The surrounding means includes light shielding means (for example, a light shielding portion 209 in FIG. 10) that can be switched between a transparent state that becomes transparent so that the predetermined area can be visually recognized and an opaque state that becomes opaque.
The light shielding unit may further include a light shielding control unit (for example, the diagram light shielding control diagram 230) that switches between the transparent state and the opaque state.
By making the surrounding means transparent, it can be easily recognized whether an object is present in the surrounding means, and by making the surrounding means opaque, the object in the surrounding means can be Images can be taken without any influence.

In addition, the information processing system can further include a presentation unit (for example, the presentation unit 210 in FIG. 10) that presents one or more states related to the information processing system.
Thereby, it is possible to easily recognize the state of the object placed in the predetermined area.

  Further, in the present specification, the term “system” means an overall apparatus composed of a plurality of devices, a plurality of means, and the like.

DESCRIPTION OF SYMBOLS 1 ... Server 2, 2-1, 2-n ... Cash register terminal 101 ... CPU
102 ... ROM
103 ... RAM
104 ... Bus 105 ... Input / output interface 106 ... Output unit 107 ... Input unit 108 ... Storage unit 109 ... Communication unit 110 ... Drive 120 ... Removable media 201 ...・ CPU
202 ... ROM
203 ... RAM
204 ... Bus 205 ... Input / output interface 206 ... Output unit 207 ... Input unit 208 ... Lighting unit 208P ... Display device 209 ... Shading unit 210 ... Presentation unit 211 -Imaging unit 212-Storage unit 213-Communication unit 214-Drive 220-Removable media 230-Shading control unit 231-Light emission control unit 232-Presentation control unit 234- ..Image acquisition unit 235... Presence recognition unit 236... Product identification unit 237 .. settlement unit 238... Display control unit 310 .. enclosure unit 320. ... Plate 401 ... DB management unit 402 ... Product DB
A ... Predetermined area (product recognition area)
D: Display unit N: Network

Claims (7)

An acquisition means for acquiring a captured image obtained as a result of imaging a predetermined area where an object can be placed at an arbitrary position;
Recognition means for recognizing the presence of the object in the predetermined area based on the acquired captured image;
Illuminating means for uniformly illuminating the entire predetermined area including the object so that the shadow of the object in the captured image does not occur or decreases;
Enclosing means for surrounding the predetermined area so that outside light does not enter the predetermined area;
With
The predetermined area is a plate surface of a plate installed inside the surrounding means. 2. The information processing system according to claim 1, wherein a plurality of the illumination units are provided, and are arranged so that at least a part of a shadow generated in the predetermined area by one illumination unit is removed by another illumination unit. . The information processing system according to claim 1, wherein the illumination unit emits light so that the color can be changed to two or more colors. The information processing system according to claim 1, wherein the illumination unit is a display device that emits light over the entire flat surface and illuminates a predetermined area. 5. The information processing system according to claim 1, wherein at least a part of the predetermined area is formed to prevent rolling of an object on the predetermined area. The surrounding means has a light shielding means that can be switched between a transparent state that becomes transparent so that the predetermined area can be visually recognized and an opaque state that becomes opaque.
The information processing system according to any one of claims 1 to 5, further comprising a light shielding control unit that switches the light shielding unit between the transparent state and the opaque state. The information processing system according to any one of claims 1 to 6, further comprising a presentation unit that presents one or more states related to the information processing system.

Images