JP2022125870A - Commodity information acquisition method and commodity information acquisition system - Google Patents

Abstract

To reduce the workload for acquiring information on arrangement of commodities displayed on a shelf.SOLUTION: A commodity information acquisition method includes: image acquisition processing of acquiring images of a shelf 13 captured by cameras 5, 7; information read processing of acquiring commodity identification information and identification information of a reference position by reading an image of a shelf tag C and an image of a position indicator M included in the captured image; position calculation processing of calculating a position of the shelf tag C on the basis of a positional relationship between the image of the shelf tag C and the image of the position indicator M in the captured image; and information storage processing of storing the commodity identification information, the position information of the shelf tag C, and the identification information of the reference position, in association with each other.SELECTED DRAWING: Figure 16

Description

The present disclosure relates to a product information acquisition method and a product information acquisition system for acquiring information regarding the arrangement of multiple products arranged on a shelf.

In stores such as convenience stores and supermarkets, a plurality of products are displayed and sold on shelves. On those shelves, a plurality of commodities are grouped and displayed for each of the same commodities. In addition, near each group of the same products on the shelf, shelf labels displaying information (product name, price, etc.) about the product are installed.

On the other hand, a store manager (or a system that manages products) needs to know the sales status of various products and the shortage status of each product. (whether or not

Therefore, there is a technology that acquires product position information based on shelf labels installed on shelves. For example, multiple electronic shelf label terminals (equivalent to shelf labels) that display product prices and descriptions are installed on shelves where products are placed, and workers use them as standard shelf labels that serve as the basis for position information. By inputting the position information of the position where the electronic shelf label terminal is installed into the handy terminal and sequentially inputting the information indicating the relative position of the other electronic shelf label terminals to the reference shelf label into the handy terminal, the electronic shelf label A system for determining location information of a terminal is known (see Patent Document 1).

Japanese Patent No. 5010664

However, in the conventional technology described in Patent Document 1, a worker manually obtains position information of an electronic shelf label terminal used as a reference shelf label and information indicating relative positions of other electronic shelf label terminals with respect to the reference shelf label. You have to type them into the terminal. Therefore, in the prior art, the workload for acquiring information on product placement is high. Further, in the conventional technology, the accuracy of information regarding the arrangement of products depends on the work accuracy of each worker.

Therefore, the main object of the present disclosure is to provide a product information acquisition method and a product information acquisition system that can reduce the workload for acquiring information about the arrangement of products arranged on shelves.

A product information acquisition method of the present disclosure is a product information acquisition method for acquiring information about the arrangement of a plurality of products arranged on a shelf, and the shelf corresponds to a group of the products grouped together for each same product. a plurality of shelf labels arranged as a unit and containing product identification information about the product, and a position display unit arranged at a predetermined reference position of the shelf and containing identification information of the reference position, An image acquisition process for acquiring a photographed image of the shelf photographed by a camera, and reading the image of the shelf label and the image of the position display unit included in the photographed image, thereby obtaining the product identification information and the reference position. information reading processing for obtaining identification information; position calculation processing for calculating the position of the shelf label from the positional relationship between the image of the shelf label and the image of the position display unit in the captured image; the product identification information; and information storage processing for storing the information on the position of the shelf label and the identification information on the reference position in association with each other.

In addition, a product information acquisition system of the present disclosure is a product information acquisition system in which a processor executes processing for acquiring information regarding the arrangement of a plurality of products arranged on a shelf, and the shelf stores products grouped for each same product. a plurality of shelf labels arranged corresponding to the group of the products arranged and containing product identification information about the products; and a position arranged at a predetermined reference position on the shelf and containing identification information of the reference position The display unit and the processor acquire a photographed image of the shelf photographed by a camera, and read the image of the shelf label and the image of the position display unit included in the photographed image to obtain the product identification information and the position display unit. The identification information of the reference position is obtained, the position of the shelf label is calculated from the positional relationship between the image of the shelf label and the image of the position display unit in the photographed image, and the product identification information and the position of the shelf label are calculated. The information on the position and the identification information on the reference position are stored in association with each other.

Advantageous Effects of Invention According to the present disclosure, it is possible to reduce the workload for acquiring information regarding the arrangement of products arranged on shelves.

Configuration diagram of a product information acquisition system according to the first embodiment Explanatory diagram showing an example of the internal configuration of a store Explanatory diagram showing the configuration of the display shelf Explanatory diagram showing an example of shelf label display Explanatory diagram showing a modified example of the display when photographing the shelf label Functional block diagram showing the configuration of the product information acquisition device Explanatory diagram showing an example of guide display when shooting with the store clerk's camera Explanatory diagram showing an example of image processing and information reading processing for one photographed image. Explanatory diagram showing an example of image processing and information reading processing for a plurality of captured images. Explanatory diagram showing an example of shelf information and a world coordinate table related to it Explanatory diagram showing an example of shelf information including world coordinates Explanatory diagram showing an example of product information and its related product table Explanatory diagram showing how to calculate the position of the shelf label on the display shelf Explanatory diagram showing an example of correction coefficients used to calculate the position of the shelf label Explanatory diagram showing an example of product information including world coordinates Flow diagram showing the flow of product location information generation processing by the product information acquisition system Explanatory diagram showing an example of product placement information Configuration diagram showing a modification of the product information acquisition system shown in FIG. Functional block diagram showing the configuration of a product information acquisition device according to the second embodiment Explanatory diagram showing the configuration of a display shelf in the product information acquisition system according to the second embodiment Flow chart showing the flow of processing for generating product search information by the product information acquisition system Explanatory diagram showing an example of product search information Explanatory diagram showing a modification of the product search information shown in FIG.

A first invention, which has been made to solve the above-mentioned problems, is a product information acquisition method for acquiring information about the arrangement of a plurality of products arranged on a shelf. a plurality of shelf labels arranged corresponding to the product group and containing product identification information about the product; and a position display section arranged at a predetermined reference position on the shelf and containing identification information of the reference position. and are provided, and an image acquisition process for acquiring a photographed image of the shelf photographed by a camera, and the product identification by reading the image of the shelf label and the image of the position display unit included in the photographed image Information reading processing for acquiring information and identification information of the reference position, and position calculation processing for calculating the position of the shelf label from the positional relationship between the image of the shelf label and the image of the position display section in the captured image. and an information storage process for storing the product identification information, the position information of the shelf label, and the identification information of the reference position in association with each other.

According to this, by reading the photographed image of the shelf including the shelf label and the position display portion, information in which the product identification information, the position information of the shelf label, and the identification information of the reference position are associated is obtained. The workload for obtaining information about the placement of the lined up products is reduced.

In the second invention, in the position calculation process, only when one or more shelf label images and a plurality of position display unit images are read from one captured image in the information reading process, The configuration is such that the position of the shelf label is calculated.

This simplifies the processing for associating the coordinates in the photographed image with the world coordinates (space coordinates) from the positions of the images of at least two or more position display units, so that the position of the shelf label can be easily calculated. can be done.

In the third invention, a part of the shelf label is arranged at the reference position and constitutes a position display shelf label including identification information of the reference position, and in the position calculation process, in the photographed image The position of the shelf label is calculated from the positional relationship between the image of the position display shelf label and the images of the other shelf labels.

According to this, by using some of the plurality of shelf labels as the position display section (position display shelf label), the number of position display sections provided separately from the shelf labels can be reduced (or omitted).

In a fourth aspect of the invention, in the image acquisition process, a plurality of the photographed images with different photographing conditions are acquired, and in the information reading process, the image of the shelf label and the position display included in the plurality of photographed images are obtained. The product identification information and the identification information of the reference position are acquired by reading the image of each part.

According to this, since the image of the shelf label and the image of the position display section are read from a plurality of photographed images obtained under different photographing conditions, the product identification information and the identification information of the reference position are stably obtained.

In a fifth aspect of the invention, in the image acquisition processing, a plurality of photographed images under different photographing conditions are acquired, and image processing is executed before the information reading processing. At least one of an image synthesizing process of synthesizing a photographed image or a plurality of images derived from the photographed image and an image processing process of processing the photographed image or an image derived from the photographed image is executed.

According to this, the image read accuracy is improved by reading the image that has been subjected to the image composition processing and the image processing processing in the information reading processing.

In a sixth aspect of the invention, the photographing conditions include at least one of the angle of the camera and the amount of exposure.

According to this, even if part of the image of the shelf label or the image of the position display unit in a certain photographed image becomes unclear due to the reflection of light, etc., these images can be obtained from other photographed images from different angles. information can be obtained.

In a seventh aspect of the present invention, the camera has a monitor for displaying a subject at the time of photographing, and the monitor displays a guide frame in which the shelf as the subject is to be placed when the shelf is photographed. configuration.

According to this, even when an unspecified person takes an image of the shelf with a camera, it is possible to obtain a photographed image suitable for image reading.

In the eighth invention, in the information reading process, it is determined whether or not the image of the shelf label and the image of the position display unit have been successfully read, and the result of the determination is notified to an administrator.

According to this, it is possible to make the manager of the product recognize whether the reading of the image of the shelf label and the image of the position display portion has been successful, and to prompt the manager to take action when the reading is unsuccessful.

In a ninth aspect of the invention, the plurality of shelf labels include electronic shelf labels, and the electronic shelf labels display an image for photographing that is different when the shelf is photographed by the camera than when not photographed. to configure.

According to this, when the shelf is photographed by the camera, an image for photographing is displayed on the electronic shelf label, so that a photographed image suitable for reading the image in the information reading process can be acquired.

In a tenth aspect of the invention, the electronic shelf label is configured such that at least part of an image other than the product identification information displayed when not photographing is not displayed in the photographing image.

According to this, by hiding images other than the product identification information, a photographed image suitable for reading the image in the information reading process is acquired.

In the eleventh invention, the electronic shelf label displays an enlarged image of the product identification information as the photographing image.

According to this, by displaying an enlarged image of the product identification information, a photographed image suitable for reading the image in the information reading process is obtained.

In a twelfth aspect of the invention, the electronic shelf label displays the image for photographing by continuously switching display patterns.

According to this, by continuously switching and displaying the display pattern, it becomes easy to acquire information in the information reading process even when the image display area of the electronic shelf label is relatively small.

A thirteenth aspect of the present invention is a product information acquisition system in which a processor executes processing for acquiring information relating to the arrangement of a plurality of products arranged on a shelf, wherein the shelf stores the products grouped for each same product. a plurality of shelf labels arranged corresponding to product groups and containing product identification information about the products; and a position display section arranged at a predetermined reference position on the shelf and containing identification information of the reference position. , the processor obtains a photographed image of the shelf photographed by a camera, and reads the image of the shelf label and the image of the position display unit included in the photographed image, thereby obtaining the product identification information and the reference position. and the position of the shelf label is calculated from the positional relationship between the image of the shelf label and the image of the position display unit in the photographed image, and the product identification information and the position information of the shelf label are calculated. , and the identification information of the reference position are stored in association with each other.

According to this, by reading the photographed image of the shelf including the shelf label and the position display portion, information in which the product identification information, the position information of the shelf label, and the identification information of the reference position are associated is obtained. The workload for obtaining information about the placement of the lined up products is reduced.

Embodiments of the present disclosure will be described below with reference to the drawings.

(First embodiment)
FIG. 1 is a configuration diagram of a product information acquisition system 1 according to the first embodiment. FIG. 2 is an explanatory diagram showing an example of the internal configuration of the store 3. As shown in FIG. In this embodiment, an example in which the product information acquisition system 1 is applied to products sold at a store 3 will be described.

As shown in FIG. 1 , the product information acquisition system 1 includes an in-store camera 5 , a store clerk camera 7 , and a product information acquisition device 9 . The in-store camera 5 , store clerk camera 7 , and product information acquisition device 9 can communicate with each other via a communication network 11 provided in the store 3 .

The in-store camera 5 generates a captured image (moving image or still image) of the display shelf 13 by constantly capturing images of the store (in particular, the display shelf 13). Images captured by the in-store camera 5 are sequentially transmitted to the product information acquisition device 9 via the communication network 11 . As shown in FIG. 2, a plurality of in-store cameras 5 are installed at locations suitable for photographing the front of each display shelf 13 (such as walls and ceilings of the store 3). An existing monitoring camera may be used as the in-store camera 5 .

The store clerk camera 7 is a portable camera used by the store clerk 15 (see FIG. 2). The store clerk 15 can generate a photographed image of the display shelf 13 in the same manner as the in-store camera 5 by sequentially photographing each display shelf 13 using the store clerk camera 7 . It should be noted that the timing and the number of times of photographing each display shelf 13 by the store clerk 15 can be appropriately changed according to the sales situation of the product.

The store clerk camera 7 is wirelessly connected to the communication network 11 via the router 17 , so that the photographed image of the display shelf 13 can be transmitted to the product information acquisition device 9 . Also, the store clerk camera 7 is communicably connected to the product information acquisition device 9 via the communication cable 21 , so that the photographed image of the display shelf 13 can be transmitted to the product information acquisition device 9 . Furthermore, the store clerk camera 7 can also transfer the photographed image of the display shelf 13 to the product information acquisition device 9 via a storage medium (USB flash drive, SD memory card, etc.).

In addition, in the product information acquisition system 1, it is sufficient if at least the photographed image of the display shelf 13 can be acquired, and one of the in-store camera 5 and the store clerk camera 7 may be omitted. Moreover, the store clerk camera 7 may be a camera operated by at least a person, and the user (photographer) of the camera is not limited to the store clerk 15 .

The product information acquisition device 9 is configured by a computer such as a personal computer or a server. The product information acquisition device 9 includes known hardware such as a processor, memory, and storage. In addition, the product information acquisition device 9 stores captured images of the display shelf 13 received from the in-store camera 5 and the store clerk camera 7 . Further, the product information acquisition device 9 acquires necessary information by reading the captured image of the display shelf 13, and stores the acquired information. As shown in FIG. 2, the product information acquisition device 9 is installed in the backyard 23 of the store 3, for example. However, the product information acquisition device 9 may be installed outside the store 3 .

Further, in the product information acquisition system 1, the product information acquisition device 9 and the camera (here, the in-store camera 5 or the store clerk camera 7) may be integrated. For example, the in-store camera 5 or store clerk camera 7 may be configured to perform all the functions of the product information acquisition device 9 . This eliminates the need to send and receive captured images and the like between the product information acquisition device 9 and the camera via the communication network 11 .

The display shelf 13 is a shelf on which a plurality of products sold at the store 3 are arranged. As shown in FIG. 2, a plurality of display shelves 13 are installed at intervals so as to face the aisle in the store. Each display shelf 13 is an object of information acquisition by the product information acquisition system 1 .

FIG. 3 is an explanatory diagram showing the configuration of the display shelf 13. As shown in FIG. FIG. 3 shows the configuration of the display shelf 13 viewed from the front. The display shelf 13 includes paired upper and lower walls 31 and 33, paired left and right walls 35 and 36, and a plurality of shelves 37A-37D.

The top wall 31 extends horizontally, and both ends thereof are connected to the left side wall 35 and the right side wall 36, respectively. The lower wall 33 also has the same configuration as the upper wall 31 .

The left side wall 35 extends vertically, and has a front surface provided with a plurality of position display portions M1, M3, and M5 spaced apart from each other. The position display portions M1, M3, and M5 are arranged at predetermined reference positions (three positions on the left side wall 35) on the display shelf 13, respectively. Further, the position display portions M1, M3, and M5 each include identification information of the corresponding reference position (hereinafter referred to as reference position ID). In this embodiment, the reference position ID is indicated as a two-dimensional code (here, QR code (registered trademark)) in the position display portions M1, M3, and M5. However, the reference position ID may be indicated in other forms (for example, barcodes such as JAN codes, characters, numbers, symbols, graphics, etc.). Each reference position is a position whose spatial coordinates (world coordinates) are known at least in the product information acquisition system 1 . Each reference position ID also functions as shelf identification information for specifying the display shelf 13 on which the position display portions M1, M3, and M5 are installed.

The position display portions M1, M3, and M5 are sheets attached to the front surface of the left side wall 35. As shown in FIG. However, the position display portions M1, M3, and M5 may be printed matter printed on the front surface of the left side wall 35. FIG. Alternatively, the position display units M1, M3, and M5 may be display devices attached to the front surface of the left wall 35 and displaying electronically rewritable screens. The configuration of this display device is, for example, the same as that of an electronic shelf label forming a shelf label C (see FIGS. 4 and 5), which will be described later.

The right side wall 36 extends in the vertical direction, and a plurality of position display portions M2, M4, and M6 are attached to the front surface of the right side wall 36 at intervals. The position display portions M2, M4, and M6 are arranged at predetermined reference positions (three positions on the right side wall 36) on the display shelf 13, respectively. The configuration of the position display portions M2, M4, and M6 is the same as that of the position display portions M1, M3, and M5 described above. The positions of the position display portions M2, M4, and M6 in the height direction are preferably the same as the positions of the position display portions M1, M3, and M5, respectively. For example, a virtual line connecting the representative position of the position display portion M1 and the representative position of the position display portion M2 (for example, a line connecting the center of gravity of each) is preferably horizontal.

The number and arrangement of the position display units M1 to M6 installed on the display shelf 13 can be changed as appropriate. However, it is preferable that the display shelf 13 is provided with at least two position display units. In addition, the positions of the position display parts M1 to M6 in the front-rear direction (the direction perpendicular to the paper surface of FIG. 3) are the same so that the positions of the position display parts M1 to M6 can be easily determined in the coordinates of the captured image. is preferred.

Each shelf 37A-37D extends horizontally and is supported at both ends by the left and right side walls 35,36. A product arrangement space is formed on the upper surface side of each of the shelf boards 37A to 37D, and a plurality of products 40 are arranged thereon. These commodities 40 are grouped by the same commodities (that is, placed close to each other).

A plurality of shelf labels C1-C10 are attached to the front surface of each shelf board 37A-37D at intervals. The shelf labels C1-C10 are arranged corresponding to a group of products 40 of the same product (that is, in the vicinity of those products). The shelf labels C1-C10 include at least product identification information (hereinafter referred to as product ID) about the products as information about the corresponding products.

The shelf labels C1-C10 are electronic shelf labels that display electronically rewritable screens. The shelf labels C1-C10 have a wireless communication function and can communicate with peripheral devices (here, the in-store camera 5, the store clerk camera 7, and the product information acquisition device 9). However, the shelf labels C1-C10 may be cards made of paper or resin. Further, the shelf labels C1 to C10 do not necessarily have to have the shape of a label, and may be of any type as long as they can fulfill the display function.

In the product information acquisition system 1, some of the shelf labels C1 to C10 may be arranged at reference positions on the display shelf 13 and configured as position display shelves including reference position IDs. As a result, some of the shelf labels C1-C10 are used instead of the position display units M1-M6, and the number of the position display units M1-M6 installed on the display shelf 13 is reduced (or omitted).

The positions of the shelf labels C1-C10 in the front-rear direction (the direction perpendicular to the paper surface of FIG. 3) are preferably the same as the positions of the position display portions M1-M6. This makes it easier to determine the positions of the shelf labels C1-C10 on the coordinates of the captured image. In this embodiment, the front surfaces of the left side wall 35 and the right side wall 36 on which the position display portions M1 to M6 are installed and the front surfaces of the shelf plates 37A to 37D on which the shelf labels C1 to C10 are installed are positioned in the front-rear direction. They are formed to be identical to each other.

The configuration of the display shelf 13 is not limited to that shown in FIG. 3, and various modifications are possible. Hereinafter, the position display sections M1 to M6 are collectively referred to as "position display section M" when there is no need to distinguish between them. Similarly, when there is no need to distinguish the shelf labels C1-C10, they are collectively referred to as "the shelf label C".

FIG. 4 is an explanatory diagram showing a display example of the shelf label C. As shown in FIG. As shown in FIGS. 4A to 4C, the display screen 43 of the shelf label C configured as an electronic shelf label can be electronically rewritten.

In this embodiment, the shelf label C indicates the product ID as a two-dimensional code 51 (here, QR code (registered trademark)). However, the product ID may be indicated in other forms (for example, barcodes such as JAN codes, characters, numbers, symbols, graphics, color codes, etc.).

FIG. 4A shows an example of normal display of the shelf label C when the product 40 is sold. The display screen 43 includes a two-dimensional code 51 including product ID information, a product name (here, orange juice) that is the name of each product 40, and a product price (here, 150 yen). A customer at the store 3 can purchase the product corresponding to the shelf label C by referring to the product name and product price.

When the normally displayed shelf label C is photographed by a camera (in this case, the in-store camera 5 and the store clerk camera 7), the photographed image of the shelf label C includes not only the two-dimensional code 51 but also the product name and product price. Contains information. On the other hand, in the product information acquisition system 1, the photographed image of the shelf label C should include at least product ID information (here, the two-dimensional code 51).

Therefore, in the product information acquisition system 1, the display screen 43 of the shelf label C when photographed by the camera is changed from the normal display (that is, the image displayed when not photographed) shown in FIG. It is possible to switch to the display at the time of shooting (that is, the image for shooting) shown in B) and (C). Such switching of the display on the display screen 43 can be performed manually by the store clerk 15 or the like, for example. Alternatively, a peripheral device (here, the in-store camera 5, the store clerk camera 7, or the product information acquisition device 9) transmits a control command (control signal) for instructing display switching to the shelf label C via wireless communication. , the display of the display screen 43 may be switched when the shelf label C receives it.

In the display (limited display) at the time of shooting shown in FIG. 4B, only the two-dimensional code 51 including the product ID information is displayed on the display screen 43, and other information related to the product included in the normal display (here, product name and product price) are not displayed.

In this manner, by hiding at least a portion of the image (here, corresponding to FIG. 4A) that is displayed when not photographing (here, other than the two-dimensional code 51), information reading processing, which will be described later, is performed. A photographed image (image of shelf label C) suitable for image reading in (see step ST104 in FIG. 16) is obtained.

In the display (enlarged display) shown in FIG. 4(C), only the two-dimensional code 51 including the product ID information is enlarged and displayed on the display screen 43, and other information included in the normal display is not displayed.

In this way, by enlarging and displaying the product ID (here, the two-dimensional code 51), a photographed image (image of the shelf label C) suitable for reading the image in the information reading process is obtained. The empty space of the display screen 43 around the enlarged two-dimensional code 51 may include other information about the product (for example, product name and product price).

FIG. 5 is an explanatory diagram showing a modified example of the display of the shelf label C when it is photographed. FIGS. 4B and 4C described above show an example in which product ID information (here, two-dimensional code) is emphasized on the display screen 43 as a display when the shelf label C is photographed. On the other hand, the product ID information may be indicated by images (videos) continuously displayed on the display screen 43 .

As shown in FIGS. 5A to 5D, the shelf label C can display a display pattern (in this case, display color) that switches at predetermined time intervals across the display screen 43 when photographed by a camera. . As a result, in the information reading process (see step ST104 in FIG. 16), which will be described later, the continuous image of the display pattern on the display screen 43 (here, the time-series change of the display color) uniquely corresponds to the display pattern. Information on the product ID to be used is acquired.

In this way, by continuously switching the display pattern and displaying it, even if the size of the image display area of the shelf label C (that is, the image of the display screen 43) is relatively small, the information can be obtained in the information reading process. easier. Note that the area where such a display pattern is displayed does not necessarily have to be the entire display screen 43 . In addition, although only four display patterns (A) to (D) are shown in FIG. 5, more display patterns can be displayed. Furthermore, instead of a display pattern based on a plurality of different display colors, a display pattern based on a plurality of simpler figures than a two-dimensional pattern may be used. Alternatively, a display pattern may be used in which the display (light source) of the shelf label C is switched by blinking at regular time intervals.

FIG. 6 is a functional block diagram showing the configuration of the product information acquisition device 9. As shown in FIG. The product information acquisition device 9 includes a communication section 53 , an electronic medium reading section 55 , an input section (input device) 57 , a display section (display device) 59 , a storage section 61 and a control section 63 .

The communication unit 53 includes a network interface (not shown) connectable to the communication network 11, and performs wired communication with other devices (here, the in-store camera 5 and the store clerk camera 7). However, the communication unit 53 may perform wireless communication with other devices by being connected to the communication network 11 (router 17) in compliance with a known wireless communication standard. In that case, the communication unit 53 has known hardware (not shown) such as an antenna, a modem, and a wireless communication circuit for performing wireless communication with other devices. The communication unit 53 also has an interface to which the communication cable 21 is connected.

The electronic medium reading unit 55 has an interface to which a storage medium (USB flash drive, SD memory card, etc.) can be connected. Further, the electronic medium reading unit 55 can function as a reader/writer that reads data from a storage medium and writes data to the storage medium.

The input unit 57 includes an input device such as a keyboard or a touch panel that receives input operations from a user (here, the administrator of the product information acquisition system 1 or the store clerk 15), and includes known hardware.

The display unit 59 includes a display device such as a liquid crystal display, and has known hardware. The display unit 59 may be configured by a touch panel together with the input unit 57 .

The storage unit 61 includes storage devices such as HDD and SSD. The storage unit 61 stores information 71 related to the display shelf 13 (hereinafter referred to as shelf information; see FIGS. 10A and 11), information 71 related to the product 40 (hereinafter referred to as product information; 15.) 73, product arrangement information 75 (see FIG. 17), shelf/product management information 77, and the like are stored. The shelf/product management information 77 includes a world coordinate table (see FIG. 10B), a product table (see FIG. 12B), and correction coefficient information (see FIG. 14) used to calculate the product position. ) and so on.

The control unit 63 has an image acquisition unit 81 , an image processing unit 83 , an information reading unit 85 , a product position calculation unit 87 and a product placement information generation unit 89 .

The image acquisition unit 81 executes processing (image acquisition processing) for acquiring, via the communication unit 53, a photographed image of the display shelf 13 generated by the camera (here, the in-store camera 5 and the store clerk camera 7). .

The image processing unit 83 executes image processing on the acquired photographed image. Image processing includes image synthesizing processing for a plurality of captured images, including multi-frame super-resolution processing, HDR (High Dynamic Range Imaging) processing, and the like. In addition, the image processing includes image processing for one captured image, including intra-frame super-resolution processing, HDR processing, edge enhancement processing, scaling processing, processing using an image restoration filter, and noise removal processing. . Furthermore, in image processing, image correction processing such as contrast correction, brightness correction, color correction, gamma correction, and conversion to a monochrome image can be executed in addition to the image composition processing and image processing processing. Such image processing improves the accuracy with which the information reading section 85 reads the photographed image.

It should be noted that the image processing by the image processing section 83 may be omitted when the captured image to be acquired is good (that is, when there is no problem in reading the captured image by the information reading section 85).

The information reading section 85 reads the image of the shelf label C and the image of the position display section M included in the photographed image to execute processing (information reading processing) of acquiring the product ID and the reference position ID. At this time, the information reading section 85 can recognize (detect) the image of the shelf label C and the image of the position display section M from the entire photographed image based on a known image recognition technique. Further, the information reading unit 85 acquires information on the image name (file name of the captured image) added to the captured image and the shooting date and time. Further, the information reading unit 85 acquires the position information (X coordinate, Y coordinate) of the image of the shelf label C and the position information (X coordinate, Y coordinate) of the image of the position display unit M based on the coordinates of the captured image. do.

As will be described later, the image processing section 83 and the information reading section 85 can process one or more captured images (or video frames).

Based on the positional relationship between the image of the shelf label C and the image of the position display section M in the photographed image, the product position calculation unit 87 calculates the position of each shelf label C as the corresponding product 40 (more strictly, the same product group). A process (position calculation process) for calculating the position of is executed.

The product placement information generation unit 89 generates product information 73 (including product ID), information on the position of the shelf label C (corresponding to the position of the product 40), and shelf information 71 (reference position ID) as information related to the placement of the product 40. ) is stored in the storage unit 61 (information storage processing).

The control unit 63 includes one or more processors (CPU, GPU, etc.) as hardware. Each function of the control unit 63 is realized by the processor executing a predetermined control program. In addition, the control unit 63 can control not only the processing of each unit described above, but also the overall operation of the product information acquisition device 9 .

FIG. 7 is an explanatory diagram showing an example of a guide display during photographing by the clerk's camera 7. As shown in FIG. In the product information acquisition system 1, a guide display is output to the monitor 90 of the clerk camera 7 when the display shelf 13 is photographed.

In the example shown in FIG. 7A, a guide frame 91 in which the display shelf 13 should be placed is displayed on the monitor 90 as a guide display. Further, on the monitor 90, a guide 92 for the photographer regarding a guide frame 91 is displayed as a guide display. By following the guide display and taking a picture so that the entire display shelf 13 fits within the guide frame 91, the shelf label C and the position display portion M can be reliably included in the frame (that is, the photographed image). As a result, even when an unspecified person (for example, a plurality of store clerks 15) takes pictures of the shelf with a camera, it is possible to acquire a photographed image suitable for image reading.

Depending on the size of the display shelf 13, if the entire display shelf 13 is accommodated within the guide frame 91, the size of the images of the shelf label C and the position display portion M in the photographed image may be reduced, making it difficult to read the image. sometimes not. Therefore, the monitor 90 may indicate that a part of the display shelf 13 (the part including the shelf label C and the position display portion M) should be accommodated within the guide frame 91 as a guide 92 to the photographer. . In that case, the photographer individually photographs a plurality of portions on one display shelf 13, so that all shelf labels C and position display portions M on the display shelf 13 are included in one of the plurality of photographed images. You may allow

In the example shown in FIG. 7B, a guide 93 is displayed on the monitor 90 as a guide display to urge the photographer to take pictures with a different angle. The photographer can obtain a plurality of photographed images with different angles by changing the angle and photographing according to the guidance 93 . As a result, even when an unspecified person photographs the shelf with a camera, it is possible to easily acquire a plurality of photographed images from different angles.

Such a guide display is executed according to an application program pre-installed in the clerk's camera 7, for example. In the case of the in-store camera 5 fixed inside the store, the angle can be changed by panning and tilting.

FIG. 8 is an explanatory diagram showing an example of image processing and information reading processing for one photographed image. The product information acquisition device 9 can appropriately change the content of image processing for the captured image and the number of times of information reading processing (here, corresponding to the number of images to be processed).

In the example shown in FIG. 8A, an image processing process and an image synthesizing process are sequentially performed on one photographed image, and an information reading process is performed on the synthesized image.

In the image processing process, a plurality of images (hereinafter referred to as “corrected images”) are processed by performing image correction (contrast correction, brightness correction, color correction, etc.) under different conditions on a single photographed image multiple times. .) is generated. Further, intra-frame super-resolution processing, HDR processing, edge enhancement processing, enlargement/reduction processing, processing using an image restoration filter, noise removal processing, and the like are performed on these corrected images as necessary, so that the final , a plurality of images (hereinafter referred to as processed images) are generated.

In the next image synthesizing process, one or more image groups (including two or more processed images) are formed from a plurality of processed images, and each image group is processed. One or more images (hereinafter referred to as composite images) are generated by this image synthesis processing.

In the subsequent information reading process, the process is executed for each composite image. Therefore, when there are K composite images (K is an integer equal to or greater than 1), the information reading process is executed K times. In each information reading process, at least part of the images of the plurality of shelf labels C and the images of the plurality of position display portions M may not be read. Therefore, in the product information acquisition device 9, by changing the content of the image processing so as to increase the number of times K of information reading processing (that is, the number of synthesized images) as necessary, more shelves can be obtained from a plurality of synthesized images. The image of tag C and position display portion M can be read.

The product information acquisition device 9 acquires the maximum information based on K times of information reading processing. In other words, the information obtained by the product information acquisition device 9 from one photographed image (here, the information of the reference position ID and the product ID) excludes redundant information from all the information read in the information reading process K times. It is what I did.

In the example shown in FIG. 8B, image processing is performed on one shot image, and information reading processing is performed on the processed image generated thereby.

In the image processing, a plurality of processed images are generated as in the case of FIG. 8(A). In the subsequent information reading process, each processed image is processed. Therefore, when there are K processed images, the information reading process is executed K times. Further, in the product information acquisition device 9, as in the case of FIG. 8A, the contents of image processing can be changed so as to increase the number of times of information reading processing as necessary.

FIG. 9 is an explanatory diagram showing an example of image processing and information reading processing for a plurality of captured images. Matters not specifically mentioned below are the same as in FIG.

The multiple shot images to be subjected to the information reading process are multiple shot images under different shooting conditions. The shooting conditions include, for example, the above-described camera angle, exposure amount, subject size (zoom magnification), and the like. When the angle is changed as a photographing condition, even if the image of the shelf label C or the image of the position display portion M in a certain photographed image becomes unclear due to the reflection of light, etc., the angle is different. From other captured images, the possibility of acquiring the information of those images increases.

In the example shown in FIG. 9A, image composition processing is performed on a plurality of (N: N is an integer equal to or greater than 2) photographed images, and information reading processing is performed on the composite image generated thereby. executed.

In image synthesizing processing, one or more image groups are formed from a plurality of captured images, and processing is performed on each image group. One or more synthesized images are generated by this image synthesizing process. In the subsequent information reading process, the process is executed for each composite image.

In the example shown in FIG. 9B, image processing is performed on a plurality of (N) captured images, and information reading processing is performed on the processed images generated thereby.

In the image processing process, a plurality of processed images are generated by performing the same processing as described above on a plurality of photographed images as necessary. Here, the same number (N sheets) of processed images as the photographed images are generated. In the subsequent information reading process, each processed image is processed.

In the example shown in FIG. 9C, image synthesizing processing and image processing processing are sequentially performed on a plurality of (N) captured images, and information reading processing is performed on the processed image generated thereby. be.

In the image synthesizing process, as in the case of FIG. 9A, one or more image groups are formed from a plurality of photographed images, and the process is executed for each image group. One or more synthesized images are generated by this image synthesizing process. In the next image processing, one or more processed images are generated by performing the same processing as described above on each synthesized image as required. In the subsequent information reading process, each processed image is processed.

In the example shown in FIG. 9D, image processing and image composition processing are sequentially performed on a plurality of (N) captured images, and information reading processing is performed on the composite image generated thereby. be.

In the image processing, as in the case of FIG. 9B, a plurality of (N) processed images are generated. In the next image synthesizing process, one or more image groups are formed from a plurality of processed images, and each image group is processed. One or more synthesized images are generated by this image synthesizing process. In the subsequent information reading process, the process is executed for each composite image.

In the example shown in FIG. 9(E), the first image processing, the image synthesis processing, and the second image processing are sequentially executed on a plurality of (N) captured images, thereby generating Information reading processing is performed on the processed image.

In the first image processing, a plurality of (N) processed images are generated as in the case of FIG. 9B. In the subsequent image synthesizing process, one or more image groups are formed from a plurality of processed images, and each image group is processed. One or more synthesized images are generated by this image synthesizing process. In the following second image processing, processing similar to the first image processing is performed on one or more composite images. One or more processed images are generated by this image processing. In the subsequent information reading process, each processed image is processed.

FIG. 10 is an explanatory diagram showing an example of the shelf information 71 and a world coordinate table related thereto. FIG. 11 is an explanatory diagram showing an example of shelf information 71 including world coordinates.

The shelf information 71 is sequentially generated for each of the position display portions M1 to M6 based on the information read from the photographed image of the display shelf 13, and stored in the storage portion 61. FIG. As shown in FIG. 10A, the shelf information 71 includes a reference position ID corresponding to each position display section M1 to M6, and an X coordinate (within the image) and a Y coordinate (inside the image) associated with each reference position ID. (within the image), the image name, and the date and time the image was taken.

The reference position ID is read from each of the images of the position display portions M1-M6 included in the captured image. The X coordinate (inside the image) and the Y coordinate (inside the image) are the values of the X coordinate and the Y coordinate of each of the position display portions M1 to M6 (representative positions) obtained based on the coordinates of the captured image. The image name and shooting date and time are information of the file name and shooting date and time added to the shot image, respectively.

The world coordinate table is information related to each of the position display units M1 to M6, and is stored in the storage unit 61 as the shelf/merchandise management information 77 before the information reading process is executed. As shown in FIG. 10B, the world coordinate table includes information such as reference position ID, X coordinate (world coordinate) and Y coordinate (world coordinate), store number, shelf location number, category number, and the like. . The shelf location number is an identifier for the area in the store where the display shelf 13 is arranged. The category number is the identifier of the store 3 in which the display shelf 13 is installed.

Each piece of information in the world coordinate table is associated with the shelf information 71 relating to each of the position display units M1-M6 by means of the reference position ID. For example, the reference position ID "Shelf01a" in the world coordinate table matches the reference position ID of the shelf information 71 regarding the position display portion M1. The matching of the reference position IDs means that the position display unit M1 is installed at the reference position of the X coordinate (world coordinates) and Y coordinate (world coordinates) "(1000, 2000)" associated with the reference position ID "Shelf01a". indicates that This also indicates that the position display unit M1 is installed on the display shelf 13 placed in the store with the store number "100" associated with the reference position ID "Shelf01a". This also indicates that the shelf location number of the display shelf 13 on which the position display portion M1 is installed is "A10". Furthermore, this indicates that the category number of the store in which the position display section M1 is installed is "309". The same applies to information associated with other reference position IDs (Shelf02a, etc.) in the world coordinate table.

The shelf information 71 is updated as information including world coordinates as shown in FIG. 11 by being associated with the world coordinate table as described above.

FIG. 12 is an explanatory diagram showing an example of the product information 73 and a product table related thereto.

The product information 73 is sequentially generated for each of the shelf labels C1 to C10 based on the information read from the photographed image of the display shelf 13, and stored in the storage unit 61. FIG. As shown in FIG. 12A, the product information 73 includes product IDs corresponding to each of the shelf labels C1 to C10, and X coordinates (in the image) and Y coordinates (in the image) associated with each product ID. , the name of the image, and the date and time the image was taken.

The product ID is read from each of the images of the shelf labels C1-C10 included in the photographed image. The X-coordinate (inside the image) and Y-coordinate (inside the image) are the values of the X-coordinate and Y-coordinate of the shelf labels C1-C10 (representative positions) obtained based on the coordinates of the captured image. The image name and shooting date and time are information of the file name and shooting date and time added to the shot image, respectively.

The product table is information related to each product 40, and is stored in the storage unit 61 as the shelf/product management information 77 before the information reading process is executed. As shown in FIG. 12B, the product table includes information such as product IDs, product names, category numbers, and category names associated with each product ID. The category number and category name are the identifier and name of the category to which each product 40 belongs, respectively. The product table may further include identification information of the store 3 associated with the product ID.

Each piece of information in the product table is associated with product information on each shelf label C1-C10 by the product ID. For example, the product ID "F714ACC0" in the product table matches the product ID of the product information 73 regarding the shelf label C1. This product ID match indicates that the product corresponding to the shelf label C1 is related to the product name “orange juice”, category number “101”, and category name “beverages” associated with the product ID “F714ACC0”. show.

FIG. 13 is an explanatory diagram showing a method of calculating the position of the shelf label C on the display shelf 13. As shown in FIG. FIG. 14 is an explanatory diagram showing an example of correction coefficients used to calculate the position of the shelf label C. As shown in FIG. FIG. 15 is an explanatory diagram showing an example of product information including world coordinates.

As described above, the position (reference position) at which each position display unit M is installed is predetermined (that is, the world coordinate values are known). Therefore, the product information acquisition device 9 can calculate the position of the shelf label C as the position (world coordinates) of the corresponding product from the positional relationship between the image of the shelf label C and the image of the position display section M in the captured image. can.

FIG. 13 shows an example in which the position of the image of the shelf label C2 (here, the two-dimensional code 51) in the captured image is calculated from the relationship between the positions of the images of the position display portions M1 and M2.

The position display units M1 and M2 are actually arranged side by side in the horizontal direction as shown in FIG. 3, but in FIG. An example is shown (taken with a camera tilted with respect to direction).

The product information acquisition device 9 (product position calculation unit 87) calculates the distance L1 between the position display unit M1 (representative position) and the position display unit M2 (representative position) in the captured image by the correction coefficient ( distance correction) (here, multiplied by a correction factor) to correct the distance L1w in world coordinates. This correction coefficient (distance correction) is the distance between the position display section M1 and the position display section M2 calculated based on the world coordinates, and the distance between the position display section M1 and the position display section M2 calculated based on the coordinates of the captured image. It is obtained from the ratio of the distances between Similarly, the product information acquisition device 9 uses the correction coefficient (distance correction) shown in FIG. can be corrected to the distance L2w in the world coordinates.

In addition, the product information acquisition device 9 detects the angle α between the straight line connecting the position display portion M1 and the position display portion M2 in the photographed image and the horizontal line, and the straight line connecting the position display portion M1 and the shelf label C2 and the horizontal line. and the angle β to be formed can be calculated.

In this case, the angles α and β in the image may differ from the angles in the actual space depending on the inclination of the camera with respect to the display shelf 13 at the time of photographing. Therefore, the product information acquisition device 9 can correct those angles using the correction coefficients (angle correction) shown in FIG. 14, for example. This correction coefficient (angle correction) is determined by tilt information of the camera body obtained from the camera, or a position display section (for example, position display section M3 or M4) aligned vertically with the position display section M1 or M2 in the captured image. can be calculated based on the positional relationship with

As a result, the product information acquisition device 9 connects the distance L2w in world coordinates between the position display portion M1 (representative position) and the shelf label C2 (representative position) in the captured image, and the position display portion M1 and the position display portion M2. The world coordinates of the shelf label C2 can be calculated based on the world coordinates of the position display section M1 and the position display section M2 from the angle α+β between the straight line and the straight line connecting the position display section M1 and the shelf label C2. The world coordinates of this shelf label C2 can be regarded as the position of the corresponding product 40 (more strictly, the same product group).

In this way, the product information acquisition device 9 may execute the position calculation process only when at least one image of the shelf label C and two or more images of the position display portions M are read from one captured image. . This simplifies the process of associating the coordinates of the captured image with the world coordinates, and the position of the shelf label C can be easily calculated.

The product information acquisition device 9 can calculate the position of the image of each shelf label C included in each captured image from the relationship with the position display section M included in the same captured image by the same method as described above.

When the world coordinates of the shelf label C calculated as described above are calculated, the product information 73 is updated as information including the world coordinates (that is, product position information) as shown in FIG.

FIG. 16 is a flow diagram showing the flow of processing for generating product placement information by the product information acquisition system 1. As shown in FIG. FIG. 17 is an explanatory diagram showing an example of product placement information.

First, when a photographed image of the display shelf 13 is generated by the camera (here, the in-store camera 5 or the store clerk's camera 7) (ST101), the product information acquisition device 9 acquires the photographed image (ST102).

Next, the product information acquisition device 9 executes image processing on the photographed image acquired from the camera (ST103). However, this image processing may be omitted.

Subsequently, the product information acquisition device 9 acquires the product ID and the reference position ID by reading the image of the shelf label C and the image of the position display section M included in the captured image (ST104). At this time, the product information acquisition device 9 can also acquire information such as the image name and the shooting date and time added to the captured image.

In the information reading process of step ST104, the product information acquisition device 9 determines whether necessary information (reference position ID, product ID, etc.) has been read from the captured image, and notifies the administrator of the determination result. can be notified. For example, the product information acquisition device 9 can notify the administrator by displaying the result of the determination of success or failure of the information reading process on the display unit 59 . As a result, the product manager can be made aware of whether the reading of the image of the shelf label C and the image of the position display portion M has been successful, and the manager can be prompted to take action when reading fails.

After that, the product information acquisition device 9 generates the shelf information 71 and the product information 73 based on the reference position ID and the product ID acquired in step ST104 (ST105).

Next, the product information acquisition device 9 calculates the position of each shelf label C as the position of the corresponding product 40 from the positional relationship between the image of the shelf label C and the image of the position display section M in the captured image (ST106). . Information (world coordinates) of the position of each shelf label C calculated there is added to the product information 73 .

After that, the product information acquisition device 9 generates product arrangement information from the product information 73 including the shelf information 71 and information on the position of the shelf label C (ST107). For example, as shown in FIG. 17, the product arrangement information includes shelf information of two or more position display units M used for calculating the position of the shelf label C and product information related to the corresponding shelf label C. As shown in FIG. The product information acquisition device 9 stores the generated product placement information in the storage unit 61 (ST108).

As described above, in the product information acquisition system 1, by reading the photographed image of the display shelf 13 including the shelf label C and the position display portion M, the product ID, the information of the position of the shelf label C, and the identification information of the shelf are obtained. In order to acquire product placement information associated with a functioning reference position ID, to acquire information related to the placement of products arranged on shelves (information indicating where each product is placed on which shelf) Work load is reduced.

FIG. 18 is a configuration diagram showing a modification of the product information acquisition system 1 shown in FIG. In FIG. 18, the same reference numerals are assigned to the same components as in the product information acquisition system 1 according to the first embodiment described above. Matters not specifically mentioned below regarding the modified example are the same as those of the above-described first embodiment.

As shown in FIG. 18, in the product information acquisition system 1 according to the modification, a plurality of stores 3 (in-store cameras 5 or store clerk cameras 7) communicably connected to the product information acquisition device 9 via the Internet 100 are installed. It is The captured images captured by the in-store camera 5 or the store clerk camera 7 of the store 3 are sequentially transmitted to the product information acquisition device 9 via the Internet 100 . In addition, identification information (for example, the identifier and name of the store 3) related to the store 3 where the captured image was captured is received from the store 3 (that is, the store camera 5, the store clerk camera 7, or the computer used at the store 3). The information is sequentially transmitted to the information acquisition device 9 .

With such a configuration, in the product information acquisition system 1 according to the modified example, one product information acquisition device 9 sequentially acquires the photographed images of the display shelves 13 respectively installed in the plurality of stores 3, and for each store 3 Product placement information can be acquired for each.

(Second embodiment)
FIG. 19 is a functional block diagram showing the configuration of the product information acquisition device 9 in the product information acquisition system according to the second embodiment. FIG. 20 is an explanatory diagram showing the configuration of the display shelf 13 in the product information acquisition system according to the second embodiment. In the diagrams relating to the second embodiment, the same reference numerals are given to the same components as in the first embodiment (including modifications). In addition, regarding the product information acquisition system according to the second embodiment, items not particularly mentioned below are the same as those of the first embodiment (including modifications).

In the product information acquisition system according to the second embodiment, as shown in FIG. 20, shelf labels C1 to C10 are installed on the display shelf 13 in the same manner as in the first embodiment. On the other hand, the display shelf 13 does not necessarily have the above-described position display section M, and a shelf information display section S including identification information of the display shelf 13 (hereinafter referred to as a shelf ID) is installed as necessary.

In the shelf information display section S, the shelf ID is displayed as a two-dimensional code (here, QR code (registered trademark)) similar to that in the position display section M. On the other hand, the shelf information display section S does not need to be arranged at a reference position like the position display section M, and is installed at an appropriate position in front of the display shelf 13 so as to facilitate photographing with a camera. Although only one shelf information display section S is shown in FIG. 20, the number and arrangement of the shelf information display sections S can be changed as appropriate.

In the product information acquisition device 9, the information reading unit 85 of the control unit 63 executes processing (information reading processing) for acquiring the product ID by reading the image of the shelf label C included in the captured image. Further, when the shelf information display section S is installed on the display shelf 13, the information reading section 85 can also acquire the shelf ID by reading the image of the shelf information display section S. FIG. Furthermore, the information reading unit 85 can acquire information such as the image name of the captured image and the date and time of shooting in the information reading process.

The control unit 63 also has a product search information generation unit 95 . The product search information generating unit 95 generates product search information 96 in which the product ID and the photographed image from which the product ID is acquired are associated with each other. The generated product search information is stored in the storage unit 61 .

FIG. 21 is a flow chart showing the flow of processing for generating product search information 96 by the product information acquisition system. FIG. 22 is an explanatory diagram showing an example of the product search information 96. As shown in FIG. FIG. 23 is an explanatory diagram showing a modification of the product search information 96 shown in FIG.

Here, steps ST201 to ST203 are the same as steps ST101 to ST103 shown in FIG. 16, respectively. Subsequently, the product information acquisition device 9 acquires the product ID by reading the image of the shelf label C included in the captured image (ST204). At this time, the product information acquisition device 9 can also acquire information such as the image name and the shooting date and time added to the captured image.

After that, the product information acquisition device 9 generates product information based on the product ID and the like acquired in step ST204 (ST205). The product information generated in step ST205 corresponds to the product information 73 shown in FIG. 12 from which the X-coordinate (inside the image) and Y-coordinate (inside the image) information has been removed.

Next, the product information acquisition device 9 generates product search information 96 in which the product information including the product name and product ID and the photographed image from which the product information is acquired are associated (ST206). For example, as shown in FIG. 22, the product search information 96 includes information such as the product ID associated with each shelf label C, the image name of the photographed image, the photographing date and time of the photographed image, and the product name. The product search information 96 may further include identification information of the store 3 . The product information acquisition device 9 stores the generated product search information 96 in the storage unit 61 (ST207).

Based on the product search information 96, the manager can search for a photographed image of a desired product. For example, the administrator inputs the product ID and product name into the input unit 57 of the product information acquisition device 9 (or a device such as a computer used by the administrator provides information on the product name). The acquisition device 9 can search the product search information 96 for a photographed image corresponding to the input product name and product ID. The product information acquisition device 9 can output (display) the retrieved photographed image to the display unit 59 .

Thereby, the manager can visually confirm the placement status of the desired product 40 on the display shelf 13 . It should be noted that such a search process for photographed images of products does not necessarily have to be executed by the product information acquisition device 9, and may be executed by another information processing device that stores the product search information 96. FIG.

The product search information 96 may include at least one piece of information that can be used as product identification information for the administrator to search for a photographed image of a desired product. Here, like the product ID of each product 40, the product name can be used as the product identification information.

Further, when the shelf information display section S is installed on the display shelf 13, the shelf ID is acquired together with the commodity ID in step ST204. Thereafter, in step ST206, product information including the product name and product ID, the photographed image obtained by acquiring the product information, and the product search information 96 associated with the shelf ID are generated.

In this case, for example, as shown in FIG. 23, the product search information 96 includes, in addition to the product ID associated with each shelf label C, the image name of the photographed image, the photographing date and time of the photographed image, the product name, and the like. ID information is included. As a result, for example, when the administrator inputs a shelf ID into the input unit 57 of the product information acquisition device 9, the product information acquisition device 9 converts the photographed image corresponding to the input shelf ID into the product search information. You can search from 96.

The product search information 96 may further include identification information of the store 3 . As the product search information 96, for example, information such as a product ID and product name may be added to the photographed image as Exif (Exchangeable image file format) information or digital watermark information.

It should be noted that in the above example, some of the components of the product information acquisition system 1 according to the first embodiment are omitted in the product information acquisition system according to the second embodiment. However, without being limited to this, the product information acquisition system according to the second embodiment may include all the components of the product information acquisition system 1 according to the first embodiment.

In that case, the display shelf 13 including the position display portion M shown in FIG. 3 is used as the object to be photographed. The reference position ID is also used as shelf identification information (shelf ID) for specifying each display shelf 13 . In addition, in the product information acquisition device 9 shown in FIG. 6 , the product placement information generation unit 89 of the control unit 63 also functions as the product search information generation unit 95 . Furthermore, the product arrangement information 75 (see FIG. 17) stored in the storage unit 61 includes information similar to the product search information 96 (with the corresponding product name added), which is used for product search. be done.

As described above, the present disclosure has been described based on specific embodiments, but these embodiments are merely examples, and the present disclosure is not limited by these embodiments.

For example, in the above-described embodiment, the products whose information is to be acquired by the product information acquisition system 1 are arranged on the display shelf 13 installed in the store 3, but the shelves on which the products are arranged are It is not limited to three display shelves 13 . For example, products for which information is to be obtained may be arranged on shelves installed in a product warehouse. In addition, the product for which information is to be obtained is not limited to the product sold directly to the customer, and may be a part of the product sold to the customer (for example, parts constituting the product to be sold).

Not all of the constituent elements of the product information acquisition method and product information acquisition system according to the present disclosure shown in the above embodiments are essential, and can be appropriately selected as long as they do not depart from the scope of the present invention. is.

The product information acquisition method and product information acquisition system according to the present disclosure have the effect of reducing the workload for acquiring information about the arrangement of products arranged on the shelf, and the arrangement of multiple products arranged on the shelf. It is useful as a product information acquisition method and product information acquisition system for acquiring information related to the product.

1: Product information acquisition system 3: Store 5: In-store camera 7: Clerk camera 9: Product information acquisition device 11: Communication network 13: Display shelf 15: Clerk 17: Router 21: Communication cable 23: Backyard 31: Upper wall 33 : Lower wall 35: Left wall 36: Right wall 37A-37D: Shelf board 40: Product 43: Display screen 51: Two-dimensional code 53: Communication unit 55: Electronic medium reading unit 57: Input unit (input device)
59: Display unit (display device)
61: storage unit 63: control unit 71: shelf information 73: product information 75: product arrangement information 77: product management information 81: image acquisition unit 83: image processing unit 85: information reading unit 87: product position calculation unit 89: product Placement information generator 90: Monitor 91: Guide frame 92: Guidance 93: Guidance 95: Product search information generator 96: Product search information 100: Internet C1-C10: Shelf label M1-M6: Position display unit S: Shelf Information display

Claims (13)

A product information acquisition method for acquiring information about the arrangement of a plurality of products arranged on a shelf,
The shelf has
a plurality of shelf labels arranged corresponding to the group of products grouped for each same product and containing product identification information related to the product;
a position display unit arranged at a predetermined reference position of the shelf and containing identification information of the reference position;
an image acquisition process for acquiring a photographed image of the shelf photographed by a camera;
an information reading process of acquiring the product identification information and the identification information of the reference position by reading the image of the shelf label and the image of the position display unit included in the captured image;
position calculation processing for calculating the position of the shelf label from the positional relationship between the image of the shelf label and the image of the position display unit in the captured image;
and an information storage process for storing the product identification information, the information on the position of the shelf label, and the identification information on the reference position in association with each other. In the position calculation process, the position of the shelf label is calculated only when one or more images of the shelf label and images of the plurality of position display units are read from one of the captured images in the information reading process. , The product information acquisition method according to claim 1. A part of the shelf label constitutes a position display shelf label arranged at the reference position and containing identification information of the reference position,
3. The method according to claim 1, wherein in the position calculation process, the position of the shelf label is calculated from the positional relationship between the image of the position display shelf label and the image of the other shelf label in the photographed image. How to obtain the product information described. In the image acquisition process, a plurality of captured images under different imaging conditions are acquired,
In the information reading process, the product identification information and the identification information of the reference position are acquired by respectively reading the image of the shelf label and the image of the position display unit included in the plurality of photographed images. The product information acquisition method according to any one of claims 1 to 3. In the image acquisition process, a plurality of shot images with different shooting conditions are acquired,
further comprising image processing performed before the information reading process;
In the image processing, at least one of an image synthesizing process of synthesizing a plurality of the captured images or a plurality of images derived from the captured images and an image processing process of processing the captured images or an image derived from the captured images is performed. 4. The product information acquisition method according to any one of claims 1 to 3, which is executed. 6. The product information acquisition method according to claim 4, wherein said photographing conditions include at least one of an angle of said camera and an exposure amount. The camera has a monitor that displays the subject during shooting,
7. The product information acquisition method according to any one of claims 1 to 6, wherein when the shelf is photographed, the monitor displays a guide frame in which the shelf as the subject is to be placed. 8. The information reading process according to any one of claims 1 to 7, wherein success or failure of reading of the image of the shelf label and the image of the position display unit is determined, and the result of the determination is notified to an administrator. method of obtaining product information. The plurality of shelf labels include electronic shelf labels,
9. The product information acquisition method according to any one of claims 1 to 8, wherein said electronic shelf label displays a photographing image different from that during non-photographing when said shelf is photographed by said camera. 10. The product information acquisition method according to claim 9, wherein said electronic shelf label hides at least part of an image other than said product identification information displayed when not photographing in said image for photographing. 11. The product information acquisition method according to claim 9, wherein said electronic shelf label displays an enlarged image of said product identification information as said image for photographing. 10. The product information acquisition method according to claim 9, wherein said electronic shelf label continuously switches display patterns to display said image for photographing. A product information acquisition system in which a processor executes processing for acquiring information regarding the arrangement of a plurality of products arranged on a shelf,
The shelf has
a plurality of shelf labels arranged corresponding to the group of products grouped for each same product and containing product identification information related to the product;
a position display unit arranged at a predetermined reference position of the shelf and including identification information of the reference position;
The processor
Acquiring a photographed image of the shelf photographed by a camera,
acquiring the product identification information and the identification information of the reference position by reading the image of the shelf label and the image of the position display unit included in the photographed image;
calculating the position of the shelf label from the positional relationship between the image of the shelf label and the image of the position display unit in the captured image;
A product information acquisition system that associates and stores the product identification information, the information on the position of the shelf label, and the identification information on the reference position.

Images