JP2015230236A - Merchandise guidance device, terminal equipment, merchandise guidance method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a merchandise guidance device for guiding a customer to a position where desired merchandise is arranged.SOLUTION: A merchandise guidance device includes: a current position information acquisition part 102 for acquiring the current position information of a customer in a store; a merchandise position information storage part 103 for storing merchandise information and merchandise position information in association; a merchandise position information acquisition part 106 for acquiring one or more merchandise position information corresponding to the merchandise information received by a merchandise information reception part 101 from the merchandise position information storage part 103; a purchase route information acquisition part 108 for acquiring purchase route information for purchasing merchandise by using route information stored in a route information storage part 107, the current position information and the merchandise position information; and an output part 109 for outputting guidance information for guiding the customer to a position where the merchandise is arranged by using the purchase route information.

Description

  The present invention relates to an apparatus for guiding a customer.

  As a conventional technique, there has been known a road guidance device that performs road guidance from a departure place to a destination based on map data including road network data and landmark data (see, for example, Patent Document 1).

JP 2003-207358 A (first page, FIG. 1 etc.)

  However, the conventional technology has a problem that it is impossible to guide a customer to a position where a target product such as a product desired to be purchased is placed in a store such as a supermarket.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to provide a product guide device that can guide a customer to a position where a desired product is arranged.

  The product guide device of the present invention acquires a product information receiving unit that receives one or more product information that is information indicating a product, and current location information that is information indicating a customer's current location in a store that provides the product. A product location information storage unit in which a location information acquisition unit, product information, and product location information, which is information indicating a location where the product indicated by the product information in the store is arranged, are stored in association with each other; A product position information acquisition unit that acquires one or more product position information corresponding to one or more product information received by the reception unit from the product position information storage unit, and passage information that is information indicating a passage in the store are stored. The product information receiving unit accepts the information using the passage information storage unit, the passage information, the current position information acquired by the current position information acquisition unit, and one or more product position information acquired by the product position information acquisition unit. A purchase route information acquisition unit that acquires purchase route information that is information indicating a purchase route that is a route for purchasing a product indicated by one or more product information, and purchase route information acquired by the purchase route information acquisition unit. And an output unit that outputs guidance information that is information for guiding a customer to a position where a product indicated by one or more product information received by the product information receiving unit is placed.

  With this configuration, the customer can be guided to a position where a desired product is arranged according to the route indicated by the purchase route information and the current location of the customer.

  Further, in the product guidance device according to the present invention, in the product guidance device, the product information reception unit receives a plurality of product information, and the product position information acquisition unit has a plurality of items corresponding to the plurality of product information received. The product location information is acquired, and the purchase route information acquisition unit uses the plurality of product location information acquired by the product location information acquisition unit to purchase a plurality of products indicated by the plurality of product information received by the product information reception unit. It is a goods guidance device which acquires purchase route information which shows a purchase route for.

  With this configuration, customers can be guided in the store so that a plurality of products can be purchased appropriately.

  The product guidance device of the present invention is a product guidance device in which a plurality of product location information is stored in association with each product information in the product location information storage unit.

  With this configuration, product position information corresponding to product information can be selected from a plurality of product position information. As a result, it is possible to obtain appropriate product position information for the current position of the customer and guide the customer appropriately.

  Moreover, the product guidance device of the present invention is the product guidance device, wherein the product location information acquisition unit is a plurality of product location information corresponding to the product information received by the product information reception unit stored in the product location information storage unit. Among these, the product location information acquisition unit acquires product location information closest to the current location acquired by the current location information acquisition unit.

  With this configuration, it is possible to acquire purchase route information using product position information that is closest to the current position of the customer, and it is possible to shorten the movement for purchase by the customer. .

  In the product guidance device according to the present invention, in the product guidance device, the product position information includes information on a height direction in which a corresponding product is arranged, and the output unit is a height in which the product is arranged. This is a product guidance device that outputs guidance information further including information for guiding the position in the vertical direction.

  With such a configuration, the height at which the product is arranged can be shown to the customer.

  The product guidance device of the present invention further includes a product name storage unit that stores product information and a general name of a product indicated by the product information in the product guide device, and the product position information acquisition unit includes: When the product indicated by the product information received by the product information receiving unit is not placed in the store, the general name corresponding to the product information is acquired from the product name storage unit, and the product position associated with the general name It is a product guide device for acquiring information.

  With such a configuration, when there is no product, the customer can be guided to the sales floor of the product to be replaced.

  Further, the product guide device of the present invention further includes an inventory information storage unit that stores product information and stock information indicating a stock status of the product indicated by the product information in the product guide device, and the output unit includes: It is a product guidance device that performs output according to the inventory information of the product indicated by the product information received by the product information receiving unit.

  With such a configuration, it is possible to appropriately guide customers according to the stock status.

  In addition, the product guide device of the present invention is a product information storage unit in which one or more product information that is information indicating a product is stored in the product guide device, and the inside of the store that provides the product is photographed. An image capturing unit that acquires an in-store video that is a video of the product, the product information receiving unit receives product information stored in the product information storage unit, and the current position related information acquiring unit The current position information is acquired using the output unit, and the output unit is a product guide device that displays the in-store video acquired by the imaging unit and an image corresponding to the guide information in an overlapping manner.

  With such a configuration, the customer can be guided to the position where the product is located in the store by the product guide device alone. In addition, it is possible to display guidance on the in-store image in which the state almost the same as the state that the user is looking at is captured, and the user can be guided more easily.

  A terminal device according to the present invention is a terminal device that constitutes a product guide system including a product guide device and a terminal device, and a product information storage unit that stores one or more product information that is information indicating products. The product information transmission unit that transmits one or more product information stored in the product information storage unit to the product guidance device, and the inside of the store that provides the product are photographed, and the in-store video that is the in-store video is acquired. A current position related information transmitting unit for transmitting current position related information, which is information related to the current position of the customer in the store, to the product guidance device, the information corresponding to the in-store video acquired by the shooting unit The guide information receiving unit that receives the guide information that is information for guiding to the position where the product indicated by the one or more product information transmitted by the product information transmitting unit is received from the product guide device, and the photographing unit acquires In the store A terminal device provided with an image, and a display unit for displaying overlapping the image corresponding to the guidance information.

  With this configuration, it is possible to display the guidance on the in-store image in which the appearance almost the same as that in the store viewed by the user is displayed, and the customer can be guided more easily.

  According to the product guidance device and the like according to the present invention, it is possible to guide a customer to a position where a desired product is arranged.

Block diagram of product guidance system in Embodiment 1 The flowchart which shows an example of operation | movement of the goods guidance apparatus of the goods guidance system The flowchart which shows an example of operation | movement of the terminal device of the goods guidance system The figure which shows the merchandise information management list of the terminal device of the merchandise guidance system The figure which shows the image in a store which the terminal device of the product guidance system transmitted The figure which shows the marker position management information of the goods guidance apparatus of the goods guidance system The figure (FIG. 7 (a)) in the state of binarizing the in-store video for explaining the product guidance system, and the figure showing the state of detecting the marker image (FIG. 7 (b)) The figure which shows the inventory information management table | surface of the merchandise guidance apparatus of the merchandise guidance system The figure which shows the merchandise position information management table of the merchandise guidance apparatus of the merchandise guidance system The figure which shows the merchandise name management table of the merchandise guidance apparatus of the merchandise guidance system The figure which shows the general merchandise position information management table of the merchandise guidance apparatus of the merchandise guidance system The figure which shows the example of a display by the terminal device of the goods guidance system (Drawing 12 (a)), the figure which shows the example of a display which shows the height of goods (Drawing 12 (b)), The example of a display when there is no handling of goods The figure which shows (Figure 12 (c)) and the figure which shows the example of display when there is no stock (Figure 12 (d)) Block diagram of commodity guidance apparatus in embodiment 2 Flow chart showing an example of the operation of the product guide device The figure which shows an example of the external appearance of the computer system in Embodiment 1,2. The figure which shows an example of a structure of the computer system

  Hereinafter, embodiments of a product guide device and the like will be described with reference to the drawings. In addition, since the component which attached | subjected the same code | symbol in embodiment performs the same operation | movement, description may be abbreviate | omitted again.

(Embodiment 1)
FIG. 1 is a block diagram of a merchandise guidance system 1000 in the present embodiment.

  The product guide system 1000 includes a product guide device 1 and one or more terminal devices 2. The product guide device 1 and the terminal device 2 are connected so as to be communicable via a network, a communication line, or the like. In FIG. 1, for convenience of explanation, a case where there is one terminal device 2 is described as an example, but a plurality of terminal devices 2 may be provided.

  The merchandise guidance device 1 includes a merchandise information reception unit 101, a current position information acquisition unit 102, a product position information storage unit 103, a product name storage unit 104, an inventory information storage unit 105, a product position information acquisition unit 106, and a passage information storage unit 107. A purchase route information acquisition unit 108 and an output unit 109.

  The terminal device 2 includes a product information storage unit 201, a product information transmission unit 202, a photographing unit 203, a current position related information transmission unit 204, a guidance information reception unit 205, and a display unit 206.

  The merchandise guidance device 1 is a server that processes information transmitted from the terminal device 2 and transmits information obtained by the processing to the terminal device 2, for example. The merchandise guide device 1 is realized by a computer or the like, for example.

  The product information receiving unit 101 receives one or more product information. The product information is information indicating a product. The product information is, for example, product identification information. The product information is, for example, a product name, a product proper noun, a product code, and the like. The product information receiving unit 101 receives, for example, product information input by a customer directly or indirectly.

  The product information receiving unit 101 may receive a list of product information including a plurality of product information.

  The reception here is usually reception of information transmitted via a wired or wireless communication line, for example, reception of information input from an input device such as a keyboard or a touch panel, a magnetic disk, a semiconductor memory, etc. It is a concept including reception of information read from a recording medium.

  In this embodiment, as an example, a case where the product information receiving unit 101 receives one or more product information transmitted from the terminal device 2 will be described as an example.

  The merchandise information receiving unit 101 is usually control software for communication means, but it can also be realized by a device driver for input means such as a keyboard, a driver for reading information on a recording medium, or the like.

  The current position information acquisition unit 102 acquires the customer's current position information. The current position information is information indicating the current position of the customer in the store that provides the product. The customer here is, for example, a customer who has input the product information received by the product information receiving unit 101. The current position information is, for example, information indicating the current position of the terminal device 2 or information indicating the current position of a customer who owns the terminal device 2. The store is, for example, a place where a product is provided or a building. The store may be considered as a commercial facility, for example. The stores are, for example, retail stores, supermarkets, department stores, and the like. A so-called shopping center where a plurality of stores gather, a shopping mall, and the like may also be considered as stores. The merchandise is, for example, daily necessities, food or clothing items, or services such as legal consultation. Rental video disks, CDs, recording media such as books, and the like may also be considered as products.

  The current position information is, for example, coordinates indicating the current position of the customer, information indicating an area or place where the customer is currently located, for example, an identifier of the area, and the like. The coordinate system of the coordinates indicating the current position may be an absolute coordinate system such as latitude and longitude, or may be a relative coordinate system set for the store.

  The acquisition of the current position information by the current position information acquisition unit 102 may be, for example, reception or reception of the current position information, or information received by the current position information acquisition unit 102 or current information It may be to acquire position information.

  The current position information acquisition unit 102 may acquire the current position information of the customer in the store. For example, the current position information acquisition unit 102 may acquire the current position information by receiving the current position information from the terminal device 2 used by the customer. Further, identification information (for example, an IP address) such as an access point or a base station included in information such as product information transmitted by the terminal device 2 used by the customer by Wi-fi (registered trademark), Bluetooth (registered trademark) or the like. Etc.) from the access point or base station provided in the store, which is accessed by the terminal device, and information indicating the location of the specified access point or base station is acquired as the current location information You may do it.

  For example, the current position information acquisition unit 102 may receive current position related information described later from the terminal device 2 and acquire the current position information according to the received information. The current position related information transmitted from the terminal device 2 is acquired from, for example, information indicating the current position of the terminal device 2, a video in the store acquired by the terminal device 2, or a video in the store acquired by the terminal device 2. AR (Augmented Reality) marker image, an identifier associated with the AR marker, and the like. A video in a store (hereinafter referred to as a video in a store) is usually a still image obtained by shooting the inside of the store, but may be a moving image. Further, it may be an image of one frame of a moving image. The format of the video in the store, the file format, the number of pixels, etc. are not questioned. Further, the in-store video may be a color image, a gray scale image, or a binarized image. The video in the store is preferably a video shot so as to include one or more AR markers arranged in the store. The AR marker will be described later.

  For example, when the current position related information transmitted by the terminal device 2 is information indicating the current position of the terminal device 2, the current position information acquisition unit 102 uses the current position related information of the customer holding the terminal device 2. You may acquire as present position information.

  Further, when the current position related information transmitted by the terminal device 2 is the in-store video acquired by the terminal device 2, the current position information acquisition unit 102 specifies the position in the store from the received in-store video, for example. An image to be detected is detected, and information on the current position is acquired from the detected image. For example, the current position information acquisition unit 102 detects an image of one or more AR markers from the received video in the store, and acquires current position information corresponding to the detected AR markers.

  When the current position related information transmitted by the terminal device 2 is an AR marker image acquired from the in-store video acquired by the terminal device 2, the current position information acquisition unit 102 corresponds to, for example, this AR marker. Get current location information.

  When the current position related information transmitted by the terminal device 2 is an identifier associated with the AR marker acquired from the in-store video acquired by the terminal device 2, the current position information acquisition unit 102, for example, The current position information corresponding to the identifier is acquired.

  The current position information acquisition unit 102 may further acquire direction information. The direction information is, for example, information indicating the direction of the customer, for example, information indicating the traveling direction or the line-of-sight direction of the customer. The direction information may be information indicating the direction of the AR marker in the in-store video, for example. For example, the direction information is represented by a direction vector or the like.

  It does not matter how the current position information acquisition unit 102 acquires the direction information. For example, the direction information may be acquired by predicting the traveling direction of the customer from the temporal change in the current position information of the customer acquired as described above.

  Further, when the current position related information transmitted by the terminal device 2 includes direction information, the current position information acquisition unit 102 may acquire the direction information. For example, the terminal device 2 has a direction sensor (not shown), a gyro (not shown), and the like, and the terminal device 2 transmits the direction information acquired by the terminal device 2 as current position related information to the product guidance device. In this case, the current position information acquisition unit 102 may acquire the direction information by receiving the direction information.

  The current position information acquisition unit 102 may acquire information indicating the direction of the AR marker as direction information from the image of the AR marker acquired by the current position information acquisition unit 102 from the in-store video.

  Hereinafter, the process in which the current position information acquisition unit 102 acquires current position information and direction information using an AR marker will be described by way of an example of acquisition from an in-store video.

  The AR marker is, for example, an index having a shape that can be identified with respect to the background of an area where the marker is arranged, a code such as a color, a pattern, an image, and a character string. The AR marker is, for example, a marker used for vision-based AR. The AR marker is a marker whose direction can be specified, for example. The AR marker is, for example, a two-dimensional image printed or pasted on a substance. The AR marker may be a three-dimensional member or the like. The AR marker used here is an AR marker that can identify each of a plurality of AR markers by, for example, a shape, a surface pattern, a surface character string, or the like. For example, each identifiable AR marker is associated with identification information of the AR marker. The identification information of the AR marker (for example, a character string that is identification information) may be written on the AR marker, for example. For example, information for acquiring identification information of the AR marker may be written in the AR marker. For example, the AR marker may have a one-dimensional or two-dimensional barcode as information for acquiring identification information, and the AR marker itself may be a barcode. The identification information of the AR marker is associated with, for example, position information indicating the position where the AR marker is arranged, for example, coordinates. The AR marker identification information may be AR marker position information.

  The current position information acquisition unit 102 detects an image of the AR marker in the in-store video. Detecting an AR marker image means, for example, detecting an area where an AR marker is shown in a store image, detecting an outline of an AR marker, and a corner portion of an AR marker. The above feature points are detected. The feature points of the AR marker are, for example, one or more portions that show features in the appearance or the like of the AR marker, or portions that have one or more shapes used to distinguish them from the background. For example, one or more corner portions of the AR marker, or a portion having a specific shape provided in the AR marker or on the outer periphery of the AR marker.

  The current position information acquisition unit 102 is designated in advance when, for example, the in-store video is a color image or a grayscale image, and the outline of the AR marker has a color whose luminance is sufficiently lower than the background. The in-store video is binarized with a threshold, and the outline of the AR marker in the in-store video is detected.

  The current position information acquisition unit 102 acquires marker related information that is information related to the recognized AR marker. The marker related information is, for example, information indicating the area where the AR marker is arranged in the in-store video, the direction of the AR marker, the shape of the AR marker, and the like, and information usable for specifying these. The marker related information may include, for example, information indicating the outline of the AR marker detected by the current position information acquisition unit 102 and information indicating one or more feature points of the AR marker. The information indicating a feature point is, for example, information indicating a coordinate indicating a feature point or an area indicating a feature point.

  The current position information acquisition unit 102 stores, for example, a plurality of ARs stored in association with the AR marker identification information in a storage unit or the like (not shown) in advance in the image of the AR marker detected in the store. Pattern matching with a marker image or the like is performed, a matching AR marker image or the like is detected, and identification information of the AR marker previously associated with the AR marker image is acquired. Then, the AR marker position information associated with the acquired identification information is acquired as the current position information. For example, information associating identification information of the AR marker with position information indicating the position in the store where the AR marker is arranged is stored in a storage unit (not shown) in advance, and this information is used. The position information corresponding to the acquired identification information of the AR marker may be acquired as the current position information.

  The AR marker image stored in advance in the storage unit (not shown) or the like may be a partial image in the AR marker. This partial image may be, for example, one or more character strings (including numbers and symbols) arranged in the AR marker.

  The current position information acquisition unit 102 acquires information indicating the positional relationship between the position of the AR marker and the shooting position using the marker related information, and a value obtained by correcting the position information of the AR marker with the acquired information. May be acquired as current position information. For example, information indicating the distance and direction between the actual AR marker and the shooting position is acquired from the size of the AR marker indicated by the marker-related information, the deformation state by the perspective of the outline of the AR marker, and the like, and this information is used. Then, a value obtained by correcting the position information of the AR marker may be acquired as the current position information.

  Further, when a plurality of AR markers are detected, for example, position information of one of the AR markers, for example, the AR marker positioned closest to the front may be acquired as current position information. Further, information on the direction of the AR marker may be acquired from the positional relationship of the plurality of markers.

  For example, the current position information acquisition unit 102 uses information indicating the direction of the AR marker included in the marker-related information, information indicating the shooting direction in which the in-store video is shot, such as a direction vector, as the customer direction information described above. You may get it. The shooting direction is, for example, the optical axis direction of the lens used during shooting.

  As described above, processing for detecting an AR marker from a captured image, processing for acquiring marker-related information such as information indicating the direction of the AR marker in the image, identification information corresponding to the AR marker, etc. Since the acquired process is a known technique, a detailed description thereof is omitted here. Examples of the references include those shown in Non-Patent Documents 1 and 2 below.

  "Non-Patent Document 1: Hirokazu Kato, Mark Billinghurst," Marker Tracking and HMD Calibration for a video-based Augmented Reality Conferencing System "Proceedings of the 2nd International Workshop on Augmented Reality, pp.85-94, Oct., 1999"

  "Non-Patent Document 2: Nao Hashimoto" Introduction to ARToolKit Augmented Reality Programming "ASCII Media Works, First Edition September 18, 2008"

  It should be noted that the current position information acquisition unit 102 acquires information such as the focal length and inclination of the lens when the in-store video is captured, and the current position information acquisition unit 102 uses the information to acquire the AR marker. Information indicating the orientation of the image may be corrected as appropriate.

  Here, an example in which the current position information acquisition unit 102 acquires current position information and direction information from an in-store video using an AR marker has been described. However, the current position information acquisition unit 102 has an image of an AR marker. When acquiring the current position information and direction information from the identification information of the AR marker or the like, a part of the processing example described above may be used as appropriate.

  The product position information storage unit 103 stores product information and product position information in association with each other. The product position information is information indicating a position where the product indicated by the product information in the store is arranged. The position is, for example, a point. The product position information is, for example, coordinates. The coordinates here are, for example, coordinates in which a store floor or a horizontal plane is considered as a coordinate plane. When a plurality of products indicated by one product information are arranged in the store, the product position information associated with the product information is a plurality of positions where the plurality of products associated with the product information are disposed. The position information of one position that represents For example, when a plurality of products indicated by one product information are arranged in the store, the product position information associated with the product information is an area in which the plurality of products associated with the product information is arranged. Among them, it may be position information of one point representing the area (for example, the center point) or one point corresponding to the area (for example, position information of the passage just before the position where the product is arranged).

  The product location information storage unit 103 may store a plurality of product location information in association with each product information. For example, the plurality of product position information may be position information indicating two or more points in an area where one or more products corresponding to one product information are arranged. It may be considered that the area includes or does not include the boundary line of the area. The product position information storage unit 103 may store product information and information indicating an area where one or more products corresponding to the product information are arranged. In this case, the region where one or more products are arranged may be considered as an aggregate of a plurality of product position information. That is, each coordinate in the area may be considered as product position information. Thus, even when product information and information indicating a region are stored in association with each other, product information and a plurality of product position information are stored. May be considered to be a form stored in association with each other. The information indicating the region is, for example, information indicating the coordinates indicating a plurality of corners of the region or an outline.

  The product position information may include information on the height direction in which the corresponding product is arranged. The height here is, for example, the height relative to the floor in the same floor. The information in the height direction is represented, for example, as a coordinate value in a direction perpendicular to the store floor or the horizontal plane, or a distance from the floor. The information in the height direction may be information (for example, the second stage, the third stage, etc.) indicating the number of shelves such as display shelves on which products are arranged.

  The product position information storage unit 103 is preferably a non-volatile recording medium, but can also be realized by a volatile recording medium. The same applies to other storage units.

  In the product name storage unit 104, one or more product information and a general name of the product indicated by the product information are stored in association with each other. The general name of a product is, for example, a commonly used name for a product. The general name of the product is, for example, milk, cellophane tape, screwdriver, laundry detergent and the like. The general name of the product may be information indicating the genre of the product. The product name storage unit 104 may store product information of products other than products arranged in the store. Note that two or more pieces of product information may be associated with one piece of product information and stored in the product name storage unit 104.

  The inventory information storage unit 105 stores one or more pieces of product information and stock information indicating the stock status of the product indicated by the product information. The stock information is information indicating whether there is a stock of merchandise, for example. The inventory information may be information indicating the number of items in stock. The product information stored in the inventory information storage unit 105 is usually product information of a product handled at a store. The product handled at the store is, for example, a product sold at the store.

  The product position information acquisition unit 106 acquires product position information corresponding to one or more product information received by the product information receiving unit 101 from the product position information storage unit 103. For example, the merchandise position information acquisition unit 106 associates the merchandise position information stored in the merchandise position information storage unit 103 with the merchandise information that respectively matches one or more pieces of merchandise information received by the merchandise information receiving unit 101. The obtained product position information is acquired by searching or the like. The match here may be a complete match or a partial match.

  When a plurality of product position information is associated with the product information and stored in the product position information storage unit 103, the product position information acquisition unit 106, for example, the product stored in the product position information storage unit 103. Of the plurality of product position information corresponding to the product information received by the information receiving unit 101, the closest product position information, for example, the closest product position information acquired by the current position information acquiring unit 102 may be acquired. For example, when product information is associated with information indicating an area where a product is arranged, the product position information acquisition unit 106 stores the product information reception unit 101 stored in the product position information storage unit 103. The position information (for example, coordinates) of the position closest to the current position acquired by the current position information acquisition unit 102 in the area indicated by the information indicating the area corresponding to the received product information may be acquired as the product position information. . The position closest to the current position is, for example, a position having the shortest distance (for example, a straight line distance) from the current position.

  The product position information acquisition unit 106 acquires a general name corresponding to the product information from the product name storage unit 104 when the product indicated by the product information received by the product information reception unit 101 is not arranged in the store. The product position information associated with the general name may be acquired.

  For example, the product position information acquisition unit 106 uses the product information stored in the stock information storage unit 105 or the product information and the stock information to determine whether the product indicated by the product information is arranged in the store. to decide. For example, the product position information acquisition unit 106 determines whether there is product information that matches the product information received by the product information receiving unit 101. If there is no matching product information, the product information receiving unit 101 receives the product information. If it is determined that the product is not placed in the store and there is matching product information, it is determined that the product is placed in the store. Even if there is matching product information, if the inventory information associated with this product information indicates that there is no product, it is determined that the product is not placed in the store, and this If the inventory information associated with the product information indicates that there is a product, it may be determined that the product is placed in the store.

  Then, when it is determined that the product indicated by the product information received by the product information receiving unit 101 is arranged in the store, the product position information acquiring unit 106 acquires the product position information of the product in the same manner as described above. To do. When it is determined that the product indicated by the product information received by the product information receiving unit 101 is not arranged in the store, a general name associated with the product information received by the product information receiving unit 101 is stored in the product name. Obtained from the unit 104. And the merchandise position information matched with this general name is acquired. For example, the product information associated with the same general name as the acquired general name is acquired from the product name storage unit 104, and the product position information corresponding to the product information is acquired in the same manner as described above. If there is more than one product information associated with the same general name as the acquired general name, it corresponds to one of those product information (for example, product information selected at random, product information detected first) Get product location information. Instead of acquiring the general name corresponding to the product information from the product name storage unit 104, the general name corresponding to the product information may be acquired by performing a search on the Internet or the like.

  In the above, instead of using inventory information, product information of a plurality of products arranged in a store, for example, a product list handled in the store, which is stored in advance in a storage unit (not shown) or the like is used. Also good.

  In addition, when the inventory information indicates that there is no inventory of the product, the product position information corresponding to the general name corresponding to the product information or the product information may not be acquired. And the output part 109 etc. which are mentioned later output the guidance information containing the information which shows that there is no stock of the goods which this product information shows, The information which shows that there is no stock of goods is the customer's terminal device 2 May be displayed.

  In addition, information that associates the general name of the product with one or more pieces of product position information indicating the position where the product indicated by the general name is stored in advance in a storage unit (not shown). Instead of acquiring the product information corresponding to the general name in the above and acquiring the product position information corresponding to the product information, the product corresponding to the general name is obtained from the information that associates the general name with the product position information. The product position information acquisition unit 106 may acquire the position information.

  The passage information storage unit 107 stores passage information that is information indicating a passage in the store. The passage information is information having, for example, a node set for the passage in the store and a link set to connect the nodes along the passage. The node is information such as coordinates indicating a plurality of points in the passage, for example. The node is set at, for example, a branch point of the passage. The link is, for example, information that associates nodes that are both ends of the link. The link may be associated with a travel time when the link is moved, a link distance, a score or cost indicating the link travel time or distance, and the like. The passage information may be other information as long as it is information capable of performing a route search or the like on the passage in the store. As the path information, for example, information that is substantially the same as road information used for route output, route search, and the like in a so-called navigation device can be used, and detailed description thereof is omitted here. .

  The purchase route information acquisition unit 108 includes the passage information stored in the passage information storage unit 107, the current position information acquired by the current position information acquisition unit 102, and the product position information acquired by the product position information acquisition unit 106. The purchase route information, which is information indicating the purchase route, is acquired. The purchase route is a route for purchasing a product indicated by one or more product information received by the product information receiving unit 101. The purchase route is, for example, a route for acquiring a product from a display shelf or selecting a product in order to purchase the product. However, the purchase route may be a route for reaching the product for the purpose of purchasing the product or a route for collecting the product for the purpose of purchase, regardless of whether the product is actually purchased or not.

  The purchase route information acquisition unit 108 uses, for example, the passage information stored in the passage information storage unit 107, the position indicated by the current position information (for example, coordinates) as a departure point, and the product position information acquisition unit 106 A route search using the position (for example, coordinates) indicated by the acquired product position information as an arrival place is performed, and purchase route information that is route information is acquired. The arrival point may be considered as the final destination. For example, the purchase route information acquisition unit 108 acquires route information suitable for the customer, preferably optimal route information as purchase route information. For example, route information with a short route length, travel time, etc., preferably route information that is the shortest is obtained. Alternatively, optimal route information may be acquired using a score, cost, or the like regarding the route length or travel time. Note that the process for performing the route search is a known technique, and thus detailed description thereof is omitted here.

  The purchase route information acquisition unit 108 uses the plurality of product position information acquired by the product position information acquisition unit 106 to purchase a plurality of products indicated by the plurality of product information received by the product information reception unit 101. The purchase route information indicating “” may be acquired.

  For example, the purchase route information acquisition unit 108 uses the path information stored in the path information storage unit 107 to set the position indicated by the current position information (for example, coordinates) as a departure place, and the product position information acquisition unit 106 A route search using a plurality of positions (for example, coordinates, etc.) indicated by the acquired plurality of product position information is performed to acquire purchase route information that is route information. In this case, the last waypoint may be considered an arrival point. For example, the purchase route information acquisition unit 108 determines a plurality of passing orders that pass through a plurality of positions indicated by a plurality of product position information in different orders, and each of the determined passing orders has a plurality of determined passing orders. Route search via the transit point is performed. Then, one piece of route information is obtained as purchase route information from the route information obtained for a plurality of route orders. For example, route information having a short route length and a short travel time, preferably the shortest route information is acquired as purchase route information. Alternatively, route information having a preferable score for the route length, travel time, etc., for example, a value closest to a preset value is acquired as purchase route information. The route search through a plurality of waypoints is, for example, the shortest route from the departure point to the first waypoint, and the pth place from the pth waypoint (where p is an integer of 2 or more) This can be realized by combining with the shortest route to. The route search in the case of passing through a plurality of waypoints in the order specified in advance can be realized by partially using, for example, a solution of the traveling salesman problem. In addition, since the route search in the case of passing through a plurality of waypoints in the order specified in advance is a known technique, detailed description thereof is omitted here.

  If the product location information acquisition unit 106 does not acquire product location information for a product that is not in stock, there is no product location information for this product, so the purchase route information acquisition unit 108 arranges this product. The purchase route information that does not go through the location to be acquired may be acquired.

  The purchase route information acquisition unit 108 does not have to re-acquire the purchase route information once the route search is performed and the purchase route information is acquired, or at a predetermined timing or trigger Accordingly, purchase route information may be reacquired. For example, when the current position information acquisition unit 102 acquires current position information, it is determined whether or not the acquired current position information is out of the route indicated by the purchase route information. The purchase route information may be acquired again using the position indicated by the position information as the departure position.

  The purchase route information acquisition unit 108 uses a position (for example, coordinates) indicated by the current position information as a departure point, and one or more positions (for example, one or more positions indicated by the one or more product position information acquired by the product position information acquisition unit 106). , Coordinates, etc.) may be used as a waypoint, and a route search may be performed using the location of the store's payment place or payment place as the destination to obtain purchase route information as route information. For example, the location information of the payment place and the payment place in the store may be stored in advance in a storage unit (not shown) or the like, and read when searching for a route.

  The output unit 109 outputs guide information using the purchase route information acquired by the purchase route information acquisition unit 108. For example, the output unit 109 transmits guidance information to the terminal device 2. The guide information is information for guiding the customer to the position where the product indicated by the one or more product information received by the product information receiving unit 101 is arranged. The guidance information is information for performing guidance for the customer to move along the route indicated by the purchase route information, for example. For example, the output unit 109 outputs guidance information corresponding to the current position information acquired by the current position information acquisition unit 102. For example, the guidance information is information indicating the traveling direction of the customer at the position indicated by the current position information acquired by the current position information acquisition unit 102.

  For example, the output unit 109 may output guide information according to the current position information acquired by the current position information acquisition unit 102 and the direction information. For example, the output unit 109 may output guidance information for guiding the customer's traveling direction to the customer who is facing the direction indicated by the direction information at the position indicated by the current position information.

  The guidance information includes, for example, images such as straight arrows and bent arrows to guide the direction of travel to the customer, a character string that conveys the direction of travel such as "straight forward" and "right turn", etc. Information used for output. For example, the output unit 109, when the customer is facing the direction indicated by the direction information from the purchase route information to the position indicated by the current position information, information on the traveling direction for moving along the purchase route information, for example, Information such as a vector is acquired, and guidance information for displaying the acquired arrow image in the traveling direction or a similar arrow image is acquired and output. The guidance information to be output may be acquired from the guidance information stored in advance in a storage unit (not shown) in association with the traveling direction information or the like. The guide information may be information on a moving image such as an animation or information specifying a moving image.

  Note that the guidance information may be any information that can output to the customer a guidance up to the position where the product is arranged as a result. An image showing the guidance or a guidance image such as 3D modeling data may be used. It may be information that can be generated, a character string indicating guidance, voice information for guidance, or a combination of one or more of these.

  For example, the guidance information may be information for causing the terminal device 2 to perform guidance to a position where the product is arranged. For example, the guidance information may be an instruction for causing the terminal device 2 to output an image, a character string, a voice, or the like indicating guidance, information for designating an image, a character string, a voice, or the like to be output, or a combination thereof. Good.

  The guidance information is, for example, an arrow for guiding the traveling direction to the customer for displaying in-store video acquired by the current position information acquisition unit 102 or in-store video acquired by the terminal device 2. The information is preferably information such as an image, an image showing a character string that conveys the direction of travel, or an instruction to display a similar image superimposed on the in-store video. The guidance information includes, for example, an image such as an arrow for guiding a traveling direction to a customer displayed in a position corresponding to the AR marker of the in-store video acquired by the current position information acquisition unit 102, and a traveling direction. Information such as an image indicating a character string to be transmitted, or an instruction to display a similar image superimposed on a position corresponding to the AR marker of the in-store video is preferable. The position corresponding to the AR marker is, for example, a position corresponding to the space above the AR marker on the AR marker of the in-store video. For example, the output unit 109 may include information indicating a position where an image or character string included in the guidance information is arranged. The guidance information is preferably information for displaying an image such as an arrow for guiding the customer in accordance with the direction in which the AR marker is arranged.

  Note that displaying the images in an overlapping manner may be considered as combining and displaying the images. For example, displaying in an overlapping manner may be combining in a so-called overlay mode or the like so that the contents of all the superimposed images can be seen, or an image arranged in the front of an image arranged in the back. It is also possible to synthesize so that the portion that overlaps with is hidden except for the mask region portion of the front image. The composition mode is not limited. The same applies to the following.

  The output unit 109 adjusts the positional relationship between the shooting position and the position where the AR marker is arranged, the distortion of the AR marker in the in-store video generated by the angle of view of the shooting unit, and the direction of the AR marker. An instruction to output a deformed image or character string or a deformed image or character string may be output as guide information.

  When the product position information includes information indicating the position in the height direction in which the product is arranged, the output unit 109 provides information for guiding the position in the height direction in which the product is arranged. May be output. For example, the output unit 109 may appropriately output guidance information including information such as a character string and an image indicating the height at which the product is arranged. Guidance information further including information for guiding the position in the height direction includes, for example, a character string indicating the number of shelves on which the product is arranged, and a position where the product is arranged in the schematic diagram of the shelf. This is information for outputting an image or the like shown in a different mode.

  The output unit 109 may perform output according to the inventory information corresponding to the product information received by the product information receiving unit 101. For example, the output unit 109 determines whether there is product information that matches the product information received by the product information receiving unit 101 in the stock information storage unit 105, and if there is, the stock associated with the product information. It is determined whether the information is information indicating that there is a stock. And when it is the information which shows that there is no stock, you may output the guidance information containing the information which shows that there is no stock about the goods. For example, guidance information having the received product information and a character string indicating that the product indicated by the product information is not in stock is output. Note that outputting the guide information using the purchase route information using the product position information acquired for the general name corresponding to the product that is not in stock may also be considered as a form of output according to the stock information. .

  Note that information such as images, 3D modeling data, and character strings used for guidance information is accumulated in advance in a storage unit (not shown), for example, and is read out by the output unit 109 as necessary. You can do it.

  The output here is usually transmission to an external device, but may be display on a display, sound output, delivery of a processing result to another processing device or another program, or the like.

  The output unit 109 may or may not include an output device such as a communication unit. Alternatively, the output unit 109 may be considered as including or not including a display, a speaker, or the like or an output device. The output unit 109 can be realized by output device driver software, or output device driver software and an output device. Further, the output unit 109 may include an MPU, a memory, dedicated hardware, and the like as a configuration for performing processing for acquiring guidance information and the like.

  The terminal device 2 is a device used by a customer. The terminal device 2 is a portable device capable of wireless communication or the like with respect to the product guidance device 1 such as a multi-function mobile phone such as a so-called smartphone, a tablet terminal, or a mobile phone.

  The product information storage unit 201 stores one or more product information that is information indicating a product. The product information storage unit 201 stores, for example, a list composed of a plurality of product information. One or more pieces of product information stored in the product information storage unit 201 are, for example, product information of products that are candidates for purchase by customers. There is no limitation on how the product information is stored in the product information storage unit 201. The storage here is a concept including temporary storage. For example, it may be considered as storing that product information previously input by a customer is stored in the product information storage unit 201. For example, a product received by an accepting unit (not shown) via an input device such as a touch panel It may be considered that information is temporarily stored in the product information storage unit 201.

  The product information transmission unit 202 transmits one or more pieces of product information stored in the product information storage unit 201 to the product guidance device 1. The merchandise information transmission unit 202 may transmit merchandise information according to any timing, trigger, or the like. For example, when a pre-designated operation for requesting in-store guidance from a customer or an operation for instructing transmission of product information is received, the product information stored in the product information storage unit 201 is read and transmitted. Also good. Further, the product information may be transmitted when the product information is accumulated in the product information storage unit 201. In addition, product information may be transmitted when a customer enters the store.

  The merchandise information transmission unit 202 is usually realized by a wireless or wired communication means. The merchandise information transmission unit 202 may or may not include a communication device.

  The photographing unit 203 photographs the inside of the store that provides the product and acquires the in-store video that is a video in the store. Since the in-store video has been described above, detailed description thereof is omitted here. The photographing unit 203 is, for example, a camera provided with the terminal device 2. The imaging unit 203 may be considered as an imaging device or the like. For example, the lens or the like of the photographing unit 203 is usually provided at a position on the back surface with respect to the monitor (not shown) of the terminal device 2, and the terminal device 2 is placed on the customer's line of sight so that the monitor faces the customer side. When arranged in the direction, the state in front of the customer's eyes is displayed on the monitor as an in-store video via the photographing unit 203.

  The current position related information transmission unit 204 is information corresponding to the in-store video acquired by the photographing unit 203 and transmits the current position related information, which is information related to the current position of the customer in the store, to the product guide device 1. The current position related information is, for example, information used for acquiring the customer's current position information and the customer's current position information. For example, the current position related information may be an in-store video acquired by the imaging unit 203, an image acquired by binarizing the video, or the like. Also, the current position related information may be an image of an AR marker that is detected and acquired from the in-store video by the current position related information transmitting unit 204, as in the case of the current position information acquiring unit 102 described above. By transmitting these pieces of information, the current position information acquisition unit 102 can acquire current position information using these pieces of information. Similarly to the current position information acquisition unit 102 described above, the AR marker identification information acquired by the current position related information transmission unit 204 by performing pattern matching or the like from the in-store video acquired by the imaging unit 203, or the AR marker The corresponding current position information may be used. Note that image information or the like necessary for pattern matching or the like may be accumulated in advance in a storage unit (not shown) or the like.

  The current position related information transmission unit 204 transmits the current position related information according to, for example, a timing or a trigger specified in advance. The current position related information transmission unit 204 may repeatedly transmit the current position related information, for example, at predetermined or indefinite predetermined timings.

  The current position related information transmission unit 204 may transmit the current position information acquired by a receiving unit (not shown) such as an indoor GPS such as the IMPS method of the terminal device 2 as the current position related information. . Moreover, the identifier read from the RFID tag located at a distance close to the product guide device 1 or the current position information corresponding thereto is obtained from a plurality of RFID tags having an identifier for identifying the position installed in the store. Then, it may be transmitted as current position related information. Also, identification information (for example, IP address) of the access point and base station connected to the terminal device 2 among the access points and base stations provided in the store is acquired, and this identification information is supported. Information indicating the location where the access point or base station is provided may be acquired as current location information and transmitted as current location related information. It does not matter how the current location information in the store that the terminal device 2 transmits as the current location related information is acquired.

  Similarly to the current position information acquisition unit 102, the current position related information transmission unit 204 is direction information acquired from the direction of the AR marker, direction information acquired by a gyro (not shown) included in the terminal device 2, an orientation sensor, or the like. May be further transmitted to the terminal device 2.

  The current position related information transmitting unit 204 is usually realized by a wireless or wired communication means. The current position related information transmission unit 204 may or may not include a communication device. The current position related information transmission unit 204 may include an MPU, a memory, and the like for performing processing for acquiring current position related information, and dedicated hardware.

  The guide information receiving unit 205 receives guide information from the product guide device 1. The guide information is information for guiding to the position where the product indicated by the one or more product information transmitted by the product information transmission unit 202 is arranged.

  The guide information receiving unit 205 is usually realized by a wireless or wired communication means. The guidance information receiving unit 205 may or may not include a communication device.

  The display unit 206 displays an image corresponding to the guide information received by the guide information receiving unit 205 on the in-store video acquired by the photographing unit 203. When the guidance information is an image, the image corresponding to the guidance information may be the guidance information itself or an image that can be acquired using the guidance information. For example, the display unit 206 displays an image for guidance that the guidance information has on the in-store video. In addition, the display unit 206 converts the character string for guidance included in the guidance information into a bitmap, acquires a display image indicating the character string, and displays the acquired image superimposed on the in-store video. Further, when the guidance information is information for designating an image to be displayed for guidance, the display unit 206 acquires an image indicated by the guidance information from images stored in advance in a storage unit (not shown). The acquired image is displayed so as to overlap with a position designated in advance or a position indicated by the guidance information. When the guide information is information for designating a character string to be displayed for guidance, the display unit 206 acquires a character string indicated by the guide information from a storage unit (not shown) and the acquired character string is a bitmap. The converted image is displayed in a superimposed manner at a position designated in advance or at a position indicated by the guidance information.

  For example, it is preferable that the display unit 206 displays an image corresponding to the guidance information in an overlapping manner at a position corresponding to the image of the AR marker in the in-store video. The position corresponding to the image of the AR marker is, for example, a position on the image of the AR marker, above the image, or a position separated by a predetermined distance in a predetermined direction with respect to the image. For example, when the information indicating the position where the guidance image or character string included in the guidance information is arranged is a position corresponding to the AR marker image in the in-store video, this information in the in-store video indicates The guidance information is superimposed on the position and displayed. When the display unit 206 or the current position related information transmission unit 204 detects an AR marker from the in-store video, an image included in the guidance information, an image of a character string, or guidance information is indicated at the detected position. An image or a character string image may be displayed. Note that portions of the AR marker in the store video other than the portion overlapping the image corresponding to the guide information may remain displayed in the store video. Further, the display unit 206 may display an image corresponding to the guidance information so as to be superimposed on the in-store video so that the entire AR marker in the in-store video is hidden.

When the guidance information includes an instruction or the like for changing the image or character string according to the AR marker, the display unit 206 displays an image deformed according to the instruction when displaying the image. May be displayed. That is, the display unit 206 may deform the image corresponding to the guidance information in accordance with the distortion of the AR marker in the in-store video, and display the deformed image on the in-store video.
Further, the display unit 206 may display an image corresponding to the guide image in an overlapping manner on a map or the like stored in advance in a storage unit (not shown). The display unit 206 may further output a voice corresponding to the guidance information. For example, you may further output the audio | voice which guides a route using the audio | voice information which guidance information has. Further, a voice that guides the route may be further output by reading out a character string included in the guidance information by using a known voice synthesis process.

  The display unit 206 may be considered as including or not including a display device such as a monitor. The display unit 206 may be realized by display device driver software, or display device driver software and a display device.

  Next, an example of operation | movement of the merchandise guidance apparatus 1 of the merchandise guidance system 1000 is demonstrated using the flowchart of FIG.

  (Step S101) The product information receiving unit 101 receives one or more product information. For example, the product information receiving unit 101 receives product information from the terminal device 2 via a network or the like.

  (Step S102) The current position information acquisition unit 102 acquires current position information. For example, the current position information acquisition unit 102 receives an in-store video from the terminal device 2 via a network or the like, and detects an AR marker from the received in-store video. Then, an identifier corresponding to the AR marker detected by pattern matching or the like is acquired, and current position information corresponding to this identifier is acquired.

  (Step S103) The purchase route information acquisition unit 108 determines whether purchase route information has been acquired. If it has been acquired, the process proceeds to step S115. If it has not been acquired, the process proceeds to step S104.

  (Step S104) The product position information acquisition unit 106 substitutes 1 for the counter k.

  (Step S105) The product position information acquisition unit 106 determines whether the stock information storage unit 105 has product information that matches the kth product information in the product information received in step S101. If there is, the process proceeds to step S106, and if not, the process proceeds to step S110.

  (Step S106) The product position information acquisition unit 106 determines whether or not the stock information associated with the product information determined to match in step S105 indicates that the product is in stock. If it indicates that there is, it proceeds to step S107, and if it indicates that it does not, it proceeds to step S112.

  (Step S107) The product position information acquisition unit 106 detects product information that matches the kth product information from the product position information storage unit 103, and acquires product position information associated with the detected product information. In addition, when there are a plurality of product position information associated with the detected product information, the position from the position indicated by the current position information acquired in step S102 is the closest of the plurality of product position information. You may make it acquire merchandise position information. The product position information acquisition unit 106 associates the acquired product position information with the kth product information and accumulates it in a storage unit (not shown) or the like.

  (Step S108) The merchandise position information acquisition unit 106 increments the value of the counter k by 1.

  (Step S109) The product position information acquisition unit 106 determines whether or not the product information received in step S101 includes the kth product information. If there is, the process returns to step S105, and if not, the process proceeds to step S114.

  (Step S110) The product position information acquisition unit 106 acquires, from the product name storage unit 104, a general name associated with the kth product information by searching.

  (Step S111) The product position information acquisition unit 106 acquires product position information corresponding to the general name acquired in step S110. For example, the product position information associated with the general name that matches the general name acquired in step S110 is acquired from a storage unit (not shown) that stores the general name and the product position information in association with each other. When there are a plurality of matching product position information, the process is the same as in step S107. The product location information acquisition unit 106 associates the acquired product location information with the general name acquired in step S110, the kth product information, and information indicating that the product is not handled, and a storage unit (not shown) To accumulate. In addition, read out the template etc. when there is no matching product prepared in advance from the storage etc. not shown, insert the general name and product information, `` "Product information '' is not handled in our shop, Information on the "general name" section. A character string or the like for introducing a sales place for a product such as “” may be acquired and stored. Then, the process proceeds to step S108.

  (Step S112) The product position information acquisition unit 106 acquires, from the product name storage unit 104, a general name associated with the kth product information by searching.

  (Step S113) The product position information acquisition unit 106 acquires product position information corresponding to the general name acquired in step S112, as in step S111. The product position information acquisition unit 106 associates the acquired product position information with the general name acquired in step S112, the k-th product information, and information indicating that there is no stock, and stores it in a storage unit (not shown). accumulate. The product position information acquisition unit 106 reads, for example, a template or the like when there is no stock prepared in advance from a storage unit or the like (not shown), inserts a general name and product information into this template, Because "information" is out of stock, we will guide you to the "general name" sales floor. A character string or the like indicating that the item is out of stock and introducing a sales place of an alternative product may be acquired and accumulated. Then, the process proceeds to step S108.

  (Step S114) The purchase route information acquisition unit 108 performs route search using the passage information stored in the passage information storage unit 107, and obtains one or more pieces of product position information acquired by the product position information acquisition unit 106. Get purchase route information as a waypoint. Purchase route information that is preferably the shortest route is acquired.

  For example, the purchase route information acquisition unit 108 acquires a plurality of different permutations, preferably all permutations when arranging one or more product position information, and for each permutation acquired, in the order indicated by the permutations. One or two or more routes passing through the positions indicated by the one or more product position information are acquired by route search. Then, the shortest route among the acquired routes is acquired as purchase route information.

  Alternatively, the purchase route information acquisition unit 108 acquires, by route search, the shortest route that passes through the position indicated by one or more product position information in the order indicated by the permutation for each permutation acquired above. Then, the shortest route among the acquired routes may be acquired as purchase route information.

  Note that the product position information acquisition unit 106 determines whether or not there is one piece of product position information. If there are a plurality of pieces of product position information acquired, the position indicated by the product position information is used as a waypoint. The purchase route information may be acquired by performing the route search described above, and in the case of one, the purchase route information may be acquired by performing a route search using the product position information as an arrival point.

  The purchase route information acquisition unit 108 accumulates the acquired route information in a storage unit (not shown) or the like. The route information includes product information associated with product location information corresponding to each waypoint, general name, information indicating that the product is not handled, information indicating that there is no stock, step S111 and step It is preferable that character string information and the like acquired in S113 is associated with each waypoint.

  (Step S115) The output unit 109 acquires guide information corresponding to the current position indicated by the current position information, using the current position information acquired in Step S102 and the purchase route information acquired in Step S114. The guidance information is information including, for example, information indicating a position corresponding to the AR marker in the in-store video and information specifying an image such as an arrow indicating a traveling direction along the purchase route information displayed at this position. is there. Further, in addition to these, information for deforming the image indicating the traveling direction may be included in accordance with the distortion of the AR marker due to perspective. In addition, the purchase route information as described above has no product information associated with the product location information associated with each waypoint, general name, information indicating that the product is not handled, or no inventory. Guidance information including information indicating the character string information and character string information obtained in step S111 and step S113 may be obtained as guidance information for guiding each waypoint.

  (Step S116) The output unit 109 outputs the guidance information acquired in Step S115. For example, the output unit 109 transmits guidance information to the terminal device 2. Then, the process returns to step S102.

  In the flowchart of FIG. 2, it is determined whether or not the current position information acquired in step S102 is the arrival point indicated by the purchase route information. If it is the arrival point, the guide information indicating that the arrival point has been reached. Or the like may be output to end the processing of the flowchart of FIG. Whether or not the current position information is an arrival point may be determined in any way, for example, when the distance between the point indicated by the current position information and the arrival point is equal to or less than a predetermined distance. You may determine that it is a point, or you may determine that it is an arrival point when the AR marker detected to acquire the current position information is the AR marker closest to the arrival point.

  In the above, in step S111, as the product position information corresponding to the general name acquired in step S110, one product information associated with the same general name as the general name acquired in step S110 is stored in the product name storage unit 104. The product position information of the detected product information may be acquired from the product position information storage unit 103 as product position information corresponding to the kth product information. At this time, guidance information including one piece of product information detected from the product name storage unit 104 may be output by the output unit 109 described later. By displaying this one item information when displaying an image indicating the kth item information, the customer is shown which route to purchase the item information instead of the kth item information. Can do. The same applies to step S113.

  In the flowchart of FIG. 2, the process is terminated by powering off or a process termination interrupt.

  Next, an example of operation | movement of the terminal device 2 of the merchandise guidance system 1000 is demonstrated using the flowchart of FIG.

  (Step S <b> 201) The product information transmission unit 202 transmits one or more product information stored in the product information storage unit 201 to the product guidance device 1. For example, the merchandise information transmission unit 202 transmits merchandise information when a reception unit (not shown) or the like receives an operation for guiding the purchase of a product from a customer.

  (Step S202) The photographing unit 203 photographs the inside of the store and acquires the in-store video. For example, when the imaging unit 203 receives an imaging instruction from a customer, the imaging unit 203 captures the inside of the store and acquires an in-store video that is a still image. In this case, the processing may be waited until the photographing instruction is accepted. Further, the photographing unit 203 may continue photographing the in-store video that is a moving image.

  (Step S203) The current position related information transmission unit 204 transmits current position related information. For example, the current position related information transmission unit 204 transmits the in-store video acquired in step S202 to the product guidance device 1 as current position related information. When the in-store video is a moving image, for example, an image of one frame in the moving image is taken out and transmitted to the product guide device 1 at every predetermined time interval or according to a predetermined trigger. May be.

  (Step S204) The guidance information receiving unit 205 determines whether or not guidance information has been received from the product guidance device 1. If received, the process proceeds to step S205. If not received, the process returns to step S204.

  (Step S205) The display unit 206 displays the image corresponding to the guidance information received in step S204 and the in-store video acquired in step S202 in an overlapping manner. Then, the process returns to step S202.

  As described above, the guide information includes product information, general name, information indicating that the product is not handled, information indicating that the product is not in stock, and a character string obtained in step S111 or step S113. Such information may also be displayed. Further, these pieces of information may be appropriately arranged and displayed on the template described in step S111 or step S113.

  In the flowchart of FIG. 3, the process ends when the power is turned off or the process is terminated.

  Hereinafter, a specific operation of the merchandise guidance system 1000 in the present embodiment will be described. Here, a case will be described in which the product guide device 1 is a server device connected to a network such as the Internet, and the terminal device 2 is a multifunctional mobile phone called a smartphone connected to the same network. The terminal device 2 is assumed to include, for example, a touch panel (not shown) as an input device.

  For example, when the customer operates the touch panel at home or the like and inputs the product names of a plurality of products desired to be purchased to the terminal device 2, such as “Bio-Z”, the product name is accepted. A reception unit (not shown) of the terminal device 2 accumulates product information that is a plurality of received product names in the product information storage unit 201. Here, it is assumed that product information of three products is accumulated.

  FIG. 4 is a product information management list for managing a plurality of product information accumulated in the product information storage unit 201.

  Next, it is assumed that a customer enters a certain store A and purchases a product to display a guide for purchasing the product indicated by the product information stored in the product information storage unit 201 in order to purchase the product. For example, assume that a button (not shown) for displaying guidance is pressed.

  The merchandise information transmission unit 202 transmits the four merchandise information stored in the merchandise information storage unit 201 to the merchandise guidance device 1 in accordance with a customer operation. Here, for example, it is assumed that the product information management list shown in FIG.

  The product information receiving unit 101 of the product guide device 1 receives product information transmitted from the terminal device 2. Here, for example, it is assumed that the product information management list shown in FIG. 4 is received. The merchandise information receiving unit 101 temporarily stores the received merchandise information management list in a storage unit or the like (not shown).

  Next, the customer photographs the inside of the store with the photographing unit 203 of the terminal device 2 in order to display the product guidance. For example, the customer photographs the traveling direction with the photographing unit 203. Thereby, the imaging unit 203 acquires the in-store video obtained from the store. Here, the in-store video is a color moving image.

  Here, a plurality of AR markers associated with different identifiers (hereinafter referred to as marker identifiers) are pasted on the aisle in the store A, and each marker identifier indicates the position of the corresponding AR marker. Is stored in advance in a storage unit (not shown) of the product guidance device 1. The AR marker is, for example, a sheet on which a predetermined figure having a circular outer shape is drawn. Here, it is assumed that different character strings are arranged in the plurality of AR markers. The size of the AR marker is such that the AR marker can be identified in the in-store image acquired by the terminal device 2, for example, the horizontal width and the vertical width are in a range from about 500 mm to 1500 mm, respectively. Is preferred. However, the size may be different from the above depending on the shape of the marker, the image drawn on the marker, and the like. For example, AR markers are arranged at intervals of several meters on a passage in a store.

  It is assumed that the in-store video captured and acquired by the customer includes the AR marker placed on the floor in the store. In addition, when a customer shoots an in-store video, it is preferable to shoot so that an AR marker arranged on the front floor is reflected as much as possible.

  The current position related information transmission unit 204 transmits the in-store video acquired by the photographing unit 203 to the product guidance device 1 as current position related information. Here, from the in-store video that is a moving image, the in-store video that is one frame image is transmitted to the product guide device 1 as current position related information at a timing specified in advance.

  FIG. 5 is a diagram showing an in-store video transmitted by the current position related information transmitting unit 204 as current position related information. As shown in the in-store video shown in FIG. 5, AR markers 51 and 52 are arranged on the floor.

  The current position information acquisition unit 102 of the merchandise guidance device 1 receives the in-store video that is current position related information from the terminal device 2.

  The current position information acquisition unit 102 acquires current position information using the in-store video that is the received current position related information. Hereinafter, processing for acquiring current position information will be described.

FIG. 6 is a diagram showing marker position management information for managing an AR marker stored in a storage unit (not shown) of the product guidance device 1 and position information associated with the AR marker. The marker position management information has attributes of “marker ID” that is an identifier of the AR marker and “marker position information” that is position information of the AR marker. Here, it is assumed that the “marker position information” is coordinates. In the coordinates, x ₁₀₁ , y _{101, and the} like are arbitrary values. Here, on the surface of the AR marker, a marker identifier that is the same character string as the attribute value of “marker ID” of each marker is arranged at a predetermined position.

  FIG. 7 is a diagram (FIG. 7 (a)) in which the in-store video is binarized for explaining the process of detecting the AR marker, and a diagram showing a state in which the marker image is detected (FIG. 7 (b)). ).

  First, the current position information acquisition unit 102 binarizes the in-store video shown in FIG. 5 as shown in FIG. 7A, and detects an AR marker image in the binarized in-store video. For example, here, assuming that the AR marker is circular, in the binarized in-store video, a rectangular area having a substantially circular shape, a substantially elliptical shape, or the like corresponding to the circular AR marker is detected. Here, the image in the detected area is called a marker image. Here, for example, as shown in FIG. 7B, marker images 71 and 72 are detected. When a plurality of marker images are detected, for example, the marker image located at the lowermost side or the marker image having the largest size is used for the process of acquiring the current position information thereafter. Here, the marker image 71 corresponding to the AR marker 51 is used for the process of acquiring current position information.

  The current position information acquisition unit 102 deforms the detected marker image 71 so that the AR marker 51 has a circular shape, and performs character recognition using image recognition using pattern matching in the deformed marker image 71. Detect a string. Here, for example, a character string “m102” is detected. The character string detected here is an identifier of the AR marker as described above.

The current position information acquisition unit 102 detects a record (row) in which the value of “marker ID” matches the identifier “m102” of the AR marker detected above in the marker position management information shown in FIG. “(X ₁₀₂ , y ₁₀₂ )” that is the attribute value of “marker position information” of the recorded record is acquired as current position information. Further, information indicating the upward direction of the detected character string, for example, information such as a vector is acquired as direction information. It is assumed that the direction indicated by the information indicating the direction is a direction designated in advance, for example, the north direction. Then, the acquired current position information, direction information, and information indicating the amount of deformation performed when the marker image is deformed as described above are temporarily stored in a storage unit (not shown).

  FIG. 8 is an inventory information management table for managing the inventory information stored in the inventory information storage unit 105. The inventory information management table has attributes of “product name” and “inventory”. “Product name” is product information. “Inventory” is “inventory information”, and the attribute value when there is stock is “present”, and the attribute value when there is no stock is “none”.

  FIG. 9 is a product location information management table for managing product location information stored in the product location information storage unit 103. The product location information management table has attributes of “product name” and “product location”. The “product position” further has attributes of “plane” and “height”. “Product name” is product information. “Product position” is product position information. “Plane” is information indicating the position of the product on the plane in the product position information, and “Height” is information indicating the height of the product in the product position information. Here, “height” indicates the number of steps counted below the display shelf on which the product is located.

  FIG. 10 is a product name management table for managing general names of products stored in the product name storage unit 104. The product name management table has attributes of “product name” and “general name”. “Product name” is product information. “General name” is a general name of a product.

  FIG. 11 is a general product position information management table for managing general names and product position information stored in a storage unit (not shown). In the figure, “general name” and “product position” indicate the same attributes as “general name” in FIG. 10 and “product position information” in FIG. 9. .

  The product location information acquisition unit 106 acquires product location information for each of a plurality of product information as shown in FIG. 4 received by the product information reception unit 101.

First, the product position information acquisition unit 106 reads out the first product information, for example, “Bio-Z”, from among the plurality of product information received by the product information receiving unit 101 from a storage unit (not shown), as shown in FIG. In the inventory information management table shown, a record (row) in which the value of “product name” matches “Bio-Z” is searched. Here, the “stock” value of the detected record for detecting a matching record is acquired, and it is determined whether or not the acquired value is “present”. Here, since it is “present”, the product position information acquisition unit 106 determines that there is a stock, and the product position information acquisition unit 106 matches “Bio Z” in the product position information management table illustrated in FIG. 9. A record (row) is detected, and an attribute value of “product position” of the detected record, here, an attribute value of “plane” and an attribute value of “height” are acquired. Since there are a plurality of attribute values of “product position”, that is, product position information here, “(x ₁₀₂ , y ₁₀₂ )” that is the current position information acquired above and each of the plurality of product position information Is calculated, and the shorter distance is obtained. For example, it is assumed here that “(x ₃₀₃ , y ₃₀₃ )” has been acquired. Then, the product name “Bio-Z” and the acquired attribute value of “Product position” are associated with each other and stored in a storage unit (not shown).

  Next, the product position information acquisition unit 106 reads the second product information, for example, “Mummy Orange” from a storage unit (not shown), and the value of “product name” is stored in the inventory information management table shown in FIG. , Search for a record (line) that matches "Mummy Orange". Here, since a matching record is detected, it is determined whether or not the “stock” value of the record is “present”. Here, since it is not “present”, the product position information acquisition unit 106 determines that there is no stock, and the product position information acquisition unit 106 matches the record (product name) with “Mummy orange” in the product name management table illustrated in FIG. Line) is detected, and “laundry detergent” that is an attribute value of “general name” of the detected record is acquired. Then, the product position information of the record whose attribute value of “general name” in the general name position information management table shown in FIG. 11 is “laundry detergent” is used, and the position information of the first product information is the product position information management. Obtained in the same way as obtained from the table. Then, the product name “Mummy Orange” is associated with the acquired “product position” attribute value and information indicating that there is no stock (for example, information such as a flag indicating that there is no stock) It accumulates in a storage unit (not shown).

  Next, the merchandise position information acquisition unit 106 reads out the third merchandise information, for example, “Clear Tape A” from the storage unit (not shown), and in the inventory information management table shown in FIG. Searches for a record (row) that matches “clear tape A”. Here, since no matching record is detected, the product position information acquisition unit 106 detects a record (row) whose “product name” is “clear tape A” in the product name management table shown in FIG. Acquire “cellophane tape” that is an attribute value of “general name” of the detected record. Then, the product location information of the record whose attribute value of “general name” in the general name location information management table shown in FIG. 11 is “cellophane tape” is used, and the location information of the first product information is the product location information management. Obtained in the same way as obtained from the table. The product name “clear tape A” corresponds to the attribute value of the acquired “product position” and information indicating that the product is not handled (for example, information such as a flag indicating that the product is not handled). Then, it accumulates in a storage unit (not shown).

  The purchase route information acquisition unit 108 uses the product position information about the three product information acquired by the product position information acquisition unit 106 and the passage information stored in the passage information storage unit 107, and uses the product position information. Are searched for the shortest route that passes through all of them in any order. Note that the passage information is the same information as road information used in a general navigation device, except that the passage information is information indicating a passage in the store, and detailed description thereof is omitted here.

  As a result of the route search, the route that passes through the position indicated by the product position information acquired in the order of “Bio-Z”, “Clear Tape A”, and “Mummy Orange” is the shortest route. If it is determined, the purchase route information acquisition unit 108 acquires route information indicating this route as purchase route information and stores it in a storage unit (not shown). Here, for each waypoint of the purchase route information, product information corresponding to each waypoint is stored in association with each other. In addition, when the product position information acquisition unit 106 acquires information indicating that the product is not handled or information indicating that the product is not in stock for the product information corresponding to the waypoint, Is also stored in the purchase route information in association with the waypoints.

  Next, the output unit 109 uses the purchase route information to face the direction indicated by the direction information acquired by the current position information acquisition unit 102 to the position indicated by the current position information acquired by the current position information acquisition unit 102 The customer who is at the location acquires guidance information for the customer to follow the route indicated by the purchase route information. Specifically, information specifying an image of an arrow indicating a course to be displayed above an area where the AR marker 51 in the in-store video is arranged, and coordinates in the in-store video displaying the arrow Obtained as guide information. Moreover, the information of the character string explaining a course is acquired as guidance information. Further, using the information on the correction amount when correcting the shape of the marker image, information on the correction amount for deforming the image of the arrow in accordance with the perspective of the in-store video is acquired as guide information. Then, the output unit 109 transmits guide information including these pieces of information to the terminal device 2.

  The guide information receiving unit 205 of the terminal device 2 receives the guide information transmitted from the product guide device 1. Then, the display unit 206 displays an image corresponding to the received guidance information so as to overlap the in-store video acquired by the imaging unit 302. Specifically, an arrow image designated by the guidance information is acquired from a storage unit (not shown), and is positioned above the AR marker 51, which is the position designated by the guidance information on the in-store video as shown in FIG. The correction information is arranged according to the correction amount of the guide information. Moreover, the character string information describing the course of the guide information and an image showing the product information are displayed below the in-store video.

  FIG. 12A is a diagram illustrating a display example by the display unit 206 of the terminal device 2.

  Similarly, the terminal device 2 acquires the in-store video and transmits it to the product guide device 1, and the current position information acquisition unit 102 of the product guide device 1 uses the received in-store video to Get information. Then, the output unit 109 acquires guide information using the newly acquired current position information, the purchase route information acquired above, and the like, and transmits it to the terminal device 2. Then, the display unit 206 of the terminal device 2 displays an image corresponding to the received guidance information on the in-store video.

  Here, the distance between the current position information acquired by the current position information acquisition unit 102 and the first transit point, that is, the product position information of “Bio-Z” that is the first product information is a predetermined distance. If it is shorter, the output unit 109 determines that the customer has arrived at the first waypoint. Then, the output unit 109 uses the “height” information of the current position information to display information indicating an arrow indicating a location in the height direction, and character string information corresponding to the “height” information Is obtained and transmitted to the terminal device 2.

  The terminal device 2 displays an image obtained by superimposing the image indicated by the guidance information, that is, the image of the arrow indicating the height at which the product is located and the image of the character string indicating the height at which the product is located on the in-store video. To do.

  FIG. 12B is a diagram illustrating a display example of an image indicating the height at which the product is located.

  If the current position information acquired by the current position information acquisition unit 102 is a position between the first waypoint and the second waypoint, the output unit 109 displays an image of an arrow indicating the traveling direction. Along with the information to be specified, the product name “Clear Tape A” which is the product information associated with the second waypoint, the general name “Cellophane Tape”, and information indicating that the product has not been handled are included. The guidance information is transmitted to the terminal device 2.

  When the guide information receiving unit 205 of the terminal device 2 receives the guide information, the display unit 206 detects that the received guide information includes information indicating that the product is not handled, and is prepared in advance. Insert the product name “Clear Tape A” and the general name “Cellophane Tape” in the guide information in the template when there is no handling, and the word “Cellophane Tape (Clear Tape A is not handled)” The image of the column is displayed in an overlapped manner with the in-store video information together with the arrow image specified by the guide information.

  FIG. 12C is a display example when the product is not handled.

  Also, assuming that the current position information acquired by the current position information acquisition unit 102 is a position between the second waypoint and the third waypoint, the output unit 109 displays an image of an arrow indicating the traveling direction. Along with the information to be specified, the product name “Mummy Orange”, which is product information associated with the third waypoint, the general name “tablet detergent”, and information indicating that the product is not in stock The guidance information is transmitted to the terminal device 2. The third waypoint is the arrival point indicated by the purchase route information here.

  When the guide information receiving unit 205 of the terminal device 2 receives the guide information, the display unit 206 detects that the received guide information includes information indicating that the product is not in stock, and is prepared in advance. Insert the product name “Mummy Orange” and the general name “Dishwashing Detergent” in the guide information in the template when there is no stock, and the string “Dishwashing Detergent (Mummy Orange is out of stock)” The image is displayed over the in-store video information together with the arrow image designated by the guide information.

  FIG. 12D is a display example when the product is out of stock.

  As described above, according to the present embodiment, a route for purchasing one or more products in a store is searched, and guidance for proceeding along the searched route is provided to the customer according to the current location of the customer. The customer can be appropriately guided to the position where the desired product is arranged in the store.

  Further, the AR marker is used to detect the current location of the customer from the video imaged in the store, and the guidance corresponding to the current location is superimposed on the video imaged in the store to display the AR marker in the store. It is possible to identify the customer's current location easily and accurately without installing equipment for identifying the customer's current location in the store, and to actually provide an image for guidance. It is possible to present the route in an easy-to-understand manner by displaying the images in the store in an overlapping manner.

  In addition, in the said embodiment, although the case where the merchandise guidance apparatus 1 performed guidance of one store was demonstrated, the merchandise guidance apparatus 1 is the merchandise position information, inventory information, and passage information which were each matched with the some store. Information on the location where the customer is located, and the purchase route information corresponding to the store where the customer is located. Information may be output. By doing in this way, guidance of goods can be performed appropriately about a plurality of stores. The same applies to the following embodiments.

  Here, the information associated with a plurality of stores is information associated with a store identifier such as a store name or a store code, for example. Moreover, you may determine how a customer is in which store. For example, as described above, the store where the customer is located may be specified from the access point or the like of the terminal device 2, or the terminal device 2 may have a GPS unit and the current position acquired by the GPS unit. May indicate that the store corresponding to the indicated position is the store where the customer is located.

(Embodiment 2)
The merchandise guidance apparatus according to the present embodiment is configured such that processing corresponding to the merchandise guidance system described above can be realized by a single merchandise guidance apparatus.

  FIG. 13 is a block diagram of the merchandise guidance device 3 in the present embodiment.

  The merchandise guidance device 3 includes a merchandise information storage unit 300, a merchandise information reception unit 301, a photographing unit 302, a current position information acquisition unit 303, a merchandise position information storage unit 103, a merchandise name storage unit 104, an inventory information storage unit 105, a merchandise position An information acquisition unit 106, a passage information storage unit 107, a purchase route information acquisition unit 108, and an output unit 304 are provided. In the figure, the same reference numerals as those in FIG. 1 denote the same or corresponding parts.

  The product information storage unit 300 stores one or more product information. It does not matter how the product information is stored in the product information storage unit 300. The product information storage unit 300 is the same as the product information storage unit 201 described above, and detailed description thereof is omitted here.

  The product information receiving unit 301 receives one or more product information stored in the product information storage unit 300. Specifically, the product information receiving unit 301 reads product information stored in the product information storage unit 300. The other configuration of the merchandise information receiving unit 301 is the same as that of the merchandise information receiving unit 101, and detailed description thereof is omitted here.

  The photographing unit 302 photographs the inside of the store that provides the product and acquires the in-store video. The imaging unit 302 is the same as the imaging unit 203 described above, and detailed description thereof is omitted here.

  The current position information acquisition unit 303 acquires the customer's current position information. The customer's current position information here is, for example, information on the current position of the product guide device 3 and the current position of the customer who owns the product guide device 3.

  For example, the current position information acquisition unit 303 acquires the current position information by the same process as when the current position related information transmission unit 204 of the terminal device 2 described above acquires the current position information as the current location related information of the terminal device 2. To do.

  For example, the current position information acquisition unit 303 acquires the current position information using the in-store video acquired by the imaging unit 302. For example, the current position information acquisition unit 303 may acquire the current position information from the in-store video by performing the same process as the current position information acquisition unit 102. For example, the current position information acquisition unit 303 may detect the AR marker or the like from the in-store video and acquire the current position information.

  Further, the current position information acquisition unit 303 may acquire direction information in the same manner as the current position information acquisition unit 102. The other configuration of the current position information acquisition unit 303 is the same as that of the current position information acquisition unit 102, and detailed description thereof is omitted here.

  The output unit 304 outputs guide information using the purchase route information acquired by the purchase route information acquisition unit 108. The output here is specifically a display. Here, outputting the guidance information is a concept including outputting an image or the like corresponding to the guidance information. For example, the output unit 304 displays the in-store video acquired by the imaging unit 302 and an image corresponding to the guidance information in an overlapping manner. In addition, regarding the process of displaying the in-store video acquired by the photographing unit 302 and the image corresponding to the guidance information in an overlapping manner, the display unit 206 described above displays the in-store video and the image corresponding to the guidance information in an overlapping manner. This process is the same as that to be performed, and detailed description thereof is omitted here. Further, other processing of the output unit 304, for example, processing for obtaining guidance information, is the same as that of the output unit 109, the display unit 206, and the like, and detailed description thereof is omitted here.

  Next, an example of operation | movement of the merchandise guidance apparatus 3 is demonstrated using the flowchart of FIG. In the figure, the same reference numerals as those in FIG. 2 denote the same or corresponding parts. In addition, in FIG. 2, the processing performed by the current position information acquisition unit 102 is performed by the current position information acquisition unit 303 here.

  (Step S1401) The product information receiving unit 301 receives product information. Specifically, the product information stored in the product information storage unit 300 is read.

  (Step S1402) The photographing unit 302 photographs the inside of the store and acquires the in-store video. This process is, for example, the same process as step S202 in FIG. Then, the process proceeds to step S102.

  (Step S1403) The output unit 304 outputs the guidance information acquired in step S115. Specifically, an image obtained by superimposing the image indicated by the guidance information acquired in step S115 and the in-store video acquired in step S1402 is displayed. This process is, for example, the same process as step S205 in FIG. Then, the process returns to step S1402.

  Hereinafter, a specific example of the product guide device 3 according to the present embodiment will be described. Here, for example, a case where the product guide device 3 is a multi-function mobile phone such as a so-called smartphone will be described.

  For example, the product information input by the customer is accumulated in the product information storage unit 300 of the product guidance device 3. When the merchandise guidance device 3 accepts an instruction to guide at a store, the merchandise information accepting unit 301 reads one or more pieces of merchandise information stored in the merchandise information storage unit 300. Further, the current position information acquisition unit 303 detects the AR marker from the in-store video acquired by the shooting unit 302 by shooting the inside of the store, and acquires current position information and direction information. The merchandise position information acquisition unit 106 acquires the merchandise position information and the like using the merchandise information and the like read out in the same manner as in the specific example of the first embodiment. Then, the purchase route information acquisition unit 108 acquires the purchase route information using the acquired product position information and the information acquired by the current position information acquisition unit 303. Then, the output unit 304 acquires the guide information using the purchase route information and the current position information, and displays the in-store video and the image corresponding to the guide information in a superimposed manner. For example, an image such as an arrow indicating the direction of travel is superimposed on the area on the AR marker of the in-store video.

  As described above, according to the present embodiment, the same effects as those of the first embodiment can be obtained, and the inside of the store can be guided only by the product guide device 3, and the system configuration can be simplified.

  In each of the above embodiments, each process (each function) may be realized by centralized processing by a single device (system), or by distributed processing by a plurality of devices. May be.

  Further, in each of the above embodiments, it goes without saying that two or more communication means (product information transmission unit, guidance information reception unit, etc.) existing in one device may be physically realized by one medium. Yes.

  In each of the above embodiments, each component may be configured by dedicated hardware, or a component that can be realized by software may be realized by executing a program. For example, each component can be realized by a program execution unit such as a CPU reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory. At the time of execution, the program execution unit may execute the program while accessing a storage unit (for example, a recording medium such as a hard disk or a memory).

  In addition, the software which implement | achieves the merchandise guidance apparatus in each said embodiment is the following programs. That is, this program includes a product position information storage unit that stores product information and product position information that is information indicating a position where the product indicated by the product information in the store is arranged, A passage information storage unit that stores passage information that is information indicating the passage of the product, a computer that can access the product information reception unit that receives one or more product information that is information indicating the product, and a store that provides the product Current position information acquisition unit for acquiring current position information, which is information indicating the current position of the customer in the product, and product position information corresponding to one or more product information received by the product information reception unit from the product position information storage unit Using the product location information acquisition unit, the passage information, the current location information acquired by the current location information acquisition unit, and the product location information acquired by the product location information acquisition unit, A purchase route information acquisition unit that acquires purchase route information, which is information indicating a purchase route that is a route for purchasing a product indicated by one or more product information received by the information receiving unit, and a purchase route information acquisition unit For functioning as an output unit that outputs guidance information that is information for guiding a customer to a position where a product indicated by one or more product information received by the product information receiving unit is received using the purchase route information It is a program.

  The software that realizes the terminal device in the above embodiment is the following program. That is, this program uses a computer that can access a product information storage unit in which one or more product information that is information indicating a product is stored, one or more product information stored in the product information storage unit, and a product guidance device. Information corresponding to the in-store image acquired by the image capturing unit, the image capturing unit for capturing the in-store image that is the image in the store, The current position related information transmitting unit that transmits the current position related information that is information related to the current position of the customer in the store to the product guide device, and one or more product information transmitted from the product guide device by the product information transmitting unit The guide information receiving unit that receives the guide information that is information for guiding to the position where the product to be displayed is arranged, the in-store video acquired by the photographing unit, and the image corresponding to the guide information are displayed in an overlapping manner. Is a program for functioning as that the display unit.

  In each of the above programs, the function realized by the program does not include a function that can be realized only by hardware. For example, a function that can be realized only by hardware such as a modem or an interface card in an acquisition unit that acquires information or an output unit that outputs information is not included in the function realized by the program.

  Further, the computer that executes this program may be singular or plural. That is, centralized processing may be performed, or distributed processing may be performed.

  FIG. 15 is a schematic diagram illustrating an example of an external appearance of a computer that executes the program and realizes the commodity guidance device according to the embodiment. The above-described embodiment can be realized by computer hardware and a computer program executed on the computer hardware.

  In FIG. 15, the computer system 900 includes a computer 901 including a CD-ROM (Compact Disk Read Only Memory) drive 905, a keyboard 902, a mouse 903, and a monitor 904.

  FIG. 16 is a diagram showing an internal configuration of the computer system 900. In FIG. 16, in addition to the CD-ROM drive 905, a computer 901 is connected to an MPU (Micro Processing Unit) 911, a ROM 912 for storing a program such as a bootup program, and the MPU 911. A RAM (Random Access Memory) 913 that temporarily stores and provides a temporary storage space, a hard disk 914 that stores application programs, system programs, and data, and a bus 915 that interconnects the MPU 911, ROM 912, and the like Prepare. The computer 901 may include a network card (not shown) that provides connection to the LAN.

  A program that causes the computer system 900 to execute the functions of the product guide device according to the above-described embodiment may be stored in the CD-ROM 921, inserted into the CD-ROM drive 905, and transferred to the hard disk 914. Instead, the program may be transmitted to the computer 901 via a network (not shown) and stored in the hard disk 914. The program is loaded into the RAM 913 when executed. The program may be loaded directly from the CD-ROM 921 or the network.

  The program does not necessarily include an operating system (OS), a third party program, or the like that causes the computer 901 to execute the function of the commodity guide device according to the above-described embodiment. The program may include only a part of an instruction that calls an appropriate function (module) in a controlled manner and obtains a desired result. How the computer system 900 operates is well known and will not be described in detail.

  The present invention is not limited to the above-described embodiments, and various modifications are possible, and it goes without saying that these are also included in the scope of the present invention.

  As described above, the product guide device according to the present invention is suitable as a device that guides the inside of a store, and is particularly useful as a device that guides the route of a product that a customer desires to purchase.

1, 3 Product guidance device 2 Terminal device 101, 301 Product information receiving unit 102, 303 Current position information acquisition unit 103 Product position information storage unit 104 Product name storage unit 105 Inventory information storage unit 106 Product position information acquisition unit 107 Passage information storage Unit 108 Purchase route information acquisition unit 109, 304 Output unit 201, 300 Product information storage unit 202 Product information transmission unit 203, 302 Imaging unit 204 Current position related information transmission unit 205 Guidance information reception unit 206 Display unit 1000 Product guidance system

Claims (12)

A product information receiving unit that receives one or more pieces of product information that is information indicating a product;
A current position information acquisition unit that acquires current position information, which is information indicating a customer's current position in a store that provides products;
A merchandise position information storage unit in which merchandise information and merchandise position information which is information indicating a position where the merchandise indicated by the merchandise information in the store is arranged are stored in association with each other;
A product position information acquisition unit that acquires one or more product position information corresponding to the one or more product information received by the product information reception unit from the product position information storage unit;
An aisle information storage unit that stores aisle information that is information indicating the aisle in the store;
Using the passage information, the current position information acquired by the current position information acquisition unit, and one or more product position information acquired by the product position information acquisition unit, one or more items received by the product information reception unit A purchase route information acquisition unit that acquires purchase route information that is information indicating a purchase route that is a route for purchasing the product indicated by the product information;
Guidance information that is information for guiding the customer to the position where the product indicated by the one or more product information received by the product information receiving unit using the purchase route information acquired by the purchase route information acquisition unit The product guidance device provided with the output part which outputs. The product information receiving unit receives a plurality of product information,
The product position information acquisition unit acquires a plurality of product position information respectively corresponding to the plurality of product information received by the product information,
The purchase route information acquisition unit uses the plurality of product position information acquired by the product position information acquisition unit to purchase a plurality of products indicated by the plurality of product information received by the product information reception unit. The merchandise guidance apparatus according to claim 1, wherein purchase route information indicating a purchase order is acquired. The product guidance device according to claim 1 or 2, wherein the product location information storage unit stores a plurality of product location information in association with each product information. The product location information acquisition unit is acquired by the current location information acquisition unit among a plurality of product location information corresponding to the product information received by the product information reception unit stored in the product location information storage unit. The product guidance device according to claim 3, wherein product location information closest to the current position is acquired. The product position information has information in the height direction where the corresponding product is arranged,
5. The product guidance device according to claim 1, wherein the output unit outputs guidance information further including information for guiding a position in a height direction where the product is arranged. A product name storage unit that stores the product information and the general name of the product indicated by the product information in association with each other;
The product position information acquisition unit acquires a general name corresponding to the product information from the product name storage unit when the product indicated by the product information received by the product information reception unit is not arranged in the store. The product guidance device according to any one of claims 1 to 5, wherein the product location information associated with the general name is acquired. An inventory information storage unit that stores product information and stock information indicating a stock status of the product indicated by the product information;
The product output device according to any one of claims 1 to 6, wherein the output unit performs output according to product inventory information indicated by the product information received by the product information receiving unit. A product information storage unit that stores one or more product information that is information indicating a product;
A shooting section that captures the inside of the store that provides the product and obtains the in-store video that is the video in the store;
The product information receiving unit receives product information stored in the product information storage unit,
The current position related information acquisition unit acquires the current position information using the in-store video,
The product output device according to any one of claims 1 to 7, wherein the output unit displays an in-store video acquired by the photographing unit and an image corresponding to the guide information in an overlapping manner. A terminal device constituting a product guidance system including a product guidance device and a terminal device,
A product information storage unit that stores one or more product information that is information indicating a product;
A product information transmission unit that transmits one or more pieces of product information stored in the product information storage unit to the product guidance device;
An imaging unit that captures the in-store video, which is an image in the store, by shooting inside the store that provides the product,
A current position related information transmitting unit that transmits information on the current position related information that is information related to the current position of the customer in the store to the product guidance device, the information corresponding to the in-store video acquired by the photographing unit;
A guide information receiving unit that receives guide information that is information for guiding from the product guide device to a position where the product indicated by the one or more product information transmitted by the product information transmitting unit is disposed;
A terminal device comprising: a display unit that displays an in-store video acquired by the photographing unit and an image corresponding to the guidance information in an overlapping manner. A current location information acquisition unit, a product location information storage unit in which product information is stored in association with product location information, which is information indicating the location where the product indicated by the product information in the store is located, A product information method performed using a position information acquisition unit, a passage information storage unit storing passage information that is information indicating a passage in a store, a purchase route information acquisition unit, and an output unit,
A product information receiving step in which the current position information acquisition unit receives one or more pieces of product information that is information indicating a product;
The current position information acquisition unit acquires current position information, which is information indicating a customer's current position in a store providing a product,
The product position information acquisition unit acquires the product position information corresponding to one or more product information received in the product information reception step from the product position information storage unit,
In the product information reception step, the purchase route information acquisition unit uses the passage information, the current position information acquired in the current position information acquisition step, and the product position information acquired in the product position information acquisition step. A purchase route information acquisition step for acquiring purchase route information that is information indicating a purchase route that is a route for purchasing the product indicated by the received one or more product information;
The output unit uses the purchase route information acquired in the purchase route information acquisition step to guide the customer to the position where the product indicated by the one or more product information received in the product information reception step is arranged. A product guide method comprising: an output step of outputting guide information as information. Product location information storage unit in which product information and product location information, which is information indicating a location where the product indicated by the product information in the store is arranged, are stored in association with each other, and information indicating a passage in the store A passage information storage unit in which passage information is stored, and a computer accessible to
A product information receiving unit that receives one or more pieces of product information that is information indicating a product;
A current position information acquisition unit that acquires current position information, which is information indicating a customer's current position in a store that provides products;
A product location information acquisition unit that acquires product location information corresponding to one or more product information received by the product information reception unit from the product location information storage unit;
Using the passage information, the current position information acquired by the current position information acquisition unit, and the product position information acquired by the product position information acquisition unit, one or more product information received by the product information reception unit A purchase route information acquisition unit that acquires purchase route information that is information indicating a purchase route that is a route for purchasing the indicated product;
Guidance information that is information for guiding the customer to the position where the product indicated by the one or more product information received by the product information receiving unit using the purchase route information acquired by the purchase route information acquisition unit To function as an output unit that outputs. A computer that can access a product information storage unit in which one or more product information that is information indicating a product is stored,
A product information transmission unit that transmits one or more pieces of product information stored in the product information storage unit to a product guidance device;
An imaging unit that captures the in-store video, which is an image in the store, by shooting inside the store that provides the product,
A current position related information transmitting unit that transmits information on the current position related information that is information related to the current position of the customer in the store to the product guidance device, the information corresponding to the in-store video acquired by the photographing unit;
A guide information receiving unit that receives guide information that is information for guiding from the product guide device to a position where the product indicated by the one or more product information transmitted by the product information transmitting unit is disposed;
A program for causing an in-store video acquired by the photographing unit and an image corresponding to guidance information to be displayed in a superimposed manner.