JP2018106482A - Image forming system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable a presentation of a location of a nearby sales store of a commercial product specified by a user together with a commercial product dealing status on a map.SOLUTION: An image forming device 1 stores respective commercial product images of multiple sales commercial products on sale at multiple sales stores in association with location information of the respective sales stores, receives, as a to-be-searched image 50, the input of the commercial product image of the commercial product for searching the sales store, searches the commercial product image of the sales commercial product matching the commercial product of the to-be-searched image 50 from a commercial product information storing unit, acquires multiple commercial product images obtained by searching in the sequence of a shorter distance from the current location to each sales store, and creates a single map 81 that indicates the locations of the respective sales stores associated with the multiple acquired commercial product images and the current location, and prints the single map 81. The single map 81 has an identification marking 84 printed thereon, and when a user views the single map 81 via an eyeglasses-type terminal 2, the identification marking 84 is recognized, and similar commercial product information is overlaid and displayed as supplementary information on the single map 81.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image forming system including an image forming apparatus and a glasses-type terminal.

  2. Description of the Related Art Conventionally, a technique for storing product information related to a product designated by a user as user memo information and displaying the stored memo information when the user logs in is known (for example, Patent Document 1). reference).

JP 2007-115220 A

  However, in the prior art, when the user actually picked up the product and confirmed it, there was a problem that it was not possible to easily show the user which store to go to.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a technique capable of presenting the location of a nearby store of a product instructed by a user on a map together with the product handling status.

The image forming system of the present invention is an image forming system including an image forming apparatus and a glasses-type terminal, and the image forming apparatus accepts an input of a product image of a product for searching for a store as a search target image. Refer to the search target image reception unit, the product information storage unit that stores the product images of the plurality of sales products sold by each of the plurality of stores in association with the position information of the respective stores, and the product information storage unit. The product image of the sales product that matches the product of the search target image received by the search target image reception unit is searched, and the product image obtained by the search is a distance from the current location to each store. A plurality of image search units that are acquired in a short order, and the sales location associated with the current location and the plurality of product images acquired by the image search unit. A map generation unit that generates a single map indicating the location of the store; and for each of the plurality of sales stores on the single map, the sales products in the order of decreasing similarity to the products of the search target image Rearranging product images to generate incidental information, and an identification mark arrangement unit for arranging an identification mark for each of the plurality of stores on the single map generated by the map generation unit A printing unit that prints the single map in which the identification mark is arranged, and a main body communication unit that transmits the auxiliary information generated by the auxiliary information generation unit to the glasses-type terminal, The terminal includes a terminal communication unit that receives the supplementary information from the image forming apparatus, a field image capturing unit that images a user's front field of view, and a single image from a field image captured by the field image capturing unit. A mark recognizing unit for recognizing the identification mark arranged on the map, and corresponding to the identification mark recognized by the mark recognizing unit, matching the incidental information to the position of the store on the single map And a display processing unit for overlay display.
The map generation unit generates a store map indicating the current location and the position of the store for each store of each of the plurality of product images acquired by the image search unit, and By overlapping the position of each dealer map from the current position to the dealer as the distance from the current position to the dealer becomes shorter, avoiding overlapping of the position of the dealer between the dealer maps based on the current location, The single map may be generated.
The map generation unit may overlap the store map located in the upper layer with a smaller map area.
The map generation unit adjusts the scale of the store map in the upper layer when the store map in the upper layer includes the position of the store in the store map in the lower layer. The map area of the store map in the lower layer may be enlarged.
In addition, the glasses-type terminal includes a motion detection unit that detects a motion of a user's hand, and the display processing unit is included in the incidental information according to the motion of the hand detected by the motion detection unit. The display of the product image may be performed.

  According to the present invention, it is possible to present the position of a nearby store of a product instructed by the user on the map together with the product handling status.

1 is a system configuration diagram of an image forming system according to an embodiment of the present invention. FIG. 2 is a functional block diagram illustrating a schematic configuration of the image forming apparatus illustrated in FIG. 1. It is a registration example of the merchandise information storage part shown in FIG. It is a functional block diagram which shows schematic structure of the glasses-type terminal shown in FIG. 3 is a flowchart showing a flow of image forming processing of the image forming apparatus shown in FIG. 2. 3 is a map for each store before a single map is generated by the map generator shown in FIG. 2. It is the single map produced | generated by superimposing the map for every store shown in FIG. It is a flowchart which shows the flow of the incidental information display process of the glasses-type terminal shown in FIG. FIG. 5 is an image diagram in which incidental information is displayed as an overlay on a single map by the glasses-type terminal shown in FIG. 4.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that, in the following embodiments, the same reference numerals are given to configurations showing similar functions.

  As shown in FIG. 1, an image forming system 100 according to the present embodiment includes an image forming apparatus 1 such as a printer, a copier, or a multifunction peripheral, and glasses connected to the image forming apparatus 1 via a network such as a wireless LAN. Type terminal 2. When the image forming apparatus 1 acquires the search target image 50 from the user, the image forming apparatus 1 searches for a store of products that match the search target image 50 and prints a single map 81 in which the store position and the identification mark are arranged. At the same time, the image forming apparatus 1 transmits similar product information similar to the search target image 50 in the merchandise handled by the store as supplementary information to the glasses-type terminal. The user can obtain an access map up to a store that sells the product of the search target image 50 by using the single map 81. When the single map 81 is viewed with the glasses-type terminal 2, the user can sell the map based on the identification mark 84. Similar product information at the store is displayed as overlay information on the single map 81 as supplementary information.

  Next, the image forming apparatus 1 will be described in detail with reference to FIG. The image forming apparatus 1 is a multifunction machine and is installed in a store such as a convenience store or a mall. The image forming apparatus 1 includes an operation unit 11, a document reading unit 12, a main body storage unit 13, an image processing unit 14, a printing unit 15, a position information acquisition unit 16, a main body communication unit 17, and a main body control unit 18.

  The operation unit 11 includes a touch panel and operation keys. The touch panel is provided with a transparent pressure-sensitive sensor on the surface of a display unit that displays various operation keys and image formation status, and functions as a display unit and an input unit. The operation keys include a numeric keypad, a reset key, a stop key, a start key, and the like.

  The document reading unit 12 emits light to a document fed by a document feeder (not shown) or a document placed on a platen glass by a user, and receives reflected light or the like to form a document image. It is a scanner that reads.

  The main body storage unit 13 is a storage unit such as a semiconductor memory or an HDD (Hard Disk Drive), and stores image data received via the main body communication unit 17, image data read by the document reading unit 12, and the like. The The main body storage unit 13 stores a product information storage unit 131, map information 132, and identification mark information 133.

  As shown in FIG. 3, the product information storage unit 131 is a database in which product images of a plurality of sales products sold at a plurality of stores are registered. In the product information storage unit 131, each product image, the store location, the store name, the price of the product, and the detailed information of the product (for example, product name, model number, color, material, quantity in stock, product description, etc.) Etc. are registered in association with each other. The store location has, for example, latitude and longitude as location information of the store. The merchandise information storage unit 131 may link the point of sales system (POS) terminal of the store with the price, the number of inventory, etc. of the merchandise.

  The map information 132 is general-purpose map information including a floor map. The identification mark information 133 is pattern data or image data for generating first to Nth identification marks 84 to be described later, and information corresponding to the first to Nth identification marks 84 is stored.

  The image processing unit 14 is means for performing predetermined image processing on the image data. In the image processing unit 14, for example, image improvement processing such as enlargement / reduction processing, density adjustment, and gradation adjustment is performed.

  The printing unit 15 is a printing unit that prints image data. For example, the printing unit 15 forms a latent image on the surface of the photosensitive drum based on the image data read from the main body storage unit 13, performs image formation using the latent image as a toner image with toner, and outputs the latent image from the photosensitive drum. The toner image is transferred to the recording paper, and the toner image is fixed on the recording paper and discharged. In addition, the printing unit 15 has a function of forming toner images using toners such as cyan, magenta, yellow, and black, respectively. By superimposing these toner images and fixing them on a recording sheet, full-color printing can be performed. It corresponds.

  The position information acquisition unit 16 has a function of acquiring the current location by GPS (Global Positioning System) or the like.

  The main body communication unit 17 has a function of transmitting / receiving various data to / from the glasses-type terminal 2 and an external terminal through wireless communication such as a wireless local area network (LAN). In addition, the main body communication unit 17 has a function of transmitting / receiving various data to / from an external terminal via a wired cable such as a LAN cable.

  The main body control unit 18 is connected to the operation unit 11, the document reading unit 12, the main body storage unit 13, the image processing unit 14, the printing unit 15, the position information acquisition unit 16, and the main body communication unit 17. The main body control unit 18 is an information processing unit such as a microcomputer including a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. The ROM stores a control program for controlling the operation of the image forming apparatus 1. The main body control unit 18 reads out a control program stored in the ROM by the CPU and develops the control program in the RAM, thereby controlling the entire apparatus. The main body control unit 18 also functions as a search target image receiving unit 181, an image search unit 182, a map generation unit 183, an incidental information generation unit 184, an identification mark arrangement unit 185, and an output control unit 186.

  The search target image receiving unit 181 has a function of receiving, as the search target image 50, an input of a product image of a product for searching for a store. Specifically, the search target image receiving unit 181 reads an article on which a product image of a product is posted by the document reading unit 12, or receives a product image transmitted from a mobile terminal possessed by the user such as a mobile phone or a smartphone. It can be received via the main body communication unit 17 and received as the search target image 50.

  The image search unit 182 refers to the product information storage unit 131 to search for a product image of a sales product that matches the product of the search target image 50 received by the search target image reception unit 181, and a product image obtained by the search Has a function of acquiring the top N items (N is a specific natural number) in the order of short distance from the current location to each store. Thereby, the image search part 182 acquires a 1st-Nth product image as a search result.

  The map generation unit 183 has a function of generating a single map 81 that indicates the current location and the respective store positions associated with the first to Nth product images acquired as search results. Specifically, the map generation unit 183 includes the current location acquired by the location information acquisition unit 16, the store location, and the store location for each store where the first to Nth products acquired by the image search unit 182 exist. Store map (first to Nth store maps) is generated. Then, the map generation unit 183 uses the current location on the first to Nth dealer maps as a base point to avoid the overlapping of the dealer positions between the first to Nth dealer maps, and the first to Nth dealer maps. Are superimposed so that the distance from the current location to the store becomes shorter as the distance from the current location to the store becomes shorter.

  The supplementary information generation unit 184 is configured to search the search target image based on the product information storage unit 131 for each store associated with the first to Nth product images on the single map 81 generated by the map generation unit 183. Sort the product images of each store in descending order of similarity to 50 products (excluding the 1st to Nth product images acquired as search results), add product information, and add 1st to Nth supplements. It has a function to generate information.

  The identification mark arrangement unit 185 is configured to display the first to Nth identification marks 84 based on the identification mark information 133 for each store of the first to Nth product images on the single map 81 generated by the map generation unit 183. And each of the generated first to Nth identification marks 84 is arranged on a single map 81.

  The output control unit 186 causes the printing unit 15 to print the single map 81 in which the first to Nth identification marks 84 are arranged by the identification mark arrangement unit 185, and the first to first items generated by the incidental information generation unit 184. It has a function of transmitting N-accompanying information to the glasses-type terminal 2 via the main body communication unit 17.

  The glasses-type terminal 2 is a glasses-type wearable terminal that a user wears on the head. As shown in FIG. 4, the glasses-type terminal 2 includes a field-of-view imaging unit 21, a screen display unit 22, a terminal communication unit 23, a terminal storage unit 24, and a terminal control unit 25.

  As shown in FIG. 1, the field-of-view imaging unit 21 is a camera configured with a solid-state imaging device such as a CCD camera or a CMOS camera arranged toward the front side of the glasses-type terminal 2. Image. Hereinafter, an image captured by the visual field imaging unit 21 is referred to as a visual field image.

  When the user wears the glasses-type terminal 2, the screen display unit 22 has a half mirror 22a positioned immediately in front of the user's eyeball as shown in FIG. 1, and projects image light onto the half mirror 22a. In this way, various screens visible to the user are displayed.

  The terminal communication unit 23 has a function of transmitting / receiving various data to / from the image forming apparatus 1 by wireless communication such as a wireless LAN.

  The terminal storage unit 24 is a storage unit such as a semiconductor memory. The terminal storage unit 24 stores identification mark information 133 used for matching the first to Nth identification marks 84. Further, the terminal storage unit 24 stores first to Nth supplementary information received by the terminal communication unit 23 from the image forming apparatus 1.

  The terminal control unit 25 is connected to the visual field imaging unit 21, the screen display unit 22, the terminal communication unit 23, and the terminal storage unit 24, and performs operation control of the entire glasses-type terminal 2. The terminal control unit 25 is an information processing unit such as a microcomputer including a CPU, a ROM, a RAM, and the like. The ROM stores a control program for controlling the operation of the glasses-type terminal 2. The terminal control unit 25 controls the entire terminal by reading a control program stored in the ROM by the CPU and developing the control program in the RAM. The terminal control unit 25 functions as a mark recognition unit 251, a display processing unit 252, and an operation detection unit 253, respectively.

  The mark recognition unit 251 has a function of recognizing the first to Nth identification marks 84 from the view image based on the identification mark information 133.

  The display processing unit 252 acquires the first to Nth supplementary information received from the image forming apparatus 1 for each of the first to Nth identification marks 84 recognized by the mark recognition unit 251 from the terminal storage unit 24, and The first to Nth auxiliary information is displayed on the screen display unit 22 in the vicinity of each of the Nth identification mark 84.

  The operation detection unit 253 has a function of detecting the user's hand from the view field image and detecting the user's swipe operation or flick operation.

  Next, the flow of the image forming operation of the image forming apparatus 1 will be described with reference to FIGS. 5 and 6.

  When a product image of a product for searching for a store is input by the user, the search target image receiving unit 181 receives the input product image as the search target image 50 (step s11). The user causes the main body communication unit to read a product article that the dealer wants to know about by using the manuscript reading unit 12, or a product image captured by a camera of a mobile terminal such as a smartphone or a product image downloaded from the Internet. 17 can be transmitted.

  Subsequently, the image search unit 182 performs an image search for the search target image 50 received by the search target image reception unit 181 with reference to the product information storage unit 131, and among the sales products of each store, the search target The product image of the sales product that matches the product of the image 50 is searched (step s12).

  Subsequently, the image search unit 182 refers to the product information storage unit 131 and obtains the search based on the store location associated with the product image obtained by the search and the current location acquired by the location information acquisition unit. The obtained product images are rearranged in the order of short distance from the current location to the store location (step s13). Subsequently, the image search unit 182 acquires the top N-th (N is a specific natural number) product images among the rearranged product images as the first to Nth product images (step s14). As a result, among the stores registered in the product information storage unit 131, N stores are identified in the order from the current location among the stores having sales products that match the products in the search target image 50. The distance may be a linear distance from the current location to the store location.

  Subsequently, the map generation unit 183 generates, based on the map information 132, the first to Nth store maps in which the current location and the respective store positions are mapped for each of the first to Nth product images (Step S1). s15).

  FIG. 6 shows a case where first to third product images 82a to 82c are acquired as search results by the image search unit 182. The store A store associated with the first product image 82a has a distance from the current location to the store location of 20 m, and the store B store associated with the second product image 82b is from the current location to the store location. The distance is 510 m. Further, the store C store associated with the third product image 82c has a distance from the current location to the store location of 3.6 km.

  The first dealer map 80-1 shown in FIG. 6A shows the current location and the position of the dealer A, and the second dealer map 80-2 shown in FIG. 6B shows the current location. And the location of the store B is shown. In addition, the current location and the location of the store C are shown in the third store map 80-3 shown in FIG. Each of the first to third dealer maps 80-1 to 80-3 has a rectangular shape, and the current location (black circle shown in FIG. 6) is arranged at one corner of the four corners (lower left in FIG. 6). As a base point, the store position (black star shown in FIG. 6) is appropriately adjusted to have a specific size (map area) so that it is included in the rectangular area.

  Specifically, the map generation unit 183 generates the Nth store map with an output paper size such as A4, for example, and the first to N-th- One dealer map is generated so as to be approximately 1 / N,..., (N−1) / N of the Nth dealer map. For example, in FIG. 6, the first dealer map 80-1 is generated so as to be approximately 1/3 the size of the third dealer map 80-3, and the second dealer map 80-2 is the third dealer map 80-2. The store map 80-3 is generated so as to be approximately 2/3 the size of the store map 80-3, and the third store map 80-3 is generated so as to be the output paper size. In addition, when there is a floor map including a store position like the first store map 80-1, the store position can be shown on the floor map.

  Subsequently, the map generation unit 183 generates the single map 81 shown in FIG. 7 by superimposing the generated first to Nth store maps (step s16). The single map 81 uses the Nth dealer map as a base map, and the N-1 dealer map, the N-2 dealer map, ..., the first dealer map in this order on the Nth dealer map. It is generated by superimposing the current location as a base point. That is, the map generation unit 183 superimposes the first to Nth store maps so that the map is positioned higher as the distance from the current location to the store decreases. In addition, the first to (N-1) th dealer maps are each generated with a size of approximately 1 / N,..., (N-1) / N of the Nth dealer map. The map area becomes smaller as the store map is located in the upper layer. For this reason, when the first to Nth dealer maps are overlapped with each other based on the current location, it is possible to avoid the overlapping of the dealer positions between the dealer maps.

  Therefore, in the single map 81 shown in FIG. 7, the third dealer map 80-3 is generated as a base map and is generated on the base map in a size approximately 2/3 of the third dealer map 80-3. The second store map 80-2 overlaps, and the first store map 80-1 generated on the second store map 80-2 with a size of about 1/3 of the third store map 80-3 is obtained. overlapping.

  In addition, the map generation unit 183 adjusts the scale of the upper-level store map when the store map located in the upper layer includes the position of the store in the lower-level store map when overlapping. And expand to the map area of the lower-layer dealer map. For example, if the store locations of the store B and the store C shown in FIG. 7 are close, the store location of the store C may be included in the second store map 80-2. In this case, the map generation unit 183 adjusts the scale of the second store map 80-2 to expand the map area of the second store map 80-2 to the third store map 80-3 (base map). To do. Thereby, since the 2nd store map 80-2 becomes a large scale, it becomes easy to confirm a store position on a map. In this case, the map generation unit 183 changes so as to map the position of the store C on the enlarged second store map 80-2. That is, both the position of the store B and the position of the store C are shown on the enlarged second store map 80-2.

  Subsequently, the map generation unit 183 arranges corresponding first to Nth product images at each store location on the single map 81, and the price and store name in the vicinity of the first to Nth product images. The merchandise information including etc. is arranged (step s17). In FIG. 7, the first to third product images 82 a to 82 c are arranged so as to indicate the positions of the store A, the store B, and the store C, respectively, and are adjacent to the first to third product images 82 a to 82 c. First to third product information 83a to 83c are respectively arranged.

  Subsequently, the incidental information generation unit 184 refers to the product information storage unit 131 and arranges the product images of the sales products in the order similar to the products of the search target image 50 for each store of the first to Nth product images. Change (step s18). Subsequently, the supplementary information generation unit 184 refers to the product information storage unit 131, adds product information such as price and detailed information to the rearranged product image group, and obtains the first to Nth supplementary information. Generate (step s19). Thereby, similar merchandise information similar to the merchandise of the search target image 50 is generated as first to Nth supplementary information at each store.

  Subsequently, the identification mark arrangement unit 185 generates the first to Nth identification marks 84 based on the identification mark information 133 and arranges them on the single map 81 in association with the first to Nth product images ( Step s20). In FIG. 7, first to third identification marks 84a to 84c are arranged on the first to third product images 82a to 82c, respectively. In other words, the first to Nth identification marks 84 are arranged for each store of the first to third product images 82a to 82c, respectively.

  Subsequently, the output control unit 186 outputs the single map 81 in which the first to Nth identification marks 84 are arranged to the printing unit 15 to print the single map 81 (step s21), and generates incidental information. The first to Nth supplementary information generated by the unit 184 is transmitted to the glasses-type terminal 2 via the main body communication unit 17 (step s22), and this process ends.

  Next, with reference to FIG. 8, the flow of the incidental information display process of the glasses-type terminal 2 will be described.

  The terminal control unit 25 of the glasses-type terminal 2 stands by until the terminal communication unit 23 receives incidental information from the image forming apparatus 1 (No in step s41 and step s41). When the auxiliary information is received by the terminal communication unit 23 (Yes in step s41), the terminal control unit 25 stores the auxiliary information in the terminal storage unit 24 (step s42). Subsequently, the terminal control unit 25 causes the visual field imaging unit 21 to image the forward visual field (step s43) and functions as the mark recognition unit 251.

  The mark recognizing unit refers to 251 and the identification mark information 133 to determine whether or not the first to Nth identification marks 84 have been detected from the view field image (step s44). If the first to Nth identification marks 84 are not detected (No in step s44), the process returns to step s43.

  When the user captures the printed single map 81 over the field of view of the glasses-type terminal 2, the mark recognition unit 251 detects the first to Nth identification marks 84 from the view image based on the identification mark information 133 (step) Yes at s44). Subsequently, the display processing unit 252 causes the screen display unit 22 to display each of the first to Nth supplementary information received by the terminal communication unit 23 in the vicinity of each of the first to Nth identification marks 84 (step s45). This process ends.

  FIG. 9 is an image diagram in which the display processing unit 252 overlays the first to third auxiliary information 85 a to 85 c on the single map 81 in the view field image 90. The first to third auxiliary information 85a to 85c are displayed in the vicinity of the first to third identification marks 84a to 84c, respectively. At that time, the display processing unit 252 determines the size of the display area of the first to third auxiliary information 85a to 85c based on the size of the display area of the first to third product images 82a to 82c in the view image 90. And the first to third auxiliary information 85a to 85c are displayed in the determined size. Thereby, the user can visually recognize the first to third product images 82a to 82c and the first to third supplementary information 85a to 85c in the single map 81 without a sense of incongruity. Each of the first to third auxiliary information 85a to 85c includes a plurality of product images, and is displayed on the product images in descending order of similarity to the product of the search target image 50 by a drum roll type user interface. In the initial display, in each of the first to third auxiliary information 85a to 85c, the product image of the sales product most similar to the product of the search target image 50 is displayed in the forefront most in each drum roll.

  When the user performs a swipe operation on any of the first to third supplementary information 85a to 85c, the operation detection unit 253 detects the swipe operation. The display processing unit 252 performs display control so that the display position of each product image in the incidental information in which the swipe operation is detected is shifted one by one in the rotation direction of the drum roll. Therefore, the user can check the products similar to the search target image 50 in descending order of similarity by the swipe operation. When the user performs a flick operation on the product image of any of the first to third auxiliary information 85a to 85c, the operation detection unit 253 detects the flick operation, and the display processing unit 252 detects the flick operation. The displayed product image and product information are enlarged. Therefore, the user can select a desired product image by a flicking operation, and can confirm the sales product of the selected product image in detail.

  As described above, the image forming system 100 according to the present embodiment is the image forming system 100 including the image forming apparatus 1 and the glasses-type terminal 2, and the image forming apparatus 1 is a product for searching for a store. A search target image receiving unit 181 that receives an input of an image as a search target image 50, and a product information storage in which product images of a plurality of sales products sold by each of a plurality of dealers are stored in association with position information of each dealer. The product image of the sales product that matches the product of the search target image 50 received by the search target image receiving unit 181 is searched with reference to the unit 131 and the product information storage unit 131, and the product image obtained by the search is the current location The image search unit 182 that acquires a plurality of items in order from the shortest distance to each store, and the plurality of the product images acquired by the current location and the image search unit 182 A map generation unit 183 that generates a single map 81 indicating the position of each sales store associated with each of the stores, and a plurality of sales stores on the single map 81 have a high degree of similarity to the product of the search target image 50 The product images of the sales products are rearranged in order, and the identification mark 84 is arranged for each of a plurality of stores on the incidental information generation unit 184 that generates incidental information and the single map 81 generated by the map generation unit 183. An identification mark placement unit 185, a printing unit 15 that prints the single map 81 on which the identification mark 84 is placed, and a main body communication unit 17 that transmits the supplementary information generated by the supplementary information generation unit 184 to the glasses-type terminal 2. The glasses-type terminal 2 includes a terminal communication unit 23 that receives incidental information from the image forming apparatus 1, a visual field imaging unit 21 that captures a user's forward visual field, and a visual field imaging unit 21. Corresponding to the identification mark 84 recognized by the mark recognition unit 251 and the mark recognition unit 251 for recognizing the identification mark 84 arranged on the single map 81 from the captured field-of-view image 90, the incidental information is converted to a single map. And a display processing unit 252 that performs overlay display in accordance with the position of the store on 81.

  As a result, by inputting the product image of the product for which the user wants to search for a store as a search target image, a single map 81 in which neighboring stores handling the product instructed for search are arranged together with product information in order from the current location. Is printed, it is possible to compare the dealers on the single map 81 and select a purchasing store. Further, when the user visually recognizes the single map 81 through the glasses-type terminal 2, the identification mark 84 in the single map 81 is recognized by the glasses-type terminal 2, and similar products in the store corresponding to the identification mark 84 are displayed. Information is overlaid on the single map 81. As a result, it is possible to grasp the product quality of a different category (for example, if the search target product is a bag, shoes, clothes, etc.) The product can be compared on the single map 81.

  The present invention is not limited to the above-described embodiment, and it goes without saying that various changes can be made without departing from the spirit of the present invention.

DESCRIPTION OF SYMBOLS 1 Image forming apparatus 2 Glasses-type terminal 11 Operation part 12 Document reading part 13 Main body memory | storage part 14 Image processing part 15 Printing part 16 Position information acquisition part 17 Main body communication part 18 Main body control part 21 Visual field imaging part 22 Screen display part 23 Terminal communication Unit 24 terminal storage unit 25 terminal control unit 50 search target image 80-1 first store map 80-2 second store map 80-3 third store map 81 single map 82a first product image 82b second product Image 82c Third product image 83a First product information 83b Second product information 83c Third product information 84 Identification mark 84a First identification mark 84b Second identification mark 84c Third identification mark 85a First additional information 85b Second additional information 85c Third supplementary information 90 Field of view image 100 Image forming system 131 Product information storage unit 132 Map information 133 Identification mark information 181 Target image reception unit 182 the image retrieval unit 183 map generator 184 supplementary information generating unit 185 identifying mark arrangement section 186 output control unit 251 mark recognition unit 252 display unit 253 operation detection unit

Claims (5)

An image forming system including an image forming apparatus and a glasses-type terminal,
The image forming apparatus includes:
A search target image receiving unit that receives an input of a product image of a product for searching for a store as a search target image;
A product information storage unit storing product images of a plurality of sales products sold by a plurality of stores in association with position information of the respective stores;
With reference to the product information storage unit, the product image of the sales product that matches the product of the search target image received by the search target image reception unit is searched, and the product image obtained by the search is the current location An image search unit that acquires a plurality of items in order from the shortest distance to each store,
A map generation unit that generates a single map indicating the current location and the positions of the respective stores associated with the plurality of product images acquired by the image search unit;
For each of the plurality of stores on the single map, an auxiliary information generation unit that generates auxiliary information by rearranging the product images of the sales products in descending order of similarity to the products of the search target image; ,
An identification mark placement unit for placing an identification mark for each of the plurality of stores on the single map generated by the map generation unit;
A printing unit for printing the single map in which the identification mark is arranged;
A main body communication unit that transmits the auxiliary information generated by the auxiliary information generation unit to the glasses-type terminal;
The glasses-type terminal
A terminal communication unit that receives the supplementary information from the image forming apparatus;
A field-of-view imaging unit that images the user's forward field of view;
A mark recognition unit for recognizing the identification mark arranged on the single map from a field-of-view image captured by the field-of-view imaging unit;
A display processing unit that displays the supplementary information according to the position of the store on the single map in correspondence with the identification mark recognized by the mark recognition unit. Forming system. The map generation unit
For each store of each of the plurality of product images acquired by the image search unit, a store map indicating the current location and the location of the store is generated, and each store is based on the current location. Avoiding overlapping of the positions of the dealers between the maps, and generating the single map by superimposing each dealer map so that the distance from the current location to the dealer is shorter, the higher the position. The image forming system according to claim 1. The map generation unit
The image forming system according to claim 2, wherein the store map located in the upper layer is overlapped with a smaller map area. The map generation unit
If the store map in the upper layer includes the position of the store in the store map in the lower layer, the scale of the store map in the upper layer is adjusted to adjust the store map in the lower layer The image forming system according to claim 2, wherein the image forming system is enlarged to a map area of the image forming apparatus. The glasses-type terminal
It has a motion detector that detects the motion of the user's hand,
The display processing unit
5. The image formation according to claim 1, wherein display of the product image included in the incidental information is controlled in accordance with the motion of the hand detected by the motion detection unit. system.

Images