JP2018128955A - Screen shot image analyzer, screen shot image analysis method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve a problem of incapability of acquiring required information by use of a received screen shot image.SOLUTION: A screen shot image analyzer includes: an image reception section receiving a screen shot image; an analysis section analyzing the screen shot image and acquiring at least one piece of information; a search section performing information search to acquire search results by use of the at least one piece of information; and a search result transmission section transmitting the search results. The screen shot image analyzer makes it possible to acquire required information by use of the received screen shot image.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a screen shot image analysis apparatus that acquires information using a screen shot image.

  Conventionally, it is possible to use a personal computer and a web browser to access a web page on the Internet and acquire and use some data such as an image existing on the web page. This is because the web page is information combining HTML and images, and can be said to be open information.

  However, recently, the number of users who start applications on user terminals such as smartphones and browse information has increased. In general, the application is closed, and a part of the screen image displayed by the application cannot be acquired and used. Examples of such applications include Instagram (registered trademark, non-patent document 1), XZ (closet) (registered trademark, non-patent document 2), Facebook (registered trademark), and LINE (registered trademark). .

  Furthermore, recently, users are increasingly using screenshots such as smartphones to leave screenshot images for recording.

"Instagram", [Search January 21, 2017], Internet [URL: https://www.instagram.com/] "XZ", [Search January 21, 2017], Internet [URL: https://www.xz-closet.jp/]

  However, in the prior art, there has been no idea of performing an information search using a screen shot image that is an image of a screen that the user is browsing while the application is activated.

  In addition, in the prior art, information such as a partial image of a screen shot image cannot be used for information retrieval. In addition, if the entire screen shot image is analyzed and information retrieval is performed without extracting an image of an appropriate area from the screen shot image, there is a problem that the accuracy of information retrieval is reduced.

  The screen shot image analysis apparatus according to the first aspect of the invention uses an image receiving unit that receives a screen shot image, an analysis unit that analyzes the screen shot image and acquires one or more pieces of information, and one or more pieces of information. A screen shot image analysis apparatus including a search unit that performs information search and obtains a search result, and a search result transmission unit that transmits the search result.

  With this configuration, necessary information can be acquired from the screenshot image.

  The screen shot image analysis apparatus according to the second aspect of the present invention further includes an extraction unit that extracts an image of a region that satisfies a predetermined condition from the screen shot image received by the image reception unit, as compared with the first aspect of the invention. The analysis unit is a screenshot image analysis device that analyzes an image of the region cut out by the extraction unit and acquires one or more pieces of information.

  With this configuration, it is possible to cut out an image of a part of information in the screenshot image and acquire necessary information from the image.

  Further, the screen shot image analysis apparatus of the third invention is different from the second invention in that the extraction unit analyzes the screen shot image and discriminates the type of the screen shot image according to the type. A screen shot image analyzing apparatus comprising an extracting unit that cuts out an image of a region that satisfies a predetermined condition.

  With this configuration, information can be acquired from an image in an area corresponding to the type of screenshot image.

  The screen shot image analysis apparatus according to the fourth aspect of the present invention is the screen for extracting the image of the rectangular area that satisfies the predetermined condition in the screen shot image, in the second or third aspect of the invention. A shot image analysis apparatus.

  With this configuration, it is possible to cut out an image of a part of information in the screenshot image and acquire necessary information from the image.

  In the screen shot image analysis device according to the fifth aspect of the present invention, in any one of the first to fourth aspects, the search unit recognizes the screen shot image, acquires a character string, Accordingly, the screen shot image analysis device obtains different search results.

  With this configuration, necessary information can be acquired using information on a character string in the screenshot image.

  The screen shot image analysis device of the sixth aspect of the present invention is the fifth aspect of the invention, in which the analysis unit recognizes the character of the screen shot image, acquires the character string, and uses the character string as user information about the user. The search unit is a screen shot image analysis device that performs information search using user information and acquires a search result.

  With this configuration, it is possible to perform information search according to user information acquired using character string information in the screenshot image.

  The screen shot image analysis apparatus of the seventh aspect of the invention is based on the sixth aspect of the invention, wherein the analysis unit recognizes the character of the screen shot image and out of the number of posts to the user's SNS and the number of followers to the user. One or more pieces of information are acquired, the degree of influence related to the degree of influence of the user on the third party is obtained from the one or more pieces of information, and the search unit performs an information search using the degree of influence and performs a search. It is a screen shot image analysis apparatus which acquires a result.

  With this configuration, it is possible to search for information according to the degree of influence acquired using the character string information in the screenshot image.

  Further, in the screen shot image analysis apparatus according to the eighth aspect of the invention, in contrast to the sixth aspect of the invention, the analysis unit recognizes characters in the screen shot image, obtains a character string, and uses the character string to other than SNS. One or more images of the page are obtained, one or more images are analyzed, user preference information relating to user preferences is obtained, and the search unit also performs information search using the user preference information, and obtains the search results. It is the screen shot image analysis device to acquire.

  With this configuration, it is possible to search for information according to user preference information acquired using character string information in the screenshot image.

  The screen shot image analysis device according to the ninth aspect of the present invention is the first to eighth aspects of the invention, in which the image receiving unit is automatically transmitted after acquiring the screen shot image at the user terminal. A screen shot image analysis apparatus for receiving a screen shot image.

  With such a configuration, necessary information can be obtained very easily from the screen shot image.

  According to the screen shot image analysis device of the present invention, necessary information can be acquired using the screen shot image.

Conceptual diagram of information system A in the first embodiment Block diagram of the information system A A flowchart for explaining the operation of the screen shot image analysis apparatus 1 Flowchart explaining the user information acquisition process Flow chart for explaining the operation of the user terminal 2 The figure which shows the screenshot image kind management table Figure showing the product information management table The figure which shows the example of the same screen display Figure showing output example of the menu The figure which shows the example of the image cut out The figure which shows the example of the same screen display Figure showing an example of the same page The figure which shows the example of the same screen display Overview of the computer system Block diagram of the computer system

  Hereinafter, embodiments of a screen shot image analysis apparatus and the like will be described with reference to the drawings. In addition, since the component which attached | subjected the same code | symbol in embodiment performs the same operation | movement, description may be abbreviate | omitted again.

  (Embodiment 1)

  In this embodiment, an information system including a screen shot image analysis apparatus that receives a screen shot image, analyzes the screen shot image, searches for information using the analysis result, and outputs the information is described.

  In this embodiment, a screen shot image is received, an image in an area in the screen shot image that satisfies a predetermined condition is analyzed, information is searched using the analysis result, and output. An information system including the screen shot image analysis apparatus will be described.

  Further, in the present embodiment, the type of the screenshot image is determined, an image of an area corresponding to the type (appropriately referred to as “target image”) is cut out, the target image is analyzed, and the analysis result is used, An information system including a screen shot image analysis apparatus that searches and outputs information will be described.

  Further, in the present embodiment, a screenshot of a rectangular region that satisfies a predetermined condition in a screenshot image, cut out, analyzes the target image, searches for information using the analysis result, and outputs a screenshot An information system including an image analysis device will be described.

  In the present embodiment, an information system including a screen shot image analysis device that analyzes a character string in a screen shot image and searches for different information according to the character string will be described.

  Further, in the present embodiment, an information system including a screenshot image analysis device that analyzes a character string in a screenshot image, acquires user information, and can perform information search according to the user information will be described. .

  Further, in the present embodiment, an information system having a user terminal to which a screenshot image is automatically transmitted after acquisition of the screenshot image will be described.

  FIG. 1 is a conceptual diagram of an information system A in the present embodiment. The information system A includes a screen shot image analysis device 1, one or more user terminals 2. Note that the screen shot image analysis apparatus 1 and the one or more user terminals 2 can communicate with each other via a network such as the Internet. The screen shot image analysis apparatus 1 is a so-called cloud server, ASP server, or the like, but the type thereof is not limited. Moreover, although the user terminal 2 is a smart phone, a portable terminal, a tablet terminal, a personal computer etc., for example, the kind is not ask | required.

  FIG. 2 is a block diagram of the information system A in the present embodiment.

  The screen shot image analysis apparatus 1 includes a storage unit 11, a reception unit 12, a processing unit 13, and a transmission unit 14. The storage unit 11 includes a product information storage unit 111, for example. The receiving unit 12 includes an image receiving unit 121. The processing unit 13 includes an extraction unit 131, an analysis unit 132, and a search unit 133. The extraction unit 131 includes, for example, a determination unit 1311 and an extraction unit 1312. The transmission unit 14 includes a search result transmission unit 141.

  The user terminal 2 includes a terminal storage unit 21, a terminal reception unit 22, a terminal processing unit 23, a terminal transmission unit 24, a terminal reception unit 25, and a terminal output unit 26.

  The storage unit 11 constituting the screen shot image analysis apparatus 1 can store various types of information. The various information is, for example, product information described later. The various types of information are, for example, one or more image cut-out rules described later. The image cut-out rule is a rule for cutting out an analysis target image from a screen shot image. The image cut-out rule includes, for example, information that specifies the type of the screen shot image and the area from which the analysis target image is cut out. The information specifying the region is, for example, two coordinate values specifying the rectangle. The information for specifying the area is, for example, information indicating the condition of the area (for example, a maximum rectangular area including the midpoint in the screenshot image). The information for specifying the area is, for example, information indicating a rectangular area specified by using relative position information from the reference point with the upper left corner of the largest rectangular area including the midpoint in the screenshot image as a reference point. It is. The information indicating the rectangular area is, for example, two coordinate values “(x1, y1) (x2, y2)” that specify the rectangle. Note that (x1, y1) is, for example, a relative coordinate value from the reference point, and is a coordinate value of the upper left point of the rectangle. Further, (x2, y2) is, for example, a relative coordinate value from the reference point, and is a value of the lower right coordinate of the rectangle. The information indicating the rectangular area is, for example, (x1, y1) and (h, w). (x1, y1) is, for example, the value of the upper left coordinate of the rectangle relative to the reference point, h is the height of the rectangle, and w is the width of the rectangle. Further, the information specifying the area may be information specifying the screen layout, for example. The format of information specifying the screen layout is not limited. The information specifying the layout of the screen is information indicating an arrangement condition, for example, user information is arranged in the upper part of the maximum rectangular area, and the number of “Like” and comments are arranged in the lower part. The image cut-out rule includes, for example, information for identifying the type of the screen shot image and the program for cutting out the analysis target image. The image cut-out rule includes, for example, a screen shot image type and a program for cutting out an analysis target image.

  The product information storage unit 111 stores one or more product information. The product information is information about the product. The product information is an example of a search target. The merchandise information includes, for example, a merchandise name, merchandise type information, merchandise image, price, URL of a website for purchase, and the like. However, the structure of product information does not matter. When the screen shot image analysis apparatus 1 searches for information other than product information using the analysis result of the screen shot image, the product information storage unit 111 does not have to exist.

  The receiving unit 12 receives various information and instructions. The various information and instructions are, for example, screen shot images and purchase instructions. The purchase instruction is an instruction to purchase a product. The purchase instruction includes, for example, a user identifier for identifying a user, a product identifier for identifying a product to be purchased, the number of products to be purchased, and the like.

  The image receiving unit 121 receives a screen shot image. The image receiving unit 121 is selected by the user, read out from the camera roll, and transmitted, for example, when a screen shot image is acquired in the user terminal 2 and the screen shot image is stored in the camera roll. Receive screenshot image. Note that the image receiving unit 121 may receive a screenshot image automatically transmitted after acquisition of the screenshot image in the user terminal 2. Further, the image receiving unit 121 may receive a screenshot image in accordance with a user instruction for a menu automatically output to the user terminal 2 after the user terminal 2 acquires the screenshot image. Here, the screen shot image is an image of the entire screen of the user terminal 2. The screenshot image is preferably an image including an image therein. The reception here is usually reception from the user terminal 2. The image in the screen shot image is preferably a photograph taken.

  The processing unit 13 performs various processes. The various processes are processes performed by the extraction unit 131, the analysis unit 132, and the search unit 133, for example. The various processes are processes for purchasing products and the like, for example. The process for purchasing a product or the like may be any process that is related to the purchase of a product or the like. The process for purchasing a product or the like includes, for example, a process for transmitting a product purchase confirmation screen to the user terminal 2, a settlement process using a credit card number managed in combination with a user identifier included in the purchase instruction, and purchase For example, processing for transmitting information related to purchase corresponding to the instruction to a settlement server (not shown). In addition, since the process for purchasing goods etc. is a well-known technique, detailed description is abbreviate | omitted.

  The extraction unit 131 cuts out an image of a partial area that satisfies a predetermined condition from the screen shot image. The partial area that satisfies the predetermined condition is an area that satisfies the predetermined condition and is a partial area of the screenshot image. Further, the partial area that satisfies the predetermined condition is, for example, a rectangular area of the image including the center point of the screenshot image. The partial area that satisfies the predetermined condition is, for example, a rectangular area in the screen shot image, which is the largest rectangular area. Further, the partial area satisfying the predetermined condition is, for example, the largest rectangular area including the center point of the screenshot image. Further, the partial area satisfying the predetermined condition is, for example, an area excluding an area including characters in the screenshot image.

  It is preferable that the extraction unit 131 cuts out an image of a rectangular area that satisfies a predetermined condition from the screen shot image. For example, the extraction unit 131 uses the image cutout rule of the storage unit 11 to cut out an image of a rectangular area that satisfies a predetermined condition in the screen shot image.

  The extraction unit 131 preferably includes a determination unit 1311 and an extraction unit 1312. Here, the determination unit 1311 analyzes the screen shot image and determines the type of the screen shot image. The type of screen shot image is, for example, identification information (for example, Instagram, closet, Facebook, LINE, etc.) of an app from which the screen shot image is acquired.

  The screen shot image analysis algorithm does not matter. For example, the storage unit 11 stores a screen shot image template corresponding to each of one or more types, an example of the screen shot image, a feature amount of the screen shot image, and the like. Is used to determine the type corresponding to the received screenshot image. For example, the determination unit 1311 applies the received screenshot image to the learning information in the storage unit 11 and determines the type corresponding to the received screenshot image by a machine learning algorithm such as deep learning or SVM. May be. Note that the learning information in the storage unit 11 is information obtained by learning one or more feature quantities of the whole or part of the screenshot image or the whole or part of the screenshot image for each type.

  The extraction unit 1312 cuts out an image of a region that satisfies a predetermined condition according to the type acquired by the determination unit 1311. For example, the extraction unit 1312 searches the storage unit 11 for an image extraction rule corresponding to the type of the screenshot image acquired by the determination unit 1311, and analyzes it from the screenshot image using information included in the image extraction rule. Cut out the target image.

  The analysis unit 132 analyzes the screenshot image and acquires one or more pieces of information. It is preferable that the analysis unit 132 analyzes the image of the region cut out by the extraction unit 131 and acquires one or more pieces of information. Here, the information acquired by the analysis unit 132 is, for example, the type of the object in the image, the name of the object in the image, one or more feature amounts of the object in the image. The object is an object that a person wears, such as clothes, shoes, a bag, and an accessory. The object is, for example, a real estate property, a scenic spot, a scenery, a spot, a dish, a gourmet, a car, a home appliance, a toy, or the like. In other words, it goes without saying that the type of the object does not matter.

  Moreover, it is preferable that the analysis unit 132 recognizes characters in the screen shot image and acquires a character string. It is preferable that the analysis unit 132 recognizes characters with respect to an image of a part of the screen shot image and acquires a character string. It is preferable that the area from which the character string is acquired is an area different from the area of the image cut out by the extraction unit 131. Further, it is preferable that the analysis unit 132 recognizes characters in an image of an area determined according to the type of the screenshot image and acquires a character string. Note that the image of the region determined according to the type is a region corresponding to the type, for example, an image of the region specified by information specifying the region managed in pairs with the type identifier.

  It is preferable that the analysis unit 132 recognizes characters in the whole or a part of the screenshot image, acquires a character string, and acquires user information using the character string. The user information is information related to the user, for example, an influence degree described later, user preference information described later, and the like.

  The analysis unit 132 recognizes the whole or part of the screen shot image, obtains one or more pieces of information of the number of posts to the user's SNS (for example, Instagram) and the number of followers for the user, and The influence degree may be acquired from one or more pieces of information. The influence degree is information relating to the degree of influence of a user on a third party. For example, the analysis unit 132 calculates the influence degree by an increasing function using the number of posts to the user's SNS as a parameter. Moreover, the analysis part 132 calculates an influence degree with the increase function which uses the number of followers as a parameter, for example. It is preferable that the analysis unit 132 character-recognizes an image in a predetermined area according to the type of the screenshot image, and obtains the number of posts to the user's SNS or the number of followers for the user. For example, the analysis unit 132 calculates the influence degree by an arithmetic expression “degree of influence = f (number of posts, number of followers)”. The function f here is an increasing function using the number of posts and the number of followers as parameters. Moreover, the analysis part 132 may acquire the posting number rank corresponding to the acquired number of postings with reference to the table of the posting number rank corresponding to the number of postings. The analysis unit 132 may acquire a follower number rank corresponding to the acquired follower number with reference to a table of follower number ranks corresponding to the number of followers. Then, the analysis unit 132 may calculate the degree of influence using one or more pieces of information of the posting number rank and the follower number rank.

  The analysis unit 132 recognizes characters in the whole or a part of the screen shot image, acquires a character string (for example, a user account name or a user ID), and uses the character string to generate another SNS. One or more images of a page may be acquired, one or more images may be analyzed, and user preference information regarding user preferences may be acquired. Note that the one or more images are images posted by the user. The analysis unit 132 is a predetermined area corresponding to the type of the screenshot image, character-recognizes an image of a part of the screenshot image, for example, acquires a character string such as a user account name, Acquire one or more other pages of the SNS using a character string such as the account name, acquire one or more images of the one or more pages, analyze one or more images, and acquire user preference information You may do it. The user preference information includes, for example, many types of information among the types of clothes (for example, jumper, jeans, skirt, tie, etc.) shown in the acquired one or more images, and many colors among the colors of clothes. Information etc.

  The search unit 133 performs an information search using one or more pieces of information acquired by the analysis unit 132 and acquires a search result. For example, the search unit 133 searches for one or more pieces of product information from the product information storage unit 111 using one or more pieces of information acquired by the analysis unit 132. That is, the search result is, for example, one or more product information.

  In the search unit 133, the analysis unit 132 may perform an information search using an analysis result of an image of a partial region of the screenshot image, and acquire the search result. In addition, the search unit 133 performs an information search by using the analysis result of the partial image of the screenshot image and the character recognition result of the partial image of the screenshot image. The result may be obtained.

  The search unit 133 obtains different search results according to the character string acquired by the analysis unit 132. For example, the search unit 133 performs an information search using the analysis result of the image acquired by the analysis unit 132 and the character string acquired by character recognition by the analysis unit 132, and acquires the search result.

  It is preferable that the search unit 133 performs information search using user information and acquires a search result. Here, the user information is, for example, one or more pieces of information of influence degree and user preference information.

  Note that it is not essential for the search unit 133 to search for product information in the product information storage unit 111. For example, a web search may be performed, or a database (not shown) other than the product information may be searched.

  The transmission unit 14 transmits various types of information to the user terminal 2. The various information is, for example, a search result.

  The search result transmission unit 141 transmits the search result acquired by the search unit 133. Note that the information transmitted by the search result transmission unit 141 may be information obtained by adding other information to the search result acquired by the search unit 133. Further, the information transmitted by the search result transmission unit 141 may be a part of the search result acquired by the search unit 133.

  The terminal storage unit 21 constituting the user terminal 2 can store various types of information. The various information includes, for example, a user ID, a user account name, a menu, and the like. The menu is, for example, a menu for selecting processing for a screen shot image.

  The terminal reception unit 22 receives various instructions and information. The various instructions and information include a screen shot image acquisition instruction, a screen shot image transmission instruction, and the like. The input means for various instructions and information may be anything such as a touch panel, a keyboard, a mouse, or a menu screen. The terminal reception unit 22 can be realized by a device driver for input means such as a touch panel and a keyboard, control software for a menu screen, and the like.

  The terminal processing unit 23 performs various processes. For example, the various processes are processes for converting instructions and information received by the terminal receiving unit 22 into a data structure to be transmitted. The terminal processing unit 23 acquires a screenshot image in response to receiving an instruction to acquire a screenshot image. That is, the terminal processing unit 23 implements a screen shot function.

  The terminal transmission unit 24 transmits various instructions and information to the screen shot image analysis apparatus 1. The various instructions and information are, for example, screen shot images.

  The terminal receiving unit 25 receives various types of information from the screen shot image analysis device 1. The various information is, for example, a search result. The search result is, for example, one or more product information.

  The terminal output unit 26 outputs various information. The various types of information are, for example, information received by the terminal receiving unit 25, information received by the terminal receiving unit 22, and the like. Various types of information are menus, for example. Here, output refers to display on a display, projection using a projector, printing with a printer, sound output, transmission to an external device, storage in a recording medium, and output to other processing devices or other programs. It is a concept that includes delivery of processing results.

  The storage unit 11, the product information storage unit 111, and the terminal storage unit 21 are preferably non-volatile recording media, but can also be realized by volatile recording media.

  The process in which information is stored in the storage unit 11 or the like is not limited. For example, information may be stored in the storage unit 11 or the like via a recording medium, information transmitted via a communication line or the like may be stored in the storage unit 11 or the like, Alternatively, information input via the input device may be stored in the storage unit 11 or the like.

  The receiving unit 12, the image receiving unit 121, and the terminal receiving unit 25 are usually realized by wireless or wired communication means, but may be realized by means for receiving a broadcast.

  The processing unit 13, the extraction unit 131, the analysis unit 132, the search unit 133, the determination unit 1311, the extraction unit 1312, and the terminal processing unit 23 can be usually realized by an MPU, a memory, or the like. The processing procedure of the processing unit 13 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The transmission unit 14, the search result transmission unit 141, and the terminal transmission unit 24 are usually realized by wireless or wired communication means, but may be realized by broadcasting means.

  The terminal output unit 26 may be considered as including or not including an output device such as a display or a speaker. The terminal output unit 26 may be implemented by output device driver software, or output device driver software and an output device.

  Next, the operation of the information system A will be described. First, the operation of the screen shot image analysis apparatus 1 will be described using the flowchart of FIG.

  (Step S <b> 301) The image reception unit 121 determines whether a screen shot image has been received from the user terminal 2. When the screen shot image is received, the process goes to step S302. When the screen shot image is not received, the process goes to step S308.

  (Step S302) The determination unit 1311 analyzes the screenshot image and determines the type of the screenshot image.

  (Step S303) The extraction unit 1312 cuts out an image of a region that satisfies a predetermined condition in accordance with the type of the screenshot image acquired in Step S302.

  (Step S304) The analysis unit 132 analyzes the image cut out in step S303, and acquires one or more pieces of information. The one or more pieces of information are, for example, the types of articles in the image (for example, skirts, shoes, ties, etc.).

  (Step S305) The analysis unit 132 recognizes characters in part or all of the screen shot image received in step S301, acquires a character string, and acquires user information using the character string. Such user information acquisition processing will be described with reference to the flowchart of FIG.

  (Step S306) The search unit 133 performs an information search using one or more information acquired in step S304, and acquires a search result. Note that the search unit 133 preferably performs an information search using one or more pieces of information acquired in step S304 and the user information acquired in step S305, and acquires a search result.

  (Step S307) The search result transmission unit 141 transmits the search result acquired in step S306 to the user terminal 2 that has transmitted the screenshot image. The process returns to step S301.

  (Step S <b> 308) The receiving unit 12 determines whether a purchase instruction has been received from the user terminal 2. When the purchase instruction is received, the process goes to step S309, and when not received, the process goes to step S310.

  (Step S309) The processing unit 13 performs processing for purchasing a product or the like in accordance with the purchase instruction received in step S308. The process returns to step S301.

  (Step S <b> 310) The receiving unit 12 determines whether another instruction or information has been received from the user terminal 2. If any other instruction or information is received, the process goes to step S311; otherwise, the process returns to step S301.

  (Step S311) The processing unit 13 and the like perform processing according to other instructions and information. The process returns to step S301.

  In the flowchart of FIG. 3, the process ends when the power is turned off or the process is terminated.

  Next, the user information acquisition process of step S305 will be described using the flowchart of FIG.

  (Step S401) The analysis unit 132 recognizes characters in an image of a predetermined area according to the type of the screenshot image acquired in step S302, and acquires one or more character strings. Note that the analysis unit 132 may perform character recognition processing on the entire screenshot image and acquire one or more character strings.

  (Step S402) The analysis part 132 acquires the information used for acquisition of the information which comprises user information, or user information from the character recognition result in step S401. The acquired information includes, for example, the number of posts and the number of followers.

  (Step S403) The analysis unit 132 acquires the degree of influence using one or more pieces of information acquired in step S402. The analysis unit 132 uses the user account name acquired by character recognition to acquire another page different from the screen shot image page from the SNS server device (not shown), and from the other page, The number of followers may be acquired. Such acquisition of the number of posts and the number of followers may be performed by character recognition processing, or may be performed by acquiring a character string in a predetermined area of the page.

  (Step S404) The analysis part 132 acquires a user account name from the character recognition result in step S401.

  (Step S405) The analysis unit 132 uses the user account name acquired in step S404 to acquire one or more pages of the user identified by the user account name. This page is, for example, a page on which many photographic images posted by the user exist.

  (Step S406) The analysis unit 132 acquires one or more images from one or more pages acquired in step S405. The one or more images are, for example, photographic images uploaded by the user to an SNS such as Instagram.

  (Step S407) The analysis unit 132 analyzes the one or more images acquired in step S406, and acquires one or more user preference information. Return to upper process.

  Next, the operation of the user terminal 2 will be described using the flowchart of FIG.

  (Step S501) The terminal receiving unit 22 determines whether a screen shot image acquisition instruction has been received. If a screen shot image acquisition instruction is received, the process proceeds to step S502. If a screen shot image acquisition instruction is not received, the process proceeds to step S508.

  (Step S502) The terminal processing unit 23 performs a screen shot process to obtain a screen shot image. And the terminal output part 26 acquires the menu which selects the process with respect to a screenshot image from the terminal storage part 21, and outputs it.

  (Step S503) The terminal receiving unit 22 determines whether a transmission instruction has been received for the menu output in Step S502. When the transmission instruction is accepted, the process goes to step S504, and when the transmission instruction is not accepted, the process goes to step S507.

  (Step S504) The terminal transmission unit 24 transmits the screenshot image acquired in Step S502 to the screenshot image analysis apparatus 1.

  (Step S505) The terminal receiving unit 25 determines whether or not the search result is received from the screen shot image analyzing apparatus 1. If the search result is received, the process goes to step S506, and if not, the process returns to step S505.

  (Step S506) The terminal output unit 26 outputs the search result received in Step S505. The process returns to step S501.

  (Step S507) The terminal processing unit 23 and the like perform processing according to other instructions. The process returns to step S501.

  (Step S508) The terminal receiving unit 22 determines whether a purchase instruction has been received. If a purchase instruction is accepted, the process proceeds to step S509. If a purchase instruction is not accepted, the process proceeds to S511.

  (Step S509) The terminal processing unit 23 configures a purchase instruction to be transmitted using the purchase instruction received in Step S508.

  (Step S510) The terminal transmission unit 24 transmits the purchase instruction configured in Step S509 to the screen shot image analysis apparatus 1. The process returns to step S501.

  (Step S511) The terminal receiving unit 22 determines whether or not other instructions have been received. If any other instruction is accepted, the process goes to step S512. If no other instruction is accepted, the process returns to step S501.

  (Step S512) The terminal processing unit 23 performs a process according to the other instructions received in Step S511. The process returns to step S501.

  Note that in step S502 in the flowchart of FIG. 5, after the terminal processing unit 23 performs the screen shot processing and acquires the screen shot image, the terminal transmission unit 24 immediately converts the acquired screen shot image into the screen shot image. You may transmit to the analysis apparatus 1.

  Hereinafter, a specific operation of the information system A in the present embodiment will be described. A conceptual diagram of the information system A is shown in FIG.

  Now, it is assumed that the storage unit 11 stores a screen shot image type management table (hereinafter referred to as “image type management table” as appropriate) shown in FIG. The image type management table is a table for managing information for each type of screenshot image. The image type management table stores one or more records having “ID”, “type identifier”, “learning information”, “image cutting rule”, “user account name acquisition information”, “post count acquisition information”, and “follower count acquisition information”. ing. “ID” is information for identifying a record. The “type identifier” is information for identifying the type of the screenshot image. “Learning information” is information for determining the type of a screenshot image. “Learning information” is an image of an area that characterizes the type of screenshot image, but it may be one or more feature amounts acquired from the image of that area, and user input information in the screenshot image is excluded. The image may be an image, or may be one or more feature amounts acquired from an image obtained by removing user input information from a screenshot image. The “image cut rule” is information for cutting an image to be analyzed. “User account name acquisition information” is information used to acquire a user account name. “Post count acquisition information” is information used to acquire the post count. “Follower number acquisition information” is information used to acquire the number of followers. Here, “user account name acquisition information”, “post count acquisition information”, and “follower count acquisition information” are, for example, information indicating the relative position from the upper left reference point of the largest rectangular area of the screenshot image. The “image extraction rule”, “user account name acquisition information”, “post number acquisition information”, and “follower number acquisition information” may be coordinate values that specify the area of the image to be extracted, or a character string that specifies the area of the image to be extracted Etc.

  Further, the product information storage unit 111 stores at least one record having a product identifier and a product attribute information group in the product information management table shown in FIG. The product attribute information group is one or more product attribute information. Here, the product attribute information group includes “product image”, “type”, “price”, “description”, and “URL”. “Type” is the type of product. “Price” is the price of a product. The “description” is a sentence explaining the product. “URL” is information indicating a link to a product detail page of an external device (not shown).

  In this state, the following two specific examples will be described. Specific example 1 cuts out an image of a partial region that satisfies a predetermined condition from a screen shot image, analyzes the cut-out image, searches for product information using the analysis result, and transmits the product information to user terminal 2 Is the case. And in the specific example 1, a user performs the process which purchases one goods from the several goods information shown.

In Specific Example 2, an image of a partial region that satisfies a predetermined condition is cut out from the screen shot image, and the cut out image is analyzed. In Specific Example 2, character recognition is performed on an image in a partial region of the screen shot image, and one or more character strings are acquired. In the second specific example, the product information is searched using one or more character strings that are the analysis result of the clipped image and the character recognition result, and transmitted to the user terminal 2.
(Specific example 1)

  It is assumed that the user A operates his / her user terminal 2 (smart phone) and displays the screen shown in FIG. The screen in FIG. 8 is an SNS screen for Instagram.

  Then, it is assumed that the user A has input a screen shot image acquisition instruction to the user terminal 2. Then, the terminal reception unit 22 of the user terminal 2 receives a screen shot image acquisition instruction. The method for instructing acquisition of the screenshot image is not limited. For example, a screen shot image acquisition instruction is input by simultaneously pressing the home button and the power button.

  Next, the terminal processing unit 23 performs screen shot processing, acquires screen shot images, and stores them in a predetermined storage area (for example, “camera roll”). And the terminal output part 26 acquires the menu which selects the process with respect to a screenshot image from the terminal storage part 21, and outputs it. An example of the output of such a menu is 901 in FIG. The menu 901 is an example of a menu. When the item “Send” in the menu 901 is selected, a screen shot image is sent to the screen shot image analysis apparatus 1. When the item “accumulate” in the menu 901 is selected, the screen shot image is accumulated in the terminal storage unit 21. Further, when the item “return” of the menu 901 is selected, the menu 901 is not displayed.

  Next, it is assumed that the user has made an input for selecting the item “transmission” in the menu 901 to the user terminal 2. Then, the terminal reception part 22 receives a transmission instruction. Then, the terminal transmission unit 24 transmits the acquired screen shot image to the screen shot image analysis device 1. Here, the terminal processing unit 23 may accumulate the screen shot image in the terminal storage unit 21.

  Note that performing the process of transmitting a screenshot image using the menu 901 in FIG. 9 is an example of a user operation and process. It goes without saying that the user may select a screenshot image to be transmitted from a predetermined storage area (for example, “camera roll”) and input an instruction to transmit the screenshot image. In such a case, a screen shot image may be transmitted to the screen shot image analysis device 1 in accordance with the instruction.

  Next, the image receiving unit 121 of the screen shot image analyzing apparatus 1 receives a screen shot image from the user terminal 2.

  Next, the determination unit 1311 analyzes the received screenshot image (see FIG. 8) and determines the type of the screenshot image. Specifically, the determination unit 1311 applies the screen shot image to “learning information” in the screen shot image type management table, and acquires the type identifier “instagram”. The discriminating means 1311 cuts out a predetermined region (here, the region 801 of the screen shot image in FIG. 8) of the screen shot image, determines learning information most similar to the image in the region, A type identifier “instagram” paired with the learning information may be acquired. In addition, the determination unit 1311 extracts one or more feature amounts of the image in the area 801, compares it with one or more feature amounts that are learning information, determines the most similar learning information, and pairs the learning information with the learning information. The type identifier “Instagram” may be acquired.

  Next, the extraction unit 1312 acquires the image cut-out rule “maximum central rectangle” that is paired with the acquired type “instagram” of the screenshot image from the image type management table. Then, the extracting unit 1312 cuts out the largest rectangular image at the center of the received screenshot image, and obtains the image of FIG.

  Next, the analysis unit 132 analyzes the image in FIG. 10 and acquires one or more pieces of information. Here, the acquired one or more pieces of information are information “tops” and “skirt” indicating the type of the object in the image, and one or more feature amounts (for example, color, shape, etc.) of each object. And

  Next, the search unit 133 searches the product information management table shown in FIG. 7 using the acquired one or more pieces of information, and searches for one or more pieces of product information. In other words, the analysis unit 132 has a product image having an attribute value “kind” of “tops” and “skirt”, and one or more feature quantities acquired for each kind and a feature quantity having a similarity equal to or greater than a threshold value. Search the product information management table from the product information management table. In addition, the search unit 133 searches for other products (for example, shoes and bags) that match the “tops” and “skirt” in the image of FIG. Here, it is assumed that the search unit 133 searches for records such as “ID = 1064” and “ID = 3311” from the product information management table shown in FIG. And the search part 133 acquires the information transmitted to the user terminal 2 from the searched record. Needless to say, the search algorithm for the merchandise information in the search unit 133 does not matter.

  Next, the search result transmission unit 141 transmits the acquired search result to the user terminal 2 of the user A.

  Next, the terminal reception unit 25 of the user terminal 2 of the user A receives the search result. And the terminal output part 26 outputs a screen as shown in FIG. This screen is a screen for purchasing a product.

  Needless to say, the search results and the display mode of the search results are not limited to FIG. 11. The search results are, for example, a product list page, and the original image (the image in FIG. 9) is switched to display the product. A list page may be displayed.

  Next, it is assumed that the user A instructs the purchase button 1101 of the menu of FIG. 11 and inputs the purchase instruction of the pants of FIG. Then, the terminal reception part 22 receives a purchase instruction. Next, the terminal processing unit 23 configures a purchase instruction to be transmitted using the received purchase instruction. Here, the purchase instruction to be transmitted has a product identifier (for example, a product identifier “1064” in the product information management table shown in FIG. 7) that specifies the product to be purchased and a user identifier of user A. It is assumed that the product identifier “1064” is included in the product information transmitted to the user terminal 2. Next, the terminal transmission unit 24 transmits the configured purchase instruction to the screen shot image analysis apparatus 1. Next, the reception unit 12 of the screen shot image analysis apparatus 1 receives a purchase instruction from the user terminal 2.

  Next, the processing unit 13 performs processing for purchasing a product or the like according to the received purchase instruction. Here, the process for purchasing a product or the like refers to, for example, payment processing using the credit card number of the user A registered in the storage unit 11 and information indicating that the purchase procedure has been completed. For example, processing to be transmitted to

Instead of the above processing, the user may select one product from the product list page, and the reception unit 12 of the screen shot image analysis apparatus 1 may receive the product identifier of the selected product. . And the process part 13 acquires the page (page of detailed description of goods) specified by URL which becomes a pair with the said goods identifier. Next, the transmission unit 14 of the screen shot image analysis apparatus 1 transmits the page to the user terminal 2. Then, the user terminal 2 receives and outputs the page.
(Specific example 2)

  In the second specific example, it is assumed that the user A performs the same operation on the user terminal 2 as the first specific example, and the terminal reception unit 22 of the user terminal 2 receives the instruction to acquire the screenshot image.

  Next, the terminal processing unit 23 performs a screen shot process and acquires a screen shot image. Then, the terminal transmission unit 24 transmits the acquired screen shot image to the screen shot image analysis device 1. Here, the terminal processing unit 23 may accumulate the screen shot image in the terminal storage unit 21. Here, as in the first specific example, the terminal output unit 26 may acquire and output a menu for selecting processing for the screenshot image from the terminal storage unit 21. Further, a screen shot stored in the camera roll may be selected and transmitted.

  Next, the image receiving unit 121 of the screen shot image analyzing apparatus 1 receives a screen shot image from the user terminal 2.

  Next, the determination unit 1311 analyzes the received screenshot image (see FIG. 8), performs the processing described in the specific example 1, and obtains the image in FIG. Further, the analysis unit 132 analyzes the image of FIG. 10 and acquires information “tops” and “skirt” indicating the type of the object in the image and one or more feature amounts (for example, color, shape, etc.) of each object. Suppose that

The analysis unit 132 obtains the user account name “mishmash_official” from the area corresponding to the acquired type “instagram” of the screenshot image. The analysis unit 132 acquires user account name acquisition information “(x ₁₁ , y ₁₁ ) (x ₁₂ , y ₁₂ )”. Then, the analysis unit 132 detects the maximum rectangle of the screen shot image. Then, the analysis unit 132 character-recognizes the image in the area indicated by the user account name acquisition information from the reference point (for example, the upper left coordinate value) of the maximum rectangle of the screen shot image, and obtains the user account name “mishmask_official”.

  Next, the analysis unit 132 acquires one or more pages of the user identified by the user account name from the SNS server (not shown) of Instagram using the acquired user account name “misch_mask”. Here, it is assumed that the analysis unit 132 has acquired the page shown in FIG.

  Next, the analysis unit 132 acquires one or more images (see 1201) from the acquired page of FIG. Then, the analysis unit 132 analyzes the acquired one or more images and acquires one or more user preference information. Here, the user preference information includes, for example, information “coat”, “one piece”, “skirt” indicating the type of the object, and includes one or more feature amounts (for example, color, shape, etc.) of each object. .

  Further, the analysis unit 132 acquires the number of posts “457” from the area 1201 corresponding to the type “instagram”.

Further, the analysis unit 132 acquires the number of followers “32,000” from the area 1201 corresponding to the type “instagram”. The analysis unit 132 receives the post number acquisition information “(x ₁₃ , y ₁₃ ) (x ₁₄ , y ₁₄ )” and the follower number acquisition information “(x ₁₅ , y ₁₅ ) (x ₁₆ , y ₁₆ )”, Obtained from the image type management table of FIG.

Next, the analysis unit 132 sends the number of posts “457” from the area 1202 specified by the reference point and the post number acquisition information “(x ₁₃ , y ₁₃ ) (x ₁₄ , y ₁₄ )” on the screen of FIG. Is obtained. Further, the analysis unit 132 determines the number of followers “3. from the region 1203 specified by the reference point and the follower number acquisition information“ (x ₁₅ , y ₁₅ ) (x ₁₆ , y ₁₆ ) ”of the screen of FIG. Get 20,000 ". Next, the analysis unit 132 acquires the degree of influence (for example, “8”) using the acquired number of posts “457” and the number of followers “32,000”. Here, it is assumed that the influence degree is, for example, 10-level evaluation from 1 to 10. For example, the higher the degree of influence, the easier it is to search for product information of a product that has at least a rare value and that raises the reputation of the store. On the other hand, for example, the lower the degree of influence, the easier it is to search for product information of products with higher profits.

  Next, the search unit 133 acquires one or more pieces of information (information “tops” and “skirt”) and one or more feature amounts (for example, color, shape, etc.) of each object acquired by the image analysis of FIG. The user information is retrieved using the user information, and the search result is obtained.The user information includes preference information (including information “coat”, “one piece”, “skirt”, and one or more feature quantities of each object ( For example, color, shape, etc.) and influence degree “8.” Note that the search algorithm of the search unit 133 does not matter.

  Next, the search result transmission unit 141 transmits the acquired search result to the user terminal 2 of the user A.

  Next, the terminal reception unit 25 of the user terminal 2 of the user A receives the search result. And the terminal output part 26 outputs a screen as shown in FIG. This screen is a screen for purchasing a product.

  Next, it is assumed that the user A instructs the purchase button 1301 of the menu in FIG. 13 and inputs the purchase instruction for the one-piece in FIG. Next, the same process as the process described in the specific example 1 is performed, and the user A can purchase the one piece.

  As described above, according to the present embodiment, it is possible to acquire necessary information using a screenshot image only by transmitting the screenshot image.

  Further, according to the present embodiment, it is possible to cut out an image of a part of information in a screenshot image and acquire necessary information from the image only by transmitting the screenshot image.

  Specifically, according to the present embodiment, a part of the image is cut out from the screen shot image of the screen of the user terminal 2 in a state where the application is activated, and the part of the image is analyzed. By using it, you can solve the problems of closed apps.

  More specifically, according to the present embodiment, a part of the image is cut out from the screen shot image of the screen of the user terminal 2 in a state where the application is activated, analyzed, and the product can be purchased. This makes it possible to use information in a closed application. More specifically, it is possible to perform information search and purchase of products that effectively use images displayed by a closed application that can browse images posted to the SNS.

  Further, according to the present embodiment, necessary information can be acquired using information on a character string in a screenshot image.

  Further, according to the present embodiment, it is possible to perform information search according to user information acquired using character string information in a screenshot image.

  Furthermore, according to the present embodiment, a screen shot image is automatically transmitted in response to the screen image being acquired by the user terminal 2, or a menu for automatically transmitting a screen shot image is provided. By displaying, necessary information can be very easily acquired from the screenshot image. Note that it is preferable that a user interface for assisting transmission of the screen shot image is displayed in response to the screen shot image being acquired by the user terminal 2. The support user interface for transmission includes, for example, the above-described menu, a panel for confirming transmission, a message for prompting transmission, and the like. Needless to say, as described above, a screenshot image stored in a camera roll or the like may be transmitted.

  Note that the processing in the present embodiment may be realized by software. Then, this software may be distributed by software download or the like. Further, this software may be recorded and distributed on a recording medium such as a CD-ROM. This also applies to other embodiments in this specification. The software that realizes the screen shot image analysis apparatus 1 according to the present embodiment is a program as described below. In other words, this program uses a computer to search for information using an image receiving unit that receives a screenshot image, an analysis unit that analyzes the screenshot image and obtains one or more pieces of information, and the one or more pieces of information. Is a program for functioning as a search unit for acquiring a search result and a search result transmission unit for transmitting the search result.

  In the above program, the computer is further caused to function as an extraction unit that extracts an image of a region that satisfies a predetermined condition from the screenshot image received by the image reception unit, and the analysis unit is extracted by the extraction unit. It is preferable that the program is for causing a computer to function as the one that analyzes the image of the region and acquires one or more pieces of information.

  In the above program, the extraction unit analyzes the screenshot image, and discriminates the type of the screenshot image, and extracts an image of an area that satisfies a predetermined condition according to the type. It is preferable that the computer program is a program for causing a computer to function as an extractor.

  In the above program, it is preferable that the extraction unit is a program for causing a computer to function as cutting out an image of a rectangular area that satisfies a predetermined condition in the screenshot image.

  In the above program, the analysis unit recognizes characters in the screenshot image and acquires a character string, and the search unit obtains different search results according to the character string acquired by the analysis unit. It is preferable that the program is for causing a computer to function.

  In the program, the analysis unit recognizes characters in the screenshot image, acquires a character string, acquires user information about the user using the character string, and the search unit includes the user information. Use to search information and get search results

  FIG. 14 shows the external appearance of a computer that executes the program described in this specification and realizes the screen shot image analysis apparatus 1 and the like of the various embodiments described above. The above-described embodiments can be realized by computer hardware and a computer program executed thereon. FIG. 14 is an overview diagram of the computer system 300, and FIG. 15 is a block diagram of the system 300.

  In FIG. 14, the computer system 300 includes a computer 301 including a CD-ROM drive 3012, a keyboard 302, a mouse 303, and a monitor 304.

  In FIG. 15, a computer 301 includes a CD-ROM drive 3012, an MPU 3013, a bus 3014, a ROM 3015, a RAM 3016, and a hard disk 3017. The ROM 3015 stores programs such as a bootup program. The RAM 3016 is connected to the MPU 3013 and temporarily stores application program instructions and provides a temporary storage space. The hard disk 3017 normally stores application programs, system programs, and data. Although not shown here, the computer 301 may further include a network card that provides connection to a LAN.

  A program that causes the computer system 300 to execute the functions of the screen shot image analysis apparatus 1 and the like of the above-described embodiment is stored in the CD-ROM 3101, inserted into the CD-ROM drive 3012, and further transferred to the hard disk 3017. Also good. Further, the program may be transmitted to the computer 301 via a network (not shown) and stored in the hard disk 3017. The program is loaded into the RAM 3016 at the time of execution. The program may be loaded directly from the CD-ROM 3101 or the network.

  The program does not necessarily include an operating system (OS), a third-party program, or the like that causes the computer 301 to execute the functions of the screen shot image analysis apparatus 1 according to the above-described embodiment. The program need only include the part of the instruction that calls the appropriate module in a controlled manner and achieves the desired result. How the computer system 300 operates is well known and will not be described in detail.

  In the above program, in the step of transmitting information, the step of receiving information, etc., processing performed by hardware, for example, processing performed by a modem or an interface card in the transmission step (only performed by hardware) Processing) is not included.

  Further, the computer that executes the program may be singular or plural. That is, centralized processing may be performed, or distributed processing may be performed.

  Further, in each of the above embodiments, it goes without saying that two or more communication units existing in one apparatus may be physically realized by one medium.

  In each of the above embodiments, each process may be realized by centralized processing by a single device, or may be realized by distributed processing by a plurality of devices.

  As described above, the screen shot image analysis apparatus according to the present invention cuts out a part of information information in the screen shot image and acquires necessary information from the screen image only by transmitting the screen shot image. This is useful as a screen shot image analysis device.

DESCRIPTION OF SYMBOLS 1 Screen shot image analyzer 2 User terminal 11 Storage part 12 Reception part 13 Processing part 14 Transmission part 21 Terminal storage part 22 Terminal reception part 23 Terminal processing part 24 Terminal transmission part 25 Terminal reception part 26 Terminal output part 111 Product information storage part 121 Image Receiving Unit 131 Extracting Unit 132 Analyzing Unit 133 Searching Unit 141 Search Result Transmitting Unit 1311 Discriminating Unit 1312 Extracting Unit

Claims (12)

An image receiver for receiving screenshot images;
Analyzing the screenshot image and obtaining one or more pieces of information;
A search unit that performs an information search using the one or more pieces of information and obtains a search result;
A screen shot image analysis device comprising: a search result transmission unit for transmitting the search result. The image receiving unit further includes an extraction unit that cuts out an image of a region that satisfies a predetermined condition from the screen shot image received,
The analysis unit
The screenshot image analysis apparatus according to claim 1, wherein the image of the region cut out by the extraction unit is analyzed to acquire one or more pieces of information. The extraction unit includes:
A discriminating means for analyzing the screenshot image and discriminating the type of the screenshot image;
The screen shot image analysis apparatus according to claim 2, further comprising an extraction unit that cuts out an image of a region that satisfies a predetermined condition according to the type. The extraction unit includes:
4. The screen shot image analysis apparatus according to claim 2, wherein an image of a rectangular area that satisfies a predetermined condition in the screen shot image is cut out. The analysis unit
Character recognition of the screenshot image, obtaining a character string,
The search unit
The screenshot image analysis device according to claim 1, wherein different search results are obtained according to the character string acquired by the analysis unit. The analysis unit
Character recognition of the screenshot image, obtaining a character string, using the character string to obtain user information about the user,
The search unit
The screen shot image analysis apparatus according to claim 5, wherein the user information is also used to perform an information search and obtain a search result. The analysis unit
Character recognition of the screen shot image, acquisition of one or more information of the number of posts to the user's SNS and the number of followers for the user, and the degree of influence of the user on the third party from the one or more information Get the degree of influence on
The search unit
The screen shot image analysis apparatus according to claim 6, wherein information search is performed also using the degree of influence to obtain a search result. The analysis unit
Recognizing the screenshot image, obtaining a character string, using the character string, obtaining one or more images of other pages of the SNS, analyzing the one or more images, Get preference information
The search unit
The screen shot image analysis device according to claim 6, wherein the user preference information is also used to perform an information search and obtain a search result. The image receiving unit
The screen shot image analysis apparatus according to any one of claims 1 to 8, wherein the user terminal receives the screen shot image automatically transmitted after the screen shot image is acquired. A terminal reception unit for receiving an instruction to acquire a screenshot image;
In response to receiving an instruction to acquire the screenshot image, a terminal processing unit that acquires a screenshot image;
A terminal transmission unit for transmitting the screenshot image to a screenshot image analysis device;
A user terminal comprising: a terminal reception unit that receives a search result acquired using the screen shot image from the screen shot image analysis device in response to transmission of the screen shot image. A screen shot image analysis method realized by an image reception unit, an analysis unit, a search unit, and a search result transmission unit,
An image receiving step in which the image receiving unit receives a screenshot image; and
An analysis step in which the analysis unit analyzes the screenshot image and acquires one or more pieces of information;
A search step in which the search unit performs an information search using the one or more pieces of information and obtains a search result;
A method of analyzing a screenshot image, comprising: a search result transmission step in which the search result transmission unit transmits the search result. Computer
An image receiver for receiving screenshot images;
Analyzing the screenshot image and obtaining one or more pieces of information;
A search unit that performs an information search using the one or more pieces of information and obtains a search result;
The program for functioning as a search result transmission part which transmits the said search result.