JP2013149194A - Pair image posting/retrieval device and pair image posting/retrieval program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a pair image posting/retrieval device and a pair image posting/retrieval program that can eliminate difficulty in keyword retrieval.SOLUTION: A pair image posting/retrieval device comprises: posting processing means for storing image data and meta-information for a single-body image about knowledge of a problem input for posting, storing image data and meta-information for a single-body image about knowledge of a solution input for the posting, or storing image data and meta-information for a single-body image about the knowledge of the problem and of the solution that constitute a pair image to be input for the posting and to be retrieved after the posting, in an image database; and retrieval processing means for retrieving a retrieval output image from the image data and meta-information that are recorded in the image database, on the basis of a key image input for the retrieval, the single-body image about the knowledge of the problem, the single-body image about the knowledge of the solution or an instructed retrieval type for retrieving the pair image about the knowledge of the problem and of the solution.

Description

  The present invention relates to a pair image posting / retrieval apparatus and a pair image posting / retrieval program for posting, searching, and viewing in units of two types of images relating to problems and knowledge of solutions.

  With the recent increase in capacity of storage media and the spread of still image and moving image search services on the Internet, users can obtain a large amount of images. Representative image search services include Google's image search site (http://www.google.com/), which is a still image search / viewing service, and YouTube video, which is also a video search / viewing service. The search site (http://www.youtube.com/) is well known. These image search viewing services employ a search method based on keyword search.

  As an example of a service related to an image search method by an image called key image search, there is a music download site (http://morawin.jp/) of Morawin. This music distribution site provides a function to search and view a music jacket image and to search and display related images from the jacket image to the jacket image.

  Further, as a service for searching and viewing information related to problem and solution knowledge, there is a service that uses a question document called a question sentence and associates answer documents related to this to provide an added value of answer answer knowledge for the problem. As a representative example, there is a Q & A service such as OK WEB (http://okwave.jp/). In this question site, an answer contributor posts an answer to each question to determine a combination of a problem and a solution.

  As a search service that uses media other than the keyword as a key, there is an integrated search / view service, that is, a search / view service in which a search key can be combined with that of various sources for a cross-search. A representative example of this integrated search viewing service is a neighbor integrated search site (http://www.naver.jp/). According to this neighbor integrated search site, images, webs, blogs, reviews, people, movies, summaries of topics, etc. can be searched at once. This function can be understood by a plurality of media searches in which the keywords are appropriate or appropriate for searching and viewing the images that are more necessary.

  As another image search method, Patent Literature 1 discloses an image search method and system for evaluating the similarity between a query image and a search target image in consideration of the similarity of local feature amounts of each feature point. This method uses not the similarity based on the text information represented by the meta tag given by the user of the image poster device and the pair image posting search program, but the image quality, that is, the similarity of the data on the digital display signal. ing.

  As a web page search system, Non-Patent Document 1 includes “natural language knowledge” that relies on formal knowledge based on conventional text, “moving image knowledge” in which “moving image knowledge” is added to “metadata”, and “metadata”. It has been proposed to search using archive information based on the three-term group.

  According to this non-patent document 1, tacit knowledge included in a moving image or still image is more intuitive as a means of planting in human memory than formal knowledge by text, and is a clue to understand the state of yourself and others. It is said that the utility value is high. Here, first, search for and refer to multiple combinations of text information in a web page and video and still image images in the same page that it is likely to mention by general text search. It proposes a method for searching for a set of more desired text information and images by re-searching text information “question text” regarding the designation of the image that seems to be intuitive and the information to be searched for as a search key. In other words, a search method based on natural language knowledge and formal knowledge called text search is added to a search method based on moving image knowledge and tacit knowledge based on image designation, and a query text is entered to further narrow down to more desirable information. Is going to be realized.

JP 2011-008507 A

Susumu Sawai, “Ternary Knowledge Representation (“ Video Knowledge ”,“ Natural Language Knowledge ”,“ Metadata ”) and its Application to Knowledge Creation Support Activity (Doctoral Dissertation)”, [online], December 2006, JAIST Graduate School of Knowledge Science, [Search August 20, 2011], Internet <URL: https://dspace.jaist.ac.jp/dspace/handle/10119/3455>

However, any of the conventional image retrieval methods described above has the following problems.
(1) Difficulty in keyword search due to image search viewer's lack of knowledge about problems and / or solutions desired to be searched,
(2) Problems due to lack of search viewing function and interface by narrowing down the problem and solution knowledge,
(3) Difficulty of mutual search due to the existence of different knowledge that complements problem and knowledge of problem and solution,
(4) Inefficiency due to an increase in problem-solving knowledge due to indiscriminate posting and search viewing process of problems and solutions.

  First, the problem (1) will be described. In the current posting search and viewing service related to images such as single still images and single videos, image contributors post images without distinction between problems and solutions, and image search viewers distinguish between problems and solutions. Search without watching. In this situation, when the image retrieval viewer wants to retrieve images of problem and solution knowledge together, a situation where there is a sufficient single image including the problem and solution knowledge is desired. If the situation cannot be expected, a situation where there is a sufficient image including knowledge of either the problem to be searched or the solution is desired. Even in such a case, it is necessary to search based on the character information which is a keyword as a feature or summary of the knowledge of either problem or solution to be searched. However, searching usually starts with the motivation of lack of knowledge. In this case, it is generally difficult to conceive character information that is a keyword for searching for an image. Therefore, in a situation where there is a lack of knowledge to sufficiently conceive character information used as keywords, it is necessary to repeatedly input many keywords and perform a search.

  Next, the problem (2) will be described. In the keyword search in the current search and viewing service regarding an image such as a single still image or a single moving image, there is no function and interface for narrowing down search and viewing regarding knowledge of problems and solutions. Therefore, unless the image contains a meta tag that indicates the type of problem and solution, or similar knowledge, and is not recognized by the image search viewer, the problem and solution will be narrowed down and searched for. It is difficult to watch.

  Next, the problem (3) will be described. In a situation where a single image relating to the knowledge of each problem and solution exists sufficiently, a search method using the image of the problem and solution as a unit for posting and searching and viewing, and a function and interface for narrowing search and viewing regarding each knowledge of the problem and solution However, since images related to problem and solution knowledge have different pieces of knowledge information that complement each other as problem solution knowledge, meta tags may not be shared, or common keywords may not be found. In such a case, an index related to the similarity of the meta tag between the key side and the search output side, and search as a single image on the search output side, which is adopted by the current search and viewing service for images such as single still images and single videos. In the search output priority criteria mainly composed of the index related to the frequency, the search between images having different knowledge information such as the image search of the solution from the problem image and the image search of the problem from the solution image is ensured. The possibility is small.

  Finally, the problem (4) will be described. In the current posting search and viewing service for images such as single still images and single videos by keyword search, image contributors post images without distinction between problems and solutions, and image search viewers are responsible for image and problem resolution In order to search and view without distinction, according to seeds for providing knowledge of problems or solutions by image contributors, needs for using knowledge of problems or solutions by search viewers, or seeds and needs of problem solving knowledge combining these two Thus, there is little possibility that the information related to the problem and the solution knowledge is efficiently increased as an image.

  Accordingly, an object of the present invention is to provide a keyword search to assist a search for an image search viewer who is biased toward one knowledge of a problem or solution from the viewpoint of acquiring knowledge of both the problem and the solution. An object of the present invention is to provide a pair image posting search device and a pair image posting search program that can eliminate the difficulty of searching.

  Another object of the present invention is to provide a pair image posting search apparatus and a pair image posting search program capable of solving the problems caused by the lack of a search viewing / viewing function and an interface related to the problem and solution knowledge.

  Still another object of the present invention is to provide a pair image posting search apparatus and a pair image posting searching program capable of solving the difficulty of mutual search caused by the existence of different knowledge that complements problem solving knowledge as problem solving knowledge. Is to provide.

  Still another object of the present invention is to provide a pair image posting search apparatus and a pair image posting search program that can eliminate the inefficiency of an increase in problem solving knowledge caused by posting without searching for problems and solutions and search viewing processes. is there.

  According to the present invention, the pair image posting search device includes a processing control device and a storage device connected to the processing control device. The storage device includes an image database that records at least image data and meta information. The processing control device includes the image data and meta information of the single image related to the knowledge of the problem input for posting, the image data and meta information of the single image regarding the knowledge of the solution input for posting, or , A post processing means for recording image data and meta information of a single image related to a problem and solution knowledge constituting a pair image input for posting and searched after posting, and input for searching Image data recorded in the image database on the basis of the key image obtained and the instruction search type for searching for a single image related to problem knowledge, a single image related to solution knowledge, or a pair image related to problem and solution knowledge, and Search processing means for searching for a search output image from the meta information.

  Based on the keyword input for the search and the instruction search type for searching the pair image related to the problem and solution knowledge, the search processing means searches the output image from the image data and meta information recorded in the image database. It is preferable to include means for performing a search.

  Image contributors can post images by specifying the type of problem and / or solution, and image search viewers can search and view images by specifying the type of problem and / or solution. For viewing, there are keyword search and key image search. In keyword search, in addition to keyword input, a problem image, a solution image, and a problem / solution pair image can be specified and viewed. . That is, according to the present invention, in keyword search, a keyword based on either problem or solution knowledge (metaphor) is input and a pair image search is performed, or an image that is not the knowledge used for the search is searched. In this case, it is possible to eliminate the difficulty of keyword search due to lack of knowledge related to knowledge desired to be searched. Also, in key image search, when searching for a solution image from a problem, a problem image from a solution, or a pair image by one image of a problem or a solution, a key image search caused by a lack of knowledge about the knowledge desired to be searched Difficulty can be eliminated.

  In the key image search, the key side image is posted to the problem image, the solution image, the problem and the solution image, and the image on the search output side is the problem image, the solution image, the problem and the solution image. Therefore, it is easy to reliably search and view each image of the problem and the solution without thinking about the character information that is characteristic of the knowledge of the problem and the solution.

  Further, the image search viewer searches and views the image by specifying the type of problem and / or solution. Seeds for providing knowledge of problems or solutions by image contributors, needs for using knowledge of problems or solutions by search viewers, knowledge of problems and solutions according to the seeds and needs of knowledge and problems of solutions combining these two There is a greater possibility that the information on the information efficiently increases as an image. Now, there is a situation where there is only a single image in which knowledge of problems and solutions is included or one of them is biased, and all images are defined as images related to either problem or solution, and combinations Assuming that the total amount of problem and solution knowledge is the same, there is a possibility that the latter situation will form more problem-solving knowledge. is there. For this reason, for example, in each of the above two situations, there are two problem solving knowledge A composed of problem knowledge A and solution knowledge A, problem knowledge B composed of problem knowledge B, and solution knowledge B. When knowledge A and solution knowledge B, problem knowledge B and solution knowledge A are in a pair relationship that complements each other as problem solution knowledge, the number of problem solution knowledge of the former is limited to two of A and B, but the latter In the situation, in addition to these two, problem knowledge A and solution knowledge B, problem knowledge B and problem knowledge B and solution knowledge A are newly added to form a total of four, all knowledge once It will be reused one by one. Therefore, there is a high possibility that the images related to the necessary problem and solution knowledge are increased based on the seeds for providing the knowledge of the problem or solution, and the use needs of the problem or solution knowledge by the search viewer. As a result, it is possible to eliminate the inefficiency of an increase in problem solving knowledge due to posting and search viewing processes that do not distinguish between problems and solutions.

  The storage device further includes an evaluation information database that records at least evaluation information in the posting and search processing, and the posting processing means for posting the meta information of the single image related to the knowledge of the problem input for posting. Decomposing elemental meta-information of a single image related to knowledge of solution input in the above, or meta-information of a single image related to problem and solution knowledge constituting a pair image input for posting and searched after posting, Meta tagging means for recording the obtained meta tag in the image database, the search processing means, the keyword or key image input for the search, the image data and the meta tag recorded in the image database, Search output image candidate determination for determining search output image candidates based on evaluation information recorded in the evaluation information database Means, keyword or key image input for search, instruction search type input for search, image data and meta tag recorded in image database, and evaluation recorded in evaluation information database Search output image determining means for determining a search output image from the determined search output image candidate based on the information is preferably provided.

  In this case, the search output image candidate determination means includes means for reading out meta tag similarity data, which is evaluation information recorded in the evaluation information database, and searching for an image with a high meta tag similarity as a search output image candidate. Is more preferable.

  Further, it is more preferable that the search output image determination means includes a means for determining a search output image according to the search priority order first and second order image types of the instruction search type for the searched search output image candidates.

  Further, the search output image determination means includes means for calculating the meta tag similarity and search frequency for the searched search output image candidates, and means for determining the search output image according to the calculated meta tag similarity and search frequency. More preferably.

  The search processing means performs a single search output image search (KW1) related to the knowledge of the solution based on the keyword related to the knowledge of the problem input for the search, and a single unit related to the knowledge of the problem based on the keyword related to the knowledge of the solution It is also preferable to include means for performing a search output image search (KW2) or performing a pair search output image search (KW3) relating to problem and solution knowledge based on a pair keyword relating to problem and solution knowledge.

  The search processing means performs a single search output image search (KG1) related to the problem knowledge based on the key image related to the solution knowledge input for the search, and the key image related to the problem knowledge input for the search A single search output image search (KG2) related to solution knowledge is performed based on the search, and a single search output image search (KG3) related to problem knowledge is performed based on the key image related to the problem knowledge input for the search. A single search output image search (KG4) related to solution knowledge is performed based on a key image related to solution knowledge input for the search, and a pair key image related to the problem and solution knowledge input for the search Based on the search result image search (KG5) related to the problem or solution knowledge, the key of the pair related to the problem and solution knowledge input for the search Search for a pair related to the problem or solution knowledge based on the image Output image search (KG6), or a pair related to the problem and solution knowledge based on the key image of the pair related to the problem and solution knowledge input for the search It is also preferable to include means for performing a search output image search (KG7).

  As described above, the heterogeneous image search in the key image search, that is, the image search from the problem to the solution and the image search from the solution to the problem are enabled. In this image search, in addition to the metatag similarity between different images, the use frequency of the combination as a key image in the pair image search between different images is also used as an index. Also in the search, an appropriate heterogeneous image can be searched even if the meta tag is not shared by adjusting the index weight of the meta tag similarity. In addition, the same image and heterogeneous image as a pair that combines and share meta tags can be searched simultaneously by pair image search. Therefore, even if the meta tag of the key image does not share the meta tag with the dissimilar image that you want to search, By sharing with the meta tag, different images can be retrieved through the mediation. Further, different types of images can be searched by repeating the same type of image search and repeating the different types of image search using the searched same type image as a key.

  In this way, images related to problem and solution knowledge have different knowledge information that complements each other as problem-solving knowledge, so even if there is a relationship that does not share meta tags or cannot find common keywords, heterogeneous images Search for heterogeneous images using the frequency of use of combinations as key images in pair image search between images, search for heterogeneous images mediated by meta tags of similar images in pair image search combining meta tags of the same type and different types of images, and search for same type images The heterogeneous image search is performed by repeating the heterogeneous image search using the same type of image searched by repetition as a key, and the search can be performed as a bridge between knowledge and problems and solutions. As a result, the difficulty of mutual search due to the different knowledge complementing the problem and solution knowledge as problem-solving knowledge is eliminated.

  According to the present invention, there is further provided a pair image posting search program for causing a computer to function as the above-described pair image posting searching device.

  ADVANTAGE OF THE INVENTION According to this invention, the difficulty of the keyword search resulting from the lack of knowledge regarding the knowledge that the image search viewer desires to search can be solved, and the problem caused by the lack of the search viewing function and interface by narrowing down the problem and knowledge of the solution. Can solve the difficulty of mutual search due to the existence of different knowledge that complements the problem and knowledge of the problem and solution. The inefficiency of the increase in solution knowledge can be eliminated.

It is a block diagram which shows roughly the structure of the hardware of one Embodiment of the pair image contribution search apparatus of this invention. It is a functional block diagram which shows roughly the structure of the central processing control apparatus and memory | storage device in embodiment of FIG. It is a figure explaining the example (it fastens with a tool) of a pair image. It is a flowchart of an example of posting operation in the embodiment of FIG. It is a flowchart of an example of search operation in embodiment of FIG. It is a flowchart of the other example of search operation in embodiment of FIG. It is a figure which shows the example of the image posting screen in embodiment of FIG. It is a figure which shows the example of the image search screen in embodiment of FIG. It is a figure which shows the condition which installs the search output image searched by the keyword search in the image search screen of embodiment of FIG. 1 as a key image in a key image tray. It is a figure which shows the example of the image search standby screen in embodiment of FIG. It is a figure which shows the condition which installs and duplicates the link of the image on the image search standby tray in the image search standby screen in embodiment of FIG. 1 on the key image tray in an image search screen as a key image. It is a figure which shows the screen state which displayed the search result in the image search screen of embodiment of FIG. It is a figure which shows the kind of search in embodiment of FIG. It is a figure which shows the three types of content of the keyword search in embodiment of FIG. It is a figure which shows the seven types of content of the key image search in embodiment of FIG. A1. Of keyword search in the embodiment of FIG. Single image search KW1. It is a figure which shows the screen state in a problem image search. A1. Of keyword search in the embodiment of FIG. Single image search KW2. It is a figure which shows the screen state in a solution image search. A2. Keyword search in the embodiment of FIG. Pair (problem / solution) image search KW3. It is a figure which shows the screen state in a pair (problem and solution) image search. B1. Of key image search in the embodiment of FIG. It is a figure which shows the screen state of KG1 solution image search (K (problem)-> T (solution)) of the heterogeneous search of a single image search. B1. Of key image search in the embodiment of FIG. It is a figure which shows the screen state of KG3 problem image search (K (problem)-> T (problem)) of the same kind search of a single image search. B2. Key image search in the embodiment of FIG. It is a figure which shows the screen state of KG5 pair image search (K (problem)-> T (problem and solution)) of single key image search of pair image search. B2. Key image search in the embodiment of FIG. It is a figure which shows the screen state of KG7 pair image search (K (problem / solution) → T (problem / solution)) of pair key image search of pair image search. It is a figure which shows the life cycle of the posting / retrieval process of the image in embodiment of FIG. It is a flowchart of the posting process in embodiment of FIG. It is a flowchart of the search process in embodiment of FIG. It is a flowchart of the process regarding acquisition and calculation of the target data of search output image determination in the embodiment of FIG. It is a figure which shows the list of the search priority order of all the 10 types of searches in embodiment of FIG. It is a figure which shows the example of a setting of three types of search priority orders of the keyword search in embodiment of FIG. It is a figure which shows the example of a setting of seven types of search priority orders of the key image search in embodiment of FIG. It is a flowchart of the search output image candidate determination process in embodiment of FIG. It is a figure which shows the structural example of the process performance database in embodiment of FIG. It is a flowchart of the meta tag similarity calculation process in embodiment of FIG. It is a flowchart of the search frequency calculation process in embodiment of FIG. It is a flowchart of the search output image determination process in embodiment of FIG. It is a detailed flowchart of the meta tag similarity calculation process in embodiment of FIG. It is a figure which shows the object data and structure of a meta tag similarity calculation process in embodiment of FIG. It is a flowchart of the time adjustment process in embodiment of FIG. It is a conceptual diagram of the time adjustment parameter in the embodiment of FIG. It is a flowchart of the search output priority calculation process in embodiment of FIG. It is a figure showing the search specification and target image in the search output priority calculation example in embodiment of FIG. It is a figure showing the example of search output priority calculation in embodiment of FIG. It is a block diagram which shows roughly the structure of the hardware of other embodiment of the pair image contribution search apparatus of this invention. It is a block diagram which shows roughly the structure of the hardware of further another embodiment of the pair image contribution search apparatus of this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals.

[Hardware configuration of pair image submission search device]
FIG. 1 schematically shows a hardware configuration of an embodiment of a pair image posting search device of the present invention, and FIG. 2 schematically shows a configuration of a central processing control device and a storage device shown in FIG. Yes. This embodiment is a case of a stand-alone type pair image posting search device mainly composed of a central processing control device and a storage device shown in FIG.

  As shown in FIG. 1, the pair image posting search device of this embodiment includes a central processing control device 101, a program memory 102 that is a read-only memory, and a read and write memory that are connected to each other via a bus 110. A cache memory 103 and an input / output interface 109. Furthermore, an input device 104, a screen display device 105, a communication control device 106, a storage device 107, and an image capturing peripheral device 108 connected to the input / output interface 109 are provided.

  The central processing control device 101 reads out and executes a boot program for starting the pair image posting search device from the program memory 102 based on an input signal from the input device 104, and further, an operating system stored in the storage device 107. Is read out.

  The central processing control device 101 controls various devices based on input signals from the input device 104 and the communication control device 106, and reads out data stored in the cache memory 103 and the storage device 107 and caches them again. In addition to loading into the memory 103, based on a program command read from the cache memory 103, a series of processes to be described later for calculating or processing data is realized. The configuration of the central processing control apparatus 101 will be described later.

  The input device 104 includes an input device such as a keyboard and a mouse through which a user who is an image contributor or an image search viewer inputs various operations, and generates an input signal based on the user's operation. The generated input signal is transmitted to the central processing control apparatus 101 via the input / output interface 109 and the bus 110.

  The screen display device 105 is a CRT (Cathode Ray Tube) display, a liquid crystal display, etc., and receives an output signal to be displayed on the display device 105 from the central processing control device 101 via the bus 110 and the input / output interface 109. The processing result of the central processing control device 101 is displayed.

  The communication control device 106 is a device such as a LAN (Local Area Network) card or a modem, and is configured to connect the pair image posting search device to a communication network such as the Internet or a LAN. Data transmitted / received to / from the communication network via the communication control device 106 is transmitted / received to / from the central processing control device 101 via the input / output interface 109 and the bus 110 as an input signal or an output signal.

  The storage device 107 is a semiconductor storage device or a magnetic disk device, and stores programs and data executed by the central processing control device 101. The configuration of the storage device 107 will be described later.

  The image capture peripheral device 108 is a device that captures image data from the outside of the device, such as an optical (magnetic) disk or a flexible disk drive, and a port on the device of a portable electronic storage medium such as a USB (Universal Serial Bus) memory. The image data read and captured by the image capturing peripheral device 108 is a digital scanner, and is transmitted and received as a digital signal to the central processing control device 101 via the input / output interface 109 and the bus 110.

  As described above, the pair image posting search program is stored in the storage device 107 of the pair image posting search device of the present embodiment. As shown in FIG. An image database 11; A processing result database 12; And an evaluation information database 13.

  A. The image database 11 includes all posted images (words) ID (Identification) (image ID, paired image ID, word ID), image data, image profile data (posted image type, image name, image comment), image A meta tag (image meta information, image meta tag, pair image meta information (combined), pair image meta tag (combined)), a word meta tag (word meta information, word meta tag), and the like are stored.

  B. The processing result database 12 stores processing result data related to processing results mainly for posting and searching.

  C. The evaluation information database 13 includes meta tag similarity data, search frequency data, pair search frequency data, search output priority data, and various parameters, search output priority calculation formulas, and search output priority calculation formulas constituting the same. , Weights of evaluations based on the first and second image types in the search priority order of the designated search type, time adjustment parameters (of the search frequency), meta tag similarity, and various thresholds of the search frequency ( The (truncated) value is recorded.

  Further, as the pair image posting search program is read and executed by the central processing control apparatus 101 of the pair image posting searching apparatus, the central processing control apparatus 101 has an image posting unit 21 and an image as shown in FIG. The search unit 22, the evaluation unit 23, and the search output unit 27 are further provided as subordinate functional units of the evaluation unit 23, a meta tag similarity calculation unit 24, a search frequency calculation unit 25, and a search output priority calculation unit 26. And are implemented.

  Each of these elements uses the data stored in the storage device 107 to perform post processing and search / view processing. Here, the posting process is the image type designation by the image contributor, whether it is a problem or resolution of the posted image, input of meta information, a single image of the problem or resolution, or a pair image of the problem and resolution Is a process of submitting two single images for the problem and solution to be configured later, and the search viewing process is a single image of whether the posted image is a problem by the image search viewer or a solution, or a problem and This is a process in which two images as a solution pair are searched and then viewed.

  In the present embodiment, the “image” refers to a visible image including a still image or a moving image. In addition, text information and color as a visible image, and audio information accompanying reproduction of a still image or a moving image are also included as information complementing the “image” information. In addition, “pair” of “pair image” refers to a pair or combination of two images that become problem solving knowledge or complement each other so as to become problem solving knowledge.

  The relationship between the two images in the pair includes “Problem and Solution” and “Problem and Solution”, “Needs and Seeds”, “Cause and Result”, “Reason for Reason and Conclusion”, “Before Improvement” And after improvement ”and“ defects and advantages ”, and these relationships are all complemented as problem solving knowledge by combination.

  In addition, the meaning of the pair image targeted by the pair image posting search apparatus of the present embodiment has a relationship that is complemented as problem solving knowledge by a combination including “problem and solution” or “problem and solution”. A combination of two images.

  FIG. 3 shows an example of a pair image with the theme “fasten with a tool”. In the image group in the left column in the figure, tools (plus screw S, plus screw A, minus screw D, nut C, nail B) that are fastened to something are displayed, and these are all tools for fastening. Is considered as the image group in question. In the image group on the right column in the figure, all the tools for holding the object to be fastened (plus driver A, plus driver C, minus driver F, bolt E, kanachi C) are displayed. Consider a group. Then, the combination in which the image group of the problem in the left column and the image group of the solution in the right column are connected is regarded as two images that may be pair images.

  Hereinafter, for the sake of explanation, it is assumed that the two images that form a pair are the problem image and the solution image, respectively, and these two are combined for use in problem solving.

  Of the keyword search and key image search provided in the present embodiment, the keyword in the keyword search refers to character information for performing a key or query (inquiry). Further, the key image in the key image search refers to an image on which a key or a query (inquiry) is performed.

  In the present embodiment, the search output image refers to an image searched and output by either keyword search or key image search, or an image targeted for search output. The search output image candidate indicates an image group that is narrowed down as a search output image candidate to be searched and output in the entire search process.

[Outline of post processing and search processing based on functional block diagram]
Hereinafter, with reference to the functional block diagram shown in FIG. 2, the outline of the posting process and the search process in the pair image posting search apparatus of the present embodiment will be described.

  In the posting process, the image posting unit 21 stores the posted image data, meta information, and the like in the A. Record in the image database 11.

  On the other hand, in the search process, the image search unit 22 executes the A.D. The image data, meta information, and the like recorded in the image database 11 are read out, and the image ID is used as a record key, and the image meta tag and the word meta tag are written in the cache memory 103 and transferred to the evaluation unit 23.

  In the meta tag similarity calculation unit 24, the search frequency calculation unit 25, and the search output priority calculation unit 26, the evaluation unit 23 performs meta tag similarity calculation, search frequency calculation, and search output priority calculation, which are subordinate processes of the search process. .

  In all the search processing steps, the search output image candidates are narrowed down using the evaluation results of these subordinate processes, and finally the search output image is determined. The search output unit 27 calculates the A.D. Image data and the like stored in the image database 11 are read out and output together with image profile data (posted image type, image name and image comment), and displayed on the image search screen.

  The result of the evaluation of each of these processes is the C.I. It is recorded in the evaluation information database 13 as meta tag similarity data, search frequency data, pair search frequency data, and search output priority data. Further, B.B. The processing result is recorded in the processing result database 12.

[Operations in the pair image submission search device]
Next, the contents of the operations performed by the image contributor and the image retrieval viewer in the pair image posting retrieval apparatus of this embodiment will be described.

  The operation of the pair image posting search apparatus of this embodiment can be roughly classified into two operations, a posting operation and a searching operation, and the users are two users, an image contributor and an image search viewer.

[Posting operation]
First, a posting operation in the pair image posting search device of this embodiment will be described with reference to a flowchart of an example of a posting operation shown in FIG. In this embodiment, FIG. 4 is a flowchart of posting operation, and FIG. 5A is a flowchart of search operation. FIG. 5b is a flowchart of the search operation in the modification mode of this embodiment.

  The image contributor first prepares an image to be posted in the storage device 107 of the pair image posting search device (step S1).

  The image data to be posted is captured by the image capturing device 108 or captured via an electronic storage medium such as USB or the Internet.

  Next, the image poster starts the pair image posting search program (step S2).

  Thereby, the image posting screen illustrated in FIG. 6 is opened (step S3). In this image posting screen example, there are two image posting trays for problems and solutions at the top, an image name input column at the bottom, a comment input column for the posted image, and a meta information input column for searching and viewing this posted image There is a post button in the upper right corner of the image submission screen.

  The image contributor installs the image to be posted prepared in the storage device 107 in step S1 in either the problem or the solution of the image posting tray shown in FIG. 6 (step S4).

  Next, the image contributor inputs the image name, comment, and meta information, which are image information, in the image name column, comment column, and meta information column shown in FIG. 6 (step S5).

  Finally, the image contributor presses (clicks) the posting button shown in FIG. 6 to post the image to be posted (step S6).

[Search operation]
Next, a search operation process performed by the pair image posting search device of this embodiment will be described with reference to a flowchart of an example of the search operation shown in FIG.

  Here, a description will be given assuming a procedure in which an image is once searched by keyword search and a search output image is searched using the image as a key image. FIG. 22 shows a life cycle of image posting / retrieval processing for a series of operations of the posting procedure and the retrieval procedure.

  The image search viewer first starts the pair image posting search program (step S11).

  Thereby, the image search screen shown in FIG. 7 is opened (step S12). In this image search screen, the upper part is a search key input area, the lower part is a search image output area, the upper search key input area has two key image trays, a keyword input field, a problem and a solution, and five search buttons. Is arranged.

  The image search viewer inputs a keyword in the keyword input field shown in FIG. 7 and searches for and determines an image as a key image (step S13). In this case, the image candidates to be used as key images are displayed in the search image output area at the bottom of the image search screen as shown in FIG. In the example of FIG. 7, the image search viewer inputs “turn the screw” as a keyword, whereby “plus screw S”, “minus screw D”, and “frame X” are searched and output as key image candidates. The image search viewer determines one of them as a key image.

  Next, the image search viewer searches the search output image using the image searched in step S13 as a key image. For this purpose, the key image is set on the key image tray shown in FIG. 7 (step S14). FIG. 8 shows the screen state in this case. That is, the image search viewer sets the determined key image “minus screw D” to the “problem” key image tray.

  Thereafter, the image search viewer searches for the search output image by pressing one of the five search buttons shown in FIGS. 7 and 8 (step S15), and until the target search output image is found. The series of operations in steps S13 to S15 are repeatedly executed to complete the image search (step S16). FIG. 11 shows a screen state where the search result on the image search screen is displayed. In this case, as shown in FIG. 11, the search button (3. Same type: K problem → T problem) is pressed.

  Next, a change mode of the search operation processing by the pair image posting search device of the present embodiment will be described with reference to a flowchart of another example of the search operation shown in FIG.

  Here, a plurality of key images are prepared in advance, and subsequently used for repeated search. FIG. 22 shows the life cycle of the image posting / retrieval process for a series of operations of the posting procedure and the retrieval procedure.

  The image search viewer first starts the pair image posting search program (step S11).

  Thereby, the image search screen shown in FIG. 7 is opened (step S12). In this image search screen, the upper part is a search key input area, the lower part is a search image output area, the upper search key input area has two key image trays, a keyword input field, a problem and a solution, and five search buttons. Is arranged.

  Next, the image search viewer opens the image search standby screen shown in FIG. 9, and installs a plurality of candidate key images on the image search standby tray (step S17). FIG. 10 shows a situation in which an image (plus screw A) on the image search standby tray of this image search standby screen is set as a key image on the key image tray in the image search screen shown in FIG. ing.

  Thereafter, the image search viewer searches for the search output image by pressing any of the five search buttons shown in FIG. 7 (step S15), and continues to step S17 and until the target search output image is found. The series of operations in step S15 is repeatedly executed to complete the image search (step S16). FIG. 11 shows a screen state in which search results are displayed on the image search screen of FIG. In this case, the search button (3. Same type: K problem → T problem) is pressed.

[Search Classification: Keyword Search and Key Image Search]
As described above, there are two methods for searching in the pair image posting search device of the present embodiment: keyword search and key image search.

  As shown in the example of the image search screen in FIG. 7, the keyword search is a method of finally searching for a search output image by inputting a keyword and pressing a search button. As shown in the example of the image search screen in FIG. 11, a search output image is searched by pressing a search button after setting a key image on a key image tray.

  As shown in FIG. 12, there are three types of search, ten types for keyword search and seven types for key image search, for a total of ten types.

[Search classification: heterogeneous search, homogeneous search and pair search]
There are three types of key image search in the pair image posting search device of this embodiment: single image heterogeneous image search, single image homogenous image search, and pair image search.

  The two types of single image heterogeneous image search and homogenous image search indicate whether the image type of the search output image to be searched is different type or the same type in terms of the image type of the key image set in the key image tray to be set. It is a kind name.

  A single image heterogeneous image search is to search for a search output image that is a solution image or a problem image using the problem image or the solution image as a key. Alternatively, a search output image that is a problem image or a resolution image is searched using the resolution image as a key, and a pair image search is a key for a problem, a resolution, or a pair image, that is, a pair of a problem and a resolution image. As described above, a pair of images, that is, a search output image that is a pair of problem and solution images is searched.

[Assign each search type function to the search button]
There are five search buttons in the pair image posting search device of the present embodiment as shown in the image search screen of FIG.

  Hereinafter, as an expression of the type of key image search, the image type of the key image and the search output image is defined as “the image type of the key image is either a problem or a solution → the image type of the search output image is either a problem or a solution”. Are expressed as “K (image type) → T (image type)”.

  Based on this expression, the image types of the key image and the search output image by the five search buttons shown on the image search screen of FIG. Single type: K (problem) → T (solution) 2. Single substance: K (solution) → T (problem) Single type: K (problem) → T (problem) Single type: K (solution) → T (solution) Pairs: K (Problem) → T (Problem / Solution), K (Solution) → T (Problem / Solution), K (Problem / Solution) → T (Problem / Solution).

[Keyword search]
The keyword search in the pair image posting search device of this embodiment uses three search buttons for each of the three types of keyword search contents shown in FIG. In FIG. 13, the search button subjected to the search operation is circled.

  About these three kinds of keyword searches, A1. KW1 problem image search, which is a single image search, A1. KW2 resolution image search, which is a single image search, and A2. The KW3 pair (problem / solution) image search is a pair image search, and these are represented by symbols KW1, KW2, and KW3, respectively.

  Screen states in these three types of keyword searches are shown in FIGS. FIG. 15 shows keyword search A1. FIG. 16 shows the screen state in the KW1 problem image search of the single image search, and FIG. FIG. 17 shows a screen state in KW2 solution image search of single image search, and FIG. The screen state in KW3 pair (problem / solution) image search of pair image search is shown.

[Key image search]
The key image search in the pair image posting search device of this embodiment uses five search buttons for each of the seven types of key image search contents shown in FIG. In FIG. 14, the search button subjected to the search operation is circled.

  Regarding these seven key image searches, B1. KG1 solution image search K (problem) → T (solution), which is a heterogeneous image search of a single image search, B1. KG2 problem image search K (solution) → T (problem), which is a heterogeneous image search of a single image search, B1. KG3 problem image search K (problem) → T (problem), which is a similar image search of single image search, B1. KG4 solution image search K (solution) → T (solution), which is a similar image search of a single image search, B2. KG5 pair image search K (problem) → T (problem / solution) that is a single key image (problem image) search of pair image search, B2. KG6 pair image search K (solution) → T (problem / solution) which is a single key image (solution image) search for pair image search, B2. KG7 pair image search K (problem / solution) → T (problem / solution) which is a pair key image search of the pair image search, and these are represented by symbols KG1, KG2, KG3, KG4, KG5, KG6 and KG7, respectively.

  Screen states in the four types of key image search out of these seven types are shown in FIGS. FIG. 18 shows the B1. FIG. 19 shows the screen state in the KG1 solution image search (K (problem) → T (solution)) of the heterogeneous search of the single image search. FIG. 20 shows a screen state in the KG3 problem image search (K (problem) → T (problem)) of the same kind search of the single image search. FIG. 21 shows a screen state in KG5 pair image search (K (problem) → T (problem / solution)) of single key image (problem image) search of pair image search. The screen state in KG7 pair image search (K (problem / solution) → T (problem / solution)) of the pair key image search of the pair image search is shown.

[Handling of simultaneous keyword search and key image search]
In addition to entering keywords in the keyword entry field shown in the image search screen of FIG. 7, if a key image is installed in either or both of the problem or solution on the key image tray, whether to return an error and stop processing A method of combining key word meta tags and key image meta tags and processing as key image processing is conceivable. In the pair image posting search device of this embodiment, the specific processing content is not described, but an error is returned and the processing is interrupted.

[Processing details for moving images to the image tray]
In the pair image posting search device of the present embodiment, the processing contents of moving and setting an image to the image tray, specifically, moving and setting the key image to the key image tray, and the key image candidate to the image search waiting tray The contents of the moving installation process are to copy link destination information for accessing various data of the corresponding image so as to be displayed on the image tray, and to call and display the image from the link destination. Specifically, this link is an A.D. of the storage device 107 shown in FIG. This is a link that allows access to and display of all data including the image data and image profile data (including post image type, image name, and image comment) with the image ID in the image database 11 as the head.

[Meta tag extraction]
The meta tag of the keyword and the key image is a search term used for the search question, and the meta tag of the search output image corresponds to the search target document.

  The extraction of meta tags refers to indexing (element decomposition) into text data elements based on meta information attached to keywords or key images and meta information attached to search output image candidates.

  The meta information is acquired from information input in the meta information input field at the bottom of the image posting screen shown in FIG. 6 at the time of posting.

  This meta information is used as text data for calculating the degree of similarity between the keyword or key image search term and the search target document of the search output image, that is, the degree of suitability in information search.

  A keyword search is performed by asking a search question using a character string directly input by a user using a keyboard or the like as a search key, and indexing a target search output image from meta information attached thereto, that is, a meta tag decomposed as an element to be searched. This is a method of collating and searching as a document.

  In key image search, an image contributor makes a search question using the meta tag indexed from the meta information entered at the time of image submission as a key, and targets the target search output image for the meta tag indexed from the meta information attached to it. This is a method of searching by matching.

  This indexing includes stemming used for general information retrieval, that is, processing for removing a part of word form change and converting it to the same expression.

[Meta tag similarity]
The meta-tag similarity refers to the similarity between the key-side word or image meta-tag and the search output-side image meta-tag, and is an evaluation index used to calculate the search output priority. This meta-tag similarity is calculated by collating these two meta tags after rearrangement based on the character code order of the meta tag of the keyword or key image and the meta tag of the search output image candidate.

[Search results]
The search results in the pair image posting search device of the present embodiment indicate two results, that is, a result that each image is used as a search output of a search output image as a key image and a result that a search output image is searched and output.

[Search frequency]
The search frequency is the number of processes in a specific time interval for obtaining the search results, and is an evaluation index used for calculating the search output priority. Each image is used as a key image for search output of the search output image. The search frequency and the search output image frequency are indicated.

  The frequency used for the search output of the search output image as the former key image is that the image once retrieved as the search output image by the keyword or the key image is selected again by the search viewer and used as the key image. When this frequency is high, it can be regarded as an image directly evaluated as a conforming document by the user.

  On the other hand, when the frequency of search output as the latter search output image is high, it can be regarded as a highly popular image including a meta tag related to a problem desired by the user and knowledge of solution.

[Search priority]
Next, the search priority order in the pair image posting search device of this embodiment will be described. FIG. 26 shows a list of 10 types of all search types and the search priority order, and the small numbers in the numbers in the search priority order column in FIG. FIG. 27 shows an example of setting of three types of search priority orders for keyword search, and the numbering on the search button in FIG. FIG. 28 shows an example of setting of seven types of search priority orders for key image search, and the small numbers in the numbers on the search button in FIG.

  The search priority order gives priority to the search output image candidate and the search output image for any of the image types of the problem, the solution, and the pair of the problem and the solution based on the search results in all 10 types of search. Search order.

  In the pair image posting search device of the present embodiment, the image search viewer can select an image posted by moving to the image posting tray shown in FIG. 6 by the image poster regardless of the image type specified at that time. It can be freely installed in the problem key image tray and the solution key image tray. Therefore, the posted image has a search result in four types, that is, the image of the problem, the solution, and the problem image as one of the pair images and the image of the solution as one of the pair images.

As an example of setting the three types of search priority order of keyword search, as shown in FIG.
A1. In KW1 problem image search, which is a single image search, Problem → 2. Resolution,
A1. In KW2 resolution image search, which is a single image search, Solution → 2. problem,
A2. In KW3 pair image search, which is a pair image search, There are only pair images.

As an example of setting the seven search priority orders for key image search, as shown in FIG.
B1. In KG1 resolution image search, which is a heterogeneous image search of a single image search, Solution → 2. problem,
B1. In the KG2 problem image search, which is a heterogeneous image search of a single image search, Problem → 2. Resolution,
B1. In the KG3 problem image search, which is a similar image search of the single image search, Problem → 2. problem,
B1. In the KG4 solution image search, which is a similar image search of the single image search, Solution → 2. Resolution,
B2. In KG5 pair image search, which is a single key image (problem image) search of pair image search, Pair image → 2. Problem Resolution,
B2. In KG6 pair image search, which is a single key image (solution image) search for pair image search, Pair image → 2. Problem Resolution,
B2. In KG7 pair image search, which is a pair key image search of a pair image search, There are only pair images.

[Processing procedure of pair image submission search device]
Hereinafter, a procedure of processing performed by the pair image posting search device of the present embodiment will be described.

[Post processing]
A post processing procedure in the pair image post search device of the present embodiment will be described with reference to the post processing flowchart shown in FIG. In the present embodiment, only the flowchart of the posting process is shown in FIG.

  When an image contributor installs an image in one of the image posting trays in the image posting screen shown in FIG. 6 or solves the problem, Are provisional image IDs.

  Next, when the image contributor clicks (presses) a post button on the image post screen shown in FIG. 6, the pair image post search device activates the input information check function and checks the input information (step) S101).

  The contents of the check include “There is a blank”, “No key image is installed”, “Two of the images, image name, comment, and meta information must be entered in all input fields. There is a check for an abnormal state such as “Images are simultaneously installed in the image posting tray in question and the image posting tray in resolution”.

  As a result of the check, it is determined whether or not any of these abnormal states is recognized (step S102). If the abnormal state is recognized, an error or warning is returned and the process is interrupted (step S103). When there is no abnormal state, that is, when the normal state is recognized, the process proceeds to the next step S104.

  From step S104 to step S108, the posted image data is recorded.

  First, the temporary image ID distributed in the posted image is replaced with the official image ID, and the official image ID is stored in the A.E. It is recorded in the image (word) ID area of the image database 11 (step S104).

  Next, the image data is stored in A. It records in the image data area of the image database 11 (step S105).

  Next, the image type indicating whether or not the problem of the posted image, which is the image profile data, the image name and the comment input via the image posting screen shown in FIG. It records in the image profile data area of the image database 11 (step S106).

  Next, when the meta information input via the image posting screen shown in FIG. 6 is indexed and a plurality of meta tags are extracted at this time, the character code system adopted by the pair image posting search device of the present embodiment is used. All meta tags are rearranged based on the character code order (step S107).

  Next, the image meta information and the image meta tag obtained by indexing and rearranging the image meta information are stored in the A. It records in the image meta tag area of the image database 11 (step S108).

  The posted image data recorded from step S104 to step S108 is related to the image ID recorded in step S104, and the A. It is recorded in the image data area of the image database 11.

  Finally, the processing results of the posting process are stored in the B. It records in the processing performance database 12 (step S109).

[Search processing]
With reference to the flowchart of the search processing shown in FIG. 24, the search processing procedure in the pair image posting search device of this embodiment will be described. The flowchart of FIG. 24 shows the main flow of the search process.

  First, search output image candidate determination processing is performed (step S111). This uses the meta tag similarity record for the meta tag of the key image in the current search process in the process record data. The ID of the previous image with a high meta tag similarity is stored in the B. process of the storage device 107 in FIG. It reads from the results database 12 and makes it a search output image candidate. This step is a step of preparing and securing search output image candidates based on the meta tag similarity at the start of the search process. The search output image candidate determination process will be described in detail later with reference to the flowchart of FIG.

  Next, the meta tag similarity is calculated for the search output image candidate (step S112). This is to compare the meta tag on the key side of the keyword or key image with the meta tags of all search output image candidates, and calculate the degree of similarity of the meta tags. This meta tag similarity calculation process will be described in detail later using the flowchart of FIG.

  Next, search frequency calculation processing is performed (step S113). This is to calculate the search frequency of all search output image candidates up to the present time. This search frequency calculation processing will be described in detail later using the flowchart of FIG.

  Next, time adjustment processing is performed (step S114). This means that the search frequency calculated in step S113 is weighted and corrected by time in order to reflect the latest search tendency. This time adjustment process will be described in detail later using the flowchart of FIG.

  Next, a process for determining a search output image is performed (step S115). This uses the search output priority calculation criteria that comprehensively evaluate the meta tag similarity, search frequency, etc. for all search output image candidates, and determines the search output image by calculating the search output priority. It is processing to do. In this process, from the viewpoint of the search output priority calculation standard, the meta tag similarity and the search frequency are determined as the search output image from the highest overall. This search output image determination process will be described in detail later using the flowchart of FIG.

  Finally, the search output image is displayed (step S116). This is to output and display the determined search output image in the search image output area shown in FIG.

  The search output image is determined by the above search process.

[Acquisition and calculation of target data for search output image determination]
Next, regarding the processing related to the acquisition and calculation of search output image determination target data in the search processing in the pair image posting search device of the present embodiment, the search output image determination target data acquisition and calculation shown in FIG. This will be described with reference to a processing flowchart. Note that the flowchart of FIG. 25 corresponds to the process of step S115 of FIG.

  Acquisition and calculation of target data for search output image determination is a search output priority calculation criterion, which is described later in [4. In the search output priority calculation formula described in the section “Calculating Search Output Priority of Search Output Image Candidate”, the search output priority calculation target data of the search output image candidate is incorporated, and the search output priority of the search output image candidate is set. It is the main step for calculating.

  This search output priority calculation target data refers to the data constituting the performance and evaluation data used for calculating the search output priority. Specifically, the meta tag similarity data, search frequency data, pair search frequency data, search output priority data of search output image candidates, and three evaluations constituting a search output priority calculation formula that are various parameters. These are weighting, evaluation weights based on the first and second-ranked image types in the search priority order of the specified search type, time adjustment parameters (of search frequency), meta-tag similarity, and various threshold (cut-off) values of search frequency. These include C.C. data together with the data of all past images other than the search output image candidate based on the designated search type (instruction search type). It is recorded in the evaluation information database 13.

[1. Setting of Meta Tag Similarity (Similarity with Key Image Based on Current Meta Tag) of Search Output Image Candidate (Step S121 in FIG. 25)]
First, after a threshold value (truncation) value: meta tag similarity search output image candidate truncation process lower than the meta tag similarity executed in step S1107 of the meta tag similarity calculation process shown in FIG. 31, the meta tag of the search output image candidate The similarity (M-Tgt) is set.

[2. Setting of search frequency as key image of search output image candidate (step S122 in FIG. 25)]
Next, after the threshold value (truncated) value: the search output image candidate that is less than the search frequency is truncated in step S1207 of the search frequency calculation process shown in FIG. 32, the search output image candidate is searched as a key image. Set the frequency (S-Key).

  In this setting, first, the first and second highest priority image types based on the current search type are (img1) and (img2), respectively. Next, search frequencies as key images of search output image candidates in these image types are set as (S-Key_img1) (search output image candidate ID) and (S-Key_img2) (search output image candidate ID), respectively.

[3. Search frequency setting of search output image candidate as search output image (step S123 in FIG. 25)]
Similarly, the threshold (cut) value executed in step S1207 of the search frequency calculation process shown in FIG. 32: after the search output image candidate is cut off below the search frequency, the search output image candidate as the search output image is displayed. A search frequency (S-Tgt) is set.

  In this setting, first, the first and second highest priority image types based on the current search type are (img1) and (img2), respectively. Next, search frequencies as key images of search output image candidates in these image types are set as (S-Tgt_img1) (search output image candidate ID) and (S-Tgt_img2) (search output image candidate ID), respectively.

[4. Calculation of Search Output Priority of Search Output Image Candidate (Step S124 in FIG. 25)]
Next, the search output priority calculated in step S1303 of the search output image determination process shown in FIG. 33 is calculated using the following search output priority calculation formula.

  The search output priority calculation formula of the present embodiment is a meta tag similarity evaluation (similarity with a key image based on the current meta tag), which is three evaluations of search output image candidates, and a search frequency evaluation as a key image. The total is obtained by weighting with the search frequency evaluation as the search output image. That is, A. of each search output image candidate. (WM-Tgt · M-Tgt), which is a metatag similarity evaluation (similarity with a key image based on the current metatag); It is a search frequency evaluation as a key image (wS-Key · wS-Key_PR_img [1or2] · S-Key_img [1or2]). It is calculated by a total of (wS-Tgt · wS-Tgt_PR_img [1or2] · S-Tgt_img [1or2]) which is a search frequency evaluation as a search output image.

  The search output priority calculation target data input to the search output priority calculation formula of the present embodiment includes the meta tag similarity recorded in the meta tag similarity calculation process shown in FIG. 31 and the search frequency calculation process shown in FIG. Finally, C.I. Similarly to the results of the search frequency in the first and second highest priority image types of the instruction search type recorded in the evaluation information database 13, C.I. From the evaluation information database 13, the meta tag similarity of the search output image candidate and the search frequency in the search priority order first and second order image types of the search result type, the three evaluation weights as various parameters, and the instruction There is a weight of evaluation based on the first and second ranked image types in the search priority order of search types.

  The search output priority is calculated by first calculating the A [1-1-1] meta tag similarity (similarity with the key image based on the current meta tag) (M-Tgt) and B [1-1-1 ] [S-Key_img [1or2]) and C [1-1-1] search output image as a search frequency as a key image (search frequency as the first and second-ranked image type of instruction search type) (S-Tgt_img [1or2]) is set as the search frequency (search frequency as the first and second-ranked image type in the search priority order of the instruction search type).

  At this time, (M-Tgt) ≧ 0, (S-Key_img [1or2]) ≧ 0, (S-Tgt_img [1or2]) ≧ 0, and (S-Key_img [1]) ≧ (S-Key_img) [2]), (S-Tgt_img [1]) ≧ (S-Tgt_img [2]).

  Next, the search priority of the search direction of the search direction in the search frequency evaluation as the B [1-1] key image, which is the weight of the evaluation based on the image type of the search priority of the search position of the specified search type. Evaluation weight based on the image type (wS-Key_PR_img [1or2]) and the evaluation based on the search priority ranking first and second order image types in the search frequency evaluation as the C [1-1] search output image Weight (wS-Tgt_PR_img [1or2]).

  At this time, (wS-Key_PR_img [1or2]) ≧ 0, (wS-Tgt_PR_img [1or2]) ≧ 0, (wS-Key_PR_img [1]) ≧ (wS-Key_PR_img [2]), (wS-Tgt_PR_img [1] ) ≧ (wS−Tgt_PR_img [2]).

  Of the search types, only the pair images are searched, and KW3 and KG7 having no distinction between the first and second highest priority image types are not considered.

  Next, the weight (wM-Tgt) of A [1] meta tag similarity evaluation (similarity to the key image based on the current meta tag), which is the evaluation weight, and the search frequency evaluation as B [1] key image Weight (wS-Key) and C [1] search frequency evaluation weight (wS-Tgt) as a search output image are set.

  At this time, (wM−Tgt) ≧ 0, (wS−Key) ≧ 0, and (wS−Tgt) ≧ 0.

  Finally, A. is a search output priority calculation formula. Meta tag similarity evaluation (similarity with key image based on current meta tag) (wM-Tgt · M-Tgt); Search frequency evaluation as key images (wS-Key, wS-Key_PR_img [1or2], S-Key_img [1or2]); The sum of the three terms of the search frequency evaluation (wS-Tgt · wS-Tgt_PR_img [1or2] · S-Tgt_img [1or2]) as the search output image is calculated.

  (Img1) or (img2) expressed as (img [1or2]) is a search frequency in the image type of the search priority order first in the instruction search type or the search priority order second in the instruction search type as the key image or the search output image. It means to include the results of.

  FIG. 39 shows search specifications and target images for the purpose of calculating search output priorities for three search output image candidates based on the search specifications in the search output priority calculation example and determining the search output order. It shows.

[Search specifications]
For the search specification, the instruction search type is KG1, that is, a solution image search of single image search (the key image is a problem image, the search output image is a solution image, indicated by K [problem] → T [solution]). As the search priority order image type, the first image is the image type of (img1) as the resolution image, and the second image is the image type of (img2) as the problem image.

[Target image]
The image name of the key image is “plus screw S”, the image name of the search output image candidate (1) is “plus driver A”, the image name of the search output image candidate (2) is “minus driver F”, and the search output image candidate The image name of (3) is “plus screw A”. These images are obtained by using the image 1 shown in FIG. 3 as a key image and the images 2, 3 and 4 as search output image candidates.

  In FIG. 39, search output image candidate ID, image itself, post-granting image type, meta tag, and meta tag similarity are shown as target image information.

  The types of image added at the time of posting of the search output image candidates (1) to (3) are “solution”, “solution”, and “problem”, respectively.

  The meta tag similarity of the search output image candidates (1) to (3) is calculated by adopting the number of common meta tags, that is, the number of common elements, as will be described later, and “1”, “0”, It is “2”.

  In this calculation example, the search frequency time adjustment shown in the search frequency time adjustment step (step S1204) in FIG. 32 is not performed. Suppose that it is constant. Further, deletion based on threshold values of meta tag similarity, search frequency, and search output priority is not applied.

[Calculation of search output priority and search output display order]
Next, the process of calculating the search output priority and determining the search output display order based on the search specification and the target image will be described with reference to the search output priority calculation example shown in FIG.

  FIG. 40 shows search output image candidates (1) to (3). <Results of the search frequency in the image type of the first and second place in the search priority order of the meta tag similarity and the instruction search type> 2. <Evaluation Weight Based on First and Second-ranked Image Types in Search Priority Order of Instruction Search Types> A numerical example of <weight of evaluation (for each performance)> is shown.

  Here, (M-Tgt) is a meta tag similarity, (S-Key) is a search frequency as a key image, and (S-Tgt) is a search frequency as a search output image, and a weight added to each track record. The main part of the variable name is shown.

  In FIG. Among the <results of search frequencies in the first and second image types in the search priority order of meta tag similarity and instruction search type>, the meta tag similarity is A [1-1-1]. Since (M-Tgt) targets the meta tag at the time of posting for each search output image candidate, regardless of the image type that is the first and second-ranked search priority of the instruction search type, B [1-1-1], which is the search frequency in the first and second image types in the search priority order of the instruction search type. (S-Key_img [1or2]) and C [1-1-1]. As for (S-Tgt_img [1or2]), two types of results for the first and second highest priority image types of the instruction search type are used.

  It should be noted that the results of the search frequency as the key image and the search output image are generally set as numerical values as being high in the same image type as each post-given image type.

  Next, 2. B [1-1] of <weight of evaluation based on first and second-ranked image types in search priority order of instruction search type>. (WS-Key_PR_img [1or2]), and C [1-1]. Also for (wS-Tgt_PR_img [1or2]), two types of weights are set for the first and second ranked image types in the search priority order of the instruction search type. In this example, since the search priority order image type based on the instruction search type KG1 is a solution, the search image as the key image and the search output image is generally given a higher weight for the resolution image than the problem image. The search output image candidate (1) and the search output image candidate (2) that are the first image types in the search priority order are twice the search output image candidate (3) that is the second image type in the search priority order. (2.0) is set.

  Next, 3. <[Evaluation Weights (for each achievement)] A [1]. (WM-Tgt), B [1]. (WS-Key), C [1]. About (wS-Tgt), one each is set. In this example, A [1]. (WM-Tgt) has a meta tag similarity of A [1]. The equivalent numerical value is larger than (wM-Tgt) B [1]. (WS-Key) and C [1]. Considering each search frequency of (wS-Tgt), A [1]. The meta tag similarity of (wM-Tgt) is set to 100 times (100).

  Since the search output priority calculation example shown in FIG. 40 and the search frequency in the image type ranked first and second in the search priority order of the instruction search type are generally different in scale (scale), In each of the meta tag similarity and the search frequency of all images to be evaluated, the weight of evaluation of each performance may be set after normalization using a rank ratio or standard deviation.

  Next, the process up to the determination of the search output order will be described.

  Search output image candidates (1) to (3) 1. shown in the search output priority calculation example of FIG. 1. <Results of search frequency in the image type of the first and second-ranked search priority order of meta tag similarity and instruction search type> <Evaluation weight based on the first and second ranked image types in the search priority order of the instruction search type> and 3. By weighting <the weight of evaluation (for each achievement)> <Evaluation of A, B, C> (WM-Tgt · M-Tgt) (meta tag similarity), B.I. (WS-Key · wS-Key_PR_img [1or2] · S-Key_img [1or2]) (search frequency as key image), C.I. (WS-Tgt · wS-Tgt_PR_img [1or2] · S-Tgt_img [1or2]) (search frequency as a search output image) is summed to obtain a search output priority.

  As a result, the search output priorities of the search output image candidate (1), the search output image candidate (2), and the search output image candidate (3) are “580”, “165”, and “528”, respectively. The order is the search output image candidate (1), search output image candidate (3), and search output image candidate (2).

[5. Determination of Search Output Image (Step S125 in FIG. 25)]
Through the above-described process, the search output priority of all search output image candidates is calculated, and the search output image in step S1309 of the search output image determination processing flow shown in FIG. 33 is determined.

[Detailed procedure for each process]
The detailed procedure of each process by the pair image posting search device of this embodiment will be described below.

[Search output image candidate decision processing]
With reference to the flowchart of the search output image candidate determination process shown in FIG. 29, the search output image candidate determination process in the pair image posting search apparatus of this embodiment will be described. Note that the flowchart of FIG. 29 corresponds to the process of step S111 of FIG.

  First, the search type (instructed search type) specified by the image search viewer is recognized (step S1001). In the case of keyword search, there are three types of states shown in FIGS. 12 and 13, and in the case of key image search, one of the seven types of states shown in FIGS. This is determined from the recognition of the search button, the input status of the character information in the keyword input field, and the image installation status on the key image tray.

  Regarding the states other than FIGS. 12 to 14, the pair image posting search device of the present embodiment returns an error and interrupts the processing.

  Next, input information is read (step S1002). The input information refers to the cache memory 103 shown in FIG. 1 so that the image search viewer 22 of the central processing control apparatus 101 shown in FIG. 2 determines search output image candidates when the image search viewer presses the search button. Is read and / or temporarily written information.

  More specifically, the input information is read in the case of the search types KW1 to KW3 (3 types) in the storage device 107 shown in FIG. In the case of the search types KG1 to KG6 (six types) for reading and writing word IDs and word meta tags stored in the image database 11, A. When reading and writing the image ID and the image meta tag stored in the image database 11 and the search type KG7, the A. Reading and writing of the pair image ID and pair image meta tag of two images stored in the image database 11.

  Next, input information is read as supplementary processing in the next two cases (first case and second case) with no search results (step S1003).

  The first case is a supplementary process for a case where there is no search result by the input keyword in the case of the search types KW1 to KW3 (3 types).

  Specifically, a new word ID is assigned to the input keyword, and A. It is recorded in the image (word) ID of the image database 11, and is divided into commas, which is a keyword meta-tagging procedure, indexing, and rearrangement based on the character code order.

  Thereafter, the meta tag of this keyword is stored in the A. An image (word) ID in the image database 11 and a word meta tag are recorded in association with each other.

  The second case is a supplementary process for a case where there is no key image search record as a pair of two images of the problem and solution in the case of the search type KG7.

  More specifically, the two image IDs are combined into one set, and a new pair image ID is assigned. Recorded in the image (word) ID of the image database 11 and divided into commas, which are procedures for meta-tagging the meta-information of two images, indexing processing, combination of meta-tags of two images, rearrangement organization based on the order of character codes, etc. I do.

  Thereafter, the A.A. The pair image (word) ID of the image database 11 and the pair image meta tag are recorded in association with each other.

  Next, the meta tag newly acquired by reading the input information (supplementary process) in step S1003 is recorded (step S1004).

  Specifically, the keyword meta tags for the search types KW1 to KW3 (3 types) and the pair image meta tags for the search type KG7 are stored in the A. Recorded in the image meta tag of the image database 11.

  Next, the keyword or key image input information is written into the cache memory 103 shown in FIG. 1 (step S1005). That is, the keyword or key image input information in each search process acquired and determined in the above processes is temporarily written in the cache memory 103.

  The input information in this process refers to A. of the storage device 107 in the case of the search types KW1 to KW3 (3 types). In the case of the search types KG1 to KG6 (six types) that are the word ID and the word meta tag in the image database 11, A. In the case of the image ID and the image meta tag in the image database 11 and the search type KG7, the A. A pair image ID and a pair image meta tag (combined) in the image database 11.

  Next, in order to specify the priority of search related to the image type of the search output image candidate ID to be searched, the search priority order is read for each search type (step S1006).

  Next, the search output image candidate ID is searched (step S1007). In the pair image posting search device of this embodiment, the A. In addition to directly searching and searching for an image meta tag from the image (word) ID of the image database 11, B. By searching for a search output image ID having a high meta tag similarity in the processing result database 12, an image ID desirable as a search output image candidate based on the instruction search type is efficiently searched.

  Finally, the search output image candidate ID is determined and written in the cache memory 103 (step S1008).

[Processing result database]
B. of the storage device 107 shown in FIG. The processing result database 12 stores the result data or history data of all the processing in the pair image posting search device of this embodiment including the processing of posting / searching, and this data is used for determining the search output image candidate. Is done.

  Hereinafter, the processing result database in the pair image posting search device of this embodiment will be described with reference to a data configuration example of the processing result database shown in FIG.

  B. Storage device 107 Examples of the data structure of the processing result database 12 include a processing ID, a processing time (at the time of search output), a search type (KW1 to KW3, KG1 to KG7), a search output image, and a search that is a profile of a key (word) image. ID and meta tag of output image and keyword or key image, search output image type (problem, solution, problem (of pair), solution (of pair)), search output rank, and search output as evaluation result There are priorities.

  The processing result database 12 includes, among search output priority calculation target data, meta tag similarity data of search output image candidates, search frequency data, pair search frequency data, search output priority data, determined search output ranks, and the like. In the course of the search process, the results recorded in the evaluation information database 13 of the storage device 107 are recorded as a history.

[Meta tag similarity calculation processing]
With reference to the meta tag similarity calculation processing flowchart shown in FIG. 31, the meta tag similarity calculation processing in the pair image posting search device of this embodiment will be described below. The flowchart of FIG. 31 corresponds to the process of step S112 of FIG.

  This process is broadly divided into two steps: a search output image candidate ID and meta tag cache memory writing step, and a meta tag similarity calculation step.

  In the step of writing the search output image candidate ID and meta tag to the cache memory, the meta tag of the search output image candidate and the meta tag of the keyword or key image written in the cache memory 103 in step S1005 of the search output image candidate determination process of FIG. Is collated in the metatag similarity calculation step, which is the next processing step, and preparations for calculating the metatag similarity are made.

  First, the search output image candidate ID and the meta tag are written into the cache memory 103 (step S1101).

  In the case of the search types KW1, KW2, and KG1 to KG6 (six types), the search output image candidate ID written in the cache memory 103 in step S1008 in FIG. 29 is read and stored using the search output image candidate ID as a key. A. of the device 107. The search output image candidate meta tag is duplicated from the corresponding part of the image meta tag of the image database 11 and written into the cache memory 103.

  In the case of the search types KW3, KG5, and KG6 (3 types), the search output pair image candidate ID written in the cache memory 103 in step S1008 is read, and the search output image candidate ID is used as a key to store the A of the storage device 107. . The pair image meta tag (combined) of the search output pair image candidate ID is read from the corresponding portion of the pair image meta tag of the image database 11 and is written in the cache memory 103.

  In the case of the search type KG7, as in the case of the search types KW3, KG5, and KG6 (three types), the search output candidate pair image ID written in the cache memory 103 is read in step S1008, and the search output image candidate ID is set. To the key, the A. The pair image meta tag (combined) of the search output pair candidate image ID is read from the corresponding part of the pair image meta tag of the image database 11 and is written in the cache memory 103.

  Next, the keyword or key image ID and meta tag written in the cache memory 103 in step S1008 of the search output image candidate determination process of FIG. 29 are read (step S1102).

  These are the keyword ID and the keyword meta tag for the search types KW1 to KW3 (3 types), the key image ID and the key image meta tag for the search types KG1 to KG6 (6 types), and the search type KG7. Is a key pair image ID and a key pair image meta tag (combined).

  Next, the ID and meta tag of the next search output image candidate written in the cache memory 103 in step S1101 are read (step S1103).

  Next, the meta tag of the keyword or key image read in step S1102 and step S1103 is collated with the meta tag of the search output image candidate, and the meta tag similarity is calculated (step S1104).

  Next, the calculated meta tag similarity is recorded (step S1105). The meta tag similarity of all search output image candidates (ID) calculated in step S1104 is written into the cache memory 103, and at the same time, the C.I. Duplicate and record in the meta tag similarity data of the evaluation information database 13.

  Next, the threshold (truncated) value: meta tag similarity in the program memory 102 shown in FIG. 1 is read (step S1106).

  Next, a threshold value (truncated) value: a search output image candidate lower than the meta tag similarity is truncated (deleted) (step S1107).

  This is for deleting search output image candidate IDs that are less than the threshold (truncated) value: meta tag similarity for the search output image candidates in the cache memory 103 arranged in order of meta tag similarity.

  Here, in order to calculate the search output priority of the search output image candidate in step S1303 of the search output image determination process flow shown in FIG. 33, the meta-tag similarity of the search output image candidate in step S121 in the search output image determination process shown in FIG. Set the degree.

  Finally, for all remaining search output image candidates, the process proceeds to the search frequency calculation process shown in the following search frequency calculation process flowchart shown in FIG. If there are no search output image candidates remaining at this time, the search process is terminated and the image search viewer is notified that there is no search output image on the image search screen shown in FIG.

[Details of meta tag similarity calculation processing]
The details of the meta tag similarity calculation process in the pair image posting search device of this embodiment will be described below with reference to the detailed flowchart of the meta tag similarity calculation process main body shown in FIG.

  The detailed flowchart of the meta tag similarity calculation processing main body in FIG. 34 corresponds to the processing in step S112 in FIG. 24 and details the meta tag similarity calculation step in step S1104 of the meta tag similarity calculation processing flow shown in FIG. It is.

  First, the meta tags on the key side are aligned and coded, and written to the cache memory 103 (step S1401).

  Here, the alignment is determination of a desired position when comparing codes on the search output side. The alignment method is based on the meta tag similarity calculation formula and calculation method. Encoding means that the character string of the meta tag is recognized as a code in the character code system of the pair image posting search device of the present embodiment so that it can be immediately used for calculation of the meta tag similarity.

  Next, the meta tags on the search output side are arranged and coded, and a list is created and written together with the image ID or the pair image ID (step S1402).

  FIG. 35 shows the structure of the meta tag encoded in step S1401 and step S1402, as a meta tag on the key side and a meta tag on the search output side of each search type.

  For KW1, KW2 and KW3 which are keyword searches, the meta tag on the key side is a word meta tag, and the meta tag on the search output side is an image meta tag because it is a single image search for KW1 and KW2, but KW3 is a pair image search. Therefore, it is a pair image meta tag (combined).

  On the other hand, for key images KG1, KG2, KG3, and KG4, the key meta tag is an image meta tag, and the search output meta tag is an image meta tag. Since KG5 and KG6 are pair image searches, the meta tag on the search output side is a pair image meta tag (combined). Similarly, for KG7 which is a key image search, since the meta tag on the key side is a pair image, it is a pair image meta tag (combined), and the meta tag on the search output side is also a pair image search, so a pair image meta tag (combined) It is.

  Next, the meta tags on the key side and the search output side are collated (step S1403). That is, the key-side meta tag code written in step S1401 is read from the cache memory 103, and the meta tag codes for the number of search output image candidates on the search output side are read and collated in order from the top in the list created in step S1402.

  Next, the meta tag code similarity is calculated (step S1404).

  The calculation of the meta tag similarity employed by the pair image posting search device of the present embodiment indicates the degree of coincidence between the elements of the two sets of the key side and the search output side.

  As an example of the calculation formula of the meta tag similarity, the set of meta tags on the key side and the search output side are set as K and T, respectively, and the meta tags included in both sets are (k1, k2,..., Kl), (t1, (t2,..., tm), and let K∪T be the meta tag common to both sets, and let (z1, z2,..., zn). Let l, m, n ≧ 0, l, m ≧ n.

  As another example of the formula for calculating the meta tag similarity, the number of common elements of K and T is divided by the total number of elements in at least one, and expressed by meta tag similarity = | K∩T | / | K∪T | There is a Jackard coefficient.

  In addition, there is a dice coefficient represented by meta tag similarity = 2 × | K∩T | / (| K | + | T |), which is obtained by dividing the number of common elements of K and T by the average number of elements in each set. .

  Also, there is a Simpson coefficient that is obtained by dividing the number of common elements of K and T by the minimum value of the number of elements in each set and represented by meta tag similarity = | K∩T | / min (| K |, | T |) .

  Jackard coefficient, dice coefficient, and Simpson coefficient are the number of common metatags divided by the number of metatags on the key side or the search output side, whichever is larger, or the average number of metatags , Divided by the smaller number of meta tags.

  It is also possible to consider using the number of common meta tags, that is, the number of common elements as a coefficient as it is.

  In addition, there is a set similarity index, which can be used, for example, as an index representing a geometric distance between vectors in an n-dimensional space represented by Euclidean distance.

  Here, in the pair image posting search device of the present embodiment, any of the above-described coefficients can be adopted, but it is assumed that there are generally few meta tags especially in the case of keywords on the key side, and the search output Since it is assumed that there are an extremely large number of meta tags depending on image candidates, the number of common meta tags, that is, the number of common elements is used as a coefficient as it is.

  Therefore, the meta tag similarity employed by the pair image posting retrieval apparatus of this embodiment is simply the number of common elements on the key side and the retrieval output side.

  Next, rearrangement based on the meta tag code similarity is performed (step S1405).

  After adding similarities to the list in which image IDs or paired image IDs and meta tag codes for the number of search output image candidates created in step S1402 are written, rearrangement is performed in descending order of similarity.

  Finally, the meta tag similarity and collation history are recorded (step S1406). That is, image IDs or pair image IDs, meta tag codes, and similarities corresponding to the number of search output image candidates, which are data on the list created in step S 1405, are stored in the B. This is added as meta tag similarity data in the processing result database 12.

  When the processing in step S1406 is completed, the process proceeds to recording of the meta tag similarity in step S1105 of the meta tag similarity calculation processing flow shown in FIG.

[Search frequency calculation processing]
Next, a search frequency calculation process in the pair image posting search apparatus of this embodiment will be described with reference to a search frequency calculation process flowchart shown in FIG. Note that the flowchart of FIG. 32 corresponds to the process of step S113 of FIG.

  This process is roughly divided into a data reading process in step S1201 and a search frequency calculation process in steps S1202 to S1205.

  First, data is read (step S1201). This is because the part of the search processing results related to the high meta tag similarity search output image candidate ID written in the cache memory 103 through the meta tag similarity calculation process shown in FIG. B. 107. The processing result data in the processing result database 12 is accessed and read as target data for the search frequency calculation.

  Next, the search frequency is calculated (step S1202).

  In the case of the search types KW1, KW2, KG1 to KG4 (six types), the B.B. In the case of the search types KW3 and KG5 to KG7 (four types) for the single search processing of the processing result data in the processing result database 12, the B.B. For the pair search processing as a pair image of the processing result database 12 and the single search processing as a single unit of each problem or solution constituting the pair image, all high meta tag similarity search output images are set for each interval. Calculate for candidates.

  Next, the calculated search frequency is written into the cache memory (step S1203).

  The search frequency of the specified time interval in the specified target period of all high meta tag similarity search output image candidates is temporarily written in the cache memory 103.

  Next, time adjustment of the calculated search frequency is performed (step S1204).

  Next, when it is desired to place importance on the latest search results with respect to the search frequency, time adjustment (weighting correction) of the search frequency is performed.

  Prior to this processing, according to the time adjustment processing flowchart shown in FIG. 36, the current time and the weight of the target period start time (WEIGHT (Wt0, Wtn)) and the weight (WtprocX) of the search point X are stored in the C.I. Read from the evaluation information database 13, read the search frequency of the specified time interval in the specified target period of all high meta tag similarity search output image candidates (single / pair) in the cache memory 103, and adjust the search frequency of each time interval (Weighting correction).

  Next, the search frequency is recorded (step S1205).

  The search frequency data adjusted in time (weight correction) is once recorded in the database.

  In the case of the search types KW1, KW2, KG1 to KG4 (six types), the C.I. In the case of search types KW3, KG5 to KG7 (four types) to the search frequency data (time adjusted) in the evaluation information database 13, the C.I. It is recorded in the pair search frequency data (time adjusted) in the evaluation information database 13.

  Next, the threshold value (truncated) value: search frequency in the program memory 102 is read (step S1206).

  Finally, a threshold (cut) value: a search output image candidate lower than the search frequency is cut (deleted) (step S1207).

  This is to delete all of the search output image candidates (ID) in the cache memory 103 arranged in the order of the search frequency, the threshold (truncated) value: those less than the search frequency.

  Finally, all the remaining search output image candidates proceed to the search output image determination process shown in FIG. If there are no search output image candidates remaining, the search process is terminated, and the image search viewer is notified that the search process has been completed or there is no search image on the image search screen shown in FIG.

  Here, in order to calculate the search output priority of the search output image candidate in step S1303 of the search output image determination processing flow shown in FIG. 33, the steps shown in the processing flow related to acquisition and calculation of target data for search output image determination shown in FIG. The search frequency as a key image of the search output image candidate in S123 is set.

[Time adjustment processing]
Hereinafter, with reference to the time adjustment process flowchart shown in FIG. 36, the time adjustment process in the pair image posting search device of the present embodiment will be described. The flow chart of FIG. 36 corresponds to the process of step S114 of FIG. 24, and corresponds to the search frequency time adjustment process in step S1204 of the search frequency calculation process flow chart shown in FIG. 32, and is shown in detail. Is.

  The time adjustment process is executed by the search frequency calculation unit 25 of the central processing control apparatus 101 shown in FIG.

  First, the current time is set to a variable (t0) (step S1501).

  Next, among the time adjustment parameters, the variable (TERM) of the time adjustment period is read, and the time adjustment target period (t0, tn (t0−TERM)) is set (step S1502).

  This is because the C.I. (TERM) of the time adjustment parameters is read from the evaluation information database 13, and the time adjustment period (TERM) is traced back from the current time (t0) to the start time (tn) of the time adjustment target period (t0− TERM) is set.

  Next, the C.I. Of the time adjustment parameters in the evaluation information database 13, the two variables (WEIGHT (Wt0)) and (WEIGHT (Wtn)) of the current time and the weight at the start of the target period are read (step S1503).

  Next, all the variables calculated in steps S1501 to S1503 and the weight (Wt-procX) of the processing time (t-procX) of (procX) which is the processing X are the calculation formulas ( (Wt0−Wtn) / tn) (tn−t−procX) + Wtn) is calculated and temporarily written in the cache memory 103 (step S1504).

  Next, the B. A search processing row as a key image in the time adjustment target period (t0-tn) is read from the processing result database 12, and preparations for searching using the processing time as a key are made (step S1505).

  Finally, the search frequency based on the processing time of all search output image candidates searched in step S1505 is calculated, and then the search frequency is adjusted in time (step S1506).

[Conceptual diagram of time adjustment parameters]
Hereinafter, the contents of the time adjustment parameter in the pair image posting search device of the present embodiment will be described with reference to the conceptual diagram of the time adjustment parameter shown in FIG.

  If the current time is (t0) and the time adjustment period is (TERM), the time adjustment target period is (t0, tn (t0-TERM)).

  The weighting ratio between the start time of the target period, that is, (tn) and the end time, that is, (t0) is set to (Wtn) and (Wt0), respectively.

  In the pair image posting search apparatus of this embodiment, the evaluation of the search frequency is weighted and weighted as the latest one.

  As described above, the weight (WtprocX) of the target time point (tprocX) of (procX) that is the process X is calculated by ((Wt0−Wtn) / tn) (tn−t−procX) + Wtn).

  The time adjustment of the search frequency is based on the past search processing of each search output image candidate, that is, all searches X ((Σ (weight of search time X (Wt-procX)) × number of search processes at time of search X (t− procX))) is calculated. (Σ) represents the total.

[Search output image decision processing]
Hereinafter, with reference to the search output image determination process flowchart shown in FIG. 33, the search output image determination process in the pair image posting search apparatus of this embodiment will be described. Note that the flowchart of FIG. 33 corresponds to the process of step S115 of FIG.

  The search output image determination process is divided into two steps: a search output priority calculation data reading process for search output image candidates and a search output priority calculation process.

  Of the search output priority calculation target data, in step S121 in the flowchart of the search output image determination process in FIG. 25, the meta tag similarity of the search output image candidate (meta tag similarity as the first and second ranked image type in the search priority order). 25) in step S122 of FIG. 25, the search frequency as the key image of the search output image candidate (search frequency as the first and second-ranked image type in the search priority order), and the search output in step S123 of FIG. The search frequency of the image candidate as the search output image (search frequency as the first and second-ranked image type in the search priority order) is the search output image candidate determination process shown in FIG. 29 up to the previous step, and the meta tag similarity shown in FIG. It has been written in the cache memory 103 through the calculation process and the search frequency calculation process shown in FIG.

  In the reading process of search output priority calculation data of the former search output image candidate, these search output priority calculation target data written in the cache memory 103 are read, and in the latter search output priority calculation process. Further, C.I. Weights related to the three evaluations constituting the search output priority calculation formulas constituting the search output priority calculation formulas that are various parameters, which are other search output priority calculation target data in the evaluation information database 13, Read the weight data of the evaluation based on the first and second-ranked image types in the search priority order of the indicated search type, combine them, calculate the search output priority of all search output image candidates, determine the search output image, It is output and displayed on the image search screen shown in FIG.

  In the search output image determination processing flowchart shown in FIG. 33, first, the search output priority calculation target data is read from the cache memory 103 (step S1301).

  Next, the C.I. A search output priority calculation formula is read from the evaluation information database 13 (step S1302).

  Next, using all the search output priority calculation target data shown in FIG. 40 read in the previous step S1301 and step S1302 and the search output priority calculation formula described above, search output of all search output image candidates. The priority is calculated (step S1303).

  Next, the search output priority and search output order calculated by the search output priority calculation are recorded in association with the search output image candidate ID (step S1304).

  Next, the search output priority and search output order of all search output image candidates are written in the cache memory 103 (step S1305).

  Next, the search output priorities of all search output image candidates are set to C.I. It is recorded as search output priority data in the evaluation information database 13 (step S1306).

  Next, the threshold (truncated) value: the number of search outputs in the program memory 102 is read (step S1307).

  Next, the threshold (cut) value: the search output image candidates exceeding the search output number are cut (deleted) (step S1308). A list of all search output image candidates (ID) in the cache memory 103 arranged in the search output order is deleted from the threshold value (truncated) value: the order in which 1 is added to the number of search outputs. If there are no search output image candidates remaining at this time, the search process is terminated and the image search viewer is notified that there is no search output image on the image search screen shown in FIG.

  Finally, the search output image to be output is determined (step S1309), and the search output image is output and displayed in the search image output area shown at the bottom of the image search screen shown in FIG. 7 based on the rank of the search output priority (step S1309). S1310). Specifically, the image IDs of the search output image candidates rearranged in the order of high search output priority are read in the order of high search output priority. From the image IDs recorded in the image database 11, these related A.D. The image data, image profile data, and image meta tag recorded in the image database 11 are read out, and output and displayed from the top in the descending order of search output priority in the search image output area shown in FIG.

[Search output priority calculation processing]
The search output priority calculation processing in the pair image posting search device of this embodiment will be described below with reference to the search output priority calculation processing flowchart shown in FIG. Note that the flowchart of FIG. 38 corresponds to the process of step S124 of FIG.

  This process shows the search output priority calculation process in step S1303 in detail in the search output image determination process flowchart shown in FIG.

  This search output priority calculation process is executed by the search output priority calculation unit 26 of the central processing control apparatus 101.

  The search output priority calculation process includes a list of search output image candidate IDs, meta tag similarity data of search output image candidates, search frequency data (time adjusted), and search output priorities. Prepared ones are prepared (step S1601).

  Next, the search output priority calculation target data is read from the cache memory 103 and written to the list (step S1602).

  Next, the C.I. A search output priority calculation formula is read from the evaluation information database 13 (step S1603).

  Next, search output priority calculation is performed (step S1604).

  The search output priority calculation target data shown in FIG. 39 for all search output image candidate IDs is taken into the search output priority calculation formula described above, and the search output priority is calculated.

  Next, the search output priorities of all the search output image candidates calculated are written in the variable string of search output priorities on the list created in step S1601 (step S1605).

  Finally, the four variable string data relating to all search output image candidates are rearranged from the top in the descending order of search output priority on the list (step S1606).

  Thereafter, the search output image obtained by searching as described above is displayed on the screen display device 105 (see FIG. 1). Hereinafter, this output process will be described.

  Referring to the search output priority of search output image candidate IDs rearranged from the top in the descending order of search output priority on the list, search output priority is given to information other than images and images on the image search screen shown in FIG. Output and display in order.

[Output format on image search screen]
The output form on the image search screen employed by the pair image posting search apparatus of the present embodiment is A1. Single image search KW1. Problem image search or A1. Single image search KW2. In the case of the solution image search, the search output images are displayed in order from the first place in the search output priority order under the problem or solution image tray on the image search screen.

  Keyword search A2. Pair (problem / solution) image search KW3. In the case of pair (problem / solution) image search, search output images are displayed in order from the first place in the search output priority order below the problem and solution image tray on the image search screen.

  Key image search B1. KG1 solution image search (K (problem) → T (solution)) of heterogeneous search of single image search, or B1. In the case of KG2 problem image search (K (solution) → T (problem)) of heterogeneous search of single image search, search on the image tray on the image search screen which is one of the knowledge to be searched or on the problem image tray The first search output image in the output priority order is displayed, and the second and subsequent search output images in the search output priority order are displayed below the problem or solution image tray.

  Key image search B1. KG3 problem image search (K (problem) → T (problem)) of similar search of single image search or B1. In the case of KG4 resolution image search (K (resolution) → T (resolution)) of the same type of single image search, search output priority is given below the problem on the image search screen or on the image tray of the resolution Search output images are displayed in order from the first place.

  B2. Key image search. KG5 pair image search for single key image search for pair image search (K (problem) → T (problem / solution)) or B2. In the case of KG6 pair image search (K (solution) → T (problem / solution)) of single key image search of pair image search, the search output priority order is displayed below the problem on the image search screen or on the image tray of the solution. Search output images are displayed in order from the first.

  B2. Key image search. Pair image search KG5 pair image search (K (problem) → T (problem / solution)) or B2. In the case of KG6 pair image search (K (solution) → T (problem / solution)) of single key image search of pair image search, the search output priority order is displayed below the problem on the image search screen or on the image tray of the solution. Search output images are displayed in order from the first.

  B2. Key image search. In the case of KG7 Pair Image Search (K (Problem / Solution) → T (Problem / Solution)) for Pair Key Image Search for Pair Image Search, the search output priority order is below the problem and solution image tray on the image search screen. Search output images are displayed in order from the first place.

[Output processing]
In the output process, the link destination information for accessing various data of the corresponding image is duplicated so as to be displayed on the image search screen, and this data is output to the area based on the output form on the image search screen. indicate. This link destination is A.D. of the storage device 107 shown in FIG. The image ID of the image database 11 is the head, and the image data and image profile data (there are post image type, image name, and image comment), and are displayed at the corresponding locations on the image search screen shown in FIG. .

  As a modification of the present embodiment, the pair image posting search device may be configured on a single stand-alone hardware as shown in FIG. It may be configured on hardware. Moreover, you may implement | achieve on the existing information system.

  As a further modification, in the pair image posting search device of the above-described embodiment, in various operations on the image posting screen shown in FIG. 6, the image search screen shown in FIG. 7, and the image search standby screen shown in FIG. As a terminal mainly composed of a keyboard and mouse shown in one input device 104 and a monitor that is a screen display device 105, an object displayed on a panel such as a mobile phone, a mobile terminal, and a pad-type terminal is directly pointed. Some models can be operated directly by touching and moving. In these models, an operation such as directly operating an image with a finger or pressing a search button may be performed.

  Furthermore, in the image posting tray shown in the image posting screen shown in FIG. 6, in addition to posting an image to be posted on either the problem or the solution, a pair of the image of the problem and the image of the solution is installed at the same time. The ability to post is also applicable. In this case, immediately after posting, the pair image ID is distributed to the combination of these two images, and the posting result is reflected in the search output priority as the pair image as the key image search frequency. May be.

  The relationship between the problem and the solution at the time of posting by the problem image and the posted image image tray shown in FIG. 6 and at the time of searching by the problem image and the key image tray of the image search screen shown in FIG. Ns (needs & seeds), "cause and result", cr (cause & result), "input and output", io (input & output) output), “raw materials and completion”, mf (material & finish), “reason and conclusion claim”, rc (reason & consolidation), “before and after improvement”, ba (before & after), pc (pros & cons), “for and against”, “ Gb (good & bad), "vs (versus)", "north and south", "west and east", ns (north & south), we (west & east) , “Old and present”, “present and future”, bp (backward & present), pf (present & future), “pair”, pr (pair), etc., and additional relationship such as pr (pair) In the search output priority calculation formula, the search frequency results based on the option specification are added as the search frequency results as key images and search output images. Anyway.

  Furthermore, if a keyword is entered in the keyword entry field shown in the image search screen of FIG. 7 and a key image is set in either or both of the problem and solution on the key image tray, an error is returned and processing is performed. A method of processing as key image processing by interrupting or combining processing of a keyword meta tag and a key image meta tag is also applicable. In the pair image posting search device of the above-described embodiment, an error is returned and the process is interrupted.

  The image meta tag is indexed by the image contributor from the meta information input at the time of image posting, but the image name specified by the image contributor and the comment input by the image contributor and the image search viewer And the like can be added as meta information of an image and captured as a meta tag.

  In this case, in the meta tag similarity evaluation (similarity to the key image based on the current meta tag), the meta information and the meta tag change due to the addition of a comment to be added after the posting, and the type of instruction search is similar to the search frequency evaluation. By weighting and calculating the results as the first and second image types in the search priority order, it is possible to realize a similarity evaluation reflecting a change in meta tags depending on the image type of problem or solution, and a search function based on this.

  Furthermore, in the calculation of the meta tag similarity, it is assumed that not only the number of common elements of the meta tag on the key side and the search output side, but also meta tags of all other search output image candidates for each meta tag in advance. It is also possible to apply a method that places importance on the scarcity of the characters.

  As the search results, when the image is a moving image, it is also possible to add a result of explicit processing by the user after the moving image search such as the number of reproductions and the number of downloads of the search output moving image.

  When reflecting the image type designation intention for the posted image by the image contributor, it is also possible to heavily evaluate the evaluation as the search output image based on the image type given at the time of posting. Specifically, with respect to the search frequency evaluation as the search output image, the evaluation item in either the first or second search priority order of the instruction search type in which the post award-giving image type as the post image matches. It is to evaluate heavily.

  In the case of KG5 and KG6 for single key image (problem image and solution image) search among the pair image search, it is assumed that the image search viewer needs knowledge of both the problem and the solution. In the first place, the problem image and the solution image are equally second. In this search priority order, the pair image may be ranked first, and either the problem image or the solution image used as a key may be ranked second and the other image ranked third.

  The function of the time adjustment processing is proportional calculation, but it is possible to change to a function such as using a quadratic function to evaluate more favorably as recently as possible, or to lower the one before a certain period in general. .

  For the search output priority calculation criteria, for example, the accuracy of the search output result is evaluated based on an index such as how much it has been reused as a key image, and each weight set in the criteria is automatically adjusted. In addition, a method of automatic control by a soft computing approach using various artificial intelligence techniques and the like can be applied.

  In addition, performance data such as meta tag similarity and search frequency included in the search output priority calculation criteria are included only for the first and second search priority orders. In a situation where the actual result data of the output image candidates is not accumulated, it is possible to apply the use of result data other than the first and second search priority orders.

  FIG. 41 schematically shows a hardware configuration of another embodiment of the pair image posting search device of the present invention. This embodiment is a case of a paired image posting search apparatus including a client terminal 201 and a server apparatus 202 connected to each other via a local area network (LAN).

  The client terminal 201 is installed with client software 203 of a pair image posting search program. The server device 202 is a database server application that stores various data such as images and meta tags, and is used for display, playback, and search of images. An image display / playback server application and a pair image posting search program are installed. The client terminal 201 is provided with services from the server device 202 via the LAN.

  When the pair image posting search program of the server device 202 is a web application, the web server 204 is installed on the server device, and the web browser 205 is installed on the client terminal 201.

  FIG. 42 schematically shows a hardware configuration of still another embodiment of the pair image posting search device of the present invention. The present embodiment is a case of a pair image posting search device including a client terminal 211 and a server device 212 that are connected to each other via the Internet.

  The client terminal 211 is installed with client software 213 of a pair image posting search program. The server device 212 is a database server application that stores various data such as images and meta tags, and is used for display, reproduction, and search of images. An image display / playback server application and a pair image posting search program are installed. The client terminal 211 is provided with services from the server device 212 via the Internet.

  When the pair image posting search program of the server device 212 is a web application, the web server 214 is installed on the server device, and the web browser 215 is installed on the client terminal 211.

  As mentioned above, although this invention was described by specific embodiment, it is understood that the description and drawing which form a part of this indication do not limit this invention, and include various embodiment etc. which are not described here. Should. Therefore, the technical scope of the present invention is defined only by the invention specific matters according to the claims. Various modifications and operational techniques will become apparent from the disclosure of the present invention, and these will also be defined by the invention-specific matters according to the claims.

DESCRIPTION OF SYMBOLS 11 Image database 12 Processing performance database 13 Evaluation information database 21 Image contribution part 22 Image search part 23 Evaluation part 24 Meta tag similarity calculation part 25 Search frequency calculation part 26 Search output priority calculation part 27 Search output part 101 Central processing control apparatus 102 Program memory 103 Cache memory 104 Input device 105 Screen display device 106 Communication control device 107 Storage device 108 Image capture peripheral device 109 Input / output interface 110 Bus 201, 211 Client terminal 202, 212 Server device 203, 213 Client software 204, 214 Web Server 205, 215 Web browser

Claims (12)

A pair image posting search device comprising a processing control device and a storage device connected to the processing control device,
The storage device includes an image database that records at least image data and meta information,
The processing control device, image data and meta information of a single image related to knowledge of a problem input for posting, image data and meta information of a single image related to knowledge of a solution input for posting, Or, a post processing means for recording image data and meta information of a single image related to the problem and knowledge of solving a pair image input for posting and searched after posting to the image database;
Recorded in the image database based on the key image input for the search and the instruction search type for searching for a single image related to problem knowledge, a single image related to solution knowledge, or a pair image related to problem and solution knowledge Search processing means for searching for a search output image from the image data and meta-information that is being processed,
A pair image posting retrieval apparatus comprising:   From the image data and meta information recorded in the image database, the search processing means is based on the keyword input for the search and an instruction search type for searching for a pair image related to the problem and solution knowledge. The pair image posting retrieval apparatus according to claim 1, further comprising means for retrieving a retrieval output image. The storage device further includes an evaluation information database that records at least evaluation information in the posting and search processing,
The post processing means is input meta information of a single image related to knowledge of a problem input for posting, meta information of a single image related to knowledge of solution input for posting, or input for posting It comprises a meta tagging means for decomposing elemental meta information of a single image related to the problem and solution knowledge constituting a pair image searched after posting, and recording the obtained meta tag in the image database,
The search processing means is based on the keyword or key image input for the search, the image data and meta tag recorded in the image database, and the evaluation information recorded in the evaluation information database. Search output image candidate determination means for determining a search output image candidate, a keyword or key image input for the search, an instruction search type input for the search, and the image database Search output image determination means for determining a search output image from the determined search output image candidates based on image data and a meta tag, and evaluation information recorded in the evaluation information database, The pair image posting search device according to claim 1 or 2.   The search output image candidate determining means includes means for reading meta tag similarity data, which is evaluation information recorded in the evaluation information database, and searching for an image having a high meta tag similarity as a search output image candidate. 4. The pair image posting retrieval apparatus according to claim 3,   The search output image candidate determination means includes means for reading search frequency data that is evaluation information recorded in the evaluation information database, and searching for images having a high detection frequency as search output image candidates. The pair image posting search device according to claim 3 or 4.   The search output image determination means includes means for determining a search output image for the searched search output image candidates according to the first and second order image types in the search priority order of the instruction search type. The pair image posting search device according to claim 4 or 5.   The search output image determining means calculates a meta tag similarity and search frequency for the searched search output image candidates, and determines a search output image according to the calculated meta tag similarity and search frequency. The pair image posting retrieval apparatus according to claim 5, wherein the pair image posting retrieval apparatus includes:   The search processing means performs a single search output image search (KW1) related to solution knowledge based on the keyword related to problem knowledge input for the search, and relates to problem knowledge based on the keyword related to solution knowledge. A means for performing a single search output image search (KW2) or performing a pair search output image search (KW3) related to problem and solution knowledge based on a pair keyword related to problem and solution knowledge is provided. The pair image posting retrieval apparatus according to claim 1, wherein the pair image posting retrieval apparatus is a pair image posting retrieval apparatus.   The search processing means performs a single search output image search (KG1) related to problem knowledge based on a key image related to solution knowledge input for search, and a key related to problem knowledge input for search A single search output image search (KG2) related to the knowledge of the solution based on the image, and a single search output image search (KG3) related to the knowledge of the problem based on the key image related to the knowledge of the problem input for the search. Perform a single search output image search (KG4) related to solution knowledge based on the key image related to solution knowledge input for search, and perform paired keys related to the problem and solution knowledge input for search Search related to problem or solution knowledge based on image Output image search (KG5) Pair for problem and solution knowledge input for search Search for a pair related to a problem or solution knowledge based on the key image Output image search (KG6), or a problem and solution knowledge based on a pair key image related to the problem and solution knowledge input for the search The pair image posting search device according to any one of claims 1 to 7, further comprising means for performing a pair search output image search (KG7). Computer
A pair image posting search device comprising a processing control device and a storage device connected to the processing control device,
The storage device includes an image database that records at least image data and meta information,
The processing control device, image data and meta information of a single image related to knowledge of a problem input for posting, image data and meta information of a single image related to knowledge of a solution input for posting, Alternatively, a post processing means for recording image data and meta information of a single image related to the problem and knowledge of solving a pair image input for posting and searched after posting to the image database, and for searching Is recorded in the image database on the basis of the key image input to the image and the instruction retrieval type for retrieving a single image related to problem knowledge, a single image related to solution knowledge, or a pair image related to problem and solution knowledge. A pair for functioning as a pair image posting retrieval device comprising retrieval processing means for retrieving a retrieval output image from image data and meta information Image post search program.   From the image data and meta information recorded in the image database, the search processing means is based on the keyword input for the search and an instruction search type for searching for a pair image related to the problem and solution knowledge. The pair image posting search program according to claim 10, further comprising means for searching for a search output image. The storage device further includes an evaluation information database that records at least evaluation information in the posting and search processing,
The post processing means is input meta information of a single image related to knowledge of a problem input for posting, meta information of a single image related to knowledge of solution input for posting, or input for posting It comprises a meta tagging means for decomposing elemental meta information of a single image related to the problem and solution knowledge constituting a pair image searched after posting, and recording the obtained meta tag in the image database,
The search processing means is based on the keyword or key image input for the search, the image data and meta tag recorded in the image database, and the evaluation information recorded in the evaluation information database. Search output image candidate determination means for determining a search output image candidate, a keyword or key image input for the search, an instruction search type input for the search, and the image database A pair image posting search comprising search output image determination means for determining a search output image from the determined search output image candidate based on image data and a meta tag, and evaluation information recorded in the evaluation information database The pair image posting search program according to claim 10 or 11, for causing the device to function as a device.

Images