JP2024046323A - Information processing program, information processing device, and information processing method - Google Patents

Abstract

[Problem] To improve the efficiency of file searches. [Solution] According to an embodiment, an information processing program is an information processing program that can be installed in a computer having a storage unit and a processor. This information processing program includes instructions for causing the processor to execute a recognition process, a reception process, an extraction process, and a presentation process. The recognition process is a process of accessing a database that accumulates a plurality of files including text and images, and calculating in advance a recognition score of an image included in the file. The reception process is a process of receiving a specification of a search key image. The extraction process is a process of extracting files including an image whose similarity to the search key image meets a predetermined standard based on the recognition score from the database. The presentation process is a process of presenting a list of the extracted files. [Selected Figure] Figure 2

Description

Embodiments of the present invention relate to an information processing program, an information processing device, and an information processing method.

Today, when all kinds of information assets have been organized into databases, search systems are an essential technology for obtaining the information required. For example, to find a template document file (document) for creating a report, a keyword search may be performed in a document file management system. Web pages for searching for images that match keywords are also known.

Patent No. 4302799 Patent No. 6314071 Patent No. 6712796

Existing search techniques using text-based keywords tend to produce a huge amount of output, and it is difficult to narrow down the search results. In particular, opening the searched files one by one and checking the contents is not efficient and takes a long time.
Therefore, an object of the present invention is to provide an information processing program, an information processing device, and an information processing method that improve file search efficiency.

According to an embodiment, the information processing program is an information processing program that can be installed in a computer having a storage unit and a processor. This information processing program includes instructions that cause the processor to execute a recognition process, a reception process, an extraction process, and a presentation process. The recognition process is a process of accessing a database that accumulates multiple files including text and images, and calculating in advance a recognition score of the image included in the file. The reception process is a process of receiving a specification of a search key image. The extraction process is a process of extracting files including an image whose similarity to the search key image meets a predetermined standard from the database based on the recognition score. The presentation process is a process of presenting a list of the extracted files.

FIG. 1 is a functional block diagram showing an example of a computer 10 according to an embodiment. FIG. 2 is a functional block diagram showing an example of the processor 11 and the storage 14. As shown in FIG. FIG. 3 is a diagram showing an example of a document file registered in the document database 14b. FIG. 4 is a flowchart showing an example of a processing procedure of the computer 10. FIG. 5 is a diagram showing an example of a GUI screen for designating a search key image. FIG. 6 is a flow chart showing an example of a processing operation of the computer 10 relating to document search. FIG. 7 is a diagram showing an example of a search result presented to the user.

Embodiments will be described with reference to the drawings. In the embodiment, a user operates the computer 10 to specify information (search key) for searching a document file. In the embodiment, an image file is given as a search key. The image as a search key (search key image) may be one stored in advance in the computer 10, or may be selected from images stored in a storage medium such as a USB memory, or may be an image acquired via a network. .

When a search key image is specified, computer 10 extracts document files that contain images similar to the search key image from the database, lists them, and presents them to the user. Details are disclosed below.

<Configuration>
FIG. 1 is a functional block diagram showing an example of a computer 10 according to an embodiment. The computer 10 includes a processor 11 , a ROM 12 , a RAM 13 , a storage 14 , a communication section 15 , an operation section 16 , and a display section 17 . These are interconnected via bus lines within the computer 10 and are capable of exchanging data.

The processor 11 controls the overall operation of the computer 10. The processor 11 also loads programs installed in the ROM 12 or storage 14 into the RAM 13 and executes them to realize various processes related to the embodiment.

ROM 12 is a non-volatile memory that prestores control programs, control data, etc. The control programs, control data, etc. stored in ROM 12 are first read and expanded in RAM 13, for example, when computer 10 is started.

RAM 13 is a volatile memory. RAM 13 temporarily stores data being processed by processor 11. RAM 13 stores various application programs based on instructions from processor 11. Data required to execute application programs and execution results of application programs are also stored in RAM 13.

The storage 14 is a non-volatile memory to which data can be written and rewritten. The storage 14 is composed of, for example, a hard disk drive (HDD), a solid state drive (SSD), or a flash memory. Basic programs, applications, and various data of the computer 10 are installed in the storage 14.

The communication unit 15 is an interface that supports, for example, wired or wireless LAN (Local Area Network) connection and communicates with other hosts via the network. The computer 10 can communicate with, for example, a host computer on the Internet via the communication unit 15.

The operation unit 16 is a user interface that includes a mouse, a keyboard, and the like, and accepts various operation inputs from the user. The operation unit 16 transmits a signal indicating the input operation to the processor 11.
The display unit 17 is, for example, a liquid crystal monitor, and visually displays various information. When the operation section 16 is composed of a touch panel, the display section 17 is formed integrally with the touch panel as the operation section 16.

FIG. 2 is a functional block diagram showing an example of the processor 11 and the storage 14. In FIG. 2, the processor 11 has a recognition function 11a, a reception function 11b, an extraction function 11c, and a presentation function 11d as processing functions according to the embodiment. The storage 14 also stores a program 14a, a document database 14b, an image recognition model 14c, and a recognition score 14d.

The program 14a is a program for information processing, and includes instructions for causing the processor 11 to execute a recognition function 11a, a reception function 11b, an extraction function 11c, and a presentation function 11d.
The recognition function 11a accesses the document database 14b and pre-calculates recognition scores for images contained in files stored in the document database 14b.
The reception function 11b receives a search key image designated by the user.
The extraction function 11c extracts files including images whose similarity to the search key image satisfies a predetermined criterion from the document database 14b, based on the recognition score.
The presentation function 11d presents a list of the extracted files on the display unit 17 of the computer 10.

The document database 14b of the storage 14 accumulates multiple document files to be searched. In the embodiment, the document files to be searched include document files that contain text and images, such as reports.

Figure 3 is a diagram showing an example of a document file registered in the document database 14b. Files to be searched include files generated by word processing software (such as Word files), files generated by presentation creation software (such as PowerPoint files), and files generated by printing software (Portable Document Format (PDF) files). Files of this type of format can embed not only text data but also images. For example, a word_001 file contains an image of a building, and a word_002 file contains an image of a collection truck. Similarly, there are files that contain images of airplanes, passenger cars, and the like. In the embodiment, a recognition score is calculated in advance for images contained in various document files stored in the database by image recognition processing using, for example, AI (Artificial Intelligence) technology.

Returning to Figure 2, we will continue the explanation. The image recognition model 14c of the storage 14 in Figure 2 is a model that has been generated in advance by training a neural network in order to calculate the recognition score of an embedded image. For example, a vector whose elements are the RGB pixel values of the pixels that make up the image data (pixel value vector) is input to the input layer of the neural network, and a recognition score is output from the output layer through a calculation based on the weighting coefficients in the hidden layer. The weighting coefficients of the image recognition model 14c converge to a certain value by repeatedly providing training data consisting of sets of pixel value vectors and recognition scores.

The processor 11 stores the calculated recognition score in the storage (recognition score 14d). The processor 11 also generates attribute information that associates the identification information (file name, identifier, etc.) of the file containing the recognized image with the recognition score of the image, and stores the attribute information in the storage 14.

<Action>
Next, the operation of the above configuration will be described.
Fig. 4 is a flowchart showing an example of a processing procedure of the computer 10. Fig. 4 shows an example of a processing procedure when the computer 10 calculates a recognition score.

In FIG. 4, the processor 11 of the computer 10 acquires one search target file from the document database 14b (step S11). Next, the processor 11 acquires an image included in the acquired search target file (step S12).

After retrieving the image, the processor 11 calculates a recognition score for the image (step S13). Next, the processor 11 reduces the size of the image and converts it into a thumbnail (step S14).

When the thumbnail is generated, the processor 11 associates the recognition score and the thumbnail with the identification information of the search target file and adds them to the attribute information (step S15). Here, if the storage 14 does not store the attribute information, the processor 11 generates the attribute information.

Next, the processor 11 determines whether there is a search target file for which an image recognition score has not been calculated (step S16). If there is a search target file for which a recognition score has not been calculated (YES in step S16), the processing procedure returns to step S11. If the recognition scores of the images included in all the search target files have been calculated (NO in step S16), the processing procedure ends.

Figure 5 is a diagram showing an example of a GUI (Graphical User Interface) displayed on the display unit 17 of the computer 10. Using such a GUI, the user can specify a search key image. As shown in Figure 5, for example, the search key image can be provided to the system by dragging and dropping the search key image into a specified area on the browser.

FIG. 6 is a flowchart showing an example of the processing procedure of the computer 10 related to document search. In the embodiment, a search service using a so-called request-response Web application will be described. In the stand-alone computer 10, this type of service is provided by specifying and accessing, for example, port 80 of the local host.

In FIG. 6, the processor 11 of the computer 10 obtains a request including a search key image specified by the user (step S21). Upon receiving the request, the processor 11 extracts a document file containing an image similar to the search key image included in the request from the document database 14b (step S22).

Here, the processor 11 separately calculates the recognition score of the search key image, and extracts document files that include images with a recognition score similar to that value (scalar or vector). The similarity between the embedded image and the search key image is determined based on a predetermined threshold, and document files that show a match equal to or greater than the threshold are picked up.

When the document file is extracted, the processor 11 acquires the file name and thumbnail image of the searched file from the attribute information, and creates list data in, for example, HTML or XML format (step S23). After generating the list data, the processor 11 transmits a response including this list data to the access source (step S24), and the processing procedure ends.

Figure 7 shows an example of a search result presented to the user. In response to the specification of a search key image of a vehicle in the GUI of Figure 5, files containing images of the vehicle are extracted from the document database 14b. The extracted files are sorted, for example, in descending order of the degree of match (degree of agreement) between the embedded image and the search key image, and are displayed in a list on the display unit 17 of the computer 10 together with the file name and thumbnail image.

＜Effects＞
As described above, in the embodiment, images embedded (published) in document files (such as Word, PowerPoint, PDF, etc.) stored in a database are analyzed/evaluated, and a recognition score is calculated in advance. Then, the similarity to a search key image designated by a user is evaluated, and document files containing images with high similarity are extracted. Furthermore, the file names of the extracted files are listed together with thumbnails of the embedded images, and are displayed as a list.

That is, according to the embodiment, unlike a web service that searches for images using a character string as a key, it is possible to use an image as a search key to extract document files that include images similar to the search key.

Therefore, according to the embodiment, for example, it becomes easy to find documents that may be useful when creating a document. In other words, instead of searching for keywords using character strings, it becomes possible to search for documents in a database using an image at hand as a key. This makes it possible to dramatically improve search efficiency.

For example, by using a current weather chart as a search key and searching a database of reports recording disaster situations, past reports containing similar weather charts can be extracted. The user can refer to the images displayed as thumbnails in the search results and view documents that seem to match their own thinking. Based on the reports from that time, the user can then quickly create documents that predict or warn of future disasters. In addition, because images in document files are displayed as thumbnails, the user does not need to open and check each document file one by one, freeing them from the tedious work.

In addition, the recognition score of the embedded image is calculated by AI using a neural network. This allows the accuracy to improve as the amount of learning increases. In other words, the more document files are stored in the database, the more the search accuracy is expected to improve.

For these reasons, according to the embodiment, it is possible to provide an information processing program, an information processing device, and an information processing method with improved file search efficiency.

Note that the present invention is not limited to the above-described embodiment. For example, the document files registered in the document database 14b may be files composed of information obtained from web pages by web scraping technology or the like. Furthermore, the document files may contain audio and video.

In addition, in this embodiment, the database is built on a standalone computer 10. This makes it possible to ensure the confidentiality of the information. On the other hand, if publicly available document files are to be handled, it is also possible to build a system in a client-server format by accessing a database built on a server other than the computer 10 via a network.

Although an embodiment has been described, this embodiment is presented as an example and is not intended to limit the scope of the invention. This new embodiment can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the gist of the invention. This embodiment and its modifications are included within the scope and gist of the invention, and are included in the scope of the invention and its equivalents described in the claims.

10...Computer 11...Processor 11a...Recognition function 11b...Reception function 11c...Extraction function 11d...Presentation function 12...ROM 13...RAM 14...Storage 14a...Program 14b...Document database 14c...Image recognition model 14d...Recognition score 15...Communication unit 16...Operation unit 17...Display unit.

Claims (5)

An information processing program installable on a computer comprising a storage unit and a processor,
the processor;
recognition processing that accesses a database that stores a plurality of files including text and images and pre-calculates a recognition score for an image included in the file;
A reception process that accepts the specification of a search key image,
an extraction process of extracting a file containing an image whose similarity with the search key image satisfies a predetermined criterion from the database based on the recognition score;
An information processing program including an instruction for executing a presentation process of presenting a list of the extracted files. The information processing program according to claim 1, wherein the presentation process presents thumbnails of images contained in the extracted files in the list. The information processing program according to claim 1, wherein the recognition process includes providing the image to an image recognition model generated by training a neural network, and outputting the recognition score. a storage unit that stores a database that stores a plurality of files including text and images;
Armed with a processor,
The processor includes:
a recognition function that calculates in advance a recognition score of an image included in the file;
A reception function that accepts the specification of a search key image,
an extraction function that extracts files containing images whose similarity to the search key image satisfies a predetermined criterion from the database based on the recognition score;
An information processing device comprising: a presentation function that presents a list of the extracted files. An information processing method by a computer having a storage unit and a processor,
The processor,
storing a database in the storage unit, the database storing a plurality of files including text and images;
pre-calculating recognition scores for images contained in said file;
Accepting a designation of a search key image;
extracting from the database, based on the recognition score, a file including an image whose similarity to the search key image satisfies a predetermined criterion;
and presenting a list of the extracted files.