WO2014119278A1 - Content retrieval device, content retrieval method, and content retrieval program - Google Patents

Abstract

A content retrieval device (10) is provided with: a retrieval icon DB (36) which stores a plurality of retrieval icons each for designating an object to be retrieved; a display control unit (CPU (11)) which causes a display unit (16) to display at least one retrieval icon among the plurality of retrieval icons stored in the retrieval icon DB (36), a condition designation object (a first condition setting unit (T2), a second condition setting unit (T3)) for designating a retrieval condition for an object to be retrieved designated by the at least one retrieval icon, and a retrieval condition disposition region (a retrieval condition setting unit (T1)) that is a region in which the retrieval icon and the condition designation object can be disposed; and a retrieval unit (CPU (11)) which executes retrieval of a content stored in a content storage unit on the basis of the disposition relationship between the retrieval icon and the condition designation object that are disposed in the retrieval condition disposition region, the object to be retrieved designated by the retrieval icon disposed in the retrieval condition disposition region, and the retrieval condition designated by the condition designation object disposed in the retrieval condition disposition region.

Description

Content search device, content search method, and content search program

The present invention relates to a content search device, a content search method, and a content search program for searching for content such as images existing on a computer or a network.

For example, there are devices described in Patent Document 1 and Patent Document 2 as content search apparatuses that search for content such as images existing on a computer or a network.

JP 2008-158680 A JP 2012-234666 A

When searching for content such as images, there is a problem that it takes time to input search conditions.

The present invention provides a content search device, a content search method, and a content search program that can facilitate content search.

A content search device according to the present invention is a content search device for searching for content,
An icon storage unit for storing a plurality of search icons for designating search targets;
At least one search icon among a plurality of search icons stored in the icon storage unit, a condition specifying object for specifying a search condition for a search target specified by at least one search icon, a search icon and a condition specifying object A display control unit that displays a search condition arrangement area, which is an area that can be arranged, on the display unit;
The relationship between the search icons and condition-specified objects placed in the search condition placement area, the search target specified by the search icon placed in the search condition placement area, and the condition designation placed in the search condition placement area A search unit that executes a search of content stored in the content storage unit based on a search condition specified by the object;
Is provided.

A content search method according to the present invention is a content search method for searching for content by a computer including an icon storage unit, a calculation unit, and a display unit,
A storage step of storing a plurality of search icons for designating search objects in the icon storage unit;
The calculation unit includes at least one search icon among a plurality of search icons stored in the icon storage unit, a condition designation object that designates a search condition for a search target designated by the at least one search icon, a search icon, A display step for displaying a search condition arrangement area, which is an area in which the condition designation object can be arranged, on the display unit;
Arithmetic unit places the search icons and condition specification objects placed in the search condition placement area, the search target specified by the search icon placed in the search condition placement area, and the search condition placement area A search step for performing a search of content stored in the content storage unit based on a search condition specified by the specified condition specifying object;
including.

The content search program according to the present invention causes a computer to function as means for executing each step of the content search method.

According to the present invention, it is possible to search for contents only by arranging a search icon or a condition specifying object in the search condition arrangement area instead of inputting characters. Therefore, content search can be facilitated. In addition, it is possible to execute a search by combining the conditions of a plurality of icons in a complex manner.

It is a block diagram which shows the hardware constitutions of the content search apparatus which concerns on this embodiment. It is a block diagram which shows the structure of the content search system containing the content search apparatus which concerns on this embodiment. It is a figure which shows the operation screen of the content search which concerns on this embodiment. It is a figure which shows the operation screen of the content search which concerns on this embodiment. It is a figure which shows the operation screen of the content search which concerns on this embodiment. It is a figure explaining the component of a search icon. It is a figure which shows an example of the description content of a search icon definition file. It is a figure which shows the functional block regarding the search icon and content registration function in the content search apparatus which concerns on this embodiment. It is a figure which shows an example of the search icon management table of search icon DB. It is a figure which shows an example of the search engine parameter (post-processing part) in an image recognition result. It is a figure which shows the structure of content DB and tag score information DB. It is a figure which shows the functional block regarding the search function in the content search device 10 concerning this embodiment. It is a flowchart when registering a search icon from a site on the network. It is a flowchart when downloading a new content to a content search apparatus and registering the search icon. It is a flowchart of the search process by a search icon. It is a block diagram which shows the structure of the content search system containing the content search apparatus which concerns on other embodiment.

Embodiments of the present invention will be described with reference to the drawings.

1. Configuration 1-1. Configuration of Content Search Device FIG. 1 is a block diagram showing a hardware configuration of a content search device according to this embodiment. The content search apparatus 10 according to the present embodiment has a content search program or the like (instructions for an electronic computer, which is an instruction to an electronic computer) on a tablet terminal equipped with, for example, Android (registered trademark) as an OS (Operation System). Combining information so that it can be obtained, and other information that is used for processing by an electronic computer and conforms to a program). The content search apparatus 10 includes a CPU 11, a ROM 12, a RAM 13, a touch panel 14, an external device connection unit 15, a display unit 16, a communication unit 17, a flash memory 18, and a data bus 19.

ROM 12 stores programs and data. For example, the ROM stores an OS, a content search application, and a database (part).

The CPU 11 reads a program and data from the ROM 12 and performs arithmetic processing based on a program command.

The RAM 13 temporarily stores programs and data that are processed by the CPU 11.

The communication unit 17 is an interface for performing communication with an external device. The communication interface includes, for example, a wired LAN interface and a wireless LAN interface.

The display unit 16 displays a video based on the video data generated by the CPU 11. The display unit 16 is configured by, for example, a liquid crystal display or an organic EL display.

The touch panel 14 is a planar pointing device for the user to input various instructions to the content search apparatus 10. The touch panel 14 is provided so as to overlap the display surface of the display unit 16 and outputs a signal related to the position touched by the user.

External device connection unit 15 includes a USB interface, a memory card connector, and the like.

The flash memory 18 stores programs and data, like the ROM 12.

The data bus 19 transmits various data between the above parts.

1-2. Configuration of Content Search System FIG. 2 is a block diagram showing a configuration of a content search system including the content search device 10 according to the present embodiment. The content search device 10 according to the present embodiment includes, for example, a digital LAN 21, a video camera 22, a smartphone 23, a recorder 24, a computer 25, and other terminals that exist inside a home or office, particularly a digital terminal, a wired LAN interface, Connection is possible via a wireless LAN interface, a USB interface, or the like. The content search system can be configured as UPnP (Universal Plug and Play) Control Point and Device. At this time, the content search apparatus 10 can use various standards and protocols such as DLNA (Digital Living Network Alliance), AirPlay (registered trademark), and SMB (Server Message Block). Manage the status of the various devices using these communication means and various standards and protocols, and acquire additional update information of content information such as photos, videos, and audio, and copy and synchronize between multiple devices. Can be done.

In addition, the content search apparatus 10 can be connected to a WEB server 26 of various external websites and a search icon server 27 that stores search icons to be described later via the Internet.

The WEB server 26 stores various website pages.

The search icon server 27 stores a search icon according to the present embodiment.

When the content search device 10 is connected to the various devices 21, 22, 23, 24, 25 and the WEB server 26, the content search device 10 searches the content stored in the WEB server 26 and acquires the content as necessary. be able to. Further, when the content search apparatus 10 is connected to the search icon server 27, the content search apparatus 10 can acquire a search icon stored in the search icon server 27. The search icon server 27 requests authentication from the content search apparatus 10 to be connected. The content search apparatus 10 can connect to the search icon server 27 to upload a search icon or acquire a search icon uploaded by another content search apparatus 10 (another user). Thereby, the search icon which recognized (learned) many images is acquirable. That is, a search icon can be exchanged between users.

1-3. Outline of Operation of Content Search Device An outline of operation of the content search device 10 will be described. FIG. 3A to FIG. 3C are diagrams showing display screens during content search displayed on the display unit 16 of the content search apparatus 10.

∙ A plurality of pull-down tabs are arranged at the top of the search screen. Specifically, a “target” tab TB1, a “face” tab TB2, a “photographing” tab TB3, an “animal” tab TB4, and an “addition” tab TB5 are arranged. When pulling down each tab, a tray containing a plurality of search icons appears.

For example, in the screen shown in FIG. 3A, when the “target” tab TB1 is pulled down, as shown in FIG. 3B, a “target” tray TR1 containing a plurality of search icons for designating content to be searched appears. In this example, the “target” tray TR1 includes, as search icons, for example, a “photo” icon IC1 for searching for content related to a photo, a “video” icon IC2 for searching content related to a video, and content related to audio. A “voice” icon IC3 and the like for searching are accommodated.

In addition, when the “Animal” tab TB4 is pulled down on the screen shown in FIG. 3A, as shown in FIG. 3C, an “Animal” tray TR4 containing a plurality of search icons for searching for content related to a specific animal appears. In this example, in the “animal” tray TR4, for example, a “tiger” icon IC4 for searching for content related to a tiger, a “giraffe” icon IC5 for searching content related to a giraffe, and a content for searching for content related to a cat. “Cat” icon IC6 and the like are accommodated.

In the screen shown in FIG. 3A, when the “face” tab TB2 is pulled down, a “face” tray (not shown) containing a plurality of search icons for searching for content related to a specific person appears. In the “face” tray, for example, a “young” icon for searching content related to young people, a “female” icon for searching for women, a “father” icon for searching content related to the user's father, A “mother” icon for searching content related to the mother, a “friend A” icon for searching content related to the user's friend A, and the like are accommodated.

Also, when the “shooting” tab TB3 is pulled down, a “shooting” tray (not shown) containing a plurality of search icons for searching for content that matches a specific shooting condition appears. The shooting condition tray contains, for example, a “flash shooting” icon for searching for flash-captured content, a “lens focal length” icon for searching content shot at a specific lens focal length, and the like.

In the “add” tab TB5, a new tray for accommodating a search icon in which other search conditions are set can be set.

As shown in FIGS. 3A to 3C, a search condition setting unit T1 (search condition arrangement area) is provided in the approximate center of the search screen. The search condition setting unit T1 includes a first condition setting unit T2 and a second condition setting unit T3 for setting a search logical condition and the like.

The first condition setting unit T2 is a part for setting the first logical condition and the like for the search. In the first condition setting unit T2, for example, characters (search condition objects) indicating logical conditions such as “to”, “ka”, and “kara” are arranged in the vertical direction. The characters “and” indicate “and” in the logical condition. The character “ka” indicates “or” in the logical condition. The letter “from” indicates a range, such as A “from” B. These characters can be scrolled up and down, and a desired character can be selected by scrolling. In this example, the state where the character “ka” is selected is shown.

The second condition setting unit T3 is a part for setting the second logical condition and the like at the time of search. In the second condition setting unit T3, for example, characters (search condition objects) indicating logical conditions such as “include ()” and “do not include ()” are arranged in the vertical direction. The character “does not contain” indicates “NOT” in the logical condition. These characters can be scrolled up and down, and a desired character can be selected by scrolling. In this example, a state where a character “including ()” is selected is shown.

The region R1 on the left side of the first condition setting unit T2 and the region R2 between the first condition setting unit T2 and the second condition setting unit T3 include a “face” tray, a “photographing” tray, or an “animal” tray TR4. The icons included in can be placed. In the region R3 on the right side of the third condition setting unit T3, search icons included in the “target” tray TR1 can be arranged.

FIG. 3A shows an example in which a “young” icon IC7 is arranged in the region R1, a “female” icon IC8 is arranged in the region R2, and a “photograph” icon IC1 is arranged in the region R3. By arranging the search icons in this way, a search condition of “photos including young people or women” is set.

In the search screen, a “search execution” button B1 for executing a search is arranged. For example, in the state shown in FIG. 3A, by touching the “execute search” button B1, the search can be executed with a search condition of “photos including young people or women”.

In the search screen, an “group elements” button B2 for grouping conditions input to the search condition setting unit T1 is arranged. By touching the “group elements” button B2, one search icon for executing a search under the conditions input to the search condition setting unit T1 can be newly generated.

The search icons accommodated in each tray can be arranged in the areas R1 to R3 of the search condition setting unit T1 by performing a predetermined operation such as dragging. For example, from the state shown in FIG. 3A, as shown in FIG. 3C, pull down the “animal” tray TR4 and then drag the “cat” icon IC6 from the “animal” tray TR4 to the region R1 of the search condition setting unit T1. Thus, the “cat” icon IC6 can be arranged in the region R1 of the search condition setting unit T1. Thereby, the state in which the search condition “photos including young people or women” shown in FIG. 3A is set is changed to the state in which the search condition “photos including cats or women” shown in FIG. 3C is set. it can.

In the above, the case where a search is performed using three search icons has been described. However, in the content search apparatus 10 according to the present embodiment, the search condition may be set using more search icons. Alternatively, using only one search icon, it is possible to search only the content conditioned on the search icon.

1-4. Configuration of Search Icon FIG. 4 is a diagram for explaining the components of the search icon.

The search icon referred to in the present embodiment is an aggregate of a plurality of files. The search icon definition file F1 is a definition file, the search icon image file F2 is an image file, the search engine file F3 is a search engine program, and feature storage. This includes four search engine parameter file F4 files. The search icon definition file F1, the search icon image file F2, the search engine file F3, and the search engine parameter file F4 constituting the search icon are stored in the search icon DB as will be described later. In the present embodiment, when the search icon is arranged or dragged on the screen of the display unit 16, the image indicated by the search icon image file F2 is arranged or dragged on the screen of the display unit 16. means.

The search icon definition file F1 is an XML format file. In the search icon definition file F1, various information related to the search icon is described. In this example, an example in which the search icon definition file F1 is an XML format file will be described, but other formats may be used as long as the above-described various information can be described.

FIG. 5 is a diagram showing an example of description contents of the search icon definition file F1.

<ID>, <ICON-IMG>, <Engine>, <Engine-Parameter>, <Tag-Information>, <ICON-Information> are provided as major items in the search icon definition file F1.

<ID> column describes the ID of the search icon (search icon ID).

In the <ICON-IMG> field, the file name (search icon image file name) of the search icon image displayed on the screen of the content search apparatus 10 is described.

In the <Engine> column, information related to the search engine is described. For example, a search engine file name, a search engine, that is, an attribute of a search icon, and the like are described. The attribute of the search engine indicates a search target of the search engine. For example, information indicating a face search engine for searching for human faces, an animal search engine for searching for animals, a landscape search engine for searching for landscapes, and the like is described.

In the <Engine-Parameter> column, search engine parameters (post-processing portion) are described. The search engine parameter is a parameter used when the search engine searches for content. The parameter relating to the post-processing portion of the search engine parameter is information recognized by image recognition such as face recognition, and is information fed back to subsequent image recognition. For example, information on face recognition, speaker consonants, vowel information, etc. in speech recognition.

By describing the face-recognized information in <Engine-Parameter>, it is possible to register information related to a plurality of human faces. For example, information on facial expressions and angles can be registered. Thereby, even if the facial expression or angle changes, the target person can be searched with high accuracy. Furthermore, by describing information about a face over a long period of time, the target person can be searched with high accuracy even when the state of the face changes due to aging.

In the <Tag-Information> column, tag information is described. The tag information includes information given at the time of recognition and operation information for the recognition result. Information given at the time of recognition is, for example, criteria information (reference information) at the time of search, search range information, and the like. The operation information for the recognition result is, for example, information related to the search result sorting condition. The information related to the sort condition includes score information such as the degree of image blur. Thereby, the images can be sorted based on the degree of blurring of the images.

In the <ICON-Information> column, search icon configuration information is described. The search icon configuration information includes group configuration information, owner information, and management information. The group configuration information is information indicating that the search icon is obtained by grouping. The owner information is information such as the owner name of the search icon. The management information is various information related to management of search icons.

Although not shown in FIG. 5, the local environment information database information may be described in the icon definition file. The local environment information database information is summary information as a result of recognition. For example, information such as the number of recognized photos. By using this information, it is possible to create a search expression that can improve the search efficiency.

The search engine file F3 is a program module for searching for content. Search engines include a face search engine for searching for human faces, an animal search engine for searching for animals, a landscape search engine for searching for landscapes, and the like. In these engines, by creating various search engine parameter files F4 according to search targets, for example, an animal search engine can be used as a dog search engine, a cat search engine, a bird search engine, or the like.

The search engine parameter file F4 records the preprocessed part of the search engine parameters. The search engine parameters (pre-processing portion) include professional parameters that cannot be set by the user and user parameters that can be set by the user. The professional parameter is a parameter for optimizing the search, for example.

1-5. Configuration of Content Search Device FIG. 6 is a diagram showing functional blocks related to a search icon and a content registration function in the content search device 10 according to the present embodiment.

The functions related to the functional blocks shown in FIG. 6 are realized by the cooperation of the CPU 11, the ROM 12, the flash memory 18, etc. of the content search apparatus 10 and the programs (software) stored in the ROM 12, the flash memory 18, etc. Is done. Specifically, it is realized by the CPU 11 executing a program or the like stored in the ROM 12, the flash memory 18, or the like.

1-5-1. The registration-type content search apparatus 10 includes a search icon acquisition unit 31, a search icon registration unit 32, a search icon management unit 33, a search engine 34, an information addition unit 35, a search icon database 36 (icon storage unit) as functional blocks related to the registration function. (Hereinafter referred to as “search icon DB 36”), recognition parameter management section 37, content input section 38, content database 39 (content storage section; hereinafter referred to as “content DB 39”), tag score information database 40 (hereinafter referred to as “tag”).・ Score information DB40 ”).

The search icon DB 36 stores, for each search icon, four files, a search icon definition file F1, a search icon image file F2, a search engine file F3, and a search engine parameter file F4 shown in FIG. The search icon DB 36 stores a search icon management table.

FIG. 7A is a diagram showing an example of a search icon management table in the search icon DB 36. As shown in FIG. FIG. 7A shows an example in which three search icons are registered. The search icon management table 60 includes a search icon ID (ID), an icon image file name (ICON), a search engine attribute (Engine Attribute), a search engine file name (Engine), and a search engine parameter file name (for each search icon. Engine Parameter 1), search engine parameter (post-processing part) item name (Engine Parameter 2), tag information (information given at the time of recognition) (Tag Information), tag information (operation information for the recognition result) (Tag Action Information), And Recognized counter are recorded. The above items are the same as part of the contents described in the icon definition file.

For example, for a search icon with a search icon ID of 1, human. As the icon image file name (ICON). icon is described. In addition, a people-ID is described as an attribute of the search engine (Engine Attribute). This indicates a face recognition engine. In addition, as a search engine file name (Engine), humanDetect. DLL, search engine parameter file name (Engine Parameter 1) as humanDetect. xml, search engine parameter (post-processing part) item name (Engine Parameter 2) and people-ID_data, and tag information (information given at the time of recognition) (Tag Information) are described as “HOGE-san”. Also, N / A is described as tag information (operation information for the recognition result) (Tag Action Information). N / A indicates that an operation such as sorting is not performed on the recognition result. Recognized counter indicates the number of contents searched by the search using the search icon.

Also, for the search icon with the search icon ID 2, the icon image file name (ICON) is set to cat. icon is described. Further, pet-cat is described as a search engine attribute (Engine Attribute). This indicates a cat recognition engine for recognizing cats. Also, WebAPI URL as the search engine file name (Engine), and catDetect. As the search engine parameter file name (Engine Parameter 1). xml is described. Further, N / A is described as a search engine parameter (post-processing part) item name (Engine Parameter 2). N / A indicates that there is no search engine parameter (post-processing portion). In addition, information of Cat [Score] RectData is described as tag information (information to be given at the time of recognition) (Tag Information). In addition, “Sort by Percentage” is described as tag information (operation information for the recognition result) (Tag Action Information). Sort by Percentage indicates that the recognition result is sorted based on the Percentage.

Also, for the search icon with the search icon ID 3, the icon image file name (ICON) is “scene. icon is described. Further, “scene” is described as a search engine attribute (Engine Attribute). This indicates a scene recognition engine for recognizing a scene such as a landscape. Also, Function SceneDetect as the search engine file name (Engine) and sceneDetect. As the search engine parameter file name (Engine Parameter 1). xml, N / A is described as a search engine parameter (post-processing part) item name (Engine Parameter 2). Also, “Scene × n [Score]” is described as tag information (information to be given at the time of recognition) (Tag Information), and “Sort by Percentage” is described as tag information (operation information for the recognition result) (Tag Action Information).

FIG. 7B is a diagram showing an example of the contents related to the search engine parameter (post-processing portion). Specifically, FIG. 7B (a) shows an example of photographic data to be recognized. Photo data 61 shown in FIG. 7B (a) is image data obtained by photographing the coast. The photograph data 61 includes three persons M1, M2, M3, a tree M4, a sky M5, and a sea M6. FIG. 7B (b) shows an example of a search engine parameter (post-processing portion) 62 obtained as a result of performing recognition processing on the photographic data 61 shown in FIG. 7B (a). In this example, face recognition processing, scene recognition processing, and animal recognition processing are performed as recognition processing. Among these recognition processes, face 1, face 2, and face 3 of humans M1, M2, and M3 are recognized by the face recognition process. As the recognition result, information regarding the location, feature data, sex, and age is obtained for each face. The information regarding the place is information indicating the position of the face on the image indicated by the photograph data 61. The feature amount data is information such as the positions and sizes of the left and right eyes. In addition, the scene recognition process can obtain information on scores and places (positions) regarding trees, sky, and sea in a photograph. Moreover, the information regarding the score and place (position) regarding the dog, cat, and bird in the photograph can be obtained by the animal recognition process. There are no dogs or cats in the image shown in this photo data 61, but the face of humans M1, M2 and M3 is somewhat similar to dogs and cats. 20 and 10 are detected.

FIG. 8 is a diagram showing the configuration of the content DB 39 and the tag / score information DB 40. FIG. 8 illustrates a case where the content DB 39 and the tag / score information DB 40 store information related to image data.

The content DB 39 is configured using a relational database such as an SQL database or a NoSQL database. The content DB 39 stores a content DB basic table. Basic information related to content is registered in the content DB basic table. For example, in the content DB basic table, a content ID (rowid), link information (url) to the content, and the like are recorded. Further, the content DB 39 stores content main body files such as images, moving images, and sounds. Note that the content DB 39 may not store content main body files such as images, moving images, and audio as long as link information to the content is recorded. Note that thumbnail images are stored so that an image corresponding to the search result can be viewed even when the user is in an environment where the network cannot be used. Thumbnail images are stored for all contents.

The tag / score information DB 40 stores tag / score information related to the content stored in the content DB 39. The tag score information is information that is referred to by the search unit 55 described later in the content search. The tag score information is information relating to the characteristics of the content. For example, in the case where the content is a photograph, the information includes facial feature amount information obtained by face recognition, information on photographing conditions, information on a user, and the like. The tag score information includes, for example, information related to the score such as the degree of blurring of the image and the number of times content has been searched. The tag / score information DB 40 stores a user information table, a user face correspondence table, a face recognition data table, and an imaging condition table.

In the user information table, a user ID (userid), a user name (username), a face feature (rcgfacefeature), and a face-recognized face ID (rcgfaceid) are recorded.

In the user face correspondence table, a user ID (userid) and a face recognized face ID (rcgfaceid) are recorded.

In the face recognition data table, information (conf,..., Eye_ry) indicating various features of the recognized face, gender, age, feature, content ID (rowid), The face ID (rcgfaceid) or the like of the face recognized is stored.

In the photographing condition table, a content ID (rowid), various photographing conditions (Make,..., GPSAltitude) relating to images and moving images constituting the content are recorded. The information recorded in the shooting condition table may be, for example, Exif information obtained when shooting with a digital camera or the like, information obtained by adding other information to Exif information, or Exif. Information selected from information included in the information may be used.

The content DB basic table of the content DB 39 and the shooting condition table of the tag / score information DB 40 are associated by a content ID (rowid). The content DB basic table and the face recognition data table are associated by a content ID (rowid). The face recognition data table and the user face correspondence table are related by a recognized face ID (rcgfaceid). The user face correspondence table and the user information table are associated by a face recognition ID (rcgfaceid) or a user ID (userid).

In FIG. 6, the search icon acquisition unit 31 downloads the search icon via the communication unit 17. The downloaded search icon includes the four files, the search icon definition file F1, the search icon image file F2, the search engine file F3, and the search engine parameter file F4.

The search icon registration unit 32 registers the downloaded search icon definition file F1 in the search icon DB 36. When registration is performed, the search icon registration unit 32 outputs a signal indicating that the registration is completed and an icon ID to the search icon management unit 33.

Upon receiving a signal indicating that registration is complete from the search icon registration unit 32, the search icon management unit 33 outputs an icon ID and a search icon output request command to the search icon DB 36.

When the search icon DB 36 receives the search icon ID and the search icon output request command from the search icon management unit 33, the search icon DB 36 outputs the search icon of the ID to the search icon management unit 33.

When receiving the search icon from the search icon DB 36, the search icon management unit 33 configures the search engine 34 by executing the search engine file F3 (search engine program) included in the search icon, that is, starts the search engine 34. Let

The search engine 34 acquires search engine parameters from the search engine parameter file F4 included in the search icon, and performs a search process on the content DB 39 based on the search engine parameters. In parallel with the search process, the search engine 34 executes a recognition process for an image or the like registered in the content DB 39. When the search process and the recognition process are completed, the search engine 34 outputs the recognized information to the information adding unit 35 as tag / score information. Further, the search engine 34 outputs the recognized learning information to the information adding unit 35 as a search engine parameter (post-processing portion). The recognized learning information is information such as the position (coordinates) and orientation (for example, face orientation) of the recognition target, for example.

The search icon management unit 33 reads the tag / score information from the search icon definition file F1 and outputs the tag / score information to the recognition parameter management unit 37.

The recognition parameter management unit 37 outputs the tag / score information acquired from the search icon management unit 33 to the information adding unit 35.

The information adding unit 35 stores the tag / score information acquired from the recognition parameter management unit 37 in the tag / score information DB 40.

In addition, the information adding unit 35 describes the recognized learning information (search engine parameter (post-processing part)) in the search icon definition file F1 of the search icon.

The content input unit 38 acquires content via an external input unit such as an SD card reader. The acquired content includes a search icon and a content body. The content input unit 38 outputs a search icon included in the acquired content to the search icon management unit 33. The content input unit 38 registers the content body included in the acquired content in the content DB 39.

The search icon management unit 33 performs the same processing as when a search icon is registered from the search icon server 27 based on the search icon included in the acquired content.

1-5-2. Search System FIG. 9 is a diagram showing functional blocks related to a search function in the content search apparatus 10 according to the present embodiment.

The content search device 10 includes a search screen display control unit 51, a search icon DB 36, a search icon arrangement configuration confirmation unit 52, a search icon configuration condition confirmation unit 53, a search expression expansion unit 54, a search unit 55, as functional blocks related to the search function. An icon grouping configuration management unit 56, a search icon grouping tag information adding unit 57, a search icon registration unit 32, a content DB 39, a tag / score information DB 40, and a search result list generation unit 58 are provided.

The search icon DB 36, content DB 39, and tag / score information DB 40 are configured as described above.

The search screen display control unit 51 controls the display of the search screen shown in FIGS. 3A to 3C.

When the search icon is arranged in the search condition setting unit T1 of the search screen by the user, the search icon arrangement configuration confirmation unit 52 includes the attribute of the search icon described in the search icon definition file F1 corresponding to the search icon. Read the search engine parameters (post-processing part). The search icon attribute is information indicating an attribute to be searched. For example, it is information indicating a classification such as whether the search target is an animal, a person or a face. The search icon includes a date icon. The date icon is an icon indicating a search target period. The search engine parameter (post-processing portion) is information for further narrowing down the target indicated by the attribute of the search icon. For example, when the attribute of the search icon is face, the search engine parameter (post-processing part) is information indicating age, sex, male, and the like. In addition, when the attribute of the search icon is a date, it is information indicating a date, a period, and the like.

The search icon arrangement configuration confirmation unit 52 optimizes the search condition based on the attribute of the search icon and the search engine parameter (post-processing part). Search condition optimization is a process for improving search efficiency. Specifically, in order to improve the search efficiency, rearrangement processing of a plurality of parameters included in the search engine parameters (post-processing portion) is performed.

The search icon composition condition confirmation unit 53 determines whether the user has touched the “group elements” button B2. Further, the search icon configuration condition confirmation unit 53 determines whether the user has touched the “execute search” button B1.

When the user does not touch the “group elements” button B2 and touches the “execute search” button B1, the search expression expansion unit 54 is based on the attribute of the search icon and the search engine parameter (post-processing portion). , Generate an optimized search expression.

The search unit 55 searches the tag / score information DB 40 based on the generated search expression. The tag score information DB 40 is based on, for example, gender information and age information in the face recognition data table when the search formula includes conditions related to gender and age of a person (face). Then, the matching content is extracted, and the content ID (rowid) corresponding to this is output to the content DB 39. The content DB 39 outputs link information such as URL (url) and item length of the matching content to the tag / score information DB 40 using the content ID (rowid) as a key. The tag score information DB 40 outputs a search result that matches the condition indicated by the search expression to the search unit 55. Search results include link information such as URLs where content that matches the search conditions exists, thumbnail images (in the case of still images), title images (in the case of moving images), and the like. The thumbnail image includes shooting date information and the like.

The search result list generation unit 58 sorts the search results output from the search unit 55 based on the information included in the search result itself and the score information included in the search icon to generate a search result list. Information displayed as a search result includes a URL where content matching the search condition exists, a thumbnail image (in the case of a still image), a title image (in the case of a moving image), shooting date / time information, and the like. Since the search result list generation unit 58 acquires thumbnail images (in the case of still images) and title images (in the case of moving images), the file names (images) of the thumbnail images (in the case of still images) and title images (in the case of moving images) are acquired. The content DB 39 is searched using the file identification ID) to obtain thumbnail images (in the case of still images) and title images (in the case of moving images). The search result list generation unit 58 outputs the generated search result list to the output unit 59.

When the search icon configuration condition confirmation unit 53 determines that the “group elements” button B2 is touched, the icon grouping configuration management unit 56 is arranged in the search condition setting unit T1 selected as the grouping target. A search icon is generated by integrating the search conditions indicated by the search icon and the search conditions indicated by the setting conditions set in the first condition setting unit T2 and the second condition setting unit T3. That is, the icon grouping configuration management unit 56 generates four files: a search icon definition file F1, a search icon image file F2, a search engine file F3, and a search engine parameter file F4 that constitute a search icon. Further, the icon grouping configuration management unit 56 describes the search formula generated by the search formula expansion unit 54 in the search icon definition file F1.

For example, the icon grouping configuration management unit 56 combines the face icons of the father, mother, son, and daughter with “to” (AND) to generate a group icon for searching for a family group photo, or “ka” ( OR) to create a group icon that searches for each family member.

In addition, for example, the icon grouping configuration management unit 56 lowers the recognition parameter threshold for an animal search icon such as a dog icon or a cat icon and combines it with a human face icon by “and” (AND) to generate a dog. It is possible to generate a group icon that enables a search such as a search for a human face similar to the face of the human (dog face search) and a search for a human face similar to the face of the cat (cat face search).

The search icon grouping tag information adding unit 57 describes tag information of a newly generated search icon in the search icon definition file F1.

The search icon registration unit 32 registers the four generated files in the search icon DB 36.

The output unit 59 outputs the information output from the search result list generation unit 58 to the display unit 16 via the OS.

2. Operation 2-1. Search Icon Registration Operation FIG. 10 is a flowchart for downloading and registering a search icon from the search icon server 27 on the network.

The search icon is downloaded from the search icon server 27 on the network via the communication unit 17 (step S11). The downloaded search icon includes four files: a search icon definition file F1, a search icon image file F2, a search engine file F3, and a search engine parameter file F4.

The search icon registration unit 32 registers the search icon definition file F1 of the downloaded search icon in the search icon DB 36 (S12). When registration is performed, the search icon registration unit 32 outputs a signal indicating that the registration is completed and the ID of the search icon to the search icon management unit 33.

When the search icon management unit 33 receives a signal indicating that registration is complete from the search icon registration unit 32, the search icon management unit 33 configures the search engine 34 by executing the downloaded search engine file F3 (search engine program). The search engine 34 is activated (S13).

The search engine 34 acquires search engine parameters from the downloaded search engine parameter file F4, and executes search processing and recognition processing based on the acquired search engine parameters (S14). When the search process and the recognition process are completed, the search engine 34 outputs the recognized information to the information adding unit 35 as tag / score information (S15). Further, the search engine 34 outputs the recognized learning information to the information adding unit 35 as a search engine parameter (post-processing part) (S16).

The search icon management unit 33 reads the tag / score information from the search icon definition file F1, and outputs the tag / score information to the recognition parameter management unit 37 (S17).

The recognition parameter management unit 37 outputs the tag / score information acquired from the search icon management unit 33 to the information adding unit 35 (S18).

The information adding unit 35 stores the tag / score information acquired from the search engine 34 in the tag / score information DB 40 (S19).

Further, the information adding unit 35 describes the search engine parameter (post-processing part) in the search icon definition file F1 of the search icon (S20).

2-2. Content Registration Operation FIG. 11 is a flowchart when new content is downloaded to the content search apparatus 10 and its search icon is registered.

Content is input via the content input unit 38. The downloaded content includes a search icon and a content body. The search icon management unit 33 extracts a search icon from the content (S21).

In steps S22 to S30, the search icon management unit 33 is similar to the processing (S12 to S20) for registering a search icon from a site on the network in FIG. 10 based on the search icon included in the acquired content. Process.

2-3. Operation of Search Process Using Search Icon FIG. 12 is a flowchart of the search process using the search icon.

When the search icon is arranged in the search screen display control unit 51 (a predetermined area in the screen) by the user, the search icon arrangement configuration confirmation unit 52 reads the contents described in the search icon definition file F1 (S31). ).

The search icon arrangement configuration confirmation unit 52 optimizes the search condition based on the search icon attribute described in the search icon definition file F1 and the search engine parameter (post-processing part) (S32). For example, a case where “young” is searched for using the “young” icon IC7 will be described. The search engine attribute of the search icon definition file F1 of the “young” icon IC7 describes that it is a “face” search engine, etc., and the age range is described in the search engine parameter (post-processing part). Yes. In addition, the search engine attribute of the search icon definition file F1 of the “female” icon IC8 describes that it is a “face” search engine, and the search engine parameter (post-processing part) has a gender as female. It is described that there is. The search icon arrangement configuration confirmation unit 52 optimizes the search condition based on the information. Specifically, in order to improve the search efficiency, a reordering process for designating the search order is performed for a plurality of parameters included in the search engine parameters (post-processing portion). For example, when searching for “photos including young and female” by combining the “young” icon IC7 and the “female” icon IC8, first, to reduce the number of parameters, the gender ( The search conditions are rearranged so that the search conditions are narrowed down based on the “generator” and then narrowed down within the age range of the search engine parameters (post-processing portion). Thereby, the load of the search process can be reduced.

The search icon composition condition confirmation unit 53 determines whether the user has touched the “group elements” button B2 (S33). Further, the search icon configuration condition confirmation unit 53 determines whether the user has touched the “execute search” button B1 (S34). If the user touches the “group elements” button B2 or does not touch the “execute search” button B1, the process returns to step S31. On the other hand, when the user does not touch the “group elements” button B2 and touches the “execute search” button B1, the search expression expansion unit 54 selects “include young people and women” based on the optimized order. A search expression related to “photograph” is generated (S35).

The search unit 55 performs a search on the tag / score information DB 40 based on the generated search expression (S36). For example, when searching for “photos including young people and women” as described above, the search unit 55 refers at least to gender information and age information in the face recognition data table of the tag score information DB 40. Then, the content that satisfies the conditions indicated by the search expression is extracted, and the content ID (rowid) corresponding to the content is output to the content DB 39. The content DB 39 outputs link information such as URL (url) and item length of the matching content to the tag / score information DB 40 using the content ID (rowid) as a key. The tag score information DB 40 outputs link information such as URL (url) and item length received from the content DB 39 to the search unit 55 as a search result. The search unit 55 outputs these pieces of information to the search result list generation unit 58.

The search result list generation unit 58 uses the search result output from the search unit 55 based on information such as the link information included in the search result itself and score information described in the tag information of the icon definition file of the search icon. (For example, based on score information indicating the degree of blurring, sorting is performed so that the blurs are displayed in order from the smallest), and a search result list is generated (S37). The search result list generation unit 58 outputs the generated search result list to the display unit 16 (S38).

According to the present embodiment as described above, the amount of data hitting the content data can be determined by executing recognition processing on the content data group at the time of icon registration. Therefore, when generating (expanding) a search expression, an efficient order expression can be generated (in such an order that the filter is searched and recognized (filtered) first under the condition that the filter rate is high with respect to the parameter. To do). For example, in the database search, the search performance can be improved by performing a search that reduces the parameter to be searched first, and the search formula can be automatically configured efficiently.

In the embodiment, for example, voice / photo that has been separately registered in the past by performing a video search by combining a speaker recognition icon for voice data search and a search icon for face recognition with “ka” (OR).・ You can search for videos all at once.

It should be noted that the tray storing the icons may be uploaded and downloaded to the search icon server 27.

3. Effects The content search apparatus 10 according to the present embodiment searches for content that matches a predetermined condition.
The content search device 10
A search icon DB 36 for storing a plurality of search icons for designating search targets;
At least one search icon among a plurality of search icons stored in the search icon DB 36, and a condition designation object (first condition setting unit T2, first condition designation object) that designates a search condition for a search target designated by at least one search icon. A display control unit (CPU 11) that causes the display unit 16 to display a two-condition setting unit T3) and a search condition arrangement region (search condition setting unit T1) that is a region where search icons and condition-designated objects can be arranged.
The relationship between the search icons and condition-specified objects placed in the search condition placement area, the search target specified by the search icon placed in the search condition placement area, and the condition designation placed in the search condition placement area A search unit (CPU 11) that executes a search for content based on a search condition specified by an object;
Is provided.

This makes it possible to search for contents only by placing a search icon or condition-designated object in the search condition placement area instead of entering characters. Therefore, content search can be facilitated. In addition, it is possible to execute a search by combining the conditions of a plurality of icons in a complex manner.

In this embodiment,
The search condition specified by the condition specifying object is an AND condition indicating that content that satisfies all the search target conditions specified by a plurality of search icons arranged in the search condition arrangement area (search condition setting unit T1) is searched. OR condition that specifies that content satisfying any of the conditions of the search target specified by the plurality of search icons is searched, or search target specified by at least one search icon of the plurality of search icons This is information specifying a NOT condition that specifies that a search is performed so as to exclude content that satisfies the above condition.

Thus, the search can be executed by specifying an AND condition, an OR condition, or a NOT condition.

In this embodiment,
To realize a search function realized by a combination of a search icon arranged in a search condition arrangement area (search condition setting unit T1) and a search condition designation object (first condition setting unit T2, second condition setting unit T3). And an icon generation unit (icon grouping configuration management unit 56, search icon grouping tag information adding unit 57) for generating one new icon.

Thus, when performing a search in which the conditions of a plurality of search icons are combined, it is possible to easily execute the search simply by selecting one search icon.

In this embodiment,
Registration means for registering a new search icon is further provided.

This makes it possible to easily perform various searches based on various icons.

In this embodiment,
The search icon
A definition file (search icon definition file F1) in which various information related to the search icon is recorded;
An image file (search icon image file F2) used when displaying the search icon on the display unit;
A search engine program (search engine file F3) that recognizes the characteristics of the searched content;
including. further,
Feature storage file (search engine parameter file F4) for storing information about features recognized by the recognition program
May be included.

This makes it possible to configure the search icon as a collection of files. In particular, when a feature storage file is included, the target content can be searched more quickly.

(Other embodiments)
FIG. 13 is a block diagram illustrating a configuration of a content search system including a content search device according to another embodiment. In the present embodiment, as shown in FIG. 13, the content search apparatus 10 </ b> A is connected to the WEB server 26. A computer 25A is connected to the network.

In the present embodiment, the communication unit 17 (see FIG. 1) of the content search apparatus 10A can be directly connected to the WEB server 26. For the connection, local loopback can be used together with various standards and protocols as described above.

The computer 25 has a browser for receiving information from the WEB server 26 and the like existing on the network and displaying a homepage and the like. A program or the like for searching the content DB 39 of the content search apparatus 10 connected to the WEB server 26 is downloaded or plugged into the browser via the Internet and the WEB server 26.

The content search device 10 can be connected to the content DB 39 and the tag / score information DB 40 of the content search device 10A connected to the WEB server 26 via the Internet. The content search devices 10 and 10A include a search icon exchange unit that enables exchange of search icons between the connected content search devices 10 and 10A. In addition, the content search device 10 includes an external search unit for performing a search on the content DB 39 of another content search device 10A.

The present invention can be widely applied to a content search system, a content search device, a content search method, and a content search program that search for content such as images existing on a computer or a network.

10 Content Search Device 11 CPU
12 ROM
13 RAM
14 Touch Panel 15 External Device Connection Unit 16 Display Unit 17 Communication Unit 18 Flash Memory 19 Data Bus 21 Digital Camera 22 Video Camera 23 Smartphone 24 Recorder 25 Computer 26 Web Server 27 Search Icon Server 31 Search Icon Acquisition Unit 32 Search Icon Registration Unit 33 Search Icon management unit 34 Search engine 35 Information adding unit 36 Search icon DB (search icon database)
37 recognition parameter management unit 38 content input unit 39 content DB (content database)
40 Tag Score Information DB (Tag Score Information Database)
51 Search Screen Display Control Unit 52 Search Icon Arrangement Configuration Confirmation Unit 53 Search Icon Configuration Condition Confirmation Unit 54 Search Formula Expansion Unit 55 Search Unit 56 Icon Grouping Configuration Management Unit 57 Search Icon Grouping Tag Information Adding Unit 58 Search Result List Generation Unit 59 Output unit 60 Search icon management table 61 Photo data 62 Search engine parameter

Claims (9)

1. A content search device for searching for content,
   An icon storage unit for storing a plurality of search icons for designating search targets;
   At least one search icon of a plurality of search icons stored in the icon storage unit, a condition specifying object for specifying a search condition for a search target specified by the at least one search icon, the search icon, and the A display control unit for displaying a search condition arrangement area, which is an area in which a condition designation object can be arranged, on a display unit;
   An arrangement relationship between search icons and condition designation objects arranged in the search condition arrangement area, a search target designated by a search icon arranged in the search condition arrangement area, and an arrangement in the search condition arrangement area A search unit that executes a search of the content stored in the content storage unit based on the search condition specified by the specified condition specifying object;
   A content search apparatus comprising:
2. The search condition specified by the condition specifying object is:
   An AND condition indicating that the content satisfying all the search target conditions specified by the plurality of search icons arranged in the search condition arrangement area is one of the search targets specified by the plurality of search icons. OR condition that specifies that content satisfying the condition is searched, or that search is performed so as to exclude content that satisfies the search target condition specified by at least one search icon of the plurality of search icons. The content search apparatus according to claim 1, wherein the content search apparatus is information specifying a NOT condition to be performed.
3. An icon generation unit that generates a new icon for realizing a search function realized by a combination of a search icon arranged in the search condition arrangement area and a search condition designation object;
   The content search apparatus according to claim 2.
4. A registration means for registering a new search icon;
   The content search device according to any one of claims 1 to 3.
5. The search icon is
   A definition file that records various information about the search icon;
   An image file used when displaying the search icon on the display unit;
   A search engine program that recognizes the characteristics of the searched content;
   including,
   The content search device according to any one of claims 1 to 3.
6. The search icon is
   further,
   Including a feature storage file for storing information on features recognized by the recognition program;
   The content search apparatus according to claim 5.
7. The content storage unit is connected to the content search device via a network.
   The content search device according to any one of claims 1 to 6.
8. A content search method for searching for content by a computer including an icon storage unit, a calculation unit, and a display unit,
   A storage step in which the arithmetic unit stores a plurality of search icons for specifying search objects in the icon storage unit;
   The calculation unit includes at least one search icon among a plurality of search icons stored in the icon storage unit; a condition specifying object for specifying a search condition for a search target specified by the at least one search icon; A display step of displaying a search condition arrangement area, which is an area in which the search icon and the condition designation object can be arranged, on a display unit;
   The calculation unit includes an arrangement relationship between a search icon and a condition designation object arranged in the search condition arrangement area, a search target designated by a search icon arranged in the search condition arrangement area, and the search condition arrangement A search step for performing a search of content stored in the content storage unit based on a search condition specified by a condition specifying object arranged in the area;
   Content search method including
9. A content search program for causing a computer to function as means for executing the steps according to claim 8.

Images