JP2016004391A - Information processing system, management device, acquisition device, information processor, information processing method and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique which allows retrieval of proper material while reflecting an intention for material provision of a material holding person in accordance with a vague retrieval request of a retrieval executing person.SOLUTION: An information processing system comprises a holding device holding material data, an acquisition device acquiring the material data, and a management device performing management of acquisition of the material data from the holding device. The management device comprises: holding means which holds material information indicating characteristics of the material data for each material data held by the holding device; retrieval means which retrieves the material information using a retrieval query generated on the basis of a policy relevant to material data collection in accordance with a request from the acquisition device; confirmation means which confirms propriety of material data provision corresponding to the retrieved material information; and instruction means which instructs the holding device holding material data permitted providing to transmit the material data to the acquisition device.

Description

  The present invention relates to an information processing system, a management device, an acquisition device, an information processing device, an information processing method, and a computer program, and more particularly, to a technique for supporting collection of materials necessary for content creation from other devices. .

  When multiple people enjoy a trip or event, each participant takes a picture using his or her own photography device and combines the photos taken by each person after the trip or event to create a commemorative album or slideshow You may be able to create and enjoy some content. In this case, it is necessary to exchange and share the pictures taken by each. In order to support such sharing, it is known to upload photos taken by each other to an online storage or the like to share all the photos and select a photo to be used for content creation.

  However, due to the growing awareness of personal information protection, it is required to apply access control to shared photos, even if they are among friends. It will be a lot of work to do. Furthermore, files such as images generally have a large data size, and uploading to online storage requires a large cost and time.

  In addition, between the participants of the event, for example, by confirming whether or not you have a photo with XX?・ In some cases, sharing methods are used. In this case, it is necessary to perform another re-search according to the holding status of other participants and their intention to share. Doing these things while considering the composition of the content is a very time-consuming act.

  Patent Document 1 describes sharing data that is distributed and held in a plurality of terminals without disclosing all data. In this configuration, it is possible to transmit and receive data safely and efficiently by simultaneously transmitting and receiving the same information to all terminals existing on the network.

  Patent Document 2 describes performing a re-search by a user's reaction to a search result. In this configuration, the search result requested by the searcher can be acquired by executing a re-search using the searcher's feedback on the search result.

JP 2007-221260 A JP 2003-216646 A

  However, the configuration of Patent Document 1 is effective only when the data requested by the requester can be specified, such as “file named xxx.jpg” or “file registered as xxx”. It is. When actually exchanging photos, it is difficult to clearly specify a file that you do not know whether or not the other party has, and in the configuration of Patent Document 1, “I want to create such content, please give me the best photo” The request cannot be met. Also, in the configuration of Patent Document 1, an attempt is made to acquire a file that matches the conditions. However, because the file is actually held, the file holder may share the corresponding file with another person. It is necessary to check whether you are thinking.

  Further, in the configuration of Patent Document 2, the search executor performs feedback on whether or not the search result is valid. Therefore, the intention of the holder of the material as to whether or not the material such as a photograph can be provided cannot be reflected in the search.

  The present invention has been made in view of the above problems, and in response to a searcher's vague search request, a technique that enables a search for an appropriate material while reflecting the intention of the material holder regarding the material provision. The purpose is to provide.

In order to achieve the above object, an information processing system according to the present invention comprises the following arrangement. That is,
An information processing system comprising a holding device that holds material data, an acquisition device that acquires material data, and a management device that manages acquisition of material data from the holding device,
The management device
For each of the material data held by the holding device, holding means for holding material information indicating the characteristics of the material data;
In response to a request from the acquisition device, a search unit that searches the material information using a search query generated based on a policy related to material data collection;
Confirmation means for confirming whether or not material data corresponding to the retrieved material information can be provided;
Instructing means for instructing a holding device holding the material data permitted to be provided to transmit the material data to the acquisition device.

  ADVANTAGE OF THE INVENTION According to this invention, according to the search execution person's vague search request, the technique which enables the search of an appropriate material can be provided, reflecting the intention regarding the material provision of the holder of a material.

Block diagram showing the configuration of an information processing system for content creation Block diagram showing functional configuration example of material acquisition device Diagram showing an example of material acquisition policy Block diagram showing functional configuration example of material holding device Block diagram showing functional configuration example of material management device The figure which shows the use scene which is assumed concerning content creation typically Flow chart showing the processing procedure of material information registration processing Flow chart showing processing procedure of material acquisition processing The figure which shows the example of the material information hold | maintained at the material holding part of a material holding device The figure which shows the example of aesthetic evaluation by the three division method The figure which shows the example of the person list hold | maintained at the material holding part of a material holding device The figure which shows the example of the material information hold | maintained at the material information holding part of a material management apparatus Sequence diagram of processing flow related to material information registration processing The figure which shows the example of the material acquisition policy setting screen The figure which shows the example of the person designation screen in material acquisition policy setting Diagram showing examples of material acquisition policy and search query Figure showing examples of search results The figure which shows the example of the information which a material acquisition availability holding part hold | maintains The figure which shows the example of the screen of material provision availability judgment The figure which shows the example of the reply to the material acquisition possibility inquiry Sequence diagram of processing flow related to material acquisition processing Block diagram showing functional configuration example of material acquisition device Block diagram showing functional configuration example of material management device Flow chart showing the processing procedure of material information registration processing Flow chart showing processing procedure of material acquisition processing The figure which shows the example of the screen which inputs the reply to the material acquisition possibility inquiry The figure which shows the example of material acquisition availability information Sequence diagram of processing flow related to material information registration processing Diagram showing examples of material acquisition policy and search query The figure which shows the example of the material information hold | maintained at the material information holding part of a material management apparatus The figure which shows the example of the reply to the material acquisition possibility inquiry The figure which shows the example of the reply to the material acquisition possibility inquiry The figure which shows the example of the material acquisition condition hold | maintained at a material acquisition condition holding part Sequence diagram of processing flow related to material acquisition processing Block diagram showing the configuration of an information processing system for content creation Block diagram showing functional configuration example of material acquisition device Block diagram showing functional configuration example of material holding device Flow chart showing the processing procedure of material information registration processing The figure which shows the use scene which is assumed concerning content creation typically The figure which shows the example of the material information hold | maintained at the material information holding part of a material management apparatus Sequence diagram of processing flow related to material information registration processing Diagram showing examples of material acquisition policy and search query The figure which shows the example of the reply to the material acquisition possibility inquiry Sequence diagram of processing flow related to material acquisition processing

<< Embodiment 1 >>
Hereinafter, a first embodiment (Embodiment 1) of the present invention will be described in detail with reference to the drawings.

(Overall configuration and functions)
FIG. 1 is a diagram showing the overall configuration of the information processing system according to the present embodiment, and shows the functions and relationships of each device for realizing material acquisition in the present embodiment. FIG. 1 shows the relationship between the communication network 1010, the material acquisition device 1020, the material holding devices 1031 and 1032, and the material management device 1040 in this embodiment. Although two material holding devices 1031 and 1032 are shown in FIG. 1, the number of material holding devices is not particularly limited. The same device may also serve as a plurality of material acquisition apparatuses, material holding apparatuses, and material management apparatuses. Hereinafter, the functions of the communication network 1010, the material acquisition device 1020, the material holding devices 1031 and 1032 and the material management device 1040 and their relationship will be described with reference to FIG.

  The communication network 1010 is used when the material acquisition device 1020, the material holding devices 1031 and 1032, and the material management device 1040 communicate with each other. Each device can communicate with each other via a communication network 1010. The communication network 1010 includes a wireless communication network such as a wireless LAN, Bluetooth (registered trademark), and mobile public wireless, and a wired communication network such as Ethernet (registered trademark), and does not particularly limit communication means and protocols. The communication network 1010 may be realized by a network such as the Internet in which wired communication and wireless communication are mixed.

  The material acquisition apparatus 1020 acquires a material for content creation from the material holding apparatuses 1031 and 1032 using the communication network 1010 and the material management apparatus 1040. The material acquisition device 1020 includes an input device such as a keyboard and a pointing device, an output device such as a liquid crystal display and a touch panel, or a combination thereof. The material acquisition device 1020 uses the input / output device to acquire the user's intention as to what kind of content he wants to create. The material acquisition device 1020 transmits the user's intention to the material management device 1040 using the communication network 1010, acquires the material from the material holding device 1031 or 1032 and stores it in the storage. The material acquisition device 1020 may be any device as long as it includes a CPU (Central Processing Unit), memory, storage (HDD, flash memory, etc.) and the like. For example, it can be configured by an information processing apparatus such as a smartphone, a personal computer, or a tablet PC.

  The material holding devices 1031 and 1032 use the communication network 1010 to transmit information related to the material held by the device itself (hereinafter referred to as “material information”) to the material management device 1040. Here, “material” (material data) refers to data such as an image used to create content. In the present embodiment, an example in which a still image is used as a content material will be described. However, the type is not particularly limited as long as it is used for creating content such as a slide show, a movie, or a photo book. Examples of materials for creating content may include illustrations, moving images, music, effects, scenarios, scripts, algorithms, layout information, and the like. “Material” is the data itself used for the content, whereas “Material information” is the characteristics of “Material” (for example, identification information, data size, type, acquisition date, subject identification information, aesthetics, etc.) They are distinguished from each other in that the information is indicative of.

  Further, the material holding devices 1031 and 1032 present the requested contents to the user of the material holding devices 1031 and 1032 in response to the material acquisition request from the material management device 1040, and determine whether or not the material acquisition is permitted. In response to the user inputting the determination result, the material holding devices 1031 and 1032 transmit the determination result to the material management device 1040. When a transmission request is received from the material acquisition device 1020 for a material for which the user has given permission for material acquisition, the material is transmitted to the material acquisition device 1020. The material holding devices 1031 and 1032 may be any devices that can transmit material information to the material management device 1040 via the communication network 1010, determine whether or not to permit material acquisition, and send materials to the material acquisition device 1020. You may be comprised with the apparatus. The material holding devices 1031 and 1032 can be configured by an information processing device such as a smartphone, a personal computer, or a digital camera, for example. Further, the device does not necessarily need to be physically configured by one device, and a device that responds to a material acquisition request and a device that holds a material and performs material transfer may be different devices.

  The material management apparatus 1040 performs processing such as material information management, search, acquisition permission request, and material transfer. The material management apparatus 1040 holds the material information received from the material holding apparatuses 1031 and 1032 using the communication network 1010 and held by the respective material holding apparatuses, and makes it searchable. In addition, a search query is generated according to the material acquisition policy received from the material acquisition device 1020, a material search is executed, and a material acquisition permission is transmitted to the material holding devices 1031 and 1032 that hold materials that are acquisition candidates. When the material acquisition permission is obtained, the material is acquired and transferred to the material acquisition apparatus 1020. If the material acquisition permission is not obtained, a material acquisition permission is requested for a different material, and if the permission is obtained, the material is acquired. The material management apparatus 1040 may be configured in any way as long as it can provide the above functions, and is configured by an information processing apparatus such as a personal computer or a workstation. However, a virtual machine or the like may be used regardless of whether physically unique hardware exists.

  In the present embodiment, a material acquisition policy is designated for a material held in each material holding device and a search is executed. Accordingly, the user of the material acquisition device can acquire the material necessary for content generation without knowing the details of the material held by the material holding device. Also, when acquiring a material, confirmation is made as to whether or not material sharing is permitted. If sharing is not permitted, another material is acquired. Through this series of processing, it is possible to smoothly share materials necessary for content creation while considering privacy in a group of a plurality of people.

  The group in the present invention is created in association with a trip or event when a plurality of people participate in the trip or event. Participants of a trip or event belong to a group of photos taken during the trip or event, and share, search, and acquire photos based on this group. Further, it is assumed that the group is configured in advance, and any group configuration method such as a manual setting method, a semi-automatic setting method, or an automatic setting method may be used. Further, the method for each photographer to assign an image to a group may be any method such as arbitrary, forced, manual, and automatic. A specific photo may belong to a plurality of groups.

(Material acquisition device)
FIG. 2 is a block diagram showing an internal configuration of the material acquisition apparatus 1020. Hereinafter, the configuration of the material acquisition apparatus 1020 will be described with reference to FIG. Each functional element of the material acquisition apparatus 1020 is realized by cooperation of an HDD, a CPU, a memory, a computer program, and the like, but may be configured by specialized hardware.

  Reference numeral 21000 denotes an information input unit that acquires information indicating what kind of content the user wants to create. The information input unit 21000 inputs information indicating the user's intention to the material acquisition policy setting unit 22001 as a policy related to material data collection.

  Reference numeral 22001 denotes a policy setting unit that generates, sets, and outputs a policy for collecting materials according to information input by the information input unit 21000. The policy in this embodiment defines the purpose and usage of content created by combining materials, the number of materials to be collected, the subject selection method, the parameters for subject selection, the percentage of designated people, the type of material to be acquired, etc. It is what. The subject selection method designates whether to select a person who will be the center when creating content, or whether to make event and travel participants appear evenly. When designating a central person, the person is designated as a parameter for subject selection. In addition, it is also specified at what rate the designated person appears. The material type to be acquired specifies what material such as an image, a movie, or music is to be acquired and content is created. The material is searched and acquired based on this information.

  Examples of policies are shown as 13001, 13002, and 13003 in FIG. 13001 is a policy that assumes creation of content such as a photo book for recording events such as barbecue. This policy is intended to select 10 images so that participants appear evenly. Therefore, no value is set for “parameter” and “specified person ratio”, and “equal” is set for “subject selection method”.

  A policy 13002 assumes that a slide show showing a child's growth and the like is created. In this policy, only 10 photos / moving pictures in which children appear are selected. Therefore, in 13002, “specified person center” with a child as a designated person is set in “subject selection method”, “specified person image” is set in “parameter”, and “100%” is set in “specified person ratio”. ing.

  13003 is a policy on the assumption that about 10 photos are selected from photos of events such as weddings and content to be shared on an online album or the like is created. Under this policy, the bride and groom, who are the main actors of the wedding, will be selected, and the photos of the participants in the wedding will be selected to some extent evenly. For this reason, the “subject selection method” is set to “designated person center” with the bride and groom as the designated person, and the “parameter” is set to “designated person image”. On the other hand, “50%” is set in the “specified person ratio” so that a material in which a person other than the bride and groom is reflected can be acquired. In addition to the photo, BGM is also selected.

  Returning to the description of FIG. An information transmission unit 22002 acquires the material acquisition policy set by the material acquisition policy setting unit 22001 and transmits it to the outside (the material management device 1040 in this embodiment) using the communication unit 23000. Reference numeral 22003 denotes a material receiving unit that receives a material from the outside (a material holding device or a material management device in this embodiment) using the communication unit 23000 and stores the material in the material holding unit 22010. Reference numeral 22010 denotes a material holding unit, which holds the material stored by the material receiving unit 22003 and outputs it in response to a request. Reference numeral 23000 denotes a communication unit that is used when the material acquisition policy transmission unit 22002 and the material reception unit 22003 communicate with the outside.

(Material holding device)
FIG. 4 is a block diagram showing the internal configuration of the material holding devices 1031 and 1032. Hereinafter, the configuration of the material holding devices 1031 and 1032 will be described with reference to FIG. Each functional element of the material holding device can be realized by cooperation of an HDD, a CPU, a memory, a program, or the like, but may be configured by specialized hardware.

  Reference numeral 31000 denotes an information input / output unit that, when a material acquisition request is made from the outside for a material held by the material holding device, whether to present the material and permit the acquisition of the material. The intention of the material holder regarding The information input / output unit 31000 includes an output device such as a liquid crystal display, an input device such as a touch panel, a button, a pointing device, a keyboard, or a combination thereof. However, any information can be used as long as it can present information on the material to the user and input a determination as to whether or not to allow acquisition of the material from the outside.

  Reference numeral 32010 denotes a material holding unit, which holds materials that may be used for content creation and outputs them as required. Reference numeral 32001 denotes a material information extraction unit that acquires a material held in the material holding unit 32010, extracts information about the material (material information), and outputs the information to the material information transmission unit 32002. Reference numeral 32002 denotes a material information transmission unit that transmits the material information extracted by the material information extraction unit 32001 to the outside using the communication unit 33000.

  Reference numeral 32003 denotes a material acquisition request handling unit that receives a material acquisition request using the communication unit 33000. If it is unconfirmed whether or not the material corresponding to the received material acquisition request is confirmed, the material provision availability confirmation unit 32004 is inquired, and the inquiry result is transmitted using the communication unit 33000. If the material corresponding to the received material acquisition request can be acquired, the material transmission unit 32005 is instructed to transmit the material.

  Reference numeral 32004 denotes a material provision availability confirmation unit that interprets the material acquisition request received by the material acquisition request handling unit 32003 and uses the information input / output unit 31000 to confirm the intention of the material holder. Then, information indicating the intention of the confirmed material holder is output to the material acquisition request responding unit 32003. Reference numeral 32005 denotes a material sending unit that sends the corresponding material to the outside using the communication unit 33000 in response to an instruction from the material acquisition request handling unit 32003. Reference numeral 33000 denotes a communication unit which is used when the material information transmission unit 32002, the material transmission unit 32005, and the material acquisition request response unit 32003 communicate with the outside.

(Material management device)
FIG. 5 is a block diagram showing the internal configuration of the material management apparatus 1040. Hereinafter, the configuration of the material management apparatus 1040 will be described with reference to FIG. The following functional elements can be realized by cooperation of the HDD, CPU, memory, program, etc., but may be realized by specialized hardware.

  A communication unit 41000 communicates with the material acquisition device 1020 and the material holding devices 1031 and 1032 via the communication network 1010. Reference numeral 42001 denotes a material information receiving unit that receives information (material information) related to materials held by the material holding devices from the material holding devices 1031 and 1032 using the communication unit 41000, and sends them to the material information holding unit 42010. Store.

  Reference numeral 42010 denotes a material information holding unit, which records material information about the material held by each material holding device output from the material information receiving unit 42001. When other functional elements request it, it outputs it accordingly. Reference numeral 42002 denotes a material acquisition policy reception unit that receives and outputs a policy regarding material acquisition from the material acquisition device 1020 using the communication unit 41000.

  Reference numeral 42003 denotes a search query generation unit that generates and outputs a search query from the material acquisition policy output by the material acquisition policy reception unit 42002. Reference numeral 42004 denotes a search execution unit, which acquires the search query generated by the search query generation unit 42003. Then, the search execution unit 42004 uses the search query to execute a search for the material information held in the material information holding unit 42010, and outputs the search result. Reference numeral 42020 denotes a search result holding unit which holds the search result output by the search execution unit 42004 and outputs it in response to a request.

  Reference numeral 42005 denotes a material acquisition availability confirmation unit, which uses the communication unit 41000 to confirm material acquisition availability with respect to the material holding devices 1031 and 1032 that hold materials corresponding to the material information held in the search result holding unit 42020. And output the result. Reference numeral 42030 denotes a material acquisition availability holding unit, which holds information (material acquisition availability information) indicating acquisition availability for each material confirmed by the material acquisition availability confirmation unit 42005, and outputs it in response to a request. A material transfer unit 42006 refers to the material acquisition availability information held in the material acquisition availability holding unit 42030 and transfers the material held in the material holding devices 1031 and 1032 to the material acquisition device 1020.

(Specific processing example)
Next, a specific processing procedure for collecting materials using the above configuration will be described. In this embodiment, as shown in FIG. 6, shooting is performed in a group including five users 70010, 70020, 70030, 70040, and 70050. At this time, the users 70010, 70020, and 70030 take images with the cameras 70011, 70021, and 70031, respectively, and register information about the taken images in the server 70100. An example in which a slide show is created using the server 70100 using the smartphone 70012 possessed by the user 70010 after the event ends will be described.

  For a slide show of an event such as a barbecue, it is considered that it is more interesting for each participant of the event that as many people as possible appear as subjects rather than just certain people appear as subjects. Therefore, an example will be described in which the material acquisition device 1020 selects a photograph so that each subject appears evenly. However, this is not limited depending on the nature of the event. For example, a photo may be selected so that a specific person is the center, or a photo may be selected so that food and scenery are the center. . In this embodiment, an image taken as a material for creating content is handled. However, a material for creating content is not limited to an image, and music, moving images, and various other information can be handled as materials.

  In this embodiment, the cameras 70011, 70021, and 70031 correspond to the material holding devices 1031 and 1032, and hold images captured by the cameras as materials. The smartphone 70012 corresponds to the material acquisition device 1020 and acquires materials for creating a slide show. The server 70100 corresponds to the material management apparatus 1040, manages information related to the material, performs material search, determination of availability, and material transfer.

  It is assumed that the cameras 70011, 70021, and 70031 have already been grouped. The group composition method is not particularly limited. For example, a group may be manually configured, or cameras that have performed some operation at a short distance at the same time using GPS information may be treated as belonging to the same group. Or you may comprise a group automatically by analyzing the content of the image | photographed image. In addition, the devices constituting the group are not limited to cameras, and any devices can be used as long as they can handle materials for creating content, such as smartphones, personal computers, and servers on the Internet. A group ID for identifying a group to which each material holding device belongs is held in each material holding device. The photographed image group is automatically assigned a group ID to each image by the material holding device and automatically belongs to the group.

Material Information Registration Processing A specific processing example will be described with reference to a processing flow diagram related to material information registration shown in FIG. 7 and a processing flow diagram related to material acquisition shown in FIG. FIG. 7 shows a flow of processing when registering image information (image information) held by the camera in the server in the cameras 70011, 70021, 70031 (corresponding to the material holding device) and the server 70100 (corresponding to the material management device). It is the flowchart which showed. The following steps are executed based on the control of the CPU of each device.

  In step S101, the material information extraction unit 32001 acquires an image held in the material holding unit 32010, and extracts image information from the image. In the present embodiment, image information is material information indicating the characteristics of an image as a material, and indicates the shooting date / time of the image, the subject, aesthetics, and the like.

  FIG. 9 shows an example of the extracted image information. In the table of FIG. 9, for each image, the material ID for identifying the image, the shooting date / time, the ID (identification information) of the subject person, and aesthetics are recorded. Material IDs are automatically assigned in the order in which images are added to the respective material holding devices. Since the material holding device in the present embodiment is a camera, material IDs are assigned in the order in which the images were taken. When there are a plurality of material holding devices, there is a possibility that the same material ID exists in the group. Therefore, in order to avoid an ID collision, for example, in the material management device, a unique ID can be created by combining the material holding device ID and the material ID.

  FIG. 10 shows a list of subject persons. In FIG. 10, the subject person ID indicates which person is reflected in which picture by recognizing the face area of the person in the image held in the material holding device. According to the table of FIG. 9, it is shown that person 1 and person 4 are shown in the image of material ID1, and person 1 and person 2 are shown in the image of material ID3. As a method for calculating a person ID, clustering is performed by extracting the face area of a person from the entire image held in the material holding apparatus, extracting the feature amount of the face area, and treating each cluster ID as a person ID. it can. Any feature amount may be used here, and the method is not limited, such as luminance distribution and HOG. The clustering method is not limited to a k-means method or hierarchical clustering method. How can a relationship between persons in a plurality of images held in the material holding device (eg, whether the person in the image of material ID3 and the person in the image of material ID5 are the same person) be derived? Various methods may be used. As shown in the table of FIG. 10, the calculated person ID is held in combination with the person ID and the face area image of the person recognized as the same person.

  Aesthetics are information indicating the degree of beauty of an image (material). In this embodiment, aesthetics are calculated from the composition of the image. For example, a three-division method can be used for composition evaluation. First, a face area existing in the image is searched, and the position of the face area in the image is specified. Next, a line that divides the image into three parts vertically and horizontally as shown in FIG. 11 is drawn, and aesthetics are calculated from the intersection of the lines and the proximity of the face area. In the present embodiment, the aesthetics are evaluated based on the composition of the image. However, any method may be used as long as it can evaluate the aesthetics of the image.

  Note that the image information does not have to be limited to the shooting date and time, the person who is the subject, and aesthetics, but the camera settings during shooting, weather, temperature, subject area, composition, photographer status, etc. Any information may be used. The material is not limited to an image, and any material such as a moving image, voice, music, or three-dimensional CAD data may be used.

  In S102, the camera as the material holding device uses the communication unit 33000 to obtain the material information as illustrated in FIG. 9 and the person list as illustrated in FIG. To the server (information management device). In the server 70100 as the material management apparatus, the material information receiving unit 42001 receives the material information using the communication unit 41000. The received material information is stored and registered in the material information holding unit 32020 as shown in FIG. 12, for example. The difference from the material information (FIG. 9) extracted by the material information extraction unit of each camera is that a material holding device ID is held according to each material information. By indicating which material holding device each material information is received from, it is possible to specify which material holding device each material is held in.

  Since the subject person ID is assigned to each material holding device, even if the subject person ID given by one material holding device is the same as the subject person ID given by another material holding device, the ID is the same. It does not necessarily indicate a person. Therefore, it is necessary to associate which subject person ID of the other material holding device is the subject person ID of each material holding device. Therefore, in the present embodiment, the subject person ID is reassigned using the face area image included in the person list received from each camera. Any method for reassigning the subject person ID may be used. For example, clustering is performed by extracting the feature amount of the face area image from the person list received from each material holding device, and the subject person ID held in a certain material holding device is held in a different material holding device. It is estimated which subject person ID is the same. Note that any feature amount may be used here, and the method is not limited, such as luminance distribution and HOG. Further, the clustering method is not limited to a k-average method or hierarchical clustering, and any method can be used as long as the subject person IDs held in different material holding devices can be linked. Also good.

  FIG. 13 shows a sequence related to material information registration. In this embodiment, when a user 70010 constituting a group presses the shutter button of the camera 70011 to perform a photographing action (13001), an image is added (13002) as a material to the camera 70011 as a material holding device. . When an image is added to the camera 70011, information (material information) about the image is extracted in S101 (material information extraction) (13003). In step S102 (image information registration), information on the material is transmitted to the server 70100 (13004) and registered (13005).

  As described above, with reference to FIGS. 6, 7, 9 to 13, etc., in the material holding device and the material management device when the image held by the camera as the material holding device is registered in the server as the material management device. The flow of processing (material information registration processing) to be performed has been described.

Material acquisition processing Next, in the material acquisition device, the material holding device, and the material management device, processing (material acquisition processing) when the material acquisition device acquires an image from the material holding device using the flowchart of FIG. The flow will be described. The following steps are executed based on the control of the CPU of each device.

  In S201, the material search query is generated according to the following procedure. First, the material acquisition policy setting unit 22001 sets a material acquisition policy. The material acquisition policy is set by selecting from a plurality of material acquisition policies using the liquid crystal touch panel of the smartphone 70012 corresponding to the information input unit 21000 of the material acquisition device 1020 and customizing it based on the policy.

  An example of the policy selection screen is shown in FIG. In the present embodiment, a template is selected from policies such as 13001, 13002, and 13003 prepared in advance in FIG. 3, and the parameters can be changed as necessary. The parameter is changed, for example, by changing the number of materials to be acquired or selecting a person who is the center of the content as shown in FIG. The person who becomes the center of the content can be selected by selecting from an image held by the content creator.

  Next, the material acquisition policy transmission unit 22002 transmits the material acquisition policy set by the material acquisition policy setting unit 22001 to a server as the material management apparatus 1040 using the communication unit 23000. In the server, the material acquisition policy reception unit 42002 receives the material acquisition policy, and the search query generation unit 42003 generates a search query. For example, when a policy for recording an event such as a barbecue shown in 16001 in FIG. 16 is received, a search query like 16002 in FIG. 16 is generated.

  The search query is actually written in a computer-readable format. The search query may be in any format such as SQL or script. In addition, a search query may be directly specified as a material acquisition policy without using the search query generation unit 43003, or a material acquisition policy may be handled as a search query if search execution is possible.

  In S202, the search execution unit 42004 executes a material search using the search query 16002 generated in S201. The material information holding unit 42010 of the material management apparatus 1040 holds image information held in each material holding terminal as shown in FIG. Since the search condition in this embodiment is 16002, search results such as 17001 and 17002 in FIG. 17 are output. Since 17001 is higher when the sum of aesthetics is compared between 17001 and 17002, in this embodiment, 17001 is the first priority acquisition material and 17002 is the second priority acquisition material. This search result is held in the search result holding unit 42020.

  In the present embodiment, two types of search results are output. However, the search results need not be two types, but may be one type or three or more types. The search method may be a brute force, a genetic algorithm, or the like, regardless of the type of search method. In addition, since the calculation time becomes longer when the number of images to be managed becomes enormous, the time may be shortened by performing branch cutting or the like. In this case, the best result may not necessarily be obtained as a search result. Needless to say, the search result holding unit 42020 may be realized by any holding means such as text data or a database.

  In step S <b> 203, the material acquisition availability confirmation unit 42005 refers to the search result held in the search result holding unit 42020 and checks whether each material included in the search result can be acquired. The search results held in the search result holding unit 42020 are, for example, 17001 and 17002 in FIG. 17 as described above. In this case, the material listed as the first candidate for the material used for content creation is held in the material ID 2 held in the material holding device ID 70011, the material ID 3 held in the material holding device ID 70021, and the material holding device ID 70031. It is 3 points of material ID2. Therefore, the terminal holding each material is inquired about whether or not the material can be acquired. The management of the inquiry result is held in a table as shown in FIG.

  In the cameras 70011, 70021, and 70031 as the material holding devices, the material acquisition request responding unit 32003 uses the communication unit 33000 to receive the material acquisition availability inquiry. When the material acquisition availability inquiry is received, the material provision availability confirmation unit 32004, as shown in FIG. 19, asks the information input / output unit 31000 that the material acquisition availability inquiry has been made, and rejects whether to accept the request. Indicates that you can select In addition, a dialog for determining whether or not to allow material acquisition is presented. The material acquisition permission / non-permission dialog is not limited to the UI as shown in this figure, and may have any shape such as a touch panel, a dial, or a button. In addition, the information to be presented is not limited to this, for example, information on a user who is trying to acquire a material, information on the purpose of use of the material, etc. Any information may be presented. Further, the material holding unit 32010 and the information input / output unit 31000 do not have to be the same device, and the material acquisition permission / non-permission may be determined via another device. Further, it is not necessary to inquire the permission / non-permission of the user of the material holding apparatus one by one, and it may be set such that the inquiry from the user is always permitted or not always permitted. Furthermore, an automatic response may be realized by specifying in advance a condition such that a photo in which a specific person is a subject is always not permitted.

  When the material acquisition is permitted or not permitted, the fact is transmitted to the material management apparatus 1040. When the material management apparatus 1040 receives information indicating whether material acquisition is permitted or not, the material management apparatus 1040 stores the result in the material acquisition permission holding unit 42030. An example of information held by the material acquisition availability holding unit 42030 is shown in FIG. The material acquisition availability holding unit 42030 indicates each material held in the material holding devices 1031 and 1032 and whether or not the material acquisition permission has been acquired, and whether or not permission has been acquired.

  In step S <b> 204, the material acquisition availability checking unit 42005 refers to the material acquisition availability holding unit 42030 to check whether acquisition permission has been obtained for all the acquisition candidate materials. If acquisition permission is obtained for all materials, it is determined that acquisition can be executed (YES in S204), and the process proceeds to S206. If acquisition permission is not obtained for all of the acquisition candidate materials (NO in S204), the process proceeds to S205.

  In step S <b> 205, the material acquisition availability confirmation unit 42005 refers to the search result holding unit 42020 and determines whether there is another search result for which material acquisition availability is to be confirmed. In this embodiment, only two patterns 17001 and 17002 are output as acquisition candidates as search results of the search execution unit 42004. Therefore, if 17001 cannot be acquired (NO in S205), it is confirmed whether the 17002 search result can be acquired, the 17002 search result is stored in the material acquisition availability holding unit 42030, and the process of S203 is performed again. Run. If all images cannot be acquired for both the first candidate and the second candidate (YES in S205), the process of S202 is executed again except for images that cannot be acquired. In this embodiment, the process waits until there is a response such as acquisition permission or acquisition refusal from all the material holding terminals. For example, when there is no response for a certain period of time, it may be regarded as acquisition refusal. Further, when the material holder and the material acquirer are the same, the material acquisition permission confirmation may not be performed in the first place.

  In S206, the material transfer unit 42006 of the server 70100 as the material management apparatus 1040 transmits a material transfer request to the cameras 70011, 70021, and 70031 as the material holding apparatuses. In the cameras 70011, 70021, and 70031, the material transmission request is received by the material transmission unit 32005. Upon receiving the material transfer request, the material sending unit 32005 obtains the target material from the material holding unit 32010 and sends it to the material obtaining device 1020 using the communication unit 33000. In this embodiment, the material is transmitted to the material acquisition apparatus 1020 via the server, but the material may be directly transmitted to the material acquisition apparatus 1020. Further, it may be transmitted via another terminal, and the transmission route and method are not limited.

  As described above, in the present embodiment, access is restricted in order to instruct the material holding device that holds the material corresponding to the material information in order from the material information of the higher rank of the search score, in order to transmit the material data. Even so, it is possible to search for an appropriate material.

  The processing flow related to material acquisition is as described above. A sequence relating to this processing is shown in FIG. However, in order to make the flow of processing easy to understand, it is assumed that there are two cameras 70011 and 70021.

  In step S201, the user 70010 designates a material acquisition policy using the smartphone 70012 (21001). The smartphone (material acquisition apparatus) 70012 transmits information on the material acquisition policy to the server 70100 as the material management apparatus 1040 (21002), and the server 70100 generates a search query (21003).

  In S202, the server 70100 searches for a material (21004). In S203, based on the result of S202, the camera 70011 and the camera 70021 confirm whether the material can be acquired (21005). In response to the inquiry, each material holding device returns whether or not material acquisition is possible (21006).

  In S204, it is determined whether or not all the materials used for content generation can be acquired based on the acquisition result confirmation result of each material obtained in S203 (21007). If it is determined that all materials can be acquired, S206 is executed.

  If it is not determined that acquisition of all materials used for content generation is possible, S205 is executed (21008). If there are different search results, S203 is executed again for the different search results. If there is no different search result, S202 is executed again.

  In S206, a material acquisition request is made to each material holding terminal (21009). In response to the material acquisition request, the material holding device relays the server 70100 to transmit the corresponding material to the material acquisition device 70012 (21010).

  The flow of processing performed by the smartphone, camera, and server when the smartphone 70012 acquires images held by the cameras 70011 and 70021 has been described above with reference to FIG.

  As described above, in the configuration of the present embodiment, by generating a search query from a search policy, the searcher collects a desired material without knowing in advance what the material that can be acquired is. be able to. In addition, it is possible to appropriately perform access control by causing the holder of the material to set whether or not to provide the material. Therefore, by acquiring materials used for content creation from a plurality of other material holding devices, it is possible to efficiently acquire materials for content creation without knowing the details of the images held by each material holding device. It becomes possible.

<< Embodiment 2 >>
Hereinafter, the second embodiment (Embodiment 2) of the present invention will be described in detail with reference to the drawings. The main difference from the first embodiment is that information related to acquisition permission is set at the time of material information registration, and transfer is performed sequentially from the material acquisition permission obtained.

(overall structure)
The function and relationship of each device for realizing the present embodiment will be described. The overall configuration of the information processing system according to the present embodiment is the same as that of the first embodiment shown in FIG. However, the material holding devices 1031 and 1032 include not only the material information but also information such as whether to permit the material acquisition device 1020 to acquire the material, whether to sequentially determine, or reject the information to the material management device 1040. Send.

  Next, the internal configuration of each device will be described. The internal configuration of the material acquisition apparatus 1020 is shown in FIG. The difference from the internal configuration (FIG. 2) of the material acquisition apparatus 1020 in the first embodiment is that it does not have the material acquisition policy transmission unit 22002 but has a search query generation unit 22004 and a search query transmission unit 22005. The search query generation unit 22004 converts the material acquisition policy set in the material acquisition policy setting unit 22001 into a search query. The search query transmission unit 22005 acquires the search query output from the search query generation unit 22004 and transmits it using the communication unit 23000.

  The internal configuration of the material holding devices 1031 and 1032 is shown in FIG. 4 as in the first embodiment. However, in the material holding apparatuses 1031 and 1032 of this embodiment, the material provision availability confirmation unit 32004 uses the information input / output unit 31000 to determine in advance whether or not material acquisition is possible for the material held in the material holding unit 32010. Is different from the first embodiment. As will be described later, the judgment result of the material provision availability confirmation unit 32004 is transmitted to the material management apparatus 1040 together with the material information, the material provision availability is retained in the material acquisition availability holding unit 42030, and the material information is retained in the material information holding unit 42010. Is done.

  The internal configuration of the material management apparatus 1040 is shown in FIG. The difference from the internal configuration (FIG. 5) of the material management apparatus 1040 in the first embodiment is that the material acquisition policy receiving unit 42002 and the search query generating unit 4003 are not included, but the search query receiving unit 42007 and the transfer status holding unit 42040 are different. It is a point to have. The search query reception unit 42007 receives the search query from the material acquisition apparatus 1020 using the communication unit 41000 and outputs the search query to the search execution unit 42004. The search execution unit 42004 does not include the material registered in the material acquisition availability holding unit 42030 as being unacceptable when executing the search. Furthermore, when confirming whether or not material acquisition is possible in the material acquisition availability confirmation unit 42005, the material information holding unit 42010 is referred to and a material registered as material acquisition availability is attempted without being inquired about whether or not material acquisition is possible. . Another difference is that confirmation of material acquisition is performed only for materials that are sequentially confirmed. The transfer status holding unit 42040 holds the transfer status of the material transferred by the material transfer unit 42006.

(Specific processing example)
Specific processing examples with reference to the configuration diagrams shown in FIG. 1, FIG. 4, FIG. 22, FIG. 23, the processing flow diagram related to material information addition shown in FIG. 24, and the processing flow diagram related to material acquisition shown in FIG. Will be explained. The usage scene of the information processing system according to the present embodiment is the same as that shown in FIG. 6 referred to in the first embodiment.

  In the present embodiment, a case where content that records the growth of the same person as a subject is created will be described as an example. Therefore, it is assumed that two or more images taken at the same date and time are not used in order to ensure a certain degree of variation in time series. However, any variation can be used. For example, even if it is the same date and time, it may be good if it is in a different time zone, or it may be selected and used if the composition etc. are different even in the same time zone. May be. In this way, the method of ensuring the variation and the definition of the variation are not limited. Depending on the nature of the content created as the image selection method, this may not be the case. For example, photos may be selected so that the people who were present on the spot are equal, and food and scenery are the center. There may be cases where a photo is selected. Also, in this embodiment, an example of handling an image (still image) taken by a camera as a material for creating content will be described, but the type of material is not limited, and videos, music, illustrations, and other various information Can be treated as a material. Further, the acquisition route of the material such as an image downloaded from the Internet as well as an image taken by the camera is not limited.

  In the present embodiment, as in the first embodiment, the cameras 70011, 70021, and 70031 in FIG. 6 correspond to the material holding device 1031 or 1032 in FIG. 1 and hold the material. The smartphone 70012 corresponds to the material acquisition device 1020 and acquires materials for creating a slide show. The server 70100 corresponds to the material management apparatus 1040, manages information related to the material, performs material search, determination of availability, and material transfer.

  Note that images taken by the cameras 70011, 70021, and 70031 are set in advance so that they are automatically tagged (associated) with the same group. The group configuration method is not particularly limited. For example, a group may be configured manually, or a plurality of cameras that have performed some operation within a certain distance at close times using GPS information can be handled as belonging to the same group. Alternatively, the contents of the captured image may be analyzed, and the group may be automatically configured when it is determined that the shooting is temporally and locally bonded. In addition, the devices constituting the group are not limited to cameras, and any devices can be used as long as they can handle materials for creating content, such as smartphones, personal computers, and servers on the Internet.

Material Information Registration Processing FIG. 24 shows processing when registering image information held by the camera in the server in the cameras 70011, 70021, 70031 (corresponding to the material holding device) and the server 70100 (corresponding to the material management device). It is the flowchart which showed the flow of material information registration processing. The following steps are executed based on the control of the CPU of each device.

  In S101, the material information extraction unit 32001 acquires a material (image) held in the material holding unit 32010, and extracts material information (image information) from the material. Since the specific processing content is the same as that of the first embodiment, the description is omitted.

  In step S <b> 199, the material provision availability confirmation unit 32004 acquires images retained in the material retention unit 32010, and uses the information input / output unit 31000 to cause the user of the material retention device to determine whether to provide each image. For example, a screen as shown in FIG. 26 is presented to the user to prompt input. The result of determining whether or not material provision is possible for each image by the user of the material holding device is acquired by the material management device 1040 and is output in association with the material ID as shown in FIG. As shown in FIG. 27, the content of material provision availability (acquisition availability) includes, for example, “OK” that permits material provision, “NG” that does not allow material provision, “Sequential judgment” that is judged for each request for material, and user response It can be set as “unanswered” or the like.

  Note that the material provision permission / non-permission dialog is not limited to the UI as shown in FIG. 26, and any shape / form such as a touch panel, a dial, or a button can be used. In addition, the contents of whether or not to be provided and registered are not limited to those described above. For example, the availability of the material may be determined and registered according to the acquirer, its attribute, the purpose of use of the material, and the like. Further, the material holding unit 32010 and the material provision availability confirmation unit 32004 do not have to be the same device, and the material provision permission / inhibition may be determined via another device. Further, it is not necessary to inquire of the user of the material holding device about permission or disapproval step by step, and it may be set such that a material acquisition permission request from a specific material acquisition device is always permitted or not permitted. Furthermore, the user of the material holding device may specify in advance a condition such that an image in which a specific person is captured as a subject is always not permitted.

  In S102, as in the first embodiment, the camera as the material holding device transmits material information, person list information, and the like to the server as the material management device 1040. However, the present embodiment is different in that not only the output result of the material information extraction unit 32001 but also the output result of the material provision availability confirmation unit 32004 are transmitted to the material management apparatus 1040 together.

  The flow of processing performed in the material holding device and the material management device when an image held by the camera as the material holding device is registered in the server as the material management device has been described above using FIG.

  FIG. 28 shows a sequence related to material information registration. In FIG. 28, unlike FIG. 13 referred to in the first embodiment, S199 (material provision availability determination extraction) is included. In S199, the user is made to determine whether or not the material can be acquired (29003), and the determination result is acquired (29004).

  In this embodiment, the user is allowed to determine whether or not to provide the material after extracting the material information. However, the material information may be extracted after allowing the user to determine whether or not the material can be provided, and these processes may be performed simultaneously. Also good.

Material Acquisition Processing Next, processing (material) when the material acquisition device acquires an image from the material holding device in the information processing system using the flowchart showing the flow of material acquisition in the present embodiment shown in FIG. The flow of acquisition processing will be described. The following steps are executed based on the control of the CPU of each device.

  In S201, unlike the first embodiment, the material acquisition device 1020 itself generates a material search query in response to a material acquisition policy input from the user. Specifically, in this embodiment, a policy for recording an event such as a barbecue indicated by 30001 in FIG. 29 is created and converted into a search query such as 30002 in FIG. Thereafter, the generated search query is transmitted to the material management apparatus 1040.

  In S202, as in the first embodiment, the search execution unit 42004 of the material management apparatus 1040 searches for a material using the received search query (for example, something like 30002 in FIG. 29).

In the search query used in the present embodiment, a person (subject person ID1 in the present embodiment) designated for all photographs appears. 30 is extracted from the material held in the material information holding unit 42010 of the material management apparatus 1040 and includes the subject person ID1. According to the search query 30002 of FIG. 29, it is necessary to increase the total aesthetics. Therefore, the materials in the table of FIG. 30 are arranged in the order of high aesthetics as follows.
Material ID 3 of material holding device ID 70031 (Aesthetics: 70).
-Material ID 21 of material holding device ID 70011 (Aesthetics: 60).
Material ID 3 of material holding device ID 70021 (Aesthetics: 60).
Material ID 6 of material holding device ID 70011 (Aesthetics: 40).
Material ID 6 of material holding device ID 70021 (Aesthetics: 40).
Material ID 8 of material holding device ID 70031 (Aesthetics: 20).
Here, since the material ID 3 of the material holding device ID 70031 and the material ID 21 of the material holding device ID 70011 are images taken at the same date and time, they do not satisfy the search condition. Therefore, the material ID 6 of the material holding device ID 70011 with the next highest aesthetics is selected without selecting the material ID 21 of the material holding device ID 70011. As a result, the material ID 3 of the material holding device ID 70031, the material ID 3 of the material holding device ID 70021, and the material ID 6 of the material holding device ID 70011 are output as search results. Further, the material ID 6 of the material holding device ID 70021 is output as a spare for the case where the acquisition of the three selected sheets is rejected.

  In S203, as in the first embodiment, the material acquisition availability confirmation unit 42005 refers to the search result held in the search result holding unit 42020 to check whether each material included in the search result can be acquired. In the example of the present embodiment, the acquisition target materials are three pieces of material ID3 of the material holding device ID 70031, material ID3 of the material holding device ID70021, and material ID6 of the material holding device ID 70011. FIG. 31 shows the acquisition permission acquisition status of these three sheets. Referring to the content held in the material information holding unit 42010 (FIG. 30), the material ID 6 of the material holding device ID 70011 is registered as acquirable when the material information is registered in the material information registration step S199. Therefore, the acquisition permission inquiry result is handled as OK without making an inquiry about whether or not acquisition is possible. Since the material ID 3 of the material holding device ID 70021 and the material ID 3 of the material holding device ID 70031 are sequentially inquired, the respective material holding devices are inquired. When each material holding device receives an inquiry about material acquisition, it responds whether or not the material can be acquired, as in the first embodiment. The result of the availability inquiry is shown in FIG.

  In S206, as in the first embodiment, the material acquisition apparatus 1020 performs processing for acquiring the material searched from the material holding apparatuses 1031 and 1032. However, the inquiry results shown in FIG. 32 include those that are determined to be unacquirable as a result of the inquiry, and therefore only those that can be acquired are acquired. Note that information related to material acquisition of the material acquired in this step is held in the transfer status holding unit 42040 of the material management apparatus 1040 in the format shown in FIG. In this embodiment, after the material acquisition is completed, the end determination S207, which is the next process, is executed. However, in an environment where a plurality of processes can be executed in parallel, the next process is executed while the transfer process is being executed. You can do it. Here, the environment in which a plurality of processes can be executed includes a pseudo environment.

  In step S <b> 207, the material transfer unit 42006 refers to the transfer status holding unit 42040 and determines whether transfer has been completed for all the materials. If the transfer has been completed (YES in S207), the process ends. If not (NO in S207), the process proceeds to S205.

  In S205, it is determined whether or not the material needs to be searched again. If the number of materials determined to be unacquirable in S203 is equal to or less than the search result output as a backup, it is determined that a re-search is unnecessary (NO in S205), and S203 is executed for the material output as a backup. . In the above-described example, acquisition permission is attempted for the material ID 6 of the material holding device ID 70021 that was output as a backup in S202. If the number of materials determined to be unacquirable in S203 is larger than the preliminary search results, or if the material acquisition permission is inquired for the preliminary search results but it is determined that acquisition is impossible, a re-search is necessary. A determination is made (YES in S205), and S202 is executed. In this embodiment, an example in which only one search result is output as a backup has been described, but a plurality of search results may be provided.

  The above is the description of the processing flow related to material acquisition. A sequence relating to this processing is shown in FIG. However, in order to make the flow of processing easy to understand, it is assumed that there are two cameras 70011 and 70021.

  In FIG. 34, in S201, the user 70010 designates a material acquisition policy using the smartphone 70012 (35001). The material acquisition device 70012 converts the material acquisition policy into a search query (35002), and sends it to the server 70100 (35003).

  In S202, the server 70100 searches for a material using the received search query (35004). In S203, based on the result of S202, the camera 70011 and the camera 70021 confirm whether the material can be acquired (35005). In response to the inquiry, each material holding device returns whether or not material acquisition is possible (35006). In S206, a material acquisition request is made to each material holding terminal (35007). In response to the material acquisition request, the material holding device transmits the corresponding material to the smartphone 70012 which is the material acquisition device (35008).

  In S207, it is determined whether all the materials have been acquired in S205 (35009). If it is determined that all materials have been acquired (YES in S207), the process ends. If it is determined that the necessary materials are not available (NO in S207), S205 is executed. In S205, it is determined whether or not to execute the search again (35010). If the search result output as a spare can be used (NO in S205), S203 is executed again. If there is no usable search result output as a spare (YES in S205), S202 is executed again.

  The flow of processing performed by the smartphone, camera, and server when the smartphone 70012 acquires images held by the cameras 70011 and 70021 has been described above with reference to FIG.

  As described above, by acquiring materials used for content creation from multiple material holding devices owned by other users, it is possible to efficiently create content without knowing the details of the images held by each material holding device It will be possible to get material for. Also, by registering information related to material acquisition at the time of material information registration, it is possible to reduce the labor related to the material acquisition availability inquiry and to efficiently collect materials. In addition, by outputting the next candidate as a preliminary search result instead of an alternative search set, the acquisition target material can be flexibly changed according to the material acquisition availability inquiry result. Further, by sequentially transferring the material from which the material acquisition permission has been acquired, efficient transfer processing is realized, and the processing time can be reduced.

<< Embodiment 3 >>
Hereinafter, the third embodiment (Embodiment 3) of the present invention will be described in detail with reference to the drawings. The main difference from the first embodiment is that material management is performed by communication between the material holding devices without providing the material management device.

(Overall configuration and functions)
FIG. 35 shows the functions and relationships of each device for realizing the present embodiment. Hereinafter, functions and relationships of each device will be described.

  FIG. 35 shows the relationship between the communication network, the material acquisition device, and the material holding device in this embodiment. In FIG. 35, 1010 is a communication network, 1020 is a material acquisition device, and 1031 and 1032 are material holding devices. Although two material holding devices 1031 and 1032 are shown in FIG. 35, the number of material holding devices is not particularly limited. The same device may also serve as a plurality of material acquisition devices and material holding devices.

  The communication network 1010 is the same as that in the first embodiment. However, since the material management apparatus 1040 does not exist in this embodiment, the material acquisition apparatus 1020 and the material holding apparatuses 1031 and 1032 are used for communication.

  The material acquisition apparatus 1020 acquires a material for content creation from the material holding apparatuses 1031 and 1032 using the communication network 1010. The material acquisition apparatus 1020 includes a means for inputting a user's intention of creating what kind of content, such as a keyboard, a pointing device, a liquid crystal display, a touch panel, or a combination thereof. The material acquisition device 1020 generates a search query, performs a search using the communication network 1010, and acquires a material as a search result. The material acquisition device 1020 may be realized by any device as long as it has such a function, and is configured by an information processing device such as a smartphone, a personal computer, or a tablet PC.

  The material holding devices 1031 and 1032 use the communication network 1010 to exchange information (“material information”) on the material held by the device itself. Also, a search is executed in response to a material search from the material acquisition device 1020, the request content is presented to the user of the material holding device in response to a material acquisition request, and whether or not material acquisition permission is permitted is determined, and the determination result is acquired. Transmit to device 1020. When a transmission request is received for a material for which material acquisition permission has been issued, the material is transmitted to the material acquisition device 1020. The material holding devices 1031 and 1032 may be any device as long as it is capable of exchanging material information through the communication network 1010, determining whether or not to permit material acquisition, and sending the material to the material acquisition device 1020. Also good. For example, the material holding device can be configured by an information processing device such as a smartphone, a personal computer, or a digital camera. In addition, the material holding device does not necessarily have to be physically configured by one device. For example, a device that responds to a material acquisition request and a device that holds material and transfers material are different devices. There may be.

  The functions of the communication network 1010, the material acquisition device 1020, and the material holding devices 1031 and 1032 and their relationship have been described above with reference to FIG.

(Material acquisition device)
FIG. 36 is a block diagram showing the internal configuration of the material acquisition apparatus 1020. Hereinafter, the configuration of the material acquisition apparatus 1020 will be described with reference to FIG.

  As can be seen by comparing FIG. 36 with FIG. 2 referred to in the first embodiment, the material acquisition apparatus 1020 of this embodiment does not include the material acquisition policy transmission unit 22002 and the material reception unit 22003 of FIG. Further, unlike the first embodiment, the material acquisition apparatus 1020 of this embodiment includes a search query generation unit 22004, a search request unit 22006, a search result holding unit 22020, a material acquisition availability confirmation unit 22007, a material acquisition availability holding unit 22030, A material transfer unit 22008 is included.

  The search query generation unit 22004 is the same as that described with reference to FIG. 22 in the second embodiment. The search request unit 22006 acquires the search query output by the search query generation unit 22004, sends a search request to the material holding device using the communication unit 23000, and receives the search result. The received search result is held in the search result holding unit 22020.

  The search result holding unit 22020 has the same function as the search result holding unit 42020 described with reference to FIG. 5 in the first embodiment. The material acquisition availability confirmation unit 22007 has the same function as the material acquisition availability confirmation unit 42005 described in Embodiment 1 with reference to FIG. The material acquisition availability holding unit 22030 has the same function as the material acquisition availability holding unit 42030 described with reference to FIG. 5 in the first embodiment. The material transfer unit 22008 has the same function as the material transfer unit 42006 described with reference to FIG. 5 in the first embodiment.

(Material holding device)
FIG. 37 is a block diagram showing the internal configuration of the material holding apparatuses 1031 and 1032. Hereinafter, the configuration of the material holding devices 1031 and 1032 will be described with reference to FIG. The difference from FIG. 4 referred to in the first embodiment is that the material information transmitting unit 32002 is not included, and the material information holding unit 32020, the material information synchronizing unit 32006, the search query receiving unit 32007, and the search executing unit 32008 are included. It is.

  The material information holding unit 32020 has the same function as the material information holding unit 42010 of FIG. 5 referred to in the first embodiment. The material information synchronization unit 32006 uses the communication unit 33000 to exchange the contents of the material information holding unit 32020 included in each of the material holding devices 1031 and 1032 and synchronizes the held material information. The search query receiving unit 32007 has the same function as the search query receiving unit 42007 of FIG. 23 referred to in the second embodiment. The search execution unit 32008 has the same function as the search execution unit 42004 in FIG. 5 referred to in the first embodiment.

(Specific processing example)
A specific processing example will be described with reference to the above-described configuration, a processing flow diagram related to material information registration shown in FIG. 38, and a processing flow diagram related to material acquisition shown in FIG.

  In the present embodiment, consider an example in which a group of five users 70010, 70020, 70030, 70040, and 70050 participates in an event such as a wedding as shown in FIG. Here, each of the users 70010, 70020, and 70030 performs shooting with the cameras 70011, 70021, and 70031. A description will be given assuming that a slide show relating to the contents of an event is created using the smartphone 70012 possessed by the 70010 after the event ends. It is assumed that the bride and groom who are the main characters of the event in this embodiment are 70040 and 70050.

  Therefore, in the present embodiment, materials are selected and acquired so as to be centered on the persons 70040 and 70050, who are the main players of the event, and distributed to some extent in time series. However, this is not limited depending on the nature of the event. For example, the material may be selected so that as many people as possible appear, or the photograph may be selected so that food and scenery are the center. In this embodiment, images taken as content creation materials are handled, but the content creation materials are not limited to images, and various other information such as videos and music can be handled as materials. it can.

  In this embodiment, the cameras 70011, 70021, and 70031 correspond to the material holding devices 1031 and 1032, hold the materials, and manage information on the materials. The smartphone 70012 corresponds to the material acquisition apparatus 1020, and acquires a material for creating a slide show by performing a material search request, an inquiry about whether to acquire the material, and transferring the material.

  The cameras 70011, 70021, and 70031 have already been grouped, and the captured image group automatically belongs to the group. The group composition method is not particularly limited. For example, a group may be configured manually, or cameras that have performed some operation near the same time using GPS information may be treated as a group, and automatically analyzed by analyzing the contents of the captured image. A group may be formed. The devices that make up the group are not limited to cameras, but any information processing device that can handle materials for creating content, such as smartphones, personal computers, and servers on the Internet. Good.

Material Information Registration Processing FIG. 38 is a flowchart showing the flow of processing when the cameras 70011, 70021, and 70031 (corresponding to the material holding device) extract, register, and synchronize information on materials held by the camera.

  In S101, as in S101 of FIG. 7 referred to in the first embodiment, the material information extraction unit 32001 acquires the material (image) held in the material holding unit 32010, and extracts the material information (image information) from the material. To do.

  In S102, the material information extracted in S101 is held in the material information holding unit 32020. An example of the material information held in the material information holding unit 32020 is shown in FIG. In this embodiment, each material holding device holds information on the material held in all the material holding devices constituting the group. Therefore, as shown in FIG. 40, material information held in other material holding devices is also held. In the present embodiment, the material information is held in a table format, but the material may be held in any format as long as the material information can be held, such as using text data or a database.

  In S104, the material information synchronization unit 32006 transmits newly added material information held by the material holding device itself to the material information holding unit 32020 to another material holding device. In addition, material information held in another material holding device is received and held in the material information holding unit 32020. As a result, the material information holding units 32020 in all the material holding devices are synchronized, and the same and the latest material information is always maintained. As a method for synchronizing material information, any method can be used as long as each material holding device can grasp the material information held in other material holding devices and can make it searchable, and only the difference is notified. And all the material information may be exchanged one by one. In addition, the material information held in all the material holding devices at a specific time does not necessarily have to be the same, and there may be a delay depending on the processing status.

  The flow of processing for extracting and managing information on materials held by the camera as the material holding device has been described above with reference to FIG. Next, FIG. 41 shows a sequence related to material information registration.

  In the present embodiment, when the user 70010, 70020 constituting the group presses the shutter button of the camera 70011, 70021, respectively, and the photographing action is performed, an image is added as a material to the material holding device that is a camera. (42001). When an image is added in the cameras 70011 and 70021, information relating to the image is extracted in S101 (42002). Through S102, information on the material is registered in the material information holding unit 32020 of the cameras 70011 and 70021 (42003). Thereafter, the material information is exchanged between the cameras 70011 and 70021 through S104 (42004).

Material Acquisition Processing Next, the flow of processing when the material acquisition device 1020 acquires images from the material holding devices 1031 and 1032 will be described using the flowchart of FIG. Although the processing flow is the same as that in the first embodiment, the material management apparatus 1040 is not provided in the present embodiment, and therefore the processing content is different from that in the first embodiment.

  In S201, as in the second embodiment, the material acquisition device 1020 itself generates a material search query in response to a material acquisition policy input from the user. Examples of the material acquisition policy and the search query in this embodiment are shown in 430001 and 430002 in FIG. In the present embodiment, images including a designated person are acquired in order of high aesthetics. However, since it is desired to select as many scene images as possible, a photograph within 5 minutes before and after the selected image is calculated assuming that the aesthetics are halved. However, it does not necessarily have to be 5 minutes before and after, for example, 10% of the time length of the event period (for example, 6 minutes for a 60-minute event) is limited to one image. You may specify how. Alternatively, a method may be employed in which the entire event is divided into four, and one event is selected from each divided period. In other words, any method may be used as long as it does not select only photos of a specific scene. In addition, when there are extremely many photographs of the same scene, it is considered that the scene is very important in the event. In such a case, a plurality of images may be selected from a short time region. In this case, for example, the image may be selected in consideration of the total number of shots, not the time of the event itself. For example, if 1000 images are taken, one image is selected from the first to 100th images, and one image is selected from the 101st to 200th images. Can be used. In addition, even if the shooting interval is short, an image may be selected if the angle or subject is different, and any method can be used as long as the image of a specific scene is not selected. May be used.

  In S202, the search request unit 22006 transmits a search query to the material holding device using the communication unit 23000, and requests execution of the search. When a search query is sent, a search is executed in the material holding device, and the result is returned via the communication network 1010 and the communication unit 23000. The search result is held in the search result holding unit 22020. In the present embodiment, a search request is transmitted to one randomly selected among the material holding apparatuses existing in the group. Here, when there are a plurality of material holding devices, the search request may be transmitted to any device, and a device with a short network distance or a device with a low CPU load may be selected. Alternatively, a search request may be transmitted to all the material holding devices, and the search result of the device from which the search result was obtained first may be used.

  On the other hand, when the material holding device receives a search query from the material acquisition device 1020, the material holding device executes a search according to the received search query. An example of material information held in the material information holding unit 32020 of the material holding device in the present embodiment is shown in the table of FIG. In this table, the material ID 21 of the material holding device ID 70031 has the highest aesthetics. When the material ID 21 of the material holding device ID 70031 is selected and the aesthetics of the images within 5 minutes before and after are considered as half and calculated, the material ID 43 of the material holding device ID 70021 has the next highest aesthetics. Similarly, the material ID 12 of the material holding device ID 70011 has the next highest aesthetics. In this embodiment, a substitute image is output in preparation for a case where each image cannot be acquired. For example, it is assumed that the material ID 21 of the material holding device ID 70031 cannot be acquired, and the material ID 43 of the material holding device ID 70021 and the material ID 12 of the material holding device ID 70011 are selected. Materials that can be candidates are the material ID 25 of the material holding device ID 70011, the material ID 38 of the material holding device ID 70021, and the material ID 39 of the material holding device ID 70031. The respective aesthetics are 32.5 and 37.5 because the material photographed within 5 minutes before and after the material already selected (the material ID 43 of the holding device ID 70021 and the material ID 12 of the material holding device ID 70011) is regarded as half. , 65.

  Therefore, the material ID 39 of the material holding device ID 70031 can be substituted for the material ID 21 of the material holding device ID 70031. Similarly, when the material ID 43 of the material holding device ID 70021 or the material ID 12 of the material holding device ID 70011 cannot be acquired, the material ID 38 of the material holding device ID 70021 with the next highest aesthetics is substituted. In the present embodiment, when there are materials that cannot be acquired, other materials that can be acquired are acquired as they are. However, in the case of a material that should be searched again including other materials, an instruction may be given to execute the search again without outputting an alternative material.

  Next, in S203, as in the first embodiment, the material acquisition availability confirmation unit 22007 refers to the search result held in the search result holding unit 22020, and determines whether each material included in the search result can be acquired. Check. The table of FIG. 43 shows an example of the acquisition target material and the acquisition inquiry result in this embodiment. In the present embodiment, an inquiry about whether or not to acquire is made for all the materials held by the material holding device, but an inquiry is not always required. For example, if the user of the material acquisition device and the user of the material holding device are the same person, the inquiry about whether or not the material acquisition is possible may be omitted. Further, it may be determined whether or not to make an inquiry according to the nature of the material and the past material acquisition permission status.

  In S204, as in the first embodiment, the material acquisition availability confirmation unit 22007 refers to the material acquisition availability holding unit 22030, and confirms whether acquisition permission has been obtained for all of the acquisition candidate materials. In the example of FIG. 43, if material acquisition permission (OK) is obtained for all the materials shown in the table of FIG. 43 (YES in S204), the process proceeds to S206 and acquisition is performed. If there are NG items, that is, if there are items for which acquisition permission has not been obtained (NO in S204), the process proceeds to S205. In the present embodiment, material acquisition is performed in S206 after material acquisition permission is obtained for all materials, but material acquisition may be performed sequentially from materials for which material acquisition permission has been obtained.

  In S206, as in the first embodiment, the material acquisition device 1020 acquires the searched material from the material holding device. However, since the material management apparatus 1040 in the first embodiment does not exist in the present embodiment, the material acquisition apparatus 1020 and the material holding apparatus holding the target material directly communicate to transfer the material. .

  In S205, first, with reference to the material acquisition permission acquisition status shown in the table of FIG. 43 and the search result held in the search result holding unit 22020, the presence of a material that can be substituted for the material for which the material acquisition permission was not obtained. Confirm. In this embodiment, an alternative material is set for each material included in the search result. For this reason, if there is only one material that cannot be acquired, re-searching is unnecessary (NO in S205), and S203 is executed for the alternative material. If there are two or more materials that cannot be acquired, it is determined that a re-search is necessary (YES in S205), the images that could not be acquired are omitted from the search target, the process returns to S202, and the material search is executed again.

  The processing flow related to material acquisition is as described above. A sequence relating to this processing is shown in FIG. However, in order to make the flow of processing easy to understand, it is assumed that there are two cameras 70011 and 70021.

  In S201, the user designates a material acquisition policy (45001), and generates a search query from the material acquisition policy (45002). In S202, the smartphone 70012 makes a search request to the camera (45003), and acquires the result of searching for the material by the cameras 70011 and 70012 (45004).

  In S203, whether or not the material can be acquired is confirmed with respect to the cameras 70011 and 70021 as the material holding devices based on the result of S202 (45005). In response to the inquiry, each camera 70011, 70021 as the material holding device returns whether or not material acquisition is possible (45006).

  In S204, it is determined whether or not all the materials used for content generation can be acquired based on the acquisition result confirmation result of each material obtained in S203 (45007). If it is determined that all materials can be acquired (YES in S204), S206 is executed. If it is not determined that acquisition of all materials used for content generation is possible (NO in S204), S205 is executed.

  In S205, it is determined whether or not to execute the search again (45008). If an alternative material for a material that cannot be acquired is set (NO in S205), S203 is executed again for the alternative material. If there is no usable search result output as a spare (YES in S205), or if there are two or more materials that could not be acquired, S202 is executed again.

  In S206, a material acquisition request is made to each material holding terminal (45009. Upon receiving the material acquisition request, the material holding device (camera) transmits the corresponding material to the smartphone 70012 as the material acquisition device (45010).

  The flow of processing performed by the smartphone and the camera when the smartphone 70012 that is the material acquisition device acquires images held by the cameras 70011 and 70021 that are the material holding devices has been described above with reference to FIG.

  As described above, by acquiring materials used for content creation from multiple other material holding devices, it is possible to efficiently create content without knowing the details of the images held by each material holding device. It becomes possible to get. Further, by managing the material information with the material holding device and exchanging the material information between the material holding devices, the material can be acquired without the need for the material management device.

  As described above, according to each embodiment of the present invention, when a plurality of people participate in a trip or event, a good content is created in a group created accompanying the trip or event. Can do. At this time, it is not necessary for the content creator to know the details of the materials held by other material holders constituting the group. In addition, when using material held by the material holder in the content, the copyright, personal information, and privacy of the photographer and subject are protected by confirming whether the material holder is used. Can be handled properly.

<< Other Embodiments >>
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

Material acquisition device 1020, material holding device 1031 and 1032, material management device 1040, communication network 1010

Claims (21)

An information processing system comprising a holding device that holds material data, an acquisition device that acquires material data, and a management device that manages acquisition of material data from the holding device,
The management device
For each of the material data held by the holding device, holding means for holding material information indicating the characteristics of the material data;
In response to a request from the acquisition device, a search unit that searches the material information using a search query generated based on a policy related to material data collection;
Confirmation means for confirming whether or not material data corresponding to the retrieved material information can be provided;
An information processing system comprising: an instruction unit that instructs a holding device holding the material data permitted to be provided to transmit the material data to the acquisition device. The acquisition device includes:
An input means for inputting the policy from a user;
Transmission means for transmitting the policy to the management device,
The management device further includes a generation unit that generates a search query based on a policy received from the acquisition device,
The information processing system according to claim 1, wherein the search unit searches the material information using a search query generated by the generation unit. The acquisition device includes:
An input means for inputting the policy from a user;
Generating means for generating a search query based on the input policy;
Transmission means for transmitting the search query to the management device;
The information processing system according to claim 1, wherein the search unit of the management device searches the material information using a search query received from the acquisition device.   4. The policy according to claim 1, wherein the policy specifies at least one of a purpose, a use, a type, an amount, and a feature of content data to be acquired. Information processing system according to item. The holding device includes a notification unit that notifies the management device in advance whether or not the material data held by the holding device can be provided.
5. The information processing system according to claim 1, wherein the confirmation unit of the management device confirms whether or not the material data can be provided with reference to a notification from the holding device. .   The confirmation unit inquires whether the material data can be provided by inquiring whether the material data is provided to the holding device that holds the material data corresponding to the material information searched by the search unit. The information processing system according to claim 1, wherein the information processing system is confirmed.   The search means further searches for alternative material information when provision of material data corresponding to the searched material information is not permitted. Information processing system described in 1.   7. The instruction unit according to any one of claims 1 to 6, wherein the instruction unit instructs transmission of material data to a holding device that holds material data corresponding to the material information in order from the material information having a higher search score. The information processing system according to any one of claims. An acquisition device that acquires material data, and a management device that can communicate with a holding device that holds material data,
For each of the material data held by the holding device, holding means for holding material information indicating the characteristics of the material data;
In response to a request from the acquisition device, a search unit that searches the material information using a search query generated based on a policy related to material data collection;
Confirmation means for confirming whether or not material data corresponding to the retrieved material information can be provided;
A management apparatus comprising: an instruction unit that instructs a holding apparatus that holds the material data permitted to be provided to transmit the material data to the acquisition apparatus. An acquisition device that can communicate with a holding device that holds material data, and a management device that manages acquisition of material data from the holding device,
An input means for inputting a material data collection policy from a user;
Generating means for generating a search query based on the input policy;
Transmitting means for transmitting the search query to the management device;
An acquisition device comprising: a receiving unit configured to receive material data searched based on the search query from the holding device. An information processing system comprising a holding device that holds material data and an acquisition device that acquires material data,
The holding device is
For each of the material data held by the holding device, holding means for holding material information indicating the characteristics of the material data;
In response to a request from the acquisition device, a search unit that searches the material information using a search query generated based on a policy related to material data collection;
Sending means for sending a search result by the search means to the acquisition device;
The acquisition device includes:
Confirmation means for confirming whether or not material data corresponding to the retrieved material information can be provided;
An information processing system comprising: a requesting unit that requests transmission of the material data to the acquisition device from a holding device that holds the material data permitted to be provided. Comprising a plurality of the holding devices;
Each of the holding devices communicates with other holding devices to hold the material information related to all material data held by the plurality of holding devices in the holding unit,
The information processing system according to claim 11, wherein the search unit searches the material information related to all the material data using the search query. The acquisition device includes:
An input means for inputting the policy from a user;
Transmission means for transmitting the policy to the holding device;
The holding device further includes generating means for generating a search query based on the policy received from the acquisition device,
The information processing system according to claim 11 or 12, wherein the search unit searches the material information using a search query generated by the generation unit. The acquisition device includes:
An input means for inputting the policy from a user;
Generating means for generating a search query based on the input policy;
Transmission means for transmitting the search query to the holding device;
The information processing system according to claim 11 or 12, wherein the search unit of the holding device searches the material information using a search query received from the acquisition device.   The confirmation unit inquires whether the material data can be provided by inquiring whether the material data is provided to the holding device that holds the material data corresponding to the material information searched by the search unit. The information processing system according to claim 11, wherein the information processing system is confirmed.   16. The search unit according to claim 11, further comprising: searching for alternative material information when provision of material data corresponding to the searched material information is not permitted. Information processing system described in 1.   17. The requesting unit requests transmission of material data to a holding device that holds material data corresponding to the material information in order from material information having a higher search score. The information processing system according to any one of claims. An information processing apparatus for collecting material data held in a plurality of terminals,
An input means for inputting a material data collection policy from a user;
Generating means for analyzing the input policy and generating a search query;
Search means for searching for material information indicating characteristics of material data held in the plurality of terminals, using the generated search query;
Confirmation means for confirming whether or not material data corresponding to the retrieved material information can be provided;
An information processing apparatus comprising: requesting means for requesting transmission of the material data to a terminal holding the material data permitted to be provided. A holding device that holds material data; an acquisition device that acquires material data; and a management device that includes holding means that holds material information indicating the characteristics of the material data for each of the material data held by the holding device. An information processing method in an information processing system having:
The management device is
In response to a request from the acquisition device, a search step for searching the material information using a search query generated based on a policy related to material data collection;
A confirmation step for confirming whether or not material data corresponding to the retrieved material information can be provided;
An information processing method comprising: an instruction step for instructing a holding device that holds the material data permitted to be provided to transmit the material data to the acquisition device. An information processing method in an information processing system comprising: a holding device that holds material data and material information indicating the characteristics of the material data; and an acquisition device that acquires material data,
The holding device is
In response to a request from the acquisition device, a search step for searching the material information using a search query generated based on a policy related to material data collection;
A sending step of sending a search result by the search step to the acquisition device;
The acquisition device is
A confirmation step for confirming whether or not material data corresponding to the retrieved material information can be provided, and transmission of the material data to the acquisition device to a holding device that holds the material data permitted to be provided. An information processing method comprising: a requesting process that requests   A computer program for causing a computer to function as each unit included in the management device according to claim 9, the acquisition device according to claim 10, or the information processing device according to claim 18.

Images