JP5870742B2 - Information processing apparatus, system, and information processing method - Google Patents

Description

  The present disclosure relates to an information processing device, a system, and an information processing method.

  Sharing photos and videos taken by users with other users via a network or the like is already widely performed. For example, Patent Document 1 discloses a technique for creating an album on a server from a user terminal via a network and browsing the album from another user terminal.

JP 2004-102548 A

  By the way, the above-described photographs and moving images are generated by users having various IT (Information Technology) literacy. Also, for many users who generate content, unlike commercial content producers, for example, it is difficult to spend a great deal of time and effort on organizing the content generated by the user. Therefore, there are many photographs and videos that have not been organized after being taken and stored on a PC (Personal Computer) or the like.

  So, for example, if you want to view content that you have created yourself, or share content with other users, you may find it difficult to find the content you want, or search for the part of your content that you are interested in In many cases, inconvenience was caused by the fact that the contents were not organized, such as taking time.

  Therefore, the present disclosure proposes a new and improved information processing apparatus, system, and information processing method capable of promoting the use of content by sharing while reducing the effort of the user who provides the content.

  According to the present disclosure, a shared user information acquisition unit that acquires information on a target user who is a content sharing partner including a still image or a moving image, and a target subject that is a subject associated with the target user, and the content, There is provided an information processing apparatus including a content extraction unit that extracts content in which the target subject is shown as target content, and a scenario generation unit that generates a scenario for configuring digest content by combining the target content.

  Further, according to the present disclosure, a first user who provides content including a still image or a moving image sets information about a target user who is a partner with whom the content is shared and a target subject that is a subject associated with the target user. A first client device that includes an operation unit that acquires an operation to perform, a shared related information acquisition unit that acquires information on the target user and the target subject, and content that shows the target subject from the content as target content A server device including a content extraction unit to extract and a scenario generation unit that generates and outputs a scenario for configuring digest content by combining the target content, and acquires the output scenario, and the content according to the scenario Generate the above digest content from System and a second client device including a content acquisition unit which provides the target user is provided.

  Further, according to the present disclosure, it is possible to acquire information on a target user who is a content sharing partner including a still image or a moving image and a target subject that is a subject associated with the target user, and from the content, the target There is provided an information processing method including extracting content showing a subject as target content and generating a scenario for combining the target content to form digest content.

  According to the above configuration, it is possible to automatically extract the content in which the target subject is shown from the shared content, and generate a digest content combining the extracted content. This automation reduces the effort of the user who provides the content. Since the digest content can be content closer to the content desired by the user who views the content, a more comfortable content viewing experience can be realized for the content sharing partner.

  As described above, according to the present disclosure, it is possible to promote the utilization of content by sharing while reducing the effort of the user who provides the content.

1 is a schematic block diagram illustrating a functional configuration of a system according to an embodiment of the present disclosure. 5 is a diagram for describing an operation of a system according to an embodiment of the present disclosure. FIG. 3 is a diagram illustrating an example of sharing settings according to an embodiment of the present disclosure. FIG. 3 is a diagram illustrating an example of sharing settings according to an embodiment of the present disclosure. FIG. 3 is a diagram illustrating an example of sharing settings according to an embodiment of the present disclosure. FIG. 3 is a diagram illustrating an example of sharing settings according to an embodiment of the present disclosure. FIG. It is a figure for demonstrating the event information in one Embodiment of this indication. It is a figure for demonstrating the scenario information in one Embodiment of this indication. It is a figure for demonstrating the scenario information in one Embodiment of this indication. FIG. 4 is a diagram for describing content reproduction using scenario information according to an embodiment of the present disclosure. 5 is a diagram for describing generation of thumbnail images and scenario information according to an embodiment of the present disclosure. FIG. 5 is a diagram for describing selection of target content according to an embodiment of the present disclosure. FIG. 4 is a diagram for describing generation of thumbnail images and thumbnail scenarios according to an embodiment of the present disclosure. FIG. 4 is a diagram for describing generation of thumbnail images and thumbnail scenarios according to an embodiment of the present disclosure. FIG. 4 is a diagram for describing generation of thumbnail images and thumbnail scenarios according to an embodiment of the present disclosure. FIG. 4 is a diagram for describing generation of thumbnail images and thumbnail scenarios according to an embodiment of the present disclosure. FIG. 5 is a diagram for describing generation of a highlight scenario according to an embodiment of the present disclosure. FIG. 5 is a diagram for describing generation of a highlight scenario according to an embodiment of the present disclosure. FIG. 5 is a diagram for describing generation of a highlight scenario according to an embodiment of the present disclosure. FIG. 5 is a diagram for describing an entire display when viewing content according to an embodiment of the present disclosure. FIG. It is a figure which shows the example of the normal mode reproduction | regeneration screen in one Embodiment of this indication. It is a figure which shows the example of the highlight mode reproduction | regeneration screen in one Embodiment of this indication. It is a figure which shows the example of the highlight mode reproduction | regeneration screen in one Embodiment of this indication. It is a figure which shows the example of the highlight mode reproduction | regeneration screen in one Embodiment of this indication. It is a figure which shows the example of the highlight mode reproduction | regeneration screen in one Embodiment of this indication. FIG. 10 is a diagram for describing an operation of a system according to another embodiment of the present disclosure. It is a block diagram for demonstrating the hardware constitutions of information processing apparatus.

  Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the present specification and drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted.

The description will be made in the following order.
1. Introduction 2. Overall configuration Share settings4 . Processing in shared server 4-1. Outline 4-2. Details 5. 5. Display when viewing content Supplement

(1. Introduction)
In describing an embodiment of the present disclosure below, related items will be described first.

  According to an embodiment of the present disclosure, a content including an image (still image) or a moving image captured by a user is uploaded to a server, and the user himself / herself views or shares the content with other users. Related to the system.

  The content generated here is, for example, an image or a moving image shot with a digital camera or the like at an event such as an athletic meet or a trip. Such content is stored, for example, as an image or movie file with a file name based on the shooting date and time. For example, in the case of a user who is not very high in IT literacy, or contentious, the content is stored and shared as it is. Even in the case of a user who is not so, in many cases, the file name is changed to include the event name, or the event name tag is attached to each file.

  When sharing such content with other users, viewing of the shared content is not always easy for other users. For example, when content is shared with the file name based on the shooting date and time, other users do not know what event the file was shot at (when the time passed, the content The same applies to the case where the user himself / herself who has taken the image views). Further, even if it is known from the file name or the like what kind of event was taken at the time of the event, the detailed contents are not known.

  For example, suppose that there is content taken by parents at a child's sports day. Some content shows the child, some shows the child's friends, and some simply shows the results of the competition. When sharing this content with grandparents, the grandparents want to watch only the content showing the grandchild (child). However, it is difficult to distinguish, for example, a file name from content that shows a grandchild and content that does not. Also, for example, even if the content showing the grandchild is shown by tagging the file, for example, in the case of video content, it is necessary to perform an uncomfortable work of watching the scene where the grandchild is not reflected while fast-forwarding. Become.

  In such a case, the parents who provide the content may select the content for sharing with the grandparents. However, for that purpose, the content must be reviewed and understood again, which takes a lot of time. It takes time and effort. For example, if the content is a video, the portion where the grandchild is reflected and the portion where the grandchild is not reflected may be mixed in one file. Preparing is not practical for many users. In addition, the part of the content that is of interest varies depending on the user viewing the content (for example, grandchildren (children) if they are grandparents, children and friends if they are children's friends, and friends if they are children themselves) The results of the competition, etc.).

  Therefore, in an embodiment of the present disclosure, in view of such a situation, by automatically extracting content suitable for sharing from content generated by the user, while reducing the effort of the user who provides the content, Realize a more comfortable viewing experience for users viewing content.

(2. Overall configuration)
Hereinafter, an overall configuration of an embodiment of the present disclosure will be described with reference to FIGS. 1 and 2. FIG. 1 is a schematic block diagram illustrating a functional configuration of a system according to an embodiment of the present disclosure. FIG. 2 is a diagram for describing an operation of a system according to an embodiment of the present disclosure.

(2-1. Functional configuration of the system)
Referring to FIG. 1, a system 10 according to an embodiment of the present disclosure includes a shared server 100, a content providing client 200, a content viewing client 300, and a content server 400.

(Shared server)
The shared server 100 is a server installed on the network, and is realized by, for example, an information processing apparatus as described later. The shared server 100 includes a content information acquisition unit 110, a content classification unit 120, a sharing related information acquisition unit 130, a content extraction unit 140, an event information generation unit 150, a frame / scene extraction unit 160, and a thumbnail image extraction unit 170 as functional configurations. , And a scenario information generation unit 180. Each functional configuration is realized by, for example, a CPU (Central Processing Unit), a RAM (Random Access Memory), and a ROM (Read Only Memory) operating according to a program stored in a storage device or a removable recording medium. .

  Note that the shared server 100 is not necessarily realized by a single device. For example, the function of the shared server 100 may be realized by cooperation of resources possessed by a plurality of devices via a network.

  The content information acquisition unit 110 acquires information related to content from the content analysis unit 420 of the content server 400. The content analysis unit 420 analyzes the content uploaded by the user who provides the content, for example, information about the event where the content was photographed, information about the person appearing in the image or video that is the content, etc. Obtained as meta information.

  The content classification unit 120 classifies content based on the information acquired by the content information acquisition unit 110. In this embodiment, content classification is classification of content for each event to be imaged. As will be described later, for example, the content analysis unit 420 of the content server 400 classifies the content for each event using the result of clustering the content based on the distribution of the shooting date and time.

  The sharing related information acquisition unit 130 acquires, from the content providing client 200, the target user who is the content sharing partner and the information on the target subject which is a subject associated with each target user. The target subject is estimated to be an object of interest when the target user views the content among the content subjects. In the above example, the target user is grandparents, and the target subject is a grandchild (child). The target user and the target subject will be described in detail later.

  Furthermore, the sharing related information acquisition unit 130 may acquire information on events that are permitted to be disclosed for each target user. Thus, it is possible to set whether to share content with the target user in units of events without finely setting the content permitted to be shared for each target user. For example, even if the content shows a child, the content shot during the trip with the grandparents is shared only with the grandparents, and the content was shot during the athletic meet. Can be used for sharing settings such as sharing not only with grandparents but also with children's friends.

  Based on the information acquired by the sharing related information acquisition unit 130, the content extraction unit 140 selects content that shows the target subject from content provided by the user (contents acquired by the content information acquisition unit 110). Extract. The content extracted here is also referred to as target content below. For example, when a target user is designated, the content extraction unit 140 extracts content showing a target subject associated with the target user as target content. Therefore, for example, when there are a plurality of target users, the target content extracted by the content extraction unit 140 may differ depending on the specified target user.

  Furthermore, the content extraction unit 140 can extract the target content for each event based on the content classification by the content classification unit 120. Thereby, for example, in the scenario generation described later, it is possible to generate a scenario for each event. In addition, the event information generation unit 150 can generate information for presenting content in units of events. In a case where the sharing related information acquisition unit 130 acquires information specifying an event that is permitted to be released for each target user, the content extraction unit 140 extracts target content for an event corresponding to the specified target user, for example. May be.

  The event information generation unit 150 generates event information based on the result of the content extraction unit 140 extracting the target content for each event, and outputs the event information to the content viewing client 300. The event information is, for example, an event including content that shows a target subject associated with the target user for a target user who views the content (that is, an event for which a scenario can be generated by the scenario information generation unit 180 described later) ). As described above, the event from which the target content is extracted may be limited by the information acquired by the sharing related information acquisition unit 130.

  The frame / scene extraction unit 160 extracts a frame or a scene that satisfies a predetermined condition from the moving image included in the target content. Here, the frame means one of the continuous images constituting the moving image. A scene means a series of frames constituting part or all of the target content. For example, the frame / scene extraction unit 160 extracts, as a target scene, a portion where the target subject is shown from a moving image included in the target content. Further, the frame / scene extraction unit 160 may select a representative scene from the target scenes for each moving image.

Further, the frame / scene extraction unit 160 may extract a frame in which the target subject is shown as a target frame from the moving image included in the target content, and select a representative frame of each moving image from the target frame. Note that an image (still image) included in the target content can be provided to the scenario information generation unit 180 and the thumbnail image extraction unit 170 without being processed by the frame / scene extraction unit 160. Details of the processing of the frame / scene extraction unit 160 will be described later.

  The thumbnail image extraction unit 170 extracts a thumbnail image that summarizes the contents of the event in a short manner using the frame extraction result by the frame / scene extraction unit 160 for the target content extracted for each event by the content extraction unit 140. To do. The thumbnail image may be, for example, a single image or frame (hereinafter also referred to as an event representative image), or an animation in which a plurality of images or frames are combined (hereinafter also referred to as a flip thumbnail). Good. In addition, as the thumbnail, a moving image (hereinafter also referred to as a thumbnail moving image) including an image and a representative scene of the moving image can be generated. This is generated by the function of the scenario information generating unit 180 as described later. .

  Here, when the thumbnail image extraction unit 170 extracts the event representative image, the thumbnail image extraction unit 170 includes the representative frame extracted by the frame / scene extraction unit 160 and the image (still image) included in the target content. It can also be said to be a representative image selection unit that selects an event representative image from the inside. For example, when the content includes only a moving image, the event representative image is selected from the representative frames.

  On the other hand, when the thumbnail image extraction unit 170 generates a flip thumbnail, the thumbnail image extraction unit 170 combines the representative frame extracted by the frame / scene extraction unit 160 and an image (still image) included in the target content. It can also be said to be an animation generation unit that generates a digest animation (a flip thumbnail). For example, when the content includes only a moving image, the flip thumbnail is generated by combining representative frames.

Thumbnail image extracted by the thumbnail image extraction unit 170, for example, be provided in the form of image data in the event information generator 150, is output together with the event information to the content viewing client 300. In the content viewing client 300, thumbnail content is presented together with event information to target users who view the content, and for example, it is easy to grasp the content of the content classified into each event.

  The scenario information generation unit 180 generates a scenario for reproducing the digest content by combining the content with respect to the target content extracted by the content extraction unit 140, and outputs the scenario to the content viewing client 300 as scenario information. As will be described later, the scenario information is used by the content acquisition unit 310 of the content viewing client 300 to access the content server 400 to acquire content and reproduce it as digest content. The scenario information generation unit 180 may use the frame or scene extraction result by the frame / scene extraction unit 160 in generating the scenario information.

  Further, the scenario information generation unit 180 can generate a scenario for the content set as the sharing target regardless of before and after the information is acquired by the sharing related information acquisition unit 130. That is, when sharing is set, the scenario information generation unit 180 generates a scenario for content that has been set as a sharing target before, and also generates a scenario for content that has been added to the sharing target after that. Generate.

  Here, the digest content includes, for example, a highlight video and a thumbnail video. The highlight moving image is a moving image that is reproduced by combining the target scene extracted from the moving image included in the target content by the frame / scene extraction unit 160 and the image (still image) included in the target content. For example, when the target content includes only a moving image, the highlight moving image is a moving image that continuously reproduces the target frame portion of each moving image. Hereinafter, a scenario for reproducing the highlight moving image is also referred to as a highlight scenario. The highlight video that is played back by the content viewing client 300 using the highlight scenario is used for viewing by the target user, for example.

  On the other hand, the thumbnail moving image is a moving image in which the representative scene selected from the target scene extracted from the moving image included in the target content by the frame / scene extraction unit 160 and the image (still image) included in the target content are combined. For example, when the target content includes only a moving image, the highlight moving image is a moving image that continuously reproduces the representative frame portion of each moving image. Hereinafter, a scenario for configuring such a thumbnail video is also referred to as a thumbnail scenario. For example, when the target user selects content to be viewed while viewing the event information, the thumbnail video that is reproduced using the thumbnail scenario by the content viewing client 300 is displayed together with the event information.

  As described above, in this embodiment, the content classification unit 120 classifies content for each event, and the content extraction unit 140 extracts target content for each event. Accordingly, the scenario information generation unit 180 also generates a scenario for each event.

(Content offer client)
The content providing client 200 is a client connected to the shared server 100 via a network, and is realized by an information processing apparatus as described later, for example. The content providing client 200 includes an operation unit 210, a display control unit 220, and a display unit 230 as functional configurations. More specifically, the content providing client 200 can be, for example, a desktop, notebook, or tablet PC that the user has, a television receiver or recorder having a network communication function, a smartphone, a mobile media player, or the like. However, the present invention is not limited thereto, and may be various devices that can have the above-described functional configuration.

  The operation unit 210 is realized by various input devices provided in the content providing client 200 or connected as an external connection device, and acquires a user operation on the content providing client 200. The operation unit 210 includes a pointing device such as a touch pad or a mouse, for example, and may provide a user with a GUI (Graphical User Interface) in cooperation with the display control unit 220 and the display unit 230 described later.

  Here, the user operation acquired by the operation unit 210 includes an operation of setting a target user who is a target to share content and a target subject associated with each target user. Note that the user who operates the content providing client 200 is a user who provides content, and in many cases, is a user who has taken the content. If it says in the above-mentioned example, the operation part 210 will acquire operation | movement which associates grandchild (child) as an object subject with the parents designating grandparents as an object user, and also grandparents who are object users. An example of such an operation will be described later together with an example of a GUI component.

  The display control unit 220 is realized by, for example, a CPU, a RAM, and a ROM operating according to a program stored in a storage device or a removable recording medium, and controls display on the display unit 230. As described above, the display control unit 220 may cause the display unit 230 to display the GUI component operated via the operation unit 210. An example of such a GUI component will be described later.

  The display unit 230 is, for example, a display device such as an LCD (Liquid Crystal Display) or an organic EL (Electro-luminescence) display that the content providing client 200 has as an output device or is connected to the content providing client 200 as an external connection device. It is realized by. The display unit 230 displays various images under the control of the display control unit 220.

(Content viewing client)
The content viewing client 300 is a client connected to the shared server 100 and the content server 400 via a network, and is realized by an information processing apparatus as described later, for example. The content viewing client 300 includes a content acquisition unit 310, a display control unit 320, a display unit 330, and an operation unit 340 as functional configurations. More specifically, the content viewing client 300 can be, for example, a desktop, notebook, or tablet PC that the user holds, a television receiver or recorder having a network communication function, a smartphone, a mobile media player, or the like. However, the present invention is not limited thereto, and may be various devices that can have the above-described functional configuration.

The content acquisition unit 310 is realized by, for example, a CPU, a RAM, and a ROM operating according to a program stored in a storage device or a removable recording medium. The content acquisition unit 310 acquires the scenario information output from the scenario information generation unit 180 of the shared server 100, and is necessary for reproducing, for example, a highlight video or a thumbnail video desired by the user based on the scenario information. Content is acquired from the content server 400. Furthermore, the content acquisition unit 310 reproduces a highlight video or a thumbnail video using the acquired content based on the scenario information. For example, the content acquisition unit 310 reproduces the acquired content in the order defined by the scenario information, and reproduces the highlight video or the thumbnail video by seeking and reproducing only the target scene or the representative scene. May be. In addition, the content acquisition unit 310 arranges the acquired content in the order defined by the scenario information, and for the video, generates a highlight video or a thumbnail video by editing processing that extracts only the target scene or the representative scene, and plays it back May be. The content acquisition unit 310 provides the reproduced moving image data to the display control unit 320.

  The display control unit 320 is realized by, for example, a CPU, a RAM, and a ROM operating according to a program stored in a storage device or a removable recording medium, and controls the display on the display unit 330. The display control unit 320 causes the display unit 330 to display the highlight video provided from the content acquisition unit 310. Further, the display control unit 320 acquires the event information output from the event information generation unit 150 of the shared server 100, and causes the display unit 330 to display, for example, an event list in which content exists based on the event information. At this time, the event representative image and flip thumbnails output together with the event information are displayed together with the event list. When a thumbnail video is generated for the event, the display control unit 320 displays the thumbnail video acquired by the content acquisition unit 310 together with the event list. Note that examples of display of a highlight video and an event list will be described later. In addition, the display control unit 320 may cause the display unit 330 to display a GUI component operated via the operation unit 340. An example of such a GUI component will also be described later.

  The display unit 330 is realized by, for example, a display device such as an LCD or an organic EL display that the content viewing client 300 has as an output device or is connected to the content viewing client 300 as an external connection device. The display unit 330 displays various images according to the control of the display control unit 320.

  The operation unit 340 is realized by various input devices provided in the content viewing client 300 or connected as an external connection device, and acquires a user operation on the content viewing client 300. The operation unit 340 may include a pointing device such as a touch pad or a mouse, for example, and may provide a user with a GUI operation in cooperation with the display control unit 320 and the display unit 330 as described above.

(Content server)
The content server 400 is a server installed on the network, and is realized by an information processing apparatus as described later, for example. The content server 400 includes a content DB 410 and a content analysis unit 420 as functional configurations.

  Note that the content server 400 is not necessarily realized by a single device. For example, the function of the content server 400 may be realized by cooperation of resources possessed by a plurality of devices via a network. Further, the content server 400 may not be separate from the shared server 100, and at least a part of functions may be realized by the same device as the shared server 100.

  The content DB 410 is realized by a storage device, for example, and stores content uploaded by a user who provides the content. The stored content is analyzed, for example, by the content analysis unit 420 and accessed by the content acquisition unit 310 of the content viewing client 300.

  The content analysis unit 420 is realized by, for example, a CPU, a RAM, and a ROM operating according to a program stored in a storage device or a removable recording medium, and analyzes the content stored in the content DB 410. For example, the content analysis unit 420 detects a person appearing in a content image or a moving image as a subject by analyzing the feature amount of the image. Further, the content analysis unit 420 may cluster the content based on, for example, the distribution of the shooting date and time of the content, and specify the event where the content was shot. The content analysis unit 420 provides the analysis result to the content information acquisition unit 110 of the shared server 100 as, for example, content meta information.

(2-2. System operation)
Next, an example of the operation of the system 10 according to the embodiment of the present disclosure described above will be further described with reference to FIG. Below, it demonstrates, showing the corresponding functional structure of FIG. 1 for every operation | movement of S1-S11.

  First, a user who provides content uploads an image or a moving image to a storage using a predetermined application (S1). Note that the application for uploading can have the same configuration as that used for uploading existing content, so the corresponding functional configuration is not shown in FIG. This application may be executed by the content providing client 200, for example, in the same manner as the sharing setting (S2), or may be executed by another device, for example, a digital camera that acquires content. The storage corresponds to the content DB 410 of the content server 400.

  Next, the user who provides the content executes sharing setting from the client device using the sharing setting application (S2). By this sharing setting, for example, a target user who is a target to share content and a target subject associated with each target user are set. Here, in the present embodiment, the target subject is a person for whom a face cluster has been generated by image analysis (face clustering) of content that has been captured and stored so far. The sharing setting application is an application provided via the operation unit 210, the display control unit 220, and the display unit 230 of the content providing client 200, for example.

  The result of sharing setting is reflected in the user DB and group DB on the server side. Here, for example, information on a target user set by the user is stored in the user DB. Further, in the group DB, information on a target subject to be associated is stored for each target user group set by the user as will be described later. Note that the above-described sharing setting may be executed after the image / video upload or content analysis.

  On the other hand, content analysis is executed on the server side for the uploaded image or video (S3). The content analysis executed here may include, for example, detection of a subject by analyzing the feature amount of the image and identification of an event by clustering based on the distribution of the shooting date / time of the content. The content analysis corresponds to the function of the content analysis unit 420 of the content server 400, for example.

  The result of the content analysis (S3) is input to the content DB as content meta information (S4). This input of meta information corresponds to the function of the content information acquisition unit 110 of the shared server 100, for example.

  Here, in the content DB, for example, meta information for each content is stored. The information stored in the content DB, the user DB, and the group DB so far is combined using, for example, the ID of the user who provides the content as a key, and is used by the scenario creation module (S5). The scenario creation module corresponds to the functions of the content classification unit 120, content extraction unit 140, event information generation unit 150, frame / scene extraction unit 160, thumbnail image extraction unit 170, and scenario information generation unit 180 of the shared server 100, for example. .

  First, the scenario creation module generates event information (S6). As described above, the event information is information for presenting an event including a content showing a target subject associated with the target user to the target user who views the content. The target user who views the content acquires event information using the scenario player application (S7). The scenario player application is an application provided via the content acquisition unit 310, the display control unit 320, the display unit 330, and the operation unit 340 of the content viewing client 300, for example.

  On the other hand, the target user who views the content selects a desired event from the event information using the scenario player application. In response to this, the scenario creation module generates, for example, a highlight movie scenario (S8). The scenario player application acquires the generated scenario (S9), and accesses the content stored in the storage based on the scenario (S10). Further, the scenario player application generates a user-desired moving image such as a highlight moving image from the accessed content and reproduces it (S11).

(3. Sharing settings)
Next, the sharing setting according to an embodiment of the present disclosure will be described with reference to FIGS. 3A to 3D are diagrams illustrating an example of sharing settings according to an embodiment of the present disclosure.

  For example, in the system 10 described with reference to FIG. 1, the sharing setting is executed using a sharing setting application provided via the operation unit 210, the display control unit 220, and the display unit 230 of the content providing client 200. The Accordingly, the sharing setting procedure described below as an example may use a GUI displayed on the display unit 230 by the display control unit 220 and operated via the operation unit 210, for example. The sharing setting described here corresponds to the sharing setting (S2) in the operation described with reference to FIG.

  FIG. 3A shows an example of a procedure for setting a target user to share content. In the illustrated example, the user U included in the target user list L1 is set as the target user. First, as shown in (a), users U1 (grandparents), U2 (daddy's friend A), and U3 (daddy's friend B) are added to the target user list L1 (daddy's friend list). In this embodiment, the addition to the target user list L1, that is, the setting as the target user requires the approval of the added user. Therefore, as shown in (b), immediately after the addition, all of the users U1 to U3 in the target user list L1 are displayed as unapproved users. Thereafter, when the users U1 (grandparents) and U3 (papa's friend B) approve the addition to the target user list L1, as shown in (c), the users U1 (grandparents) and U3 ( Daddy's friend B) is displayed as an approved user.

  FIG. 3B shows an example of a procedure for setting and grouping target users. In the illustrated example, first, a user U0 (a dad = a user who executes sharing settings) and a user U3 (a daddy's friend B) each create an account for a service that provides content sharing. Next, as described with reference to FIG. 3A, the user U0 displays the target user list L1 (daddy's friend list), and adds the user U3 thereto. At this time, the user U3 in the target user list L1 is displayed as an unapproved user.

  On the other hand, the sharing setting application transmits a notification (invitation) that the user U3 has been added to the target user list L1 by the user U0. User U3 receives the invitation using an appropriate application and accepts it. The share setting application that has received the notification of acceptance validates the user U3 added to the target user list L1. That is, the user U3 in the target user list L1 is displayed as an approved user.

  Next, the user U0 creates a target user group G1 (golf companion Gp), and adds the user U3 from the target user list L1 to the group G1. As a result, the user U3 is classified into the target user group G1. Furthermore, the user U0 adds information on the person F1 (papa's friend B, that is, the user U3) to the target user group G1. The information on the person added here corresponds to information on the target subject associated with the target user classified into the target user group G1. As a result, the user U3 (daddy's friend B) can use the appropriate application for the content that the user U0 (daddy) has set as the sharing target and the person F1, that is, the user U3 himself / herself is shown. Can be shared.

  In addition, said procedure can be suitably changed, for example by applying another application. For example, the target user list L1 may be created by the user as described above, or may be created by importing friend settings of other services such as SNS (Social Network Service). Therefore, the sharing setting application does not necessarily need to execute all the processes related to the setting of the target user shown in the above example.

  FIG. 3C shows an example of a procedure for associating person information with a group of target users. In the illustrated example, the target user group G2 (grandparent group) into which the user U1 (grandparent) is classified is set by the procedure as shown in FIGS. 3A and 3B. For example, the sharing setting application provides a GUI for associating information on the desired person F from the person list FL (person list) to the target user group G2. In the illustrated example, persons F2 to F7 are displayed in the person list FL. The user who executes the sharing setting associates the information of the person F with the target user group G2 by dragging the desired person F from the person list FL to the area of the target user group G2, for example.

  In the illustrated example, persons F2 (Hanako) and F3 (Taro) are dragged to the area of the target user group G2 (grandparent group). As a result, persons F2 (Hanako) and F3 (Taro) are associated as target subjects with the user U1 (grandparents) classified into the target user group G2. As a result, the user U1 (grandparents) can share content showing the persons F2 (Hanako) and F3 (Taro). In this way, grouping target users makes it easy to set subjects common to a plurality of target users. In addition, similar to the target user, subjects may be grouped.

  FIG. 3D shows another example of a procedure for associating person information with a group of target users. In the illustrated example, the target user groups G1 to G4 are set by the procedure as shown in FIGS. 3A and 3B. In the sharing setting application, a plurality of target user groups G can be set as described above. In this case, for example, the target user groups G1 to G4 are arranged around the person list FL (person list), and the user who executes the sharing setting selects a desired person F in the person list FL as one of the target user groups G. An operation of dragging to the area may be executed.

  In the illustrated example, as in the example of FIG. 3C, by dragging persons F2 (Hanako) and F3 (Taro) to the area of the target user group G2 (grandparent group), the user U1 (the target user) The persons F2 (Hanako) and F3 (Taro) are associated with the grandparents as target subjects. In the same manner, the user who executes the sharing setting drags the person F5 (papa's friend A) to the target user group G1 (papa's friend group), for example. In addition, the persons F2 (Hanako) and F3 (Taro) are dragged to the target user group G4 (family group). As described above, the setting of the person F as the target subject may be different between the target user groups G, or a part or all of them may overlap.

(4. Processing on shared server)
(4-1. Overview)
Next, an overview of processing in the shared server 100 according to an embodiment of the present disclosure will be described with reference to FIGS. FIG. 4 is a diagram for describing event information according to an embodiment of the present disclosure. 5A and 5B are diagrams for describing scenario information according to an embodiment of the present disclosure. FIG. 6 is a diagram for describing content reproduction using scenario information according to an embodiment of the present disclosure.

(Processing related to event information)
FIG. 4 shows processing related to event information. The illustrated process corresponds to the functions of the content classification unit 120, the content extraction unit 140, and the event information generation unit 150 of the shared server 100 in the system 10 described with reference to FIG. Further, the illustrated processing corresponds to the generation of event information (S6) and the acquisition of event information (S7) in the operation described with reference to FIG.

  In the illustrated example, event information for the user U1 (grandparent) is generated. Persons F2 (Hanako) and F3 (Taro) are associated with the target user group G2 (grandparent group) to which the user U1 belongs. That is, the person F2, F3 is associated with the user U1 who is the target user as the target subject. In the shared server 100, the sharing related information acquisition unit 130 acquires these pieces of information. Such information of the persons F2 and F3 (information of the target subject) can be a first input for generating event information.

  On the other hand, for example, content provided by the user U0 (papa = user who executes sharing setting) is clustered by the content analysis unit 420 described with reference to FIG. In the illustrated example, the content is clustered into three events I1 to I3. In shared server 100, content classification unit 120 classifies content according to the result of clustering. The content classified into the events I1 to I3 in this way can be a second input for generating event information.

Using these inputs, event information Iinfo for user U1 (grandparents) is generated. In the shared server 100, first, the content extraction unit 140 extracts content including persons F2 (Hanako) and F3 (Taro) as subjects from the contents classified into the events I1 to I3. As a result, contents including person F2 (Hanako) and F3 (Taro) as subjects are extracted at events I1 (athletic meet) and I2 (family trip). On the other hand, in the event I3 (golf), only content including the persons F0 (daddy) and F5 (daddy's friend A) as subjects was extracted.

  In response to this result, the event information generation unit 150 generates event information Iinfo including information on the events I1 (athletic meet) and I2 (family trip) and presents them to the user U1 (grandparents) via the content viewing client 300. . Thus, the user U1 can easily select and view the content showing the subject of interest (Hanako, Taro) on an event basis.

  As described above, the event I included in the event information Iinfo may be limited, for example, by the setting of the user U0 (papa = a user who executes sharing setting). For example, when the event I2 (family trip) is a secret event for the user U1 (grandparents), the user U0 is set not to include the event I2 in the event information Iinfo presented to the user U1. Is also possible. For such setting, the content providing client 200 may be provided with a preview function of event information for each target user, for example.

(Processing related to scenario information)
FIG. 5A shows processing related to scenario information. The illustrated process corresponds to the functions of the content extraction unit 140, the frame / scene extraction unit 160, and the scenario information generation unit 180 of the shared server 100 in the system 10 described with reference to FIG. The illustrated process corresponds to scenario generation (S8) and scenario acquisition (S9) in the operation described with reference to FIG.

  In the illustrated example, scenario information for the user U1 (grandparent) is generated as a continuation of the above event information example. Persons F2 and F3 are associated with the target user, user U1 (grandparents) as target subjects. In the shared server 100, the sharing related information acquisition unit 130 acquires these pieces of information. Such information of the persons F2 and F3 (information of the target subject) can be a first input for generating scenario information.

  On the other hand, for example, it is assumed that a user U1 (grandparents) presenting events I1 (athletic meet) and I2 (family trip) as event information Iinfo selects event I1 (athletic meet). In that case, the content extraction unit 140 and the frame / scene extraction unit 160 execute processing for the content belonging to the event I1. In the illustrated example, a moving image A and a moving image B are shown as two contents belonging to the event I1. Content belonging to such an event I can be a second input for generating scenario information.

  Using these inputs, the frame / scene extraction unit 160 extracts a scene in which at least one of the persons F2 and F3 is shown from the moving picture A and the moving picture B. In the illustrated example, scenes A-2, A-3, and A-5 are extracted from the moving image A, and a scene B-2 is extracted from the moving image B, respectively. The scenario information generation unit 180 combines these scenes to form a highlight movie.

  However, in the illustrated example, the scenario information generation unit 180 does not output the highlight moving image as it is, but outputs it as a highlight scenario HS. As will be described later, the highlight scenario HS is information used by the content viewing client 300 to access the content and obtain a highlight moving image. In the illustrated example, the highlight scenario HS is shown as an xml file indicating the location of the content and the start and end of the scene, but what is the format of the highlight scenario HS? May be.

  FIG. 5B shows the process related to the scenario information shown in FIG. 5A in more detail. Similar to FIG. 5A, the first input for generating scenario information is information on the persons F2 and F3 (information on the target subject). The second input is content belonging to the event I1 (athletic meet). However, in the example of FIG. 5B, unlike the example of FIG. 5A, the event I1 includes six contents of image A, image B, image C, movie D, movie E, and movie F.

  Using these inputs, first, the content extraction unit 140 extracts content including persons F2 and F3 as subjects from content belonging to the event I1. In the illustrated example, the images A, C, D, E, and F include persons F2 and F3 as subjects. Therefore, the content extraction unit 140 extracts the image A, the image C, the moving image D, the moving image E, and the moving image F as target content. On the other hand, the image B that does not include the persons F2 and F3 as subjects is not used in the subsequent processing related to scenario information generation.

  Next, the frame / scene extraction unit 160 extracts scenes in which the persons F2 and F3 are shown for the moving picture D, the moving picture E, and the moving picture F. In the illustrated example, scenes D-2, D-3, D-5, and D-8 from the video D, scenes E-2 from the video E, scenes F-1 and F-4 from the video F, respectively. Extracted.

  In the illustrated example, the scenario information generation unit 180 generates two types of scenario information from the images and scenes extracted above. One is the highlight scenario HS described in FIG. 5A above. The highlight scenario HS is a scenario for obtaining a highlight moving image in which all extracted images and scenes are arranged in an order such as time series.

  The other is a thumbnail scenario TS. The thumbnail scenario TS is a scenario for further extracting images and scenes satisfying a predetermined condition from the extracted images and scenes and obtaining a thumbnail moving image in which the images and scenes are arranged. In the illustrated example, from the extracted images and scenes, scenes D-3, E-2, and images that are images and scenes (indicated by smile marks) with high smile levels of the people F2 and F3 that are shown C is extracted to constitute a thumbnail video.

(Reproduction of contents using scenario information)
FIG. 6 shows a process of reproducing content using scenario information output by the process shown in FIG. 5A, for example. The illustrated processing corresponds to the function of the content acquisition unit 310 of the content viewing client 300 in the system 10 described with reference to FIG. Further, the illustrated processing corresponds to content access (S10) and reproduction (S11) in the operation described with reference to FIG.

  In the illustrated example, a highlight movie including scenes A-2, A-3, and A-5 of movie A and scene B-2 of movie B is defined by the highlight scenario HS that is scenario information. . The content acquisition unit 310 that has acquired the highlight scenario HS accesses the content entities of the moving image A and the moving image B stored in the content server 400, for example, and acquires the moving image including the scene specified by the highlight scenario HS. . Further, the content acquisition unit 310 seeks and reproduces the designated scene, thereby providing the display control unit 320 with a moving image in which the acquired scenes are arranged on the time axis as shown in the figure, thereby highlighting the scene. The moving image is displayed on the display unit 330.

  Here, the content acquisition unit 310 may acquire the entire moving image A and moving image B from the content server 400, for example, and may perform editing processing of the moving image on the client side to reproduce the highlight moving image. Alternatively, the content acquisition unit 310 may transmit the highlight scenario HS to the content server 400, edit the video on the content server 400 side based on this, and provide the content viewing client 300 with, for example, streaming.

(4-2. Details)
Next, details of processing in the shared server 100 according to an embodiment of the present disclosure will be described with reference to FIGS. 7 to 10C. FIG. 7 is a diagram for describing generation of thumbnail images and scenario information according to an embodiment of the present disclosure. FIG. 8 is a diagram for describing selection of target content according to an embodiment of the present disclosure. 9A to 9D are diagrams for describing generation of thumbnail images and thumbnail scenarios according to an embodiment of the present disclosure. 10A to 10C are diagrams for describing generation of a highlight scenario according to an embodiment of the present disclosure.

(About generation of thumbnail images and scenario information)
FIG. 7 shows the overall flow of the processing described below with reference to FIGS. 8 to 10C, and the reference numerals (S101 to S108) of the processing in FIG. 7 denote the respective processing in FIGS. 8 to 10C. Corresponds to the sign of.

  Referring to FIG. 7, first, when outputting one image or frame (event representative image) as a thumbnail image, the content extraction unit 140 selects the target content (S101), and the frame / scene extraction unit 160 is extracted. A score is extracted for the target content (S102). Based on this result, the thumbnail image extraction unit 170 selects an event representative image (S103).

  Also, when outputting an animation (a flip thumbnail) consisting of a plurality of images or frames as thumbnail images, the frame / scene extraction unit 160 extracts the score for the target content (S102), as in the case of the event representative image. The thumbnail image extraction unit 170 generates a flip thumbnail (S104).

  On the other hand, when generating a thumbnail scenario, the frame / scene extraction unit 160 extracts a score for the target content in the same manner as in the above two cases (S102), and then executes the process of scene cutout A (S105). Based on this result, the scenario information generation unit 180 generates a thumbnail scenario (S106).

  When generating a highlight scenario, after the content extraction unit 140 selects the target content (S101), the frame / scene extraction unit 160 executes the process of scene cutout B (S107). Based on this result, the scenario information generation unit 180 generates a highlight scenario (S108).

(About selection of target content)
FIG. 8 is a diagram showing in more detail an example of target content selection (S101) by the content extraction unit 140. FIG. In the illustrated example, as in the example of FIG. 5B described above, the information of the persons F2 and F3 (information on the target subject) as the first input, the images A to C belonging to the event I1 as the second input, and Six contents of moving pictures D to F are given respectively.

  In the selection of the target content (S101), content in which the persons F2 and F3 are shown as subjects is picked up from the content given as input. For the images A to C, the content extraction unit 140 acquires information on the subject shown in each image from the content information acquisition unit 110, for example, and selects the persons F2 (Hanako) and F3 (Taro) shown as subjects. To detect. In the illustrated example, it is detected that the person F2 (Hanako) is shown in the image A, and both the persons F2 (Hanako) and F3 (Taro) are shown in the image C.

  On the other hand, for the moving images D to F, the content extraction unit 140 acquires captured images (frames) at a predetermined frame rate from the respective moving images, and obtains information on the subject reflected in each frame, for example, the content information acquiring unit 110. And the persons F2 and F3 appearing as subjects are detected. In the illustrated example, as a result of the capture image being acquired at the frame rate of 1 fps, the person F2 (Hanako) is in frame D # 1 of the moving picture D, the person F3 (Taro) is in frame D # 3, and the frame D # 9, It is detected that both of the persons F2 (Hanako) and F3 (Taro) are shown in D # 10. Further, it is detected that the person F2 (Hanako) is reflected in the frames F # 1 and F # 5 of the moving image F.

  As a result of the above detection, the content extraction unit 140 includes four images, that is, an image A, an image C, a moving image D, and a moving image F as content in which at least one of the persons F2 (Hanako) and F3 (Taro) is reflected. Select. On the other hand, the image B and the moving image E in which neither the person F2 (Hanako) nor F3 (Taro) is shown are used for the subsequent generation of thumbnail images and scenario information. However, the image B and the moving image E are not necessarily unnecessary contents. For example, if another subject (for example, Hanako's friend) is shown in the moving image E, the moving image E can be selected by the content extraction unit 140 when the target user is Hanako's friend or Hanako himself.

(About generation of thumbnail images and thumbnail scenarios)
FIG. 9A is a diagram illustrating an example of score extraction (S102) by the frame / scene extraction unit 160 in more detail. In the illustrated example, as a continuation of the example of FIG. 8 described above, the image A, the image C, the moving image D, and the moving image F selected by the content extraction unit 140 are given as inputs.

  In the score extraction (S102), the content unit score is set based on a predetermined standard for the content given as input. In the illustrated example, smile levels (indicated by numbers with smile marks) of the persons F2 (Hanako) and F3 (Taro) included as target subjects are extracted as the scores of the respective contents. It should be noted that any known technique such as the technique described in Japanese Patent Application Laid-Open No. 2008-311819 can be applied to smile level detection.

  In the illustrated example, the frame / scene extraction unit 160 performs smile level detection as it is for the image A and the image C, and sets the smile level as a content score. On the other hand, for the moving image D and the moving image F, similar to the processing in the content extraction unit 140, a captured image (frame) is acquired at a predetermined frame rate, and smile level detection is executed for a frame in which the target subject is shown. That is, in the illustrated example, among the captured images acquired at the frame rate of 1 fps, the frames D # 1, D # 3, D # 9, and D # 10 of the moving image D and the frames F # 1 and F of the moving image F are captured. Smile level detection is executed for # 5. The frame / scene extraction unit 160 sets, for each moving image, the highest smile level acquired in each frame as the content score.

  Further, in the illustrated example, when there are a plurality of target subjects, the frame / scene extraction unit 160 performs smile level detection for each target subject. That is, the frame / scene extraction unit 160 for the image F in which both the persons F2 (Hanako) and F3 (Taro) are shown, and the frames D # 9 and D # 10 of the moving picture D, the person F2 (Hanako). Both the smile level and the smile level of the person F3 (Taro) are detected. The degree of smile detected in this way is indicated as “70/30” for the image C, for example. This indicates that the smile degree for the person F2 (Hanako) is 70 and the smile degree for the person F3 (Taro) is 30.

  Based on the result of the score extraction as described above, the frame / scene extraction unit 160 determines the score of each content. Further, when the content is a moving image, the frame / scene extraction unit 160 may set a frame corresponding to the score as a representative frame of the content. In the illustrated example, the frame / scene extraction unit 160 sets the smile level 10 of the person F2 (Hanako) for the image A and the smile level 70/30 of the person F2 (Hanako) / F3 (Taro) for the image C. For movie D, representative frame D # 1 and person F2 (Hanako) smile level 100 are set, and for movie F, representative frame F # 5 and person F2 (Hanako) smile level 80 are set.

  Note that the score and the representative frame extracted here affect the selection of, for example, an image or a moving image displayed as a thumbnail. Therefore, the frame / scene extraction unit 160 adjusts the setting in consideration of the balance between subjects, for example. May be. For example, the frame / scene extraction unit 160 may preferentially select a frame including both the person F2 (Hanako) and the person F3 (Taro) as the representative frame. In this case, for example, in the moving image D, the frame D # 1 in which both the person F2 (Hanako) and the person F3 (Taro) are shown is shown, not the frame D # 1 in which the person F2 (Hanako) is shown alone and the smile level is high. # 9 or frame D # 10 is set as the representative frame, and the smile level 50/90 of person F2 (Hanako) / F3 (Taro) is set as the content score.

  FIG. 9B is a diagram illustrating in more detail an example of event representative image selection (S103) and flip thumbnail generation (S104) by the thumbnail image extraction unit 170. In the illustrated example, as a continuation of the example of FIG. 9A described above, the score of the content (image A, image C, moving image D, and moving image F) set by the frame / scene extraction unit 160 and the information of the representative frame are as follows. Given as input.

  In event representative image selection (S103), an image with the highest score set by the frame / scene extraction unit 160 or a frame of a moving image is set as the representative image of the event. In the illustrated example, the smile degree 100 of the person F2 (Hanako) in the representative frame D # 1 of the moving image D is the highest score. Therefore, the thumbnail image extraction unit 170 selects the frame D # 1 as a representative image of the event I1 (athletic meet). As described above, the event representative image can be displayed together with the event information in the content viewing client 300, for example.

  On the other hand, in the flip thumbnail generation (S104), an image in which a score is set by the frame / scene extraction unit 160 and an animation in which a representative frame of a moving image is sequentially displayed is generated. In the illustrated example, in the flip thumbnail, frame D # 1 (representative frame) of image A, moving image D, image C, and frame F # 5 (representative frame) of moving image F are sequentially displayed every 5 seconds. The time for displaying each image is not necessarily 5 seconds. These images may be displayed repeatedly. Similar to the event representative image, the flip thumbnail can be displayed together with the event information in the content viewing client 300, for example.

  FIG. 9C is a diagram showing in more detail an example of scene cutout A processing (S105) by the frame / scene extraction unit 160. In the illustrated example, as a continuation of the example of FIG. 9A described above, the score of the content (image A, image C, moving image D, and moving image F) set by the frame / scene extraction unit 160 and the information of the representative frame are as follows. Given as input.

In the process of scene cutout A (S105), the sections before and after the representative frame are cut out as representative scenes for the moving picture contents (moving picture D and moving picture F). In the illustrated example, for example, the following is set as a scene cut-out rule.
-When the frames in which the subject is shown are continuous, these frames are treated as an integrated frame group.
・ If it is within 2 seconds between the frame in which the target subject is shown and the next frame in which the target subject is shown (other target subjects may be shown), these frames are combined into a single frame group. Treat as.
Cut out as a single scene up to 1 second before and after the frame or frame group in which the target subject is shown.
One scene is at least 3 seconds, and if the frame in which the target subject is shown is the first frame or the last frame, cut out up to 2 seconds before and after the frame.

  In the example shown in the drawing, the scene cut-out criterion is set in units of one second, so that the scene cut-out determination is performed using the captured image acquired at the frame rate of 1 fps as in the case of selecting the target content described above. Is possible. For example, when the scene cutout reference is set in units of 0.1 seconds, scene cutout determination can be performed using a captured image acquired at a frame rate of 10 fps.

For the moving image D, since the first frame D # 1 is the representative frame, 3 seconds from the frame D # 1, that is, the frames D # 1 to D # 3 are set as the representative scene D # 1. For the moving image F, since the intermediate frame F # 5 is the representative frame, 3 seconds in 1 second before and after the frame F # 5, that is, the frames F # 4 to F # 6 are set as the representative scene F # 5. Is done.

  The range of the representative scene is set by the captured images having a predetermined frame rate as described above, but the actual representative scene is a moving image section corresponding to these captured images. That is, the representative scene D # 1 is not a three-frame frame, but is a part of a moving image constituted by a whole series of frames between, for example, the frames D # 1 to D # 3.

  For example, the representative scenes D # 1 and F # 5 of the moving images D and F acquired by the scene cutout A processing as described above and the images A and C selected in the target content selection (S101) are, for example, A thumbnail video is obtained by arranging in time series.

  FIG. 9D is a diagram illustrating an example of thumbnail scenario generation (S106) by the scenario information generation unit 180 in more detail. In the illustrated example, as a continuation of the above example of FIG. 9C, the representative scenes D # 1 and F # 5 of the image A, the image C, and the moving images D and F are given as inputs.

  The scenario information generation unit 180 defines the thumbnail video by arranging these contents, for example, in time series. Furthermore, the scenario information generation unit 180 generates a thumbnail scenario TS corresponding to the thumbnail video. The thumbnail scenario TS may be an xml file indicating the location of the content and the start and end of the scene, as in the example of the highlight scenario HS described above, but is not limited thereto.

  The generated thumbnail scenario TS is output to the content viewing client 300. When the content acquisition unit 310 of the content viewing client 300 acquires content according to the thumbnail scenario TS, a thumbnail video as illustrated is reproduced.

  In the example of the thumbnail movie shown in the figure, after the image A is displayed for 3 seconds, the scene D # 1 is reproduced, the image C is further displayed for 3 seconds, and finally the scene F # 5 is reproduced. In the thumbnail video, the image display time does not necessarily have to be 3 seconds. However, in the illustrated example, the lengths of the scene D # 1 and the scene F # 5 are both 3 seconds. Images A and C are also displayed for 3 seconds in accordance with these. Further, the thumbnail video may be reproduced repeatedly.

  As shown in the figure, the thumbnail video can be displayed together with event information (event details), for example, in the content viewing client 300. At this time, information on the subject included in the content (a person who is shown) is added. May be displayed. In the illustrated example, people F2 and F3 are displayed.

(About creating a highlight scenario)
FIG. 10A is a diagram showing in more detail an example of scene cutout B processing (S107) by the frame / scene extraction unit 160. In the illustrated example, as a continuation of the example of FIG. 8 described above, the image A, the image C, the moving image D, and the moving image F selected by the content extraction unit 140 are given as inputs.

In the process of scene cutout B (S107), the section in which the target subject is shown is cut out as a representative scene for the moving picture content (moving picture D and moving picture F). In the illustrated example, for example, the following is set as a scene cut-out rule (the difference from the rule in the scene cut-out A process is indicated by <>).
-When the frames in which the subject is shown are continuous, these frames are treated as an integrated frame group.
・ If the frame between the subject subject and the next subject subject frame (may be another subject subject) is within <5 seconds>, these frames are integrated Treat as a frame group.
Cut out as a single scene up to <2 seconds> before and after the frame or frame group in which the subject is shown.

  A highlight video is generated as a target for viewing by a user, unlike a thumbnail video generated to shorten content. Therefore, it is desirable that the highlight moving image sufficiently includes a portion that is a target of interest of the user who views the content. For this reason, as in the above example, the criteria for scene cutout may be different from those for thumbnail videos.

  For example, if only a frame in which a target subject is shown in a moving image is cut out as a scene, an unnatural highlight moving image may be generated. Therefore, in the scene cutout B process for the highlight video, the interval for handling the frame in which the target subject is shown as an integral frame group and the minimum length of one scene are smaller than in the case of the thumbnail video. Can be set longer. As a result, for example, when the target subject is reflected but not recognized by image analysis, or when the target subject is facing backward, the portion where the target subject is reflected but is not recognized as a person Can be included in the scene to be cut out.

In the illustrated example, in accordance with the above rules, in the moving image D, the frame D # 1 and the frame D # 3 are handled as an integrated frame group, and the frame 2 seconds after the frame D # 3 from the frame D # 1 Up to D # 5 is cut out as scene D # 1. Also, the frame D # 9 and the frame D # 10 are treated as an integrated frame group, and the frame D # 7 from 2 seconds before the frame D # 9 to the frame D # 10 are cut out as the scene D # 2. . On the other hand, in the moving image F, the frame F # 1 and the frame F # 5 are treated as an integrated frame group, and the frame F # 1 to the frame F # 7 that is 2 seconds after the frame F # 5 are the scene F #. Cut out as 1.

  Note that the range of the scene cut out here is set by the captured images having a predetermined frame rate as described above, but the actual scene is a moving image section corresponding to these captured images. That is, the scene D # 1 is not a frame of five frames but a part of a moving image constituted by a whole series of frames from, for example, the frames D # 1 to D # 5.

  10B and 10C are diagrams illustrating an example of the highlight scenario generation (S108) by the scenario information generation unit 180 in more detail. In the illustrated example, as a continuation of the example of FIG. 10A described above, scenes D # 1 and D # 2 cut out from image A, image C, and movie D, and scene F # 1 cut out from movie F are shown. , Given as input.

  As shown in FIG. 10B, the scenario information generation unit 180 defines the highlight moving image by arranging these contents in time series, for example. Further, the scenario information generation unit 180 generates a highlight scenario HS corresponding to the highlight moving image. The highlight scenario HS may be, for example, an xml format file indicating the location of the content and the start and end of the scene, but is not limited thereto.

  As illustrated in FIG. 10C, the generated highlight scenario HS is output to the content viewing client 300. When the content acquisition unit 310 of the content viewing client 300 acquires content according to the highlight scenario HS, a highlight moving image as illustrated is reproduced.

  In the example of the highlighted movie shown in the figure, after the image A is displayed for 3 seconds, the scene D # 1 is reproduced, the scene D # 2 is subsequently reproduced, and the image C is further displayed for 3 seconds. F # 1 is reproduced. In the highlight movie, the time for displaying an image is not necessarily 3 seconds. The time for which the image is displayed may be dynamically set according to, for example, the length of the moving image scene included elsewhere or the entire length of the highlight moving image. Note that the highlight moving image is, for example, intentionally selected and viewed by the user, and therefore unlike the thumbnail moving image, in many cases, it is not repeatedly reproduced.

(5. Display when viewing content)
Next, with reference to FIGS. 11 to 13D, display during content viewing according to an embodiment of the present disclosure will be described. FIG. 11 is a diagram for describing the entire display during content viewing according to an embodiment of the present disclosure. FIG. 12 is a diagram illustrating an example of a normal mode playback screen according to an embodiment of the present disclosure. 13A to 13D are diagrams illustrating examples of a highlight mode reproduction screen according to an embodiment of the present disclosure.

  Referring to FIG. 11, in one embodiment of the present disclosure, for example, a login screen 1100, an event list screen 1200, a normal mode playback screen 1300, and a highlight mode playback screen 1400 are displayed when viewing content. These displays can be displayed on the display unit 330 by the display control unit 320 in the content viewing client 300, for example.

  For example, when the user starts viewing the shared content on the content viewing client 300, the display control unit 320 first displays the login screen 1100 on the display unit 330. The login screen 1100 has an input area for an ID and a password as shown in the figure, for example. Using this login screen, the user logs in to an account of a service that provides content sharing, for example. Thereby, for example, the shared server 100 can identify which target user is using the content viewing client 300.

  If the user login is successful, an event list screen 1200 is displayed. The event list screen 1200 is a screen that displays, for example, event information generated by the event information generation unit 150 of the shared server 100 as a list. In the event list screen, for example, event representative images and flip thumbnails generated by the thumbnail image extraction unit 170 or thumbnail videos acquired according to the thumbnail scenario generated by the scenario information generation unit 180 are displayed together with event information. May be. In this case, whether to display the event representative image, the flip thumbnail, or the thumbnail moving image may be determined based on, for example, the rendering capability of the content viewing client 300.

  Here, the events displayed on the event list screen 1200 may correspond to specific events such as “Athletic meet”, “Birthday”, “Family trip”, for example, As in “2008.05.08-2008.05.09”, it may simply correspond to the shooting date and time range. Since the content is not necessarily shot at a specific event, the event may be defined by the range of shooting date and time as in the illustrated example.

  In the event list screen 1200, information for identifying a user who provides content may be displayed together with an event name (or date). In the illustrated example, an e-mail address such as “aaa@bb.cc” is displayed as information for identifying the user providing the content, but is not limited to this, for example, a user ID, a nickname, or the like. May be displayed.

  When the user selects any event on the event list screen 1200 using, for example, the operation unit 340, reproduction of content corresponding to the event is started. The content reproduced here is, for example, a highlight movie acquired according to a highlight scenario generated by the scenario information generation unit 180 of the shared server 100.

  In the illustrated example, the content is first played on the normal mode playback screen 1300. Here, when the user performs a mode switching operation using the operation unit 340, for example, the display is changed to the highlight mode playback screen 1400 while the playback of the content is continued. Here, between the normal mode playback screen 1300 and the highlight mode playback screen 1400, the display of the progress bar for displaying the progress of the content is different. The display of the progress bar will be described in more detail in the following part.

  In the normal mode playback screen 1300 or the highlight mode playback screen 1400, when the user instructs the end of playback using, for example, the operation unit 340, the playback of content ends and the display returns to the event list screen 1200.

(Normal mode playback screen)
FIG. 12 shows an example of a normal mode playback screen. On the normal mode playback screen 1300, a normal progress bar 1310 is displayed. As the progress bar 1310, any known technique such as the technique described in Japanese Patent Application Laid-Open No. 2008-67207 can be applied. In the illustrated example, the progress bar 1310 displays the entire content to be played. Of the progress bar 1310, the part that has already been reproduced and the part that has not yet been reproduced are displayed in different colors. The boundaries of these differently colored parts correspond to the part currently being played back. The user can jump to an arbitrary position in the content by selecting an arbitrary position of the progress bar 1310 using the operation unit 340.

(Highlight mode playback screen)
FIG. 13A shows an example of a highlight mode playback screen. On the highlight mode playback screen 1400, a progress bar 1410 particularly suitable for playback of highlight content such as a highlight movie is displayed. Here, the highlight content is a content in which a part (first section) of the section included in the original content is reproduced and the other part (second section) is not reproduced, like a highlight video. It is.

  When such highlight content is reproduced, for example, the user may want to view the cut portion. In such a case, for example, if the cut portion is not displayed on the progress bar, it is difficult for the user to recognize the cut portion (whether the portion that the user wants to watch has been cut or not originally exists). do not know). However, if the progress bar is displayed including all the cut parts, there will be many meaningless parts on the progress bar (parts that will not be played if the user does not want to watch), and the display becomes redundant. turn into.

  Therefore, in this embodiment, a progress bar 1410 particularly suitable for reproduction of highlight content is displayed so that the user can view the content more comfortably even in the above case.

  The progress bar 1410 includes a + button 1411 and a − button 1412. Further, on the highlight mode playback screen 1400, a playback position display 1420 and a person display 1430 are further displayed.

  First, the progress bar 1410 is different from the progress bar 1310 displayed on the normal mode playback screen in that it does not necessarily display the entire content to be played back. The progress bar 1410 flows from right to left as the content is played back so that the currently played portion matches the playback position display 1420 (the position is fixed) (center focus). Accordingly, as described later, the + button 1411 and the − button 1412 arranged on the progress bar 1410 appear from the right end of the progress bar 1410 when the user does not perform any particular operation in a state where the content is played back, and the playback position. It passes through the position of the display 1420, flows to the left end of the progress bar 1410, and disappears.

(+ Button /-button)
Next, the + button 1411 and the − button 1412 displayed on the progress bar 1410 will be further described with reference to FIG. 13B.

  The + button 1411 is a button indicating that there is a cut portion in the content being played back. As described above, the content being played back in the illustrated example is a highlight moving image acquired according to a highlight scenario generated by the scenario information generation unit 180, for example. In the highlight moving image, for example, as shown in the example of FIG. 10B, there is a portion that is not extracted as a scene in the moving image.

  Here, it is assumed that the user presses the + button 1411 via the operation unit 340, for example. Then, the + button 1411 changes to a − button 1412, and the cut portion CUT in the content indicated by the + button 1411 is displayed. In FIG. 13A and FIG. 13B, the portions displayed in a color different from the other portions on both sides of the − button 1412 are cut portions CUT. The cut part CUT represents a part (second section) of the original content that is not reproduced as a highlight moving image. Therefore, the cut portion CUT is displayed in a different overview from the other portions of the progress bar 1410, for example, in a different color.

  Therefore, in the progress bar 1410, a part other than the cut part CUT is displayed in succession on the first bar representing the first section of the content, and the cut part CUT is displayed in succession on the first bar to represent the second section. It may be expressed as a second bar. Then, it can be said that the + button 1411 is displayed on the first bar instead of the second bar and indicates that the second section is not reproduced.

  The cut portion CUT displayed together with the − button 1412 is not only displayed but actually reproduced. For example, when the + button 1411 on the progress bar 1410 located on the right side of the reproduction position display 1420 is pressed, the cut portion corresponding to the + button 1411 is displayed as the cut portion CUT and the cut portion CUT. When the position of the playback position display 1420 is reached, the content is played back including the part that was originally cut.

  This is because, for example, in response to an operation in which the content acquisition unit 310 of the content viewing client 300 presses the + button 1411 acquired via the operation unit 340, the data of the cut part of the content is newly added to the content server. It can be obtained by acquiring the data from 400 and providing the data to the display control unit 320.

  By the operation as described above, the total length of the content to be played is extended. However, the progress bar 1410 originally does not correspond to the entire length of the content, and is the center focus. Therefore, for example, in the example of FIG. 13A, even if the + button 1411 is changed to the − button 1412 on the left side of the playback position display 1420 and the cut portions CUT are additionally displayed on both sides of the − button 1412, The display position of another + button 1411 or the like displayed on the right side of the playback position display 1420 does not change, for example, only a part of the left part (the part that is not originally cut) is not displayed.

  On the other hand, when the user presses the − button 1412 via the operation unit 340, for example, the − button 1412 changes to the + button 1411, and the cut portions CUT displayed on both sides of the − button 1412 are not displayed. In this state, the cut portion is not played back as in the original highlight movie, for example. In this case, the total length of the content to be reproduced is reduced. However, as described above, the progress bar 1410 originally does not correspond to the entire length of the content, and is the center focus, so that the change of the − button 1412 to the + button 1411 is, for example, on the progress bar 1410. Thus, the display of the area on the opposite side across the reproduction position display 1420 is not affected.

  In the above example, as a default setting, a highlight moving image is to be played back. Therefore, at the start of playback, playback of content that does not include a cut portion is scheduled. At this time, the cut portion CUT and the − button 1412 are not displayed on the progress bar 1410, and the + button 1411 is displayed for all the cut portions of the content.

(Person display)
Next, an example of the person display 1430 displayed on the highlight mode playback screen 1400 will be further described with reference to FIG. 13C. The person display 1430 indicates the position on the content where the target subject (person) associated with the target user (the user viewing the content in the illustrated example) is reflected in the content sharing setting.

  In the example of FIG. 13C, the person display 1430 is displayed at a position on the content where each person starts to appear. For example, the person display 1430a is displayed at a position where the persons P1 (Hanako) and P2 (Taro) are shown for the first time. Therefore, the person display 1430a includes displays of persons P1 (Hanako) and P2 (Taro). Also, the person display 1430b is displayed at a position where the person P3 (Jiro) is shown for the first time, and after the persons P1 (Hanako) and P2 (Taro) are no longer shown, they are shown again. Therefore, the person display 1430b includes displays of persons P1 (Hanako), P2 (Taro), and P3 (Jiro).

On the other hand, the person display 1430c is displayed at a position where the person P1 (Hanako) once appears and then starts to appear again. At this time, the persons P2 (Taro) and P3 (Jiro) are not included in the person display 1430c because they are continuously reflected from the time when the previous person display 1430b is displayed. Accordingly, the person display 1430c includes only the display of the person P1 (Hanako). In addition, the person display 1430d is displayed at a position where the persons P2 (Taro) and P3 (Jiro) once appear and then start to appear again. At this time, the person P1 (Hanako) is not included in the person display 1430d because the person P1 (Hanako) continues to appear from the time when the previous person display 1430c was displayed. Thus, the person display 1430 d is the person P2 (Taro), including only the display of the P3 (Jiro).

  By the person display 1430 as described above, for example, a user who views the content can recognize the timing at which each person starts to appear. In this way, by displaying the timing of appearance instead of the section in which the image is shown, for example, when viewing content, when a certain person P no longer appears, the position where that person P appears next is searched. It is possible to satisfy the user's request for wanting.

  As another example, the person display 1430 may be displayed for each scene or still image constituting the highlighted moving image being reproduced. As described above, the highlight moving image is configured by a scene or a still image in which the target subject is shown. Therefore, the target subject that appears is often changed for each scene or still image. Therefore, by displaying the person display 1430 for each scene or still image delimited by the + button 1411, it is possible to inform the user which target subject appears in the display of each scene or still image. For example, in the case of a scene, the person display 1430 may be displayed at the start position of the scene (position of the + button 1411), or may be displayed in the middle of the scene (intermediate between the previous and next + buttons 1411).

  Whether or not to display the person display 1430 can be switched by, for example, setting of a scenario player application (see FIG. 2). Further, for example, when the user selects the person display 1430 via the operation unit 340, the reproduction may jump to that place.

  Furthermore, as another example of the person display 1430, as shown in FIG. 13D, when an arbitrary position on the progress bar 1410 is selected via the operation unit 340 or the like, it is reflected in a portion corresponding to the position of the content. The person display 1430 including the display of the person P may be displayed. This display may be possible even when the person display 1430 is set to non-display by the setting of the scenario player application, for example.

(6. Supplement)
(Other embodiments)
In addition, about operation | movement of the system demonstrated above with reference to FIG. 2, other embodiment which is demonstrated with reference to FIG. 14 below, for example is also possible. FIG. 14 is a diagram for describing an operation of a system according to another embodiment of the present disclosure. In this embodiment, the functional configuration of the system is the same as that of the above-described embodiment, and detailed description thereof is omitted.

  FIG. 14 shows operations similar to S1 to S11 described in FIG. However, in this embodiment, each operation is executed by being distributed to a plurality of servers as shown in the figure. In the illustrated example, processing related to the client-side application is executed by the application server. Also, the analysis server that executes content analysis (S3) and the scenario creation server that executes scenario generation (S8) are separate servers. In this case, an analysis connector server may be provided for meta information input (S4) between these servers.

  Note that the operations described above with reference to FIGS. 2 and 14 are only examples of the embodiment of the present disclosure. That is, both FIG. 2 and FIG. 14 show only an example of implementation of the functional configuration of the present disclosure as shown in FIG. 1, for example. As described above, the functional configuration of the present disclosure can be realized by an arbitrary system configuration including, for example, one or a plurality of server devices and one or a plurality of client devices.

(Hardware configuration)
Next, a hardware configuration of the information processing apparatus 900 that can realize the shared server 100, the content providing client 200, the content viewing client 300, the content server 400, and the like according to the embodiment of the present disclosure will be described with reference to FIG. To do. FIG. 15 is a block diagram for explaining a hardware configuration of the information processing apparatus.

  The information processing apparatus 900 includes a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 903, and a RAM (Random Access Memory) 905. Further, the information processing apparatus 900 may include a host bus 907, a bridge 909, an external bus 911, an interface 913, an input device 915, an output device 917, a storage device 919, a drive 921, a connection port 923, and a communication device 925. The information processing apparatus 900 may include a processing circuit such as a DSP (Digital Signal Processor) instead of or in addition to the CPU 901.

  The CPU 901 functions as an arithmetic processing device and a control device, and controls the overall operation or a part of the information processing device 900 according to various programs recorded in the ROM 903, the RAM 905, the storage device 919, or the removable recording medium 927. The ROM 903 stores programs and calculation parameters used by the CPU 901. The RAM 905 primarily stores programs used in the execution of the CPU 901, parameters that change as appropriate during the execution, and the like. The CPU 901, the ROM 903, and the RAM 905 are connected to each other by a host bus 907 configured by an internal bus such as a CPU bus. Further, the host bus 907 is connected to an external bus 911 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 909.

  The input device 915 is a device operated by the user, such as a mouse, a keyboard, a touch panel, a button, a switch, and a lever. The input device 915 may be, for example, a remote control device that uses infrared rays or other radio waves, or may be an external connection device 929 such as a mobile phone that supports the operation of the information processing device 900. The input device 915 includes an input control circuit that generates an input signal based on information input by the user and outputs the input signal to the CPU 901. The user operates the input device 915 to input various data and instruct processing operations to the information processing device 900.

  The output device 917 is configured by a device capable of visually or audibly notifying acquired information to the user. The output device 917 can be, for example, a display device such as an LCD (Liquid Crystal Display), a PDP (Plasma Display Panel), an organic EL (Electro-Luminescence) display, an audio output device such as a speaker and headphones, and a printer device. . The output device 917 outputs the result obtained by the processing of the information processing device 900 as video such as text or an image, or outputs it as audio such as voice or sound.

The storage device 919 is a data storage device configured as an example of a storage unit of the information processing device 900. The storage device 919 is, for example, HDD (Hard Disk Drive) a magnetic storage device such as a semiconductor storage device, an optical storage device, or a magneto-optical storage device. The storage device 919 stores programs executed by the CPU 901, various data, various data acquired from the outside, and the like.

  The drive 921 is a reader / writer for a removable recording medium 927 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and is built in or externally attached to the information processing apparatus 900. The drive 921 reads information recorded on the attached removable recording medium 927 and outputs the information to the RAM 905. In addition, the drive 921 writes a record in the attached removable recording medium 927.

  The connection port 923 is a port for directly connecting a device to the information processing apparatus 900. The connection port 923 can be, for example, a USB (Universal Serial Bus) port, an IEEE 1394 port, a SCSI (Small Computer System Interface) port, or the like. Further, the connection port 923 may be an RS-232C port, an optical audio terminal, an HDMI (High-Definition Multimedia Interface) port, or the like. By connecting the external connection device 929 to the connection port 923, various types of data can be exchanged between the information processing apparatus 900 and the external connection device 929.

  The communication device 925 is a communication interface configured with, for example, a communication device for connecting to the communication network 931. The communication device 925 can be, for example, a communication card for wired or wireless LAN (Local Area Network), Bluetooth (registered trademark), or WUSB (Wireless USB). The communication device 925 may be a router for optical communication, a router for ADSL (Asymmetric Digital Subscriber Line), or a modem for various communication. The communication device 925 transmits and receives signals and the like using a predetermined protocol such as TCP / IP with the Internet and other communication devices, for example. The communication network 931 connected to the communication device 925 is a wired or wireless network, such as the Internet, a home LAN, infrared communication, radio wave communication, or satellite communication.

  The imaging device 933 uses various members such as an imaging element such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) and a lens for controlling the formation of a subject image on the imaging element, for example. It is an apparatus that images a real space and generates a captured image. The imaging device 933 may capture a still image or may capture a moving image.

  The sensor 935 is various sensors such as an acceleration sensor, a gyro sensor, a geomagnetic sensor, an optical sensor, and a sound sensor. The sensor 935 acquires information about the state of the information processing apparatus 900 itself, such as the posture of the information processing apparatus 900, and information about the surrounding environment of the information processing apparatus 900, such as brightness and noise around the information processing apparatus 900, for example. To do. The sensor 935 may include a GPS sensor that receives a GPS (Global Positioning System) signal and measures the latitude, longitude, and altitude of the apparatus.

  Heretofore, an example of the hardware configuration of the information processing apparatus 900 has been shown. Each component described above may be configured using a general-purpose member, or may be configured by hardware specialized for the function of each component. Such a configuration can be appropriately changed according to the technical level at the time of implementation.

(Summary of embodiment)
In the embodiment of the present disclosure, for each content sharing partner, for example, a subject that can be an interest of the sharing partner is set. In accordance with this setting, the shared server automatically generates a scenario for configuring the digest content. The digest content is, for example, content that combines scenes and images that are meaningful to the sharing partner, and is exemplified as a highlight video in the above embodiment. The shared server can automatically generate a scenario or the like for content added after the above settings are made. Therefore, the user who provides the content only needs to add the content to the sharing target without performing any additional operation after setting the sharing once.

  In addition, a thumbnail video may be generated as the digest content. A thumbnail video is content in which scenes and images that are meaningful to a sharing partner are gathered together in a shorter time. The highlight video is mainly used for viewing by the sharing partner, while the thumbnail video is presented to the sharing partner, for example, as the thumbnail of the highlight video, together with information indicating the event to be captured of the content, for example. To make it easier.

  As information for facilitating selection of a viewing target, a thumbnail image may be displayed. In the above embodiment, the thumbnail images are exemplified as the content representative image and the flip thumbnail. Since the thumbnail image is a single image (still image) or an animation composed of a plurality of still images, it can be displayed even when the image processing capability of the client device is low, for example.

  In the above embodiment, the shared server provides the scenario information to the client, and the digest content such as the highlight video is generated on the client side according to the scenario. However, for example, when the image processing capability of the client device is low or the communication state is stable, the shared server or the content server may generate digest content according to the scenario and provide it to the client.

  In addition, an operation for setting a sharing partner or a subject to be associated with the sharing partner can be acquired using, for example, a GUI as described in the above embodiment. For example, the sharing partner and the subject may be displayed as icons, and the sharing partner and the subject may be associated according to a drag operation for associating each icon. As a result, the user can make sharing settings through intuitive operations.

  In addition, when viewing a content in which a part of the original content is extracted (other parts are cut), such as the above-described highlight video, for example, the portion provided as the highlight video is As a result of viewing by the user, it may be desired to view the cut portions before and after that. In such a case, the part extracted using the + button and-button and the cut part are displayed intuitively, and the part cut by the operation of these buttons is played back. If a GUI that can be used is provided, the user's experience of viewing content becomes more comfortable.

  In the above description, the embodiments of the present disclosure mainly related to the information processing apparatus have been introduced. For example, a method executed in these information processing apparatuses, a program for realizing functions in the information processing apparatus, and A recording medium on which such a program is recorded can also be realized as an embodiment of the present disclosure.

  The preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, but the technical scope of the present disclosure is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field of the present disclosure can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that it belongs to the technical scope of the present disclosure.

The following configurations also belong to the technical scope of the present disclosure.
(1) a sharing related information acquisition unit that acquires information on a target user who is a sharing partner of content including a still image or a moving image, and a target subject that is a subject associated with the target user;
A content extraction unit that extracts, as the target content, content in which the target subject appears from the content;
An information processing apparatus comprising: a scenario generation unit configured to generate a scenario for configuring digest content by combining the target content.
(2) a scene extracting unit that extracts, as a target scene, a portion in which the target subject appears from a moving image included in the target content;
The information processing apparatus according to (1), wherein the scenario generation unit generates a scenario for configuring digest content by combining the target scenes.
(3) The information processing apparatus according to (2), wherein the scenario generation unit generates a scenario for configuring digest content by combining the target scene and a still image included in the target content.
(4) The scene extraction unit selects a representative scene of the moving image from the target scene,
The information processing apparatus according to (2) or (3), wherein the scenario generation unit generates a scenario for configuring digest content by combining the representative scenes.
(5) The target subject is a person,
The information processing apparatus according to (4), wherein the scene extraction unit selects the representative scene based on a smile level of the person.
(6) a frame extraction unit that extracts a frame in which the target subject is shown as a target frame from a moving image included in the target content, and selects a representative frame of the moving image from the target frame;
The information processing apparatus according to any one of (1) to (5), further including: an animation generation unit that generates a digest animation of the target content by combining the representative frames.
(7) The information processing apparatus according to (6), wherein the animation generation unit generates the digest animation by combining the representative frame and a still image included in the target content.
(8) a frame extraction unit that extracts a frame in which the target subject is shown as a target frame from a moving image included in the target content, and selects a representative frame of the moving image from the target frame;
The information processing apparatus according to any one of (1) to (5), further including: a representative image selection unit that selects a representative image of the target content from the representative frame.
(9) The information processing apparatus according to (8), wherein the representative image selection unit selects the representative image from the representative frame and a still image included in the target content.
(10) a content classifying unit that classifies the content for each event to be photographed;
The information processing apparatus according to any one of (1) to (9), wherein the scenario generation unit generates the scenario for each event.
(11) The information processing apparatus according to (10), further including an event information generation unit that generates event information including information on the event for which the scenario is generated.
(12) The information processing apparatus according to (10) or (11), wherein the sharing related information acquisition unit further acquires information on the event that is permitted to be disclosed to the target user.
(13) The sharing-related information acquisition unit according to any one of (1) to (12), wherein the sharing-related information acquisition unit acquires information on the target user group and the target subject associated with each group. Information processing device.
(14) The scenario generation unit may acquire information about the target user and the target subject for the content set as a sharing target before the information about the target user and the target subject is acquired. The information processing apparatus according to any one of (1) to (13), wherein the scenario is also generated for the content added to the sharing target.
(15) The information processing apparatus according to any one of (1) to (14), wherein the scenario generation unit outputs the scenario to an external apparatus where the target user views the digest content.
(16) An operation in which a first user who provides content including a still image or a moving image sets a second user who is a share partner of the content and a target subject which is a subject associated with the second user. A first client device including an operation unit to obtain;
A shared related information acquisition unit for acquiring information on the second user and the target subject;
A server device comprising: a content extraction unit that extracts content showing the target subject as target content from the content; and a scenario generation unit that generates and outputs a scenario for configuring digest content by combining the target content When,
A second client device including a content acquisition unit that acquires the output scenario, generates the digest content from the content according to the scenario, and provides the digest content to the second user.
(17) The first client device further includes a display control unit configured to display an icon indicating a candidate subject that is a candidate for the target subject on a display unit,
The system according to (16), wherein the operation unit acquires an operation of the first user who sets the target subject by selecting an icon corresponding to a desired subject from the icons.
(18) acquiring information on a target user who is a partner with whom content including a still image or a moving image is shared, and a target subject that is a subject associated with the target user;
Extracting the content in which the target subject appears from the content as target content;
Generating a scenario for configuring the digest content by combining the target content.

DESCRIPTION OF SYMBOLS 10 System 100 Shared server 110 Content information acquisition part 120 Content classification part 130 Sharing related information acquisition part 140 Content extraction part 150 Event information generation part 160 Frame / scene extraction part 170 Thumbnail image extraction part 180 Scenario information generation part 200 Content provision client 210 Operation unit 220 Display control unit 230 Display unit 300 Content viewing client 310 Content acquisition unit 320 Display control unit 330 Display unit 340 Operation unit 400 Content server 410 Content DB
420 Content analysis part 1410 Progress bar 1411 + button 1412-button 1430 Person display CUT Cut part

Claims (17)

A shared user information acquisition unit that acquires information on a target user who is a partner to share content including a still image or a video, and a target subject that is a subject associated with the target user;
A content extraction unit that extracts, as the target content, content in which the target subject appears from the content;
A scene extraction unit that extracts a target scene from a moving image included in the target content based on a portion where the target subject is shown, and extracts a representative scene based on an extraction criterion that is shorter than the extraction criterion of the target scene; ,
An information processing apparatus comprising: a scenario generation unit that generates a highlight scenario for forming a highlight moving image by combining the target scenes , and generates a thumbnail scenario for combining the representative scene to form a thumbnail moving image . The information processing apparatus according to claim 1 , wherein the scenario generation unit generates the highlight scenario for configuring the highlight moving image by combining the target scene and a still image included in the target content. The scene extraction unit selects the representative scene from the target scene, the information processing apparatus according to claim 1 or 2. The target subject is a person,
The information processing apparatus according to claim 3 , wherein the scene extraction unit selects the representative scene based on a smile level of the person. A frame extraction unit that extracts a frame in which the target subject is shown as a target frame from a moving image included in the target content, and selects a representative frame of the moving image from the target frame;
Further comprising a animation generator for generating a digest animation of the target content by combining the representative frames, the information processing apparatus according to any one of claims 1-4. The information processing apparatus according to claim 5 , wherein the animation generation unit generates the digest animation by combining the representative frame and a still image included in the target content. A frame extraction unit that extracts a frame in which the target subject is shown as a target frame from a moving image included in the target content, and selects a representative frame of the moving image from the target frame;
Further comprising an information processing apparatus according to any one of claims 1-4 and a representative image selecting section for selecting a representative image of the target content from the representative frame. The information processing apparatus according to claim 7 , wherein the representative image selection unit selects the representative image from the representative frame and a still image included in the target content. A content classification unit that classifies the content for each event to be photographed;
The scenario generation unit generates the highlight scenario and the thumbnail scenarios for each of the event, the information processing apparatus according to any one of claims 1-8. The information processing apparatus according to claim 9 , further comprising an event information generation unit configured to generate event information including information on the event for which the highlight scenario and the thumbnail scenario are generated. The information processing apparatus according to claim 9 or 10 , wherein the sharing related information acquisition unit further acquires information on the event that is permitted to be disclosed to the target user. The shared related information obtaining unit, the target user group, and acquires said target object information associated with each group, information processing apparatus according to any one of claims 1-10. The scenario generation unit may also share the content set as a sharing target before the information about the target user and the target subject is acquired after the information about the target user and the target subject is acquired. for even the content that has been added to the subject, it generates the highlight scenario and the thumbnail scenario, the information processing apparatus according to any one of claims 1 to 12. The information according to any one of claims 1 to 13 , wherein the scenario generation unit outputs the highlight scenario and the thumbnail scenario to an external device where the target user views the highlight video and the thumbnail video. Processing equipment. An operation in which a first user who provides content including a still image or a moving image acquires an operation of setting a second user who is a partner to share the content and a target subject which is a subject associated with the second user. A first client device including a section;
A shared related information acquisition unit for acquiring information on the second user and the target subject;
A content extraction unit that extracts, as the target content, content in which the target subject appears from the content;
A scene extraction unit that extracts a target scene from a moving image included in the target content based on a portion where the target subject is shown, and extracts a representative scene based on an extraction criterion that is shorter than the extraction criterion of the target scene; ,
A scenario generation unit that generates a highlight scenario for configuring a highlight movie by combining the target scenes , and generates and outputs a thumbnail scenario for configuring a thumbnail movie by combining the representative scenes ;
A server device including:
Obtains the output highlight scenario and thumbnail scenario, the highlight generates the highlight video from the content according to the scenario, the product to the second user the thumbnail video from the content in accordance with the thumbnail scenario A second client device including a content acquisition unit to be provided. The first client device further includes a display control unit configured to display an icon indicating a candidate subject that is a candidate for the target subject on a display unit,
The system according to claim 15 , wherein the operation unit acquires an operation of the first user that sets the target subject by selecting an icon corresponding to a desired subject from the icons. Obtaining information on a target user who is a partner to share content including a still image or a video, and a target subject that is a subject associated with the target user;
Extracting the content in which the target subject appears from the content as target content;
Extracting a target scene from a moving image included in the target content based on a portion where the target subject is shown, and extracting a representative scene with an extraction criterion that is shorter than the extraction criterion of the target scene;
An information processing method comprising: generating a highlight scenario for composing a highlight video by combining the target scenes , and generating a thumbnail scenario for composing a thumbnail video by combining the representative scenes .

Images