JP7158564B2 - Information processing device, server device, user device and information processing system - Google Patents

Description

The present invention relates to an information processing device, a server device, a user device and an information processing system.

Patent Literature 1 discloses a technique of converting content displayed on a display unit into keywords, narrowing down the keywords based on a user's action history, and generating an icon for displaying information about the narrowed down keywords. . In addition, in Patent Document 2, based on the user's action history, a feature amount of an object of interest to the user is specified, and using the specified feature amount, the image of the object to be displayed is narrowed down, and the narrowed down object image is displayed. A technique for commenting on an image has been disclosed.

JP 2015-154195 A JP 2014-16882 A

The prior art does not disclose how to distribute the processing load applied to each of the keyword narrowing process and the object image narrowing process.

In order to solve the above problems, an information processing apparatus according to a preferred aspect of the present invention extracts images of a plurality of objects from a moving image based on action information indicating a user's action history and the degree of first narrowing down. A keyword generation unit that identifies and generates candidate keywords that are candidates for comments for each of the identified images of a plurality of objects; A specifying unit that specifies one or more target keywords from a plurality of candidate keywords generated by the keyword generating unit, and a comment generating unit that generates comments related to the target keywords for each of the one or more target keywords. and an adjustment unit that adjusts the degree of first narrowing down and the degree of second narrowing down according to processing information related to the processing of the keyword generating unit and the specifying unit.

Further, in order to solve the above problems, an information processing system according to a preferred aspect of the present invention is an information processing system comprising a user device managed by a user and a server device, wherein the user device comprises the A plurality of images of objects are identified from the moving image based on the action information indicating the user's action history and the degree of the first narrowing down, and candidate keywords that are candidates for comments are determined for each of the plurality of identified images. A comment transmitted from the server device by transmitting a keyword generation unit to be generated, the action information, a plurality of candidate keywords generated by the keyword generation unit, and processing information related to the processing of the keyword generation unit to the server device. and a display control unit for displaying the comment on a display device, wherein the server device receives the action information, the plurality of candidate keywords and the keyword generation transmitted from the user device a second communication device that receives processing information about processing of a part and transmits the comment to the user device; a specifying unit that specifies a keyword from the plurality of candidate keywords; a comment generating unit that generates, as the comment, a comment related to the target keyword for each of the one or more target keywords; and the degree of the first narrowing down. and the degree of the second narrowing down according to processing information about the processing of the keyword generating unit and processing information about the processing of the specifying unit.

According to the information processing apparatus or information processing system according to the present invention, the processing load of narrowing down to identify a plurality of object images and the processing load of narrowing down to identify a target keyword from a plurality of candidate keywords are adjusted according to the processing power. can do.

It is a block diagram showing the whole service system composition concerning a 1st embodiment of the present invention. It is a block diagram which illustrates the hardware constitutions of the server apparatus used for the same embodiment. 3 is a functional block diagram showing functions of a server device used in the same embodiment; FIG. It is an explanatory view showing an example of an image of an object in the same embodiment. It is a flowchart which shows operation|movement of the server apparatus used for the same embodiment. FIG. 11 is a block diagram illustrating the hardware configuration of a user device used in the second embodiment; FIG. FIG. 8 is a functional block diagram showing functions of a user device used in the second embodiment; It is a flowchart which shows operation|movement of the user apparatus used for the same embodiment. FIG. 12 is a block diagram illustrating the hardware configuration of a server device used in the third embodiment; FIG. 3 is a functional block diagram showing functions of a server device used in the same embodiment; FIG. It is a block diagram which illustrates the hardware constitutions of the user's apparatus used for the same embodiment. 3 is a functional block diagram showing functions of a user device used in the same embodiment; FIG. It is a flowchart which shows each operation|movement of a server apparatus and a user apparatus which are used for the same embodiment.

[1. First Embodiment]
[1.1. Service system configuration]
FIG. 1 is a block diagram showing the overall configuration of a service system according to the first embodiment of the invention. A service system 1 shown in FIG. 1 provides a video distribution service. For example, a moving image distribution service provides content of movies or terrestrial digital broadcasting.

As illustrated in FIG. 1, the service system 1 includes user devices 20_1 to 20_m (m is an integer equal to or greater than 1) managed by users U_1 to U_m, a network NW, and a server device . The network NW is a telecommunications line including a mobile communication network or the Internet. In the following description, only common numbers among reference numerals are used, such as user device 20 and user U, when similar elements are not distinguished.

The user device 20 is an information processing device that processes various types of information. The user device 20 is, for example, a portable information processing device such as a smart phone or a tablet terminal. However, any information processing device can be adopted as the user device 20 . The user device 20 may be, for example, a terminal-type information processing device such as a personal computer.

The user device 20 can communicate with the server device 10 via the network NW. The user device 20 receives the image signal Sa transmitted from the server device 10 and displays an image corresponding to the image signal Sa, or transmits the image signal Sa to the television receiver 30 and 30 can be caused to display an image. Also, the user device 20 transmits behavior information to the server device 10 via the network NW. The action information indicates the user's action history. The behavior information is information in which time information is associated with each of position information indicating the user's position, purchase information regarding the user's purchase of goods or services, browsing information regarding web browsing, and playback information regarding video or music playback. is.

The server device 10 is an information processing device having a moving image distribution function of transmitting an image signal Sa representing a moving image to the user device 20 and a comment generating function. The comment generating function narrows down images of objects of interest to the user U based on the action history of the user from among a plurality of images of objects included in one screen image of the moving image indicated by the image signal Sa, and selects the images of the narrowed down objects. It is a function to generate comments about images.

[1.2. Configuration of server device 10]
FIG. 2 is a block diagram illustrating the hardware configuration of the server device 10. As shown in FIG. The server device 10 is implemented by a computer system comprising a processing device 11A, a storage device 12A, a communication device 14A, and a bus 19. FIG. The processing device 11A, storage device 12A, and communication device 14A are connected by a bus 19 for communicating information. The bus 19 may be composed of a single bus, or may be composed of different buses between devices. Note that each element of the server device 10 may be composed of one or more devices, and some elements of the server device 10 may be omitted.

The processing device 11A is a processor that controls the entire server device 10, and is composed of, for example, one or more chips. The processing device 11A is composed of, for example, a central processing unit (CPU) including an interface with peripheral devices, an arithmetic device, registers, and the like. Some or all of the functions of the processing device 11A are realized by hardware such as a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array). may The processing device 11A executes various processes in parallel or sequentially.

The storage device 12A is a recording medium readable by the processing device 11A. The storage device 12A stores a plurality of programs including a control program PRa executed by the processing device 11A, a feature amount table TBLa, a comment table TBLb, service data DS, and various data used by the processing device 11A. The storage device 12A is composed of, for example, one or more types of storage circuits such as ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), and RAM (Random Access Memory).

The feature amount table TBLa associates and stores words indicating types of objects and feature amounts. Words indicating types of objects are, for example, nouns such as Maltese dogs and wine. The feature amount includes a first feature amount regarding the shape of the object and a second feature amount regarding the color of the object. For example, the feature quantity stored in the feature quantity table TBLa in association with the word maltese includes a first feature quantity relating to the shape of the maltese and a second feature quantity relating to the color of the maltese. For an object that does not have a specific shape, such as a liquid, the feature quantity table TBLa associates a word indicating the type of the object with a first feature value related to the shape of the container that stores the object. A second feature quantity relating to the color of the object is stored. For example, the feature quantity stored in the feature quantity table TBLa in association with the word wine includes a first feature quantity relating to the shape of the wine bottle and a second feature quantity relating to the color of the wine.

The service data DS is moving image data representing a moving image distributed by streaming from the server device 10 to the user device 20 via the network NW. The service data DS is compressed and has an I frame (I-frame: Intra-coded frame), a P frame (P-frame: Predicted Frame), and a B frame (B-frame: Bi-directional Predicted Frame). . An I-frame is an uncompressed frame. Each of the P-frames and B-frames is a differential frame. The processing device 11A expands the service data DS to generate the image signal Sa.

Examples of the communication device 14A include, for example, network devices, network controllers, network cards, or communication modules. The communication device 14A communicates with the user device 20 via the network NW under the control of the processing device 11A. The communication device 14A transmits the image signal Sa to the user device 20. FIG. The communication device 14A also receives action information transmitted from the user device 20 and transmits comments generated by the comment generation function to the user device 20 .

[1.3. Function of server device 10]
FIG. 3 is a functional block diagram showing functions of the server device 10. As shown in FIG. The processing device 11A functions as a keyword generation unit 110A, a specification unit 120, a comment generation unit 130, and an adjustment unit 140 by reading and executing the control program PRa from the storage device 12A.

Keyword generation unit 110A identifies images of a plurality of objects of interest to the user from the moving image based on the behavior information received from user device 20 via network NW and the degree of first narrowing down.

The first control information SDa is provided from the adjuster 140 to the keyword generator 110A. The first control information SDa is an example of information indicating the degree of first narrowing down. The first control information SDa designates the degree of narrowing down in the keyword generator 110A. The first control information SDa designates levels 1 to 3, for example. Level 1 means "low" degree of refinement. Level 2 means "medium" degree of refinement. Level 3 means "high" degree of narrowing down. A "low" degree of narrowing down means that narrowing down is performed by broadly and shallowly analyzing the action history indicated by the action information. A "high" degree of narrowing down means that the action history indicated by the action information is deeply analyzed and narrowed down. When level 1 is specified by the first control information SDa, the degree of narrowing down is low, so the number of candidate keywords KW generated by the keyword generation unit 110A increases. On the other hand, when level 3 is specified by the first control information SDa, the degree of narrowing down is high, so the number of candidate keywords KW narrowed down by the keyword generation unit 110A is small. The higher the degree of narrowing down, the deeper the analysis of the action history, so the processing load on the keyword generation unit 110A increases.

As shown in FIG. 3, the keyword generation unit 110A includes a determination unit 111, a feature amount generation unit 112, an extraction unit 113A, and a conversion unit 114.

The determination unit 111 determines the type of object to be extracted from the moving image based on the result of evaluating the user's degree of interest in the object and the first control information SDa. An evaluation function is used to evaluate the user's degree of interest in the object. The evaluation function uses time information, position information, purchase information, browsing information, and playback information indicating behavior information for a predetermined period (for example, the past month) as variables, and evaluates the degree of interest of the user in the object of interest. Output as a value. More types of objects are determined as the degree of narrowing indicated by the first control information SDa is lower.
Specifically, the determination unit 111 determines the type of object to be extracted from the moving image by comparing the threshold corresponding to the degree of narrowing down and the evaluation value. For example, the playback information may indicate that a video of a soccer game was played. It is also assumed that the reproduced soccer match was a match between soccer team A and soccer team B. Also assume that the purchase information indicates the purchase of a uniform for a particular soccer player belonging to soccer team A. Further, let the evaluation value for the object related to "soccer" be X1, the evaluation value for the object related to "soccer team A" be X2, and the evaluation value for the object related to a specific soccer player be X3. In this case, the concept of "soccer" is the broadest and the concept of "a particular soccer player" is the narrowest. And the concept of "soccer team A" is intermediate between the concept of "soccer" and the concept of "a particular soccer player". That is, it can be said that the user's interest is higher in the order of "specific soccer player", "soccer team A", and "soccer". The evaluation values are X3>X2>X1.
Here, it is assumed that the threshold corresponding to level 1 of the degree of narrowing down is R1. Assume that the threshold corresponding to level 2 of the degree of narrowing down is R2. Assume that the threshold corresponding to level 3 of the degree of narrowing down is R3. Furthermore, assume that X3>R3>X2>R2>X1>R1.
When the degree of narrowing down is level 1, the determining unit 111 determines “objects related to soccer” as the type of object to be extracted from the moving image. When the degree of narrowing down is level 2, the determining unit 111 determines “objects related to soccer team A” as the type of object to be extracted from the moving image. When the degree of narrowing down is level 3, the determining unit 111 determines “objects related to a specific soccer player” as the type of object to be extracted from the moving image. The determining unit 111 determines the type of object to be extracted from the moving image by comparing the threshold corresponding to the degree of narrowing down and the evaluation value.

For the type of object determined by the determination unit 111, the feature amount generation unit 112 generates a feature amount of the object of the type. The feature quantity generation unit 112 reads the feature quantity corresponding to the type of the object determined by the determination unit 111 from the feature quantity table TBLa to generate the feature quantity of the object.

113 A of extraction parts extract the image of the object which has the feature-value produced|generated by the feature-value production|generation part 112 from the service data DS. The extraction unit 113A extracts the image of the object from the I-frame image among the I-frame, P-frame, and B-frame included in the service data DS. There are many object images in one I-frame image. When the I-frame images are those shown in FIG. 4, the extraction unit 113A extracts images OB1 to OB5, for example.

The conversion unit 114 generates a candidate keyword KW as a comment target candidate for each of the plurality of object images OB extracted by the extraction unit 113A. Specifically, the conversion unit 114 converts each of the plurality of object images OB into candidate keywords KW. The conversion unit 114 converts the image OB of the object into the candidate keyword KW using, for example, an image recognition model learned by machine learning. For example, image OB1 shown in FIG. 4 is converted to "wine", image OB2 to "wine glass", image OB3 to "clock", image OB4 to "candle", and image OB5 to "Western food". be.

The identifying unit 120 identifies one or more target keywords KX to be commented from among the plurality of candidate keywords KW generated by the keyword generating unit 110A by narrowing down using the behavior information. As described above, the action information includes time information, location information, purchase information, browsing information, and playback information. The specifying unit 120 calculates an evaluation value for each of the plurality of candidate keywords KW using an evaluation function using time information, position information, purchase information, browsing information, and reproduction information as variables. The identifying unit 120 ranks the plurality of candidate keywords KW in descending order of evaluation value. Next, the specifying unit 120 specifies one or a plurality of target keywords KX to be commented on based on the rank of each of the plurality of candidate keywords KW and the second control information SDb. The second control information SDb is an example of information indicating the degree of second narrowing down.

The second control information SDb is generated by the adjusting section 140 and provided from the adjusting section 140 to the specifying section 120 . The second control information SDb designates levels 1 to 3, for example, like the first control information SDa. The meanings of level 1, level 2 and level 3 are the same as those of level 1, level 2 and level 3 in the first control information SDa described above.

The identification unit 120 identifies more target keywords KX as the degree of narrowing indicated by the second control information SDb is lower. When level 1 is specified by the second control information SDb, the degree of narrowing down is low, so the number of target keywords KX obtained by narrowing down in the specifying unit 120 increases. On the other hand, when level 3 is specified by the second control information SDb, the degree of narrowing down is high, so the number of target keywords KX obtained by narrowing down in the specifying unit 120 is small. As the degree of narrowing down increases, the processing load on the identification unit 120 increases.

The comment generating unit 130 generates related comments for each of one or more target keywords KX. A comment means an explanation or commentary on the target keyword KX. A comment is a concept including recommendations. For this reason, the comment includes information about the product that the user U is recommended to purchase and the store that sells the product in relation to the target keyword KX. The comment generation unit 130 generates a comment by reading a comment corresponding to the target keyword KX from the comment table TBLb storing comments in association with keywords. Alternatively, the comment generation unit 130 may access a search site connected to the network NW, acquire information related to the target keyword KX, and use the acquired information as a comment. The comment generated by the comment generator 130 is transmitted to the user device 20 by the communication device 14A, for example, by e-mail.

The adjustment unit 140 adjusts the degree to which the keyword generation unit 110A narrows down the types of objects (first degree of narrowing down) and the degree to which the specifying unit 120 narrows down one or more target keywords KX from a plurality of candidate keywords KW (second degree of narrowing down). degree of narrowing down) is adjusted according to processing information relating to the processing of the keyword generation unit 110A and the processing of the identification unit 120. FIG. The processing information is information indicating the processing capability of the keyword generation unit 110A and the processing capability of the identification unit 120. FIG. The processing capacity of the keyword generating section 110A and the processing capacity of the specifying section 120 change according to the processing capacity of the CPU or the resources of the CPU allocated to the functions of the keyword generating section 110A and the specifying section 120, respectively.

If the processing power of the CPU exceeds a predetermined threshold value, or if the resources allocated to the function of the identification unit 120 are greater than the resources allocated to the function of the keyword generation unit 110A, the adjustment unit 140 performs the identification It is determined that the processing capability of the unit 120 is high. The size of resources is determined, for example, by at least one of the number of cores and the number of threads. For example, assume that the number of cores and the number of threads allocated to the keyword generation unit 110A is "1" and the number of threads is "1", and the number of cores and the number of threads allocated to the identification unit 120 is "1" and "2". . In this case, it is determined that the processing capability of the identification unit 120 is higher than the processing capability of the keyword generation unit 110A.
When determining that the processing capability of the identifying unit 120 is high, the adjusting unit 140 decreases the degree of narrowing down in the keyword generating unit 110A and increases the degree of narrowing down in the identifying unit 120 . Specifically, adjustment section 140 generates first control information SDa indicating level 1 and second control information SDb indicating level 3. FIG. By providing the first control information SDa indicating level 1, the processing load of the keyword generation unit 110A is reduced. On the other hand, when the second control information SDb indicating level 3 is given, the processing load on the identifying unit 120 increases.

When determining that the processing capability of the identifying unit 120 is low, the adjusting unit 140 increases the degree of narrowing down in the keyword generating unit 110A and decreases the degree of narrowing down in the identifying unit 120 . Specifically, adjustment section 140 generates first control information SDa indicating level 3 and second control information SDb indicating level 1 . Given the first control information SDa indicating level 3, the processing load on the keyword generation unit 110A increases. On the other hand, given the second control information SDb indicating level 3, the processing load on the identification unit 120 is reduced.

The reason for lowering the degree of narrowing down in the keyword generation unit 110A and increasing the degree of narrowing down in the specifying unit 120 when the processing capability of the specifying unit 120 is high is as follows. 110 A of keyword production|generation parts extract the object which the user U is interested in by analyzing an image in 113 A of extraction parts. However, the accuracy of object extraction using image processing is low. Therefore, when the processing capability of the identification unit 120 is high, the degree of narrowing down in the keyword generation unit 110A is lowered. By lowering the degree of narrowing down, the number of candidate keywords KW generated by the keyword generation unit 110A increases compared to when the degree of narrowing down is high. However, since the identifying unit 120 has high processing capability, the identifying unit 120 can narrow down the target keyword KX from the increased candidate keywords KW.
On the other hand, if the processing capability of the identifying unit 120 is low, the processing of the identifying unit 120 will be delayed if the number of candidate keywords KW that exceeds the processing capability is generated. As a result, it becomes difficult to generate comments at appropriate times. In such a case, it is desirable to reduce the number of candidate keywords KW generated by the keyword generator 110A at the expense of the accuracy of the candidate keywords KW.
The adjustment unit 140 adjusts the extent to which the keyword generation unit 110A narrows down the types of objects in the keyword generation unit 110A and the identification unit 120 selects one or more from a plurality of candidate keywords KW according to the processing capabilities of the keyword generation unit 110A and the identification unit 120. The extent to which the target keyword KX is narrowed down is adjusted. As a result, the accuracy of the specific part 120 comment improves. In addition, comments are generated at appropriate timing by suppressing processing delays.
Furthermore, the processing capacity of the keyword generating section 110A and the processing capacity of the identifying section 120 may be dynamically changed. An example in which the processing capacity is dynamically changed is when the resources allocated to the keyword generation unit 110A and the resources allocated to the identification unit 120 decrease as the processing load of the processing device 11A increases. In such a case, the adjusting unit 140 adjusts the degree of narrowing down of the keyword generating unit 110A and the degree of narrowing down of the identifying unit 120 according to the processing ability of the keyword generating unit 110A and the processing ability of the identifying unit 120. . Therefore, the accuracy of comments is improved. Comments are generated in a timely manner.

[1.4. Operation of server device 10]
Next, operations of the server device 10 will be described. FIG. 5 is a flow chart showing the operation of the server device 10. As shown in FIG.

First, the processing device 11A generates first control information SDa and second control information SDb according to processing information (step S1). The server device 10 has a CPU with higher processing power than the user device 20, and the processing power of the CPU exceeds a predetermined threshold. In step S1, the processing device 11A determines that the capability of the identifying unit 120 is high, and generates first control information SDa indicating level 1 and second control information SDb indicating level 3. FIG.

Next, the processing device 11A determines the type of object based on the action information and the degree of narrowing down according to the first control information SDa generated in step S1. Since the first control information SDa generated in step S1 of this operation example indicates level 1, the degree of object narrowing down in step S2 is low.

Next, the processing device 11A reads the feature quantity corresponding to the type of object determined in step S2 from the feature quantity table TBLa (step S3).

Next, the processing device 11A extracts the image of the object from the I frame of the moving image indicated by the service data DS based on the feature amount read out in step S3 (step S4A). A single frame image usually includes a plurality of object images. Therefore, the processing device 11A extracts images of a plurality of objects in the process of step S4A.

Next, the processing device 11A converts each of the plurality of object images extracted in step S4A into candidate keywords KW (step S5).

Next, the processing device 11A identifies the target keyword KX from among the plurality of candidate keywords KW obtained by the process of step S5 based on the action information and the second control information SDb (step S6). Since the second control information SDb generated in step S1 of this operation example indicates level 3, the action information is narrowed down to be deeply analyzed in step S6.

Next, the processing device 11A generates a comment related to the target keyword KX (step S7). In the processing of step S7, the processing device 11A generates a comment by reading the comment corresponding to the target keyword KX from the comment table TBLb. The processing device 11A transmits the generated comment to the user device 20 by e-mail or the like.

Further, the processing device 11A functions as the adjustment unit 140 in the processing of step S1, and functions as the keyword generation unit 110A in the processing of steps S2 to S5. More specifically, the processing device 11A functions as the determination unit 111 in the process of step S2, functions as the feature amount generation unit 112 in the process of step S3, functions as the extraction unit 113A in the process of step S4A, and functions as the extraction unit 113A in the process of step S5. It functions as the conversion unit 114 in the processing of . Furthermore, the processing device 11A functions as the specifying unit 120 in the processing of step S6, and functions as the comment generating unit 130 in the processing of step S7.

The server device 10 of this embodiment is an example of an information processing device having a comment generation function, that is, an information processing device of the present invention. The server device 10 narrows down a plurality of images each corresponding to a plurality of objects from the moving image based on action information indicating the user's action history, and candidates for commenting on each of the plurality of narrowed down images. A keyword generation unit 110A for generating keywords is provided. The server device 10 also includes a specifying unit 120 that specifies one or more target keywords to be commented on by narrowing down the plurality of candidate keywords generated by the keyword generating unit 110A based on the behavior information. Furthermore, for each of one or a plurality of target keywords, the server device 10 includes a comment generation unit 130 that generates a comment related to the target keyword, an extent to which a plurality of images are narrowed down in the keyword generation unit 110A, and and an adjustment unit 140 that adjusts the degree to which a plurality of candidate keywords are narrowed down according to processing information related to the processing of the keyword generation unit 110A and the identification unit 120 .

According to this aspect, the degree of narrowing down in the keyword generation unit 110</b>A and the degree of narrowing down in the specifying unit 120 can be adjusted according to the processing capability of the specifying unit 120 . In the server device 10 of the above-described embodiment, since the processing capability of the identification unit 120 is high, the accuracy of comments is improved by adjusting the degree of narrowing down in the keyword generation unit 110A to be low and the degree of narrowing down in the identification unit 120 to be high. can be made

[2. Second Embodiment]
In the service system 1 of the second embodiment, the server device 10 has only the moving image distribution function, and the user device 20 has the comment generation function.

FIG. 6 is a diagram showing a hardware configuration example of the user device 20 of the second embodiment. The user device 20 includes a processing device 11 B, a storage device 12 B, a communication device 14 B, a display control section 15 and a bus 19 . In addition to the processing device 11B, the storage device 12B, the communication device 14B, the display control unit 15, and the bus 19, the user device 20 includes an output device including a display device and a speaker, an input device such as a touch panel, and a short-range wireless communication device. and GPS devices. A short-range wireless communication device is a device that communicates with another device by short-range wireless communication. Short-range wireless communication includes, for example, Bluetooth (registered trademark), ZigBee (registered trademark), or WiFi (registered trademark). A GPS device is a device that receives radio waves from a plurality of satellites and generates position information from the received radio waves.

Each of the processing device 11B, the storage device 12B, and the communication device 14B corresponds to each of the processing device 11A, the storage device 12A, and the communication device 14A in the first embodiment. The storage device 12B differs from the storage device 12A in that it stores the control program PRb instead of the control program PRa, that it stores action information, and that it does not store the service data DS. . The communication device 14B differs from the communication device 14A in that it communicates with the server device 10 under the control of the processing device 11B. The communication device 14B receives the image signal Sa transmitted from the server device 10 via the network NW.

The display control unit 15 controls the operation of the display device or the television receiver 30 . The display control unit 15 causes the display device or the television receiver 30 to display the moving image indicated by the image signal Sa. In addition, the display control unit 15 generates an image of the comment generated by the comment generation function as an overlay image for the moving image, and displays the overlay image on the frame of the moving image on the display device or the television receiver 30. Let It should be noted that the comment display is not limited to being performed in real time while viewing the moving image, and may be performed after viewing the moving image.

FIG. 7 is a functional block diagram showing functions of the processing device 11B of the second embodiment. The processing device 11B functions as a keyword generation unit 110B, a specification unit 120, a comment generation unit 130, and an adjustment unit 140 by reading and executing the control program PRb from the storage device 12B. Further, in the processing device 11B of the second embodiment, the action information stored in the storage device 12B is provided to the keyword generating section 110B and the specifying section 120. FIG.

As shown in FIG. 7, the keyword generation unit 110B includes a determination unit 111, a feature amount generation unit 112, an extraction unit 113B, and a conversion unit 114. The image signal Sa received via the network NW is supplied to the extraction unit 113B. The image signal Sa is composed of a plurality of frames. Since the image signal Sa is a signal obtained by decompressing the service data DS, each frame represented by the image signal Sa does not distinguish between I-frames, P-frames and B-frames. The extraction unit 113B is different from the extraction unit 113A of the first embodiment in that it extracts an image of the type of object determined by the determination unit 111 from the image of each frame represented by the image signal Sa.

Next, operation of the user device 20 of the second embodiment will be described. FIG. 8 is a flow chart showing the operation of the user device 20 of the second embodiment. In the user device 20 of the second embodiment, the processing device 11B executes steps S1, S2, S3, S4B, S5, S6 and S7 shown in FIG. 7 in this order. The processing device 11B functions as the adjusting section 140 in the process of step S1. However, since the user device 20 has a CPU with lower processing power than the server device 10, the processing device 11B determines that the processing power of the identification unit 120 is low in step S1, and the first control indicating level 3 Information SDa and second control information SDb indicating level 1 are generated.

It functions as the keyword generator 110B in the processing from step S2 to step S5. More specifically, the processing device 11B functions as the determination unit 111 in the process of step S2, functions as the feature amount generation unit 112 in the process of step S3, functions as the extraction unit 113B in the process of step S4B, and functions as the extraction unit 113B in the process of step S5. It functions as the conversion unit 114 in the processing of . In this operation example, the first control information SDa generated in step S1 indicates level 3, so the degree of narrowing down in step S2 is high. Further, in the process of step S4B, the processing device 11B extracts an image of the object from each of the plurality of frames of the moving image indicated by the image signal Sa, based on the feature amount read out in step S3. Since images of a plurality of objects usually exist in one frame image, the processing device 11B extracts images of a plurality of objects in the process of step S4B.

Furthermore, the processing device 11B functions as the specifying unit 120 in the process of step S6, and functions as the comment generation unit 130 in the process of step S7. The processing device 11B causes the display device or the television receiver 30 to display the comment generated by the processing of step S7 using the display control unit 15 . In this operation example, the second control information SDb generated in step S1 indicates level 1, so the degree of narrowing down in step S6 is low, and the processing load on the identification unit 120 is low.

The user device 20 of this embodiment is an example of an information processing device having a comment generation function, that is, an information processing device of the present invention. The user device 20 identifies images of a plurality of objects from the moving image based on the behavior information indicating the behavior history of the user and the degree of the first narrowing down, and candidates for comments on each of the identified images. A keyword generation unit 110B that generates a candidate keyword that becomes Also, the user device 20 is a specifying unit that specifies one or a plurality of target keywords to be commented from a plurality of candidate keywords generated by the keyword generating unit 110B, based on the behavior information and the degree of the second narrowing down. 120. Furthermore, for each of one or a plurality of target keywords, the user device 20 includes the comment generation unit 130 that generates comments related to the target keyword, and the keyword generation unit 110B that narrows down the plurality of images (first narrowing down). degree) and the degree of narrowing down a plurality of candidate keywords in the identifying unit 120 (second degree of narrowing down) according to processing information about the processing of the keyword generating unit 110B and the identifying unit 120; Prepare.

According to this aspect, the degree of narrowing down in the keyword generation unit 110</b>B and the degree of narrowing down in the specifying unit 120 can be adjusted according to the processing capability of the specifying unit 120 . In the user device 20 of the second embodiment, since the processing capability of the specifying unit 120 is low, the degree of narrowing down in the keyword generating unit 110B is adjusted to be high, and the degree of narrowing down in the specifying unit 120 is adjusted to be low.

[3. Third Embodiment]
FIG. 9 is a diagram showing a hardware configuration example of the server device 10 included in the service system 1 of the third embodiment. The server device 10 comprises a processing device 11C, a storage device 12C, a communication device 14C (second communication device), and a bus 19 . Each of the processing device 11C, the storage device 12C, and the communication device 14C corresponds to each of the processing device 11A, the storage device 12A, and the communication device 14A in the first embodiment. The storage device 12C differs from the storage device 12A in that the control program PRc is stored in place of the control program PRa and the feature amount table TBLa is not stored. The communication device 14C is common to the communication device 14A in that it transmits the image signal Sa and the comment to the user device 20 under the control of the processing device 11C. However, the communication device 14C differs from the communication device 14A in that it receives from the user device 20 processing information and action information relating to the processing in the keyword generation unit 110B, and in that it transmits the first control information SDa to the user device 20. do.

FIG. 10 is a functional block diagram showing functions of the server device 10 of the third embodiment. The processing device 11C functions as a specifying unit 120, a comment generating unit 130, and an adjusting unit 140 by reading and executing the control program PRc from the storage device 12C. Further, in the server device 10 of the third embodiment, the processing information transmitted from the user device 20 via the network NW is given to the adjustment unit 140, and the first control information SDa generated by the adjustment unit 140 is It differs from the server device 10 of the first embodiment in that it is transmitted to the user device 20 via the network NW.

FIG. 11 is a diagram showing a hardware configuration example of the user device 20 included in the service system 1 of the third embodiment. The user device 20 includes a processing device 11D, a storage device 12D, a communication device (first communication device) 14D, a display control section 15, and a bus 19. FIG. Each of the processing device 11D, the storage device 12D, and the communication device 14D corresponds to each of the processing device 11B, the storage device 12B, and the communication device 14B in the second embodiment. The storage device 12D differs from the storage device 12B in that it stores the control program PRd instead of the control program PRb and that it does not store the comment table TBLb. The communication device 14D is common to the communication device 14B in that it receives the image signal Sa from the server device 10 under the control of the processing device 11D. However, the communication device 14D transmits to the server device 10 processing information, action information, and a plurality of candidate keywords KW relating to the processing in the keyword generation unit 110B, and receives first control information SDa and comments from the server device 10. It differs from the communication device 14B in one point.

FIG. 12 is a functional block diagram showing functions of the user device 20 of the third embodiment. The processing device 11D functions as a keyword generator 110B by reading and executing the control program PRd from the storage device 12D. Further, the user device 20 of the third embodiment transmits processing information to the server device 10 via the network NW, and the first control information SDa transmitted from the server device 10 via the network is a keyword generator. 110B is different from the user device 20 of the second embodiment.

Next, operations of the server device 10 and the user device 20 of the third embodiment will be described. FIG. 13 is a flow chart showing operations of the server device 10 and the user device 20 of the third embodiment. In the user device 20 of the third embodiment, the processing device 11D executes steps S1_1, S2, S3, S4B, and S5 shown in FIG. 7 in this order. On the other hand, in the server device 10 of the third embodiment, the processing device 11C executes steps S1, S6, and S7 shown in FIG. 12 in this order.

In the process of step S1_1, the processing device 11D transmits the processing information regarding the processing of the keyword generating section 110B and the action information stored in the storage device 12D to the server device 10 via the network NW.

In the server device 10, the processing of step S1 is executed with the reception of the action information and the processing information as a trigger. In the process of step S1, the processing device 11C functions as the adjusting section 140. FIG. The processing information received from the user device 20 indicates the processing capability of the keyword generator 110B. As described above, the processing capability of the user device 20 is lower than the processing capability of the server device 10, and resources that can be allocated to the keyword generation unit 110B in the user device 20 can also be allocated to the function of the identification unit 120 in the server device 10. resources. In step S1, the processing device 11D determines that the processing capability of the identifying unit 120 is high, and generates first control information SDa indicating level 1 and second control information SDb indicating level 3. FIG. The first control information SDa generated by the process of step S1 is transmitted to the user device 20 via the network NW.

In the user device 20, the processing from step S2 onward is executed with the reception of the first control information SDa transmitted from the server device 10 as a trigger. In the processing from step S2 to step S5, the processing device 11D functions as the keyword generation section 110B. More specifically, the processing device 11D functions as the determination unit 111 in the processing of step S2, functions as the feature amount generation unit 112 in the processing of step S3, functions as the extraction unit 113B in the processing of step S4B, and functions as the extraction unit 113B in the processing of step S4B. It functions as the conversion unit 114 in the processing of . The candidate keyword KW generated by the process of step S5 is transmitted to the server device 10 via the network NW. In this operation example, since the first control information SDa transmitted from the server device 10 to the user device 20 indicates level 1, the degree of narrowing down in step S2 is low, and the processing load on the keyword generation unit 110B is low.

In the server device 10, when the candidate keyword KW transmitted from the user device 20 is received, the process from step S6 onwards is executed. 11 C of processing apparatuses function as the specific part 120 in the process of step S6, and function as the comment production|generation part 130 in the process of step S7. The comment generated by the process of step S7 is transmitted from the server device 10 to the user device 20 by e-mail or the like. When receiving a comment by e-mail or the like, the user device 20 displays an image of the comment on a display device or a television receiver. In this operation example, since the second control information SDb indicating level 3 is generated in step S1, the degree of narrowing down in step S6 is high, and the processing load on the identification unit 120 is high.

The service system 1 of this embodiment is an example of an information processing system having a comment generation function, that is, an information processing system of the present invention, and includes a user device 20 managed by a user and a server device 10 . The user device 20 identifies images of a plurality of objects from the moving image based on the behavior information indicating the behavior history of the user and the degree of the first narrowing down, and candidates for comments on each of the identified images. A keyword generation unit 110B that generates a candidate keyword that becomes, action information, a plurality of candidate keywords generated by the keyword generation unit 110B, and processing information related to the processing of the keyword generation unit 110B are transmitted to the server device 10, and the server device 10 and a communication device 14D (first communication device) that receives comments transmitted from and a display control unit 15 that displays the comments on the display device. The server device 10 receives action information transmitted from the user device 20, a plurality of candidate keywords generated by the keyword generation unit 110B, and processing information related to the processing of the keyword generation unit 110B, and transmits comments to the user device 20. a device 14C (second communication device); a specifying unit 120 that specifies one or more target keywords to be commented from a plurality of candidate keywords based on the behavior information and the degree of the second narrowing down; For each of a plurality of target keywords, a comment generating unit 130 that generates related comments, a degree of narrowing down a plurality of images in the keyword generating unit 110B (a first degree of narrowing down), and a plurality of candidate keywords in the specifying unit 120 are selected. The adjusting unit 140 adjusts the degree of narrowing down (second degree of narrowing down) according to the processing information regarding the processing of the keyword generating unit 110B and the processing information regarding the processing of the identifying unit 120. FIG.

According to this aspect, the degree of narrowing down in the keyword generating unit 110B and the degree of narrowing down in the identifying unit 120 are according to the processing ability of the keyword generating unit 110B of the user device 20 and the processing ability of the identifying unit 120 of the server device 10. and can be adjusted. In the service system 1 of the third embodiment, the keyword generation unit 110B has a lower processing capacity than the identification unit 120, so the degree of narrowing down in the keyword generation unit 110B is low, and the degree of narrowing down in the identification unit 120 is adjusted to be high. and can improve the relevance of comments.

[4. Modification]
The present invention is not limited to the embodiments exemplified above. Specific modification modes are exemplified below. Two or more aspects arbitrarily selected from the following examples may be combined.

(1) In the above-described first embodiment, the frames from which the extraction unit 113A extracts the image of the object from the service data DS may be the following frames.
First, the extraction unit 113A may extract the image of the object in a frame with a high audience rating among the plurality of frames in the image. In this case, the extraction unit 113A may acquire the audience rating from the external device in real time. Specifically, the extraction unit 113A extracts an image of an object from a frame in which the obtained audience rating exceeds a predetermined audience rating. A frame with a high audience rating is presumed to be of high interest to the user U compared to other frames. Therefore, since the image of the object is extracted from the image of the frame in which the user U is highly interested, a comment beneficial to the user U can be generated.
Secondly, the extraction unit 113A may receive the audio signal of the user U from the user device 20, and extract the object image from the frame to which the user U responds well based on the audio signal. For example, an object image may be extracted from a frame in which the user U cheers.
Thirdly, the extraction unit 113A may extract an object image in a frame that is the theme of the program based on the program information. For example, the extraction unit 113A may analyze the service data DS and identify frames that are the subject of the program. In this case, the extraction unit 113A may acquire program information from the external device via the network NW.
Similarly, in the second and third embodiments, the extraction unit 113B extracts the image of the object from the image signal Sa. It may be a frame that becomes

(2) The adjustment unit 140 adjusts the degree of narrowing down (first degree of narrowing down) in the determining unit 111 and the degree of narrowing down in the specifying unit 120 according to the quality of the moving image instead of or in addition to the processing power. (degree of second narrowing down) may be adjusted. Video quality is included in the processing information. A specific example of the quality of the moving image is the frame rate of the moving image or the resolution of each frame of the moving image. When the processing information includes the quality of the moving image, the adjustment unit 140 reduces the extent to which the plurality of images are narrowed down in the keyword generation unit compared to when the quality of the moving image is high and when the quality of the moving image is low. Increase the degree of narrowing down multiple candidate keywords. This is because the processing load on the keyword generation unit is higher when the quality of the moving image is high than when the quality of the moving image is low.

(3) In the above embodiment, the keyword converted from the image of the object specified based on the action information is directly used as the candidate keyword. However, in the narrowing down by the keyword generating unit 110A, the broader concept of the keyword converted from the image of the object identified based on the action information is used as a candidate keyword instead of or in addition to the keyword, and the narrowing down in the identifying unit 120 is performed. can be narrowed down to subordinate concepts. For example, if the keyword converted from the image of the object specified based on the action information is "SUV" (Sport Utility Vehicle), the narrowing down by the keyword generation unit 110A will be "car" which is a superordinate concept of "SUV". is used as a candidate keyword together with or instead of "SUV", and the target keyword is narrowed down to "SUV" by narrowing down in the specifying unit 120. FIG. Further, when the keyword converted from the image of the object specified based on the action information is "rose", the keyword generation unit 110A narrows down "flower", which is a superordinate concept of "rose", to "rose". The target keyword may be narrowed down to "rose" by using a candidate keyword together with or instead of "rose" and narrowing down in the specifying unit 120. FIG. It should be noted that the degree of hyper-conceptualization may be adjusted according to the processing capability of the keyword generating section 110A. In addition, for the narrowing down in the keyword generation unit 110B, the hypernymization may be similarly adopted.

(4) In the first and third embodiments, comments are sent from the server device 10 to the user device 20 by e-mail or the like. However, the image signal Sa overlaid with the comment may be transmitted from the server device 10 to the user device 20 . Moreover, the display of the comment is not limited to real-time display, and may be performed after the viewing of the moving image is finished.

(5) As an example of the information processing apparatus of the present invention, the server device 10 was mentioned in the first embodiment, and the user device 20 was mentioned in the second embodiment. However, the information processing apparatus of the present invention identifies images of a plurality of objects from a moving image based on behavior information indicating a user's behavior history and the degree of first narrowing down, and for each of the identified plurality of images, A keyword generation unit that generates candidate keywords that are candidates for comments, and one or more target keywords that are targets for comments are generated by the keyword generation unit based on the behavior information and the degree of second narrowing down. A specifying unit that specifies from a plurality of candidate keywords, a comment generating unit that generates comments related to each of one or more target keywords, and a degree of narrowing down a plurality of images in the keyword generating unit (first degree of narrowing down) and degree of narrowing down the plurality of candidate keywords in the specifying unit (second degree of narrowing down) according to processing information relating to the processing of the keyword generating unit and the specifying unit. It is not limited to the server device or the user device.

For example, a relay device (switching hub, router, gateway, etc.) that relays the image signal Sa transmitted from the video distribution server to the user device may include a keyword generation unit, a specification unit, a comment generation unit, and an adjustment unit. . According to this relay device, the image signal Sa transmitted from the moving image distribution server can be overlaid with the image of the comment generated by the comment generation unit and transferred to the user device. Further, according to this relay device, it is possible to adjust the degree of narrowing down in the keyword generating unit and the degree of narrowing down in the identifying unit according to the processing capability of the identifying unit.

A server device having a comment generation function, that is, the server device of the present invention includes an information processing device having the keyword generation unit, the specification unit, the comment generation unit, and the adjustment unit, and an action transmitted from the user device managed by the user. a communication device for receiving information and transmitting comments to the user device. Similarly, a user device having a comment generation function, that is, the user device of the present invention includes an information processing device having the keyword generation unit, the specification unit, the comment generation unit, and the adjustment unit, and a display control unit for displaying the comment on the display device. and .

(6) The block diagrams used to describe each of the above-described embodiments show blocks for each function. These functional blocks (components) are implemented by any combination of hardware and/or software. Further, means for realizing each functional block is not particularly limited. That is, each functional block may be implemented by one device physically and/or logically coupled, or may be implemented by two or more physically and/or logically separated devices directly and/or indirectly. These multiple devices may be connected together (eg, wired and/or wirelessly). For example, the function of the determination unit 111 may be provided by another server device connected via the network NW. Similarly, the function of the feature quantity generation unit 112 may also be provided by another server device connected via the network NW, and the feature quantity table TBLa may also be provided in another server device.
Also, the term "apparatus" used in the description of each of the above-described embodiments may be replaced with other terms such as circuit, device, or unit.

(7) As long as there is no contradiction, the order of the processing procedures, sequences, flowcharts, etc. in each of the above-described embodiments may be changed. For example, the methods described herein present elements of the various steps in a sample order, and are not limited to the specific order presented.

(8) In each of the above-described embodiments, input/output information and the like may be stored in a specific location (for example, memory) or managed in a management table. Input/output information and the like can be overwritten, updated, or appended. The output information and the like may be deleted. The entered information and the like may be transmitted to another device.

(9) In each of the above-described embodiments, the determination may be made by a value represented by 1 bit (0 or 1), or by a true/false value (Boolean: true or false). and may be performed by numerical comparison (eg, comparison with a predetermined value).

(10) The storage device 12A in the first embodiment described above is a recording medium readable by the processing device 11A. Applications discs, Blu-ray (registered trademark) discs), smart cards, flash memory devices (e.g. cards, sticks, key drives), CD-ROMs (Compact Disc-ROMs), registers, removable discs, hard disks, floppies (registered (trademark) disks, magnetic strips, databases, servers, or other suitable storage media. The storage device 12B in the second embodiment, and the storage devices 12C and 12D in the third embodiment are similar to the storage device 12A. Also, the program may be transmitted from the network NW. Also, the program may be transmitted from a communication network via an electric communication line.

(11) Each of the above-described embodiments is LTE (Long Term Evolution), LTE-A (LTE-Advanced), SUPER 3G, IMT-Advanced, 4G, 5G, FRA (Future Radio Access), W-CDMA (registered trademark) ), GSM (registered trademark), CDMA2000, UMB (Ultra Mobile Broadband), IEEE 802.11 (Wi-Fi), IEEE 802.16 (WiMAX), IEEE 802.20, UWB (Ultra-WideBand), Bluetooth (registered Trademarks), other suitable systems, and/or future generation systems enhanced based on these.

(12) In each of the embodiments described above, the information, signals, etc. described may be represented using any of a variety of different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may refer to voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, light fields or photons, or any of these. may be represented by a combination of
The terms explained in this specification and/or terms necessary for understanding this specification may be replaced with terms having the same or similar meanings.

(13) Each function illustrated in FIGS. 3, 7, 10, and 12 is implemented by any combination of hardware and software. Also, each function may be implemented by a single device, or may be implemented by two or more devices configured separately from each other.

(14) The programs exemplified in each of the above embodiments, whether referred to as software, firmware, middleware, microcode, hardware description language, or by any other name, may include instructions, instruction sets, code, code segments. , program code, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, procedures or functions, or the like.
Software, instructions, etc. may also be sent and received over a transmission medium. For example, the software may use wired technologies such as coaxial cable, fiber optic cable, twisted pair and Digital Subscriber Line (DSL) and/or wireless technologies such as infrared, radio and microwave to create websites, servers, or other When transmitted from a remote source, these wired and/or wireless technologies are included within the definition of transmission media.

(15) In each of the embodiments described above, the terms "system" and "network" are used interchangeably.

(16) In each of the above-described embodiments, information, parameters, etc. may be represented by absolute values, may be represented by relative values from a predetermined value, or may be represented by corresponding separate information. good too.

(17) In each of the embodiments described above, the user equipment 20 may be a mobile station. A mobile station is defined by those skilled in the art as a subscriber station, mobile unit, subscriber unit, wireless unit, remote unit, mobile device, wireless device, wireless communication device, remote device, mobile subscriber station, access terminal, mobile terminal, wireless It may also be called a terminal, remote terminal, handset, user agent, mobile client, client, or some other suitable term.

(18) in each of the above embodiments, the term "connected" or any variation thereof means any direct or indirect connection or coupling between two or more elements; It can include the presence of one or more intermediate elements between two elements that are "connected" to each other. Connections between elements may be physical, logical, or a combination thereof. As used herein, two elements are referred to by the use of one or more wires, cables and/or printed electrical connections and, as some non-limiting and non-exhaustive examples, radio frequency They can be considered to be "connected" to each other through the use of electromagnetic energy, such as electromagnetic energy having wavelengths in the microwave, light (both visible and invisible) regions.

(19) In each of the embodiments described above, the phrase "based on" does not mean "based only on," unless expressly specified otherwise. In other words, the phrase "based on" means both "based only on" and "based at least on."

(20) Any reference to elements using the "first," "second," etc. designations used herein does not generally limit the quantity or order of those elements. These designations may be used herein as a convenient method of distinguishing between two or more elements. Thus, references to first and second elements do not imply that only two elements may be employed therein, or that the first element must precede the second element in any way.

(21) To the extent that "including," "comprising," and variations thereof are used in each of the above-described embodiments in the specification or claims, these terms include: Like the term "comprising," it is intended to be inclusive. Furthermore, the term "or" as used in this specification or the claims is not intended to be an exclusive OR.

(22) Throughout this application, where articles have been added by translation, e.g. a, an and the in English, these articles shall be used unless the context clearly indicates otherwise. Including multiple.

(23) It will be clear to those skilled in the art that the present invention is not limited to the embodiments described herein. The present invention can be implemented as modifications and changes without departing from the spirit and scope of the present invention determined based on the description of the claims. Accordingly, the description herein is for illustrative purposes only and is not meant to be limiting in any way. Also, a plurality of aspects selected from the aspects exemplified in this specification may be combined.

Reference Signs List 1 service system 10 server device 11A, 11B, 11C, 11D processing device 20 user device 12A, 12B storage device 14A, 14B, 14C, 14D communication device 15 display unit 19 Bus 110A, 110B Keyword generation unit 111 Determination unit 112 Feature amount generation unit 113A, 113B Extraction unit 114 Conversion unit 120 Specification unit 130 Comment generation unit 140 Adjustment unit , PRa, PRb, PRc, PRd... control program, TBLa... feature amount table, TBLb... comment table, DS... service data, KW... candidate keyword, KX... target keyword.

Claims (9)

A plurality of images of objects are specified from a moving image based on action information indicating a user's action history and a degree of first narrowing down, and candidates for commenting on each of the specified images of a plurality of objects. a keyword generation unit that generates keywords;
a specifying unit that specifies one or more target keywords to be commented from a plurality of candidate keywords generated by the keyword generating unit, based on the behavior information and the degree of second narrowing down;
a comment generation unit that generates a comment related to the target keyword for each of the one or more target keywords;
an adjusting unit that adjusts the degree of first narrowing down and the degree of second narrowing down according to processing information related to processing of the keyword generating unit and the specifying unit;
Information processing device. 2. The information processing apparatus according to claim 1, wherein said processing information includes processing capability of said specifying unit. The adjustment unit
When the processing power of the specific unit is high, compared to when the processing power is low, the degree of the first narrowing down is lowered and the degree of the second narrowing down is raised.
The information processing apparatus according to claim 2. 2. The information processing apparatus according to claim 1, wherein the processing information includes information regarding quality of the moving image. The adjustment unit
When the quality of the moving image is high, compared to when the quality of the moving image is low, the degree of the first narrowing down is lowered and the degree of the second narrowing down is raised.
The information processing apparatus according to claim 4. The adjustment unit outputs control information indicating the degree of the first narrowing down to the keyword generation unit,
The keyword generation unit is
a determination unit that determines the type of object to be extracted from the moving image based on the result of evaluating the user's degree of interest in the object based on the behavior information and the control information;
a feature quantity generating unit for generating a feature quantity of an image of the object of the type determined by the determining unit;
an extraction unit that extracts an image of an object having the feature amount from the moving image;
with
The keyword generation unit is
a conversion unit that converts the image of the object extracted by the extraction unit into the candidate keyword;
The information processing apparatus according to any one of claims 1 to 5. an information processing apparatus according to any one of claims 1 to 6;
a communication device that receives the action information transmitted from a user device managed by the user and transmits the comment to the user device;
A server device comprising an information processing apparatus according to any one of claims 1 to 6;
a display control unit for displaying the comment on a display device;
A user equipment comprising: An information processing system comprising a user device managed by a user and a server device,
The user device
Based on the action information indicating the action history of the user and the degree of first narrowing down, a plurality of images respectively corresponding to the plurality of objects are specified from the moving image, and a comment is made for each of the specified images of the plurality of objects. a keyword generating unit that generates candidate keywords that are candidates for the target;
A first communication device that transmits the action information, a plurality of candidate keywords generated by the keyword generation unit, and processing information related to processing of the keyword generation unit to the server device, and receives comments transmitted from the server device. When,
A display control unit for displaying the comment on a display device,
The server device
a second communication device that receives the action information, the plurality of candidate keywords, and processing information related to processing of the keyword generation unit transmitted from the user device, and transmits the comment to the user device;
a specifying unit that specifies one or more target keywords to be commented on from the plurality of candidate keywords based on the behavior information and the degree of second narrowing down;
a comment generating unit that generates, for each of the one or more target keywords, a comment related to the target keyword as a comment to be transmitted to the user device;
a degree of narrowing down the plurality of images in the keyword generating unit; an adjustment unit that adjusts according to information and processing information related to processing of the identification unit;
Information processing system.

Images