JP7209441B2 - Screen configuration analysis device, screen configuration analysis method and program - Google Patents

Description

The present invention relates to a screen configuration analysis device, a screen configuration analysis method, and a program.

In a broadcast program, when evaluating a certain program after broadcasting, various configurations of the program, such as the arrangement of multiple TV cameras, evaluation of how to use the cameras, confirmation of the progress of a variety program, and audience rating of the program Therefore, there is a demand for the audience watching the program to evaluate and confirm the program after the fact from the viewpoint of which statements of the performers received.

In order to solve this problem, the invention of Patent Document 1 is proposed.

The invention of Patent Document 1 creates a still image of each shot of a video broadcast on TV, and this still image, TV station name, channel name, program title, broadcast date, broadcast start time, broadcast elapsed time, program A content summary and an index with time codes are stored in association with each other. This index, also called metadata, is information about the program.

JP 2013-162337 A

However, in the invention of Patent Document 1, the index does not include information on the time-series presence or absence of the performers on the screen, the layout, the screen configuration such as the occupancy ratio, etc. Visually check the screen composition. On the other hand, as a producer, what kind of impact will the appearance of the performers on the screen in chronological order, and the screen composition of the video, such as placement and occupancy ratio, not appear in the metadata, on the viewer? I want to know what to give

Accordingly, the present invention has been made in view of the above problems, and it is an object of the present invention to provide a screen configuration analysis device, a screen configuration analysis method, and a program capable of analyzing the screen configuration of a content video.

According to one aspect of the present invention, an image recognition unit that recognizes, from a content video of a content, a designated component of interest among screen components forming the content video, and the content when the component of interest is focused on a screen configuration analysis unit that analyzes the screen configuration of a video image and outputs screen configuration information as an analysis result and temporal position information on the content video of the image to be analyzed for the screen configuration; By using an index acquisition unit that acquires time information of an index, temporal position information on the content video, and time information of the index, the screen configuration information and the index are associated on the same time axis, It is a screen configuration analysis device having an association unit that stores in a storage unit.

One aspect of the present invention is to recognize, from a content video of a content, a designated component of interest among screen constituent elements that constitute the content video, and reproduce an image of the content video when focusing on the component of interest. analyzing a screen configuration, outputting screen configuration information as an analysis result and temporal position information on the content video of an image to be analyzed for the screen configuration, acquiring an index and time information of the index; In the screen configuration analysis method, the screen configuration information and the index are associated on the same time axis using the temporal position information on the content video and the time information of the index, and stored in a storage unit.

According to one aspect of the present invention, image recognition processing for recognizing, from a content video of a content, a designated component of interest among screen components forming the content video; screen configuration analysis processing for analyzing the screen configuration of a video image and outputting screen configuration information as an analysis result and temporal position information on the content video of the image whose screen configuration is to be analyzed; The screen configuration information and the indicator are associated on the same time axis by using an indicator acquisition process for acquiring the time information of the indicator and the temporal position information on the content video and the time information of the indicator, It is a program that causes a computer to execute association processing stored in a storage unit.

INDUSTRIAL APPLICABILITY The present invention can analyze the screen configuration of content video.

FIG. 1 is a block diagram showing a screen configuration analysis device according to the first embodiment. FIG. 2 is a diagram showing an example of recognition of the position and size on the screen of the constituent elements of the target screen by the image recognition unit 1. In FIG. FIG. 3 is a diagram for explaining screen configuration analysis and screen configuration information. FIG. 4 is a diagram for explaining screen configuration analysis and screen configuration information. FIG. 5 is a diagram for explaining screen configuration analysis and screen configuration information. FIG. 6 is a diagram for explaining the association between screen configuration information and indicators. FIG. 7 is a block diagram of a screen configuration analysis device configured by a computer system. FIG. 8 is a diagram showing an example of a line graph of the occupancy ratio of performer A in drama X and the audience rating of drama X output to the output unit 7 . FIG. 9 is a block diagram of Modification 1 of the first embodiment. FIG. 9 is a block diagram of Modification 2 of the first embodiment. FIG. 10 is a block diagram of a screen configuration analysis device according to the second embodiment. FIG. 11 is a diagram showing an example of a line graph of the presence or absence of display (appearance) of performer B on the video output to the output unit 7, the numerical value of the voice level of performer B, and the audience rating of content Y. In FIG. FIG. 13 is a block diagram of a screen configuration analysis device according to the third embodiment. FIG. 14 is a block diagram showing an application example of the third embodiment.

<First embodiment>
A screen configuration analysis apparatus according to a first embodiment of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing a screen configuration analysis device according to this embodiment. The screen configuration analysis device includes an image recognition section 1 , a screen configuration analysis section 2 , an index acquisition section 3 , an association section 4 , a storage section 5 , an output control section 6 and an output section 7 .

The image recognition unit 1 recognizes, from an input content video, a specified target screen component among the screen components that configure the screen (video) of the content video.

Here, the screen constituent elements refer to the respective elements that constitute the screen (video) of the content video. People, various objects (buildings, sky, trees, and other natural objects), telops, and the like appear in the content video, and a screen (video) is composed of these elements. In the present embodiment, these elements that form the screen of the content video are described as screen constituent elements. Among the screen constituent elements, a screen constituent element to which attention is focused when analyzing the screen configuration of the content video is described as a target screen constituent element. Typical examples of the focused screen component are specific performers, products, telops, etc. appearing in the content video. Any type of content video can be used as long as the screen components change over time. There are various contents. Moreover, it does not matter whether the content video input to the image recognition unit 1 is a real-time video.

Designation of the target screen constituent element is performed by designating the screen constituent element that the user wishes to analyze to the image recognition section 1 . The designation of the screen component of interest can be done by specifying individual individuals (e.g., specifying a specific moderator or leading actor, etc.) or by not specifying individual individuals (e.g., a person appearing on the method to specify ). Also, the number of target screen components to be specified may be singular or plural.

As image recognition methods for recognizing such a target screen constituent element, there are image recognition techniques using pattern matching and machine learning using techniques such as deep learning.

When the image recognizing unit 1 recognizes the specified target screen constituent element from the content video, the image recognition unit 1 identifies the target screen constituent element (for example, the name if the target screen constituent element is a person, the name if the attention screen constituent element is a telop, (for example, telop, etc.) and attention screen component recognition information including the position, size, etc. of the attention screen component on the screen are output. Recognition and attention screen component recognition information are output at predetermined time intervals (for example, every 1 second or 2 seconds).

As for the recognition of the position and size on the screen of the target screen component by the image recognition unit 1, there is an example shown in FIG. The example of FIG. 2 is an example in which a person is used as the target screen component. As for the detected person, the image recognition unit 1 sets an area surrounding the recognized person as an attention area as shown in FIG. 2, and sets the intersection of diagonal lines of the attention area as the center point. Then, the size of the recognized person is set as the area of the attention area, and the center point is set as the positional coordinates of the recognized person on the screen. This example is an example, and other methods may be used.

The screen configuration analysis unit 2 analyzes the screen configuration of the content video when focusing on at least one or more target screen components recognized by the image recognition unit 1, and generates screen configuration information. In this analysis of the screen configuration, from what point of view the target screen constituent element should be focused on is set, and the screen configuration is analyzed from that point of view.

Screen configuration analysis and screen configuration information will be described with reference to FIGS. 3 to 5. FIG.

Assuming that a performer A appearing in a drama is the screen component of interest, the screen configuration differs depending on how the performer A appears on the screen. The example of FIG. 3 is an example focusing on the screen configuration from the viewpoint of the occupancy rate of performer A (single) on the screen. If A is shown with a screen ratio of 70% or less with the background, scene, and many extras, and if Performer A is shown with a background, scene, and many extras, with a screen ratio of 50% or less, and if Performer A is the background, The example shows a case where the screen ratio is 30% or less with a large number of scenes and extras. In such a case, although they are common in that the performer A is shown, the screen configurations are different, and the respective images (screens) have different effects on the viewer. The same is true when a plurality of performers appearing in a drama are set as the screen constituent elements of interest. Furthermore, the attention screen constituent elements are the characters of various supers of the video of the drama, and the position of the various supers and the characters in the supers (program name, performer name, etc.) The impact is different.

Further, when the focused screen component is the moderator (MC) of an information program, the screen configuration differs depending on how the moderator (MC) appears on the screen. The example in FIG. 4 is an example focusing on the positional relationship between the moderator (MC) and other components. When the person (MC) is shown on the screen with Flip (or CG screen), and when the moderator (MC) is shown (or talking) with other performers (or commentators), A case where the moderator (MC) is shown on the wipe screen on the screen is exemplified. In such cases, the same moderator (MC) is shown, but the screen configurations are different, and the respective images (screens) have different effects on the viewers.

Further, when a certain product is selected as a target screen component, the screen configuration differs depending on how the product appears on the screen. The example in Fig. 5 is an example that focuses on the display form of the product, and illustrates the case where the product is shown alone in a large (close-up) image in the center of the screen and the case where a VTR or CG image introducing the product is shown. However, although they are common in that they show the same product, the screen configuration is different, and the impact of each image (screen) on the viewer is different.

In this way, the viewpoint of attention is set for at least one or more of the attention screen constituent elements recognized by the image recognition unit 1, and the screen composition of the content video is analyzed. As a specific analysis method, an analysis standard that serves as a standard of a viewpoint to be focused on is specified, and a screen configuration is determined based on the analysis standard when attention is paid to a focused screen constituent element.

The simplest analysis criterion is whether or not a specific person such as a performer or moderator appears on the screen when a specific person is designated as a focused screen component. For example, when the performer A is specified as the target screen constituent element, the screen configuration analysis unit 2 sets "1" when the performer A appears on the screen, and "1" when the performer A does not appear on the screen. If so, it is analyzed as "0". Then, the digitized data is generated as screen configuration information. Also, when a plurality of persons are specified as the target screen component, the number of specified target screen components (persons) appearing on the screen may be used. Furthermore, the screen configuration analysis unit 2 also acquires temporal position information (such as the time code of the content video) of the content video of the image to be analyzed. The screen configuration analysis unit 2 then outputs the screen configuration information and the temporal position information.

Also, when a specific person is set as a target screen component, the occupancy rate of the screen occupied by the specific person may be used as an analysis criterion. In this case, the screen configuration analysis unit 2 calculates the ratio of the area of the attention area to the screen area in the example of FIG. 2 as the occupancy rate, and generates the numerical value of the occupancy rate as the screen configuration information. Furthermore, the screen configuration analysis unit 2 also acquires temporal position information (such as the time code of the content video) of the content video of the image to be analyzed. The screen configuration analysis unit 2 then outputs the screen configuration information and the temporal position information. It should be noted that the occupancy rate may be a stepwise numerical value. For example, if the occupation rate of performer A occupying the screen is 0%, "0", if the occupation rate of performer A occupying the screen is 0% or more and less than 30%, "1", performer A "2" if the screen occupation rate is 30% or more and less than 50%, "3" if the screen occupation rate of performer A is 50% or more and less than 70%, and performer A occupies the screen If the occupancy rate is 70% or more, it is set to "4". Then, the digitized data is generated as screen configuration information. Furthermore, the screen configuration analysis unit 2 also acquires temporal position information (such as the time code of the content video) of the content video of the image to be analyzed. The screen configuration analysis unit 2 then outputs the screen configuration information and the temporal position information.

Alternatively, a telop or the like may be set as a screen component of interest, and the position of the telop or the like may be used as the analysis criterion. For example, the screen is divided into nine areas, and identification numbers for identifying the areas are assigned in order from the upper left (for example, upper left area: 1, . . . , lower right area: 9). The screen configuration analysis unit 2 determines to which area the position coordinates of the center point of the target screen component recognized by the image recognition unit 1 belongs, and outputs the identification number of the area to which it belongs as screen configuration information. Furthermore, the screen configuration analysis unit 2 also acquires temporal position information (such as the time code of the content video) of the content video of the image to be analyzed, and outputs the temporal position information.

The analysis of the screen configuration and the screen configuration information by the screen configuration analysis unit 2 described above are examples, and other methods may be used.

The index acquisition unit 3 acquires the index and the time information of this index. The indicators include not only indicators directly related to content, but also indicators obtained by sensing human behavior over time, although they are not directly related to content. The index can also be said to be an index of behavior of viewers of content. It is preferable that the indices to be acquired are indices at predetermined time intervals.

Indexes related to content video content are, for example, audience ratings of broadcast programs, number of time-series accesses to web pages related to content, time-series number of tweets related to content, time-series playback count of content, etc. be. These indicators are indicators that directly reflect the contents of the content, and are important indicators when considering the screen configuration of the content video.

On the other hand, indices obtained by sensing people's behavior over time do not directly reflect the contents of content, but they may be useful indices in examining the screen composition of content video. For example, the presence or absence of the time series of people in front of the TV receiver obtained by sensing with the human sensor attached to the TV receiver, the time series obtained by sensing with the water meter installed in the house. water volume, etc. In soccer game content, there are often people in front of the TV receiver to watch it during the match, but at halftime they do housework, etc. that they had put up with until then, so the content is left displayed. , tend to move away from the front of the TV receiver. Then, the human sensor will stop detecting people, and the amount of water supply will increase. In this way, unlike the audience rating of content, it is not a direct indicator of content, but it can be a useful indicator when considering the structure of content.

Alternatively, time-series data from sensors provided in each home appliance may be used as an index. For example, it is time-series data of turning on/off the power of a television receiver, pressing various buttons of a television remote controller, opening and closing a refrigerator door, and the like. These data are also useful indicators for analyzing interest in content.

In addition, when a display device (for example, a television receiver) that displays content is provided with a line-of-sight detection sensor, the viewer's gaze area on the screen in chronological order may be used as an index. Furthermore, if the line-of-sight detection sensor can also detect the size of the viewer's pupil, the time-series change in the size of the viewer's pupil may be used as an index. This is because there is an experimental result that the pupil diameter widens for "interesting" objects and narrows for uninteresting objects, which is a useful index for analyzing interest in content.

Moreover, when various sensors are attached to the human body, time-series data of changes in the human body obtained from the various sensors may also be used as an index. This is because the viewer's body changes due to the viewer's interest in the content.

The association unit 4 associates the screen configuration information acquired from the screen configuration analysis unit 2 and the index acquired from the index acquisition unit 3 on the same time axis and stores them in the storage unit 5 . For example, with the time code of the content as the time axis, the screen configuration information and the index having the same time information are associated with each other and stored in the storage unit 5 . In the example of FIG. 6, screen configuration information A and index A are stored in association with content time code 1:00:25, and screen configuration information B and index B are associated with content time code 1:00:27. The screen configuration information C and the index C are associated and stored at the content time code 1:00:29, and the screen configuration information D and the index D are associated and stored at the content time code 1:00:31. It is

The storage unit 5 is a recording medium such as a hard disk.

When the analysis standard and the index are specified, the output control unit 6 outputs to the output unit 7 time-series information representing the relationship between the screen configuration information of the analysis standard and the index in time series. Note that the output screen configuration information and the index may not be in a one-to-one relationship. A plurality of indices may be output simultaneously. In this case, the output screen configuration information and the index are preferably output in association with the same time axis.

The output unit 7 is output means such as a display.

Specifically, the screen configuration analysis device described above can be realized by a computer system (information processing device) having a processor that performs various kinds of arithmetic processing. FIG. 7 is a block diagram of a screen configuration analysis device configured by a computer system.

The screen configuration analyzer includes hardware such as a processor 10, a memory (ROM or RAM) 11, a storage device (hard disk, semiconductor disk, etc.) 12, an input device (keyboard, mouse, touch panel, etc.) 13, a display device 14, a communication device 15, and the like. It can be configured by a general-purpose computer having hardware resources.

The program stored in the storage device 12 is loaded into the memory 11 and executed by the processor 10 to perform image recognition processing, screen configuration analysis processing, index acquisition processing, association processing, Output control processing is realized. The image recognition processing corresponds to the image recognition unit 1, the screen configuration analysis processing corresponds to the screen configuration analysis unit 2, the index acquisition processing corresponds to the index acquisition unit 3, the association processing corresponds to the association unit 4, Output control processing corresponds to the output control unit 6 .

Next, a specific embodiment of the screen configuration analysis device described above will be described.

In this embodiment, the content is the drama X of the broadcast program, the attention component is the performer A appearing in the drama X, the analysis criterion is the screen occupancy ratio of the performer A (single), and the index is the drama X. An example of the audience rating of .

The image recognition unit 1 recognizes the performer A from the drama X video. Recognition shall be performed every second. Then, the image recognition unit 1 outputs the area of the attention area of the performer A and the time code of the video in which the performer A is recognized.

Based on the area of the attention area of performer A received from the image recognition unit 1 and the time code, the screen configuration analysis unit 2 obtains the occupation ratio of the area of the attention area of performer A on the screen specified by the time code. . The occupancy ratio here ranges from 0 to 10, where 0 is when performer A is not displayed on the video screen, and 10 is when the area of the attention area of performer A matches the area of the screen. demand. The screen configuration analysis unit 2 outputs the obtained occupancy ratio of performer A and the time code at that time to the association unit 4 .

The index acquisition unit 3 acquires the audience rating of the drama X in 1-second units. Note that audience rating data can be obtained from existing services.

The association unit 4 converts the occupancy ratio (screen configuration information) of the performer A received from the screen configuration analysis unit 2 and the audience rating of the drama X received from the index acquisition unit 3 into the time code of the drama X. is stored in the storage unit 5 in association with .

When time-series information of the occupation ratio (screen configuration information) of performer A in drama X and the audience rating (indicator) of drama X, which are the analysis criteria, are designated, the output control unit 6 sets the horizontal axis to the time code. (time), the first vertical axis is the audience rating, and the second vertical axis is the occupation ratio of the performer A in the drama X, and the line graph of the audience rating of the drama X. , is output to the output unit 7 as time-series information. FIG. 8 shows an example of a line graph of the occupancy ratio of cast A in drama X and the audience rating of drama X output to the output unit 7 . It can be seen from FIG. 8 that there is a correlation between the occupancy ratio of performer A and the increase in audience rating.

In this way, by arranging the relationship between the screen configuration information of the image of the content video and the index when focusing on the component of interest in chronological order, the screen configuration of the content video can be examined and evaluated objectively.

Note that the output control unit 6 may display the content video in time series along with the time series information.

Furthermore, the screen configuration analysis unit 2 generates screen configuration information and also acquires temporal position information of the content video such as a time code, which is stored in the recording unit. It can also be used as metadata for For example, if the screen configuration information indicates whether or not the performer appears on the screen, it is possible to select only the video in which the performer appears using the temporal position information of the screen configuration information in which the performer appears. can.

<Modification 1 of the first embodiment>
Modification 1 of the first embodiment will be described.

FIG. 9 is a block diagram of Modification 1 of the first embodiment.

Modification 1 of the first embodiment removes the output control unit 6 and the output unit 7 from among the components of the image analysis apparatus in the first embodiment described above, and removes the image recognition unit 1 and screen configuration analysis. The image analysis apparatus 100 is configured by the unit 2 , the index acquisition unit 3 , the association unit 4 and the storage unit 5 . A terminal 200 connected to the image analysis apparatus 100 via a network such as a LAN includes an output control section 6 and an output section 7 .

With such a configuration, a plurality of terminals can browse the time-series relationship between the screen configuration and the indices analyzed and collected by the image analysis apparatus 100 .

<Modification 2 of the first embodiment>
Modification 2 of the first embodiment will be described.

Modification 2 of the first embodiment is provided with a plurality of image analysis apparatuses 100 of Modification 1 of the first embodiment. It is configured so that the output control unit 6 of the terminal 200 can compare and output.

FIG. 10 is a block diagram of Modification 2 of the first embodiment.

In modification 2 of the first embodiment, a plurality of image analysis apparatuses 100 are provided, and each image analysis apparatus 100 stores different program images M, program images N, . . . , or the program image Z is input.

The constituent elements of interest are designated to each image analysis apparatus 100. When the constituent elements of interest are performers, each program video M, program video N, . . . , the performers of the program video Z are different, so the target components to be designated are also different. This is because, for example, the program image M is the performer A, and the program image N is the performer B. However, it is preferable that the concept of the component of interest to be focused on is common, such as the performers appearing in the program.

On the other hand, the analysis criteria specified for each image analysis apparatus 100 are preferably the same analysis criteria. For example, each program video M, program video N, . . . , the occupation ratio of the screen of the cast of the program image Z, and the like.

Further, the indices acquired by the index acquisition unit 3 are preferably of the same type from the viewpoint of comparison. For example, each program video M, program video N, . . . , is the audience rating of each program of the program video Z.

In this way, each image analysis device 100 can generate each program image M, program image N, . . . , the program image Z is analyzed, and the screen configuration information and the index are stored in association with each other on the same time axis.

When the analysis criteria and indices are specified, the output control 6 reads out the screen configuration information and indices from each image analysis device 100, and expresses the relationship between each screen configuration information and each index on the same screen in time series. The resulting time-series information is output to the output unit 7 . For example, each program video M, program video N, . . . , screen configuration information of program video Z (for example, the screen occupation ratio of performers in each program) and each program video M, program video N, . . . , and indexes of the program video Z (for example, audience rating of each program) are arranged in chronological order on the same screen to output time-series information.

By configuring in this way, it is possible to compare the relationship between a plurality of content videos and the index, and to evaluate the transition of the viewer's program accompanying the change of the screen configuration information.

Note that program video M, program video N, . . . , program video Z is not analyzed in real time, each program video M, program video N, . . . , program video Z, the output control 6 analyzes each program video M, program video N, . . . , screen configuration information of program video Z and each program video M, program video N, . . . , and the index of the program image Z may be output as time series information arranged in time series on the same screen.

<Second Embodiment>
A second embodiment will be described.

The sound contained in the video of the content is also considered to have a great influence on the viewer. Therefore, in the second embodiment, an example will be described in which the content video is analyzed in addition to the speech included in the content video as a recognition target.

FIG. 11 is a block diagram of a screen configuration analysis device according to the second embodiment.

The screen configuration analysis device of the second embodiment includes a speech recognition unit 20 in addition to the screen configuration analysis device of the first embodiment.

The speech recognition unit 20 recognizes the speech of the component of the screen of interest. In order to recognize the voice of the target screen constituent element, the voice recognition unit 20 learns the voice of the target screen constituent element by machine learning or the like. The voice to be recognized is the voice of a person when the target screen component is a person, and the sound produced by the target screen component when the target screen component is an object such as a musical instrument. Note that the speech recognition unit 20 is not limited to the sound of the constituent element of the screen of interest. For example, when the sound exists alone instead of as an image, such as an opening song or background music, the sound itself is recognized. You may make it recognize as an attention component.

The information of the voice recognized by the voice recognition unit 20 includes not only the presence or absence of the voice of the target screen component, but also the level, frequency, pitch, speed, and loudness value of the voice emitted by the target screen component. Then, attention component speech recognition information, which is information about the content of the identified speech, is generated.

The screen configuration analysis unit 2 uses the target screen component recognition information received from the image recognition unit 1 and the target component voice recognition information received from the voice recognition unit 20 to analyze the content video screen according to the set analysis criteria. Analyze your configuration. Examples of analysis criteria include a person's voice level, frequency, pitch, speed, loudness value, etc. when the person appears on the video when the target screen configuration is a person.

The screen configuration analysis unit 2 analyzes the screen configuration of the content video according to the analysis standard, and outputs screen configuration information and temporal position information.

The operations of the index acquisition unit 3, the association unit 4, the storage unit 5, the output control unit 6, and the output unit 7 are the same as in the first embodiment.

Explaining a specific embodiment In this embodiment, the components of interest are a certain performer B appearing in the content Y and the voice of the performer B, and the analysis criterion is the video of the performer B (single) An example will be described in which presence/absence of display (appearance) and voice level of performer B are used, and the audience rating of content Y is used as an index.

The image recognition unit 1 recognizes the performer B from the content Y video. Recognition shall be performed every second. Then, the image recognition unit 1 outputs the area of the attention area of the performer B and the time code of the video in which the performer B is recognized.

The voice recognition section 20 recognizes the voice of performer B and outputs the voice data to the screen configuration analysis section 2 .

Based on the area of attention area of performer B received from image recognition unit 1 and the time code, screen configuration analysis unit 2 determines whether or not performer B is displayed (appears) in the image on the screen specified by the time code. to decide. That is, if the area of the attention area of performer B is not zero, it means that performer B is displayed (appears) on the video. The screen configuration information here is 1 when the performer B is displayed (appears) on the video, and is 0 when the performer B is not displayed (appears) on the video.

Further, the screen configuration analysis unit 2 determines the voice level of the performer B based on the voice data of the performer B from the voice recognition unit 20 . The level is determined in 10 stages, and the higher the level, the higher the numerical value. Then, the screen configuration analysis unit 2 outputs the numerical value of the audio level as the screen configuration information.

The index acquisition unit 3 acquires the audience rating of the content Y in units of one second.

The association unit 4 acquires the screen configuration information of the performer B received from the screen configuration analysis unit 2 (presence or absence of display (appearance) of the performer B on the video and the numerical value of the voice level of the performer B) and an index. The viewing rate of the content Y in units of one second received from the unit 3 is stored in the storage unit 5 in association with the time code of the content Y, the screen configuration information corresponding to the time code, and the viewing rate of the content Y. .

When the screen configuration information of the performer B of the content Y and the time-series information of the audience rating of the content Y, which are the analysis criteria, are specified, the output control unit 6 sets the horizontal axis as the time code (time) and the vertical axis. The first axis is the audience rating of content Y, and the second axis of the vertical axis is the presence or absence of display (appearance) on the video of performer B and the numerical value of the tone of performer B's voice on the video of performer B The presence or absence of display (appearance), the numerical value of the voice level of performer B, and the line graph of the audience rating of content Y are output to the output unit 7 as time-series information. FIG. 12 shows an example of a line graph of the occupancy ratio of cast A in drama X and the audience rating of drama X output to the output unit 7 . In FIG. 12, it can be seen that there is a correlation between the voice level of performer B and the increase in audience rating.

In this way, by arranging the relationship between the screen configuration information of the image of the content video and the index when focusing on the component of interest in chronological order, the screen configuration of the content video can be examined and evaluated objectively.

It should be noted that the second embodiment can also be configured in combination with the modification of the first embodiment described above.

<Third Embodiment>
FIG. 13 is a block diagram showing a screen configuration analysis device according to the third embodiment. The third embodiment has a correlation analysis section 30 in addition to the second embodiment.

The analysis unit 30 uses the screen configuration information and the index stored in the storage unit 5 as input data and analyzes the correlation between the screen configuration information and the index. For example, it is information such as how the index changes as the screen configuration information changes, and presentation of the screen configuration information when a characteristic change occurs in the index. These can be performed by analyzing only one content video, but it is more effective to analyze the screen configuration information and indices of a plurality of content as described in Modification 2 of the first embodiment. Become.

As an example of the analysis by the correlation analysis unit 30, the correlation analysis unit 30 detects a characteristic change in the index, such as a rapid increase or a sudden drop in the audience rating of a specific content, or a simultaneous decrease in the detection value of the human sensor, The screen configuration information of the content when the characteristic change occurs and its index data are output. Further, the correlation analysis unit 30 detects periodic characteristic changes in the index, such as appearing at the same time every day or on the same day of the week. and

In addition, the analysis unit 30 is made to learn a plurality of pieces of screen configuration information and index data stored in the storage unit 5, and what kind of index tends to change with what kind of screen configuration information, or , what kind of index tends to change when what kind of screen configuration information changes, or what kind of screen configuration information tends to affect the value of the index Analytical information can also be provided. Such analysis methods include those based on pattern matching and machine learning using techniques such as deep learning.

Such analysis results of the correlation analysis unit 30 can also contribute to automation of content editing.

FIG. 14 is a block diagram showing an application example of the third embodiment.

In the example of FIG. 14 , the analysis result of the correlation analysis section 30 is input to the content production system 40 . To explain an example of the content production system 40, as a result of analysis, the screen configuration information that is recognized to increase the audience rating, for example, the main character is displayed on the video with a screen share of 50 to 70% during a specific time period of the content. It is assumed that there is an analysis result that being reflected greatly contributes to an increase in audience rating. In the case of creating a single content video from images shot with multiple cameras, the screen share of the main actor in each camera's video is monitored, and during a specific time period, the main actor accounts for 50-70% of the screen. By automatically selecting a camera image captured with a screen occupation ratio of , the switching processing of the camera image can be automated. The same is true for camera work (zoom, pan, etc.) performed by a single camera.

The third embodiment can estimate the relationship between the screen configuration information and the index by providing the correlation analysis unit. Furthermore, by using the analysis results, it is possible to contribute to the automation of content production.

Although the present invention has been described above with reference to preferred embodiments, it is not necessary to include all the configurations of the embodiments, and not only can they be combined as appropriate, but the present invention is not necessarily limited to the above embodiments. However, it can be modified and implemented in various ways within the scope of its technical ideas.

1 image recognition unit 2 screen configuration analysis unit 3 index acquisition unit 4 association unit 5 storage unit 6 output control unit 7 output unit 20 voice recognition unit 30 correlation analysis unit 40 content production system 100 screen configuration analysis device

Claims (6)

an image recognition unit for recognizing a designated person by image recognition from content video of content;
analyzing a specified person occupancy ratio, which is an occupancy ratio of the specified person on the screen of the content video , converting the specified person occupancy ratio into a representative value of a class to which the specified person occupancy ratio belongs , and a screen configuration analysis unit that outputs temporal position information on the content video of the image to be analyzed for the specified person occupancy ratio;
an index acquisition unit that acquires the audience rating of the content and the time information of the audience rating of the content;
an associating unit that associates the representative value with the audience rating of the content on the same time axis using the temporal position information on the content video and the time information of the audience rating of the content, and stores the result in a storage unit; ,
an output control unit that outputs time-series information of the representative value and the audience rating of the content using the representative value and the audience rating of the content that are associated on the same time axis;
A screen configuration analyzer having providing a plurality of screen configuration analysis devices each having the image recognition unit, the screen configuration analysis unit, the index acquisition unit, and the association unit;
causing each of the screen configuration analysis devices to analyze different content videos;
The output control unit outputs time-series information of a representative value of each content video and an audience rating of the content.
The screen configuration analysis device according to claim 1 . a correlation analysis unit that analyzes the correlation between the representative value and the audience rating of the content, using the representative value and the audience rating of the content stored in association on the same time axis;
3. The screen configuration analysis device according to claim 1 or 2 . A video production system that controls video production equipment using the analysis result of the correlation analysis unit
The screen configuration analysis device according to claim 3 . The information processing device
Recognize the designated person by image recognition from the content video of the content,
analyzing a specified person occupancy ratio, which is an occupancy ratio of the specified person on the screen of the content video , converting the specified person occupancy ratio into a representative value of a class to which the specified person occupancy ratio belongs , and outputting temporal position information on the content video of the image to be analyzed for the specified person occupancy ratio;
Acquiring the audience rating of the content and the time information of the audience rating of the content,
By using the temporal position information on the content video and the time information of the audience rating of the content, the representative value and the audience rating of the content are associated on the same time axis and stored in a storage unit;
outputting time-series information of the representative value and the audience rating of the content using the representative value and the audience rating of the content that are associated on the same time axis;
Screen composition analysis method. Image recognition processing for recognizing a specified person from content video of content by image recognition;
The specified person occupancy ratio, which is the occupancy ratio of the specified person on the screen of the content video, is analyzed, the specified person occupancy ratio that is the analysis result is converted into a representative value of the class to which the specified person occupancy ratio belongs, and the representative screen configuration analysis processing for outputting a value and temporal position information on the content video of the image targeted for analysis of the specified person occupancy ratio;
an index acquisition process for acquiring the audience rating of the content and the time information of the audience rating of the content;
an association process of associating the representative value and the audience rating of the content on the same time axis using the temporal position information on the content video and the time information of the audience rating of the content, and storing the result in a storage unit; ,
an output process of outputting time-series information of the representative value and the audience rating of the content using the representative value and the audience rating of the content that are associated on the same time axis;
A program that makes a computer run

Images