JP7137825B2 - Video information provision system - Google Patents

Description

The present invention relates to a video information providing system, and more particularly to a video information providing system that determines the frequency of appearance of video content appearing on TV and on the web, and timely provides users with video content that has a high frequency of appearance.

Currently, a large amount of diverse video information is continuously distributed to viewers and users via TV and the web. It is becoming difficult for viewers and users to appropriately and timely extract useful video content from a large amount of video information that is automatically provided 24 hours a day. .

In other words, busy modern people visually recognize all the multiple programs and a large amount of video information distributed from the broadcasting station over a predetermined period of time in front of the TV receiver, and select from among them what they need. There is a situation where it is gradually becoming impossible to obtain video content and information that are of interest to the public.

In addition, there is a similar situation regarding the acquisition of information from the Web. That is, when trying to obtain necessary information from the web, information is obtained by keyword search using an Internet search engine such as Google (registered trademark). Even information similar to the desired information, referred to as "noise", is extracted in large amounts.

Therefore, even if the user uses a search engine, the user still needs to extract the desired information from a large amount of related information, and the search work is still troublesome. .

By the way, conventionally, for example, there is known a technique of searching for a video similar to a pre-registered video for a certain period of time, from among videos output by TV broadcasting or the like as video content.

Such search technology can detect, for example, a specific title roll in a television broadcast signal to start/stop real-time recording, or detect the same news material broadcasted at different times and on different stations to detect the structure of a video. It is used in techniques such as analysis (see, for example, Patent Document 1).

In addition, such a search technique is not limited to television broadcasting, and can also target distribution data such as video content received via an Internet line (see, for example, Patent Document 2).

Furthermore, such search techniques are not limited to video, but can also work with text, for example. Specifically, in addition to the caption text included in the video content, metadata such as the broadcast start time, broadcast end time, performers, and a summary of the contents of each corner of the broadcast program are displayed after the end of the broadcast program. It is possible to adopt metadata provided by the service provider of the paid service (also called program metadata service) to be distributed, or text that describes the video content that the user inputs by operating the keyboard, etc. (See Patent Document 2, for example).

JP 2010-262413 A JP 2012-038239 A

However, these techniques target, for example, one program or video content, and the viewer or user, as described above, has a high degree of importance or interest among all the programs provided on TV. Requests for time-efficient acquisition of useful video content with high interest rate, and time-efficient acquisition of important or high-interest useful video content from all video content uploaded on the web. It was not something that could respond to the request that I want to do it.

The object of the present invention is to provide a video information providing system capable of quickly and appropriately supplying useful video content to users from among a large number of distributed video content, in order to solve the above problems. to provide.

In order to achieve the above object, a video information providing system according to the present invention comprises: receiving means for receiving distributed video content; storage means for storing and accumulating the video content received by the receiving means; component appearance for determining the frequency of appearance of the same component by analyzing the components of the video content thus obtained and comparing the analyzed component with a large number of past video contents stored in the storage means. frequency determining means, and based on the determination by said component appearance frequency determining means, video content containing said component when the appearance frequency is equal to or greater than a predetermined value, is selected from a large number of past video contents accumulated in said storage means. video content extracting means for extracting; and video content providing means for providing a viewer with the video extracted by the video content extracting means, wherein the component appearance frequency determining means is included in the distributed video content. Analyzing the constituent elements, comparing the analyzed constituent elements with a large number of the past video contents accumulated in the storage means, judging the appearance frequency of the same constituent elements, and determining the appearance of the predetermined constituent elements. When it is determined that the frequency has reached a predetermined value in a predetermined period of time, the video content extracting means extracts the video content including the component from a large number of video content stored in the storage means, and extracts the video content. The video content extracted by a providing means is provided to a viewer .

In the distributed video content, the component appearance frequency determination means analyzes the components included in the video content and compares them with the video content stored in the past to determine the appearance frequency of the same component. be done. Here, the video content is mainly moving images, but still images are also included. In addition, voices and texts, which are image components existing in moving images, are also subject to determination.

Then, when it is determined by the component appearance frequency determining means that the appearance frequency of a predetermined component has reached a predetermined value in a predetermined period, the video contents including the component are accumulated by the video content extracting means. Extracted from a large number of video contents.

After that, the video content extracted by the video content providing means is provided to the viewer. In this case, provision of the video content includes not only provision of the video content by display, but also recording/distribution of the video content and other modes of output.

In this case, AI (artificial intelligence) is used for the component appearance frequency determination means and the video content extraction means, and high-speed data processing is performed.

In the invention according to claim 2, the constituent element appearance frequency determination means is characterized in that the appearance frequency of the constituent elements is determined based on viewing wishes registered by the viewer.

Therefore, the viewer, who is the user of this system, can register the video he or she desires to view, the classification, type, genre, etc. of the video within an appropriate designated range, and the component appearance frequency determining means can Based on the above, the frequency of appearance of the constituent elements is determined.

The invention according to claim 3 is characterized in that the constituent element is a character, a voice, a video object that is the theme of the video content, a character, or a background of the video object.

In the case of video images, how the component appearance frequency determination means recognizes and identifies components in specific video content is described in terms of "characters, voices, video objects that are themes of the video content, characters or from the point of view of the "background of the image object".

In this case, the "characters" are characters such as telops that appear in moving images, and are performed based on so-called character recognition technology, using morphological analysis technology.

Also, "sound" refers to various sounds and human voices included in moving images and the like, including background sounds, theme songs, voices of characters, sound effects, and the like. If necessary, only the speaking voice of a specific character on the theme of the video is extracted from the various voices, and the frequency of appearance is determined by the component appearance frequency determination means.

In this case, when specifying only the speaking voice of a person who is the subject of a specific video, it is possible to extract only the speaking voice of the person by specifying the frequency band of the voice of the person.

In addition, "theme video object" is the subject of the video, and "character" includes a person who appears as a main character or a supporting character in relation to the theme of the video. . In this case, facial recognition technology or the like is used to identify characters. Also, the 'background of an image object' refers to what appears as the background of a character or an object that is the subject of an image, such as a building, the sea, a mountain, the sky, a plain, and the like.

A fourth aspect of the present invention is characterized in that the constituent element appearance frequency determination means judges the appearance frequency of the constituent elements by referring to the title portion displayed by characters in the video screen.
That is, in the case of a news video distributed from a TV broadcasting station, a title portion composed of text may be displayed in the upper right portion of the video displayed on the screen. , perform video recognition, voice recognition, person recognition, background recognition, etc., and determine the frequency of appearance of constituent features.
The invention according to claim 5 is characterized in that the video content is distributed from a server via an electric communication line.

Therefore, in the invention according to claim 5, the element appearance frequency determination means determines the application frequency of the same element for video content including moving images distributed via the web. Here, "video distributed via the web" means various videos uploaded on the web, videos on SNS (social network service), videos distributed via the Internet (for example, NETFLIX [registered trademark], amazon FIRE TV), etc.
Also, in the following description, the term "web" is not limited to web systems viewed with a browser, and includes cloud systems. Therefore, video content includes cases where it resides on either a web server or a cloud server.

The invention according to claim 6 is characterized in that the video content is distributed from a broadcasting station via an electric communication line.

Therefore, in the sixth aspect of the present invention, the component appearance frequency determination means determines the application frequency of the same component for all video contents displayed as TV images.

In the video information providing system according to the invention of claim 7, there is provided a summary creation means for creating a summary by converting audio data or video data in the plurality of video contents extracted from each video signal of the plurality of video contents into text. weighting specific to one or more predetermined conditions included in the summary for the plurality of video contents while learning the optimum condition based on the accumulation result of the summary created by the summary creation means. and weighting means for assigning weights, wherein the element appearance frequency determination means determines the appearance frequency of the elements based on the summary.

In the seventh aspect of the present invention, the constituent element appearance frequency determination means determines the frequency of appearance of the video content, ie, the degree of importance, based on the summary of the video content.

That is, the summarization means determines the frequency of appearance of the constituent elements based on a textual summary of the audio data or video data of the plurality of video contents extracted from each video signal of the plurality of video contents.

In this case, the summary created by the summary creation means is applied to the plurality of video contents by applying weights specific to at least one or more predetermined conditions to the plurality of video contents by the weighting means. To provide users with more useful information of high importance and interest.

The invention according to claim 8 is the video information providing system according to claim 7, wherein the weighting means refers to duplicate text from audio data or video data included in the plurality of video contents, When the reference result is equal to or greater than a predetermined value, the text is determined to be an important text that meets the predetermined condition and is weighted.

For example, in daily broadcast news programs, the web, SNS, and the like, it can be said that topics with great social news value, information on incidents and accidents, etc., are of high social importance and interest. As a result, information about such social events often appears as multiple video contents in multiple media.

Therefore, in multiple video contents, overlapping texts are judged to be the same constituent element by morphological analysis of the text, etc., and according to the frequency of appearance, for example, the number of appearances, the time of appearance, the appearance rate, etc. The importance is appropriately set.

According to a ninth aspect of the invention, there is provided the video information providing system of the eighth aspect, wherein the weighting means determines whether or not a plurality of video contents within a preset period are important texts. Characterized by

The period required for judging the degree of social importance of information such as the topic, incident or accident is judged within a predetermined period.

The invention according to claim 10 is the video information providing system according to claim 8 or 9, wherein the weighting means determines that new video content is video content including the important text. is characterized by starting recording of the video content.

In news programs that are broadcast daily, it can be said that major incidents and accidents are of high social importance and interest. Therefore, if such incidents, accidents, etc., are recorded and stored in a large-capacity storage section separately from those used for summarization, for example, they can be edited by operators, etc., and left as recorded images.

According to the eleventh aspect of the invention, in the video information providing system according to the sixth aspect, the weighting means assigns a specific It is characterized by calculating a CM conversion value from the frequency of appearance of people and objects.

The specific conditions mentioned above can include specific persons, objects, incidents, and accidents. Here, for example, by targeting a specific person (including corporate name, various organizations, etc.) as a specific condition, the frequency of appearance of a specific person from a plurality of video contents, such as the number of appearances, the appearance time, The appearance rate, etc. can be calculated.

Then, based on the number of appearances, appearance time, appearance rate, etc., it is possible to calculate a CM conversion value that serves as a guideline of how much a person's value (for example, setting a guarantee) is in the world of mass media and information. . In addition, when targeting a specific corporate name, analysis such as the ratio of commercials provided to the entire commercial, the importance of the various products sold by the corporation (degree of effort and identification of new products), and the correlation with stock price fluctuations It can also be used for (weighting) and the like.

In the invention according to claim 12, the weighting means targets a specific corporate name from the text included in at least one of the audio data and the video data of the plurality of video contents. do.

Therefore, in the invention according to claim 12, since the specific corporate name is used as a component for determining the appearance frequency, the video content in which the specific corporate name appears is extracted, and the viewer as the user is extracted. provided to the legal entity that is

In the inventions according to claims 1 to 12, more useful information can be automatically provided to the user from the video system side for a plurality of video contents.

That is, the video content related to the delivered moving image or still image is analyzed by the element appearance frequency determination means, and the elements included in the video content are analyzed and compared with the video content accumulated in the past. Appearance frequency of a component is determined, and when it is determined that the appearance frequency of a predetermined component reaches a predetermined value in a predetermined period, a large number of video contents in which video contents including the component are accumulated by the video content extracting means. Since the video content extracted by the video content providing means is configured to be provided to the viewer, the viewer does not need to constantly watch the video content delivered to the TV or the web, and automatically It is possible to check and comprehend important information by viewing only the video content that appears on TV or the web more than a predetermined frequency that is publicly provided.

As a result, video content that appears frequently is often important information or trendy information at that time, so viewers always check multiple TV channels and web information. There is no need, and only video content consisting of important information provided by the system according to the present invention and trendy information at that time can be viewed in a time efficient manner without being bound by time to a TV receiver, personal computer or smartphone. can be viewed.

Therefore, even if the viewer is in a busy environment, he or she does not need to constantly check the TV program on the TV receiver or frequently check the web information on the personal computer or smart phone. It is effective in being able to acquire information efficiently.

In the invention according to claim 2, the component appearance frequency determination means is configured to determine the appearance frequency of the component based on a viewing request registered by a viewer, who is a user of the system. The viewer can register the video he/she wishes to view, the classification, type, genre, etc. of the video in an appropriate designated range, and the component appearance frequency determination means determines the appearance frequency based on the request. Therefore, the user registers the genre, type, etc. of the video content that the user desires in the system in advance, and automatically and efficiently displays the video content that appears frequently within the range of the genre and type. It is possible to watch.

Also, for example, if a specific company that is a user wishes to collectively manage information related to the company that appears on TV or the web, register the company name, corporate identity, company name abbreviation, etc. By doing so, it becomes possible to automatically collect and view all TV video content or video information on the web related to the company, and use it for business management of the company from various points of view.
In the invention according to claim 4, regarding the video information transmitted from the TV broadcasting station, for example, in the case of a news program, the title of the news is displayed in a short text at a position such as the upper right in the screen. may be displayed. In such a case, when video recognition, voice recognition, person recognition, and background recognition are performed to determine the appearance frequency of the constituent elements, the title part can be referred to. In this way, when video recognition, voice recognition, person recognition, and background recognition are performed by referring to the title part, more accurate recognition can be performed, and more efficient determination of the appearance frequency of constituent elements can be performed. It becomes possible to

In the invention according to claim 5, the constituent element appearance frequency determination means detects various moving images uploaded on the web, all moving images on SNS (social network service), various moving images distributed over the Internet, etc. Since the appearance frequency is determined as a target, we extract video content with high importance from all the video content that is circulated in the world through the web, and comprehensively identify video content with high importance for the viewers who are users. It will be possible to search and provide quickly and automatically.

According to the sixth aspect of the invention, the element appearance frequency determination means determines the application frequency of the same element for all video contents displayed as TV images. It is possible to extract video content of high importance from the video content of the user, comprehensively search for video content of high importance to the viewer who is the user, and provide it quickly and automatically.

In the inventions according to claims 7 to 12, since the component appearance frequency determination means is configured to determine the frequency of appearance of the video content, that is, the degree of importance, based on the summary of the video content, Compared to narrowing down the subject of each video directly from the video content using various advanced technologies such as face recognition technology, voice recognition technology, and morphological recognition technology, it is possible to determine the frequency of appearance of the relevant component more quickly and accurately. can be done.

In the invention according to claim 7, information related to social events often appears as multiple video contents in multiple media. Since the text is judged to be the same constituent element, and the importance is appropriately set in stages according to the frequency of appearance, for example, the number of appearances, the time of appearance, the appearance rate, etc. The frequency of appearance of the constituent elements, that is, the degree of social importance is determined accurately, and video content with an appropriate degree of importance is provided to the user.

According to the ninth aspect of the invention, since the period required for judging social importance is determined within a predetermined period, it is possible to quickly provide socially important video contents to users. It becomes possible.

According to the invention of claim 10, if socially important incidents, accidents, etc., are recorded and stored in a large-capacity storage unit separately from those used for summarization, they can be edited and recorded by operators, etc. It is possible to provide a video information providing system that can retain and reuse collected video content that is important to the user for analysis or the like as appropriate.

The invention according to claim 11 can provide a video information calculation system that can use the CM conversion value calculated by the weighting means as an analysis index for various economic activities.

In the twelfth aspect of the invention, since the specific corporate name is used as a component for determining the appearance frequency, the video content in which the specific corporate name appears is extracted, and the viewer is the user. Since it is provided to the corporation concerned, for example, it is possible to quickly, appropriately and comprehensively collect information on the company's social evaluation, reputation, etc., and to promptly reflect it in the management of the company.

1 is a block diagram showing the overall configuration of a first embodiment of a video information providing system according to the present invention; FIG. 1 is a block diagram showing the overall configuration of a second embodiment of a video information providing system according to the present invention; FIG. FIG. 10 is a block diagram showing the overall configuration of a summary creation system according to a second embodiment of the video information providing system according to the present invention; FIG. 4 is a block diagram showing a speech-to-text unit in the second embodiment of the video information providing system according to the present invention, where (a) is a block diagram and (b) is a diagram showing the flow of processing; FIG. 4 is a block diagram showing a telop text conversion unit of a summary creation system in the second embodiment of the video information providing system according to the present invention, where (a) is a block diagram and (b) is a diagram showing the flow of processing; FIG. 4 is a block diagram showing a background image text conversion unit of a summary creation system in the second embodiment of the video information providing system according to the present invention, where (a) is a block diagram and (b) is a diagram showing the flow of processing; . FIG. 10A is a block diagram showing a logo mark text conversion unit of a summary creating system in a second embodiment of the video information providing system according to the present invention, and FIG. FIG. 4 is a block diagram showing a text integration unit of a summary creation system in the second embodiment of the video information providing system according to the present invention; FIG. 10 is a block diagram showing a summary creation unit of a summary creation system in the second embodiment of the video information providing system according to the present invention; FIG. 10 is a flow chart showing an operation process of a summary creating system in the second embodiment of the video information providing system according to the present invention; FIG. 1 shows an application example of the video information providing system according to the present invention, (A) is an explanatory diagram when it is determined that the desired condition is met by character recognition, and (B) is the desired condition by voice recognition. FIG. 11 is an explanatory diagram when it is determined that

FIG. 1 is a block diagram showing a first embodiment for realizing a video information providing system 1 according to the present invention.
The video information providing system 1 according to the present embodiment aims to quickly and appropriately supply useful video content for users from a large number of distributed video content. A component appearance frequency determining means 5a for determining the appearance frequency of the same component based on the element, and a video containing the component when the appearance frequency is equal to or higher than a predetermined value based on the determination by the component appearance frequency determining means 5a. It comprises video content extracting means 5b for extracting content, and video content providing means 7 for providing the video extracted by the video content extracting means 5b to viewers.

In this embodiment, as shown in FIG. 1, the video information providing system 1 can be used only with a playback device 9 such as a television, a personal computer, a smart phone, a tablet terminal, etc., having a single computer function. A receiving unit 2 equipped with a tuner or the like for receiving video signals related to content from a television station (television broadcasting station) or a web video distribution server, and an operation unit (including a remote control, etc.) equipped in the playback device 9 3, a storage unit 4 storing applications for realizing various functions of the playback device 9, a control circuit unit 5 processing various functions based on the applications stored in the storage unit 4, and storing video content . It has a large-capacity storage unit 6 and an output unit 7 including a speaker for audio output and a monitor for video output.

The receiving unit 2 includes a receiving antenna for television broadcasting including satellite broadcasting, a modem for telecommunication lines such as the Internet line, a tuner, and the like, and can receive all data related to video content.

When the playback device 9 is a receiver such as a television (including a video deck, etc.), the operation unit 3 includes a main power supply, a channel selection operation, a volume increase/decrease switch, etc., which are operated by the user to view desired video content. A device (including a remote control device, etc.) for performing operations. When the playback device 9 is a personal computer or the like, the operation unit 3 is a mouse, keyboard, touch panel, or the like for the user to operate to view desired video content.

The storage unit 4 stores, for example, various applications executed by the control circuit unit 5 in order for the playback device 9 to operate. Further, the applications stored in the storage unit 4 include applications for functions to be realized by the present embodiment.

The control circuit unit 5 enables viewing of the received video content based on the application stored in the storage unit 4, and has component appearance frequency determination means 5a and video content extraction means 5b.

The large-capacity storage unit 6 stores the video content data received by the receiving unit 2 . In addition, the large-capacity storage unit 6 can accumulate the results of video content analysis performed by the control circuit unit 5 .

Specifically, video content distributed from a web server (cloud server) or a television broadcasting station or the like and received by the receiving unit 2 is accumulated in the large-capacity storage unit 6 .

Each component such as text, voice, video object, character or screen background that is the theme of the video content contained in the stored video content is determined according to the application stored in the storage unit 4. The appearance frequency of each component is determined by the determining means 5a.

That is, in order to sequentially calculate the frequency of appearance of the target component, for example, the number of appearances, the time of appearance, the appearance rate, etc., the control circuit unit 5 stores data in which the calculation result and each component correspond to each other. Stored in the capacity storage unit 16 .

The control circuit section 5 includes component appearance frequency determination means 5a and video content extraction means 5b.

The control circuit unit 5 determines whether or not the appearance frequency of each component that is sequentially calculated has reached a predetermined value or more, and if it is determined that the appearance frequency has reached a predetermined value or more, the component is included. The video content is extracted and displayed on the display screen of the output unit 7, which is content providing means.

The output unit 7 may be configured by a storage or a slot for storing the extracted video content in an external recording medium (eg, SD memory card, USB memory stick, etc.) not shown.

Further, when recording the extracted video content data in such an external recording medium, for example, backup video data in which the resolution of the video data in the video content stored in the large-capacity storage unit 6 is reduced is generated and stored in the storage unit 4. It may also be configured such that the backup video data is recorded after the video data is temporarily stored in the .

In such a configuration, the distributed video content is analyzed by the component appearance frequency determination means 5a for the component elements included in the video content, and compared with the video content stored in the past. A component's frequency of occurrence is determined. Here, video content is mainly composed of moving images, but still images are also included.

When the component appearance frequency determining means 5a determines that the appearance frequency of a predetermined component has reached a predetermined value in a predetermined period, the video content extracting means 5b accumulates a large number of video contents including the component. extracted from the video content of

After that, the video content extracted by the video content providing means 7 is provided to the viewer. The provision of video content in this case includes not only display of video content via an appropriate display, but also recording/distribution of video content and other modes of output.

In this case, AI (artificial intelligence) is used for the component appearance frequency determination means 5a and the video content extraction means 5b, and data processing is performed at high speed.

That is, the distributed video content related to moving images and still images is analyzed by the component appearance frequency determination means 5a for elements included in the video content, and is compared with the video content accumulated in the past to determine whether the video content is the same. is determined, and when it is determined that the appearance frequency of a predetermined component has reached a predetermined value in a predetermined period of time, the video content extracting means 5b accumulates a large number of video contents including the component Since the video content extracted from the video content and extracted by the video content providing means 7 is configured to be provided to the viewer, the viewer does not need to watch the video content delivered to the TV or the web all the time. Instead, it is possible to view video content that appears on TV or the web more than a predetermined frequency that is automatically provided.

As a result, video content that appears frequently is often important information or trendy information at that time, so viewers always check multiple TV channels and web information. There is no need, and only video content consisting of important information provided by the system according to the present invention and trendy information at that time can be viewed in a time efficient manner without being bound by time to a TV receiver, personal computer or smartphone. can be viewed.

Therefore, even if the viewer is in a busy environment, he or she does not need to constantly check the TV program on the TV receiver or frequently check the web information on the personal computer or smart phone. It is effective in being able to acquire information efficiently.

Further, in this embodiment, the control circuit section 5 is provided with a desired video registration means 5C. Therefore, the viewer, who is the user of the present system, registers the video that the user himself/herself desires to view, the classification, type, genre, etc. of the video in an appropriate designated range by the desired video registration means 5C via the operation unit 3. The constituent element appearance frequency determination means 5a judges the appearance frequency of the constituent elements based on the registered desired image.

In the present embodiment, regarding how the component appearance frequency determination means 5a recognizes and identifies specific video content with respect to a moving image, "a video object that is the theme of text, voice, and video content, This is done from the point of view of the character or the background of the image object.

Therefore, in the present embodiment, the components whose frequency of appearance is to be determined are text, voice, video objects that are themes of video content, characters, or backgrounds of video objects.

In this case, the "characters" are characters such as telops that appear in moving images, and are performed based on so-called character recognition technology, using morphological analysis technology.

Also, "sound" is a variety of sounds included in moving images and the like, and includes background sounds, theme songs, voices of characters, sound effects, and the like.

If necessary, only the speaking voice of a specific character on the theme of the video is extracted from the various voices, and the frequency of appearance is determined by the component appearance frequency determination means 5a.

In this case, when specifying only the speaking voice of a person who is the subject of a specific video, it is possible to extract only the speaking voice of the person by specifying the frequency band of the voice of the person.

In addition, "theme video object" is the subject of the video, and "character" includes a person who appears as a main character or a supporting character in relation to the theme of the video. . In this case, facial recognition technology or the like is used to identify characters. Also, the "background of an image object" refers to what appears as the background of a character or an object that is the subject of an image, such as a building, the sea, a mountain, the sky, a plain, and the like.

Thus, the component appearance frequency determination means 5a is configured to determine the frequency of appearance of the component based on the viewing desire registered by the viewer. It is possible to register the video, the classification, type, genre, etc. of the video in an appropriate designated range, and the component appearance frequency determination means can determine the appearance frequency based on the user's desire. The genre, type, etc. of the video content to be viewed are appropriately registered in the system, and it is possible to automatically and efficiently view the video content having a high frequency of appearance within the range of the genre and type.

In addition, for example, if a specific company that is a user wishes to collectively manage information related to the company that appears on TV or the web, register the company name, corporate identity, company name abbreviation, etc. By doing so, it is possible to automatically collect and view all TV video content or video information on the web related to the company, and use it for business management of the company from various points of view.

If the video content is distributed from a web server (or cloud server) via an electric communication line, the component appearance frequency determination means determines whether the same component is targeted for the video content including the distributed video etc. determine the filing frequency of Here, "video distributed via the web" refers to all types of videos uploaded to the web, videos on SNS (social network services), videos on services that provide videos via the Internet, etc. can be

In this way, the component appearance frequency determination means determines the appearance frequency of various videos uploaded on the web, all videos on SNS (social network service), and videos distributed over the Internet. It is possible to extract video content with high importance from all the video content circulating in the world through the web, and comprehensively and automatically provide video content with high importance or interest to viewers who are users. becomes.

Further, when the video content is distributed from a broadcasting station via an electric communication line, the constituent element appearance frequency determination means 5a applies the same configuration to all the video content displayed as TV video. Determine the filing frequency of elements.

Therefore, since the constituent element appearance frequency determination means 5a judges the filing frequency of the same constituent element for all video contents displayed as TV images, the important It is possible to extract video content with a high degree of importance or interest and comprehensively and automatically provide video content with a high degree of importance or interest to viewers who are users.

The control circuit unit 5 can also determine the appearance frequency of the constituent elements under various weighting conditions, which will be described later.
Further, in this embodiment, regarding the video information transmitted from the TV broadcasting station,
For example, in the case of a news program, the title of the news may be displayed in short text at a position such as the upper right of the screen. In such a case, when video recognition, voice recognition, person recognition, and background recognition are performed to determine the appearance frequency of the constituent elements, the title part can be referred to.
In this way, when video recognition, voice recognition, person recognition, and background recognition are performed by referring to the title part, more accurate recognition can be performed, and more efficient determination of the appearance frequency of constituent elements can be performed. It becomes possible to

Next, a second embodiment configured to create a summary based on video content and use the summary to determine the frequency of appearance of constituent elements will be described.
FIG. 2 is a block diagram showing the overall configuration of the video information providing system 11 for realizing the video information providing system according to the embodiment of the present invention. The basic configuration of the system as a whole is similar to that of the first embodiment, but the present embodiment differs in that a summary creation system 10 is provided.

That is, the video information providing system according to the second embodiment is based on the configuration of the video information providing system 1 according to the first embodiment, and furthermore, has a plurality of video contents extracted from each video signal. A summary creating means 10 for converting audio data or video data in a video content into text to create a summary; weighting means 15d for giving weights specific to one or more predetermined conditions included in the summary, and component appearance frequency determination means 15a determines the frequency of appearance of the components based on the summary. is configured to
Further, in this embodiment, the control circuit section 15 has a desired video registration means 15c, and the user can select the video the user desires to view and classify the video through the operation section 3 as appropriate. , type, genre, etc., can be registered by the desired video registration means 15C within an appropriate designated range, and the constituent element appearance frequency determination means 15a determines the frequency of appearance of the constituent elements based on the registered desired video. The video content is extracted by the content extraction means 15b and displayed on a display or the like by the output section 17 constituting the video content output means, as in the first embodiment.

In the present embodiment, the summary creation system 10 that constitutes the video information providing system 11 can be provided in the playback device 19, or the summary creation system 10 can be configured by a dedicated management server or the like. The video output system part that operates based on the summary created by the server can be realized by, for example, a TV, personal computer, smartphone, tablet terminal, etc. (hereinafter referred to as "playback device 19") having computer functions. It is possible. It should be noted that the playback device 19 can be used not only by one device but also by a plurality of devices.

In addition, in the following explanation, the case of television broadcasting will be mainly explained, and the unique cases of web video distribution will be explained as appropriate. Description of the form is omitted.

Television broadcasting includes terrestrial digital broadcasting, satellite broadcasting, one-segment broadcasting, Internet broadcasting, etc., and the form of broadcasting and reception is not limited.

As shown in FIGS. 2 and 3, the video information providing system 11 includes a receiving unit 12 having a tuner or the like for receiving a video signal related to video content from a TV station 30 or a web video distribution server 40, and An operation unit (including a remote controller, etc.) 13, a storage unit 14 storing applications for realizing various functions of the playback device 19, and a control circuit unit processing various functions based on the applications stored in the storage unit 14. 15, the summary creation system 10 described above, a large-capacity storage unit 16 for storing the created summary and various data for recording video content, and an output unit 17 including a speaker for audio output and a monitor for video output. and

The abstract creation system 10 is composed of a video signal separation unit 20, a video signal processing unit 18, a text integration unit 500, and a summary creation unit 600. The video signal processing unit 18, as shown in FIG. It is composed of a text conversion unit 100 , a telop text conversion unit 200 , a background image text conversion unit 300 and a logo mark text conversion unit 400 .

As shown in FIG. 2, the playback device 19 has a receiving section 12, an operating section 13, a storage section 14, a control circuit section 15, a summary processing system 11, a large capacity storage section 16 and an output section 17. FIG.
Further, the output unit 17 constitutes a video content providing unit having a function of outputting the result of weighting calculated by the control circuit unit 15 as a weighting unit to a monitor or a printer.
The control circuit section 15 is the same as in the first embodiment in that it has a component appearance frequency determination means 15a, a video content extraction means 15b, and a desired video registration means 15C. It further has weighting means 15d.

<Create a summary (video metadata)>
Here, as an example of producing video metadata as a summary, a case of processing television broadcast contents in Japanese and creating a database will be described. Also, in this case, the video content is exemplified for one program or segment.

"Immediateness" and "accuracy" are important in television programs, particularly in news and broadcast programs that are broadcast every moment.

On the other hand, in such news and broadcast programs in television broadcasting, some news content may be broadcast in other news programs etc. with different time zones (regardless of the difference in broadcasting stations). However, the same program will not be rebroadcast on different days of the week, and it can be said that it is disappearing information.

Information that has such "immediacy" and "accuracy" has continuity depending on conditions such as social importance and needs, or new information becoming clear depending on the news content. Therefore, it is desirable to weight the news according to the degree of importance/needs, such as the frequency of appearance reaching a predetermined value, for example. For the frequency of appearance, for example, the number of times of appearance, time of appearance, rate of appearance, etc. can be applied.

Here, since the importance and needs have short-term, long-term, and seasonal elements, for example, weekly, monthly, seasonal (seasonal), yearly, graph weighted by another statistic It is also possible to create At this time, the created graph can be output from the output unit 17 to a monitor or a printer.

As a result, it is possible to suppress the situation where the importance/needs are high in a short period of time, but the importance/needs are low throughout the year. can be given a weight.

Specifically, effective weighting can be applied to cases such as "forecast of cherry blossom blooming", "famous cherry blossom viewing", "Olympics", etc., where the degree of importance/need increases in a specific cycle.

Metadata for newly broadcast video information can be successively updated with a time lag of about 10 minutes, and can be updated to the degree of importance based on the latest information. At this time, it is possible to simultaneously receive programs from a plurality of broadcasting stations and update to the latest information.

In addition to basic information such as broadcast stations and broadcast times, metadata includes text information such as news titles, content abstracts, commentator names and conspicuous logos, as well as image authentication such as background images, caster face authentication, More detailed video metadata can be produced and distributed by voiceprint analysis and the like.

Furthermore, the results can be confirmed by the user through the web or by e-mail. Therefore, if the user saves and accumulates the video metadata in a large-capacity storage medium such as a hard disk, it can be used in various situations.

Specifically, from the daily news broadcast, it is possible to close up the specific commentator's words and actions, record the details, and later verify the contents.
If the specific commentator is a condition, it can be configured to always notify the user of the viewing when the specific commentator appears, and the news program currently being broadcast or the recorded In a news program, when the commentator is making a comment, it is also possible to notify the user of the news program by increasing the volume in a spot so as to prompt the user to view the program.
In the case of such a configuration, the user does not have to watch all the program images on the TV, and only needs to watch them when they are notified. It enables acquisition of information.

If the user is busy and does not have time to watch TV all the time, a condition corresponding to a predetermined image is set in the summary, and the image is displayed when the condition is met. It can also be configured to notify the user to the effect that the
In this case, when content that meets the notification conditions is included, the viewer, who is the user of the video content being output, may be notified by increasing the volume as described above or by uttering a message or the like. of the user's hearing.
Further, in addition to the user's auditory notification, for example, the user's visual notification such as repetition of brightness inversion of the display image 7a shown in FIG. In addition, both hearing and vision may be used. Furthermore, it can be used not only as a mere notification, but also as a trigger signal for starting other operations (for example, recording).

Further, for example, in a news program, if the user wants to know only the results of sports news, and if the user has registered that effect by the desired video registration means 15C, the user does not view the entire news program. For example, as shown in FIG. ), the user can be notified when the newscaster reads out an announcement manuscript containing "sports".

In addition, as a mode of utilization of video metadata, characters appearing in videos played in the program, for example, the comment time and tone analysis of the above-mentioned specific commentator, politicians introduced in the broadcast content (during the program) In addition to creating a database of video metadata such as broadcast times including videos of (political parties) and athletes, and clustering (a function that automatically classifies data without external criteria), It is also possible to apply weighting such as calculating the CM conversion value of an object.

As described above, the summary creation process (AI processing), which is a function of the summary creation system 10 that learns the optimal summary creation settings using the accumulated past summary creation result input data and output data as teaching materials, is used. When assigning such weighting, separate from AI processing, the CM conversion value is evaluated for each person by manual input by the operator based on the information not included in the video metadata such as the viewing rate or the video metadata such as newspapers and magazines. You may convert into a price (unit price per unit time).

Furthermore, the weighted commercial conversion values are used to produce video metadata for, for example, a single broadcasting station, a single program, multiple broadcasting stations (e.g., key stations in the Kanto area), etc. quarterly)/half year/full year/unit. The collected data can be output to a monitor or a printer from the output unit 7 in the form of graphs, lists (for example, for the top 100 people), or the like.

In addition, textual video metadata can be used as text broadcast during simulcast, for example, TV news/broadcast programs, wide shows, debate programs, political/economic programs, political/economic variety, etc. It is also possible to use as text information for the operator to create detailed information on the contents and article units of the information programs of state-run broadcasting stations and commercial key stations, which total more than 100 hours per day.

<Playback Device 19>
The playback device 19 has, as the receiving unit 2, a tuner function for television broadcasting (including terrestrial digital broadcasting, satellite broadcasting, and one-seg) or a receiving function for receiving Internet-delivered video, as shown in FIG. Furthermore, since the image can be output to the display screen 17a of the output unit 7, a television, personal computer, smart phone, tablet terminal, or the like can be used.

The receiving unit 12 receives TV radio waves from a TV station or communication radio waves transmitted via the Internet. Further, when a summary is created outside the reproducing device 19, it may be configured to have a function of receiving summary information supplied via communication means. The abstract received by the receiving unit 12 is stored or updated in the large-capacity storage unit 16 .

The operation unit 13 can use various switches and the like attached to the television, remote control devices attached to the television, mice and keyboards for computers, various switches and touch panels attached to smartphones and tablet terminals, and the like.

By the way, in the above-mentioned television broadcasting, when a certain incident occurs in the news, a plurality of television broadcasting stations repeatedly broadcast the same scene. In such a case, it is often not possible to easily recognize the whole picture even if one understands what, when and how each television media broadcasted.

Therefore, if such incidents are set as desired conditions, the contents of all designated news broadcast programs can be converted into text data in units of seconds and stored in a database to create a summary.

Then, by analyzing the results of classifying (clustering) the contents of the summary by the same theme, for example, if the user or the operator of the contracted dedicated company processes, what, when, which station, and how much will be broadcast It is also possible to obtain information that has been measured or quantified.

By setting such quantified information as desired conditions, it becomes possible to set more up-to-date and accurate conditions in subsequent news broadcasts. It is possible to make selections such as partial viewing of the program by notification and viewing of the entire program for other news broadcasts.

In this quantification, for example, it is possible to give weight to the output function such as automatically recording the video portion of the incident (for example, the video portion related to the progress of the accident processing at the nuclear power plant) in the large-capacity storage unit 16. .

In addition, as mentioned above, weighting is given to usage forms such as graphing how much broadcasts related to such incidents and accidents have been covered in television media, and which stations have dealt with which theme in terms of time and number of times. It is also possible to

Furthermore, such summaries are not limited to news broadcasts, and it is possible to analyze the contents of various entertainment programs from various angles.

As a result, for example, based on a comprehensively constructed entertainment database, entertainment programs such as dramas, movies, and variety shows can be analyzed from multiple perspectives, such as content, genre comparison, and time zone understanding. .

Furthermore, the appearance time of the specific performer can be converted, and the performer value can be easily calculated from, for example, the appearance rate on a daily/weekly/monthly basis.

Also, the voice of the performer described above can be weighted to convert the dialect into a standard language at the time of morphological analysis for text conversion after voice recognition.

The control circuit unit 15 appropriately (or sequentially) receives the summaries created by the summary creation system 10, stores them in the large-capacity storage unit 16, and determines the optimum conditions for weighting based on the summaries. While learning, weights specific to one or more predetermined conditions included in the summary can be assigned to multiple video contents.

In this way, the video information providing system 11 applies weights specific to at least one or more predetermined conditions to a plurality of video contents by the weighting means to the summary created by the summary creating means. Therefore, more useful information can be supplied to the user for a plurality of video contents.

In addition, it is possible to appropriately set the degree of importance (in stages, etc.) according to the frequency of appearance in a plurality of video contents. In addition, in daily broadcast news programs, etc., it can be said that the degree of social importance (or degree of interest) of major incidents and accidents is high. Therefore, by targeting a relatively short period of importance of such incidents, accidents, etc., it is possible to automatically set a predetermined condition.

In addition, if such incidents, accidents, etc., are recorded and stored in the large-capacity storage unit 6 separately from, for example, for preparing summaries, they can be edited by operators, etc., and can be left as recorded images.

The specific conditions mentioned above can include specific persons, objects, incidents, and accidents. Here, by targeting, for example, a specific person, company, various groups, etc., as a specific condition, the frequency of appearance of a specific person, company, or group from a plurality of video contents, such as the number of appearances, the appearance time, The appearance rate, etc. can be calculated.

Then, based on the appearance frequency, it is possible to calculate a CM conversion value, which is a measure of how much character value (for example, setting of a guarantee, etc.) exists.

In addition, when targeting a specific corporate name, analyzes such as the ratio of commercials provided to the entire commercial, the proportion of various products sold by the corporation (degree of emphasis and identification of new products), and the correlation with stock price fluctuations It can also be used for (weighting) and the like.

Appearance frequencies can be aggregated on a daily/weekly/monthly/yearly basis, and can also be aggregated separately for each TV broadcasting station and for each time period.

Also, the frequency of appearance can include, for example, a case where the number of appearances in a specific program is once per program, and a case where the number of appearances within a program is targeted, as weighting conditions.

Here, the number of appearances in the program may include the number of appearances in the video or the appearance time, and the number of times the name is called out by the moderator, etc., for example. can.

In addition, when calling by name, in the case of the full name "○×△□", in the case of only the name such as "○×-san", in the case of the name or name such as "△□-chan" or "△-chan" can be a condition.

Furthermore, if the performer belongs to a group, it is possible to include (as weighting) the case of addressing by group name and the case of addressing by name. Similarly, for example, even if a junior player is called "○○ second generation" because he has achieved good results in a field such as sports in the past, both the predecessor and the successor are used as weighting conditions. be able to.

For example, in the case of calling by the name of “○×”, although it is rare that a person with the same name is included in the same program, there is a high possibility that there are multiple people in the industry as a whole. Therefore, in such a case, only persons appearing in the program can be included as weighting conditions, excluding persons not appearing in the program.

Conversely, for example, if the name or image of another person belonging to the group to which the performer belongs, even though the person did not appear on the program, that person will not appear on the program. It can also be treated as if it exists.

As described above, in the present embodiment, a summary creation means 600 for creating a summary by converting audio data or video data in a plurality of video contents extracted from each video signal of the plurality of video contents into text, and a summary creation means a weighting means 15d that, while learning the optimum conditions based on the accumulation results of the summaries created in step 600, gives weights specific to one or more predetermined conditions included in the summaries to a plurality of video contents; and the component appearance frequency determination means 15a is configured to determine the appearance frequency of the component based on the summary.

The component appearance frequency determination means 15a determines the frequency of appearance of the video content, that is, the degree of importance, based on the summary of the video content.

That is, the summarization means 600 determines the appearance frequency of the components of the video content based on the text summary of the audio data or video data of the plurality of video contents extracted from each video signal of the plurality of video contents. be.

In this case, the summary created by the summary creating means 600 is obtained by assigning weights specific to at least one or more predetermined conditions to the plurality of video contents by the weighting means 15d. To provide users with more useful information of high importance or interest.

Therefore, in the video information system 11 according to the second embodiment, the component appearance frequency determination means 15a is configured to determine the frequency of appearance of the video content, that is, the degree of importance, based on the summary of the video content. Therefore, compared to the case where various advanced technologies such as face recognition technology, voice recognition technology, and morphological recognition technology are used to narrow down the subject of each video directly from the video content, it is possible to identify the constituent elements more quickly and accurately. It becomes possible to judge the appearance frequency.

Further, the weighting unit 15d refers to overlapping text from audio data or video data included in a plurality of video contents, and if the reference result is equal to or greater than a predetermined value, the text meets a predetermined condition. The text is judged to be important text that has been read and weighted.

For example, in daily broadcast news programs, the web, SNS, and the like, it can be said that topics with great social news value, information on incidents and accidents, etc., are of high social importance and interest. As a result, information about such social events often appears as multiple video contents in multiple media.

Therefore, in multiple video contents, overlapping texts are judged to be the same constituent element by morphological analysis of the text, etc., and according to the frequency of appearance, for example, the number of appearances, the time of appearance, the appearance rate, etc. The importance is appropriately set.

In this way, information related to social events often appears as multiple video contents in multiple media. Therefore, in multiple video contents, it is possible to judge overlapping text as the same component by morphological analysis of the text. In addition, since it is configured to appropriately set the degree of importance, such as in stages, according to the appearance frequency, for example, the number of appearances, the appearance time, the appearance rate, etc., the appearance frequency of the constituent elements can be accurately set. That is, the degree of social importance is determined, and appropriate video content is provided to the user.

Also, the weighting unit 15d determines whether or not a plurality of video contents within a preset period are important texts.

The period required for judging the social importance of information such as topics, incidents or accidents is judged within a predetermined period. Therefore, since the period required for determining the degree of social importance can be determined within a predetermined period, it is possible to quickly provide socially important video content to users.

In this way, by recording and storing socially important incidents, accidents, etc. in a large-capacity storage unit separately from, for example, summaries, they can be edited and recorded by operators, etc. It is possible to provide a video information providing system capable of appropriately reusing the collected video content important to the user for analysis or the like.

Further, when the new video content is determined to be video content including important text, the weighting unit 15d starts recording the video content.

In news programs that are broadcast daily, it can be said that major incidents and accidents are of high social importance and interest. Therefore, if such incidents, accidents, etc., are recorded and stored in a large-capacity storage section separately from those used for summarization, for example, they can be edited by operators, etc., and left as recorded images.

Further, the weighting unit 15d calculates a CM conversion value from the frequency of appearance of a specific person or object from texts included in at least one of audio data and video data of a plurality of video contents.

The specific conditions mentioned above can include specific persons, objects, incidents, and accidents. Here, for example, by targeting a specific person (including corporate name, various organizations, etc.) as a specific condition, the frequency of appearance of a specific person from a plurality of video contents, such as the number of appearances, the appearance time, The appearance rate, etc. can be calculated.

Then, based on the number of appearances, appearance time, appearance rate, etc., it is possible to calculate a CM conversion value that serves as a guideline of how much a person's value (for example, setting a guarantee) is in the world of mass media and information. . In addition, when targeting a specific corporate name, analyzes such as the ratio of commercials provided to the entire commercial, the proportion of various products sold by the corporation (degree of emphasis and identification of new products), and the correlation with stock price fluctuations can also be used for

Therefore, it is possible to provide a video information calculation system that can use the CM conversion value calculated by the weighting means 15d as an analytical index for various economic activities.

Also, the weighting means 15d targets a specific corporate name from texts included in at least one of audio data and video data of a plurality of video contents.

Therefore, since a specific corporate name is used as a component for determining the appearance frequency, video content in which the specific corporate name appears is extracted and provided to the corporation, which is a viewer as a user.

Therefore, since a specific corporate name is a component that is the target of appearance frequency determination, video content in which a specific corporate name appears is extracted and provided to the corporation, who is a viewer as a user, For example, it is possible to quickly, appropriately, and comprehensively collect information about the company's social evaluation, reputation, etc., and promptly reflect it in the management of the company.

The summary creation system 10 according to the second embodiment will be described below.
<Overall Configuration of Abstract Creation System 10>
As shown in FIG. 3, the summary creation system 10 includes a video signal separation unit 20, a speech text conversion unit 100, a telop text conversion unit 200, a background image text conversion unit 300, a logo mark text conversion unit 400, a text integration unit 500, and a summary creation unit 600 . In this embodiment, summarizing system 10 obtains the video signal from a television broadcast from television station 30 . Note that the video signal can be obtained from images on the Internet.

A video signal V including an audio signal and a video signal is separated into an audio signal A and a video signal B by a video signal separating section 20 . The audio signal A is input to the speech text conversion unit 100 , and the video signal B is input to the telop text conversion unit 200 , the background image text conversion unit 300 , and the logo mark text conversion unit 400 .

<Speech text conversion unit 100>
As shown in FIG. 4, the speech-to-text conversion unit 100 receives the audio signal A and outputs speech text, which is a text describing the content of a person's speech in the video content. The speech text conversion unit 100 includes a speech information extraction unit 110 , a speech content recognition unit 120 , and a speech content text conversion unit 130 .

The speech information extraction unit 110 extracts speech information from the audio signal A of the video signal V. FIG. Namely, the noise in the audio signal A is removed and the information of the human speech is extracted. This speech information can include sound effects and characteristic music.

The utterance content recognition unit 120 recognizes the utterance content from the utterance information. That is, the utterance information is acoustically and grammatically analyzed to recognize the contents of the utterance as language. The parameters, conditions, and the like used for this recognition can be generated by machine learning from accumulated past speech text generation data, as will be described later.

The utterance content text conversion unit 130 converts the utterance content into text and outputs it. The parameters, conditions, and the like used for this recognition can be generated by machine learning from input data and generated data of past voice texts that have been accumulated as will be described later.

<telop text conversion unit 200>
As shown in FIG. 5, the telop text converting unit 200 receives the video signal B and outputs telop text, which is text describing the content of the telop in the video content. The telop text conversion unit 200 includes a telop information extraction unit 210 , a telop content recognition unit 220 , and a telop content text conversion unit 230 .

The telop information extraction unit 210 extracts telop information from the video signal B of the video signal V. FIG. That is, the background in the video signal B is removed and the information of only the telop image is extracted.

The utterance content recognition unit 120 recognizes the telop content from the telop image information. That is, the telop information is linguistically and grammatically analyzed to recognize the telop display content as a language. The parameters, conditions, and the like used for this recognition can be generated by machine learning from accumulated past telop text input data and generated data, as will be described later.

The telop content text conversion unit 230 converts the telop content into text and outputs it. The parameters, conditions, and the like used for this recognition can be generated by machine learning from accumulated past telop text input data and generated data, as will be described later.

<Background image text conversion unit 300>
As shown in FIG. 6, the background image text conversion unit 300 receives the video signal B and outputs background image text, which is text describing the content of the background image in the video content. Background images can include people, belongings of people, facial expressions of people, landscapes, building conditions, indoor conditions, animals, vehicles, and other items. The background image text conversion unit 300 includes a background image information extraction unit 310 , a background image content recognition unit 320 , and a background image content text conversion unit 330 .

Background image information extraction section 310 extracts background image information from video signal B of video signal V. FIG. That is, telops and unclear images in the video signal B are removed, and information of only the recognizable background image is extracted.

The background image content recognition unit 320 recognizes the content of the background image from the background image information. In other words, the person, belongings of the person, expression of the person, scenery, building conditions, indoor conditions, animals, vehicles, and other items represented by analyzing the background image information are recognized. The parameters, conditions, and the like used for this recognition can be generated by machine learning from past background image text input data and generated data that have been accumulated as will be described later.

The background image content text conversion unit 330 converts the background image content into text and outputs it. The parameters, conditions, and the like used for this recognition can be generated by machine learning from past background image text input data and generated data that have been accumulated as will be described later.

<Logo Mark Text Conversion Unit 400>
As shown in FIG. 7, the logo mark text converting unit 400 receives the video signal B and outputs logo mark text, which is text describing the content of the logo mark in the video content. Logomarks include trademarks and other marks that indicate the origin of goods.

The logo mark text conversion unit 400 includes a logo mark image information extraction unit 410 , a logo mark content recognition unit 420 , and a logo mark content text conversion unit 430 .

The logo mark image information extraction unit 410 extracts logo mark image information from the video signal B of the video signal V. FIG. That is, telops and background images in the video signal B are removed, and information of only the recognizable logo mark image is extracted.

The logo mark content recognition unit 420 recognizes the content of the logo mark from the logo mark image information. That is, the product, service, store, facility, etc. represented by analyzing the logo mark image information are recognized. The parameters, conditions, and the like used for this recognition can be generated by machine learning from past logo mark text input data and generated data accumulated as will be described later.

The logo mark content text conversion unit 430 converts the content of the logo mark image into text and outputs the text. The parameters, conditions, and the like used for this recognition can be generated by machine learning from past logo mark text input data and generated data accumulated as will be described later.

<Text Integration Unit 500>
As shown in FIG. 8, the text integration unit 500 includes the speech text from the speech text conversion unit 100, the telop text from the telop text conversion unit 200, the background text from the background image text conversion unit 300, and the logo mark text conversion unit 400. Integrate background text from . That is, correct contradictions and errors in each text to generate an integrated text. The parameters, conditions, etc. used for this text integration can be generated by machine learning from the input and output data of past text integration, which has been accumulated as described later.

<Summary Creation Unit 600>
As shown in FIG. 9, the summarizer 600 summarizes the integrated text from the text integrator 500 . That is, the contents of the summary text are summarized to the specified number of characters. The parameters, conditions, etc. used for this summarization can be generated by machine learning from the input data and output data of past winning combination processing accumulated as will be described later.

Next, machine learning processing of each unit will be described.
<Machine learning processing of the speech-to-text unit 100>
FIG. 3 is a block diagram showing an utterance-to-text section of the summary creation system. The utterance text conversion unit 100 includes an utterance information extraction unit 110 , an utterance content recognition unit 120 , an utterance content text conversion unit 130 , a machine learning unit 140 , a content recognition text creation setting unit 150 , and a comparison evaluation unit 160 . An existing data storage unit 700 is also connected to the speech text conversion unit 100 .

The speech text conversion unit 100 performs machine learning based on the existing video data stored in the existing data storage unit 700 and the existing speech text, and optimizes the speech content recognition unit 120 and the speech content text conversion unit 130 . An existing data storage unit 700 stores an existing video data storage unit 710 storing a large number of video data used when a person created an utterance text in the past, and an utterance text created from the utterance contents of this video data. An existing speech text storage unit 720 is provided. These video data and spoken texts serve as teaching materials for machine learning.

The speech-to-text unit 100 also includes switching units 170 and 180 for switching data output when performing machine learning and when creating speech content text from new video data.

The content recognition text creation setting section 150 stores the settings of the speech content recognition processing of the speech content recognition section 120 and the settings of the text conversion processing of the speech content text conversion section 130 . The speech content recognition unit 120 and the speech content text conversion unit 130 recognize and text the speech content according to the conditions and parameters set by the content recognition text creation setting unit 150 .

The comparison evaluation section 160 includes a comparison section 161 and an evaluation section 162 . The comparison unit 161 compares the speech text created by the speech content text conversion unit 130 upon receiving the existing video data from the existing video data storage unit 710 and the existing speech text from the existing speech text storage unit 720 . The evaluation unit 162 evaluates based on the comparison result of the comparison unit 161, and gives a high score when there is a good match.

The machine learning unit 140 receives the evaluation from the evaluation unit 162 and changes the setting state of the content recognition text creation setting unit 150 . This processing is repeated for the same video data, and the evaluation value of the evaluation unit 162 is made as high as possible. This process can be repeated for multiple pieces of video data.

By performing such machine learning, the capabilities of the utterance content recognition unit 120 and the utterance content text conversion unit 130 are improved. After completing the predetermined machine learning, the speech-to-text conversion unit 100 processes the new video data and is ready to output the optimum speech text.

<Machine learning of telop text conversion unit 200>
FIG. 4 is a block diagram showing a telop-text conversion unit of the summary creation system. The telop text conversion unit 200 includes a telop information extraction unit 210, a telop content recognition unit 220, a telop content text conversion unit 230, a machine learning unit 240, a content recognition text creation setting unit 250, and a comparison evaluation unit 260. An existing data storage unit 700 is also connected to the telop text conversion unit 200 .

The telop text conversion unit 200 performs machine learning based on existing video data and existing telop text stored in the existing data storage unit 700 to optimize the telop content recognition unit 220 and the telop content text conversion unit 230 . An existing data storage unit 700 stores an existing video data storage unit 710 storing a large number of video data used when a person created telop text in the past, and telop text created from the utterance content of this video data. An existing telop text storage unit 730 is provided. These video data and spoken texts serve as teaching materials for machine learning.

Further, the telop text conversion unit 200 includes switching units 270 and 280 for switching data output when performing machine learning and when creating speech content text from new video data.

The content recognition text creation setting section 250 stores the settings of the text content recognition processing of the telop content recognition section 220 and the settings of the text conversion processing of the telop content text conversion section 230 . The telop content recognition unit 220 and the telop content text conversion unit 230 recognize the content of the telop and convert it into text according to the conditions and parameters set by the content recognition text creation setting unit 250 .

The comparison evaluation section 260 includes a comparison section 261 and an evaluation section 262 . The comparison unit 261 compares the telop text created by the telop content text conversion unit 230 upon receiving the existing video data from the existing video data storage unit 710 and the existing telop text from the existing telop text storage unit 730 . The evaluation unit 262 evaluates based on the comparison result of the comparison unit 261, and gives a high score when there is a good match.

The machine learning section 240 receives the evaluation from the evaluation section 262 and changes the setting state of the content recognition text creation setting section 250 . This processing is repeated for the same video data, and the evaluation value of the evaluation unit 262 is made as high as possible. This process can be repeated for multiple pieces of video data.

By performing such machine learning, the capabilities of the telop content recognition unit 220 and the telop content text conversion unit 230 are improved. After completing the predetermined machine learning, the telop text conversion unit 200 processes the new video data and is ready to output the optimum telop text.

<Machine learning of the background image text conversion unit 300>
FIG. 6 is a block diagram showing the background image text conversion section of the summary creation system. The background image text conversion unit 300 includes a background image information extraction unit 310, a background image content recognition unit 320, a background image content text conversion unit 330, a machine learning unit 340, a content recognition text creation setting unit 350, and a comparison evaluation unit 360. Prepare. An existing data storage unit 700 is connected to the background image text converting unit 300 .

The background image text conversion unit 300 performs machine learning based on the existing video data stored in the existing data storage unit 700 and the existing background image text, and optimizes the background image content recognition unit 320 and the background image content text conversion unit 330. do. The existing data storage unit 700 stores an existing video data storage unit 710 storing a large number of video data used when a person created telop text in the past, and a background image text created from the utterance content of this video data. An existing background image text storage unit 740 is provided. These video data and background image text serve as teaching materials for machine learning.

The background image text conversion unit 300 also includes switching units 370 and 380 for switching data output when machine learning is performed and when speech content text is created from new video data.

The content recognition text creation setting section 350 stores the settings of the background image content recognition processing of the background image content recognition section 320 and the settings of the text conversion processing of the background image content text conversion section 330 . The background image content recognition unit 320 and the background image content text conversion unit 330 perform content recognition and text conversion of the background image according to the conditions and parameters set by the content recognition text creation setting unit 350 .

The comparison evaluation section 360 includes a comparison section 361 and an evaluation section 362 . The comparison unit 361 compares the background image text created by the background image content text conversion unit 330 upon receiving the existing video data from the existing video data storage unit 710 and the existing background image text from the existing background image text storage unit 740. do. The evaluation unit 362 performs evaluation based on the comparison result of the comparison unit 361, and gives a high score when there is a good match.

The machine learning section 340 receives the evaluation from the evaluation section 362 and changes the setting state of the content recognition text creation setting section 350 . This processing is repeated for the same video data, and the evaluation value of the evaluation unit 362 is made as high as possible. This process can be repeated for multiple pieces of video data.

By performing such machine learning, the capabilities of the background image content recognition unit 320 and the background image content text conversion unit 330 are improved. After completing the predetermined machine learning, the background image text conversion unit 300 is ready to process the new video data and output the optimum background image text.

<Machine learning of the logo mark text conversion unit 400>
FIG. 7 is a block diagram showing a logo mark text conversion unit of the same summary creation system. The logo mark text conversion unit 400 includes a logo mark image information extraction unit 410 , a logo mark content recognition unit 420 , a logo mark content text conversion unit 430 , a machine learning unit 440 , a content recognition text creation setting unit 450 , a comparison evaluation unit 460 . Prepare. An existing data storage unit 700 is connected to the logo mark text conversion unit 400 .

The logo mark text conversion unit 400 performs machine learning based on the existing video data stored in the existing data storage unit 700 and the existing logo mark text, and optimizes the logo mark content recognition unit 420 and the logo mark content text conversion unit 430. become The existing data storage unit 700 stores an existing video data storage unit 710 storing a large number of video data used when people created logo mark texts in the past, and a logo mark text created from the utterance content of this video data. A stored existing logo mark text storage unit 750 is provided. These video data and logo mark text serve as teaching materials for machine learning.

Further, the logo mark text conversion unit 400 is provided with switching units 470 and 480 for switching data output when performing machine learning and when creating speech content text from new video data.

The content recognition text creation setting section 450 stores the settings of the logo mark image content recognition processing of the logo mark content recognition section 420 and the settings of the text conversion processing of the logo mark content text conversion section 430 . The logo mark content recognition section 420 and the logo mark content text conversion section 430 perform the content recognition and text conversion of the logo mark according to the conditions and parameters set by the content recognition text creation setting section 450 .

The comparison evaluation section 460 includes a comparison section 461 and an evaluation section 462 . The comparison unit 461 compares the text created by the logo mark content text conversion unit 430 upon receiving the existing video data from the existing video data storage unit 710 and the existing background image text from the existing logo mark text storage unit 750 . The evaluation unit 462 evaluates based on the comparison result of the comparison unit 461, and gives a high score when there is a good match.

The machine learning section 440 receives the evaluation from the evaluation section 462 and changes the setting state of the content recognition text creation setting section 450 . This processing is repeated for the same video data, and the evaluation value of the evaluation unit 462 is made as high as possible. This process can be repeated for multiple pieces of video data.

By performing such machine learning, the capabilities of the logo mark content recognition unit 420 and the logo mark content text conversion unit 430 are improved. After completing the predetermined machine learning, the logo mark text conversion unit 400 is ready to process the new video data and output the optimum background image text.

<Machine Learning of Text Integration Unit 500>
FIG. 8 is a block diagram showing the text integration section of the summary creation system. The text integration unit 500 includes an integrated text creation unit 510 , an integrated text creation setting unit 520 , a machine learning unit 530 and a comparison evaluation unit 540 . An existing data storage unit 700 is connected to the text integration unit 500 .

The text integration unit 500 performs machine learning based on existing integrated texts stored in the existing data storage unit 700, namely, speech text, telop text, background text, logo mark text, and existing integrated texts. Optimize the behavior of The existing data storage unit 700 includes an existing text storage unit 760 that stores various text data used when integrated texts were created in the past, and an existing integrated text storage unit 770 that stores integrated texts created from these various texts. and These various texts and integrated texts serve as teaching materials for machine learning.

The text integration unit 500 also includes switching units 570 and 580 for switching data output when performing machine learning and when creating a new integrated text from various new texts.

The integrated text creation setting section 520 stores settings for text integration processing of the integrated text creation section 510 . The integrated text creation section 510 performs text integration processing according to the conditions and parameters set by the integrated text creation setting section 520 .

The comparison evaluation section 540 includes a comparison section 541 and an evaluation section 542 . The comparison unit 541 compares the integrated text created by the integrated text creation unit 510 upon receipt of the various existing texts from the existing various text storage unit 760 and the existing integrated text from the existing integrated text storage unit 770 . The evaluation unit 542 performs evaluation based on the comparison result of the comparison unit 541, and gives a high score when there is a good match.

The machine learning section 530 receives the evaluation from the evaluation section 542 and changes the setting state of the integrated text creation setting section 520 . This processing is repeated for the same text data, and the evaluation value of the evaluation unit 542 is made as high as possible. This process can be repeated for a plurality of various text data.

By performing such machine learning, the ability of the integrated text creation unit 510 is improved. After completing the predetermined machine learning, the text integrator 500 is ready to process the new video data and output the optimal integrated text.

<Machine learning of summary creation unit 600>
FIG. 9 is a block diagram showing the abstract creating section of the same abstract creating system. The summary creation section 600 includes a summary text creation section 610 , a summary creation setting section 620 , a machine learning section 630 and a comparison evaluation section 640 . An existing data storage unit 700 is connected to the summary creation unit 600 .

The summary creation section 600 performs machine learning based on the integrated text and the summary text stored in the existing data storage section 700 to optimize the operation of the summary text creation section 610 . The existing data storage unit 700 includes an existing integrated text storage unit 770 that stores integrated text data used when summarizing texts were created in the past, and an existing integrated text storage unit 780 that stores a summary text created from this integrated text. and These integrated texts and summary texts serve as teaching materials for machine learning.

The summary creation unit 600 also includes switching units 670 and 680 for switching data output when performing machine learning and when creating a new summary text from a new integrated text.

The summary creation setting section 620 stores the settings of the summary processing of the summary text creation section 610 . The summary text creation section 610 performs text summary processing according to the conditions and parameters set by the summary creation setting section 620 .

The comparison evaluation section 640 includes a comparison section 641 and an evaluation section 642 . Comparing section 641 compares the summary text created by summary text creation section 610 upon receiving the existing integrated text from existing integrated text storage section 770 and the summary text from existing summary text storage section 780 . The evaluation unit 642 evaluates based on the comparison result of the comparison unit 641, and gives a high score when there is a good match.

The machine learning section 630 receives the evaluation from the evaluation section 642 and changes the setting state of the summary creation setting section 620 . This processing is repeated for the same text data, and the evaluation value of the evaluation unit 642 is made as high as possible. This process can be repeated for a plurality of integrated text data.

By performing such machine learning, the ability of the summary text creation unit 610 is improved. After completing the predetermined machine learning, the digester 600 is ready to process the new video data and output the optimal text summary.

Next, processing of the abstract creation system 10 will be described. FIG. 10 is a flow chart showing the operation of the abstract creation system.

First, an existing video data storage unit 710, an existing speech text storage unit 720, an existing telop text storage unit 730, an existing background image text storage unit 740, an existing logo mark text storage unit 750, and various existing text storage units of the existing data storage unit 700. 760, the existing integrated text storage section 770, and the existing summary text storage section 780, the existing video signal and various text data are read (step ST1).

Next, machine learning processing is performed in the speech text conversion unit 100, the telop text conversion unit 200, the background image text conversion unit 300, and the logo mark text conversion unit 400 (steps ST2a, ST2b, ST2c, ST2d). This learning process can also be performed sequentially.

Next, existing input data and output data are read into the existing data storage unit 550 of the text integration unit 500 and the existing data storage unit 650 of the summary creation unit 600 (step ST3). Next, machine learning processing is performed in the text integration section 500 and the summary creation section 600 (steps ST3a and ST3b). This learning process can also be performed sequentially. What is the machine learning processing of the utterance text conversion unit 100, the telop text conversion unit 200, the background image text conversion unit 300, and the logo mark text conversion unit 400, and the machine learning processing of the text integration unit 500 and the summary creation unit 600? Regardless of the order of processing, it can be performed in the reverse order.

When the learning process is completed (yes in step ST4), the video signal to be processed is input to the video signal separating section 20 (step ST5). As a result, the speech text conversion unit 100, the telop text conversion unit 200, the background image text conversion unit 300, and the logo mark text conversion unit 400 execute text conversion processing (steps ST6a, ST6b, ST6c, ST6d).

Then, each text is integrated by the text integration unit 500 (step ST7), the integrated text is summarized by the summary generation unit 600 (step ST8), and the summary text is output. finish.

From the next digest creation process, the machine learning process (steps ST1 to ST4) is not performed, and the optimum digest can be created only by immediately inputting the video signal to be summarized (step ST5). Also, machine learning processing can be performed as needed.

The above system includes a CPU (Central Processing Unit) as a processing device, RAM (Random Access Memory), ROM (Read Only Memory), HDD (Hard Disc Drive), SSD (Solid State Drive), etc. as storage devices. It can be implemented by running application software on a computer system. Moreover, each part does not have to be arranged at the same place, and can be realized by arranging a part on a web and connecting with a network. In addition, since these processes target a large amount of data, it is preferable to process them using a GPU (Graphics Processing Unit).

That is, the integrated text is simply textualization of voices, characters, background images, etc., and is data about a huge amount of sentences. Therefore, high-speed processing can be achieved by specializing the GPU to text processing.

Text input by the text integration unit 500 is not limited to speech text, telop text, background image text, and logo mark text.

For example, when targeting television programs (digital terrestrial television broadcast programs), characters and voices obtained from an electronic program guide (EPG), closed-caption broadcasts, and commentary broadcasts can be acquired as text and input. As a result, the quality and quantity of the integrated text can be improved, and the versatility and taste of the text can be improved.

Similarly, in the case of Internet video distribution, evaluations (including comments) and reputations of third parties can be acquired as text and input. As a result, the quality and quantity of the integrated text can be improved, and the versatility and taste of the text can be improved.

For "weighting", it is also possible to use the number of views (number of playbacks) and third party evaluation (good/bad), and use the evaluation ratio to the number of views, or the ratio of good/bad. is.

As described above, in the video information providing system 11 according to the second embodiment, the summary created by the summary creating means 600 is weighted by the weighting means 15d for a plurality of video contents, and at least one or more By assigning weights specific to the predetermined conditions of (1), more useful information can be supplied to the user for a plurality of video contents.

In the second embodiment, data processing is performed at high speed and appropriately by AI (artificial intelligence) processing and summarized. AI processing can be realized by machine learning (ML) described above. Furthermore, as machine learning, supervised learning can be employed in which existing data is the correct answer. Moreover, it is effective to use deep learning (DL: Deep Learning) as machine learning.

In deep learning, learning is performed using a large number of existing video data, various text data corresponding to each video data, integrated text, and summary text as big data. Each machine learning unit has an input layer, multiple intermediate layers, and an output layer, and performs processing using a neural network with a large number of neurons. That is, the new video data input to the summary creation system according to the present invention, various texts, integrated texts, and abstracts based on this video data are input, and the intermediate layer is arranged so that the output approaches the existing various texts, integrated texts, and abstracts. The weights and parameters in the neurons of are optimized by methods such as the method of least squares.

The scope of rights of the present invention is not limited to the configurations described in the first and second embodiments, but extends to all configurations included within the scope of the present invention.

1: Video information providing system 2: Reception unit 3: Operation unit 4: Storage unit 5: Control circuit unit 5a: Component appearance frequency determination means 5b: Video content extraction means 5C: Desired video registration means 6: Mass storage unit 7 : Output unit (video content providing means, output means)
7a: Display video object 9: Playback device 10: Summary creation system (summary creation means)
11: Video information providing system 12: Reception unit 13: Operation unit 14: Storage unit 15: Control circuit unit 15a: Constituent element appearance frequency determination means 15b: Video content extraction means 15c: Desired video registration means 15d: Weighting means 16: Mass storage unit 17: output unit (output means)
17a: Display screen 18: Video signal processing unit 19: Playback device 20: Video signal separation unit 30: TV station 40: Internet 100: Speech text conversion unit 110: Speech information extraction unit 120: Speech content recognition unit 130: Speech content text conversion Unit 140: Machine learning unit 150: Content recognition text creation setting unit 160: Comparative evaluation unit 161: Comparison unit 162: Evaluation unit 170: Switching unit 180: Switching unit 200: Telop text conversion unit 210: Telop information extraction unit 220: Telop Content recognition unit 230: Telop content text conversion unit 240: Machine learning unit 250: Content recognition text creation setting unit 260: Comparative evaluation unit 261: Comparison unit 262: Evaluation unit 270: Switching unit 280: Switching unit 300: Background image text conversion Section 310: Background image information extraction section 320: Background image content recognition section 330: Background image content text conversion section 340: Machine learning section 350: Content recognition text creation setting section 360: Comparative evaluation section 361: Comparison section 362: Evaluation section 370 : Switching section 380: Switching section 400: Logo mark text conversion section 410: Logo mark image information extraction section 420: Logo mark content recognition section 430: Logo mark content text conversion section 440: Machine learning section 450: Content recognition text creation setting section 460: Comparative evaluation unit 461: Comparison unit 462: Evaluation unit 470: Switching unit 480: Switching unit 500: Text integration unit 510: Integrated text creation unit 520: Integrated text creation setting unit 530: Machine learning unit 540: Comparative evaluation unit 541 : Comparison unit 542: Evaluation unit 550: Existing data storage unit 570: Switching unit 580: Switching unit 600: Summary creation unit (summary creation means)
610: Summary text creation unit 620: Summary creation setting unit 630: Machine learning unit 640: Comparative evaluation unit 641: Comparison unit 642: Evaluation unit 650: Existing data storage unit 670: Switching unit 680: Switching unit 700: Existing data storage unit 710: Existing video data storage unit 720: Existing speech text storage unit 730: Existing telop text storage unit 740: Existing background image text storage unit 750: Existing logo mark text storage unit 760: Existing various text storage unit 770: Existing integrated text storage Section 780: Existing summary text storage section

Claims (12)

receiving means for receiving the distributed video content;
a storage means for storing and accumulating the video content received by the receiving means;
A component that analyzes the components of the distributed video content, compares the analyzed components with a large number of past video contents accumulated in the storage means, and determines the appearance frequency of the same component. appearance frequency determination means;
Video content extraction for extracting video content containing the component when the appearance frequency is equal to or greater than a predetermined value based on the determination by the component appearance frequency determination means , from a large number of past video contents accumulated in the storage means. means and
a video content providing means for providing a viewer with the video extracted by the video content extraction means;
The component appearance frequency determination means analyzes components included in the distributed video content, and compares the analyzed component with a large number of past video contents accumulated in the storage means to determine if they are the same. determine the frequency of occurrence of the constituent elements of
When it is determined that the appearance frequency of the predetermined component has reached a predetermined value in a predetermined period of time, the video content extracting means extracts the video content including the component from a large number of video contents stored in the storage means. and providing the video content extracted by the video content providing means to a viewer . 2. The video information providing system according to claim 1, wherein said component appearance frequency determination means determines the appearance frequency of said component based on a viewing request registered by a viewer. 2. A video information providing system according to claim 1, wherein said constituent elements are characters, sounds, video objects that are themes of said video content, characters, or backgrounds of video objects. 4. The structural element appearance frequency determining means determines the frequency of appearance of the structural elements by referring to the title portion displayed by characters in the video screen. or the image information providing system according to item 1. 5. The video information providing system according to any one of claims 1, 2, 3, and 4, wherein said video content is distributed from a server via an electric communication line. 5. The video information providing system according to any one of claims 1, 2, 3, and 4, wherein said video content is distributed from a broadcasting station via an electric communication line. a summary creation means for creating a summary by converting audio data or video data in the video content extracted from each video signal of the video content into text;
Appropriate weighting is applied to one or more predetermined conditions included in the summary to the plurality of video contents while learning the optimum condition based on the accumulation result of the summary created by the summary creation means. 2. A video information providing system according to claim 1, further comprising weighting means, and wherein the element appearance frequency determination means determines the appearance frequency of the elements based on the summary. The weighting means refers to overlapping text from audio data or video data included in the plurality of video contents, and if the reference result is equal to or greater than a predetermined value, the text meets the predetermined condition. 8. The video information providing system according to claim 7, wherein the image information providing system determines that the text is important text that has been read and weighted. 9. A video information providing system according to claim 8, wherein said weighting means determines whether or not a plurality of video contents within a preset period are important texts. 10. The weight assigning means starts recording of new video content when it is determined that the new video content includes the important text. The video information providing system according to any one of 1. The weighting means calculates a CM conversion value from the frequency of appearance of a specific person or object from the text included in at least one of the audio data and the video data of the plurality of video contents. 8. The video information providing system according to claim 7. 8. The video information providing system according to claim 7, wherein said weighting means targets a specific corporate name from text included in at least one of said audio data and said video data of said plurality of video contents. .

Images