JP7137815B2 - Recording playback system - Google Patents

Description

The present invention relates to a recording/playback system for selecting and playing back desired content from recorded content.

2. Description of the Related Art Conventionally, there has been known a technique for selecting and playing back desired content from a large amount of captured content.

That is, in such a technology, metadata such as caption text included in the content, broadcast start time, broadcast end time for each corner of the broadcast program, performers, and a summary of the corner are collected, and based on the input keyword The desired content is selected and reproduced by retrieving the metadata based on the information. This metadata may be data provided by a service provider of a paid service (also called a program metadata service) delivered after the end of a broadcast program, or input by a user operating a keyboard or the like. 1).

In addition, the applicant of the present patent has a recording means having a recording file for recording video, a character information obtaining means for obtaining character information from the video recorded in the recording file, and the character information obtained by the character information obtaining means. Character information documenting means for aggregating and documenting character information, and storing the character information documented by the character information documenting means in a metadata storage file as metadata of video recorded in the recording file. proposed a metadata generation system including metadata storage means (see Patent Document 2).

JP 2010-262413 A JP-A-2018-33048

These technologies collate the text displayed in the video and character information from the audio to generate metadata. There is a demand for accurately reproducing the

SUMMARY OF THE INVENTION It is an object of the present invention to provide a recording/reproducing system capable of accurately reproducing desired contents from recorded contents.

The invention according to claim 1 for solving the above-mentioned problem is a video recording means for recording content consisting of video data, a transmission source of the recorded content, transmission date and time, title, performers, scheduled transmission text, voice and content content. and metadata storage means for storing the metadata, wherein the metadata creation means outputs summary text summarizing the contents of the content. said summary creation means comprises summary setting learning means for learning optimum settings for creating a summary by using input data and output data of accumulated past summary creation results as teaching materials; specific information recognition means for interpreting the instruction information to recognize content specific information; content specific means for identifying at least part of the content by searching the metadata based on the content specific information; video data transmission means for transmitting video data of the specified content, wherein the specific information recognition means performs optimal specific information recognition using input data and output data of accumulated past specific information recognition processing as teaching materials. Specific information recognition setting learning means for learning processing settings, wherein the content specifying means learns optimum content specifying settings using input data and output data of accumulated past content specifying results as teaching materials. wherein the summary creation means is composed of speech text describing human utterances in the content extracted from a video signal of the content, and telop text describing display content of telops in the content extracted from the video signal. , a background image text describing at least one description of scenes, situations, articles, and events appearing in the video signal; logo mark text describing marks including logo marks, trademarks, symbols, and symbols appearing in the video signal; text integration means for integrating at least two texts from among and outputting an integrated text; text integration setting means for learning optimum text integration settings by using the input data and output data of the text integration process as teaching materials; and as teaching materials to learn the optimal summarization settings. The video recording/playback system is characterized by comprising summary setting learning means for learning .

According to the present invention, the summary creation means summarizes the content and outputs the summary text. Here, the summarization means has summarization setting learning means for learning optimal summarization settings by using input data and output data of accumulated past summarization results as teaching materials, and learns the optimal summarization settings. . For this reason, the optimal summary creation state is set for the scenes and situations of the content. Therefore, an optimal summary text can be created.

According to the first aspect of the present invention, the specific information recognition means learns the optimum specific information recognition processing settings using input data and output data of past specific information recognition processing stored as teaching materials. Have the means.

According to the present invention, the specific information recognition means learns the optimum specific information recognition setting by using the input data and the output data accumulated in the past specific information recognition processing by the specific information recognition means setting learning means. Therefore, the most suitable specific information can be recognized. Therefore, the optimum specific information is recognized from the input instruction information.

2. The recording/playback system according to claim 1, wherein said content identification means comprises content specific setting learning means for learning optimum content specific settings using input data and output data of accumulated past content identification results as teaching materials. .

According to the present invention, the content specifying means learns the optimum content specifying settings by using the input data and the output data of the accumulated past content specifying results by the content specifying setting learning means as teaching materials. Therefore, the most suitable content can be identified. Therefore, the optimum content is selected from the input instruction information.

2. The recording/reproducing system according to claim 1, wherein said summary creation means includes a speech text describing human utterances in said content extracted from a video signal of said content, and a telop in said content extracted from said video signal. telop text describing display content, background image text describing at least one of scenes, situations, goods, and events appearing in the video signal, and marks including logo marks, trademarks, symbols, and symbols appearing in the video signal. and text integration means for outputting an integrated text by integrating at least two texts of the logo mark text describing the text integration; The means comprises text integration setting means for learning optimal text integration settings using accumulated past input data and output data of text integration processing as teaching materials, and said summary text creation means is adapted to create summaries of the accumulated past text integration settings. A summary setting learning means is provided for learning optimum settings for creating a summary using the resulting input data and output data as teaching materials.

According to the present invention, in the video recording/playback system, the text integration means includes: the summarization means; the utterance text describing the content uttered by a person in the content extracted from the video signal of the content; and the content extracted from the video signal. telop text describing the display content of the telop in the video signal, background image text describing at least one of scenes, situations, goods, and events appearing in the video signal, logo marks, trademarks, symbols, and codes appearing in the video signal Integrate at least two of the logomark texts that describe the mark containing the

Since the text integration means includes the text integration setting means, the text integration setting means learns the optimum text integration settings using the input data and output data of the past text integration processing that have been accumulated as teaching materials.

Therefore, the text integration means can generate the speech text, the telop text, the background image text, and the logo mark text in an optimal state, and integrate each text.

According to a second aspect of the present invention, there is provided, in the recording/reproducing system of the first aspect, means for extracting speech information from an audio signal of the video signal of the content, and recognizing the content of the speech from the speech information. an utterance-to-text conversion unit comprising means for converting the content of the utterance into text and outputting the utterance text; means for extracting telop information for displaying a telop from the video signal of the video signal; and a telop from the telop information. A telop text conversion unit comprising: means for recognizing content; and means for converting the content of the telop into text and outputting the telop text; means for extracting background image information from the video signal of the video signal; A background image text converting unit comprising means for recognizing background image content from information and means for converting the background image content into text and outputting the background text, and extracting logo mark image information from the video signal of the video signal. means for recognizing the contents of the logo mark from the logo mark image information; and means for converting the contents of the logo mark into text and outputting the logo mark text. at least one of the utterance text conversion unit, the telop text conversion unit, the background image text conversion unit, and the logo mark text conversion unit is optimized from input data and output data accumulated in past text conversion processing. text conversion learning means for learning text creation settings.

According to the present invention, the utterance content text conversion unit extracts utterance information from the audio signal of the video signal, recognizes the utterance content, and generates the utterance text. In addition, the telop text conversion unit extracts telop information from the video signal of the video signal, recognizes the display contents of the telop, and generates telop text. The background image text conversion unit extracts background image information from the video signal of the video signal, recognizes the content of the background image, and generates background text. Further, in the logo mark text conversion unit, logo mark text information is recognized from the video signal of the video signal to generate the logo mark text.

Therefore, text is generated from speech information, telop information, background image information, and logo mark information, respectively, and text is generated based on various information from the video signal of the content, and the optimal summary for the scene and situation of the content is generated. can be created.

Further, according to the present invention, each text conversion unit learns based on the input data and output data of the past text conversion processing accumulated by the text conversion learning means.

Therefore, it is possible to learn optimal settings for creating text from a large number of summaries created in the past, and to create optimal summaries for scenes and situations of content similar to the settings.

Further, according to the invention of claim 3, in the recording/playback system of claim 1, the content is acquired from a broadcast program broadcast by a television broadcasting station or a video image distributed over the Internet. and
Therefore, the recording/reproducing system can record/reproduce various contents.

Further, according to a fourth aspect of the present invention, in the recording/playback system of the first aspect, the recording means, the metadata creating means, the metadata storing means, the specific information recognition means, the content specifying means, and the The image data sending means is characterized by being arranged at one place.

According to the present invention, since all the means are arranged in one place, for example, a recording/playback system can be constructed independently at home, and the most appropriate content can be selected from the recorded content according to the voice or text instructions of the user. video signal is played.

Further, the invention according to claim 5 is the recording/playback system according to claim 1, wherein at least the metadata creating means is arranged in a different location from the recording means and is connected to the Internet. do.

According to the present invention, the metadata generating means that requires the most processing power is installed in an external server device, and processing can be performed by connecting to the server device from home via the Internet, and a recording/playback system can be easily constructed.

According to the recording/reproducing system of the present invention, it is possible to accurately reproduce desired contents from recorded contents.

That is, according to the recording/reproducing system of claim 1, the summary creation means summarizes the details of the content and outputs the summary text. Here, the summarization means has summarization setting learning means for learning optimal summarization settings by using input data and output data of accumulated past summarization results as teaching materials, and learns the optimal summarization settings. . For this reason, the optimal summary creation state is set for the scenes and situations of the content. Therefore, an optimal summary text can be created.

Therefore, it is possible to create a summary text that is optimal for the scene or situation of the content by integrating the spoken text, the telop text, the background image text, and the logo mark text.

Further, according to the recording/reproducing system of claim 1, the specific information recognition means optimizes the input data and output data accumulated in the past specific information recognition processing as teaching materials by the specific information recognition means setting learning means. specific information recognition settings. Therefore, the most suitable specific information can be recognized. Therefore, the optimum specific information is recognized from the input instruction information.

Further, according to the recording/reproducing system of claim 1, the content specifying means uses the input data and output data of the accumulated past content specifying results as teaching materials by the content specifying setting learning means to obtain optimum content specifying settings. to learn. Therefore, the most suitable content can be identified. Therefore, the optimum content is selected from the input instruction information.

Further, according to the video recording/playback system of claim 1, in the video recording/playback system, the text integration means includes the summary creation means, the utterance text describing the content uttered by the person in the content extracted from the video signal of the content. , a telop text describing display contents of a telop in the content extracted from the video signal, a background image text describing at least one of scenes, situations, articles, and events appearing in the video signal; At least two texts of a logomark, a trademark, a symbol, a logomark text describing a mark containing a code appearing are integrated.

Here, the text integration means comprises text integration setting means. The text integration setting means learns the optimum text integration setting using the accumulated input data and output data of past text integration processing as teaching materials.

Therefore, the text integration means can generate the speech text, the telop text, the background image text, and the logo mark text in an optimal state, and integrate each text.

Further, according to the recording/playback system of claim 2, the speech content text conversion unit extracts speech information from the audio signal of the video signal, recognizes the speech content, and generates the speech text. In addition, the telop text conversion unit extracts telop information from the video signal of the video signal, recognizes the display contents of the telop, and generates telop text. The background image text conversion unit extracts background image information from the video signal of the video signal, recognizes the content of the background image, and generates background text. Further, in the logo mark text conversion unit, logo mark text information is recognized from the video signal of the video signal to generate the logo mark text.

Therefore, text is generated from speech information, telop information, background image information, and logo mark information, respectively, and text is generated based on various information from the video signal of the content, and the optimal summary for the scene and situation of the content is generated. can be created.

Further, according to the present invention, each text conversion unit learns based on the input data and output data of the past text conversion processing accumulated by the text conversion learning means.

Therefore, it is possible to learn optimal settings for creating text from a large number of summaries created in the past, and to create optimal summaries for scenes and situations of content similar to the settings.

Further, according to the recording/reproducing system of claim 3, it is possible to acquire from a broadcast program broadcast by a television broadcasting station or from a video image distributed over the Internet. Therefore, the recording/reproducing system can record/reproduce various contents.

Further, according to the recording/reproducing system of claim 4, since all the means are arranged in one place, for example, the recording/reproducing system can be constructed independently at home, and the user can instruct by voice or text. A video signal of the most suitable content is reproduced from the recorded content.

Furthermore, according to the recording/playback system of claim 5, the metadata generation means that requires the most processing power is placed in an external server device, and processing can be performed by connecting to this from home via the Internet, You can easily build a recording and playback system.

1 is a block diagram showing the overall configuration of a recording/playback system according to an embodiment of the present invention; FIG. It is a flowchart which shows operation|movement of the same recording reproduction system. FIG. 4 is a block diagram showing a summary creating system of a metadata creating unit in the recording/playback system; FIG. 2 shows an utterance-to-text part of the summary creation system, where (a) is a block diagram and (b) is a diagram showing the flow of processing. FIG. 2 shows a telop-text conversion unit of the summary creation system, where (a) is a block diagram and (b) is a diagram showing the flow of processing. FIG. 2 shows a background image text conversion unit of the abstract creation system, where (a) is a block diagram and (b) is a diagram showing the flow of processing. FIG. 2 shows a logo mark text conversion unit of the abstract creation system, where (a) is a block diagram and (b) is a diagram showing the flow of processing. It is a block diagram which shows the text integration part of the same summary creation system. It is a block diagram which shows the summary preparation part of the same summary preparation system. It is a flowchart which shows the processing procedure of the same abstract creation system. It is a figure which shows the example of a process of the same abstract creation system.

[First Embodiment]
A recording/playback system according to an embodiment of the present invention will be described. First, the overall configuration of the recording/playback system according to the first embodiment will be described. FIG. 1 is a block diagram showing the overall configuration of a summary creation system according to an embodiment of the present invention.

<Overall Configuration of Recording/Playback System 900>
As shown in FIG. 1, the recording/playback system 900 according to the present embodiment includes a video data acquisition unit 910, a metadata creation unit 920 as metadata creation means, a metadata storage unit 930 as metadata storage means, and a recording means. and a video data sending unit 950 serving as video data sending means. The recording/playback system 900 also includes user input means 960, a specific information recognition section 970 as specific information recognition means, a metadata search section 980, and a content identification section 990 as content identification means.

The video data acquisition unit 910 acquires video data of content transmitted from the television broadcasting station 30 or the Internet 40 . The video data acquisition unit 910 is realized, for example, by a multi-tuner that acquires video data from a plurality of television stations, or a web browser that acquires net moving images transmitted via the Internet.

The metadata creating unit 920 creates metadata. This metadata includes broadcast start time data, broadcast end time data, text data obtained from a pay service after the end of the broadcast program, text data entered by the user by operating a keyboard or the like, and metadata. It includes various text data generated by the summary creation system 10 incorporated in the data generation unit 920, abstract text data summarizing these data, and data externally acquired by the metadata generation unit 920. FIG.

The summary creation system 10 includes a description of at least one of the content uttered by a person in the content, the display content of the telop in the content, the scene, the situation, the article, and the event, and the description of the mark including the logo mark, trademark, symbol, and code. At least two of them are integrated to output a summarized text.

The metadata storage section 930 stores metadata generated by the metadata creation section 920 . This metadata is associated with the content.

The recording unit 940 can record a plurality of contents acquired at the same time, and is composed of, for example, a multi-channel video recorder. Image data of multiple contents are stored in the hard disk drive. Each content stores the metadata described above as a tag.

The video data sending unit 950 sends the video data of the designated content to the video/audio reproducing unit 50 to output video and audio. The video/audio reproducing unit 50 is composed of, for example, a liquid crystal display device and a speaker or a television device.

The user input means 960 is composed of a device for inputting broadcast date and time, broadcast station, keywords, and other data in order for the user to specify the content that the user desires to reproduce. User input means 960 are implemented by, for example, a microphone, keyboard, mouse, and joystick. Voice is input from the microphone and text is input from the keyboard. Information is input by selecting a predetermined position from the image displayed on the video/audio reproducing unit 50 with the mouse or joystick.

The specific information recognition unit 970 interprets instruction information from the user input means 960 to recognize and output content specific information that is information for specifying content. A specific information recognition unit 970 recognizes information for specifying content from broadcast date/time, broadcasting station, keywords, and other data. To generate the most appropriate information for specifying contents from information such as listed words.

In the case of voice, the specified information is often input in spoken language. The specific information recognition unit 970 interprets this spoken language and outputs the optimum specific information that identifies the content. For example, it is assumed that "I want to see XX player in XX tennis tournament last month" is input. The specific information recognition unit 970 extracts utterance information from the voice information including this utterance, and further interprets the language. As a result, "month", "○○ tennis tournament", "XX player", etc. are extracted as specific information. Note that this process may not be performed when text is input.

In order to improve the accuracy of this extraction, in the recording/playback system 900 according to this embodiment, the specific information recognition section 970 is provided with a learning section 971 as specific information recognition setting learning means.

The learning unit 971 learns optimal settings for recognizing specific information using accumulated past input data for specific information recognition and output data as recognition results as teaching materials. In learning, data processing is processed quickly and appropriately by AI (Artificial Intelligence). AI processing can be realized by the above-described machine learning (ML). Furthermore, as machine learning, supervised learning can be employed in which existing data is the correct answer. Moreover, it is effective to use deep learning (DL: Deep Learning) as machine learning.

The metadata search unit 980 refers to the metadata storage unit 930 and searches for metadata based on the information specified by the specific information recognition unit 970 . The retrieved metadata is sent to the content specifying section 990 .

The content identification unit 990 identifies predetermined content from a large number of contents identified by the metadata searched by the metadata search unit 980 .

The content identifier 990 may identify multiple pieces of content. For example, content having metadata such as "month", "○○ tennis tournament", and "XX player" is a wide variety of programs such as live broadcasts, news, and other programs.

The content specifying unit 990 selects content of which the designator (user) has a high interest from a large number of selected contents. This selection criterion varies depending on the selector. For this reason, the content specifying unit 990 includes a learning unit 991 as content specifying setting learning means for learning the tendency of the selector.

The learning unit 991 learns settings for optimum specific information recognition using accumulated past input data for content specification and output data as recognition results as teaching materials. In learning, data processing is processed quickly and appropriately by AI (Artificial Intelligence). AI processing can be realized by the above-described machine learning (ML). Furthermore, as machine learning, supervised learning can be employed in which existing data is the correct answer. Moreover, it is effective to use deep learning (DL: Deep Learning) as machine learning.

In the above case, for example, if it is determined that the designated person constantly selects a specific news program, the XX news of the XX station will introduce "the result of the XX player's match at the XX tennis tournament". Select a corner to do. When a plurality of contents are selected, it is possible to display candidates of contents to be played back and designate them by voice, keyboard, mouse, or the like.

Then, the content selected by the content specifying unit 990 is output from the recording unit 940 to the video/audio reproducing unit 50 via the video data transmitting unit 950, and the desired video and audio are output from the audio/video reproducing unit 50.

Next, the processing flow of the recording/playback system 900 will be described. FIG. 2 is a flow chart showing the operation of the recording/playback system.

In the recording/playback system 900 according to the present embodiment, learning is first performed in each unit, that is, the summary creation system 10, the specific information recognition unit 970, and the content identification unit 990 (step S1). This learning can also be performed while the recording/playback system 900 is in operation.

In the recording/playback system 900, in parallel with the learning of each unit, the video data acquired from the video data acquisition unit 910 is recorded by the recording unit 940 (step S2). Then, the metadata creating section 920 immediately acquires the summary text by the summary creating system 10 and other texts from the video data and stores them as metadata in the metadata storage section 930 (step S3). Content recording and metadata storage continue until instructed to stop.

In this state, the user can reproduce the desired content or portion of the content from a large number of recorded contents. For example, when the user inputs by voice (step S4), the voice input from the user input means 960 is recognized by the specific information recognition unit 970 (step S5), and the specific information designation for specifying the content is output. be. Then, the metadata search unit 980 refers to the metadata storage unit 930 and selects one or a plurality of contents and content parts, such as news and wide show corners, as candidates.

The content specifying unit 990 specifies the optimum content from the content candidates (step S7), and causes the video data transmitting unit 950 to transmit the content from the recording unit 940 to be reproduced by the video/audio reproducing unit 50 (step S8). If there is no instruction to end (No in step S9), steps S4 to S8 are repeated as necessary. If there is an instruction to end the reproduction (Yes in step S9), the reproduction ends.

As described above, according to the recording/playback system 900, it is possible to quickly and accurately play back the content desired by the user from the recorded content.

The video data acquisition unit 910, the metadata creation unit 920, the metadata storage unit 930, the recording unit 940, the user input means 960, the specific information recognition unit 970, the metadata search unit 980, and the content identification unit of the recording/playback system 900 990 can be located at one location (eg, the user's home) as a single device. That is, the recording/playback system 900 can be configured as, for example, a multi-channel digital recorder. As a result, the recording/playback system 900 can be used independently at home, and the video signal of the most appropriate content can be played back from the recorded content according to the user's voice or text instructions.

Further, in the recording/playback system 900, the video data acquisition unit 910, the user input means 960, the recording unit 940, and the video data transmission unit 950 are arranged as one device in one place (user's home), and the other The parts, that is, the metadata creating section 920, the metadata storing section 930, the specific information recognizing section 970, the metadata searching section 980, and the content identifying section 990, can be arranged at other locations, such as a service provider company, and connected via the Internet. The metadata generation means that requires the most processing power is placed outside (in the cloud), and processing can be performed by connecting to it from home via the Internet, making it possible to easily build a recording/playback system.

<Summary Production System 10>
The summary creation system 10 arranged in the metadata creating section 920 in the recording/playback system 900 according to the embodiment will be described below.

<Overall Configuration of Abstract Production System 10>
FIG. 3 is a block diagram showing a summary creating system of the metadata creating section in the recording/playback system. As shown in FIG. 3, the summary production system 10 includes a video signal separation unit 20, a speech text conversion unit 100, a telop text conversion unit 200, a background image text conversion unit 300, a logo mark text conversion unit 400, and text integration means. It comprises a text integration section 500 and a summary creation section 600 which is a summary creation means. In this embodiment, the digest production system 10 acquires video signals from broadcast programs from television stations 30 and moving images distributed over the Internet 40 . Note that the video signal can be obtained from moving images on the Internet. At least two portions selected from the telop text conversion unit 200, the background image text conversion unit 300, the logo mark text conversion unit 400, and the text integration unit 500 can be provided.

A video signal V including an audio signal and a video signal is separated into an audio signal A and a video signal B by a video signal separating section 20 . The audio signal A is input to the speech text conversion unit 100 , and the video signal B is input to the telop text conversion unit 200 , the background image text conversion unit 300 , and the logo mark text conversion unit 400 .

<Speech text conversion unit 100>
The speech-to-text conversion unit 100 receives the audio signal A and outputs a speech text, which is a text describing the contents of a person's speech in the content. The speech text conversion unit 100 includes a speech information extraction unit 110 , a speech content recognition unit 120 , and a speech content text conversion unit 130 .

The speech information extraction unit 110 extracts speech information from the audio signal A of the video signal V. FIG. Namely, the noise in the audio signal A is removed, and the information of the human speech is extracted. This speech information can include sound effects and characteristic music.

The utterance content recognition unit 120 recognizes the utterance content from the utterance information. That is, the utterance information is acoustically and grammatically analyzed to recognize the contents of the utterance as language. The parameters, conditions, and the like used for this recognition can be generated by machine learning from accumulated past speech text generation data, as will be described later.

The utterance content text conversion unit 130 converts the utterance content into text and outputs it. The parameters, conditions, and the like used for this recognition can be generated by machine learning from input data and generated data of past voice texts that have been accumulated as will be described later.

<telop text conversion unit 200>
A telop text converting unit 200 receives the video signal B and outputs telop text, which is text describing the content of the telop in the content. The telop text conversion unit 200 includes a telop information extraction unit 210 , a telop content recognition unit 220 , and a telop content text conversion unit 230 .

The telop information extraction unit 210 extracts telop information from the video signal B of the video signal V. FIG. That is, the background in the video signal B is removed and the information of only the telop image is extracted.

The utterance content recognition unit 120 recognizes the telop content from the telop image information. That is, the telop information is linguistically and grammatically analyzed to recognize the telop display content as a language. The parameters, conditions, and the like used for this recognition can be generated by machine learning from accumulated past telop text input data and generated data, as will be described later.

The telop content text conversion unit 230 converts the telop content into text and outputs it. The parameters, conditions, and the like used for this recognition can be generated by machine learning from accumulated past telop text input data and generated data, as will be described later.

<Background image text conversion unit 300>
The background image text conversion unit 300 receives the video signal B and outputs background image text, which is text describing the background image content in the content. Background images include at least one of scene, situation, article, and event, such as a person, a person's belongings, a person's facial expression, a landscape, a building situation, an indoor situation, an animal, a vehicle, and other articles. can be mentioned. The background image text conversion unit 300 includes a background image information extraction unit 310 , a background image content recognition unit 320 , and a background image content text conversion unit 330 .

Background image information extraction section 310 extracts background image information from video signal B of video signal V. FIG. That is, telops and unclear images in the video signal B are removed, and information of only the recognizable background image is extracted.

The background image content recognition unit 320 recognizes the content of the background image from the background image information. That is, the person, the belongings of the person, the expression of the person, the scenery, the state of the building, the state of the room, the animal, the vehicle, and other articles represented by analyzing the background image information are recognized. The parameters, conditions, and the like used for this recognition can be generated by machine learning from past background image text input data and generated data that have been accumulated as will be described later.

The background image content text conversion unit 330 converts the background image content into text and outputs it. The parameters, conditions, and the like used for this recognition can be generated by machine learning from past background image text input data and generated data that have been accumulated as will be described later.

<Logo Mark Text Conversion Unit 400>
A logo mark text conversion unit 400 receives the video signal B and outputs logo mark text, which is text describing the content of the logo mark in the content. Logomarks may include trademarks, symbols, marks containing symbols, and other indicia that indicate the origin of goods. The logo mark text conversion unit 400 includes a logo mark image information extraction unit 410 , a logo mark content recognition unit 420 , and a logo mark content text conversion unit 430 .

The logo mark image information extraction unit 410 extracts logo mark image information from the video signal B of the video signal V. FIG. That is, telops and background images in the video signal B are removed, and information of only recognizable logo mark images is extracted.

The logo mark content recognition unit 420 recognizes the content of the logo mark from the logo mark image information. That is, the product, service, store, facility, etc. represented by analyzing the logo mark image information are recognized. The parameters, conditions, and the like used for this recognition can be generated by machine learning from past logo mark text input data and generated data accumulated as will be described later.

The logo mark content text conversion unit 430 converts the content of the logo mark image into text and outputs the text. The parameters, conditions, and the like used for this recognition can be generated by machine learning from past logo mark text input data and generated data accumulated as will be described later.

<Text Integration Unit 500>
The text integration unit 500 integrates the speech text from the speech text conversion unit 100, the telop text from the telop text conversion unit 200, the background text from the background image text conversion unit 300, and the background text from the logo mark text conversion unit 400. . That is, it corrects contradictions and errors in each text to generate an integrated text. The parameters, conditions, etc. used for this text integration can be generated by machine learning from the input and output data of past text integration, which has been accumulated as described later.

<Summary Creation Unit 600>
Summarizer 600 summarizes the integrated text from text integrator 500 . That is, the contents of the summary text are summarized to the specified number of characters. The parameters, conditions, etc. used for this summarization can be generated by machine learning from the input data and output data of past winning combination processing accumulated as will be described later.

Next, machine learning processing of each unit will be described.
<Machine learning processing of the speech-to-text unit 100>
FIG. 4 shows an utterance-to-text unit of the same summary creation system, where (a) is a block diagram and (b) is a diagram showing the flow of processing. As shown in FIG. 1(a), the utterance text conversion unit 100 includes an utterance information extraction unit 110, an utterance content recognition unit 120, an utterance content text conversion unit 130, a machine learning unit 140 as text conversion learning means, a content A recognition text creation setting unit 150 and a comparison evaluation unit 160 are provided. An existing data storage unit 700 is also connected to the speech text conversion unit 100 .

The speech text conversion unit 100 performs machine learning based on the existing video data stored in the existing data storage unit 700 and the existing speech text, and optimizes the speech content recognition unit 120 and the speech content text conversion unit 130 . An existing data storage unit 700 stores an existing video data storage unit 710 storing a large number of video data used when a person created an utterance text in the past, and an utterance text created from the utterance contents of this video data. An existing speech text storage unit 720 is provided. These video data and spoken texts serve as teaching materials for machine learning.

The speech-to-text unit 100 also includes switching units 170 and 180 for switching data output when performing machine learning and when creating speech content text from new video data.

The content recognition text creation setting unit 150 sets the utterance information extraction processing of the utterance information extraction unit 110, sets the utterance content recognition processing of the utterance content recognition unit 120, and sets the text conversion processing of the utterance content text conversion unit 130. and are stored. The utterance information extraction unit 110 , the utterance content recognition unit 120 , and the utterance content text conversion unit 130 perform speech information extraction, speech content recognition, and text conversion according to the conditions and parameters set by the content recognition text creation setting unit 150 .

The comparison evaluation section 160 includes a comparison section 161 and an evaluation section 162 . The comparison unit 161 compares the speech text created by the speech content text conversion unit 130 upon receiving the existing video data from the existing video data storage unit 710 and the existing speech text from the existing speech text storage unit 720 . The evaluation unit 162 evaluates based on the comparison result of the comparison unit 161, and gives a high score when there is a good match.

The machine learning unit 140 receives the evaluation from the evaluation unit 162 and changes the setting state of the content recognition text creation setting unit 150 . This processing is repeated for the same video data, and the evaluation value of the evaluation unit 162 is made as high as possible. This process can be repeated for multiple pieces of video data.

By performing such machine learning, the capabilities of the utterance content recognition unit 120 and the utterance content text conversion unit 130 are improved. After completing the predetermined machine learning, the speech-to-text conversion unit 100 processes the new video data and is ready to output the optimum speech text.

Processing of the speech-to-text conversion unit 100 will be described. As shown in FIG. 4B, first, feature amounts for speech recognition and text conversion are set in the content recognition text creation setting unit 150 (step SA1). This setting is performed based on the learning result of the machine learning unit 140 .

Next, utterance information extraction section 110 extracts speech from a large amount of speech signals based on the set features (step SA2).

Furthermore, the utterance content recognition unit 120 analyzes the extracted voice based on the set features (step SA3).

Then, the utterance content text conversion unit 130 converts the voice into text based on the set features and outputs the utterance text (step SA4).

<Machine learning of telop text conversion unit 200>
FIG. 5 shows the telop-text conversion unit of the summary creation system, where (a) is a block diagram and (b) is a diagram showing the flow of processing. As shown in FIG. 2A, the telop text conversion unit 200 includes a telop information extraction unit 210, a telop content recognition unit 220, a telop content text conversion unit 230, a machine learning unit 240, and a content recognition text creation setting unit 250. , and a comparison evaluation unit 260 . An existing data storage unit 700 is also connected to the telop text conversion unit 200 .

The telop text conversion unit 200 performs machine learning based on existing video data and existing telop text stored in the existing data storage unit 700 to optimize the telop content recognition unit 220 and the telop content text conversion unit 230 . An existing data storage unit 700 stores an existing video data storage unit 710 storing a large number of video data used when a person created telop text in the past, and telop text created from the utterance content of this video data. An existing telop text storage unit 730 is provided. These video data and spoken texts serve as teaching materials for machine learning.

Further, the telop text conversion unit 200 includes switching units 270 and 280 for switching data output when performing machine learning and when creating speech content text from new video data.

The content recognition text creation setting unit 250 sets the telop information extraction of the telop information extraction unit 210, the text content recognition processing setting of the telop content recognition unit 220, and the text conversion setting of the telop content text conversion unit 230. stored. The telop information extractor 210, the telop content recognition section 220, and the telop content text conversion section 230 perform telop extraction, content recognition, and text conversion according to the conditions and parameters set by the content recognition text creation setting section 250. FIG.

The comparison evaluation section 260 includes a comparison section 261 and an evaluation section 262 . The comparison unit 261 compares the telop text created by the telop content text conversion unit 230 upon receiving the existing video data from the existing video data storage unit 710 and the existing telop text from the existing telop text storage unit 730 . The evaluation unit 262 evaluates based on the comparison result of the comparison unit 261, and gives a high score when there is a good match.

The machine learning section 240 receives the evaluation from the evaluation section 262 and changes the setting state of the content recognition text creation setting section 250 . This processing is repeated for the same video data, and the evaluation value of the evaluation unit 262 is made as high as possible. This process can be repeated for multiple pieces of video data.

By performing such machine learning, the capabilities of the telop content recognition unit 220 and the telop content text conversion unit 230 are improved. After completing the predetermined machine learning, the telop text conversion unit 200 processes the new video data and is ready to output the optimum telop text.

The processing of the telop text conversion unit 200 will be described. As shown in FIG. 5B, first, feature amounts for telop information extraction, content extraction, and text conversion are set in the content recognition text creation setting unit 250 (step SB1). This setting is performed according to the learning result of the machine learning unit 240 (step SB2).

Next, the telop information extraction unit 210 extracts telops from a large amount of video signals based on the set features (step SB2).

Further, the telop content recognition unit 220 analyzes the extracted telop based on the set features (step SB3).

Then, the telop content text conversion unit 230 converts the content of the telop into text based on the set features and outputs it as telop text (step SB4).

<Machine learning of the background image text conversion unit 300>
FIG. 6 shows a background image text conversion unit of the same summary creation system, where (a) is a block diagram and (b) is a diagram showing the flow of processing. As shown in FIG. 4A, the background image text conversion unit 300 includes a background image information extraction unit 310, a background image content recognition unit 320, a background image content text conversion unit 330, and a machine learning function as text integration setting means. It comprises a section 340 , a content recognition text creation setting section 350 and a comparative evaluation section 360 . An existing data storage unit 700 is connected to the background image text converting unit 300 .

The background image text conversion unit 300 performs machine learning based on the existing video data stored in the existing data storage unit 700 and the existing background image text, and optimizes the background image content recognition unit 320 and the background image content text conversion unit 330. do. The existing data storage unit 700 stores an existing video data storage unit 710 storing a large number of video data used when a person created telop text in the past, and a background image text created from the utterance content of this video data. An existing background image text storage unit 740 is provided. These video data and background image text serve as teaching materials for machine learning.

The background image text conversion unit 300 also includes switching units 370 and 380 for switching data output when machine learning is performed and when speech content text is created from new video data.

The content recognition text creation setting unit 350 sets the background image extraction processing of the background image information extraction unit 310, sets the background image content recognition processing of the background image content recognition unit 320, and sets the text conversion of the background image content text conversion unit 330. Processing settings are stored. The background image information extraction unit 310, the background image content recognition unit 320, and the background image content text conversion unit 330 perform background image extraction, background image content recognition, and text conversion according to the conditions and parameters set by the content recognition text creation setting unit 350. conduct.

The comparison evaluation section 360 includes a comparison section 361 and an evaluation section 362 . The comparison unit 361 compares the background image text created by the background image content text conversion unit 330 upon receiving the existing video data from the existing video data storage unit 710 and the existing background image text from the existing background image text storage unit 740. do. The evaluation unit 362 performs evaluation based on the comparison result of the comparison unit 361, and gives a high score when there is a good match.

The machine learning section 340 receives the evaluation from the evaluation section 362 and changes the setting state of the content recognition text creation setting section 350 . This processing is repeated for the same video data, and the evaluation value of the evaluation unit 362 is made as high as possible. This process can be repeated for multiple pieces of video data.

By performing such machine learning, the capabilities of the background image content recognition unit 320 and the background image content text conversion unit 330 are improved. After completing the predetermined machine learning, the background image text conversion unit 300 is ready to process the new video data and output the optimum background image text.

Processing of the background image text conversion unit 300 will be described. As shown in FIG. 6B, first, the feature amounts for background image information extraction, background image recognition, and text conversion are set in the content recognition text creation setting unit 350 (step SC1). This setting is performed based on the learning result of the machine learning unit 340 .

Next, the background image information extraction unit 310 extracts a background image from a large amount of video signals based on the set features (step SC2).

Further, the background image content recognition unit 320 analyzes the extracted background image based on the set features (step SC3).

Then, the background image content text conversion unit 330 converts the content of the background image into text based on the set features and outputs the text as background image text (step SC4).

<Machine learning of the logo mark text conversion unit 400>
FIG. 7 shows a logo mark text conversion unit of the abstract creation system, where (a) is a block diagram and (b) is a diagram showing the flow of processing. The logo mark text conversion unit 400 includes a logo mark image information extraction unit 410 , a logo mark content recognition unit 420 , a logo mark content text conversion unit 430 , a machine learning unit 440 , a content recognition text creation setting unit 450 , a comparison evaluation unit 460 . Prepare. An existing data storage unit 700 is connected to the logo mark text conversion unit 400 .

The logo mark text conversion unit 400 performs machine learning based on the existing video data stored in the existing data storage unit 700 and the existing logo mark text. Optimize the content texting unit 430 . The existing data storage unit 700 stores an existing video data storage unit 710 storing a large number of video data used when people created logo mark texts in the past, and a logo mark text created from the utterance content of this video data. A stored existing logo mark text storage unit 750 is provided. These video data and logo mark text serve as teaching materials for machine learning.

Further, the logo mark text conversion unit 400 is provided with switching units 470 and 480 for switching data output when performing machine learning and when creating speech content text from new video data.

The content recognition text creation setting section 450 stores the settings of the logo mark image content recognition processing of the logo mark content recognition section 420 and the settings of the text conversion processing of the logo mark content text conversion section 430 . The logo mark image information extraction unit 410 , the logo mark content recognition unit 420 , and the logo mark content text conversion unit 430 perform logo mark extraction, content recognition, and text conversion according to the conditions and parameters set by the content recognition text creation setting unit 450 .

The comparison evaluation section 460 includes a comparison section 461 and an evaluation section 462 . The comparison unit 461 compares the text created by the logo mark content text conversion unit 430 upon receiving the existing video data from the existing video data storage unit 710 and the existing background image text from the existing logo mark text storage unit 750 . The evaluation unit 462 evaluates based on the comparison result of the comparison unit 461, and gives a high score when there is a good match.

The machine learning section 440 receives the evaluation from the evaluation section 462 and changes the setting state of the content recognition text creation setting section 450 . This processing is repeated for the same video data, and the evaluation value of the evaluation unit 462 is made as high as possible. This process can be repeated for multiple pieces of video data.

By performing such machine learning, the capabilities of the logo mark content recognition unit 420 and the logo mark content text conversion unit 430 are improved. After completing the predetermined machine learning, the logo mark text conversion unit 400 is ready to process the new video data and output the optimum background image text.

Processing of the logo mark text conversion unit 400 will be described. As shown in FIG. 7B, first, the feature amount of the logo mark is set in the content recognition text creation setting unit 450 (step SC1). This setting is performed based on the learning result of the machine learning unit 340 .

Next, the logo mark image information extraction unit 410 extracts the logo mark from a large amount of video signals based on the set features (step SD2).

Further, the logo mark content recognition unit 420 analyzes the extracted background image based on the set features, automatically confirms and registers it (step SD3).

Further, the logo mark content recognition unit 420 recognizes a registered logo mark or a specific logo mark as a logo mark (step SD4).

Then, the logo mark content text conversion unit 430 converts the content of the logo mark into text based on the set features and outputs it as logo mark text (step SD5).

<Machine Learning of Text Integration Unit 500>
FIG. 8 is a block diagram showing the text integration section of the summary creation system. The text integration unit 500 includes an integrated text creation unit 510 , an integrated text creation setting unit 520 , a machine learning unit 530 and a comparison evaluation unit 540 . An existing data storage unit 700 is connected to the text integration unit 500 .

The text integration unit 500 performs machine learning based on existing integrated texts stored in the existing data storage unit 700, namely, speech text, telop text, background text, logo mark text, and existing integrated texts. Optimize behavior. The existing data storage unit 700 includes an existing text storage unit 760 that stores various text data used when integrated texts were created in the past, and an existing integrated text storage unit 770 that stores integrated texts created from these various texts. and These various texts and integrated texts serve as teaching materials for machine learning.

The text integration unit 500 also includes switching units 570 and 580 for switching data output when performing machine learning and when creating a new integrated text from various new texts.

The integrated text creation setting section 520 stores settings for text integration processing of the integrated text creation section 510 . The integrated text creation section 510 performs text integration processing according to the conditions and parameters set by the integrated text creation setting section 520 .

The comparison evaluation section 540 includes a comparison section 541 and an evaluation section 542 . The comparison unit 541 compares the integrated text created by the integrated text creation unit 510 upon receipt of the various existing texts from the existing various text storage unit 760 and the existing integrated text from the existing integrated text storage unit 770 . The evaluation unit 542 performs evaluation based on the comparison result of the comparison unit 541, and gives a high score when there is a good match.

The machine learning section 530 receives the evaluation from the evaluation section 542 and changes the setting state of the integrated text creation setting section 520 . This processing is repeated for the same text data, and the evaluation value of the evaluation unit 542 is made as high as possible. This process can be repeated for a plurality of various text data.

By performing such machine learning, the ability of the integrated text creation unit 510 is improved. After completing the predetermined machine learning, the text integrator 500 is ready to process the new video data and output the optimal integrated text.

<Machine learning of summary creation unit 600>
FIG. 9 is a block diagram showing the abstract creating section of the same abstract creating system. The summary creation section 600 includes a summary text creation section 610 , a summary creation setting section 620 , a machine learning section 630 as summary setting learning means, and a comparison evaluation section 640 . An existing data storage unit 700 is connected to the summary creation unit 600 .

The summary creation section 600 performs machine learning based on the integrated text and the summary text stored in the existing data storage section 700 to optimize the operation of the summary text creation section 610 . The existing data storage unit 700 contains an existing integrated text storage unit 770 that stores integrated text data used when summarizing texts were created in the past, and past summary creation results that store summary texts created from this integrated text. and an existing summary text store 780 . These integrated texts and summary texts serve as teaching materials for machine learning.

The summary creation unit 600 also includes switching units 670 and 680 for switching data output when performing machine learning and when creating a new summary text from a new integrated text.

The summary creation setting section 620 stores the settings of the summary processing of the summary text creation section 610 . The summary text creation section 610 performs text summary processing according to the conditions and parameters set by the summary creation setting section 620 .

The comparison evaluation section 640 includes a comparison section 641 and an evaluation section 642 . The comparison unit 641 compares the summary text created by the summary text creation unit 610 upon receiving the existing integrated text from the existing integrated text storage unit 770 and the summary text from the existing summary text storage unit 780 . The evaluation unit 642 evaluates based on the comparison result of the comparison unit 641, and gives a high score when there is a good match.

The machine learning section 630 receives the evaluation from the evaluation section 642 and changes the setting state of the summary creation setting section 620 . This processing is repeated for the same text data, and the evaluation value of the evaluation unit 642 is made as high as possible. This process can be repeated for a plurality of integrated text data.

By performing such machine learning, the ability of the summary text creation unit 610 is improved. After completing the predetermined machine learning, the digester 600 is ready to process the new video data and output the optimal text summary.

Next, the processing of abstract production system 10 will be described. FIG. 10 is a flow chart showing the operation of the abstract creation system. First, an existing video data storage unit 710, an existing speech text storage unit 720, an existing telop text storage unit 730, an existing background image text storage unit 740, an existing logo mark text storage unit 750, and various existing text storage units of the existing data storage unit 700. 760, the existing integrated text storage section 770, and the existing summary text storage section 780, the existing video signal and various text data are read (step ST1).

Next, machine learning processing is performed in the speech text conversion unit 100, the telop text conversion unit 200, the background image text conversion unit 300, and the logo mark text conversion unit 400 (steps ST2a, ST2b, ST2c, ST2d). This learning process can also be performed sequentially.

Next, existing input data and output data are read into the existing data storage unit 550 of the text integration unit 500 and the existing data storage unit 650 of the summary creation unit 600 (step ST3). Next, machine learning processing is performed in the text integration section 500 and the summary creation section 600 (steps ST3a and ST3b). This learning process can also be performed sequentially. What is the machine learning processing of the utterance text conversion unit 100, the telop text conversion unit 200, the background image text conversion unit 300, and the logo mark text conversion unit 400, and the machine learning processing of the text integration unit 500 and the summary creation unit 600? Regardless of the order of processing, it can be performed in the reverse order.

When the learning process is completed (yes in step ST4), the video signal to be processed is input to the video signal separating section 20 (step ST5). As a result, the speech text conversion unit 100, the telop text conversion unit 200, the background image text conversion unit 300, and the logo mark text conversion unit 400 execute text conversion processing (steps ST6a, ST6b, ST6c, ST6d).

Then, each text is integrated by the text integration unit 500 (step ST7), the integrated text is summarized by the summary generation unit 600 (step ST8), and the summary text is output. finish.

From the next digest creation process, the machine learning process (steps ST1 to ST4) is not performed, and the optimum digest can be created by simply inputting the video signal to be summarized (step ST5). Also, machine learning processing can be performed as needed.

An example of processing of the abstract production system 10 will be described below. FIG. 11 is a diagram showing a processing example of the same abstract creation system. Video data includes video data including video 810 and audio data including narration 850 . This example is a part of a news program announcing the results of a tennis tournament.

First, in the utterance text conversion unit 100, the utterance information extraction unit 110 extracts the narration 850 from the voice data, and the utterance content recognition unit 120 recognizes the content. Then, this recognized content is converted into text by the utterance content text conversion unit 130 .

In this example, the utterance content text conversion unit 130 generates the utterance text as follows: "At the XX Open Championship held in ▲▲, on XX day on XX, Japan's XX player won the semi-finals and advanced to the finals. The final will be held on the coming day at □.The game will be broadcast live on △△ TV from ♦." is output.

Also, in the telop text conversion unit 200, the telop information extraction unit 210 extracts a telop image from the video data. In this example, the telop displayed in the telop display portion 811, the news name 812 displayed above the video 810, and the provider company name 813 are extracted. Further, the telop content recognition unit 220 recognizes the content of the telop, and the telop content recognition unit 220 recognizes the content of the telop. Then, the background image content text conversion unit 330 converts the recognized content into text.

In this example, the telop content text conversion unit 230 displays the telop display unit 811 with “Breaking news ○○ player XX open finals!” Company” is output as telop text.

Furthermore, in the background image text conversion unit 300 , the background image information extraction unit 310 extracts the person, the article, the situation, etc. from the background image 820 . In this example, images of a tennis player 830, face 831, cap 832, tennis wear 833, tennis pants 834, racket 835, ball 836, tennis court 837, etc. are extracted. Based on this extracted image, the background image content recognition unit 320 recognizes the content of each background image. For example, gender, race, age, and expression are recognized from the image and face 831 of the tennis player 830 . Also, the style, color, etc. of the cap 832, tennis wear 833, and tennis pants 834 are recognized. Further, the color, model number, etc. of the racket 835 and the ball 836 are also recognized. Then, the background image content text conversion unit 330 converts the recognized content into text.

In this example, the tennis player 830 is "male", "○○ type", "twenties", etc.; "Short sleeve", "white" and "shorts" for tennis pants 834, "manufacturer" and "model number" for racket 835, "yellow" for ball 836, and "clay" for tennis court 837 are output as background image text. be.

Then, in the logo mark text conversion unit 400 , the logo mark image information extraction unit 410 extracts the logo mark from the image 810 . In this example, the logo mark 841 attached to the top of the video 810 together with the news name 812, the logo mark 842 attached to the tennis wear 833, and the logo mark 843 attached to the tennis pants 834 are extracted. At this time, since the logo mark 842 and the logo mark 843 are written on the clothes, they are deformed by the movement of the tennis player 830 . In the present example, the extraction is performed in consideration of this deformation. Based on the extracted logo marks, the background image content recognition unit 320 recognizes the content of each logo mark. Then, the content of the recognized logo mark is converted into text by the logo mark content text conversion unit 430 .

In this example, the background image content text conversion unit 330 has the logo mark 841 described as “ΔΔN” and is “ΔΔ Television Co., Ltd.”, and the logo mark 842 as “AAA” and “BBB Inc.”. and "BB" for the logo mark 843, which is the description of "BB brand" of "CC Co., Ltd.", are output as the logo mark text.

The text integration unit 500 combines the speech text from the speech text conversion unit 100, the telop text from the telop text conversion unit 200, the background image text from the background image text conversion unit 300, and the logo mark text from the logo mark text conversion unit 400. Integrate.

In this example, "At the XX Open Championship held in ▲▲, on XX, XX, Japan won the semi-finals and advanced to the finals. The finals will be held on the coming XX days △△TV will be broadcasting this game live from ◆.”, “Breaking news ○○ player”, advanced to the open finals! ”, “△△ News”, “Provided ◆◆ Co., Ltd.”, “Male”, “○○ system”, “20s”, “Sweat”, “Painful”, “Cap white”, “Tennis wear white”, "Tennis Wear Short Sleeves", "Tennis Pants White", "Shorts", "Racquet Maker", "Model Number", "Ball Yellow", "Tennis Court Clay", "△△N", "△△ Television Co., Ltd." , "logomark AAA", and texts such as "BBB Co., Ltd.", "AAA brand", "logomark BB", "CC Co., Ltd.", and "BB brand" are comprehensively integrated and output.
Although this text for integration is a huge amount of data, it is suitable for machine search.

The summarizing unit 600 receives the integrated text from the text integrating unit 500, summarizes it within a predetermined number of characters, for example, 200 characters, and outputs it. In this example, the summary creation unit 600 writes, "Japanese XX player won the semi-finals of the XX Open Championship on XX day and advanced to the final. The XX Open Championship is being held in ▲▲. , held on a clay court.○○ won the match wearing a white cap, AAA brand white tennis wear, and BB brand white tennis pants.The final match will be held on Monday, December, at will be broadcast live on △△ TV from Since the summary is easy to grasp the content, people can understand it at a glance.

The recording/playback system described above includes a CPU (Central Processing Unit) as a processing device, and RAM (Random Access Memory), ROM (Read Only Memory), HDD (Hard Disc Drive), SSD (Solid State Drive), etc. as storage devices. It can be realized by executing application software on a computer system equipped with. Moreover, each part does not need to be arranged at the same place, and a part thereof can be arranged on the cloud and connected by a network. Moreover, since these processes target a large amount of data, it is desirable to process them using a GPU (Graphics Processing Unit).

That is, the integrated text is simply textualization of voices, characters, background images, etc., and is data about a huge amount of sentences. Therefore, high-speed processing can be achieved by specializing the GPU to text processing.

Furthermore, the text integration unit 500 can be input with text obtained from electronic program guide data in addition to speech text, telop text, background image text, and logo mark text in the case of television programs. Furthermore, in the case of moving images distributed over the Internet, evaluations and reputations of third parties can be acquired as text and input. This can improve the quality and quantity of the integrated text.

[Other embodiments]
In the recording/reproducing system according to the present invention, data processing is performed at high speed and appropriately by AI (Artificial Intelligence). AI processing can be realized by the above-described machine learning (ML). Furthermore, as machine learning, supervised learning can be employed in which existing data is the correct answer. Moreover, it is effective to use deep learning (DL: Deep Learning) as machine learning.

In deep learning, learning is performed using a large number of existing video data, various text data corresponding to each video data, integrated text, and summary text as big data. Each machine learning unit has an input layer, multiple intermediate layers, and an output layer, and performs processing using a neural network with a large number of neurons. That is, the new video data input to the abstract creation system according to the present invention, various texts, integrated texts, and abstracts based on this video data are input, and the intermediate layer is added so that the output approaches the existing various texts, integrated texts, and abstracts. The weights and parameters in the neurons of are optimized by methods such as the method of least squares.

INDUSTRIAL APPLICABILITY The present invention is widely applicable to recording/reproducing systems and recording/reproducing devices for television and Internet moving images, and has industrial applicability.

10: Summary Production System 20: Video Signal Separating Section 30: Television Broadcasting Station 40: Internet 50: Video/Audio Reproducing Section 100: Speech Text Conversion Section 110: Speech Information Extraction Section 120: Speech Content Recognition Section 130: Speech Content Text Conversion Section 140: machine learning unit 150: content recognition text creation setting unit 160: comparison evaluation unit 161: comparison unit 162: evaluation unit 170: switching unit 180: switching unit 200: telop text conversion unit 210: telop information extraction unit 220: telop content Recognition unit 230: Telop content text conversion unit 240: Machine learning unit 250: Content recognition text creation setting unit 260: Comparative evaluation unit 261: Comparison unit 262: Evaluation unit 270: Switching unit 280: Switching unit 300: Background image text conversion unit 310: Background image information extraction unit 320: Background image content recognition unit 330: Background image content text conversion unit 340: Machine learning unit 350: Content recognition text creation setting unit 360: Comparative evaluation unit 361: Comparison unit 362: Evaluation unit 370: Switching section 380: Switching section 400: Logo mark text conversion section 410: Logo mark image information extraction section 420: Logo mark content recognition section 430: Logo mark content text conversion section 440: Machine learning section 450: Content recognition text creation setting section 460 : Comparative evaluation unit 461: Comparison unit 462: Evaluation unit 470: Switching unit 480: Switching unit 500: Text integration unit 510: Integrated text creation unit 520: Integrated text creation setting unit 530: Machine learning unit 540: Comparative evaluation unit 541: Comparison unit 542: Evaluation unit 550: Existing data storage unit 570: Switching unit 580: Switching unit 600: Summary creation unit 610: Summary text creation unit 620: Summary creation setting unit 630: Machine learning unit 640: Comparative evaluation unit 641: Comparison Unit 642: Evaluation unit 650: Existing data storage unit 670: Switching unit 680: Switching unit 700: Existing data storage unit 710: Existing video data storage unit 720: Existing speech text storage unit 730: Existing telop text storage unit 740: Existing background Image text storage unit 750: Existing logo mark text storage unit 760: Existing various text storage unit 770: Existing integrated text storage unit 780: Existing summary text storage unit 810: Video 811: Telop display unit 812: News name 813: Provider company name 820: Background image 830: Tennis player 831: Face 832: Cap 833: Tennis wear 834: Tennis pants 835: Racket 836: Ball 837: Tennis court 841: Logo mark 842: Logo mark 843: Logo mark 850: Narration 900: Recording/playback system 910: Video data acquisition unit 920: Metadata creation unit 930: Metadata storage unit 940: Recording unit 950: Video data transmission unit 960: User input means 970: Specific information recognition unit 971: Learning Part (specific information recognition setting learning means)
980: metadata search unit 990: content identification unit 991: learning unit (content identification setting learning means)

Claims (5)

a recording means for recording content consisting of video data;
Metadata creation means for creating metadata including the source of recorded content, date and time of transmission, title, performers, scheduled transmission text, audio, and a summary of the contents of the recorded content;
and metadata storage means for storing the metadata,
the metadata creation means comprises summary creation means for outputting a summary text summarizing the contents of the content;
The summarization means comprises summarization setting learning means for learning optimal summarization settings using input data and output data of accumulated past summarization results as teaching materials,
specific information recognition means for interpreting the instruction information and recognizing the content specific information when the instruction information is input;
content identification means for identifying at least part of the content by searching the metadata based on the content identification information;
video data sending means for sending video data of the identified content;
with
The specific information recognition means is
a specific information recognition setting learning means for learning an optimum specific information recognition process setting using accumulated past input data and output data of specific information recognition process as teaching materials;
The content specifying means is
a content specific setting learning means for learning optimum content specific settings using input data and output data of accumulated past content specific results as teaching materials;
The summary creation means is
an utterance text describing human utterances in said content extracted from a video signal of said content;
a telop text describing display contents of a telop in the content extracted from the video signal;
background image text describing at least one of scenes, situations, objects and events appearing in said video signal;
Logomark text describing marks, including logomarks, trademarks, symbols, and symbols appearing in the video signal;
a text integration means for integrating at least two texts among and outputting an integrated text;
summary text creation means for outputting a summary text summarizing the integrated text;
with
The text integration means comprises text integration setting means for learning optimal text integration settings using the accumulated input data and output data of past text integration processing as teaching materials,
The recording/playback system, wherein the summary text creating means comprises a summary setting learning means for learning optimum settings for creating the summary using input data and output data of accumulated past summary creation results as teaching materials. An utterance text comprising means for extracting utterance information from the audio signal of the video signal of the content, means for recognizing the utterance content from the utterance information, and means for converting the utterance content into text and outputting the utterance text. chemical department and
A telop comprising: means for extracting telop information for displaying a telop from a video signal of said video signal; means for recognizing telop content from said telop information; and means for converting said telop content into text and outputting telop text. a textualization unit;
A background image comprising means for extracting background image information from the video signal of the video signal, means for recognizing background image content from the background image information, and means for converting the background image content into text and outputting the background text. a textualization unit;
A logo comprising means for extracting logo mark image information from the video signal of said video signal, means for recognizing logo mark content from said logo mark image information, and means for converting said logo mark content into text and outputting logo mark text. a mark textualization unit;
comprising at least two textualizations selected from
At least one of the utterance text conversion unit, the telop text conversion unit, the background image text conversion unit, and the logo mark text conversion unit selects an optimum text conversion unit from input data and output data accumulated in past text conversion processing. 2. The recording/reproducing system according to claim 1, further comprising text conversion learning means for learning text creation settings. 2. The recording/reproducing system according to claim 1, wherein said content is acquired from a broadcast program broadcast by a television broadcasting station or a video image distributed over the Internet. 2. The apparatus according to claim 1, wherein said recording means, said metadata creating means, said metadata storing means, specific information recognizing means, said content specifying means, and said video data transmitting means are arranged in one place. The recording and playback system described. 2. The recording/playback system according to claim 1, wherein at least said metadata creating means is arranged at a location different from said recording means and is connected to the Internet.

Images