JP7241131B2 - Information extraction device and information extraction program - Google Patents

Description

The present invention relates to an information extracting device and an information extracting program for extracting information from video of moving image contents (hereinafter referred to as programs) such as programs distributed by television broadcasting or communication.

Programs produced by broadcasters are important assets for the broadcasters themselves as program archives.

The recorded program archive data is stored together with the program video, with detailed metadata (additional information) attached to each program for information retrieval and secondary use at a later date.

By the way, conventionally, the extraction and input work of metadata as described above are performed in specialized departments.

The input of such information is often manually input by the person in charge, requiring a great deal of labor and time. In particular, programs related to news and culture are broadcast for a long time every day, and it takes time to grasp the content.

Therefore, development of an information extracting device and the like for automatically extracting information from programs related to news reports and culture is underway.

For example, Japanese Patent Laid-Open No. 2002-200001 is cited as a technology related to an information extraction device and the like.

Japanese Patent Application Laid-Open No. 2001-111910

However, in the information extracting apparatus according to the prior art, various and complicated character information (hereinafter referred to as "super") superimposed and displayed on the programs related to news reports and education can be accurately and efficiently grasped. There is a problem that it is not possible to accurately extract the items and the gist of the program.

SUMMARY OF THE INVENTION In view of the above circumstances, it is an object of the present invention to provide an information extracting device and an information extracting program capable of accurately and efficiently grasping supermarkets and accurately extracting items and gists in a program. .

An information extracting apparatus according to an embodiment comprises video data acquiring means for acquiring video data from moving images constituting a program, converting means for converting the video data into still images at predetermined time intervals, and text information acquisition means for recognizing character information and acquiring text information, wherein the text information acquisition means displays character information included in the still image at a predetermined position on a screen displaying the video data. summary character string extracting means for extracting the character string as character string information summarizing the contents of the television program; and information generating means.

Also, the predetermined position of the screen may include an upper right area of the screen.

Further, list information processing means for performing a predetermined process on the primary list information is further provided, and the list information processing means determines whether or not the character string exceeds a predetermined lower limit of the number of characters, If it is determined that the lower limit value is not exceeded, the same processing can be performed for the next character string.

Further, the list information processing means retains the character string as a comparison target 1 when it is determined that the lower limit value is exceeded, and then stores the character string as The comparison object 2 is held, and the comparison object 1 and the comparison object 2 are compared based on a predetermined algorithm to determine whether the character string of the comparison object 2 corresponds to the last character string in the primary list information. If it is determined that it corresponds to the last character string, it is determined whether or not a predetermined threshold is satisfied, and if it is determined that the predetermined threshold is satisfied, the last character string may be recorded as secondary list information.

Also, the predetermined algorithm can be an algorithm for comparing character strings based on the Levenshtein distance.

Further, when it is determined that the comparison object 2 is not the last character string in the primary list information and satisfies a preset threshold value, the list information processing means converts these character strings into the second list information. You may make it record as next list information.

In addition, the list information processing means determines whether this state is the first time or the second time or later when the number of consecutive times determined not to satisfy the threshold does not reach a predetermined specified number of times. If it is determined to be the first time, the character string of the comparison object 2 at this time is recorded in the primary list information as a candidate for the next comparison object 1, and if it is determined to be the second time or later, can be processed by using the following character string as the comparison object 2.

Further, when the number of consecutive times determined not to satisfy the threshold has reached a specified number of times set in advance, the list information processing means converts the character string being compared at this point into the secondary list information. It is possible to determine whether or not the recorded number of times has reached a specified number of times, and to record the character string in the program item list information when it is determined that the specified number of times has been reached.

Further, when recording the character strings in the program item list information, the list information processing means records the character string having the largest number of completely matching parts among the character strings recorded in the secondary list information as the program item list information. Can be adopted as a name.

An information extraction program according to another embodiment comprises a video data acquisition step of acquiring video data from a moving image that constitutes a program, a conversion step of converting the video data into still images at predetermined time intervals, and a text information step of recognizing included character information to obtain text information; a summary character string information extracting step of extracting character string information summarizing contents; a primary list information generating step of generating primary list information in which time information at which the character string is obtained is associated with the character string information; and the primary list and a list information processing step for performing predetermined processing on the information, which is executed by the information extraction device.

According to the present invention, there is provided an information extracting device and an information extracting program capable of accurately and efficiently grasping a super for a program relating to news reports or education, and accurately extracting items and gists in the program. be able to.

It is a functional block diagram showing the functional configuration of the information extraction device according to the embodiment. 4 is a flow chart showing a processing procedure of information extraction processing executed by the information extraction device according to the embodiment; 6 is a flow chart showing a processing procedure of list information processing executed by the information extraction device according to the embodiment; 7 is a flowchart showing the continuation of the processing procedure of list information processing executed by the information extraction device according to the embodiment; 7 is a flowchart showing the continuation of the processing procedure of list information processing executed by the information extraction device according to the embodiment; FIG. 4 is an explanatory diagram showing a display example by the information extraction device according to the embodiment; FIG. 7 is an explanatory diagram showing an example of an item extraction result by the information extraction device according to the embodiment;

BEST MODE FOR CARRYING OUT THE INVENTION An embodiment as an example of the present invention will be described in detail below with reference to the drawings. Here, in the accompanying drawings, the same members are denoted by the same reference numerals, and redundant description is omitted. It should be noted that the present invention is not limited to the best mode for carrying out the present invention, since the description here is the best mode for carrying out the present invention.

[Configuration example of information extraction device]
An information extraction device M1 according to an embodiment will be described with reference to FIG.

FIG. 1 is a functional block diagram showing the functional configuration of an information extraction device M1 according to this embodiment.

The information extraction device M1 is composed of, for example, a general-purpose computer, server, or the like.

Here, the information extracting device M1 is a device that converts the filed program video into still images at regular time intervals, uses OCR technology to detect characters from the images, and converts them into text. is.

Note that among the characters displayed in the image, the super displayed in the upper right part of the screen (also called the upper right side super) is often a character string that summarizes the content of the broadcast at that time. , the character strings displayed in this area and the time information at that time are put together into list information (primary list information to be described later) in chronological order for each program.

As shown in FIG. 1, the information extraction device M1 includes video data acquisition means 101 for acquiring video data D1 from moving images that constitute a program produced by a broadcaster B1.

The information extraction device M1 also includes conversion means 102 configured by software or hardware that converts the video data D1 into still images D2 at predetermined time intervals (for example, four seconds).

The information extraction device M1 also includes text information acquisition means 103 configured by software or hardware for recognizing the character information T1 included in the still image D2 and acquiring the text information T2.

Furthermore, the text information acquiring means 103 extracts the character information T1 included in the still image D2 from a predetermined position (for example, the upper right corner of the monitor screen 501 shown in FIG. 4) on which the video data D1 is displayed (for example, the monitor screen 501 shown in FIG. 4). A summary character string extracting means 104 configured by software or hardware is provided for extracting the character string T1 displayed in the area 501a (see FIG. 4) as character string information T3 summarizing the content of the television program.

The text information acquisition means 103 also includes a primary list information generation means 105 that generates primary list information L1 in which the time information at which the character string T1 is acquired is associated with the character string information T3.

The text information acquisition means 103 also includes a list information processing means 106 that performs predetermined processing on the primary list information L1.

The list information processing means 106 determines whether or not the character string T1 exceeds a preset lower limit U1 (for example, 10 characters).

Then, when it is determined that the lower limit value U1 is not exceeded, the same processing is performed on the next character string.

Further, the list information processing means 106 holds the character string as comparison object 1 when it is determined that the lower limit value U1 has been exceeded. It is held as comparison target 2.

Next, comparison object 1 and comparison object 2 are compared based on a predetermined algorithm, and it is determined whether or not the character string of comparison object 2 corresponds to the last character string in the primary list information L1.

Then, if it is determined that the character string corresponds to the last character string, it is determined whether or not a predetermined threshold value (an arbitrary value from 0 to 1 such as 0.5) is satisfied, and it is determined that the predetermined threshold value is satisfied. If so, the last character string is recorded as the secondary list information L2.

Note that the predetermined algorithm described above can be an algorithm that compares character strings based on the Levenshtein distance.

Here, the OCR technology, which is also applied to the information extraction device M1, causes erroneous recognition at a certain rate when analyzing a complicated language such as Japanese. Therefore, if the same character string is taken out from the primary list information as one in-program item, there is a risk that the erroneously recognized character string will become a different item. It may be divided into items. A similar phenomenon also occurs when a part of the character string is actually changed but the content is almost the same.

To avoid such events, some degree of ambiguity is required when comparing strings.

In order to compare character strings in consideration of such ambiguity, the concept of the Levenshtein distance is applied in the information extraction device M1 according to the present embodiment.

By applying the concept of the Levenshtein distance and comparing the strings in the list while allowing a certain amount of difference, the effects of misrecognition and changes in strings of only a few characters can be reduced and the intention can be improved. It is possible to create a list of items according to

Various OCR techniques using the concept of the Levenshtein distance have already been developed, and these existing OCR techniques can be appropriately selected and applied to character recognition of character strings in the information extraction device M1. .

If it is determined that the comparison object 2 is not the last character string in the primary list information L1 and satisfies a preset threshold value, the list information processing means 106 adds these character strings to the secondary list. It may be recorded as information L2.

Further, when the number of consecutive times determined not to satisfy the threshold does not reach the specified number of times set in advance, the list information processing means 106 determines whether this state is the first time or the second time or later. If it is determined to be the first time, the character string of the comparison object 2 at this time is recorded in the primary list information L1 as a candidate for the next comparison object 1, and if it is determined to be the second time or later, , and subsequent character strings can be processed as comparison target 2.

If the number of consecutive times determined not to satisfy the threshold has reached the specified number set in advance, the list information processing means 106 records the character string being compared at this point in the secondary list information L2. It is determined whether or not the number of times the program has been performed has reached a specified number of times, and if it is determined that the specified number of times has been reached, the character string can be recorded in the program item list information L3.

Also, when recording the character strings in the program item list information L3, the list information processing means 106 selects the character string having the largest number of completely matching parts among the character strings recorded in the secondary list information L2 as the program item list information L3. Can be adopted as a name.

The information extracting device M1 includes a recording means 107 composed of a hard disk drive or the like for storing data such as character string extraction results and program item names.

It also has a display means 500 comprising a liquid crystal monitor or the like for displaying the result of character string extraction, program item names, and the like.

A specific display example and the like will be described later.

[About information extraction processing]
A processing procedure of information extraction processing executed by the information extraction device M1 according to the present embodiment will be described with reference to the flowchart of FIG.

When this process is started, first, in step S101, the video data D1 is obtained from the moving image that constitutes the program produced and broadcast by the broadcaster B1, and the process proceeds to step S102.

In step S102, the video data D1 is converted into still images D2 at predetermined time intervals, and the process proceeds to step S103.

In step S103, for example, the character string T1 in the designated area 500a on the upper right of the screen is converted into text by image recognition, and the process proceeds to step S104.

In step S104, character string information summarizing the content of the program is extracted, and the process proceeds to step S105.

In step S105, the primary list information L1 is generated from the combination of the time information and the character string, and the process proceeds to step S106.

In step S106, a subroutine of list information processing, which will be described later, is executed, and in step S107, the predetermined character string is recorded in the program item list information L3, and the process ends.

Thus, program information can be efficiently managed by manipulating the moving image file and utilizing the program item list information L3.

[About list processing]
The processing procedure of the list information processing subroutine will be described with reference to the flow charts of FIGS. 3A to 3C.

In step S201, the generated primary list information L1 is acquired, and the process proceeds to step S202.

In step S202, the character string T1 is arranged in chronological order, and the character string detected first is sequentially processed.

In step S203, it is determined whether or not the number of characters of the character string T1 satisfies the set lower limit value U1.

If the determination result is "No", the process moves to step S204, and returns to step S201 to repeat the process until a character string that satisfies the lower limit value U1 is detected.

If the determination result is "Yes", the process proceeds to step S205.

In step S205, this character string T1 is taken as the first comparison object 1 to be compared by an algorithm using the Levenshtein distance, and the process proceeds to step S206.

In step S206, a character string that satisfies the lower limit value U1 of the number of characters after comparison object 1 is set as comparison object 2, and the process proceeds to step S207.

In step S207, comparison objects 1 and 2 are compared by the above-described algorithm using the Levenshtein distance, and the process proceeds to step S208.

In step S208, it is determined whether or not the primary list information L1 is processed last.

Then, if the determination result is "Yes", the process proceeds to step S209, and it is determined whether or not the comparison result of the character string satisfies the threshold value.

If the determination result is "No", the process proceeds to step S218. Also, if the determination result is "Yes", it is recorded in the secondary list information L2 in step S210, and then the process proceeds to step S218.

On the other hand, if the determination in step S208 is "No", the process proceeds to step S211.

In step S211, it is determined whether or not the character string comparison result satisfies a threshold value, and if "Yes", the process proceeds to step S212.

In step S212, it is recorded in the secondary list information L2, and the subsequent character string is set as comparison object 2, and the process returns to step S207.

On the other hand, if the determination in step S211 is "No", the process proceeds to step S213.

In step S13, it is determined whether or not the number of consecutive failures to satisfy the threshold has reached a designated number.

If the determination result is "Yes", the process proceeds to step S218, and if "No", the process proceeds to step S214.

In step S214, it is determined whether or not it is the first time when the threshold is not met. If the determination result is "No", the process proceeds to step S215, determines that it is the second or later character string, and returns to step S207 with the subsequent character strings as comparison target 2. FIG.

If "Yes", the process moves to step S216 to record the character string of the current comparison target 2 in the temporary storage list, and moves to step S217.

In step S217, the subsequent character string is set as comparison target 2 and the process returns to step S207.

On the other hand, in step S218, it is determined whether or not the number of times recorded in the secondary list information of the currently compared character string has reached the designated number.

If the determination result is "No", the process moves to step S219, discards these character strings without adopting them as program items, and moves to step S221.

If the determination result is "Yes", the process proceeds to step S220, and the program item list is recorded after a predetermined process (organization of information such as the start and end time of the corresponding program item) is added to the program item list. The process proceeds to step S221.

In step S221, the character string recorded in the temporary storage list is set as comparison target 1, and the process proceeds to step S222.

In step S222, it is determined whether or not it is the last process of the primary list information L1. If "No", the process proceeds to step S207, and if "Yes", the process ends.

Such processing makes it possible to accurately and efficiently grasp the super (character string) of, for example, a news report or a cultural program, and accurately extract the items and the gist of the program.

In particular, of the character strings recorded in the secondary list information L2, the character string with the greatest number of completely matching parts can be adopted as the program item name.

[Display example]
A display example of the display means (liquid crystal display device or the like) 500 of the information extraction device M1 will be described with reference to the explanatory diagram of FIG.

In FIG. 4, a moving image display area 501, a still image display area 502, a reproduction information display area 503, and a program information display area 504 are set near the left side of the screen of the display means 500. FIG.

A registered person display area 505 and a person information display area 506 are set near the center of the screen.

A detected character information display area 507 is set near the right side of the screen.

Then, in the moving image display area 501, the moving image (corresponding to the image data D1 acquired by the image data acquiring means 101) of the program whose displayed character string (super) is to be automatically extracted is displayed.

The example shown in FIG. 4 shows a case where one scene of a moving image of the program name "News N" is displayed. Here, the character string displayed in the upper right display area (also referred to as upper right side super) 501a of the moving image display area 501 is to be extracted.

In the example shown in FIG. 4, the upper right supermarket 501a displays "Breaking news today, 394 infected in Tokyo". Also, in the moving image display area 501, one person H is displayed.

In the still image display area 502, the still image data D2 converted from the video data D1 by the conversion means 102 is displayed. For convenience of explanation, in the example shown in FIG. 4, the images displayed in the moving image display area 501 and the still image display area 502 are assumed to be in the same state.

Then, in the still image display area 502, the face recognition processing of the person H is performed, and the person H is identified as, for example, the announcer H2. Note that various existing techniques can be applied to face recognition processing.

The identified announcer H2 is collated with the registered person data displayed in the registered person display area 505, and, for example, the face photograph of the announcer H2 is highlighted.

At the same time, in the person information display area 506, the person information regarding the announcer H2 is displayed.

In addition, the character string T1 of "Breaking news today, 394 people infected in Tokyo" displayed as the upper right side supermarket 501a is subject to item extraction.

Then, each means of the information extraction device M1 shown in FIG. 1 and the information extraction processing shown in the flow charts of FIGS. and the above-described processing is performed by the list information processing.

As a result, the character string "Tokyo 394 people" is extracted as the third item name 3 from the top of the detected character information display area 507, for example.

In addition, in the reproduction information display area 503, information such as the reproduction time of the moving image of the program name "News N" is displayed.

In addition, in the detected character information display area 507, by the same processing for the moving image of the program name "News N", for example, the item name 1 is "Y submits defendant's resignation request", and the item name 2 is "crisis about the Olympics". Examples of such extracted are shown.

As described above, according to the information extraction device M1 according to the present embodiment, for example, a super (character string) can be automatically and efficiently grasped for a program relating to news reports or lifestyle information, and an item in the program can be extracted. You can accurately extract the gist of the text.

[Example of item extraction results]
An example of an item candidate extraction result by the information extraction device M1 according to the present embodiment will be briefly described with reference to FIG.

The example shown in FIG. 5 shows correspondence between still images and item candidate extraction results.

That is, for example, still image 001 . jpg, the item candidate of "Preliminary report today Tokyo 394 people infected" is still image 002.jpg. jpg, the item candidate of "announced with 394 people in Tokyo" is the still image 003.jpg. jpg, the case where item candidates of "394 new infections in Tokyo" are extracted respectively is illustrated.

By performing the above-described processing on such an extraction result, it is possible to extract "Tokyo 394 people", which has a high frequency as a completely matching part, as an item name.

Thus, according to the information extraction device M1 according to the present embodiment, it is possible to reduce erroneous detection of character strings and select a more optimal item name.

Although the invention made by the present inventor has been specifically described based on the embodiments, the embodiments disclosed in this specification are illustrative in all respects and are not limited to the disclosed technology. should not be considered. That is, the technical scope of the present invention should not be construed in a restrictive manner based on the description of the above embodiments, but should be construed according to the description of the scope of claims. All changes that come within the scope of the technology described and equivalents and the claims are included.

For example, in the program item list, in addition to character strings, time information of the beginning and end of the secondary list information L2 can be recorded together. This allows you to record the start and end times of the item.

M1 Information extraction device 101 Video data acquisition means 102 Conversion means 103 Text information acquisition means 104 Summary character string extraction means 105 Primary list information generation means 106 List information processing means 500 Display means

Claims (3)

video data acquisition means for acquiring video data from moving images constituting a program;
conversion means for converting the video data into still images at predetermined time intervals;
text information acquisition means for acquiring text information by recognizing character information included in the still image;
with
The text information acquisition means is
summary character string extracting means for extracting a character string displayed at a predetermined position on a screen displaying the video data from the character information contained in the still image as character string information summarizing the content of the program ;
primary list information generating means for generating primary list information in which time information when the character string is obtained is associated with the character string information;
list information processing means for performing predetermined processing on the primary list information;
with
The primary list information is list information in which the character string and time information at the time when the character string was obtained are integrated in chronological order for each program,
The list information processing means is
Arranging the character strings in time series, determining whether or not the number of characters of the first detected character string satisfies a predetermined lower limit, and if the character string does not satisfy the lower limit, the character string satisfying the lower limit An information extracting apparatus characterized by changing the character string to be processed from the current character string to the next character string and repeatedly determining whether or not the number of characters satisfies the lower limit value until is detected. 2. The information extracting apparatus according to claim 1, wherein the predetermined position of the screen includes an upper right area of the screen. a video data obtaining step of obtaining video data from moving images constituting a program;
a conversion step of converting the video data into still images at predetermined time intervals;
a text information acquisition step of acquiring text information by recognizing character information included in the still image;
a summary character string information extracting step of extracting a character string displayed at a predetermined position on a screen displaying the video data from the character information contained in the still image as character string information summarizing the content of the program;
a primary list information generating step of generating primary list information in which time information at which the character string was obtained is associated with the character string information;
a list information processing step of performing a predetermined process on the primary list information;
has
The primary list information is list information in which the character string and time information at the time when the character string was obtained are integrated in chronological order for each program,
The list information processing step includes:
Arranging the character strings in time series, determining whether or not the number of characters of the first detected character string satisfies a predetermined lower limit, and if the character string does not satisfy the lower limit, the character string satisfying the lower limit until the information extraction device detects the An information extraction program characterized by:

Images