JP2018156344A - Video stream consistency determination program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a video stream consistency determination program that compares a plurality of resource videos with on-air videos currently broadcasting and can determine the consistency between a resource video and a broadcast-using video even if there is a spatial difference or a temporal difference.SOLUTION: A computer extracts feature quantities of the latest predetermined number of frames of a resource video and the latest one frame of a broadcast-using video, calculates the degree of consistency between the latest predetermined number of frames of the resource video and the latest one frame of the broadcast-using video based on the extracted feature quantities, calculates a sum of degrees of consistency calculated during a period from the present time to a predetermined time before for each of the latest predetermined number of frames of the resource video, calculates an average by dividing the sum by a predetermined number, determines whether the average is equal to or less than a predetermined threshold, and when the average of the degrees of consistency is equal to or less than the predetermined threshold, determines that the resource video having the average of the degrees of consistency equal to or less than the threshold among the plurality of resource videos coincides with the broadcast-using video.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an apparatus for rapidly matching two video streams having spatial and temporal differences. More specifically, the present invention relates to a video stream matching determination program that can be used for managing the use status of various videos in a television broadcasting station.

  Conventionally, identification of resource (material) video used on-air for broadcasting in a television broadcasting station has been performed by adding a signal to an ancillary area of a video signal. However, this method requires a function of reading the signal in various devices in a television broadcasting station, and when this method is newly introduced, it is necessary to repair and update all existing devices. Therefore, in this method, the introduction cost becomes enormous when realizing the identification function of the resource video used on the air for broadcasting.

  When this is realized without using the ancillary area of the video signal, a method of capturing the resource video and the on-air video and comparing them by image processing can be considered. However, a character super is added to the on-air video, and there is a pattern that is further reduced and used as a part of the screen (spatial difference). In addition, an indefinite delay occurs between the resource video and the on-air video due to the digital video compression / decompression process (temporal difference). Therefore, when compared and realized by image processing, it is necessary to deal with both spatial differences and temporal differences.

  As a technique for comparing and judging the coincidence of images, there are methods such as Patent Document 1 and Patent Document 2. Patent Document 1 is a device that discriminates similar images, and aims to detect chapter switching and to divide chapters. Further, Patent Document 2 is a system that constructs a database by accumulating videos and searches for the corresponding video from the database. The technology of Patent Document 2 relates to a method for detecting whether a copy of an input video file has been illegally uploaded on the network. The video is handled as a file instead of a stream, and the search target is enormous. Thus, the prerequisites are different from the identification of resource (material) video that is used on-air for broadcasting in a television broadcasting station.

  As a method for comparing and determining the coincidence of video corresponding to a spatial difference, a method of performing detection using a feature amount has been conventionally used (for example, see Patent Document 3). However, this method alone cannot cope with the time difference.

  In addition, as a method for comparing and judging video coincidence corresponding to temporal differences, a video signal delay measurement method has been proposed (see, for example, Patent Documents 4 and 5).

International Publication No. 2009/031398 JP2011-237879A JP 2016-115226 A JP 11-346196 A JP 2005-33314 A

  However, in the method of Patent Document 4, since a delay amount is measured by adding a TS packet for delay time measurement to a video TS signal, a TS packet is added. There is also a problem that the input signal is limited to only the TS signal.

  The method of Patent Document 5 is a delay measurement device that does not require a signal generator or the like, but does not consider spatial differences, and thus handles both temporal differences and spatial differences. Can not.

  Therefore, by comparing multiple resource videos with videos currently being aired on the broadcast, even if there are spatial differences or temporal differences, the video stream that can determine the consistency between the resource video and the broadcast video An object is to provide a coincidence determination program.

  A video stream match determination program according to an embodiment of the present invention is a video stream match determination program for determining matching between a resource video and a broadcast-use video. The feature amount of the latest predetermined number of frames of the resource video and the latest one frame of the broadcast use video is extracted, and based on the feature amount, the latest predetermined number of frames of the resource video and the broadcast use The degree of coincidence with the latest one frame of the video is calculated, and for each of the latest predetermined number of frames of the resource video, the total degree of coincidence calculated from the present to a predetermined time before is calculated, and the total Is divided by the predetermined number to calculate an average value of the degree of coincidence, and whether the average value of the degree of coincidence is a predetermined threshold value or less. And when the average value of the degree of coincidence is equal to or less than a predetermined threshold, among the plurality of resource videos, it is determined that the resource video having the average value of the degree of coincidence equal to or less than the threshold matches the broadcast use video. To do.

  Video stream match determination that can compare resource video and broadcast-use video even if there is a spatial difference or temporal difference by comparing multiple resource videos with videos that are currently on air A program can be provided.

1 is a diagram illustrating a video processing system 10 including a computer in which a video stream matching determination program according to an embodiment is installed. FIG. It is a functional block of the CPU 120 and the flow of data. It is a flowchart which shows the process performed by CPU120. It is a figure which shows the high-speed matching determination process of the video stream performed by CPU120.

  An embodiment to which the video stream matching determination program of the present invention is applied will be described below.

<Embodiment>
FIG. 1 is a diagram illustrating a video processing system 10 including a computer in which a video stream matching determination program according to an embodiment is installed.

  The video processing system 10 includes a PC (Personal Computer) 100, an inserter 150, and a speaker 160. The video processing system 10 is deployed, for example, in a television broadcasting station (key station) having a broadcasting area in the whole country, a local station having a broadcasting area in a part of the country. Here, a form deployed in a local station will be described. Hereinafter, unless otherwise specified, the local station refers to a broadcasting station in which the video processing system 10 is provided.

  The PC 100 includes video input terminals 101A and 101B, a synchronization signal input terminal 102, a video output terminal 103, an audio output terminal 104, an HD (High Definition) -SDI (Serial Digital Interface) input / output board 110, and a CPU (Central Processing Unit) 120. And a memory 130.

  Resource video and broadcast use video are input to the video input terminals 101A and 101B, respectively.

  The resource video is a content (program) of video (with audio) that can be broadcast (on-air) at that point in time at the local station where the video processing system 10 is deployed. Here, as an example, one screen is divided into 16 Then, 16 resource videos are displayed. The 16 resource videos include, for example, video of each program such as a news program produced by the local station, video video of a program held by the local station, weather camera video held by the local station or key station, key station There is a relay video from. The resource video is output from a multi-viewer device (not shown) included in the video processing system 10 and input to the video input terminal 101A.

  The broadcast use video is a video of one program selected from 16 resource videos by the local station and on-air. The broadcast use video is a video that is selected and output as a broadcast use video from the 16 resource videos by the HD-SDI input / output board 110. A broadcast use video to be compared is input to the video input terminal 101B. For example, the broadcast program sent from the key station to the local station, the broadcast program sent from the local station to the radio tower, the video obtained by decoding the broadcast wave received by the local station antenna, etc. are input. .

  For this reason, the broadcast use video input to the video input terminal 101B is delayed from the resource video output from the multi-viewer device and input to the video input terminal 101A, and there is a temporal difference with respect to the resource video. Have. In addition, while the broadcast use video is displayed on one screen, the resource video is displayed on one section of the 16 screens, and thus the number of pixels is different. For this reason, there is a spatial difference between the broadcast use video and the resource video. Further, for example, when the broadcast use video is 4K or 8K, the resource video is not 4K or 8K, and this also causes a spatial difference between the broadcast use video and the resource video.

  In addition, there may be a spatial difference between the broadcast video and the resource video even when the resource video is reduced and used as a part of the video. For example, a pattern in which a small rectangular screen is displayed beside an announcer's video in a studio, and a weather camera video is synthesized in the screen is applicable. Furthermore, a character super or the like may be added on the resource video, and even in such a case, a spatial difference occurs between the broadcast use video and the resource video.

  The synchronization signal input terminal 102 is a terminal to which a synchronization signal used by the HD-SDI input / output board 110 for various synchronization processes is input.

  The video output terminal 103 outputs a tally signal representing a tally video. A tally is an on-air resource out of 16 resource videos in one screen in order to indicate a resource video of one program selected from the 16 resource videos at the local station and being aired. It is a red frame displayed to surround the video. The tally signal is a signal representing an image of a frame displayed in red on an on-air video image in which only one of 16 resource videos in one screen is selected in a black background. In FIG. 1, the tally is shown in white. In actual operation, there are cases where the same video is selected for 16 resources, so that the displayed tally is not limited to one. Specifically, in the 16-divided resource video, there may be a video with a large delay transmitted through the IP system in addition to a video with a small delay transmitted through the dedicated line. Two images are taken by the same camera, but when the transmission route is different and the delay values are different, a plurality of tally are displayed simultaneously.

  The audio output terminal 104 is a terminal that outputs an audio signal indicating, for example, that the resource video is on the air. Such an audio signal is generated by the CPU 120.

  The CPU 120 has a video stream matching determination program installed therein, and executes the video stream matching determination program. As a result, the PC 100 functions as a high-speed match determination device for video streams.

  The memory 130 represents a RAM (Random Access Memory), a ROM (Read Only Memory), a hard disk, and the like included in the PC 100 as one block.

  The inserter 150 receives a resource video and a tally signal, and outputs a video obtained by adding (super) a tally image represented by the tally signal to the resource video. As a simpler method, the PC 100 may output the resource video in a synthesized state. This eliminates the need for an inserter, reduces the hardware scale, and reduces the installation cost. However, since the video composition / output processing increases, the speed of the main image processing of the PC is somewhat reduced. Therefore, a method in which the user selects and uses these two methods (referred to as inserter mode / tally synthesis mode) is desirable.

  The speaker 160 converts the sound signal output from the CPU 120 into sound and outputs the sound.

  FIG. 2 is a diagram illustrating a functional block of the CPU 120 and a data flow. The CPU 120 implements the function shown in FIG. 2 by executing a video stream matching determination program.

  The CPU 120 includes a video reading unit 121, a feature amount extraction unit 122, a feature amount comparison unit 123, a delay amount estimation unit 124, a match determination unit 125, and a tally control unit 126. 2 shows a video composition unit 151 included in the inserter 150 as an external component of the CPU 120, and resource data, broadcast use video, video with tally, and audio alarm as data.

  Here, the function of each unit will be described with reference to FIG. FIG. 3 is a flowchart showing processing executed by the CPU 120.

  The CPU 120 first performs initialization (step S1). This processing is performed by the main control unit of the CPU 120 that performs processing other than the processing performed by the video reading unit 121, the feature amount extraction unit 122, the feature amount comparison unit 123, the delay amount estimation unit 124, the coincidence determination unit 125, and the tally control unit 126. Do.

  In this initialization, the array and variables to be handled in the process are initialized, and various parameters set are read. For example, in the matching cost matrix, all elements are initially initialized to a maximum value. In addition, preparation for processing is performed such as securing a memory based on parameters specified by the user such as the number of frames to be compared.

  The resource video and the broadcast use video are input to the video reading unit 121 by the main control unit. The resource video is also input to the video composition unit 151 via a video distributor (not shown).

  The video reading unit 121 reads the image of the latest frame so that the two video streams of the resource video and the broadcast use video can be captured and handled as images (step S2). At that time, the resource video is captured in accordance with the coordinate position designated by the user, and is captured by being divided into a plurality of resource videos before being synthesized by the multi-viewer device. The capture is performed at a frame rate specified by the user, and frames captured for the number of frames specified by the user are stored in the memory 130.

  The feature amount extraction unit 122 performs image processing on the image of the latest frame read by the video reading unit 121 and extracts the feature amount (step S3). The feature amount extraction unit 122 stores data representing the extracted feature amount in the memory 130.

  The feature amount comparison unit 123 compares the feature amount of the latest frame of the broadcast use video extracted by the feature amount extraction unit 122 with the feature amount of the past frame (about 2 to 3 seconds) from the latest frame of the resource video. Then, a matching process between feature quantities is performed (step S4).

  As an algorithm for the matching process, for example, an ORB ("An efficient alternative to SIFT or SURF", Ethan Rublee et al., IEEE International Conference on Computer Vision, Nov. 2011) is used as an existing feature extraction method. it can. The matching result obtained here is used as a matching cost. The matching cost represents the degree of matching between feature quantities, and the lower the matching cost, the higher the degree of matching.

  Specifically, the matching cost is obtained as 1− (number of feature points determined to be a match) / (total number of feature points of resource video). Thereby, the matching cost is represented by a decimal number between 0 and 1, and is 0 when all the feature points match, and 1 when they do not match at all. In addition, when matching is performed between feature points, erroneous detection may occur due to outliers (noise in data). Therefore, it is desirable to suppress the influence of outliers by applying outlier processing called k-nearest neighbor method or RANSAC.

  The delay amount estimation unit 124 obtains an average value of matching costs calculated by the feature amount comparison unit 123. The average value is an average value for the past predetermined number of frames for the latest frame of the resource video, an average value for the predetermined number of past frames for the latest previous frame of the resource video,... Is the average value for the past predetermined number of frames for the latest six previous frames. The predetermined number of frames in the past is, for example, 7 frames.

  The details of this calculation method will be described later with reference to FIG. 4. The average value from the latest frame of the resource video to the latest six previous frames of the resource video is calculated using the broadcast video for the resource video. This is to hit (estimate) the delay time of up to 7 frames.

  Then, the delay amount estimation unit 124 obtains the minimum value of the seven average values from the average value for the latest frame of the resource video to the average value for the latest six previous frames of the resource video ( Step S5). This is because the time difference between the frame that gives the minimum value among the seven average values and the latest frame is estimated to be the delay time of the broadcast use video with respect to the resource video. The reason for this will be described later with reference to FIG.

  The coincidence determination unit 125 determines whether the minimum value of the average value obtained by the delay amount estimation unit 124 in step S5 is below a threshold set by the user, so that the broadcast use video and the resource video match. It is determined whether or not (step S6).

  The coincidence determination unit 125, when the minimum value among the average values obtained in step S5 by the delay amount estimation unit 124 is below the threshold (the broadcast use video and the resource video match), Is determined to be used on-air as a broadcast. In this case, the flow proceeds to step S7A.

  On the other hand, the coincidence determination unit 125, if the minimum value of the average values obtained in step S5 by the delay amount estimation unit 124 is not less than the threshold value (broadcast video and resource video do not match), This resource video is determined not to be used on-air as a broadcast. In this case, the flow proceeds to step S7B.

  The tally control unit 126 outputs a tally signal for displaying the tally and an audio signal (step S7A). When the match determination unit 125 determines that the broadcast-use video and the resource video match, the tally control unit 126 determines that the broadcast-use video matches the display of one screen including 16 resource videos. The tally signal in which the tally is displayed in the portion representing the resource video thus output is output to the video composition unit 151, and the audio signal “the resource video has been on air” is output to the speaker 160.

  The tally control unit 126 outputs a tally signal that hides the tally (step S7B). When the match determination unit 125 determines that the broadcast use video and the resource video do not match, the tally control unit 126 outputs a tally signal in which the tally is not displayed to the video synthesis unit 151. In this case, an audio signal “resource video is not on the air” may be output to the speaker 160.

  When the processing of step S7A or S7B is completed, the main control unit stores a log indicating the start time and end time when the resource video is used as air as a broadcast, the resource name, and the frame of the broadcast use video when it is on air. 130 is written (step S8). Based on these data, a resource operation log is output at an arbitrary timing required by the user. When the main control unit finishes the process of step S8, the flow returns to step S1.

  FIG. 4 is a diagram illustrating a high-speed matching determination process for video streams executed by the CPU 120. FIG. 4A shows 20 frames A1 to A15 and A11 to A15 of the resource video, and 20 frames B1 to B7, A5 to A11, and C1 to C6 of the broadcast use video.

  The resource video shown in FIGS. 4A to 4E is one of 16 resource videos in one screen. Here, the processing described with reference to FIG. 4 is similarly performed in parallel for each of the 16 resource videos included in one resource video and the broadcast use video.

  In FIG. 4A, the horizontal axis is time, and the left frame becomes a past frame, and the right axis becomes a new frame. For this reason, in the state shown as an example in FIG. 4A, the rightmost frame A20 of the resource video is the latest (current) frame, and the rightmost frame C6 of the broadcast-use video is the latest (current) frame. .

  The alphabets of the frames A1 to A20 and the frames B1 to B7, A5 to A11, and C1 to C6 represent the types of contents (programs). Alphabet suffixes indicate the order in the time direction, with smaller numbers indicating older and larger numbers indicating newer.

  Therefore, the frames A1 to A20 of the resource video indicate 20 frames A1 to A20 that are continuous in time series of one type of the same content (program) A.

  Also, the frames B1 to B7, A5 to A11, and C1 to C6 of the broadcast use video broadcast the content (program) A after broadcasting seven frames B1 to B7 that are continuous in time series of the content (program) B. This shows that seven frames A5 to A11 continuous in time series are broadcasted, and then six frames C1 to C6 continuous in time series of content (program) C are broadcast.

  Also, in FIG. 4A, numbers 1 to 20 are assigned from the past frame to the latest frame in order to show the order of 20 of the resource video and the broadcast use video.

  Here, when the resource video and the broadcast use video shown in FIG. 4A are focused on the frame A5, it can be seen that the broadcast use video is delayed by three frames compared to the resource video. As described above, this is because the local station broadcasts the broadcast use video from the radio tower and inputs the broadcast use video received by the antenna of the local station to the video input terminal 101B. It is a delay.

  In the case of operation at a local station, a broadcast program sent from a key station can be used for broadcast use video. Further, by detecting the broadcast program sent from the key station in this way, it is possible to detect the video of the local station used in the key station. In the case of this method, it is possible to detect even in a situation where the resource video possessed by the local station is actually on-air for broadcasting, and another resource video possessed by the local station is used by the key station side.

  In addition, when it is desired to detect the video used at the local station, the video signal sent to the radio tower can be used as the broadcast video. Thereby, the delay between the resource video and the broadcast use video can be reduced. However, since the encoding / decoding process is often performed on the key station side or the local station side, the delay does not become zero.

  In this way, when the broadcast video is delayed with respect to the resource video, the identity of the frame of the resource video and the frame of the broadcast video is determined as follows.

  4B to 4E show seven consecutive frames (that is, the latest seven frames) from the latest frame of the resource video to the seven previous frames in the horizontal axis direction, and the left side is the left side. This is the latest resource video frame, and the right side is a past (older) resource video frame. In addition, in the vertical axis direction, seven consecutive frames (that is, the latest seven frames) from the latest frame of the broadcast use video to the previous seven frames are shown, and the upper side is the latest broadcast use video frame. The lower side is a frame of a past (older) broadcast use video.

  The latest seven frames of resource video and broadcast-use video (seven consecutive frames from the latest frame to the previous seven frames) are replaced one by one each time the frames are switched over time. .

  In addition, 49 matching costs between each frame of the resource video and the broadcast use video are shown in a matrix. Each time a frame is switched over time, seven matching costs are obtained for the latest broadcast-use video and the latest seven resource videos, and values are stored in the top row of 49 matrices. Then, every time the latest seven frames of the resource video and the broadcast use video are replaced one by one with the passage of time, the average value of the seven shifts downward by one row.

  Therefore, the top row of the 49 matrices is the seven matching costs obtained for the latest broadcast use video and the latest seven resource videos. Of the 49 matrices, the one row below the top row is the seven matching costs obtained for the latest broadcast use video and the latest seven resource videos at the time one frame before. Similarly, in the 49 matrixes, 6 rows below the top row (bottom row) are obtained for the latest broadcast use video and the latest 7 resource videos at the time of 6 frames before. This is the matching cost of the piece.

  Further, below the 49 matrixes, the total value and the average value of the matching costs of each frame of the resource video and the seven frames of the broadcast use video are shown. That is, the average value of each matching cost is the seven matching costs obtained for the latest one frame of the broadcast use video for each of the latest seven frames of the resource video in each cycle in which the frames are switched. The average value of (seven matching costs arranged vertically) is shown.

  Here, using the average value of such matching costs, it is determined whether the 16 resource videos having spatial differences and temporal differences match the broadcast use videos. Here, one of 16 processes performed in parallel will be described.

  In the following, it is assumed that the minimum value of the matching cost is 0 and the maximum value of the matching cost is 1. In addition, the threshold value of the average value of matching costs when determining whether or not the broadcast use video and the resource video match in step S6 of FIG. 3 is 0.7 as an example. Such a threshold value may be determined to an optimum value through experiments or the like.

  First, FIG. 4 (B) shows the latest broadcast-use video, the latest broadcast-use video obtained at the time when each frame was the latest frame for the seven frames from the latest frame of the broadcast-use video to six frames before. 7 matching costs obtained for the 7 resource videos. In FIG. 4B, as an example, frames A1 to A7 and B1 to B7 of resource video and broadcast use video are shown.

  The 49 matching cost values shown in FIG. 4B are obtained by the processing of steps S3 to S5 when the CPU 120 repeats the processing of steps S2 to S8 shown in FIG. 3 seven times.

  In the example of FIG. 4B, all 49 matching costs are 1.0. For this reason, the average value of seven matching costs obtained by the CPU 120 is 1.0. This corresponds to the process of step S5 shown in FIG. Here, since the average value of the seven matching costs is 1.0, the minimum value is 1.0. Since this value is larger than the threshold (0.7), the CPU 120 determines NO in step S6, hides the tally in step S7B, writes the log in step S8, and returns the flow to step S2.

  Next, the CPU 120 executes the process of step S2 to acquire the latest frames of the broadcast video and resource video, and as shown in FIG. Are updated one by one, and processing is performed for frames A2 to A8 and frames B2 to B7 and A5. And CPU120 calculates | requires seven matching costs according to the process of step S3-S5 about the part of the newest frame of a broadcast utilization image | video and a resource image | video.

  In the example of FIG. 4C, the matching cost (see the top row) between the resource video frames A2 to A8 and the broadcast-use video frame A5 is lower than 1.0. The matching cost between the frame A5 of the resource video and the frame A5 of the broadcast video is 0, which is the minimum value. This is the matching cost between the frame A5 of the resource video and the frame A5 of the broadcast video, and there are differences in pixels and scales (spatial differences) and delays in the broadcast video with respect to the resource videos (temporal differences). Even if it exists, since the same image | video is compared fundamentally, the matching cost is the minimum.

  4C, the average value of the seven matching costs obtained by the process of step S5 executed by the CPU 120 is 0.90, 0.89, 0.87, 0.86, 0. 87, 0.89, and 0.90. That is, the minimum value is 0.86.

  In FIG. 4C, since the latest frame A5 of the broadcast use video is delayed by 3 frames with respect to the frame A5 of the resource video, the delay time is estimated to be 3 frames.

  However, since the minimum value of the average value of matching costs is larger than the threshold (0.7), the CPU 120 determines NO in step S6, hides the tally in step S7B, writes the log in step S8, and the flow. To step S2.

  Next, the CPU 120 executes the process of step S2 to acquire the latest frames of the broadcast video and resource video, and as shown in FIG. Are updated one by one, and processing is performed for frames A3 to A9 and frames B3 to B7, A5, and A6. And CPU120 calculates | requires seven matching costs according to the process of step S3-S5 about the part of the newest frame of a broadcast utilization image | video and a resource image | video.

  In the example of FIG. 4D, the matching cost between the latest seven frames A3 to A9 of the resource video and the latest frame A6 of the broadcast use video is lower than 1.0. The matching cost between the frame A6 of the resource video and the frame A6 of the broadcast video is 0, which is the minimum value. Even if there is a spatial difference and a temporal difference, the matching costs are minimized because basically the same images are compared.

  4D, the average value of seven matching costs obtained by the process of step S5 executed by the CPU 120 is 0.76, 0.74, 0.73, 0.71, 0,. 73, 0.74, and 0.76. That is, the minimum value is 0.71.

  In FIG. 4C, since the latest frame A6 of the broadcast use video is delayed by 3 frames with respect to the frame A6 of the resource video, the delay time is estimated to be 3 frames.

  However, since the minimum value of the matching cost is larger than the threshold (0.7), the CPU 120 determines NO in step S6, hides the tally in step S7B, writes the log in step S8, and flows the flow to step S2. Return to

  Thereafter, the CPU 120 repeats the processes of steps S2 to S8, thereby repeatedly updating the latest seven frames of the resource video and the broadcast use video one by one. In FIG. 4E, a case where processing is performed for frames A9 to A15, frames A6 to A11, and C11 will be described.

  The CPU 120 executes the process of step S2 to acquire the latest frames of the broadcast use video and the resource video, thereby processing the frames A9 to A15, the frames A6 to A11, and C11 as shown in FIG. I do. And CPU120 calculates | requires the average value of seven matching cost according to the process of step S3-S5 about the newest frame of a broadcast utilization image | video and the latest seven frame of a resource image | video.

  In the example of FIG. 4E, the matching cost between the latest frame C1 of the broadcast video and the latest seven frames A15 to A9 of the resource video is 1.0.

  In FIG. 4E, the average value of seven matching costs obtained by the process of step S5 executed by the CPU 120 is 0.40, 0.31, 0.23, 0.14,. 23, 0.31, and 0.40. That is, the minimum value is 0.14.

  Since the minimum value of the matching cost is equal to or less than the threshold (0.7), the CPU 120 determines YES in step S6 (broadcast video and resource video match), displays a tally in step S7A, and performs step S8. The log is written in, and the flow returns to step S2.

  Here, one of the 16 processes performed in parallel for each of the 16 resource videos included in one screen and the broadcast use video has been described, but in step S7A, the broadcast use video and A tally red frame is displayed at the coordinates at which the resource video determined to match is displayed in a one-screen display including 16 resource videos.

  As a result, the user can recognize which of the 16 resource videos is the on-air resource video.

  As described above, according to the embodiment, the matching cost between the latest one broadcast use video frame and the latest seven resource video frames is obtained, and the average value of the matching costs over the past seven frames is obtained. Then, each of the resource video from the latest frame to the 6th previous frame is obtained.

  Then, when the minimum value of the seven average values of the matching costs is equal to or less than the threshold value, it is determined that the resource video is on air, and a tally is displayed. This is because the resource video that gives the minimum of the seven average values of the matching cost has the average value obtained for the frame corresponding to the time estimated to be the delay time of the broadcast video for the resource video.

  Therefore, by comparing multiple resource videos and videos currently on the air, even if there are spatial differences or temporal differences, the video stream that can determine the consistency between the resource video and the broadcast video A coincidence determination program can be provided.

  In the above description, the mode in which the resource video includes 16 resource videos in one screen has been described. However, the number of resource videos included in one screen is not limited to 16. For example, the number may be four or one.

  Further, instead of outputting the audio signal to the speaker 160, the audio signal may be provided to a relay site or the like via a digital hybrid and a telephone line. In this case, it is possible to grasp that the video at the relay site is on air at the relay site or the like. The digital hybrid is sometimes referred to as a telephone hybrid.

  In the above description, the mode in which the video processing system 10 is deployed in a local station has been described. In the local station, basically, a broadcast program transmitted from the key station and a broadcast program created in the local station are selectively transmitted and broadcast. The local station's weather camera and other resource images are sent to the key station via an optical line (private line) or IP line. Therefore, the key station can use the resource video held by the local station on the broadcast program. However, the local station side does not know at what timing the key station side uses the local station side resource video. Therefore, it is possible to confirm that the key station has used the resources on the local station side by comparing the broadcast use video and the resource video.

  On the other hand, considering the case of operation on the key station side, the key station side has its own information on which resource video is used. Therefore, unlike a local station, visualization (attaching a tally) of a currently used video can be easily performed to some extent. However, according to the present invention, it is possible to perform a match determination by simply comparing two video streams without using such resource video usage information. Therefore, the scale of the system is small and the threshold for introduction is considered to be low, and the system is suitable not only for local stations but also for key stations.

  Although the video stream matching determination program of the exemplary embodiment of the present invention has been described above, the present invention is not limited to the specifically disclosed embodiment, and departs from the scope of the claims. Various modifications and changes can be made without this.

10 Video processing system 100 PC
101A, 101B Video input terminal 102 Synchronization signal input terminal 103 Video output terminal 104 Audio output terminal 110 HD-SDI input / output board 120 CPU
121 video reading unit 122 feature amount extraction unit 123 feature amount comparison unit 124 delay amount estimation unit 125 coincidence determination unit 126 tally control unit 130 memory 150 inserter 151 video synthesis unit 160 speaker

Claims (3)

A video stream match determination program for determining matching between resource video and broadcast use video,
Computer
For each resource video of a plurality of resource videos, the feature amount of the latest predetermined number of frames of the resource video and the latest one frame of the broadcast use video is extracted,
Based on the feature amount, the degree of coincidence between the latest predetermined number of frames of the resource video and the latest one frame of the broadcast video is calculated,
For each of the latest predetermined number of frames of the resource video, the total degree of coincidence calculated from the present to a predetermined time before is calculated,
By calculating the average value of the degree of coincidence by dividing the total by the predetermined number,
Determining whether the average value of the degree of coincidence is a predetermined threshold value or less;
A video stream that, when the average value of the degree of coincidence is equal to or less than a predetermined threshold, determines that, among the plurality of resource videos, the resource video having the average value of the degree of coincidence equal to or less than the threshold matches a broadcast video. Match determination program. The plurality of resource videos are displayed in a format in which one screen is divided,
2. The video stream matching according to claim 1, wherein when it is determined that the resource video and the broadcast use video match, a tally is displayed at a position of the one screen where the resource video determined to match is displayed. Judgment program.   3. The video stream matching determination program according to claim 1, wherein, when it is determined that the resource video matches the broadcast use video, an audio guidance indicating that the resource video is on-air is output.