JP2007053604A - Video signal frame synchronization system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a video image frame synchronization system which is capable of performing frame synchronization without having to transmit video signals, while secondary data (side information), such as time codes, are superimposed on the video signals. <P>SOLUTION: Feature volume data extracted from video images of two families (systems A and B) are accumulated in the feature volume buffer (system A) 11, and the feature volume buffer (system B) 12 of an image quality monitoring terminal 6. A PSNR deducing unit 13 sets the image feature volume of a certain frame in one of the systems A and B as a reference, considers a frame delay difference between one and another frame as one frame in front and back, and obtains objective image qualities (Ppp, Ppc, Pcp, Pcc, Pnc, Pcn, and Pnn), when seven patterns of delay changes occur that are considered possible. A delay change detecting unit 14 decides the existence or nonexistence and the direction of delay change, and outputs a delay change volume determination value, based on the magnitude relation of the objective image quality on the seven patterns. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a frame synchronization system for video signals, and in particular, compensates for transmission delay of image feature data at two arbitrary points (for example, transmission point / reception point) on a transmission path, and enables comparison of both signal values. The present invention relates to a frame synchronization method for video signals.

  As a method for synchronizing frame information for quality evaluation of video signals for television transmission, information (time code) indicating correspondence between video data of each frame and reproduction time as in Patent Document 1 below. There is a method of obtaining signal synchronization of two systems by superimposing in a transmission signal and transmitting this information.

Further, as an application method of the method of Patent Document 1 to image quality evaluation / monitoring, there is a recommendation shown in Non-Patent Document 1. Non-Patent Document 1 is a recommendation indicating a framework for feature quantity extraction type transmission quality monitoring. Among these, image feature quantity data extracted from video data of each frame and frame playback time, or each frame A format in which information indicating a relationship with a uniquely assigned time code is separately transmitted over a data line is shown.
JP 2003-234726 A ITU-T Recommendation J.220 “Framework for remote monitoring of transmitted picture Signal-to-Noise ratio using spread-spectrum and orthogonal transform” (J.SSOT)

  When referring to the time code superimposed on the video signal by the method of Patent Document 1 and Non-Patent Document 1, it is ensured that the feature data of the same frame is the same time code. Reference can be made only when digital video signals for commercial broadcasting equipment are transmitted, and time codes cannot be used for analog signals and consumer video signals.

  Furthermore, even when such a commercial signal standard for broadcasting stations is applied, when transmission is accompanied by compression encoding, many transmission devices (codecs) erase the superimposed time code, so before and after compression. It was difficult to compare these images based on the time code.

  An object of the present invention is to provide a frame of a video signal that can perform frame synchronization without superimposing and transmitting secondary data (side information) such as a time code on the video signal in view of the above-described problems of the prior art. It is to provide a synchronization method.

  In order to achieve the above-described object, the present invention is based on image feature quantity extraction means for extracting an image feature quantity from each frame of two systems of video signals, and on the basis of the image feature quantity of a frame of one system. The objective image quality calculating means for obtaining the objective image quality of each frame based on the comparison of the image feature values of the two systems while sequentially changing the frame delay difference with the other frame, and the point where the objective image quality is maximized There is a first feature in that a delay difference detecting means for detecting as a delay difference is provided, and the delay difference is reduced to synchronize the frames.

  Further, the objective image quality calculation means sets the frame delay difference with the other frame as one frame before and after the image feature amount of a frame of the one system as a reference, and has seven possible delay change patterns. There is a second feature in that the objective image quality in the case of occurrence of the delay is obtained, and the delay difference detecting means determines the presence and the direction and the direction of the delay change based on the objective image quality relationship of the seven patterns.

  According to the present invention, even if side information such as time information and time code is not transmitted in addition to the feature amount data, only the objective image quality measurement based on the comparison of the feature amount data can be used to determine the video feature amounts of the two systems on the transmission path. Frame synchronization is possible. Further, since side information is not used, it is possible to measure image quality even for a video signal on which no time code is superimposed.

  In addition, since it is not necessary to send the time code and time information together with the feature amount extraction data, the scale of mounting the feature amount extraction apparatus is reduced. In addition, since the amount of information for the time code is reduced, the data bandwidth for image quality monitoring can be reduced.

  Hereinafter, the present invention will be described in detail with reference to the drawings. FIG. 1 is a system diagram showing the configuration of a feature quantity extraction type image quality monitoring system to which the frame synchronization system of the present invention is applied.

FIG. 1 assumes one-to-one video transmission, and compression encoding (encoder 1) and decoding (decoder 3) are applied before and after the transmission path 2. The image feature quantity extraction devices 4 and 5 are installed at the preceding stage A point of the encoding device (encoder 1) and the subsequent stage B point of the decoding device (decoder 3), respectively, and arbitrary bits from each pixel block of the baseband image. A number of image feature values F _A [i] and F _B [i] are extracted. The image feature values F _A [i] and F _B [i] extracted at the points A and B are transmitted to the image quality monitoring terminal 6 through the monitoring line 7 and then compared with each other to compare the objective image quality. Is measured. Then, frame synchronization is performed based on the objective image quality.

Another video transmission mode to which the present invention can be applied is one-to-many transmission as shown in FIG. The transmission image is compressed and encoded by the encoder 1 and then sent to the content distribution network 8. In this case, there are a plurality of receivers for one sender, and image feature amounts extraction 4 and 5 are performed at the subsequent points A and B of the respective decoding devices 3a and 3b on the receiving side. The extracted image feature quantities F _A [i] and F _B [i] are sent to the image quality monitoring terminal 6 via the monitoring line 7. The image quality monitoring terminal 6 compares the feature amounts F _A [i] and F _B [i]. Also in this case, the image feature amount extracted at each of the points A and B is transmitted to the monitoring point, and the objective image quality is derived based on the comparison of the feature amount there. Then, frame synchronization is performed based on the objective image quality.

In addition, as a specific method of the image feature quantity extraction method, Japanese Patent Laid-Open No. 2005-64679 “Image feature quantity extraction method and image quality evaluation method” can be cited. This publication discloses a feature quantity extraction method using spectrum spreading and orthogonal transformation. Specifically, if the image feature amounts in the same image frame extracted at points A and B on the transmission path are F _A [i] and F _B [i], the objective image quality PSNR of the frame is as follows ( 1) It is defined by the formula.

Here, i each pixel block in the frame (for example, 8 pixels × 8 lines) to an applied index number, N _B is the total number of pixel blocks in a frame. (1) In FIG. 1, a delay occurs in video signal transmission between points A and B. In FIG. 2, a delay occurs in video signal transmission between encoders 1-A and between encoders 1-B. 2) Especially in FIG. 1, compression coding is performed between A and B. (3) The feature data transmission delay from point A to image quality monitoring terminal 6 is different from point B to image quality monitoring terminal 6 For the reason, the feature amount data of the same frame generally does not arrive at the image quality monitoring terminal 6 at the same time. Therefore, the image quality monitoring terminal 6 must store the received feature data in the buffer in units of frames while distinguishing the transmission sources of the points A and B, and then detect which data in the buffer is for the same frame. I must.

  FIG. 3 is a block diagram of an embodiment of the image quality monitoring terminal device 6 according to the present invention. In FIG. 3, the same reference numerals as those in FIGS. 1 and 2 denote the same or equivalent components.

  As shown in the figure, the image quality monitoring terminal device 6 includes a feature amount buffer (system A) 11 for temporarily storing the feature amount of the input image extracted by the feature amount extraction device (system A) 4, and feature amount extraction. The feature quantity buffer (system B) 12 that temporarily stores the feature quantity of the input image extracted by the device (system B) 5, the feature quantity buffer (system A) 11, and the feature quantity buffer (system B) 12 are accumulated. PSNR estimation unit 13 serving as an objective image quality calculation means for obtaining an objective image quality from the previous frame feature value, the current frame feature value, and the subsequent frame feature value, and a delay difference (change) of the image frame from the PSNR estimate value. It comprises a delay change detecting unit 14 as a delay difference detecting means for detecting.

  Although details of the operation of the image quality monitoring terminal device 6 will be described later, a delay change amount determination value is output from the delay change detection unit 14 and the delay change amount determination value is stored in the feature amount buffer (system A) 11. And sent to the feature amount buffer (system B) 12. This is because, when a delay change (advance or delay of an image frame) is detected, the arrangement of the feature values of each frame stored in the buffer is adjusted (the feature data is discarded or the current frame data is used again). Etc.) because it is necessary.

  Now, the positional relationship of the same frame in the buffer does not change once it is detected, but it may change over time due to the effects of frame discard, repetition, return, etc. Each process is required. An example is shown in FIG.

  FIG. 4 shows frame information of two systems of video signals passing through the devices 1 and 2. It is assumed that the frames (Z, Z ′), (A, A ′), (B, B ′),. Here, consider a case in which the frame A is discarded only on the device 2 side, and then video feature values are extracted from the video signals of the respective systems. Then, even if the synchronization of the feature amount data can be acquired by some method up to the frame (Z, Z ′), the alignment of the feature amount data is generated from the subsequent frames, so the synchronization is lost. Therefore, it is necessary to detect synchronization for each frame so as to always cope with such inconsistencies in the frame order.

  In the present invention, attention is paid to the differential power (MSE) between two systems of video signals, and a set of feature quantities that minimizes this is detected as data of the same frame. This is based on the property that the difference power between frames becomes smaller as the frame interval becomes closer (of course, the difference is 0 in the same frame) when there is no scene change in the same scene. . Actually, the two systems of video are not completely the same signal due to processing conditions during transmission such as compression coding, so the difference between the video features of the same frame will not be 0, but still Since it is considered to have the above-mentioned basic properties, in the present invention, it is the same by selecting a set of video feature quantities having a smaller differential power (which can be paraphrased as a set having the maximum PSNR). Detect frames.

  In the following, an operation of the image quality monitoring terminal 6 of FIG.

  Feature quantity data extracted from video signals of two systems (systems A and B) is stored in the feature buffer (system A) 11 and the feature buffer (system B) 12 of the image quality monitoring terminal 6. Are separated and stored for each system without impairing the frame order. Also, a frame for which synchronization is determined is referred to as a “current frame”, a frame immediately before it is referred to as a “previous frame”, and a frame immediately after is referred to as a “back frame”. In addition, it is assumed that synchronization can be acquired up to the previous frame by an appropriate method.

  Here, the feature value data of the previous frame, current frame, and subsequent frame of the A system are represented as (Ap, Ac, An), and the B system is represented as (Bp, Bc, Bn). The PSNR value is described as shown in FIG. The PSNR value Ppn between the feature data AP of the previous frame of the A system and the feature data Bn of the subsequent frame of the B system, the feature data An of the subsequent frame of the A system, and the feature data Bn of the previous frame of the B system. Although the PSNR value Pnp can be defined, unlike the other PSNR values, the frame interval is a PSNR value in a state of 2 frames. Therefore, these values are not used in the present invention.

  The relationship between the deviation between Pxx and feature data is expressed as shown in FIG. Ppp, Pcc, and Pnn are all PSNR values when there is no frame shift. Ppc and Pcn are PSNR values assuming that the A system is delayed by one frame. PSNR value when In other words, Ppp, Pcc, and Pnn are respectively the previous one frame of one system and the other system, the current one frame of one system and the other system, the one frame of the other system, and the other system. The PSNR values of the next one frame, Pcp and Pcn are the PSNR values of the current frame of one system and the previous frame of the other system, the current frame of one system and the back frame of the other system, respectively. Ppc and Pnc can be said to be PSNR values of the previous frame of one system and the current frame of the other system, and the PSNR value of the subsequent frame of one system and the current frame of the other system, respectively.

  For the above-described reason, it is considered that the PSNR value in a state where synchronization is obtained is the highest. Therefore, the above-mentioned seven PSNR values may be compared to find a set of PSNR values showing the highest value. This procedure is shown in the flowchart of FIG.

  Since it is assumed in FIG. 7 that the synchronization of the previous frame is confirmed, Ppp which is a PSNR value obtained by comparing the feature quantities of the previous frames of the A and B systems shows a high value. Conceivable. Therefore, first, Pcc obtained by comparing the feature values of the current frames of the A and B systems and Pnn obtained from the feature values of the next frames are obtained, and the magnitude relationship between these and Ppp is obtained (step S1). . Since Pcc and Pnn are PSNR values assuming that there is no change in the delay amount, it can be determined that no frame delay occurs when these values are higher than Ppp.

  On the other hand, if Ppp is higher than Pcc or Pnn (Yes in step S1), the process proceeds to step S2 because there is a possibility of delay variation. In step S2, Pcn assuming that the A system has advanced by one frame is compared with Pnc assuming that the B system has advanced by one frame, and there is a possibility that the delay has moved to the one showing a high PSNR value. to decide. Next, Pcc or Pnc having a higher value is compared with Pcc (PSNR based on no delay) (steps S3 and S5).

  Here, when Pcc is higher than Pcn (No in step S3) or Pcc is higher than Pnc (No in step S5), Pcc is lower than Ppp, Since it can be determined that the delay is higher than the PSNR on the premise that the delay fluctuates, it is determined that the delay does not fluctuate, and the determination of the frame ends.

  Conversely, if the value of Pcn is higher than Pcc (Yes in step S3) or if the value of Pnc is higher than Pcc (Yes in step S5), there is a high possibility that a delay variation has occurred. Conceivable. Therefore, finally, a comparison is made between Pcp and Ppc assuming reverse delay in FIG. 6 or Pnc and Ppc assuming reverse delay (steps S4 and S6). If the value is higher (Yes in steps S4 and S6), it is finally determined that the delay has changed. On the other hand, if the value of Pcn or Pnc is lower (No in steps S4 and S6), the direction of the shift is indeterminate, so that there is no frame delay change in the sense of deferment. Even if the suspension is made, the processing of FIG. 7 is repeated every frame, so that no trouble occurs.

  As described above, if seven magnitudes of PSNR values obtained on the assumption of delay fluctuation are as assumed, fluctuation in delay is detected. In the present invention, it is assumed that a delay variation of a plurality of frames does not occur in one frame. However, even if a plurality of frames are shifted at a time, if the above procedure is repeated with a Kaku frame, each frame has a correct direction. Since the correction is performed frame by frame, the synchronization can be finally obtained. For example, when three frames are shifted at a time, it is possible to synchronize the correct feature amount data by performing the above determination for three frames in succession. Even if a scene change occurs and frame synchronization becomes impossible, the same scene that does not include the frame of the scene change will come, so the above correction will be applied after entering the same scene. Is possible.

It is a system diagram showing a configuration of a feature quantity extraction type image quality monitoring system to which the present invention is applied. It is a system diagram showing another configuration of a feature quantity extraction type image quality monitoring system to which the present invention is applied. FIG. 3 is a block diagram illustrating a specific configuration of the image quality monitoring terminal of FIGS. 1 and 2. FIG. 11 is an explanatory diagram of an example in which frame synchronization is lost due to frame discard in a video device. It is explanatory drawing of the PSNR value obtained by the combination of each front, present, and back frame of A and B systems. It is explanatory drawing which represented FIG. 5 in the time series. It is a flowchart explaining operation | movement of the delay change detection part of an image quality monitoring terminal.

Explanation of symbols

4, 5 ... feature quantity extraction device, 6 ... image quality monitoring terminal, 11, 12 ... feature quantity buffer, 13 ... PSNR estimation section, 14 ... delay change detection section.

Claims (4)

Image feature amount extraction means for extracting an image feature amount from each frame of two video signals;
Objective image quality for obtaining the objective image quality of each frame based on the comparison of the image feature values of the two systems while sequentially changing the frame delay difference with the other frame, based on the image feature value of a frame of one system Computing means;
A delay difference detecting means for detecting a point at which the objective image quality is maximized as a correct frame delay difference;
A frame synchronization method for video signals, characterized in that the delay difference is reduced to perform frame synchronization. The video signal frame synchronization method according to claim 1,
The objective image quality calculation means uses the image feature amount of a frame of one system as a reference, and sets a frame delay difference with the other frame as one frame before and after, and generates 7 possible delay change patterns. To obtain objective image quality
The video signal frame synchronization system, wherein the delay difference detecting means determines presence / absence and direction of delay change based on the objective image quality relationship of the seven patterns. The video signal frame synchronization system according to claim 1 or 2,
A video signal frame synchronization method using a PSNR value as a measure of the objective image quality. In the frame synchronization system of the video signal according to any one of claims 1 to 3,
A video signal characterized by using a PSNR value derived by estimating a differential power (MSE) between two systems of video signals using a method using spread spectrum and orthogonal transformation as the image feature extraction method. Frame synchronization method.

Images