KR101721224B1 - Apparatus and method for detecting real-time video and audio distortion - Google Patents

Abstract

A process of separating a digital broadcast signal or a multimedia content into video and audio signals, a process of inserting a watermark signal of the same intensity for each block into video and audio, a process of outputting a watermark signal embedded video and audio to a test device A process of capturing a video and audio output in which a watermark signal output from the testing device is inserted, and a process of detecting the presence or absence of a watermark in the video and audio by detecting a watermark signal from the captured video and audio.

Description

[0001] The present invention relates to a real-time video and audio distortion detection method and apparatus,

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a video and audio distortion detection system for a digital multimedia device, and more particularly to a system and method for detecting video and audio distortion in a digital multimedia device Real-time image and speech distortion detection method and apparatus.

In general, digital multimedia devices such as video, BD / DVD player, digital TV, digital set-top box, and DMB test product abnormality and image quality after the manufacturing process is completed.

Such a test is performed by subjectively evaluating several evaluation items by the inspector's subject while watching the test image familiar to him / her.

At this time, the examinee evaluates evaluation items such as sharpness of image quality, contrast, brightness, color reproduction, and noise. Such a test method has a problem that the evaluation standard is changed according to the examiner because it is manually tested by a person .

In addition, there is a difference in evaluation results depending on various viewing environments that affect image quality, and it takes a long time to perform image quality test.

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and it is an object of the present invention to provide a digital watermarking method and a digital watermarking method capable of detecting the loss and distortion of video and audio data reproduced in a digital multimedia device in real time, A method and an apparatus are provided.

According to an aspect of the present invention, there is provided a method for detecting image and speech distortion,

Separating a digital broadcast stream or a multimedia content into video and audio signals;

Inserting a watermark signal having the same intensity for each block into the video and audio signals;

Outputting the video and audio signal in which the watermark signal is embedded to a test apparatus;

Capturing the video and audio output in which the watermark signal output from the testing apparatus is embedded;

Detecting a watermark signal from the captured video and audio, and determining whether or not a watermark is present in the video and audio.

And storing the separated video and audio signals in a separate storage area in a file unit.

And the watermark signal is inserted in the same block by block for the video and audio to be amplified or modified.

The watermark signal includes an image watermark signal inserted into the image signal on a block-by-block basis and a voice watermark signal inserted on a time block basis with respect to the voice signal.

The watermark embedding process

A block size and a block time for determining image and speech distortion in a video and audio signal to be watermarked;

A step of forming a plurality of video and audio signals into blocks with a predetermined block size;

And inserting the video watermark signal and the audio watermark signal of the same intensity for each block of the video and audio signal according to the entire watermark embedding time and the divided embedding time.

And the block size and the block time for the image and speech distortion determination are adjusted according to the type of the device under test.

The method of claim 1,

Detecting a corresponding watermark on a block-by-block basis for captured video and audio signals;

And analyzing the watermark of each of the detected blocks to determine whether there is an abnormality of the watermark.

And reporting the analysis result of the video and audio.

The reporting process

And the video block or the audio block including the watermark having the distortion according to the time setting of the user is stored for the preset time.

According to another aspect of the present invention, there is provided an image and speech distortion detection apparatus,

A signal processor for separating digital broadcasting signals or multimedia contents into video and audio signals and storing the separated video and audio signals in a corresponding storage area;

A watermark generating unit for generating video and audio watermark;

A watermark embedding scheduler for inserting the video and audio watermarks of the same intensity for each block into the reference video and audio signals stored in the signal processor;

An output scheduler for outputting video and audio in which the video and audio watermark is embedded in the watermark embedding scheduler to a testing apparatus;

A signal capturing unit capturing video and audio output from the test apparatus;

And a distortion analyzer for detecting video and audio watermark from the captured video and audio to determine whether or not a watermark is present in the video and audio.

The signal processing unit

A signal separator for separating a digital broadcast signal or a multimedia content into video and audio signals;

And a frame storage unit for storing the video and audio signals separated by the signal separation unit in the respective storage areas.

The watermark generating unit

And a robust watermark is generated for amplification and modification of video and audio, which are unique functions of a multimedia device.

The watermark embedding scheduler

A blocking unit for blocking a plurality of blocks with a block size and a block time set for video and audio signals;

A controller for adjusting the block size and the block time according to a defect margin;

And a watermark inserting unit for inserting the video and audio watermark into the video and audio blocks blocked in the blocking unit according to the block size and the block time adjusted by the controller.

The signal capture unit

A video signal output from the testing apparatus is captured by a camera or a line and a voice signal output from the testing apparatus is captured by a microphone or a line.

The distortion analyzer

A watermark detection unit for detecting a watermark for each of the captured video and audio signals for each block;

A watermark block judging unit for analyzing the watermark of the detected video and audio blocks to judge whether the watermark is abnormal;

And a reporting unit for reporting an analysis result of the video and audio determined by the watermark block determination unit.

And a signal storage unit for storing a video block or a voice block including a watermark having a distortion according to a time setting of the user in the reporting unit for a preset time.

FIG. 1 illustrates a real-time image and speech distortion detection system according to an embodiment of the present invention.
2 is a diagram showing an embodiment of real-time image distortion detection according to the present invention.
3 is a diagram showing an embodiment of real-time voice distortion detection according to the present invention.
4 is a block diagram of a real-time image and speech distortion detection apparatus according to an embodiment of the present invention.
5 is a detailed block diagram of the watermark embedding scheduler unit of FIG.
6 is a detailed view of the distortion analyzing unit of FIG.
7 is a flowchart illustrating a method for detecting real-time image and speech distortion according to an exemplary embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.

FIG. 1 illustrates a real-time image and speech distortion detection system according to an embodiment of the present invention.

The real-time image and sound distortion detection system of FIG. 1 includes an image and sound distortion detection apparatus 110 and a test apparatus 120.

Generally, digital watermarking is embedded in multimedia contents such as text, motion picture, still image, and voice in a form that can not be visually or audibly perceived, is concealed, and the concealed information is extracted, It is used as additional information for monitoring video.

The video and audio distortion detecting apparatus 110 includes a watermark embedding apparatus 112 and a watermark detecting apparatus 114. The watermark embedding device 112 receives a digital broadcast stream or a multimedia content file from a TV receiving antenna or STB or BD / DVD in real time, inserts a distortion judgment signal using a watermark into a broadcast signal or an AV signal, 120). The watermark detecting apparatus 114 detects a watermark signal from the video and audio signals output from the testing apparatus 120, determines whether there is a watermark abnormality or not from the detected watermark signal, and reports the determination result. The watermark detection device 114 also stores defect images and audio according to the result reporting.

The test apparatus 120 receives a reference image and a voice signal in which a watermark is embedded from the video and audio distortion detecting apparatus 110 and reproduces the reference video and audio signal through a display or a speaker, (110). The test apparatus 120 may use DTV or BD / DVD, but it is applicable to all multimedia devices using video and sound.

2 is a diagram showing an embodiment of real-time image distortion detection according to the present invention.

2 (a), an original video stream compressed in the MPEG2 or DIVX video format is received from a TV receiving antenna or STB or DVD. For example, the original image has 1080 X 1920 pixels.

Referring to FIG. 2 (b), a compressed image of a predetermined format is decoded and a frame image is divided into a plurality of blocks. For example, an image of one frame is divided into blocks of 32 x 32 pixels. At this time, the blocks may be varied to control the defect margin. Then, a watermark is inserted by applying a watermarking technique based on a spatial domain to the blocks of 32 x 32 pixels partitioned (the position where the watermark is inserted is indicated by an arrow). Then encode the image with the watermark. One frame is 2040 blocks, and the same watermark is inserted for each block.

2 (c), it is checked whether or not the watermark is lost due to the encoding.

2 (d), when the test apparatus is a DTV, the encoded image is converted into an RF signal through a stream server.

2 (e), an image output through the stream server is displayed on a test apparatus such as a DTV.

2 (f), the image output from the test apparatus is captured by the camera. Capture with the camera After setting the threshold automatically according to the test environment before capture, proceed with capture. At this time, there may occur a problem in watermark detection due to a synchronization problem with the watermark embedded image captured by the camera by the FRC (Frame Rate Converter) of the DTV. Accordingly, the synchronization problem can be solved by inserting the same watermark in each of the video blocks.

2 (g), a watermark signal is detected from the captured image to determine whether or not a watermark is present in the image, and the result of the determination is automatically stored in the video file for the set time and the capture screen for reporting. Also, the result report is displayed on the screen so that the user can check it.

3 is a diagram showing an embodiment of real-time voice distortion detection according to the present invention.

3 (a), an original voice stream compressed in an audio format such as AAC or MP3 from a TV receiving antenna or STB or DVD is received.

Referring to FIG. 3 (b), a compressed speech of a predetermined format is decoded and then the decoded speech is divided into a plurality of time blocks. For example, speech for a predetermined time is divided into units of time blocks of 500 ms. At this time, the time blocks may be varied to control the defect margin.

Then, a watermark is inserted into the time blocks partitioned by applying the watermarking technique based on the spatial domain. And encode the voice with a watermark. At this time, the same watermark is inserted for each time block. It is also possible to apply watermarking techniques based on domains other than the spatial domain.

Referring to FIG. 3 (c), it is checked whether the watermark is lost due to the encoding.

Referring to FIG. 3 (d), when the test apparatus is a DTV, the encoded voice is converted into an RF signal through a stream server.

3 (e), the encoded speech is output to a test apparatus such as BD / DVD or DTV and reproduced through the speaker of the test apparatus.

In FIG. 3 (f), the voice output from the DTV is micro-captured and the voice output from the BD / DVD is transmitted through the line.

3 (g), the micro-captured voice signal is generated as an electrical voice signal.

3 (h), the voice signal input from the microphone is PCM-modulated. At this time, the threshold value is automatically set by detecting the deformation of the watermark information by the PCM modulation.

3 (i), a watermark signal is detected from the PCM-modulated audio to determine whether or not a watermark is present in the audio, and the result of the determination is automatically stored in the audio file for the set time. The report is also displayed on the screen so that the user can check it.

4 is a block diagram of a real-time image and speech distortion detection apparatus according to an embodiment of the present invention.

The real-time image and sound distortion detection apparatus of FIG. 4 is roughly divided into a watermark embedding apparatus 112 and a defective detection apparatus 114. The watermark embedding apparatus 112 includes a signal processing unit 410, a watermark embedding scheduler unit 430, a watermark generating unit 440 and an output scheduler unit 450, A signal capturing unit 460, and a signal analyzing unit 470. The signal processing unit 410 is divided into an A / V separating unit 410 and a frame storing unit 420.

First, the A / V separator 412 separates a digital broadcast stream or a multimedia content into video and audio signals by a predetermined signal processing algorithm.

The frame storage unit 414 stores the video and audio signals separated by the A / V separator 412 in real time on a file unit basis.

The frame storage unit 414 may be, for example, a magnetic recording medium such as a hard disk, or a nonvolatile memory such as an EEPROM or a flash memory, but is not limited thereto.

The watermark generating unit 440 generates an image watermark inserted into the image signal on a block-by-block basis and a voice watermark inserted on a time block basis for the voice signal based on the watermarking technique based on the spatial domain. At this time, the watermark generating unit 440 generates robust video and audio watermark for the correction and amplification of video and audio.

In one embodiment, the watermark generating unit 440 is divided into a video watermark generating unit (not shown) and a voice watermark generating unit (not shown).

The image watermark generation unit determines a watermark pattern indicating information related to playback of the original image or the original image, and converts the watermark pattern into a video data format to generate video watermark data. The voice watermark generation unit also determines a watermark pattern indicating information related to playback of the original voice or the original voice, and converts the watermark pattern into a voice data format to generate voice watermark data. At this time, the watermark pattern is determined based on information related to reproduction of the original image or the original image. For example, the watermark pattern may indicate information on the device that generates the watermark embedding information, that is, identification information of the device, position, and information on the reproduction time. As another example, the work mark pattern may include a TDFD (Temporal Drop-Frame Detection) signal for determining a frame drop and an SDBD (Spatial Distort-Block Detection) signal for determining image distortion in an image frame. The watermark pattern can be generated in the form of a random number sequence. Typically, the types of watermarks can be classified into Robust Watermark, Semi Watermark, and Fragile Watermark depending on the degree of resistance to external attack or deformation. The types of watermarks that can be used in the embodiments of the present invention are those that can withstand the amplification and deformation such as compression and A / D conversion, D / A conversion, noise addition, FRC (Frame Rate Converter) Use robust watermark.

The watermark embedding scheduler 430 inserts video and audio watermarks of the same intensity for each block in real time on the video and audio signals stored in the frame storage 420.

At this time, the watermark signal is inserted for each video and audio block in order to block watermark loss and distortion due to amplification and deformation of video and audio.

The output scheduler unit 450 outputs the video and audio in which the video and audio watermark are embedded in the watermark insertion scheduler unit 430 to the test apparatus 120 such as DTV or BD / DVD.

The signal capture unit 460 captures the video and audio output from the test apparatus 120. In one embodiment, the signal capture unit 460 captures an image signal output from the test apparatus 120 to a camera or a line, and captures a voice signal output from the test apparatus 120 to a microphone or a line.

The signal analyzing unit 470 detects video and audio watermark from the video and audio captured by the signal capturing unit 460 to determine whether or not there is a watermark in the video and audio, do. In one embodiment, the signal analyzer 470 automatically detects the audio distortion of the test equipment by applying a blind watermark technique to the captured audio signal.

Therefore, according to the present invention, a watermark is embedded in real-time broadcast contents (image, moving image, voice) in real time, thereby enabling continuous and continuous testing of problems that may occur in a real market environment.

5 is a detailed block diagram of the watermark embedding scheduler 430 of FIG.

The blocking unit 510 blocks the video and audio signals into a plurality of preset block sizes and a plurality of block times. At this time, the block size and the plurality of block times can be adjusted for defect margins.

The controller 520 adjusts the block size of the image to be blocked and the block time of the voice in the blocking unit 510. That is, the defect margin can be adjusted by adjusting the block size of the image and the block time of the voice. In addition, the controller 520 may adjust the block size and the block time according to the type of the apparatus to be tested for image and speech distortion.

The watermark inserting unit 530 inserts blocks of video and audio signals blocked in the blocking unit 510 based on the video block size, the audio block time, the entire watermark embedding time and the divided embedding time adjusted by the control unit 520, The video and audio watermark are inserted into the video and audio watermark.

In another embodiment, the watermark inserting unit 530 determines a watermark embedding strength for a specific pixel in each of the partitioned blocks, and inserts the watermark into the image according to the watermark embedding strength determined for each block.

FIG. 6 is a detailed view of the distortion analyzer 470 of FIG.

The watermark detection unit 610 detects a watermark for each of the captured video and audio signals on a block-by-block basis.

The watermark block determination unit 620 analyzes the watermark of the video and audio blocks detected by the watermark detection unit 610 to determine whether or not the watermark is abnormal.

The reporting unit 630 reports the analysis result of the video and audio determined by the watermark block determination unit 620. The reporting unit 630 generates a list by dividing the image frame into whether or not the image distortion has occurred in the image frame and the position where the image distortion has occurred.

The signal storage unit 640 stores a video frame or a speech block including a watermark having a distortion according to the user's time setting in the reporting unit 630 for a set time.

7 is a flowchart illustrating a method for detecting real-time image and speech distortion according to an exemplary embodiment of the present invention.

First, in operation 710, a digital broadcasting stream or a multimedia content is separated into a video signal and a voice signal from a TV receiving antenna, a STB (SetTopBox), or a BD / DVD. At this time, the separated video and audio signals are stored in each storage area on a frame basis.

Subsequently, the video and audio watermark signals of the same intensity are inserted into the video and audio blocks for the stored video and audio signals (operation 720). At this time, the video and audio watermark signals are generated based on the watermarking technique based on the spatial domain in one embodiment. In addition, the watermark signal is inserted for each video and audio block in order to block watermark loss and distortion due to amplification and deformation of video and audio.

Subsequently, the watermarked image and audio are output to a test apparatus such as BD / DVD or DTV (step 730). At this time, the test apparatus reproduces the reference image and the audio signal in which the watermark is inserted by the display or the speaker, and outputs the video and audio signal to be reproduced.

Next, in step 740, the video and audio output of the watermark embedded in the test apparatus is captured using a camera or a microphone.

Next, a watermark signal is detected from the captured image and voice to determine whether or not a watermark exists in the image and the voice (operation 750).

After checking whether the watermark of the video and audio is finally abnormal, the result is automatically reported (step 760).

Therefore, according to the present invention, real-time embedding of watermarks in real-time broadcast contents (image, moving picture, voice) enables continuous and continuous testing of problems that may occur in a real market environment.

The present invention can also be embodied as computer-readable codes on a computer-readable recording medium. A computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer system is stored. Examples of the computer-readable recording medium include a ROM, a RAM, a CD-ROM, a magnetic tape, a hard disk, a floppy disk, a flash memory, and an optical data storage device. The computer readable recording medium may also be distributed over a networked computer system and stored and executed as computer readable code in a distributed manner.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention. Accordingly, the scope of the present invention is not limited to the above-described embodiments, but should be construed to include various embodiments falling within the scope of the claims.

Claims (17)

A video and audio distortion detection method comprising:
A process of separating a digital broadcast stream or multimedia content reproduced in a digital multimedia device into video and audio signals;
Inserting a watermark signal having the same intensity for each block into the video and audio signals;
Outputting the video and audio signal in which the watermark signal is embedded to a test apparatus;
Capturing the video and audio output in which the watermark signal output from the testing apparatus is embedded;
Detecting a watermark signal from the captured video and audio to determine whether or not a watermark is present in the video and audio, detecting whether the video and audio are distorted according to presence or absence of a watermark in the video and audio, And determining whether there is an abnormality. The method of claim 1, further comprising storing the separated video and audio signals in a separate storage area in a file unit. The method as claimed in claim 1, wherein the watermark signal is inserted into the image and the voice to be amplified or transformed for each block. The video and audio distortion detection method according to claim 1, wherein the watermark signal includes an image watermark signal embedded in a block-by-block basis for a video signal and a voice watermark signal inserted in a time- . The method of claim 1, wherein the step of inserting the watermark signal comprises:
A block size and a block time for determining image and speech distortion in a video and audio signal to be watermarked;
A step of forming a plurality of video and audio signals into blocks with a predetermined block size;
And embedding the video watermark signal and the audio watermark signal of the same intensity for each block of the video and audio signal according to the entire watermark embedding time and the divided embedding time. The method of claim 1, wherein the block size and the block time for the image and speech distortion determination are adjusted according to the type of the device under test. The method of claim 1, wherein the step of determining whether or not the watermark of the video and audio is abnormal includes:
Detecting a corresponding watermark on a block-by-block basis for captured video and audio signals;
And analyzing a watermark of each of the detected blocks to determine whether there is an abnormality of the watermark. The method of claim 1, further comprising reporting an analysis result of the video and audio. 9. The method of claim 8,
Wherein a video block or a voice block including a watermark having a distortion according to a user's time setting is stored for a set time.
A video and audio distortion detecting apparatus comprising:
A signal processor for separating digital broadcasting signals or multimedia contents reproduced by the digital multimedia device into video and audio signals and storing the separated video and audio signals in a corresponding storage area;
A watermark generating unit for generating video and audio watermark;
A watermark embedding scheduler for inserting the video and audio watermarks of the same intensity for each block into the reference video and audio signals stored in the signal processor;
An output scheduler for outputting video and audio in which the video and audio watermark is embedded in the watermark embedding scheduler to a testing apparatus;
A signal capturing unit capturing video and audio output from the test apparatus;
The video and audio watermark is detected from the video and audio captured by the signal capturing unit to determine whether or not a watermark is present in the video and audio, and whether or not the video and audio distortion is detected And a distortion analyzer for determining whether or not the digital multimedia device is abnormal. 11. The apparatus of claim 10, wherein the signal processing unit
A signal separator for separating a digital broadcast signal or a multimedia content into video and audio signals;
And a frame storage unit for storing the video and audio signals separated by the signal separation unit in the respective storage areas.
The apparatus of claim 10, wherein the watermark generating unit
Wherein a robust watermark is generated for amplification and deformation of video and audio, which are inherent functions of the digital multimedia device.
The watermark embedding apparatus according to claim 10, wherein the watermark embedding scheduler
A blocking unit for blocking a plurality of blocks with a block size and a block time set for video and audio signals;
A controller for adjusting the block size and the block time according to a defect margin;
And a watermark inserting unit for inserting the video and audio watermark into the video and audio blocks blocked by the blocking unit according to the block size and the block time adjusted by the control unit.
11. The apparatus of claim 10, wherein the signal capture unit
Wherein the video signal output from the testing apparatus is captured by a camera or a line and the audio signal output from the testing apparatus is captured by a microphone or a line.
The apparatus of claim 10, wherein the distortion analyzer
A watermark detection unit for detecting a watermark for each of the captured video and audio signals for each block;
A watermark block judging unit for analyzing the watermark of the detected video and audio blocks to judge whether the watermark is abnormal;
And a reporting unit for reporting an analysis result of the video and audio determined by the watermark block determination unit. 16. The apparatus of claim 15, further comprising a signal storage unit for storing a video block or a voice block including a watermark having a distortion according to a user's time setting in the reporting unit for a set time, Device. 10. A computer-readable recording medium on which a program for implementing the method of any one of claims 1 to 9 is recorded.

Images