JP2003256385A - Data processing device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve a problem that, as a device for performing real-time processing of digital watermark embedding for high resolution moving images, a device that can handle enormous computing power and enormous volumes of data is required, but the achievement of the device is difficult. <P>SOLUTION: A plurality of processor groups in which a plurality of processors are linearly combined via channel connections, a processor for distributing input data, and a processor for aggregating output data are provided. A processor at one end of the plurality of processor groups and the processor for distributing input data are connected through a shared bus for distribution. A processor at the other end of the plurality of processor groups and the processor for aggregating output data are connected through a shared bus for aggregation. An inputted moving image is partitioned into areas by the processor for distributing input data, and each one is sent to the processor groups. The partitioned moving image is submitted to watermark embedding processing by pipeline processing with the processor groups and the result is sent to the processor for aggregation. The processor for aggregation combines partitioned moving images for which the watermark embedding processing has been performed into an image, thereby outputting it as a moving image. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

TECHNICAL FIELD The present invention relates to content information such as moving images.
The present invention relates to a data processing device that embeds information such as copy control information and copyright information.

[0002]

2. Description of the Related Art Content information such as digitized music and moving images has come to be distributed via a communication network, and the importance of copyright protection of content information is increasing. Embedding copyright protection information, etc. in technology that protects the copyright of content information by making changes to the content information that humans cannot notice.
There is digital watermark technology.

For example, Japanese Patent Laid-Open No. 2000-175019 discloses an example of a digital watermark technique for embedding copyright protection information in a moving image.

In the digital watermark technology, it is necessary to suppress deterioration of content information. Further, the embedded information is required to be resistant to deterioration even if image processing such as reduction is performed. To meet these two requirements, Japanese Unexamined Patent Application Publication 2000-2000
Japanese Patent Publication No. 175019 discloses a method of detecting a motion between frames forming a moving image and using the result to change the method of embedding copyright information.

Further, in Japanese Patent Laid-Open No. 2000-4350,
A method of suppressing deterioration of content information by using a filter developed for medical use is disclosed. The medical filter of the present technology detects the edge direction of the image for each pixel,
Smoothing processing is performed based on the one-dimensional information in the edge direction.

The watermark embedding apparatus combines the above-described motion detection, medical filter, and many other algorithms to proceed with the processing. Therefore, a programmable electronic computer system is generally used for the watermark embedding device.

[0007]

In recent years, the amount of content information has increased. Also, it is required to process the embedding of copyright information in real time. However, in the above-described motion detection and medical filter processing, several hundreds of calculations are required for each pixel. For example, horizontal 192
In the case of image data having 0 pixels and 1080 pixels and a resolution of 30 frames per second, it is necessary to process about 62 M pixels per second. Therefore, in order to process digital watermark embedding in the above resolution video in real time, at least:
It requires several tens of G operations per second. Here, assuming that the amount of data for one pixel is 2 bytes, it is necessary to input / output approximately 324 Mbytes of data per second. In addition to this, since there is intermediate data for calculation, a total of several GB of data must be transferred per second.

There is a multiprocessor system as a device for processing a large number of operations in this way. In order to process the above processing amount in the multiprocessor system, several tens of processors capable of 1 G / s operation are required. The connection method of each arithmetic unit in such a multiprocessor system is described in “Parallel processing mechanism” (Takahashi Yoshizo, 1989,
Maruzen) shown, such as shared bus coupling and channel coupling. However, when processing a large amount of data content information in a multiprocessor system, the bandwidth of data transfer on the bus is insufficient with a single shared bus coupling or a single line coupling configuration using channel coupling. . In addition, if several tens of processors are completely connected by using channel connection, the number of channels becomes huge, which is difficult to realize. That is, the conventional multiprocessor system cannot cope with an increase in the amount of content information, and cannot satisfy the request for processing the embedding of copyright information in real time.

An object of the present invention is to provide a data processing device for processing in real time the embedding of a digital watermark in content information having a huge amount of data.

[0010]

In order to achieve the above object, a data processor of the present invention is a multiprocessor system having two independent shared buses for input image data and output image data. To do. A processor group in which at least two processors are line-coupled using channel coupling is connected to the input image data shared bus and the output image data shared bus.

Further, the data processing apparatus of the present invention includes a data separating apparatus for separating a part of input data, a separated data storage apparatus for storing the separated data, a data processed by the multiprocessor system and a separated data. It may be configured to have a data synthesizing device that synthesizes and outputs data in the storage device.

[0012]

BEST MODE FOR CARRYING OUT THE INVENTION The data processing apparatus of the present invention will be described below with reference to FIGS. In the following embodiment,
A case where the data processing device is a moving image watermark embedding device will be described. In addition, in the present embodiment, the moving image output from the television camera is used as the content information. An example will be described in which the moving image watermark embedding device changes the luminance value of a pixel to embed electronic watermark information in a moving image and outputs the result to a video tape recorder.

FIG. 1 is a diagram for explaining the configuration of the moving picture digital watermark embedding device of the present invention and the connection with peripheral devices.
The moving picture digital watermark embedding device 1 is connected to a moving picture signal generating device 10 of a television camera and a moving picture storage device 11 of a video tape recorder. The moving image data output from the moving image signal generator 10 is input to the moving image input / output device 2 and sent to the multiprocessor device 3. The moving image data in which the copyright information is embedded by the multiprocessor device 3 is output to the moving image storage device 11 via the moving image input / output device 2.

The structure and operation of the moving image input / output device 2 will be described. The moving image input device 20 receives the moving image data from the moving image signal generating device 10 and sends it to the image identifier adding device 21.
There is a signal indicating a frame break in the moving image data, and the image identifier adding device 21 detects the signal and distinguishes the frame image data. The image identifier adding device 21 gives an image identifier, for example, a serial number, to the distinguished frame images. The image data to which the image identifier is added is sent to the image information separating device 22.

The image information separating device 22 converts the image data into
It is divided into data related to watermark embedding processing and other data. As shown in FIG. 6, the image data 4 in one frame includes an effective image area 40, a vertical blanking area 41, and a horizontal blanking area 42. Of these, the areas other than the effective image area 40 are not related to the watermark embedding processing of the present embodiment. Each pixel is composed of a luminance component and a color component. However, in this embodiment, the color component is not related to the watermark embedding process. Image information separation device 22
Sends data not related to such watermark embedding processing to the separation information storage device 23 and data related to watermark embedding processing to the multiprocessor device 3. An image identifier is added to both data.

The separation information storage device 23 stores the data and the image identifier sent from the image information separation device 22 in association with each other. Further, the separation information storage device 23 sends the data and the image identifier associated with the data to the image information composition device 24.

The image information synthesizing device 24 receives the data and the image identifier which have been watermark-embedded by the multiprocessor device 3. The image information composition device 24 sends the received image identifier to the separation information storage device 23, and receives the data separated by the image information separation device 22 from the separation information storage device 23. Then, the image information synthesizer 24
The data received from the multiprocessor device 3 and the data received from the separation information storage device 23 are recombined and sent to the moving image output device 25 as moving image data. The moving image output device 25 outputs the moving image data received from the image information synthesizer 24 to the moving image signal receiving device 11.

The configuration of the multiprocessor device 3 will be described. The multiprocessor device 3 includes a distribution processor 34 that receives moving image data from the moving image input / output device 2, an aggregation processor 35 that outputs moving image data to the moving image input / output device 2, and a pipeline processing processor in which a plurality of arithmetic processors are linearly connected by channels. A group 33, a distribution processor 34, a shared bus 31 for data distribution to which a processor at one end of the pipeline processing processor group 33 is connected, an aggregation processor 35, and the pipeline processing processor group 33.
The shared bus 32 for data aggregation to which the processor at the other end is connected. The number of pipeline processor groups 33 is arbitrary.

A console 30 is connected to the distribution processor 34, and controls the operation of the entire multiprocessor device 3. The console 30 is used for instructing the multiprocessor device 3 about a value of a constant above an algorithm such as information to be embedded in data or embedding strength. Since these algorithm constants differ depending on the content handled and the copyright holder of the content, it is necessary to specify for each content. Each processor is a program storage type electronic computer.

FIG. 2 shows a pipeline processor group 3.
It is a figure which shows the structure of 3. The pipeline processor group 33 is composed of a distribution bus side arithmetic processor 330, an aggregate bus side arithmetic processor 331, and an arbitrary number of intermediate processing arithmetic processors 332. However, the arithmetic processor 332 for intermediate processing may be omitted. The intermediate processing arithmetic processor 332 is connected by a channel 333.

FIG. 3 shows a distribution bus side arithmetic processor 330.
3 is a diagram showing a configuration of an aggregate bus side arithmetic processor 331. FIG. The distribution bus side arithmetic processor 330 and the aggregate bus side arithmetic processor 331 are the central processing unit 3300,
The storage device 3301, the shared bus connection device 3302, and the channel communication device 3303 are the internal local bus 3305.
It has a configuration connected by.

FIG. 4 is a diagram showing the configuration of the intermediate processing processor 333. Intermediate processing processor 333
The central processing unit 3300, the storage device 3301, the distribution bus side channel communication device 3323, and the aggregation bus side channel communication device 3324 are the internal local bus 3305.
It has a configuration connected by.

The distribution bus side processor 330 is connected to the data distribution shared bus 31 via a shared bus connection device 3302. Aggregate bus side arithmetic processor 33
1 is connected to the data aggregation shared bus 32 via the shared bus connection device 3302. The channel communication device 3303 of the distribution bus side arithmetic processor 330 is connected to the distribution bus side channel communication device 3323 of the intermediate processing arithmetic processor 333 or the channel communication device 3303 of the aggregate bus side arithmetic processor 331. The channel communication device 3303 of the aggregation bus side arithmetic processor 331 is
The aggregate bus side channel communication device 3324 of the intermediate processing arithmetic processor 333 or the distribution bus side arithmetic processor 3
It is connected to 30 channel communication devices 3303 via a channel 333.

The distribution bus side channel communication device 3323 of the intermediate processing arithmetic processor 332 is, as shown in FIG. 2, the aggregate bus side channel communication device 3324 of the adjacent intermediate processing arithmetic processor 333 or the distribution bus side arithmetic processor. It is connected to the channel communication device 3303 of 330.
The aggregate bus side channel communication device 3324 of the intermediate processing arithmetic processor 332 is, as shown in FIG. 2, the distribution bus side channel communication device 3 of the left adjacent intermediate processing arithmetic processor 332.
323 or the channel communication device 3303 of the aggregation bus side arithmetic processor 331.

FIG. 5 shows the data distribution shared bus 31 and the data aggregation shared bus 32 of the multiprocessor device 3.
FIG. 10 is a diagram showing an embodiment in which a hierarchical shared bus is used instead of a physically one-stage shared bus.

The first bus bridge device 311 transfers data between the first sub shared bus 3110 and the main shared bus 310. Similarly, the second bus bridge device 312 is
Data is transferred between the sub shared bus 3120 and the main shared bus 310. Further, the third bus bridge device 313 is
Data is transferred between the third sub shared bus 3130 and the main shared bus 310. With such a configuration, it logically functions as one shared bus, and many processors can be connected without increasing the electrical load of each bus.

In the embodiment of FIG. 5, the distribution processor 34 and the aggregation processor 35 have the configuration shown in FIG. 3, like the distribution bus side arithmetic processor 330. The video input / output device 2 is connected using a channel communication device 3303.

Next, details of the operation of the multiprocessor device 3 will be described. The distribution processor 34 uses the moving image input / output device 2
Receives the moving image data and the image identifier sent from, and divides the image data. Then, the distribution processor 34
The divided data and the image identifier are sent to separate pipeline processor groups 33 via the shared bus 31 for data distribution.

FIG. 7 is a diagram showing a method of dividing data in the distribution processor 34. The data of the effective image area 40 is divided into strips from the first divided data area 400 to the seventh divided data area.
It is divided into seven divided data areas 406. At this time, the first divided data area 400 and the second divided data area 401
But let the areas overlap a little. The degree of overlap of the data areas depends on the range of pixels used for motion detection and edge detection used. For example, when the motion search range is 16 pixels vertically and horizontally, and 4 pixels vertically and horizontally for edge detection, the overlap is 16 pixels.

Similarly, the second divided data area 401 and the third divided data area 401
Division data area 402, third division data area 402 and fourth division data area 403, fourth division data area 403 and fifth division data area 404, fifth division data area 404
And sixth divided data area 405, sixth divided data area 40
The fifth and seventh divided data areas 406 are also divided so that the areas slightly overlap each other.

The divided data is distributed to the distribution processor 34.
According to the
Are distributed through the shared bus 31 for data distribution. In the present invention, the number of data divisions must be less than or equal to the number of pipeline processor groups 33. If the number of data divisions is smaller than the number of pipeline processor groups 33, the distribution processor 34
However, the pipeline processor group 33 that should process the divided data is arbitrarily selected and the data is distributed.
By doing so, it becomes possible to perform processing by separate processors as processing of continuous images.

Each pipeline processor group 33
The distribution bus side arithmetic processor 330 receives the area-divided data and the image identifier. The distribution bus side arithmetic processor 330, the intermediate processing arithmetic processor 332, and the aggregate bus side arithmetic processor 331 perform watermark embedding processing on the divided image data by pipeline processing. The data in which the watermark is embedded and the image identifier are sent from the aggregation bus side arithmetic processor 331 to the aggregation processor 35 via the data aggregation shared bus 32.

Pipeline processing in the pipeline processor group 33 will be described with reference to FIG. In FIG. 8, the number of processors in one pipeline processor group 33 is three. Here, the motion detection processor 334 is the distribution bus side arithmetic processor 33 shown in FIG.
0, the embedded processor 338 corresponds to the aggregate bus side arithmetic processor 331 shown in FIG. 2, and the embedded strength determination filter processor 336 corresponds to the intermediate processing arithmetic processor 332 shown in FIG.

Motion detection processor 3 that received the data
34 performs a data motion detection process. The motion detection processor 334 selects a parameter for the watermark embedding operation based on the result of the motion detection. The parameters of the watermark embedding operation, the image data, and the image identifier are sent to the embedding strength determination filter processor 336 via the channel communication line 335. The embedding strength determination filter processor 336 obtains, for each pixel, a data change range that does not interfere with human image reference. This operation result is
This is called the embedding strength determination result.

The embedding strength judgment result, the embedding operation parameter, the image data and the image identifier are sent to the embedding processor 338 via the channel communication line 337. The embedding processor 338 embeds the watermark in the image data using the embedding strength determination result and the embedding operation parameter. The image data in which the watermark is embedded and the image identifier are transmitted via the shared bus 32 for data aggregation.
It is sent to the aggregation processor 35.

The aggregation processor divides the divided data sent from each pipeline processor group 33 into
The moving image input / output device 2 is reassembled into the same image configuration as before being divided by the distribution processor 34, and is added together with the image identifier.
Output to.

With the configuration of the present invention, the amount of data transferred in the device can be reduced. First, the image information separating device 22 transfers only a part of the moving image data used for the watermark processing to the multiprocessor device 3, so that the amount of data transferred in the multiprocessor device 3 becomes half or less.

Further, since the data distribution shared bus 31 and the data aggregation shared bus 32 are provided independently, the amount of data transferred by each shared bus is half that of one shared bus. Further, the intermediate data of the calculation is often larger than the input / output data, but in the configuration of the present invention, since the data is transferred using the channel communication line 333 in the pipeline processor group 33, it is common. It is easier to secure the bandwidth than when using a bus. In addition, the transfer of the intermediate data for calculation does not consume the bandwidth of the data distribution shared bus 31 and the data aggregation shared bus 32.

For the above reason, even in a multiprocessor system having several tens of processors, the present invention can be used to realize a configuration capable of ensuring communication bandwidth. Moreover, since the multiprocessor system distributes the calculation load, it is possible to secure the calculation capacity for the moving image watermark embedding process.

Further, the pipeline processor group 33
Is connected to the data distribution shared bus 31 and the data aggregation shared bus 32, the number of pipeline processor groups 33 can be easily increased or decreased. Therefore, it is possible to easily change the calculation capability by expanding or reducing the function.

For example, in the present invention, it is possible to change the number of pipeline processor groups 33 for processing data according to the resolution of the target moving image. Specifically, when the resolution of the moving image to be processed is low, the number of pipeline processor groups 33 is small. On the other hand, when the resolution of the moving image to be processed is high, the pipeline processing processor group 33 is used in a large number. Further, the distribution processor 34 changes the number of divisions of data according to the input image size, and changes the number of pipeline processing processor groups 33 that send the divided data.

[0042]

As described above, according to the present invention, the load generated by calculation is distributed to a plurality of processors, and the bandwidth of the communication circuit between the processors can be reduced. Therefore, the digital watermark embedding in the content information such as a moving image can be performed in real time. Can be processed in.

[Brief description of drawings]

FIG. 1 is a diagram showing a configuration of a moving image digital watermark embedding device of the present invention.

FIG. 2 is a diagram illustrating a configuration of a pipeline processing processor group.

FIG. 3 is a diagram illustrating a configuration of a distribution bus side arithmetic processor.

FIG. 4 is a diagram illustrating a configuration of an intermediate processing arithmetic processor.

FIG. 5 is a diagram illustrating a configuration example of a shared bus for data distribution.

FIG. 6 is a diagram illustrating a configuration of image data of one frame as a whole.

FIG. 7 is a diagram illustrating an example of division of image data.

FIG. 8 is a diagram illustrating allocation of processing contents to pipeline processing processors.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Moving image digital watermark embedding device, 2 ... Moving image input / output device, 3 ... Multiprocessor device, 4 ... Image data for 1 frame, 10 ... Moving image signal generating device, 11 ... Moving image signal receiving device, 20 ... Moving image input device, 21 ... Image identifier adding device, 22 ... Image information separating device, 23 ... Separation information storage device, 24 ... Image information combining device, 25 ... Movie output device,
30 ... Console, 31 ... Shared bus for data distribution, 32
... shared bus for data aggregation, 33 ... pipeline processing processor group, 34 ... distribution processor, 35 ... aggregation processor, 40 ... effective image area, 41 ... vertical blanking area, 42 ...
Horizontal blanking area.

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Koichi Terada             1099 Ozenji, Aso-ku, Kawasaki City, Kanagawa Prefecture             Ceremony company Hitachi Systems Development Laboratory (72) Inventor Iwagaki Ashikaga             1099 Ozenji, Aso-ku, Kawasaki City, Kanagawa Prefecture             Ceremony company Hitachi Systems Development Laboratory F term (reference) 5B045 AA01 EE07 FF08 GG15                 5B057 AA20 CA08 CA12 CA16 CB08                       CB12 CB16 CC01 CE08 CH05                       CH08 CH14                 5C063 AB03 AB05 AC01 CA23 DA07                       DA13 DB09

Claims (4)

[Claims] 1. A plurality of processor groups having a plurality of processors, a data distributor for distributing image data to the plurality of processor groups, and a data distribution share for connecting the data distributor to the plurality of processor groups. A bus, a data collection unit that collects data processed by the plurality of processor groups, and a data aggregation shared bus that connects the data collection unit and the plurality of processor groups, the processor group comprising: The plurality of processors included in the processor group are channel-coupled, any one of the plurality of processors is connected to the data distribution shared bus, and the other one is connected to the data aggregation shared bus. A data processing device characterized by the above. 2. An image information separation unit that divides moving image data into two pieces of processing target data and data other than processing target data, a separation information storage unit that stores data other than the processing target data, and data that has undergone data change processing. 2. The data processing device according to claim 1, further comprising: an image information composition unit that recomposes data in the separation information storage device. 3. The data distribution unit has means for dividing data into a plurality of pieces, and each of the plurality of processor groups processes one of the divided pieces of data. The data processing device according to any one of claims 1 and 2. 4. The processor group according to claim 1, wherein the plurality of processor groups perform a process of embedding a watermark in the image data. Data processing device.