CN103139522B - A kind of processing method of multi-view image - Google Patents

Abstract

The invention discloses a kind of processing method of multi-view image, first it adopt histogram matching to set up the color map relation of source images and target image, and according to Gamma correction, watermarking images to be embedded is embedded in source images, obtain the source images being embedded with watermark, again coding transmission is carried out to the source images and target image that are embedded with watermark, and devise color correction and copyright protection two kinds of modes process the decoded source images being embedded with watermark, like this while guarantee multi-view image color correction accuracy, effectively can protect the copyright information of multi-view image, improve the validity of multi-view video system.

Description

Processing method of multi-viewpoint image

Technical Field

The present invention relates to an image processing method, and more particularly, to a multi-viewpoint image processing method.

Background

In recent years, with the rapid development of networks and information technologies, digital multimedia information and works such as digital images, audio and video can be directly obtained and widely spread in a digital form, and pirates can very easily copy or spread the digital multimedia information and works, so that copyright works are widely copied and spread under unauthorized conditions, and the legal rights and interests of original authors are seriously infringed. Therefore, effective copyright protection measures must be taken to prevent illegal copying and distribution of digital multimedia information and works. Digital watermarking is an effective method for realizing copyright protection, characteristics such as invisibility, robustness and safety are important indexes for evaluating the digital watermarking method, and the research of the existing digital watermarking method is mainly focused on two-dimensional video images.

Copyright protection of multi-view video content is also very important in three-dimensional video applications. For the copyright owner of the three-dimensional video content, the copyright owner not only has the copyright of the originally acquired multi-view video content, but also protects other view images generated according to the original multi-view video. Therefore, copyright protection for multi-view images should be different from that of existing two-dimensional video images because: on one hand, how to ensure that effective watermark information can still be extracted from the processed multi-viewpoint images after the multi-viewpoint images are subjected to various operations such as color correction, virtual viewpoint rendering and the like is a problem which needs to be solved urgently; on the other hand, how to embed watermark information into a multi-view image so that the embedded watermark information is not affected by various processes and how to accurately detect the watermark information from color correction are problems that need to be studied and solved in the process of processing the multi-view image.

Disclosure of Invention

The technical problem to be solved by the invention is to provide a method for processing multi-viewpoint images, which can accurately realize the color consistency among viewpoints and does not need to refer to the original viewpoint images when extracting watermarks.

The technical scheme adopted by the invention for solving the technical problems is as follows: a processing method of multi-view image is characterized in that the processing procedure is as follows: firstly, establishing a color mapping relation between a target image and a source image by adopting a histogram matching method; secondly, embedding the watermark image to be embedded into the source image according to the function representation form of gamma correction to obtain the source image embedded with the watermark; then, a multi-view video encoder is adopted to encode and transmit the target image and the source image embedded with the watermark, and the encoded target image and the encoded source image embedded with the watermark are decoded at a user end to obtain a decoded target image and a decoded source image embedded with the watermark; and finally, performing color correction on each color component of each pixel point in the decoded source image embedded with the watermark to obtain a corrected image of the decoded source image embedded with the watermark after the color correction, or extracting a watermark marking image from the decoded source image embedded with the watermark, and then determining whether the source image embedded with the watermark is protected by copyright according to the watermark marking image.

The processing method of the multi-view image specifically comprises the following steps:

① one of the multi-viewpoint images captured by the multi-viewpoint parallel camera system at the same time is defined as a target image and is recorded as a^(T)I_i(x₁,y₁) Define other viewpoint images as source images, and record as-^(S)I_i(x₁,y₁) Where i ═ 1,2,3 denote a luminance component Y, a first chrominance component U, and a second chrominance component V, respectively, of the three color components of the YUV color space, (x) denotes₁,y₁) Representing the coordinate positions of pixel points in the target image and the source image, x is more than or equal to 1₁≤W,1≤y₁H is less than or equal to H, W represents the width of the target image and the source image, H represents the height of the target image and the source image,^(T)I_i(x₁,y₁) Representing^(T)I_i(x₁,y₁) The position of the middle coordinate is (x)₁,y₁) The color value of the ith color component of the pixel point of (1),^(S)I_i(x₁,y₁) Representing^(S)I_i(x₁,y₁) The position of the middle coordinate is (x)₁,y₁) The color value of the ith color component of the pixel point;

② Using histogram matching method, creating a ready object image^(T)T_i(x₁,y₁) Each color value in (f) with a source image^(S)I_i(x₁,y₁) The color mapping relationship of the corresponding color values in the (q = f)_i(p) wherein f_i() A functional representation representing the color mapping of the ith color component, p representing the source image^(S)I_i(x₁,y₁) Colour value in, q represents a source image^(S)I_i(x₁,y₁) Color corrected color value ofThe color value p is more than or equal to 0 and less than or equal to 255, and q is more than or equal to 0 and less than or equal to 255;

③ functional representation according to gamma correction r = α× p^βWatermark image to be embedded { P (x)₂,y₂) Embedding into a source image^(S)I_i(x₁,y₁) In the method, a source image embedded with a watermark is obtained and recorded asWherein α and β are control parameters and p represents the source image^(S)I_i(x₁,y₁) Colour value in (r) representing a source image^(S)I_i(x₁,y₁) The color value of the color value in the color image is gamma-corrected, p is more than or equal to 0 and less than or equal to 255, r is more than or equal to 0 and less than or equal to 255,representing a watermarked source imageThe middle coordinate position is (x)₁,y₁) (ii) the color value of the ith color component of the pixel point of (x)₂,y₂) Representing the watermark image to be embedded P (x)₂,y₂) X is more than or equal to 1₂≤N,1≤y₂N, N represents the watermark image to be embedded P (x)₂,y₂) The horizontal and vertical dimensions of the (X) } size, P (x)₂,y₂) Representing the watermark image to be embedded P (x)₂,y₂) The position of the middle coordinate is (x)₂,y₂) The watermark information of the pixel points;

④ A multi-view video encoder for a target image^(T)I_i(x₁,y₁) And watermark embedded source imageCoding transmission is carried out, the coded target image and the coded source image embedded with the watermark are decoded at the user end to obtain the decoded target image and the decoded embedded imageThe watermark source images are respectively correspondingly recorded asAndwherein,representing decoded target imageThe middle coordinate position is (x)₁,y₁) The color value of the ith color component of the pixel point of (1),representing decoded watermarked source imagesThe middle coordinate position is (x)₁,y₁) The color value of the ith color component of the pixel point;

⑤ different ways to decode the watermarked source imageProcessing is performed, if the color correction mode is selected, step ⑥ is performed, and if the copyright protection mode is selected, step ⑦ is performed;

⑥ according to q = f_i(p) for the decoded watermarked source imagePerforming color correction on each color component of each pixel point to obtainThe corrected image after color correction is recorded as^(C)I_i(x₁,y₁) Will be^(C)I_i(x₁,y₁) The position of the middle coordinate is (x)₁,y₁) The color value of the ith color component of the pixel point is recorded as^(C)I_i(x₁,y₁)，

⑦ according to r = α× p^βFrom decoded watermarked source imagesExtracting watermark image, recording as^(Q)I_i(x₁,y₁) Will be^(Q)I_i(x₁,y₁) The position of the middle coordinate is (x)₁,y₁) The color value of the ith color component of the pixel point is recorded as^(Q)I_i(x₁,y₁)，Then judging whether it can be directly mapped from watermark image^(Q)I_i(x₁,y₁) Observe the embedded watermark image { P (x) }₂,y₂) Watermark information of each pixel point in the image, if yes, determining the source image embedded with the watermarkIs copyright protected, otherwise, the source image embedded with watermark is determinedAre not protected by copyright.

The concrete process of the second step is as follows:

② -1 calculating object image^(T)I_i(x₁,y₁) A histogram of the ith color component of said image, said histogram being recorded as^(T)H_i(k) |0 ≦ k ≦ 255}, and then calculating |^(T)H_i(k) A cumulative histogram of |0 ≦ k ≦ 255},is noted as^(T)C_i(k)|0≤k≤255}，Wherein,^(T)H_i(k) representing an object image^(T)I_i(x₁,y₁) The color value of the ith color component of the pixel is the number of all pixel points of k,^(T)H_i(j) representing an object image^(T)I_i(x₁,y₁) The color value of the ith color component is the number of all pixel points of j,^(T)C_i(k) representing an object image^(T)I_i(x₁,y₁) The color value of the ith color component of the pixel is not more than the accumulated number of all pixel points of k;

② -2 calculating source image^(S)I_i(x₁,y₁) A histogram of the ith color component of said image, said histogram being recorded as^(S)H_i(k) |0 ≦ k ≦ 255}, and then calculating |^(S)H_i(k) The cumulative histogram of |0 ≦ k ≦ 255} is recorded as |0 ≦ k ≦ 255^(S)C_i(k)|0≤k≤255}，Wherein,^(S)H_i(k) representing a source image^(S)I_i(x₁,y₁) The color value of the ith color component of the pixel is the number of all pixel points of k,^(S)H_i(j) representing a source image^(S)I_i(x₁,y₁) The color value of the ith color component is the number of all pixel points of j,^(S)C_i(k) representing a source image^(S)I_i(x₁,y₁) The color value of the ith color component of the pixel is not more than the accumulated number of all pixel points of k;

② -3 picture of taking source^(S)I_i(x₁,y₁) The color value of the ith color component of } is not greater than p₀The accumulated number of all pixel points^(S)C_i(p₀) Then calculate^(S)C_i(p₀) To a target image^(T)I_i(x₁,y₁) The accumulated number of all pixel points of which the color value of the ith color component is not more than q^(T)C_i(q) distance between (q) is noted^(S)C_i(p₀)-^(T)C_i(q) |, based on^(S)C_i(p₀)-^(T)C_i(q) | obtaining source image | (pocket)^(S)I_i(x₁,y₁) Color value p in₀To a target image^(T)I_i(x₁,y₁) Colour value q in₀Is marked as q₀=f_i(p₀)，Wherein p is not less than 0₀≤255，0≤q₀Is less than or equal to 255, q is less than or equal to 0 and less than or equal to 255, "|" is an absolute value symbol,is shown to make^(S)C_i(p₀)-^(T)C_i(q) the smallest value of q;

② -4, selecting 255 and p₀Color values which are different from each other, respectively denoted as p₁,p₂,…,p_j′，…,p₂₅₅The image is then respectively obtained in the same way as in the operation of step ② -3^(S)I_i(x₁,y₁) Color value p in₁,p₂,…,p_j′，…,p₂₅₅To a target image^(T)I_i(x₁,y₁) Colour value q in₁,q₂,…,q_j,…,q₂₅₅The color mapping relations in one-to-one correspondence are respectively marked as q₁=f_i(p₁),q₂=f_i(p₂),…,q_j′=f_i(p_j′),…,q₂₅₅=f_i(p₂₅₅)， Wherein j' is more than or equal to 1 and less than or equal to 255, q is more than or equal to 0 and less than or equal to 255, and p is more than or equal to 0₁,p₂,…,p_j',…,p₂₅₅≤255，0≤q₁,q₂,…,q_j′,…,q₂₅₅≤255，q₁,q₂,…,q_j′，…,q₂₅₅Value of (a) and q₀The values of (a) and (b) are different from each other,^(S)C_i(p₁) Representing a source image^(S)I_i(x₁,y₁) The color value of the ith color component of } is not greater than p₁The cumulative number of all the pixel points of (1),^(S)C_i(p₂) Representing a source image^(S)I_i(x₁,y₁) The color value of the ith color component of } is not greater than p₂The cumulative number of all the pixel points of (1),^(S)C_i(p_j′) Representing a source image^(S)I_i(x₁,y₁) The color value of the ith color component of } is not greater than p_j'The cumulative number of all the pixel points of (1),^(S)C_i(p₂₅₅) Representing a source image^(S)I_i(x₁,y₁) The color value of the ith color component of } is not greater than p₂₅₅The cumulative number of all pixel points.

The concrete process of the step III is as follows:

③ -1, according to r = α× p^βObtaining the optimum density value of the embedded watermark, and recording as p^*，Wherein,in order to take the function of the gradient,is shown to makeThe maximum p value;

③ -2, based on the optimal density value p of the embedded watermark^*Obtained at the source image^(S)I_i(x₁,y₁) The starting position of the embedded watermark is recorded as Wherein, 1≤x₀≤W-N+1，1≤y₀≤H-N+1，^(S)I_i(x₁,y₁) Representing a source image^(S)I_i(x₁,y₁) The position of the middle coordinate is (x)₁,y₁) The color value of the 1 st color component of the pixel point, "|" is the absolute value symbol,is shown to makeMaximum (x)₀,y₀)；

③ -3 for containing a source image^(S)I_i(x₁,y₁) On the abscissaInterval(s)Inner and ordinate onDefining the pixel point currently being processed in the interval as a current pixel point;

③ -4, assume the coordinate position of the current pixel point to be (x)₁',y₁') and then judges the watermark image to be embedded { P (x) }₂,y₂) The coordinate position in the equation isWhether the watermark information of the pixel point is 1 or not, if so, modifying the color value of the 1 st color component of the current pixel point, and recording the modified color value of the 1 st color component of the current pixel point as the color value Otherwise, directly taking the color value of the 1 st color component of the current pixel point as the color value of the 1 st color component of the modified current pixel point, and recording the color value of the 1 st color component of the modified current pixel point as the color value of the 1 st color component of the modified current pixel point Wherein, $x_1^{*}\≤{x_1}^{\′}\≤x_1^{*}+N-1,$ $y_1^{*}\≤{y_1}^{\′}\≤y_1^{*}+N-1,$ ^(S)I_i(x₁',y₁') denotes the source image^(S)I_i(x₁,y₁) The position of the middle coordinate is (x)₁',y₁') the color value of the 1 st color component of the pixel point, Δ being the intensity factor;

③ -5 for containing a source image^(S)I_i(x₁,y₁) On the abscissaWithin a range and with the ordinate atTaking the next pixel point to be processed in the interval as the current pixel point, then returning to the step ③ -4 to continue executing until the image leaf^(S)I_i(x₁,y₁) On the abscissaWithin a range and with the ordinate atAfter all pixel points in the interval are processed, obtaining a source image embedded with the watermark and recording the source image as the watermarkWherein,representing a watermarked source imageThe middle coordinate position is (x)₁,y₁) The color value of the ith color component of the pixel point.

And taking alpha =1 and beta =0.2 in the step (c).

And step three-3, taking delta = 3.

Compared with the prior art, the invention has the advantages that:

1) the method achieves the aims of color correction and copyright protection by changing the color mapping relation of a source image and a target image, realizes accurate color correction by adopting a histogram matching method, ensures the colors between viewpoints to be consistent, enhances the strength of the embedded watermark by adopting gamma correction, has very simple watermark embedding and extracting methods, does not need to refer to an original viewpoint image when extracting the watermark, and can be effectively applied to various multi-viewpoint video systems.

2) The method of the invention designs two modes of color correction and copyright protection to process the decoded source image embedded with the watermark, thus ensuring the color correction accuracy of the multi-view image, effectively protecting the copyright information of the multi-view image and improving the effectiveness of the multi-view video system.

Drawings

FIG. 1 is a block flow diagram of the method of the present invention;

FIG. 2a is a target image of the "flamenco 1" multi-view test set;

FIG. 2b is a source image of the "flamenco 1" multi-view test set;

FIG. 2c is a target image of the multi-view test set of "objects 2";

FIG. 2d is a source image of the multi-view test set of "objects 2";

FIG. 3a is an embedded "Ningbo university" watermark image;

FIG. 3b is an embedded "information college" watermark image;

FIG. 3c is an embedded "image processing" watermark image;

FIG. 3d is an embedded "electronic information" watermark image;

FIG. 3e is an embedded "one, two, three, four" watermark image;

FIG. 4a is a color corrected image of the image of FIG. 2b with the embedded watermark image of FIG. 3a obtained by the method of the present invention;

FIG. 4b is a diagram of a watermark image obtained by the method of the present invention after the watermark image shown in FIG. 3a is embedded in the image shown in FIG. 2 b;

FIG. 4c is a color corrected image of the image of FIG. 2d with the embedded watermark image of FIG. 3a obtained by the method of the present invention;

FIG. 4d is a diagram illustrating a watermark image obtained by the method of the present invention after the watermark image shown in FIG. 3a is embedded in the image shown in FIG. 2 d;

FIG. 5a is a color corrected image of the image of FIG. 2b embedded with the watermark image of FIG. 3b, obtained by the method of the present invention;

FIG. 5b is a diagram of a watermark image obtained by the method of the present invention after the watermark image shown in FIG. 3b is embedded in the image shown in FIG. 2 b;

FIG. 5c is a color corrected image of the image of FIG. 2d with the embedded watermark image of FIG. 3b obtained by the method of the present invention;

FIG. 5d is a diagram of a watermark image obtained by the method of the present invention after the watermark image shown in FIG. 3b is embedded in the image shown in FIG. 2 d;

FIG. 6a is a color corrected image of the image of FIG. 2b embedded with the watermark image of FIG. 3c obtained by the method of the present invention;

FIG. 6b is a diagram of a watermark image obtained by the method of the present invention after the watermark image shown in FIG. 3c is embedded in the image shown in FIG. 2 b;

FIG. 6c is a color corrected image of the image of FIG. 2d with the embedded watermark image of FIG. 3c obtained by the method of the present invention;

FIG. 6d is a diagram of a watermark image obtained by the method of the present invention after the watermark image shown in FIG. 3c is embedded in the image shown in FIG. 2 d;

FIG. 7a is a color corrected image of the image of FIG. 2b with the embedded watermark image of FIG. 3d obtained by the method of the present invention;

FIG. 7b is a diagram of a watermark image obtained by the method of the present invention after the watermark image shown in FIG. 3d is embedded in the image shown in FIG. 2 b;

FIG. 7c is a color corrected image of the image of FIG. 2d with the embedded watermark image of FIG. 3d obtained by the method of the present invention;

FIG. 7d is a diagram of a watermark image obtained by the method of the present invention after the watermark image shown in FIG. 3d is embedded in the image shown in FIG. 2 d;

FIG. 8a is a color corrected image of the image of FIG. 2b with the embedded watermark image of FIG. 3e obtained by the method of the present invention;

FIG. 8b is a diagram of a watermark image obtained by the method of the present invention after the watermark image shown in FIG. 3e is embedded in the image shown in FIG. 2 b;

FIG. 8c is a color corrected image of the image of FIG. 2d with the embedded watermark image of FIG. 3e obtained by the method of the present invention;

fig. 8d is a watermark image obtained by the method of the present invention for the image embedded with the watermark image shown in fig. 3e in fig. 2 d.

Detailed Description

The invention is described in further detail below with reference to the accompanying examples.

The flow of the processing method of the multi-view image provided by the invention is shown in fig. 1, and the processing process is as follows: firstly, establishing a color mapping relation between a target image and a source image by adopting a histogram matching method; secondly, embedding the watermark image to be embedded into the source image according to the function representation form of gamma correction to obtain the source image embedded with the watermark; then, a multi-view video encoder is adopted to encode and transmit the target image and the source image embedded with the watermark, and the encoded target image and the encoded source image embedded with the watermark are decoded at a user end to obtain a decoded target image and a decoded source image embedded with the watermark; and finally, performing color correction on each color component of each pixel point in the decoded source image embedded with the watermark to obtain a corrected image of the decoded source image embedded with the watermark after the color correction, or extracting a watermark marking image from the decoded source image embedded with the watermark, and then determining whether the source image embedded with the watermark is protected by copyright according to the watermark marking image.

The processing method of the multi-viewpoint image of the invention specifically comprises the following steps:

① one of the multi-viewpoint images captured by the multi-viewpoint parallel camera system at the same time is defined as a target image and is recorded as a^(T)I_i(x₁,y₁) Define other viewpoint images as source images, and record as-^(S)I_i(x₁,y₁) Where i ═ 1,2,3 denote a luminance component Y, a first chrominance component U, and a second chrominance component V, respectively, of the three color components of the YUV color space, (x) denotes₁,y₁) Representing the coordinate positions of pixel points in the target image and the source image, x is more than or equal to 1₁≤W,1≤y₁H is less than or equal to H, W represents the width of the target image and the source image, H represents the height of the target image and the source image,^(T)I_i(x₁,y₁) Representing^(T)I_i(x₁,y₁) The position of the middle coordinate is (x)₁,y₁) The color value of the ith color component of the pixel point of (1),^(S)I_i(x₁,y₁) Representing^(S)I_i(x₁,y₁) The position of the middle coordinate is (x)₁,y₁) The color value of the ith color component of the pixel point.

In the present embodiment, multi-view video test sets "flamencol" and "objects 2" provided by KDDI corporation of japan are adopted as the original multi-view video images. Fig. 2a and 2b are respectively a target image and a source image of a "flamenco 1" multi-viewpoint test set, fig. 2c and 2d are respectively a target image and a source image of an "objects 2" multi-viewpoint test set, and the size of the target image and the size of the source image are both 320 × 240. As can be seen from fig. 2a, 2b, 2c and 2d, the color appearance of the target image and the source image is significantly different, and therefore color correction thereof is necessary.

② since the histogram can reflect the statistical distribution information of image color, the invention adopts histogram matching method to establish the destination image^(T)I_i(x₁,y₁) Each color value in (f) with a source image^(S)I_i(x₁,y₁) The color mapping relationship of the corresponding color values in the (q = f)_i(p) wherein f_i() A functional representation representing the color mapping of the ith color component, p representing the source image^(S)I_i(x₁,y₁) Colour value in, q represents a source image^(S)I_i(x₁,y₁) Color corrected color values of the color values in (f), i.e. representing a corresponding target image^(T)I_i(x₁,y₁) P is more than or equal to 0 and less than or equal to 255, and q is more than or equal to 0 and less than or equal to 255.

In this embodiment, the specific process of step two is:

② -1 calculating object image^(T)I_i(x₁,y₁) A histogram of the ith color component of said image, said histogram being recorded as^(T)H_i(k) |0 ≦ k ≦ 255}, and then calculating |^(T)H_i(k) The cumulative histogram of |0 ≦ k ≦ 255} is recorded as |0 ≦ k ≦ 255^(T)C_i(k)|0≤k≤255}，Wherein,^(T)H_i(k) representing an object image^(T)I_i(x₁,y₁) The color value of the ith color component of the pixel is the number of all pixel points of k,^(T)H_i(j) representing an object image^(T)I_i(x₁,y₁) The color value of the ith color component is the number of all pixel points of j,^(T)C_i(k) representing an object image^(T)I_i(x₁,y₁) The color value of the ith color component of the pixel is not more than the accumulated number of all pixel points of k.

② -2 calculating source image^(S)I_i(x₁,y₁) A histogram of the ith color component of said image, said histogram being recorded as^(S)H_i(k) |0 ≦ k ≦ 255}, and then calculating |^(S)H_i(k) The cumulative histogram of |0 ≦ k ≦ 255} is recorded as |0 ≦ k ≦ 255^(S)C_i(k)|0≤k≤255}，Wherein,^(S)H_i(k) representing a source image^(S)I_i(x₁,y₁) The color value of the ith color component of the pixel is the number of all pixel points of k,^(S)H_i(j) representing a source image^(S)I_i(x₁,y₁) The color value of the ith color component is the number of all pixel points of j,^(S)C_i(k) representing a source image^(S)I_i(x₁,y₁) The color value of the ith color component of the pixel is not more than the accumulated number of all pixel points of k.

② -3 picture of taking source^(S)I_i(x₁,y₁) The color value of the ith color component of } is not greater than p₀The accumulated number of all pixel points^(S)C_i(p₀) Then calculate^(S)C_i(p₀) To a target image^(T)I_i(x₁,y₁) Colour value of ith colour component ofAccumulated number of all pixel points not greater than q^(T)C_i(q) distance between (q) is noted^(S)C_i(p₀)-^(T)C_i(q) |, based on^(S)C_i(p₀)-^(T)C_i(q) | obtaining source image | (pocket)^(S)I_i(x₁,y₁) Color value p in₀To a target image^(T)I_i(x₁,y₁) Colour value q in₀Is marked as q₀=f_i(p₀)，Wherein p is not less than 0₀≤255，0≤q₀Is less than or equal to 255, q is less than or equal to 0 and less than or equal to 255, "|" is an absolute value symbol,is shown to make^(S)C_i(p₀)-^(T)C_iThe value of (q) | is the smallest q value.

② -4, selecting 255 and p₀Color values which are different from each other, respectively denoted as p₁,p₂,…,p_j′，…,p₂₅₅The image is then respectively obtained in the same way as in the operation of step ② -3^(S)I_i(x₁,y₁) Color value p in₁,p₂,…,p_j′，…,p₂₅₅To a target image^(T)I_i(x₁,y₁) Colour value q in₁,q₂,…,q_j′,…,q₂₅₅The color mapping relations in one-to-one correspondence are respectively marked as q₁=f_i(p₁),q₂=f_i(p₂),…,q_j′=f_i(p_j′),…,q₂₅₅=f_i(p₂₅₅)， Wherein j' is more than or equal to 1 and less than or equal to 255, q is more than or equal to 0 and less than or equal to 255, and p is more than or equal to 0₁,p₂,…,p_j',…,p₂₅₅≤255，0≤q₁,q₂,…,q_j′,…,q₂₅₅≤255，q₁,q₂,…,q_j′，…,q₂₅₅Value of (a) and q₀The values of (a) and (b) are different from each other,^(S)C_i(p₁) Representing a source image^(S)I_i(x₁,y₁) The color value of the ith color component of } is not greater than p₁The cumulative number of all the pixel points of (1),^(S)C_i(p₂) Representing a source image^(S)I_i(x₁,y₁) The color value of the ith color component of } is not greater than p₂The cumulative number of all the pixel points of (1),^(S)C_i(p_j′) Representing a source image^(S)I_i(x₁,y₁) The color value of the ith color component of } is not greater than p_j'The cumulative number of all the pixel points of (1),^(S)C_i(p₂₅₅) Representing a source image^(S)I_i(x₁,y₁) The color value of the ith color component of } is not greater than p₂₅₅The cumulative number of all pixel points.

③ since gamma correction can extend smaller color deviations into a larger color range, the present invention is based on the functional representation of gamma correction in the form r = α× p^βWatermark image to be embedded { P (x)₂,y₂) Embedding into a source image^(S)I_i(x₁,y₁) In the method, a source image embedded with a watermark is obtained and recorded asWherein α and β are control parameters, and in the embodiment, are takenα =1, β =0.2, p representing a source image ∑^(S)I_i(x₁,y₁) Colour value in (r) representing a source image^(S)I_i(x₁,y₁) The color value of the color value in the color image is gamma-corrected, p is more than or equal to 0 and less than or equal to 255, r is more than or equal to 0 and less than or equal to 255,representing a watermarked source imageThe middle coordinate position is (x)₁,y₁) (ii) the color value of the ith color component of the pixel point of (x)₂,y₂) Representing the watermark image to be embedded P (x)₂,y₂) X is more than or equal to 1₂≤N,1≤y₂N, N represents the watermark image to be embedded P (x)₂,y₂) The horizontal and vertical dimensions of the (X) } size, P (x)₂,y₂) Representing the watermark image to be embedded P (x)₂,y₂) The position of the middle coordinate is (x)₂,y₂) The watermark information of the pixel points.

In this embodiment, the specific process of step (c) is:

③ -1, according to r = α× p^βObtaining the optimum density value of the embedded watermark, and recording as p^*，Wherein,in order to take the function of the gradient,is shown to makeThe maximum p value.

③ -2, based on the optimal density value p of the embedded watermark^*ObtainingAt source image^(S)I_i(x₁,y₁) The starting position of the embedded watermark is recorded as Wherein, 1≤x₀≤W-N+1，1≤y₀≤H-N+1，^(S)I_i(x₁,y₁) Representing a source image^(S)I_i(x₁,y₁) The position of the middle coordinate is (x)₁,y₁) The color value of the 1 st color component of the pixel point, "|" is the absolute value symbol,is shown to makeMaximum (x)₀,y₀)。

③ -3 for containing a source image^(S)I_i(x₁,y₁) From the starting positionBeginning, abscissa atWithin a range and with the ordinate atThe pixel point currently being processed in the interval is defined as a current pixel point.

③ -4, assume the coordinate position of the current pixel point to be (x)₁',y₁') and then judges the watermark image to be embedded { P (x) }₂,y₂) The coordinate position in the equation isWhether the watermark information of the pixel point is 1 or not, if so, modifying the color value of the 1 st color component of the current pixel point, and recording the modified color value of the 1 st color component of the current pixel point as the color value Otherwise, directly taking the color value of the 1 st color component of the current pixel point as the color value of the 1 st color component of the modified current pixel point, and recording the color value of the 1 st color component of the modified current pixel point as the color value of the 1 st color component of the modified current pixel point Wherein, $x_1^{*}\≤{x_1}^{\′}\≤x_1^{*}+N-1,$ $y_1^{*}\≤{y_1}^{\′}\≤y_1^{*}+N-1,$ ^(S)I_i(x₁',y₁') denotes the source image^(S)I_i(x₁,y₁) The position of the middle coordinate is (x)₁',y₁') color value of the 1 st color component of the pixel point, Δ is the intensity factor, and in this embodiment, Δ =3 is taken.

③ -5 for containing a source image^(S)I_i(x₁,y₁) On the abscissaWithin a range and with the ordinate atTaking the next pixel point to be processed in the interval as the current pixel point, then returning to the step ③ -4 to continue executing until the image leaf^(S)I_i(x₁,y₁) On the abscissaWithin a range and with the ordinate atAfter all pixel points in the interval are processed, obtaining a source image embedded with the watermark and recording the source image as the watermarkWherein,representing a watermarked source imageThe middle coordinate position is (x)₁,y₁) The color value of the ith color component of the pixel point.

④ A multi-view video encoder for a target image^(T)I_i(x₁,y₁) And source maps embedded with watermarksImageCoding and transmitting are carried out, the coded target image and the coded source image embedded with the watermark are decoded at the user end, the decoded target image and the decoded source image embedded with the watermark are obtained and respectively and correspondingly recorded asAndwherein,representing decoded target imageThe middle coordinate position is (x)₁,y₁) The color value of the ith color component of the pixel point of (1),representing decoded watermarked source imagesThe middle coordinate position is (x)₁,y₁) The color value of the ith color component of the pixel point.

In this embodiment, the multiview video encoder employs the well-known HBP coding prediction structure.

⑤ different ways to decode the watermarked source imageProcessing is performed, and if the color correction mode is selected, step ⑥ is executed, and if the copyright protection mode is selected, step ⑦ is executed.

⑥ according to q = f_i(p) for the decoded watermarked source imagePerforming color correction on each color component of each pixel point to obtainThe corrected image after color correction is recorded as^(C)I_i(x₁,y₁) Will be^(C)I_i(x₁,y₁) The position of the middle coordinate is (x)₁,y₁) The color value of the ith color component of the pixel point is recorded as^(C)I_i(x₁,y₁)，

⑦ according to r = α× p^βFrom decoded watermarked source imagesExtracting watermark image, recording as^(Q)I_i(x₁,y₁) Will be^(Q)I_i(x₁,y₁) The position of the middle coordinate is (x)₁,y₁) The color value of the ith color component of the pixel point is recorded as^(Q)I_i(x₁,y₁)，Then judging whether it can be directly mapped from watermark image^(Q)I_i(x₁,y₁) Observe the embedded watermark image { P (x) }₂,y₂) Watermark information of each pixel point in the image, if yes, determining the source image embedded with the watermarkIs copyright protected, otherwise, the source image embedded with watermark is determinedAre not protected by copyright.

The subjective performance of the color correction and watermark extraction of the two sets of multi-view video test sets "flamenco 1" and "objects 2" by the method of the present invention is compared below.

Fig. 3a, fig. 3b, fig. 3c, fig. 3d, and fig. 3e respectively show five groups of binary watermark images to be embedded, and the resolution of each group of watermark images is 80 × 80. FIG. 4a is a color corrected image obtained by the method of the present invention of the image after the watermark image shown in FIG. 3a is embedded in FIG. 2b, FIG. 4c is a color corrected image obtained by the method of the present invention of the image after the watermark image shown in FIG. 3a is embedded in FIG. 2d, FIG. 5a is a color corrected image obtained by the method of the present invention of the image after the watermark image shown in FIG. 3b is embedded in FIG. 2b, FIG. 5c is a color corrected image obtained by the method of the present invention of the image after the watermark image shown in FIG. 3b is embedded in FIG. 2d, FIG. 6a is a color corrected image obtained by the method of the present invention of the image after the watermark image shown in FIG. 3c is embedded in FIG. 2b, FIG. 7a is a color corrected image obtained by the method of the present invention of the image after the watermark image shown in FIG. 2b is embedded in FIG. 3d, fig. 7c is a color corrected image obtained by the method of the present invention from the image of fig. 2d embedded with the watermark image shown in fig. 3d, fig. 8a is a color corrected image obtained by the method of the present invention from the image of fig. 2b embedded with the watermark image shown in fig. 3e, and fig. 8c is a color corrected image obtained by the method of the present invention from the image of fig. 2d embedded with the watermark image shown in fig. 3e, and it can be seen from the subjective effects of the images shown in fig. 4a, fig. 4c, fig. 5a, fig. 5c, fig. 6a, fig. 6c, fig. 7a, fig. 7c, fig. 8a, and fig. 8c that the color corrected image obtained by the method of the present invention has a color appearance very close to the target image and the embedded watermark information cannot be perceived from the color corrected image, compared to the target image of "flamenco 1" shown in fig. 2a and the target image of "objects 2" shown in fig. 2c, the watermark embedding process illustrating the method of the invention is efficient.

FIG. 4b is a watermark image obtained by the method of the present invention from the image in which the watermark image shown in FIG. 3a is embedded in FIG. 2b, FIG. 4d is a watermark image obtained by the method of the present invention from the image in which the watermark image shown in FIG. 3a is embedded in FIG. 2d, FIG. 5b is a watermark image obtained by the method of the present invention from the image in which the watermark image shown in FIG. 3b is embedded in FIG. 2b, FIG. 5d is a watermark image obtained by the method of the present invention from the image in which the watermark image shown in FIG. 3b is embedded in FIG. 2d, FIG. 6b is a watermark image obtained by the method of the present invention from the image in which the watermark image shown in FIG. 3c is embedded in FIG. 2b, FIG. 7b is a watermark image obtained by the method of the present invention from the image in which the watermark image shown in FIG. 3d is embedded in FIG. 2b, fig. 7d is a watermark image obtained by the method of the present invention from the image in which the watermark image shown in fig. 3d is embedded in fig. 2d, fig. 8b is a watermark image obtained by the method of the present invention from the image in which the watermark image shown in fig. 3e is embedded in fig. 2b, fig. 8d is a watermark image obtained by the method of the present invention from the image in which the watermark image shown in fig. 3e is embedded in fig. 2d, and the embedded watermark image can be easily observed from the watermark images shown in fig. 4b, fig. 4d, fig. 5b, fig. 5d, fig. 6b, fig. 6d, fig. 7b, fig. 7d, fig. 8b, and fig. 8d, which shows that the watermark extraction process by the method of the present invention is effective.

Claims (5)

1. A processing method of multi-view image is characterized in that the processing procedure is as follows: firstly, establishing a color mapping relation between a target image and a source image by adopting a histogram matching method, wherein the target image is one viewpoint image in multi-viewpoint images shot by a multi-viewpoint parallel camera system at the same time, and the source image is other viewpoint images; secondly, embedding the watermark image to be embedded into the source image according to the function representation form of gamma correction to obtain the source image embedded with the watermark; then, a multi-view video encoder is adopted to encode and transmit the target image and the source image embedded with the watermark, and the encoded target image and the encoded source image embedded with the watermark are decoded at a user end to obtain a decoded target image and a decoded source image embedded with the watermark; finally, performing color correction on each color component of each pixel point in the decoded source image embedded with the watermark to obtain a corrected image of the decoded source image embedded with the watermark after the color correction, or extracting a watermark marking image from the decoded source image embedded with the watermark, and then determining whether the source image embedded with the watermark is protected by copyright according to the watermark marking image;

the processing method specifically comprises the following steps:

① one of the multi-viewpoint images captured by the multi-viewpoint parallel camera system at the same time is defined as a target image and is recorded as a^(T)I_i(x₁,y₁) Define other viewpoint images as source images, and record as-^(S)I_i(x₁,y₁) Where i ═ 1,2,3 denote a luminance component Y, a first chrominance component U, and a second chrominance component V, respectively, of the three color components of the YUV color space, (x) denotes₁,y₁) Representing the coordinate positions of pixel points in the target image and the source image, x is more than or equal to 1₁≤W,1≤y₁H is less than or equal to H, W represents the width of the target image and the source image, H represents the height of the target image and the source image,^(T)I_i(x₁,y₁) Representing^(T)I_i(x₁,y₁) The position of the middle coordinate is (x)₁,y₁) The color value of the ith color component of the pixel point of (1),^(S)I_i(x₁,y₁) Representing^(S)I_i(x₁,y₁) The position of the middle coordinate is (x)₁,y₁) The color value of the ith color component of the pixel point;

② Using histogram matching method, creating a ready object image^(T)I_i(x₁,y₁) Each color value in (f) with a source image^(S)I_i(x₁,y₁) The color mapping relationship of the corresponding color values in the description is denoted as q ═ f_i(p) wherein f_i() A functional representation representing the color mapping of the ith color component, p representing the source image^(S)I_i(x₁,y₁) Colour value in, q represents a source image^(S)I_i(x₁,y₁) P is more than or equal to 0 and less than or equal to 255, and q is more than or equal to 0 and less than or equal to 255;

③ function representation according to gamma correction, r α× p^βWatermark image to be embedded { P (x)₂,y₂) Embedding into a source image^(S)I_i(x₁,y₁) In the method, a source image embedded with a watermark is obtained and recorded asWherein α and β are control parameters and p represents the source image^(S)I_i(x₁,y₁) Colour value in (r) representing a source image^(S)I_i(x₁,y₁) The color value of the color value in the color image is gamma-corrected, p is more than or equal to 0 and less than or equal to 255, r is more than or equal to 0 and less than or equal to 255,representing a watermarked source imageThe middle coordinate position is (x)₁,y₁) (ii) the color value of the ith color component of the pixel point of (x)₂,y₂) Representing the watermark image to be embedded P (x)₂,y₂) X is more than or equal to 1₂≤N,1≤y₂N, N represents the watermark image to be embedded P (x)₂,y₂) The horizontal and vertical dimensions of the (X) } size, P (x)₂,y₂) Representing the watermark image to be embedded P (x)₂,y₂) The position of the middle coordinate is (x)₂,y₂) The watermark information of the pixel points;

④ A multi-view video encoder for a target image^(T)I_i(x₁,y₁) And watermark embedded source imageCoding and transmitting are carried out, the coded target image and the coded source image embedded with the watermark are decoded at the user end, the decoded target image and the decoded source image embedded with the watermark are obtained and respectively and correspondingly recorded asAndwherein,representing decoded target imageThe middle coordinate position is (x)₁,y₁) The color value of the ith color component of the pixel point of (1),representing decoded watermarked source imagesThe middle coordinate position is (x)₁,y₁) The color value of the ith color component of the pixel point;

⑤ different ways to decode the watermarked source imageProcessing is performed, if the color correction mode is selected, step ⑥ is performed, and if the copyright protection mode is selected, step ⑦ is performed;

⑥ according to q ═ f_i(p) for the decoded watermarked source imagePerforming color correction on each color component of each pixel point to obtainThe corrected image after color correction is recorded as^(C)I_i(x₁,y₁) Will be^(C)I_i(x₁,y₁) The position of the middle coordinate is (x)₁,y₁) The color value of the ith color component of the pixel point is recorded as^(C)I_i(x₁,y₁)， ${I_{\left(C\right)}}_i(x_1,y_1)=f_i\left({\hat{I}_{\left(S\right)}}_i\right(x_1,y_1\left)\right);$

⑦ according to r- α× p^βFrom decoded watermarked source imagesExtract the watermark imageIs noted as^(Q)I_i(x₁,y₁) Will be^(Q)I_i(x₁,y₁) The position of the middle coordinate is (x)₁,y₁) The color value of the ith color component of the pixel point is recorded as^(Q)I_i(x₁,y₁)，Then judging whether it can be directly mapped from watermark image^(Q)I_i(x₁,y₁) Observe the embedded watermark image { P (x) }₂,y₂) Watermark information of each pixel point in the image, if yes, determining the source image embedded with the watermarkIs copyright protected, otherwise, the source image embedded with watermark is determinedAre not protected by copyright.

2. The method for processing multi-viewpoint images according to claim 1, wherein the specific process of the step (ii) is as follows:

② -1 calculating object image^(T)I_i(x₁,y₁) A histogram of the ith color component of said image, said histogram being recorded as^(T)H_i(k) |0 ≦ k ≦ 255}, and then calculating |^(T)H_i(k) The cumulative histogram of |0 ≦ k ≦ 255} is recorded as |0 ≦ k ≦ 255^(T)C_i(k)|0≤k≤255}，Wherein,^(T)H_i(k) representing an object image^(T)I_i(x₁,y₁) The color value of the ith color component of the pixel is the number of all pixel points of k,^(T)H_i(j) representing an object image^(T)I_i(x₁,y₁) The color value of the ith color component is the number of all pixel points of j,^(T)C_i(k) representing an object image^(T)I_i(x₁,y₁) The color value of the ith color component of the pixel is not more than the accumulated number of all pixel points of k;

② -2 calculating source image^(S)I_i(x₁,y₁) A histogram of the ith color component of said image, said histogram being recorded as^(S)H_i(k) |0 ≦ k ≦ 255}, and then calculating |^(S)H_i(k) The cumulative histogram of |0 ≦ k ≦ 255} is recorded as |0 ≦ k ≦ 255^(S)C_i(k)|0≤k≤255}，Wherein,^(S)H_i(k) representing a source image^(S)I_i(x₁,y₁) The color value of the ith color component of the pixel is the number of all pixel points of k,^(S)H_i(j) representing a source image^(S)I_i(x₁,y₁) The color value of the ith color component is the number of all pixel points of j,^(S)C_i(k) representing a source image^(S)I_i(x₁,y₁) The color value of the ith color component of the pixel is not more than the accumulated number of all pixel points of k;

② -3 picture of taking source^(S)I_i(x₁,y₁) The color value of the ith color component of } is not greater than p₀The accumulated number of all pixel points^(S)C_i(p₀) Then calculate^(S)C_i(p₀) To a target image^(T)I_i(x₁,y₁) The accumulated number of all pixel points of which the color value of the ith color component is not more than q^(T)C_i(q) distance between cells marked as +^(S)C_i(p₀)-^(T)C_i(q) |, again according to^(S)C_i(p₀)-^(T)C_i(q) | obtaining source image | (pocket)^(S)I_i(x₁,y₁) Color value p in₀To a target image^(T)I_i(x₁,y₁) Colour value q in₀Is marked as q₀＝f_i(p₀)，Wherein p is not less than 0₀≤255，0≤q₀Is less than or equal to 255, q is less than or equal to 0 and less than or equal to 255, "|" is an absolute value symbol,is represented as-^(S)C_i(p₀)-^(T)C_i(q) the smallest value of q;

② -4, selecting 255 and p₀Color values which are different from each other, respectively denoted as p₁,p₂,…,p_j',…,p₂₅₅The image is then respectively obtained in the same way as in the operation of step ② -3^(S)I_i(x₁,y₁) Color value p in₁,p₂,…,p_j',…,p₂₅₅To a target image^(T)I_i(x₁,y₁) Colour value q in₁,q₂,…,q_j',…,q₂₅₅The color mapping relations in one-to-one correspondence are respectively marked as q₁＝f_i(p₁),q₂＝f_i(p₂),…,q_j'＝f_i(p_j'),…,q₂₅₅＝f_i(p₂₅₅)， $q_1=\arg \underset{q}{min}|{C_{\left(S\right)}}_i\left(p_1\right)-{C_{\left(T\right)}}_i\left(q\right)|,q_2=\arg \underset{q}{min}|{C_{\left(S\right)}}_i\left(p_2\right)-{C_{\left(T\right)}}_i\left(q\right)|,$ $q_{j^{\′}}=\arg \underset{q}{min}|{C_{\left(S\right)}}_i\left(p_{j^{\′}}\right)-{C_{\left(T\right)}}_i\left(q\right)|,q_{255}=\arg \underset{q}{min}|{C_{\left(S\right)}}_i\left(p_{255}\right)-{C_{\left(T\right)}}_i\left(q\right)|,$ Wherein j' is more than or equal to 1 and less than or equal to 255, q is more than or equal to 0 and less than or equal to 255, and p is more than or equal to 0₁,p₂,…,p_j',…,p₂₅₅≤255，0≤q₁,q₂,…,q_j',…,q₂₅₅≤255，q₁,q₂,…,q_j',…,q₂₅₅Value of (a) and q₀The values of (a) and (b) are different from each other,^(S)C_i(p₁) Representing a source image^(S)I_i(x₁,y₁) The color value of the ith color component of } is not greater than p₁The cumulative number of all the pixel points of (1),^(S)C_i(p₂) Representing a source image^(S)I_i(x₁,y₁) The color value of the ith color component of } is not greater than p₂The cumulative number of all the pixel points of (1),^(S)C_i(p_j') Representing a source image^(S)I_i(x₁,y₁) The color value of the ith color component of } is not greater than p_j'The cumulative number of all the pixel points of (1),^(S)C_i(p₂₅₅) Representing a source image^(S)I_i(x₁,y₁) The color value of the ith color component of } is not greater than p₂₅₅The cumulative number of all pixel points.

3. The method according to claim 2, wherein the step (c) comprises the following steps:

③ -1, according to r α× p^βObtaining the optimum density value of the embedded watermark, and recording as p^*，Wherein,in order to take the function of the gradient,is shown to makeThe maximum p value;

③ -2, based on the optimal density value p of the embedded watermark^*Obtained at the source image^(S)I_i(x₁,y₁) The starting position of the embedded watermark is recorded as $(x_1^{*},y_1^{*})=\underset{\left(x_0,y_0\right)}{\arg max}\underset{x_1=x_0}{\overset{x_0+N-1}{\Σ}}\underset{y_1=y_0}{\overset{y_0+N-1}{\Σ}}\left|{I_{\left(S\right)}}_1\right(x_1,y_1)-p^{*}|,$ Wherein,1≤x₀≤W-N+1，1≤y₀≤H-N+1，^(S)I₁(x₁,y₁) Representing a sourceImage^(S)I_i(x₁,y₁) The position of the middle coordinate is (x)₁,y₁) The color value of the 1 st color component of the pixel point, "|" is the absolute value symbol,is shown to makeMaximum (x)₀,y₀)；

③ -3 for containing a source image^(S)I_i(x₁,y₁) On the abscissaWithin a range and with the ordinate atDefining the pixel point currently being processed in the interval as a current pixel point;

③ -4, assume the coordinate position of the current pixel point to be (x)₁',y₁') and then judges the watermark image to be embedded { P (x) }₂,y₂) The coordinate position in the equation isWhether the watermark information of the pixel point is 1 or not, if so, modifying the color value of the 1 st color component of the current pixel point, and recording the modified color value of the 1 st color component of the current pixel point as the color value Otherwise, directly taking the color value of the 1 st color component of the current pixel point as the color value of the 1 st color component of the modified current pixel point, and dividing the 1 st color of the modified current pixel pointColor value of the quantity is noted ${\tilde{I}_{\left(S\right)}}_1({x_1}^{\′},{y_1}^{\′})={I_{\left(S\right)}}_1({x_1}^{\′},{y_1}^{\′}),$ Wherein, $x_1^{*}\≤{x_1}^{\′}\≤x_1^{*}+N-1,y_1^{*}\≤{y_1}^{\′}\≤y_1^{*}+N-1,$ ^(S)I₁(x₁',y₁') denotes the source image^(S)I_i(x₁,y₁) The position of the middle coordinate is (x)₁',y₁') the color value of the 1 st color component of the pixel point, Δ being the intensity factor;

③ -5 for containing a source image^(S)I_i(x₁,y₁) On the abscissaWithin a range and with the ordinate atTaking the next pixel point to be processed in the interval as the current pixel point, then returning to the step ③ -4 to continue executing until the image leaf^(S)I_i(x₁,y₁) On the abscissaWithin a range and with the ordinate atAfter all pixel points in the interval are processed, obtaining a source image embedded with the watermark and recording the source image as the watermarkWherein,representing a watermarked source imageThe middle coordinate position is (x)₁,y₁) The color value of the ith color component of the pixel point.

4. A method according to claim 3, wherein α -1 and β -0.2 are taken in said step (c).

5. The method according to claim 4, wherein Δ -3 is taken from said step- (3).