WO2017113850A1 - Method and apparatus for obtaining parallax parameters of stereoscopic film source - Google Patents

Abstract

A method and an apparatus for obtaining parallax parameters of a stereoscopic film source. The method comprises: reading an image frame from a stereoscopic film source file, wherein the image frame comprises a left-eye image and a right-eye image; determining multiple first image areas in one of the left-eye image and the right-eye image; searching the other one of the left-eye image and the right-eye image for multiple second image areas corresponding to contents of the multiple first image areas, wherein the first image area and the second image area that correspond to each other have a same size; separately calculating a horizontal distance between each first image area and a second image area corresponding thereto, to obtain multiple parallax parameters; and determining a maximum parallax parameter and a minimum parallax parameter, to obtain a variation range of parallax of a stereoscopic film source.

Description

Method and device for obtaining stereoscopic source parallax parameters

The present application claims priority to Chinese Patent Application No. 2015110009115, entitled "Method and Apparatus for Obtaining Stereoscopic Source Parallax Parameters", filed on December 28, 2015, the entire contents of which are incorporated herein by reference. In this application.

Technical field

The present application relates to the field of image processing, for example, to a method and apparatus for acquiring stereo disc source parallax parameters.

Background technique

With the development of 3Dimensional (3D) technology, the number of stereoscopic sources is increasing, and more and more devices supporting viewing stereoscopic sources are also available. The principle of 3D technology is to simultaneously capture the same object through two shooting devices at the same horizontal line but separated by a certain horizontal distance, and obtain two images of the left and right corresponding to the object. When the stereoscopic film source is played, the left side of the left and right eyes respectively allows the user to view the left image and the right eye to view the right image. Since there is a horizontal interval between the photographing devices for simulating the pupil distance of the human eye, there is a certain parallax between the objects respectively presented by the left and right images. When the user views the stereoscopic source, the left and right images are simultaneously transmitted to the brain through the retina, and the brain uses the parallax to generate the depth of field effect of the object, thereby obtaining a stereoscopic effect.

In order to play a better stereoscopic effect, it is necessary to obtain the parallax parameter of the stereoscopic sheet source in advance. The stereo source itself does not provide a parallax parameter, and manual calculation is required to obtain the parallax parameter. The relevant person finds the same object in the left and right images respectively, for example, finds the object "water cup" in the left and right images, and then manually calculates the number of pixels in the horizontal direction of the two "cups" to obtain the horizontal distance on the X axis. , thus obtaining the parallax parameter between the two "cups". Generally, the stereoscopic degree of different objects in the 3D image is different, so the horizontal distances of different objects in the left and right images are different from each other. When the parallax parameters are acquired, the professional needs to select and calculate different objects separately, and obtain multiple parallax parameters. And select the maximum and minimum values of the parallax parameters to determine the range of parallax variation of the stereoscopic source.

Since the related technology mainly relies on the subjective recognition and judgment of the relevant personnel to perform object selection, and the calculation of the horizontal distance is also manually performed by a professional, when the number of objects in the 3D image is large, the efficiency of acquiring the parallax parameter is very low. The related method can still process static stereoscopic pictures, but for stereoscopic video composed of hundreds of thousands of frames or even millions of frames, if the parallax parameters of each frame are manually obtained manually, the time cost will be very high. Big.

Summary of the invention

The present application provides a method and apparatus for obtaining a parallax parameter of a stereoscopic slice source, and solves the problem of inefficiently obtaining a parallax parameter of a stereoscopic slice source.

In one aspect, the embodiment of the present application provides a method for obtaining a parallax parameter of a stereoscopic slice, including:

Reading an image frame from the stereoscopic source file, the image frame including a left eye image and a right eye image;

Determining a plurality of first image regions in one of a left eye image and a right eye image;

Searching, in another image of the left eye image and the right eye image, a plurality of second image regions corresponding to the plurality of first image region contents, wherein the first image region and the second image region corresponding to each other have the same size;

Calculating a horizontal distance between each of the first image regions and the second image region corresponding thereto, respectively, to obtain a plurality of parallax parameters;

The maximum parallax parameter and the minimum parallax parameter are determined to obtain a parallax variation range of the stereoscopic slice source.

On the other hand, the embodiment of the present application further provides an apparatus for acquiring a parallax parameter of a stereoscopic slice, including:

a reading unit configured to read an image frame from the stereoscopic source file, the image frame including a left eye image and a right eye image;

a region determining unit configured to determine a plurality of first image regions in one of a left eye image and a right eye image;

a searching unit configured to search for a plurality of second image regions corresponding to the plurality of first image region contents in another of the left eye image and the right eye image, the first image region and the second image region size corresponding to each other the same;

a calculating unit, configured to separately calculate a horizontal distance between each of the first image regions and the second image region corresponding thereto, to obtain a plurality of parallax parameters;

The parameter determining unit is configured to determine a maximum parallax parameter and a minimum parallax parameter to obtain a parallax variation range of the stereoscopic slice source.

On the other hand, an embodiment of the present application further provides an electronic device, including:

At least one processor; and,

a memory communicatively coupled to the at least one processor; wherein

The memory stores instructions executable by the one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

Reading an image frame from a stereoscopic source file, the image frame including a left eye image and a right eye image;

Determining a plurality of first image regions in one of the left eye image and the right eye image;

Finding and the plurality of first image regions in another of the left eye image and the right eye image a plurality of second image regions corresponding to the domain content, wherein the first image region and the second image region corresponding to each other have the same size;

Calculating a horizontal distance between each of the first image regions and the second image region corresponding thereto, respectively, to obtain a plurality of parallax parameters;

The maximum parallax parameter and the minimum parallax parameter are determined to obtain a parallax variation range of the stereoscopic slice source.

In another aspect, the embodiment of the present application further provides a non-transitory computer readable storage medium, wherein the non-transitory computer readable storage medium stores computer instructions, where the computer instructions are used to cause the computer to execute the above A method of obtaining a stereoscopic source parallax parameter.

In another aspect, the embodiment of the present application further provides a computer program product, including a computing program stored on a non-transitory computer readable storage medium, the computer program including program instructions, when the program instructions are executed by a computer And causing the computer to perform the above-described method of acquiring a stereoscopic source parallax parameter.

The method and device for acquiring a stereoscopic source parallax parameter provided by the embodiments of the present application can automatically read an image frame from a stereoscopic source file, and determine a plurality of identical and one-to-one correspondences in the left and right eye images in the image frame. a first image area and a second image area, respectively calculating a horizontal distance of each pair of the first and second image areas, obtaining a parallax parameter corresponding to each pair of the first and second image areas, and then from the plurality of parallax parameters A maximum parallax parameter and a minimum parallax parameter are determined, thereby obtaining a parallax variation range of the stereoscopic sheet source. The time taken to obtain parallax parameters is greatly shortened, and the processing efficiency is improved.

DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, a brief description of the drawings used in the embodiments or the prior art description will be briefly described below. Obviously, the drawings in the following description It is a certain embodiment of the present invention, and other drawings can be obtained from those skilled in the art without any creative work.

FIG. 1 is a flowchart of a method for obtaining a parallax parameter of a stereoscopic slice according to an embodiment of the present disclosure;

FIG. 2 is a flowchart of another method for obtaining a stereoscopic source parallax parameter according to an embodiment of the present disclosure;

FIG. 3 is a flowchart of still another method for obtaining a parallax parameter of a stereoscopic slice according to an embodiment of the present disclosure;

4a to 4d are schematic diagrams of determining a first image area according to an embodiment of the present application;

4 e is a schematic diagram of determining a second image area according to an embodiment of the present application;

5a to 5d are schematic diagrams of determining pixel points according to an embodiment of the present application;

FIG. 6 is a structural block diagram of an apparatus for acquiring a parallax parameter of a stereoscopic slice according to an embodiment of the present disclosure;

FIG. 7 is a structural block diagram of another apparatus for acquiring a stereoscopic source parallax parameter according to an embodiment of the present application;

FIG. 8 is a schematic diagram of a physical structure of an electronic device for acquiring a parallax parameter of a stereoscopic slice according to an embodiment of the present disclosure.

Implementation

The technical solutions in the embodiments of the present application are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present application. It is a part of the embodiments of the present application, and not all of the embodiments.

An embodiment of the present application provides a method for obtaining a parallax parameter of a stereoscopic slice. As shown in FIG. 1 , the method includes:

In step 110, an image frame is read from the stereoscopic source file.

The stereoscopic source in this embodiment may be a stereoscopic image or a stereoscopic video. When a stereoscopic image is used, the stereoscopic source file usually includes only one image, and for stereoscopic video, the stereoscopic source file generally includes many images. frame. In order to facilitate the unified expression, the embodiment refers to an image in a stereoscopic picture as an image frame.

As mentioned above, the stereoscopic film source is obtained by two imaging devices at the same horizontal line and separated by a certain distance, so whether it is a stereoscopic picture or a stereoscopic video, each image frame is composed of a left eye image and a The right eye image is composed of. Since there is a horizontal interval between the two imaging devices for simulating the eyelid distance of the human eye, both the left eye image and the right eye image have a certain positional offset in the horizontal direction, and this positional offset is the implementation. The parallax parameter to be calculated in the example.

In this step, the computer can read the stereoscopic source file from the storage area or the database according to the default storage path, or directly receive the stereoscopic source file sent by the network side through the preset application interface, or through a preset operation. The interface receives the stereoscopic source file uploaded by the user. This embodiment does not specifically limit the manner in which the stereoscopic source file is obtained.

In step 120, a plurality of first image regions are determined in one of the left eye image and the right eye image.

A prerequisite for obtaining the parallax parameter is to determine two image regions corresponding to the same content, that is, the first image region and the second image region, respectively, in the left and right images. In this embodiment, the first image region is first determined therein based on any one of the left and right images, and then the second image region is determined correspondingly in the other image. For convenience of description, the present embodiment will be described by taking the first image region in the left eye image as an example. Of course, in the actual application, the first image region may be first determined based on the right eye image. The manner of obtaining the parallax parameter is the same as that of the former method, and the present embodiment does not repeat the description.

Usually, a variety of different content is involved in the left and right images, such as for Street View images, where It will include pedestrians, cars, trees, streets, and shops. Although theoretically, the disparity of all content in the left and right images should be consistent, that is, pedestrian 1 and pedestrian 2 (note: pedestrian 1 is the pedestrian in the left eye image, pedestrian 2 is the corresponding pedestrian in the right eye image, the same) Parallax, the parallax of car 1 and car 2 should be the same, but because the depth of field of different objects is different, the disparity of different content may be different in actual situation, which needs to be determined in the left eye image. A plurality of first image regions are determined, and a plurality of second image regions are correspondingly determined in the subsequent right eye images, and thereby a plurality of parallax parameters are calculated.

In this embodiment, the objects in the left eye image can be identified by image recognition technology, and then different first image regions including different objects are determined according to the position and size of the different objects. In this way, the parallax parameters between the objects can be accurately calculated from the naturally formed objects, but because of the use of image recognition technology, a large amount of calculation is involved in practical applications. In order to reduce the processing load of the computer and improve the acquisition speed of the parallax parameter, the first image region may also be subjected to blur determination in the embodiment, that is, the first image region is directly determined without the object recognition as a premise requirement for determining the first image region. . For example, a plurality of first image regions are determined by randomly selecting positions in the left eye image, or several first image regions are determined at specific positions according to the size ratio of the left eye image. This method does not require recognition of the content of the image, and is faster to implement.

In step 130, a plurality of second image regions corresponding to the plurality of first image region contents are searched for in another of the left eye image and the right eye image.

The content correspondence means that the content displayed in the second image area is the same as the content displayed in the first image area. From the perspective of the control variable, only the difference in position between the same content and the left and right images can be used to reflect the disparity of the content in the left and right images. Therefore, the content in the first and second image regions is required in this embodiment. It is corresponding. Under this premise, the first image area and the second image area should be identical at least in the following aspects: 1. The content is the same; 2. The size of the area is the same; 3. The number of areas is the same.

In this step, the computer can find the content corresponding to the content of the first image region in the right eye image through an image matching technology (for example, the method provided in OpenCV), and determine the location of the second image region according to the found content.

It should be noted that when the second image region is determined in the right eye image, the content in the right eye image needs to be identified by the image matching technology, and the purpose is to make the determined second image region and the left eye image. The first image regions correspond to each other in content, and as a precondition for calculating the parallax parameters, such a process of content recognition is necessary. In 120, in the image recognition performed when determining the first image region, since the content of the first image region is not required to correspond to some content, the image recognition process in step 120 may be omitted to improve the processing speed. .

In step 140, a horizontal distance between each of the first image regions and the second image region corresponding thereto is separately calculated to obtain a plurality of parallax parameters.

After obtaining the first image area and the second image area of equal numbers, respectively calculating a horizontal distance between the first image area and the second image area corresponding to the content, thereby obtaining a parallax parameter corresponding to each pair of the first/second image areas . Exemplarily, assuming that four first image regions are determined in step 120, then corresponding four second image regions are determined in step 130. In step 140, the parallax parameter a between the first image region 1 and the second image region 1, the parallax parameter b between the first image region 2 and the second image region 2, the first image region 3, and the first The parallax parameter c between the two image regions 3 and the parallax parameter d between the first image region 4 and the second image region 4 obtain four parallax parameters equal in number to the first image region or the second image region.

In this embodiment, the horizontal distance may be calculated based on a certain pixel point in the first image area and the second image area, or the horizontal distance may be calculated based on the same area edge of the first image area and the second image area. The horizontal distance may also be calculated based on the entirety of the first image area and the second image area. This embodiment does not specifically limit the object for calculating the horizontal distance reference.

In this embodiment, the horizontal distance refers to the difference in distance between the first image area and the second image area on X, and the unit may be the number of pixels, or may be a unit of length in a general sense, such as millimeters, centimeters, and the like. This embodiment does not limit this, but in general, the distance difference is positive and negative.

In step 150, the maximum parallax parameter and the minimum parallax parameter are determined to obtain a parallax variation range of the stereoscopic slice source.

As mentioned above, the disparity between different content may be different. In practice, obtaining the parallax variation range of the entire image frame is more practical than obtaining the parallax parameter of each region in the image frame. Therefore, in this step, the computer finds a maximum parallax parameter and a minimum parallax parameter from all the obtained parallax parameters, and determines the parallax variation range of the image frame as the upper and lower boundary values of the parallax variation range.

For a stereoscopic picture, since it only contains one image frame, the range of parallax variation obtained in this step is the range of parallax variation of the stereoscopic source; for stereoscopic video, since it contains many image frames, all can be All the parallax parameters in the image frame are put together, and a maximum parallax parameter and a minimum parallax parameter are found out to determine the range of parallax variation of the stereoscopic video.

In the following, the embodiment of the present application will provide a method for performing parallax parameter acquisition on a stereoscopic slice source including a plurality of image frames. In practical applications, a stereoscopic source containing a plurality of image frames is usually a stereoscopic video, but it is not excluded that a partial stereoscopic image is composed of at least two image frames, and the latter is equally applicable to the method. As shown in Figure 2, the method includes:

In step 210, at least two image frames are extracted from the stereoscopic source file as sample frames.

In this embodiment, the computer calculates a parallax parameter for at least two image frames in the stereoscopic source file, and puts the parallax parameters of all the image frames together, and selects a maximum parallax parameter and a minimum parallax parameter.

In this step, the computer extracts image frames from the stereoscopic source file in accordance with predetermined rules. Extracting several image frames and extracting which image frames are defined by the preset rule. The following two implementations of extracting image stitches are given:

Mode 1 is extracted according to sampling interval

Usually, the image frames have a certain order in the stereo source files. This order is generally determined by the order in which the image frames are played. In this manner, at least two sampling frames may be extracted according to the predetermined sampling interval based on the ranking. The sampling interval may be an interval frame number or an interval time. For the former implementation manner, it may be specified that one image frame is extracted every four image frames from the first image frame, or one image frame is extracted every 100 image frames; for the latter implementation, the image frame may be played according to the image frame. Timing, starting with the moment when the first image frame is played, extracting one image frame every 100 milliseconds, or extracting one image frame every 15000 milliseconds. The specific numerical values in this mode are for illustrative purposes only and are not intended as a limitation of the actual application.

Mode 2 random extraction

The computer can extract the sampled frames using a random number generation algorithm. In an example of the mode, the computer randomly generates a random number whose value is not greater than the total number of image frames by using a random number generating function, and extracts an image frame with the same image frame number and the random number as the extracted frame. In this example, You need to ensure that the generated random number is a positive integer. In another example of the method, the computer may randomly generate a floating-point random number greater than 0 and less than or equal to 1 by using a random number generating function, and then multiply the random number by the total number of image frames to obtain a random extraction. The number of image frames, this method needs to optimize the algorithm to ensure that the product of the random number and the total number of image frames is a positive integer.

In addition to the above two methods, the computer can also receive the extraction rule set by the staff through a preset human-computer interaction mode, and the extraction rule can only limit the number of extracted image frames, and can also specifically define which specific image frames are extracted. This embodiment does not limit this. In an extreme mode of this embodiment, all image frames in the stereoscopic source file can be extracted.

In step 220, parallax parameters for each sample frame are obtained separately.

In this step, the computer performs step 120 to step 140 in FIG. 1 for each sample frame to obtain the parallax parameters of each sample frame. Exemplarily, it is assumed that there are 130 sampling frames, and each sampling frame needs to calculate 4 parallax parameters, that is, 4 first/second image regions are respectively determined in the left and right images of each sampling frame. Then, after performing step 120 to step 140 for each sample frame, a total of 4*130=520 parallax parameters are obtained.

It should be noted that, in practical applications, the number of parallax parameters obtained by calculating each sampling frame may be For the same, for example, four of the above examples, the number of differentiated parallax parameters may also be determined for different sampling frames, for example, three parallax parameters are acquired for a partial sampling frame, and seven parallax parameters are acquired for another partial sampling frame. This embodiment does not limit the number of disparity parameters acquired in each sampling frame, and does not limit the number of disparity parameters acquired in each sampling frame to be the same.

In this step, the computer may obtain the parallax parameters for each sampling frame according to the sampling frame number from small to large, or may obtain the parallax parameters according to the sequence of the extracted sampling frames. In practical applications, when there are enough idle computing resources, the computer can simultaneously obtain parallax parameters for some or all of the sampled frames through a preset number of parallel threads.

In step 230, among the parallax parameters of all the sampling frames, the maximum parallax parameter and the minimum parallax parameter are determined, and the parallax variation range of the stereoscopic slice source is obtained.

The maximum disparity parameter and the minimum disparity parameter are selected from all disparity parameters of all sample frames. In the above example, when a total of 520 parallax parameters are acquired for 130 sample frames, the computer selects one of the 520 parallax parameters with the largest parallax parameter and the smallest value parallax parameter.

Optionally, as a refinement of the method shown in FIG. 1 and FIG. 2, the embodiment of the present application further provides a method for obtaining a parallax parameter of a stereoscopic slice source. For ease of description, the method is described in terms of obtaining parallax parameters from an image frame. It should be clarified that although the expression angle has certain limitations, it does not affect the application of the method in the case of FIG. As shown in FIG. 3, the method includes:

In step 310, an image frame is read from the stereoscopic source file.

In step 320, a plurality of preset image size first regions are determined at preset positions in one of the left eye image and the right eye image.

Taking the first image region from the left eye image as an example, the computer determines a plurality of preset image first image regions from preset positions in the left eye image. Variables that can be automatically set by the computer or manually set manually include: 1. coordinates of the preset position; 2. the number of preset positions (ie, the number of first image areas); 3. size of the first image area . Several examples of determining the first image area in this embodiment are given below:

As shown in FIG. 4a, in an implementation manner of the embodiment, the computer randomly selects three rectangular regions in the left eye image by using a random algorithm, and determines the first image region (the length and width values of the rectangle are not shown). .

As shown in FIG. 4b, in another implementation manner of the embodiment, the computer selects four square regions according to the layout position of the four-square grid, and determines the first image region. The size of the square is reduced by a preset ratio based on the size of the square grid (the square length value of the square is not shown).

As shown in FIG. 4c, in another implementation manner of this embodiment, the computer is arranged according to the layout of the nine squares. Position, 9 square areas are selected and determined as the first image area. The size of the square is reduced by a preset ratio based on the size of the nine squares (the square length value of the square is not shown).

As shown in FIG. 4d, in another implementation manner of the embodiment, the computer selects eight square regions in the diagonal position of the left eye image, and determines the first image region (the square length value of the square is not shown). Out).

In step 330, the first image area is filtered to eliminate the first image area whose pixel chromaticity is less than a predetermined degree of dispersion.

In order to improve the accuracy of calculating the parallax parameter, in the alternative of the embodiment, the determined first image region may be filtered to remove the image region with a single chroma. In practical applications, for a single chrominance image region (for example, an all white or all green image region), since the chromaticities of the respective pixels in the image region are the same or similar, when subsequently comparing with the second image region, The horizontal offset of the image cannot be accurately determined, thereby making the calculated parallax parameter larger or smaller than the actual parallax parameter. Therefore, in this step, the first image region with a single chroma can be eliminated, thereby eliminating inaccurate parallax parameters. It should be noted that the image region of a single chromaticity includes not only an image region (for example, an all white region) involving only one chromaticity, but also an image region involving a plurality of chromaticities but relatively close chromaticity. For example, the image area involving blue sky and white clouds, although it contains both white and blue hues, in some specific climatic conditions, the chromaticity of white clouds is very similar to the chromaticity of the blue sky as the background, there is no obvious difference between the two. For this image area, this step will also be rejected.

The computer can cull the first image region of a single chrominance by calculating the degree of dispersion of the pixel chromaticity. The degree of dispersion of pixel chrominance reflects the degree of chromaticity difference between pixels in the image region. The greater the degree of dispersion, the greater the difference in pixel chromaticity. The preset dispersion degree is used as a boundary condition for distinguishing a single chrominance image region and an image region whose chromaticity dispersion degree satisfies the calculation requirement. In actual application, the preset dispersion degree can be obtained by the computer according to the accuracy of the historical parallax parameter, and also It can be obtained by the staff based on experience.

In an implementation of this embodiment, the degree of dispersion of pixel chrominance can be quantified by standard deviation, and the standard deviation is used to characterize the degree of deviation of all values relative to the average, wherein the average is the average of all values. The value, the degree of deviation is derived from the individual difference contribution of each value to the mean. Therefore, the standard deviation can be reflected as a whole for the degree of dispersion of all values (relative mean). In this mode, the computer calculates that each pixel in the first image region corresponds to a standard difference of three color components of red, green, and blue, and obtains three differences of a red standard deviation value, a green standard deviation value, and a blue standard deviation value. Value parameter. Taking the red standard deviation as an example, the standard deviation reflects the degree of deviation of the red color chromaticity mean between different pixels on the red component. The larger the red standard deviation, the greater the dispersion of each pixel on the red component. The chromaticity of the entire image region on the red component is not uniform. In After obtaining the red standard deviation value, the green standard deviation value and the blue standard deviation value, the computer sums the three to obtain the comprehensive standard deviation value, and then compares the integrated standard deviation value with the preset standard deviation threshold value. The first image area whose integrated standard deviation is less than the standard deviation threshold is eliminated.

Alternatively, the standard deviation is calculated as follows:

Where σ is the standard deviation value, N is the number of pixel points in the first image region, x _i is the chromaticity value of the ith pixel point, and μ is the chromaticity mean of all the pixel points.

First, the computer calculates the standard deviation of all pixels on the red component by the following formula:

Where r represents the red component, σ _r is the standard deviation of the first image region on the red component, x _ri is the chromaticity value of the i-th pixel on the red component, and μ _r is the red component of all pixels The average of the chromaticity.

Second, the computer calculates the standard deviation of all pixels on the green component by the following formula:

Where g is the green component, σ _g is the standard deviation of the first image region on the green component, x _gi is the chromaticity value of the ith pixel on the green component, and μ _g is the pixel on all the green components The average of the chromaticity.

The computer then calculates the standard deviation of all pixels on the blue component by the following formula:

Where b represents the blue component, σ _b is the standard deviation of the first image region on the blue component, x _bi is the chromaticity value of the ith pixel on the blue component, and μ _b is the pixel of all pixels The chromaticity mean on the blue component.

It should be noted that when calculating the standard deviation, it is necessary to calculate the degree of dispersion of the relative mean of each pixel, but the obtained standard deviation is a value reflecting the degree of overall deviation of all the pixels, and therefore, on one color component, An image area corresponds to a standard deviation.

Thereby, the computer obtains standard deviation values σ _r , σ _g and σ _b corresponding to the three color components of the first image region. Then σ _r , σ _g and σ _b are summed to obtain a comprehensive standard deviation σ _c . Finally, the computer compares σ _c with a preset standard deviation threshold. If σ _{c is} greater than the standard deviation threshold, the dispersion of the pixel chromaticity in the first image region is sufficiently large to retain the first image region; If σ _{c is} smaller than the standard deviation threshold, it indicates that the degree of dispersion of the pixel chromaticity in the first image region is not large enough, and the first image region is eliminated. This completes the screening of the first image area. In the example shown in Figure 4a, the first image area in the lower right corner will be rejected.

The above method is only one of the implementation methods for screening the first image region. In practical applications, a simpler algorithm can also be used to filter the first image region. For example, calculate the average difference of each color component according to the following formula:

Where V is the average difference of a certain color component, and the characters and arithmetic symbols in the absolute value symbol correspond to the content in the foregoing formula. Compared with the standard deviation algorithm, the average difference algorithm can reduce the calculation amount of the computer to some extent, but the accuracy of the standard deviation algorithm is higher from the perspective of reflecting the degree of color deviation. In addition, in practical applications, the edge of the object in the image may be identified according to the degree of change in the brightness of the pixel, and the first image region containing less objects may be removed accordingly.

It should be noted that step 330 is an optional step in FIG. 3, and in actual application, step 340 may be directly executed after step 320 is performed, that is, the first image area is not filtered.

In step 340, a plurality of second image regions corresponding to the plurality of first image region contents are searched for in the other of the left eye image and the right eye image.

In this embodiment, the other image is a right eye image, and the computer determines a plurality of second image regions in the right eye image, the number and size of the second image regions, and the number and size of the first image regions in the left eye image. the same. However, it is worth noting that the position of the second image area in the right eye image is generally different from the position of the first image area in the left eye image due to the horizontal spacing between the left and right images. Of course, for an object that does not exhibit a stereoscopic effect, the positions of the two image regions in the left and right images are the same.

In this step, the computer needs to ensure that the content involved in the second image area is the same as the content involved in the first image area, and the computer searches for the content involved in the first image area in the right eye image through image matching technology, and then in the content Upper, the second image area is determined according to the size of the first image area. Illustratively, FIG. 4e shows the second image area determined in the right eye image, and it can be seen that the two correspond to the same number, size and content as compared with the first image area in the left eye image. However, due to the horizontal spacing, the position of the second image area is slightly to the left relative to the first image area.

In step 350, in the first image region and the second image region corresponding to each other, select the same Pixels.

For the first image area and the second image area corresponding to each other in the left and right images (ie, the first image area and the second image area involving the same content in the left and right images), the computer selects the same pixel point from the two image areas respectively. Calculate the reference basis for the parallax parameter. The same pixel refers to a pixel having the same relative coordinate in the image region, for example, the fourth row and the fifth pixel in the first image region, and the fourth row and the fifth pixel in the second image region. Two pixels belong to the same pixel. Since the sizes of the first image area and the second image area are the same, the concept of using relative coordinates ensures that the selected pixels are the same. However, it should be noted that in practical applications, it is not appropriate to use the absolute coordinate concept in the left and right images to select pixel points, because there is a horizontal spacing between the first image area and the second image area, and the two are in the left and right images. The position is different. If the absolute coordinate concept of the relative image is used, the position of the selected two pixels in the first/second image area will be different.

In one example of this embodiment, the computer determined pixel points may be as shown in Figure 5a. In practical applications, it is also possible to select pixels at a particular location, such as a pixel on the four corners of the image area, a central pixel in the image area, or a pixel on the edge of the image area. The latter three cases are shown in Figures 5b, 5c and 5d, respectively.

In step 360, the coordinate difference of the pixel on the X-axis is calculated to obtain a parallax parameter.

After determining the pixel point, the computer calculates the coordinate difference value of the two pixel points on the X-axis, that is, obtains the parallax parameter corresponding to the pair of first/second image regions. The computer sequentially performs steps 350 and 360 for each pair of first/second image regions to obtain a plurality of parallax parameters corresponding to the number of first/second image regions.

In this embodiment, the computer can obtain the absolute coordinates of the two pixel points in the left and right images, and then subtract the X-axis component values of the two absolute coordinates to obtain the coordinate difference of the pixel points on the X-axis.

In step 370, the maximum parallax parameter and the minimum parallax parameter that exceed the reasonable range of the preset parallax are eliminated.

In practical applications, the parallax parameter has positive and negative points. This difference is used to reflect whether the second image area is shifted to the left or to the right relative to the first image area. Based on different reference standards, the leftward and rightward offsets visually represent the object protrusions and depressions, respectively, or the object depressions and protrusions, respectively.

In general, the absolute value of the parallax parameter reflects the stereoscopic degree of the object protruding or concave. Generally, the stereoscopic degree of the object in the stereoscopic source is limited, and the degree of convexity and concavity is too large to make the user's eyes feel uncomfortable. Based on this, when the maximum parallax parameter obtained in the above step is too large, or the minimum parallax parameter is too small (negative value), the calculation result of the parallax parameter is inaccurate, and this part needs to be regarded as The difference parameter is removed.

For the tradeoff of the maximum parallax parameter, the computer searches for all the parallax parameters whose value is greater than 0, and calculates the first parallax average value, and takes the preset multiple of the first parallax average value as the boundary condition for rejecting the maximum parallax parameter. If the maximum parallax parameter is less than a preset multiple of the first disparity average, the maximum disparity parameter is retained, otherwise the maximum disparity parameter is eliminated. After rejecting the maximum parallax parameter that does not meet the requirement, the computer further searches for the maximum parallax parameter in the remaining parallax parameters, and performs the tradeoff according to the above manner until there is no parallax parameter larger than the above boundary condition, and the computer will remain the parallax at this time. The parallax parameter with the largest value in the parameter is determined as the final maximum parallax parameter.

For the tradeoff of the minimum parallax parameter, the computer searches for all the parallax parameters whose value is less than 0, and outputs the second parallax average value, and takes the preset multiple of the second parallax average value as the boundary condition for eliminating the minimum parallax parameter. If the minimum disparity parameter is less than a preset multiple of the second disparity average, the minimum disparity parameter is retained, otherwise the minimum disparity parameter is eliminated. After rejecting the minimum parallax parameter that does not meet the requirement, the computer further finds the minimum parallax parameter in the remaining parallax parameters, and performs the tradeoff according to the above manner until the parallax parameter less than the above boundary condition does not exist, and the computer will remain the parallax at this time. The parallax parameter with the smallest value in the parameter is determined as the final minimum parallax parameter.

In practical applications, the above preset multiple can take "1" or "2". An example of this step is given below: The computer obtains six parallax parameters, "+3", "+6", "+30", "-1", "-2", and "-12". The parallax parameters "+3", "+6", and "+30" are averaged to obtain a first parallax average "+13", and then the first parallax average "+13" is multiplied by a preset multiple "2" Get "+26". Compared with "+26", the parallax parameter "+30" exceeds this value, and the computer rejects it. The parallax parameter "+6" does not exceed this value, and the computer determines it as the maximum parallax parameter. The computer then averages the parallax parameters "-1", "-2", and "-12" to obtain a second parallax average "-5", and then sets the second parallax average "-5" with a preset multiple "2". Multiply by "-10". Compared with "-10", the parallax parameter "-12" exceeds this value, and the computer rejects it. The parallax parameter "-2" does not exceed this value, and the computer determines it as the minimum parallax parameter.

Similar to step 330, step 370 also pertains to the optional steps in the flow shown in FIG. When step 370 is not performed, step 380 is directly executed after step 360 is performed; when step 370 is performed, the range of parallax variation obtained by the process shown in FIG. 3 is more accurate.

In step 380, a parallax variation range of the stereoscopic slice source is defined based on the obtained maximum parallax parameter and minimum parallax parameter.

In the above example, the final determined maximum parallax parameter is "+6" and the minimum parallax parameter is "-2", whereby the parallax variation range of the stereoscopic sheet source can be obtained as [-2, +6].

Optionally, as an implementation of the method shown in FIG. 1 , FIG. 2 , and FIG. 3 , the embodiment of the present application further provides an apparatus for acquiring a parallax parameter of a stereoscopic slice source. The device may be integrated in the form of software or hardware on the client side, or on the server side, to perform the calculation of the inspection parameters for the stereoscopic source provided by the local storage or the external database. As shown in FIG. 6, the apparatus includes: a reading unit 610, an area determining unit 620, a searching unit 630, a calculating unit 640, and a parameter determining unit 650;

The reading unit 610 is configured to read an image frame from the stereoscopic source file, the image frame including a left eye image and a right eye image;

The area determining unit 620 is configured to determine a plurality of first image areas in one of the left eye image and the right eye image;

The searching unit 630 is configured to search, in another of the left eye image and the right eye image, a plurality of second image regions corresponding to the plurality of first image region contents, the first image region and the second image region corresponding to each other Same size;

The calculating unit 640 is configured to separately calculate a horizontal distance between each of the first image regions and the second image region corresponding thereto, to obtain a plurality of parallax parameters;

The parameter determining unit 650 is configured to determine the maximum parallax parameter and the minimum parallax parameter to obtain a parallax variation range of the stereoscopic slice source.

Optionally, as shown in FIG. 7, the device further includes:

The extracting unit 660 is configured to: when there are multiple image frames in the stereoscopic source file, extract at least two image frames from the stereoscopic source file as sampling frames;

The calculating unit 640 is configured to obtain a parallax parameter of each sampling frame separately;

The parameter determining unit 650 is configured to determine a maximum parallax parameter and a minimum parallax parameter among the parallax parameters of all the sampling frames to obtain a parallax variation range of the stereoscopic slice source.

Optionally, as shown in FIG. 7, the extracting unit 660 includes:

The interval extraction sub-unit 6610 is configured to extract at least two sampling frames according to a preset sampling interval based on the ordering of the image frames in the stereoscopic source file, and the sampling interval includes the interval frame number and/or the interval duration; or

The random extraction sub-unit 6620 is arranged to randomly extract at least two sample frames by a random number generation algorithm.

Optionally, the area determining unit 620 is configured to determine a plurality of preset image size first regions in a preset position in one of the left eye image and the right eye image; wherein the preset position is randomly determined, or Calculate according to the size ratio of the image.

Optionally, as shown in FIG. 7, the device further includes:

The screening unit 670 is configured to perform the first image region after determining the plurality of first image regions The first image area in which the degree of dispersion of the pixel chromaticity is less than the preset degree of dispersion is excluded.

Optionally, as shown in FIG. 7, the screening unit 670 includes:

The first calculating sub-unit 6710 is configured to calculate a standard difference value of the three color components corresponding to the red, green and blue pixels in the first image region;

The second calculating subunit 6720 is configured to perform a summation calculation on the red standard deviation value, the green standard deviation value, and the blue standard deviation value to obtain a comprehensive standard deviation value;

The culling sub-unit 6730 is arranged to reject the first image region whose integrated standard deviation is less than the standard deviation threshold.

Optionally, the computing unit 640 is configured to:

Selecting the same pixel point in the first image area and the second image area corresponding to each other;

Calculate the coordinate difference of the pixel on the X-axis to obtain the parallax parameter.

Optionally, the parameter determining unit 650 is configured to reject the maximum parallax parameter and the minimum parallax parameter that exceed a reasonable range of the preset parallax.

Optionally, as shown in FIG. 7, the parameter determining unit 650 includes:

The first determining subunit 6510 is configured to calculate an average value of all the disparity parameters whose value is greater than 0, to obtain a first disparity average value, and retain the maximum parallax parameter when the maximum disparity parameter is less than a preset multiple of the first disparity average value, otherwise Eliminate the maximum parallax parameter;

The second determining subunit 6520 is configured to calculate an average value of all the disparity parameters whose value is less than 0, to obtain a second disparity average value, and when the minimum disparity parameter is greater than a preset multiple of the second disparity average value, retain the minimum parallax parameter, otherwise Eliminate the minimum parallax parameter.

The apparatus for acquiring the stereoscopic source parallax parameter provided by the embodiment of the present application can automatically read an image frame from the stereoscopic source file, and determine a plurality of identical and one-to-one correspondences in the left and right eye images in the image frame. An image area and a second image area respectively calculate a horizontal distance of each pair of the first and second image areas, obtain a parallax parameter corresponding to each pair of the first and second image areas, and then determine one from the plurality of parallax parameters The maximum parallax parameter and a minimum parallax parameter, thereby obtaining a parallax variation range of the stereoscopic sheet source. Compared with the related art, the processes of acquiring an image frame, determining an image area, and calculating a parallax parameter in the embodiment of the present application are all automatically completed by a computer, and the relevant personnel only need to provide a storage path of the stereoscopic source file to the computer. Since the manual operation is not involved, the embodiment of the present application can greatly shorten the time-consuming of obtaining parallax parameters and improve the processing efficiency, especially when processing stereoscopic video.

It should be noted that, for the foregoing apparatus for acquiring the stereoscopic source source parallax parameter, the functions of the unit or the subunit used in the embodiment of the present application can be implemented by a hardware processor.

Exemplarily, as shown in FIG. 8 , FIG. 8 is a schematic diagram of a physical structure of an electronic device for acquiring a stereoscopic source parallax parameter according to an embodiment of the present application. The electronic device may include: a processor 810 , A communication interface 820, a memory 830, and a bus 840, wherein the processor 810, the communication interface 820, and the memory 830 complete communication with each other through the bus 840. Communication interface 820 can be arranged to transmit and/or receive information over a communication network. The processor 810 can call the logic instructions in the memory 830 to perform any of the above methods for acquiring the stereoscopic source parallax parameters, including: reading an image frame from the stereoscopic source file, the image frame including the left eye image and the right eye image Determining a plurality of first image regions in the left eye image or the right eye image; searching a plurality of second image regions corresponding to the plurality of first image region contents in the other image, the first image regions corresponding to each other, and the first image region The two image regions have the same size; respectively calculate a horizontal distance between each of the first image regions and the second image region corresponding thereto, to obtain a plurality of parallax parameters; determine a maximum parallax parameter and a minimum parallax parameter, and obtain a parallax variation of the stereoscopic source range.

In addition, the logic instructions in the memory 830 described above may be implemented in the form of a software functional unit and sold or used as a stand-alone product, and may be stored in a computer readable storage medium. Based on the understanding, the technical solution of the embodiment of the present application may be embodied in the form of a software product stored in a storage medium, including a plurality of instructions for causing a computer device (which may be a personal computer, a server, Or a network device or the like) performs all or part of the steps of the method described in the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like. .

The embodiment of the present application further provides a non-transitory computer readable storage medium, wherein the non-transitory computer readable storage medium stores computer instructions for causing the computer to execute any one of the above A method of obtaining a stereoscopic source parallax parameter.

The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, ie may be located A place, or it can be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the embodiment.

Through the description of the above embodiments, those skilled in the art can clearly understand that the above embodiments can be implemented by means of software and a general hardware platform, and of course, by hardware. Based on such understanding, the above technical solution may be embodied in the form of a software product, which may be stored in a computer readable storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., including a plurality of fingers. The method described in some of the above-described embodiments or embodiments is performed to cause a computer device (which may be a personal computer, server, or network device, etc.). In addition, in the case of no conflict, various features in the embodiments of the present application may be arbitrarily combined with each other, and the technical solutions after the combination still fall within the protection scope of the present application.

Industrial applicability

The embodiment of the present application provides a method and a device for acquiring a parallax parameter of a stereoscopic slice source, which solves the problem of low efficiency when acquiring a parallax parameter of a stereoscopic slice source.

Claims (21)

1. A method for obtaining a parallax parameter of a stereoscopic sheet source, the method for applying to an electronic device includes:

   Reading an image frame from a stereoscopic source file, the image frame including a left eye image and a right eye image;

   Determining a plurality of first image regions in one of the left eye image and the right eye image;

   Searching a plurality of second image regions corresponding to the plurality of first image region contents in the other of the left eye image and the right eye image, the first image region and the second image region corresponding to each other Same size;

   Calculating a horizontal distance between each of the first image regions and the second image region corresponding thereto, respectively, to obtain a plurality of parallax parameters;

   The maximum parallax parameter and the minimum parallax parameter are determined to obtain a parallax variation range of the stereoscopic slice source.
2. The method according to claim 1, wherein when there are a plurality of image frames in the stereoscopic slice source file, at least two image frames are extracted from the stereoscopic slice source file as sampling frames;

   Obtaining parallax parameters for each sample frame separately;

   Among the parallax parameters of all the sampling frames, the maximum parallax parameter and the minimum parallax parameter are determined, and the parallax variation range of the stereoscopic slice source is obtained.
3. The method of claim 2, wherein the extracting at least two image frames from the stereoscopic source file as sample frames comprises:

   And extracting at least two sampling frames according to a preset sampling interval, where the sampling interval includes an interval frame number and/or an interval duration, or based on the sorting of the image frames in the stereoscopic source file; or

   At least two sampling frames are randomly extracted by a random number generation algorithm.
4. The method according to any one of claims 1 to 3, wherein the determining the plurality of first image regions in one of the left eye image and the right eye image comprises:

   Determining a plurality of preset image size first regions in a preset position in one of the left eye image and the right eye image; wherein the preset position is randomly determined, or according to an image size The scale calculation is determined.
5. The method of claim 1, after determining the plurality of first image regions, the method further comprising:

   The first image area is filtered to eliminate the first image area whose pixel chromaticity is less than a predetermined degree of dispersion.
6. The method according to claim 5, wherein the filtering the first image region to eliminate the first image region in which the degree of dispersion of the pixel chromaticity is less than a predetermined degree of dispersion comprises:

   Calculating, by the pixel points in the first image region, standard deviation values corresponding to three color components of red, green, and blue;

   Calculate the sum of the red standard deviation, the green standard deviation, and the blue standard deviation to obtain a comprehensive Standard difference;

   The first image region whose integrated standard deviation is less than the standard deviation threshold is rejected.
7. The method according to any one of claims 1 to 6, wherein the calculating the horizontal distance between each of the first image regions and the second image region corresponding thereto to obtain a plurality of parallax parameters comprises:

   Selecting the same pixel point in the first image area and the second image area corresponding to each other;

   A coordinate difference value of the pixel on the X-axis is calculated to obtain a parallax parameter.
8. The method of any one of claims 1 to 7, wherein said determining a maximum parallax parameter and a minimum parallax parameter comprises:

   The maximum parallax parameter and the minimum parallax parameter that exceed the reasonable range of the preset parallax are excluded.
9. The method according to claim 8, wherein the culling the maximum parallax parameter and the minimum parallax parameter beyond a reasonable range of the preset parallax comprises:

   Calculating an average of all the parallax parameters whose values are greater than 0, and obtaining a first parallax average value;

   If the maximum disparity parameter is less than a preset multiple of the first disparity average, retaining the maximum disparity parameter, otherwise rejecting the maximum disparity parameter;

   Calculating an average of all parallax parameters whose values are less than 0 to obtain a second parallax average;

   If the minimum disparity parameter is greater than a preset multiple of the second disparity average, the minimum disparity parameter is retained, otherwise the minimum disparity parameter is eliminated.
10. An apparatus for obtaining a parallax parameter of a stereoscopic sheet source, comprising:

    a reading unit configured to read an image frame from a stereoscopic sheet source file, the image frame including a left eye image and a right eye image;

    a region determining unit configured to determine a plurality of first image regions in one of the left eye image and the right eye image;

    a searching unit configured to search for a plurality of second image regions corresponding to the plurality of first image region contents in the other of the left eye image and the right eye image, and the first image regions corresponding to each other Same size as the second image area;

    a calculating unit configured to separately calculate a horizontal distance between each of the first image regions and the second image region corresponding thereto to obtain a plurality of parallax parameters;

    The parameter determining unit is configured to determine a maximum parallax parameter and a minimum parallax parameter to obtain a parallax variation range of the stereoscopic slice source.
11. The apparatus of claim 10 further comprising:

    The extracting unit is configured to: when there are multiple image frames in the stereoscopic source file, extract at least two image frames from the stereoscopic source file as sampling frames;

    The calculating unit is configured to obtain a parallax parameter of each sample frame separately;

    The parameter determining unit is configured to determine a maximum parallax parameter and a minimum parallax parameter among the parallax parameters of all the sampling frames to obtain a parallax variation range of the stereoscopic slice source.
12. The apparatus according to claim 11, wherein the extracting unit comprises:

    The interval extraction subunit is configured to extract at least two sampling frames according to a preset sampling interval based on the order of the image frames in the stereoscopic source file, where the sampling interval includes the interval frame number and/or the interval duration; or

    A random extraction sub-unit is arranged to randomly extract at least two sampling frames by a random number generation algorithm.
13. The apparatus according to any one of claims 10 to 12, wherein the area determining unit is configured to determine a plurality of pre-positions at preset positions in one of the left-eye image and the right-eye image The first image area of the size is set; wherein the preset position is randomly determined or determined according to the size ratio of the image.
14. The apparatus of claim 10 further comprising:

    The screening unit is configured to filter the first image region after determining the plurality of first image regions, and reject the first image region in which the degree of dispersion of the pixel chromaticity is less than a preset degree of dispersion.
15. The apparatus of claim 14, wherein the screening unit comprises:

    a first calculating subunit, configured to calculate a standard difference value of the three color components corresponding to the red, green, and blue pixels in the first image region;

    a second calculating subunit, configured to perform a summation calculation on the red standard deviation value, the green standard deviation value, and the blue standard deviation value to obtain a comprehensive standard deviation value;

    The culling subunit is set to reject the first image region where the integrated standard deviation is less than the standard deviation threshold.
16. The apparatus of any of claims 10-15, wherein the computing unit is configured to:

    Selecting the same pixel point in the first image area and the second image area corresponding to each other;

    A coordinate difference value of the pixel on the X-axis is calculated to obtain a parallax parameter.
17. The apparatus according to any one of claims 10-16, wherein the parameter determination unit is configured to reject the maximum parallax parameter and the minimum parallax parameter that exceed a reasonable range of the preset parallax.
18. The device according to claim 17, wherein the parameter determining unit comprises:

    a first determining subunit, configured to calculate an average value of all the disparity parameters whose value is greater than 0, to obtain a first disparity average value, when the maximum disparity parameter is less than a preset multiple of the first disparity average value, retaining the Maximum parallax parameter, otherwise the maximum parallax parameter is eliminated;

    a second determining subunit, configured to calculate an average value of all the disparity parameters whose value is less than 0, to obtain a second disparity average value, when the minimum disparity parameter is greater than a preset multiple of the second disparity average value, retaining the The minimum parallax parameter, otherwise the minimum parallax parameter is eliminated.
19. An electronic device comprising:

    At least one processor; and,

    a memory communicatively coupled to the at least one processor; wherein

    The memory stores instructions executable by the one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

    Reading an image frame from a stereoscopic source file, the image frame including a left eye image and a right eye image;

    Determining a plurality of first image regions in one of the left eye image and the right eye image;

    Searching a plurality of second image regions corresponding to the plurality of first image region contents in the other of the left eye image and the right eye image, the first image region and the second image region corresponding to each other Same size;

    Calculating a horizontal distance between each of the first image regions and the second image region corresponding thereto, respectively, to obtain a plurality of parallax parameters;

    The maximum parallax parameter and the minimum parallax parameter are determined to obtain a parallax variation range of the stereoscopic slice source.
20. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the acquisition of any one of claims 1-9 A method of stereoscopic source parallax parameters.
21. A computer program product comprising a computing program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions that, when executed by a computer, cause the computer to perform claims 1-9 A method of obtaining a stereoscopic source parallax parameter according to any one of the preceding claims.

Images