CN110858896A

CN110858896A - VR image processing method

Info

Publication number: CN110858896A
Application number: CN201810975482.0A
Authority: CN
Inventors: 孟宪民; 李小波; 赵德贤
Original assignee: BEIJING HENGXIN CAIHONG INFORMATION TECHNOLOGY Co Ltd
Current assignee: BEIJING HENGXIN CAIHONG INFORMATION TECHNOLOGY Co Ltd
Priority date: 2018-08-24
Filing date: 2018-08-24
Publication date: 2020-03-03
Anticipated expiration: 2038-08-24
Also published as: CN110858896B

Abstract

The application discloses a VR image processing method, which comprises the following steps: determining an image needing to be processed; selecting at least one focus area; determining a focus area profile of the focus area; performing fusion processing on the focus area and the background image according to the contour of the focus area; and transmitting the fused image to a VR head display. The application has reached the high definition part of guaranteeing to transmit the image and partly links up naturally with non-high definition, the technological effect of effectual promotion user experience's comfort level.

Description

VR image processing method

Technical Field

The application relates to the technical field of VR image transmission, in particular to a VR image processing method.

Background

The requirement for resolution in image transmission is higher and higher nowadays, especially in the VR image transmission field, because VR needs binocular output, there is a demand for double-size image output in transmission. According to the feasible scheme at present, if the transmitting end uses X264 coding compression and then uses h264 decoding at the receiving end, the compressed video stream data can be transmitted in real time, and the network transmission data volume can be effectively reduced. However, in the field of network transmission, there are situations that high-definition and ultra-high-definition videos are transmitted to cause delay aggravation and the possibility that the videos cannot be observed instantly, particularly, the transmission pressure of the 4K or 8K videos is higher, the higher the compression ratio of the corresponding video coding field is, the lower the definition is, the possibility that the videos can be transmitted instantly and watched by a client cannot be met, and the requirement of VR binocular output panoramic high-definition images cannot be met.

Disclosure of Invention

The application aims to provide a VR image processing method, which ensures that a high-definition part and a non-high-definition part of a transmission image are naturally linked, and effectively improves the comfort level of user experience.

In order to achieve the above object, the present application provides a VR image processing method, including the steps of: determining an image needing to be processed; selecting at least one focus area; determining a focus area profile of the focus area; performing fusion processing on the focus area and the background image according to the contour of the focus area; and transmitting the fused image to a VR head display.

Preferably, the focal region profile is determined using the Camshift algorithm.

Preferably, the method for determining the profile of the focal region specifically comprises: determining a first edge profile; determining a second edge profile; and storing the first edge outline and the second edge outline into a focus object file library.

Preferably, the second edge profile is determined from the first edge profile inward contraction calculation.

Preferably, the inward shrinkage calculation formula of each pixel on the first edge profile is as follows: the shrunk pixel value is (first contour pixel value-center point) × shrinkage factor.

Preferably, the shrinkage factor is 0.6 to 0.9.

Preferably, the shrinkage factor is 0.8.

Preferably, the transmission method of the fused image is as follows: transmitting the focus area in high definition and synchronizing the background image in non-high definition.

The beneficial effect that this application realized is as follows:

(1) the high-definition part and the non-high-definition part of the transmission image are naturally connected, and the comfort level of user experience is effectively improved.

(2) The technical effects of reducing the data volume of network transmission, avoiding transmission delay and meeting the requirement that a client watches a high-definition video can be achieved.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the present application, and other drawings can be obtained by those skilled in the art according to the drawings.

FIG. 1 is a flow diagram of one embodiment of a VR image processing method;

FIG. 2 is a flow chart of one embodiment of a method for determining a focal region profile.

Detailed Description

The technical solutions in the embodiments of the present invention are clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

As shown in fig. 1, the present invention provides a VR image processing method, including the following steps:

s110: and playing the video and determining the images needing to be processed in the video.

Specifically, the staff plays the video through the video playing tool, in the playing process, the staff pauses the video at a position where a focus needs to be set, and the frame of image displayed on the video playing tool after pausing is the image needing to be processed.

S120: at least one focal region is selected from the image to be processed.

Specifically, the staff performs framing on at least one region in the image by using the mouse, the framed region is a focus region, and after the focus region is determined, S130 is performed.

S130: a focal region profile of the focal region is determined.

Further: as an embodiment, the focus area profile is determined using the CamShift algorithm (continuous adaptive mean-Shift algorithm).

Specifically, a Camshift algorithm (continuous Adaptive Mean-Shift algorithm) is an improved algorithm for the Meansshift algorithm, and can adjust the size of a Search Window in real time along with the change of the size of a tracking target in the tracking process, make all frames of a video image into the Meansshift algorithm, use the result of the previous frame (i.e. the centroid and size of the Search Window) as the initial value of the Search Window of the Meansshift algorithm of the next frame to find the optimal iteration result, use the centroid in the optimal iteration result in the zero-order moment M00 and the first-order moment M10 of the Window, and calculate the size of the zero-order moment in M01, where the zero-order moment is the integral of all pixels in the Search Window, and the integral of all pixels in the Search Window is equal to the size of the Search Window, and the size of the Search Window is the focus area.

Further, the operations of Camshift and MeanShift are performed on the back projection image, a histogram of the H component of the target region (i.e., the focal region) is generated through back projection calculation, and the pixel value of each pixel point on the target region is replaced by the value of the histogram bin corresponding to the bin where the current pixel value is located. After reverse projection, most pixel values of a search window of a target area are normalized to be a maximum value of 255, a threshold value is determined according to a test result in use, if the calculated zero-order moment is larger than the threshold value, the target is paved on the whole search window, whether the target area exists in an area outside the search window needs to be judged, and if the target area exists, the size of the search window needs to be increased; if the zeroth order moment is less than the threshold, the search window needs to be reduced in size. Therefore, when the size of the target changes, the size of the search window can be adjusted in a self-adaptive mode through the Camshift algorithm, the target area is tracked, and the outline of the focus area is obtained.

Further, as shown in fig. 2, as another embodiment, the method for determining the focal region profile includes the following steps:

s210: a first edge profile is determined.

Specifically, the worker uses a mouse to draw a first edge contour on the image to be processed, and the area surrounded by the first edge contour is larger than the focus area to be framed.

S220: a second edge profile is determined.

Specifically, a second edge profile is computed from the first edge profile inward scaling.

Specifically, all paths of all pixel points on the track are calculated according to the moving track of the mouse when the staff draws the first edge profile. And calculating the average value of all paths to obtain the central point. And (3) inwards shrinking all the pixel points to calculate to obtain a second edge profile, wherein the calculation formula is as follows:

pixel value after shrinkage (first contour pixel value-center point) x shrinkage factor

The value range of the shrinkage coefficient is 0.6-0.9, the value of the shrinkage coefficient is a value obtained by a user according to real-time test, and preferably, the value of the coefficient is 0.8. The connecting line of the shrunk pixel values of all the pixels on the image to be processed is the second edge contour.

And S230, storing the first edge contour and the second edge contour into a focus object file library.

Specifically, the data such as the image size, the focus area, the first edge profile, the second edge profile, and the like of each frame of image are transmitted to the focus object file library for storage.

S140: and carrying out fusion processing on the focus area and the background image according to the contour of the focus area.

When the video is played, the computer reads the data of the focus object, the first edge contour, the second edge contour, the image size and the like corresponding to each frame of image in the played video from the focus object file library. And synchronously transmitting the definition data, the first edge profile, the second edge profile and the video data to the VR client. After receiving the data, the VR client renders a non-high-definition image as a background image by using the original image, then renders a layer of high-definition image of a focus area in the corresponding focus area on the background image, and gradually fuses to the second edge outline according to the first edge outline to form a fused area, so that the processed image is ensured to have comfortable visual sense when displayed. After the fusion process of the focus area and the background image is completed, S150 is performed.

S150: and transmitting the fused image to a VR head display.

Further, the transmission method of the fused image is as follows: transmitting the focus area in high definition and synchronizing the background image in non-high definition.

The beneficial effect that this application realized is as follows:

While the preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all alterations and modifications as fall within the scope of the application. It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.

Claims

1. A VR image processing method is characterized by comprising the following steps:

determining an image needing to be processed;

selecting at least one focus area;

determining a focus area profile of the focus area;

performing fusion processing on the focus area and the background image according to the contour of the focus area;

and transmitting the fused image to a VR head display.

2. The VR image processing method of claim 1, wherein the focus area profile is determined using a CamShift algorithm.

3. The VR image processing method of claim 1, wherein the method of determining the focus area profile is specifically:

determining a first edge profile;

determining a second edge profile;

and storing the first edge outline and the second edge outline into a focus object file library.

4. The VR image processing method of claim 3, wherein the second edge profile is determined from a first edge profile inward scaling calculation.

5. The VR image processing method of claim 4, wherein the inward shrinkage for each pixel on the first edge profile is calculated by: the shrunk pixel value is (first contour pixel value-center point) × shrinkage factor.

6. The VR image processing method of claim 5 wherein the shrinkage factor is 0.6 to 0.9.

7. The VR image processing method of claim 6, wherein the contraction coefficient is 0.8.

8. The VR image processing method of claim 1, wherein the fused image is transmitted by: transmitting the focus area in high definition and synchronizing the background image in non-high definition.