CN105979224A - Head mount display, video output device and video processing method and system - Google Patents

Head mount display, video output device and video processing method and system Download PDF

Info

Publication number
CN105979224A
CN105979224A CN201610466833.6A CN201610466833A CN105979224A CN 105979224 A CN105979224 A CN 105979224A CN 201610466833 A CN201610466833 A CN 201610466833A CN 105979224 A CN105979224 A CN 105979224A
Authority
CN
China
Prior art keywords
video
head
area information
data
output device
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201610466833.6A
Other languages
Chinese (zh)
Inventor
赵文慧
徐建军
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Qingdao Goertek Co Ltd
Original Assignee
Qingdao Goertek Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Qingdao Goertek Co Ltd filed Critical Qingdao Goertek Co Ltd
Priority to CN201610466833.6A priority Critical patent/CN105979224A/en
Publication of CN105979224A publication Critical patent/CN105979224A/en
Priority to PCT/CN2016/114042 priority patent/WO2017219652A1/en
Pending legal-status Critical Current

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/18Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/167Position within a video image, e.g. region of interest [ROI]

Abstract

The invention discloses a head mount display, a video output device and a video processing method and system. The head mount display comprises an image collection unit which photographs head and eye images of a user in real time; an image processing unit which calculates the eyeball movement track of the user according to the head and eye images, thereby obtaining the information of an important region in a display screen currently watched by eyeballs; a data transmission unit which sends the information of the important region to the video output device, thereby enabling the device to carry out regional compressed coding on a current frame image according to the information of the important region, wherein the data transmission unit also receives the frame image after the regional compressed coding sent by the device; a video decoding unit which carries out regional decoding on the frame image after the regional compressed coding; and a display unit which displays and outputs the decoded frame image, thereby displaying the data in the frame image corresponding to the important region in a high definition mode, wherein the data corresponding to other regions is displayed in a weakened mode. According to the head mount display, the video output device and the video processing method and system, a frame video is divided into regions by use of the watching positions of the eyeballs of the user, and the user experience of the head mount display is optimized.

Description

A kind of head-mounted display, picture output device and method for processing video frequency, system
Technical field
The present invention relates to wear Display Technique field, be specifically related to a kind of head-mounted display, video frequency output sets Standby and method for processing video frequency, system.
Background technology
At present, head-mounted display achieves development at full speed, head-mounted display (Head Mount Display, It is called for short HMD) video obtained frame by frame from the picture output device such as smart mobile phone or computer can be shown Out watch for wearer.Fig. 1 is the transmission of video scene schematic diagram of prior art head-mounted display, ginseng See Fig. 1, owing to the transmission quantity of video data is relatively big, so needing to be compressed video data processing. As a example by smart mobile phone, video transmitting procedure includes: mobile phone uses Coding Compression Algorithm to each in video Frame is compressed coding, then is sent to wear by HDMI connecting line by the frame of video after compressed encoding Display, head-mounted display shows after being decoded.
Video compressing and encoding method has multiple, on a large scale, can be divided into lossless compression-encoding and damage Compression.Lossless compression-encoding advantage is: the video after lossless compression-encoding is permissible decoding when Recovering completely, such as: Huffman encodes, Run-Length Coding, and count coding, shortcoming is the data of data transmission Amount is the biggest, takies the transfer resource transmission time that is more and that spend longer.And lossy compression method coding Algorithms most in use has: vector quantization coding, it was predicted that coding, model image coding etc., this compressed encoding is Irreversible, after compression data and decode after and initial data between there is certain difference, but permissible Obtain higher compression ratio.
From the foregoing, it will be observed that the video play in the head-mounted display of prior art is all according to the most so Whole frame carry out lossy compression method or lossless compress after incoming, video transmission efficiency and data integrity cannot Taking into account, Consumer's Experience is the best.
Summary of the invention
The invention provides a kind of head-mounted display, picture output device and method for processing video frequency, system, It is all after whole frame carries out lossy compression method or lossless compress to solve broadcasting video in prior art head-mounted display Incoming, video transmission efficiency and integrity of video cannot be taken into account, the problem that Consumer's Experience is the best.
In order to achieve the above object, the technical scheme is that and be achieved in that:
According to an aspect of the invention, it is provided a kind of head-mounted display, this head-mounted display includes figure As collecting unit, graphics processing unit, data transmission unit, video decoding unit and display unit:
Image acquisition units, for the head of captured in real-time user and eyes image and by the head of user and Eyes image is sent to graphics processing unit;
Graphics processing unit, for calculating the fortune of user's eyeball according to the user's head received and eyes image Dynamic track, obtains the key area information of user's eyeball current fixation display screen, is believed this key area Breath is sent to data transmission unit;
Data transmission unit, for sending to the video being connected with head-mounted display defeated by key area information Go out equipment so that the current frame image in video to be output is believed by picture output device according to this key area Breath carries out subregion compressed encoding process;And, for receiving the subregion pressure that picture output device sends Two field picture after contracting coded treatment, and this two field picture is sent to video decoding unit;
Video decoding unit, the two field picture after processing the subregion compressed encoding received utilizes pressure The inverse algorithm reducing the staff code carries out subregion decoding process, and decoded two field picture is sent to display unit;
Display unit, is used for showing the decoded two field picture of output so that with key area in this two field picture Corresponding data high definition shows, the data reduction display that remaining region is corresponding.
Alternatively, key area is with the coordinate figure of user's eyeball fixes as the center of circle on display screen, with in advance Measured length is the region of radius.
Alternatively, data transmission unit includes: area information sends subelement and video data receives son list Unit;
Area information sends subelement, for information transmission in key area be extremely connected with head-mounted display Picture output device;
Video data receives subelement, for receiving at the subregion compressed encoding that picture output device sends Two field picture after reason;
Area information sends subelement and video data reception subelement is wireless transport module or wire transmission Module.
Alternatively, graphics processing unit, it is additionally operable to this key area information is sent to video decoding unit;
Video decoding unit, specifically for the two field picture profit after processing the subregion compressed encoding received Inverse algorithm with compressed encoding and the key area information from graphics processing unit acquisition carry out subregion Decoding process.
According to another aspect of the present invention, it is provided that a kind of picture output device, picture output device bag Include:
Communication unit, for receiving the weight of user's eyeball current fixation display screen that head-mounted display sends Point area information, is sent to territory, video partition processing unit by this key area information;
Territory, video partition processing unit, is used for the present frame in video to be output according to key area information Image division is high definition region and atenuator region, and the data in high definition region are not compressed coding or Carry out lossless compression-encoding process, the data in atenuator region are carried out lossy compression method coded treatment, wherein, Attach most importance to region corresponding to region in high definition region;
Communication unit, is additionally operable to the two field picture after being processed by subregion compressed encoding and sends to head-mounted display.
Alternatively, picture output device is smart mobile phone or computer, and in this smart mobile phone or computer, storage has Video data.
Alternatively, communication unit includes: area information receives subelement and video data sends subelement;
Area information receives subelement, for receiving user's eyeball current fixation of head-mounted display transmission The key area information of display screen;
Video data sends subelement, and the two field picture after being processed by subregion compressed encoding sends to head Wear display;
Area information receives subelement and video data transmission subelement is wireless communication module or wire communication Module.
According to a further aspect of the invention, it is provided that a kind of method for processing video frequency, the method includes:
The head of captured in real-time head-mounted display user and eyes image;
Head and the movement locus of eyes image calculating user's eyeball to user, obtains user's eyeball current Watch the key area information of display screen attentively;
Key area information is sent to the picture output device being connected with head-mounted display so that video is defeated Go out equipment and the current frame image in video to be output is carried out subregion compression volume according to this key area information Code processes;And, receive the two field picture after the subregion compressed encoding process that picture output device sends;
Two field picture after processing the subregion compressed encoding received utilizes the inverse algorithm of compressed encoding to carry out Subregion decoding process;
Two field picture after display output decoding process so that data corresponding with key area in this two field picture High definition shows, the data reduction display that remaining region is corresponding.
According to a further aspect of the invention, it is provided that a kind of processing system for video, this system includes: as The head-mounted display of one aspect of the invention, and such as the picture output device of another aspect of the invention; Wired or wireless data cube computation is set up between head-mounted display and picture output device.
The invention has the beneficial effects as follows: the technical scheme of the embodiment of the present invention, by Real-time Collection user's Head and eyes image, and calculate user's eyeball current fixation display screen based on the image collected Key area, is then sent to the video frequency output being connected with head-mounted display by the information of this key area Equipment so that picture output device can be according to this key area information, to present frame in video to be output Image carries out subregion compressed encoding: the data of the key area that those correspond to user's viewing are carried out nothing Damage compression, and the video data in remaining region is carried out lossy compression method.Picture output device is again by subregion Frame of video after territory processes is sent to head-mounted display, and head-mounted display shows defeated after carrying out subregion decoding Go out, thus by the lossless compress of video in key area that user is watched or do not compress, it is ensured that regard The high definition of frequency shows and the non-destructive of data, further through the data acquisition lossy compression method to remaining region, subtracts Little data volume makes video can be transferred to head-mounted display wirelessly, and improves data and pass Defeated efficiency, saves the transmission time, optimizes user's experience.
Accompanying drawing explanation
Fig. 1 is head-mounted display and picture output device transmission of video schematic diagram in prior art;
Fig. 2 is the structured flowchart of a kind of head-mounted display of one embodiment of the invention;
Fig. 3 is the subregional schematic diagram of two field picture of one embodiment of the invention;
Fig. 4 is the structured flowchart of the picture output device of one embodiment of the invention;
Fig. 5 is the structured flowchart of the processing system for video of one embodiment of the invention;
Fig. 6 is the schematic flow sheet of the method for processing video frequency of one embodiment of the invention.
Detailed description of the invention
The design concept of the present invention is: be all one whole for the video play in prior art head-mounted display Frame, a whole frame compression process after incoming, result in the problem that Consumer's Experience is the best.The present invention implements Example proposes movement locus based on head-mounted display wearer's eyeball and calculates wearer and watch display screen The key area of curtain, and this key area information is sent to picture output device so that video frequency output sets It is passed to wear display for after frame of video is carried out subarea processing according to the key area information dynamically obtained In device, process such that it is able to realize the lossy compression method of data in atenuator region so that video can pass through Wireless way for transmitting is to head-mounted display, and high definition area video does not processes or lossless compress processes, both Reduce data volume and also ensure that the integrity of video data in key area simultaneously.
Embodiment one
Fig. 2 is the structured flowchart of a kind of head-mounted display of one embodiment of the invention, sees Fig. 2, should Head-mounted display 100 includes: image acquisition units 101, graphics processing unit 102, data transmission unit 103, video decoding unit 104 and display unit 105:
Image acquisition units 101, for the head of captured in real-time user and eyes image and by the head of user Portion and eyes image are sent to graphics processing unit 102;
Graphics processing unit 102, for calculating user's eyeball according to the user's head received and eyes image Movement locus, obtain the key area information of user's eyeball current fixation display screen, by this emphasis district Domain information is sent to data transmission unit 103;
Data transmission unit 103, for sending key area information to regarding of being connected with head-mounted display Frequently outut device so that picture output device to the current frame image in video to be output according to this emphasis district Domain information carries out subregion compressed encoding process;And, for receiving the subregion that picture output device sends Two field picture after the compressed encoding process of territory, and this two field picture is sent to video decoding unit 104;
Video decoding unit 104, the two field picture profit after the subregion compressed encoding received is processed Carry out subregion decoding process by the inverse algorithm of compressed encoding, decoded two field picture is sent to display Unit 105;
Display unit 105, is used for showing the decoded two field picture of output so that with emphasis in this two field picture Data high definition in the region that region is corresponding shows, the data in remaining region weaken display.
By the head-mounted display shown in Fig. 2, the head of Real-time Collection wearer and eyes image, and base Calculate the key area of wearer's current fixation in the image collected, then this key area is believed Breath is sent to picture output device so that picture output device regards to be transmitted according to this key area information A two field picture in Pin carries out subregion compressed encoding process and is sent to head-mounted display, by wearing display Device shows after being decoded.Owing to a two field picture can be carried out territory, dynamic partition compressed encoding thus with existing There is the processing mode of a whole frame in technology to compare, both can guarantee that video high definition in the key area that user watches Display and the non-destructive of data, can improve again data transmission efficiency, saves the transmission time.
Embodiment two
Fig. 3 is the subregional schematic diagram of two field picture of one embodiment of the invention.The present embodiment combines figure Two field picture subregion structural representation shown in 3, decoding and display to this head-mounted display carry out emphasis and say Bright, other guide sees other embodiments of the present invention.
The present embodiment first passes through in head-mounted display arrange image acquisition units collection user head and The image of eye.Here image acquisition units can be photographic head, and photographic head may be mounted at head Wear in the space between the display screen of display and user's eye, by photographic head captured in real-time user's Head and eyeball obtain real time imaging.
It should be noted that in the concept that ocular movement is followed the tracks of, the main movement of eyeball has three kinds of forms: Watch attentively, beat and smooth and trail tracking.Research conclusion according to prior art understands oculomotor angle Scope is about 18 degree, is accomplished by head movement more than 12 degree and completes to coordinate it.So to eye The when that ball being followed the tracks of, head to be considered and two factors of eyeball.Based on this, the present embodiment utilizes shooting Head gathers head and the eyes image of user.The original image that photographic head dynamic acquisition is watched to human eye, sends out Give graphics processing unit.
Original image is processed by graphics processing unit according to tracing of human eye algorithm, calculates user's eyeball Movement locus, and determine the key area that user's eyeball is watched, then by key area information (such as circle The coordinate figure of the heart and radius) it is transferred to picture output device so that and picture output device is compiled in video compress During code, territory, video partition is processed.Graphics processing unit carries out the basic fundamental of human eye sight tracking and can be divided into Hardware and software.Method based on hardware mainly user brings the special helmet or uses head Fixed support obtains the track of eyeball, but the method is not convenient and also error is bigger.With software as base The eye tracking technology of plinth is to obtain eye and the image of face of people first with video camera, is reprocessed by image After crossing, calculate the track of eyeball, and transfer data to video processing equipment, thus control video Process and display.I.e. have employed software mode in the present embodiment, graphics processing unit uses image procossing core Sheet, and according to image processing algorithm and Technical Follow-Up ocular movement.
Here key area is with the coordinate figure of user's eyeball fixes as the center of circle on display screen, with predetermined The region of a length of radius.In head-mounted display, when user watches display screen, eyeball is corresponding aobvious The upper position coordinates of display screen curtain, then can observe the empirical value of regional extent according to eyeball and determines one Individual radius, i.e. can determine that the emphasis viewing areas of eyeball.Here position coordinates is display on-screen pixels The coordinate of point.The display resolution of such as head-mounted display is 1680 × 1050, i.e. horizontal direction has 1680 Individual pixel, vertical direction has 1050 pixels, then the position coordinates of eyeball fixes can be The 800th pixel in (800,500), i.e. horizontal direction and the 500th pixel of vertical direction determine Position coordinates, after being calculated the position coordinates that eyeball is corresponding, in conjunction with the length of radius, the most available Key area.
Seeing Fig. 3, in the present embodiment, the key area illustrated is the border circular areas of dotted line limit, Key area is high definition region when two field picture shows, the region except high definition region in two field picture is all It it is atenuator region.
Owing to display resolution frame sign each with incoming video is consistent, so head-mounted display After calculating the key area of user's eyeball current fixation, key area information is sent to video frequency output and sets Standby, a two field picture can be divided into corresponding high definition according to key area information by such picture output device Region, and using the region outside high definition region as atenuator region, carry out dynamic subarea processing.
In the present embodiment, data transmission unit includes: area information sends subelement and video data connects Receive subelement;Area information sends subelement, for key area information being sent extremely and head-mounted display The picture output device connected;Video data receives subelement, for receiving what picture output device sent Two field picture after the compressed encoding process of subregion;Area information sends subelement and video data receives son list Unit is wireless transport module or wire transmission module.
It should be noted that due in prior art frame of video be essentially all the process side according to a whole frame Formula, this mode relative data amount is relatively big, causes the transmission of video approach of existing main flow head-mounted display It is all by wire transmission mode, as it is shown in figure 1, pass through between picture output device and head-mounted display HDMI (High Definition Multimedia Interface is called for short HDMI) is even Wiring connects, and passes to head by HDMI data wire after the video data whole frame compression in picture output device Wear display.This data transfer mode needs to carry HDMI connecting line, and if in order to improve biography Defeated efficiency and use whole frame lossy compression method mode be compressed process, then can affect integrity and the stream of video Smooth degree, and use lossless compress to ensure video data integrity, data transmission efficiency can be sacrificed again With the transmission time.
Send subelement for area information in this situation, preferably the present embodiment and video data receives Subelement is wireless transport module, such as, and WiFi wireless transport module.So can with omitted data line, And by calculating the key area information that user watches attentively, then this key area information is issued and regard Frequently outut device makes subregion compressed encoding process, it is ensured that video lossless in key area, remaining Area data lossy compression method.On the one hand, due in a frame data in remaining region by lossy compression method, thus Substantially reduce data volume so that video wireless transmission is possibly realized.On the other hand, owing to ensure that weight The non-destructive of some area video, also makes subsequent decoding can be entered by this key area video data in processing The lossless reduction of row, does not affect the integrity of video.
Video decoding unit is decoded for convenience, and in the present embodiment, graphics processing unit is additionally operable to this Key area information is sent to video decoding unit, and video decoding unit is specifically for the subregion received Two field picture after territory compressed encoding processes utilizes the inverse algorithm of compressed encoding and obtains from graphics processing unit The key area information taken carries out subregion decoding process.
Concrete, picture output device is when carrying out the compressed encoding of a two field picture, according to head-mounted display The key area information sent carries out territory, dynamic partition compressed encoding, and the two field picture after coding is sent to head Wear display, the data transmission unit of head-mounted display receive coding after two field picture after be sent to video Decoding unit.Video decoding unit is when decoding, according to the emphasis district dynamically obtained from graphics processing unit The coding information that domain information and coded data carry is decoded, and coded data here carries Coding information may include that high definition area information, and data in high definition region and encryption algorithm thereof are (e.g., Huffman encodes), and atenuator region information, in atenuator region, data and encryption algorithm thereof are (such as vector quantity Change) etc..Owing to video decoding unit has got key area information from graphics processing unit, and encode The high definition area information of rear data is consistent with key area information, so video decoding unit can be known Data in high definition region, road are the data that the key area of user's eyeball current fixation is corresponding, then may be used To use the inverse algorithm of key area data encoding algorithm to be decoded, thus it is corresponding to obtain key area Video data.
Same, video decoding unit according to the key area information obtained it will also be appreciated that frame figure after Bian Ma The atenuator region of picture, and be decoded obtaining according to the inverse algorithm of data encoding algorithm in this atenuator region The data that atenuator region is corresponding, are then sent to the two field picture that decoding obtains display unit and carry out showing defeated Go out, it is achieved the data high definition in region corresponding with key area in this two field picture shows, remaining region Data weaken display.
Embodiment three
Fig. 4 is the structured flowchart of the picture output device of one embodiment of the invention, emphasis in the present embodiment Structure and work process to picture output device illustrate, and other guide sees other realities of the present invention Execute example.
Seeing Fig. 4, this picture output device 400 includes:
Communication unit 401, for receiving user's eyeball current fixation display screen that head-mounted display sends Key area information, this key area information is sent to territory, video partition processing unit 402;
Territory, video partition processing unit 402, was used for working as in video to be output according to key area information Prior image frame is divided into high definition region and atenuator region, and is not compressed the data in high definition region compiling Code or carry out lossless compression-encoding process, the data in atenuator region are carried out lossy compression method coded treatment, Wherein, attach most importance to region corresponding to region in high definition region;
Communication unit 401, is additionally operable to the two field picture after being processed by subregion compressed encoding and sends aobvious to wearing Show device.
In the present embodiment, picture output device 400 is smart mobile phone or computer, this smart mobile phone or computer Middle storage has video data.
In the present embodiment, communication unit 401 includes: area information receives subelement and video data sends son Unit;Area information receives subelement, for receiving user's eyeball current fixation that head-mounted display sends The key area information of display screen;Video data sends subelement, for by subregion compressed encoding Two field picture after process sends to head-mounted display;Area information receives subelement and video data sends son Unit is wireless communication module or wire communication module.
Concrete, area information receives subelement and the region letter in the data transmission unit of head-mounted display Breath sends subelement and connects, it is achieved the transmission of key area information.Video data sends subelement and wears Video data in the data transmission unit of display receives subelement and connects, it is achieved the transmission of video data. It addition, the transmission means that area information reception subelement sends subelement with area information should be consistent, example As, both of which uses wireless transmission method to carry out data transmission.And video data sends subelement and regards Frequently the transmission means of data receiver subelement should also be as unanimously.
In the present embodiment, the data in high definition region are not compressed by territory, video partition processing unit 402 Encode or carry out lossless compression-encoding process, the data in atenuator region are carried out lossy compression method coded treatment Time coding standard be H.264.H.264 Video coding advantage is: low bit-rate, high quality graphic, appearance Wrong ability is strong, network availability strong, and the advantage of its maximum is to have the highest data compression ratio, same Under conditions of picture quality, compression ratio H.264 is more than 2 times of MPEG-2, is MPEG-4 1.5~2 times.H.264 while there is high compression ratio, also have an image that high-quality is smooth, just because of To this end, the video data after H.264 standard code compresses, required in network transmission process Bandwidth is less, the most more economically.
The structure of picture output device as shown in Figure 4 and work process understand, and the video of the present embodiment is defeated Go out equipment 400 receive head-mounted display by eye follow the tracks of user that is that calculate in real time and that send watch Key area information after, the territory, two field picture dynamic partition of video to be output is compressed, this two field picture is pressed (high definition region is corresponding with key area with atenuator region to be divided into high definition region according to key area information Region, atenuator region is the region in a two field picture beyond high definition region), then, the number to high definition region According to using lossless compression-encoding algorithm to carry out lossless compression-encoding, or it is not compressed encoding to ensure to regard The non-destructive of frequency evidence;And the data acquisition for atenuator region processes by lossy compression method, thus reduce data Transmission quantity, improves efficiency of transmission, saves the transmission time.
It addition, picture output device 400 has H.264 regarding of the highest data compression ratio by employing Frequency coding, decreases data volume, saves data transfer bandwidth.Further, communication unit 401 can be adopted Realize video wireless transmission with communication, eliminate the data wires such as HDMI, thus be greatly promoted Consumer's Experience.
Embodiment four
Fig. 5 is the structured flowchart of the processing system for video of one embodiment of the invention, sees Fig. 5, and this regards Frequently processing system 500 includes: head-mounted display 100, and picture output device 400, head-mounted display Wired or wireless data cube computation is set up between 100 and picture output device 400.
Below as a example by setting up wireless data connection between head-mounted display 100 and picture output device 400 Processing system for video is illustrated.
In the present embodiment, the structure of processing system for video 500 is as it is shown in figure 5, picture output device 400 Can be the input equipment such as mobile phone and computer, effect be to provide video frequency output;
Image acquisition units in head-mounted display 100, takes use by wearer's head and eye photo Carry out eye tracking process later;Here image acquisition units can be photographic head;
The image that image acquisition units is collected by the graphics processing unit in head-mounted display 100 processes Calculate the key area of wearer's eyeball fixes, issue data transmission unit;
The data transmission unit of head-mounted display 100 includes: area information sends subelement and video data Receiving subelement, concrete, area information sends the communication unit of subelement and picture output device 400 Middle area information receives subelement and sets up wireless connections, key area information is sent to area information and receives Subelement;
The area information of picture output device 400 receives subelement by the real-time emphasis of user's eyeball fixes Area information is sent to territory, video partition processing unit;
After territory, the video partition processing unit of picture output device 400 receives new key area information, root According to this key area information, to the two field picture zoning that will export, i.e. a two field picture is divided For high definition region and atenuator region, high definition region is corresponding with key area, atenuator region be in a frame except The region in high definition region.Then, the data in high definition region are carried out lossless compression-encoding, to atenuator region Data carry out lossy compression method coding, compressed encoding here uses H.264 standard.Picture output device Video data after coding is sent to video data transmitting and send son single by territory, the video partition processing unit of 400 Unit, is sent the video data in the subelement data transmission unit with head-mounted display 100 by video data Receive subelement and set up wireless connections, thus the two field picture after the compressed encoding of subregion is wirelessly transmitted to head Wear display 100.
After the video data of head-mounted display 100 receives the two field picture that subelement receives, it is sent to video Decoding unit, video decoding unit is come video solution by the inverse algorithm of video compress after receiving two field picture Code, is sent to display unit by decoded image.Display unit carries out display output to two field picture.
By the way, the video output system of the offer of the present embodiment, wireless mode can be used to exist Transmitting video data between picture output device and head-mounted display, is greatly improved Consumer's Experience, and Video processing when, it is determined by the emphasis viewing areas of user, in emphasis viewing areas, depending on Frequently Lossless transport, at remaining area video Weakening treatment.So it is possible not only to reach key area video high Clearly also can guarantee that video compress efficiency of transmission.
It addition, other structures of head-mounted display 100 of including of this processing system for video 500 and work Process, can be found in the description of previous embodiment one.The video frequency output that this processing system for video 500 includes Other structures of equipment 400 and work process, can be found in the description of previous embodiment three, the most superfluous State.
Embodiment five
Fig. 6 is the schematic flow sheet of the method for processing video frequency of one embodiment of the invention, sees Fig. 6, and this regards Frequently processing method comprises the steps:
Step S610, the head of captured in real-time head-mounted display user and eyes image;
Step S620, head and the movement locus of eyes image calculating user's eyeball to user, used The key area information of family eyeball current fixation display screen;
Step S630, sends key area information to the picture output device being connected with head-mounted display, Current frame image in video to be output is carried out point by picture output device according to this key area information Area compresses coded treatment;And, after receiving the subregion compressed encoding process that picture output device sends Two field picture;
Step S640, the two field picture after processing the subregion compressed encoding received utilizes the inverse of compressed encoding Subregion decoding process is carried out to algorithm;
Step S650, the two field picture after display output decoding process so that with key area in this two field picture Corresponding data high definition shows, the data reduction display that remaining region is corresponding.
It should be noted that this method for processing video frequency of the present embodiment is and the work of aforementioned head-mounted display Making process corresponding, therefore, in the present embodiment, method for processing video frequency realizes not description in step Content may refer to the related description in the present invention aforementioned head-mounted display embodiment, repeat no more here.
In sum, the technical scheme of the embodiment of the present invention, by head and the eye of Real-time Collection user Image, and the key area of user's eyeball current fixation display screen is calculated based on the image collected, Then the information of this key area is sent to the picture output device being connected with head-mounted display so that Current frame image in video to be output can be carried out point by picture output device according to this key area information Area compresses encodes: the data of the key area that those correspond to user's viewing are carried out lossless compress, and Video data in remaining region is carried out lossy compression method.Picture output device is again by after subarea processing Frame of video is sent to head-mounted display, and head-mounted display shows output after carrying out subregion decoding, thus logical Cross in the key area to user's viewing the lossless compress of video or do not compress, it is ensured that the high definition of video shows Show the non-destructive with data, make further through the data acquisition lossy compression method in remaining region is reduced data volume Obtain video and can be transferred to head-mounted display wirelessly, and, improve data transmission efficiency, joint Save the transmission time, optimize user's experience.
The foregoing is only presently preferred embodiments of the present invention, be not intended to limit the protection model of the present invention Enclose.All any modification, equivalent substitution and improvement etc. made within the spirit and principles in the present invention, all Comprise within the scope of the present invention.

Claims (9)

1. a head-mounted display, it is characterised in that this head-mounted display includes image acquisition units, figure As processing unit, data transmission unit, video decoding unit and display unit:
Described image acquisition units, for the head of captured in real-time user and eyes image and by described user Head and eyes image be sent to graphics processing unit;
Described graphics processing unit, for calculating user's eyeball according to the user's head received and eyes image Movement locus, obtain the key area information of user's eyeball current fixation display screen, by this emphasis district Domain information is sent to data transmission unit;
Described data transmission unit, for by described key area information send to described head-mounted display The picture output device connected so that described picture output device is to the current frame image in video to be output Subregion compressed encoding process is carried out according to this key area information;And, it is used for receiving described video defeated Go out the two field picture after the subregion compressed encoding process that equipment sends, and this two field picture is sent to video solution Code unit;
Described video decoding unit, the two field picture profit after the subregion compressed encoding received is processed Carry out subregion decoding process by the inverse algorithm of compressed encoding, decoded two field picture is sent to display Unit;
Described display unit, is used for showing the decoded two field picture of output so that with emphasis in this two field picture Data high definition corresponding to region shows, data corresponding to remaining region weaken display.
Head-mounted display the most according to claim 1, it is characterised in that described key area is aobvious With region as the center of circle, with predetermined length as radius of the coordinate figure of user's eyeball fixes on display screen curtain.
Head-mounted display the most according to claim 1 and 2, it is characterised in that described data are transmitted Unit includes: area information sends subelement and video data receives subelement;
Described area information sends subelement, for sending described key area information to wearing with described The picture output device that display connects;
Described video data receives subelement, for receiving the subregion pressure that described picture output device sends Two field picture after contracting coded treatment;
Described area information sends subelement and video data reception subelement is wireless transport module or wired Transport module.
Head-mounted display the most according to claim 1, it is characterised in that described graphics processing unit, It is additionally operable to this key area information is sent to described video decoding unit;
Described video decoding unit, specifically for the frame figure after processing the subregion compressed encoding received As utilizing the inverse algorithm of compressed encoding and entering from the key area information that described graphics processing unit obtains Row subregion decoding process.
5. a picture output device, it is characterised in that described picture output device includes:
Communication unit, for receiving the weight of user's eyeball current fixation display screen that head-mounted display sends Point area information, is sent to territory, video partition processing unit by this key area information;
Territory, described video partition processing unit, is used in video to be output according to described key area information Current frame image be divided into high definition region and atenuator region, and the data in high definition region are not pressed Reduce the staff code or carry out lossless compression-encoding process, the data in atenuator region are carried out at lossy compression method coding Reason, wherein, described high definition region is the region that described key area is corresponding;
Described communication unit, is additionally operable to the two field picture after being processed by subregion compressed encoding and sends to described head Wear display.
Picture output device the most according to claim 5, it is characterised in that described video frequency output sets Standby for smart mobile phone or computer, in this smart mobile phone or computer, storage has video data.
7. according to the picture output device described in claim 5 or 6, it is characterised in that described communication unit Unit includes: area information receives subelement and video data sends subelement;
Described area information receives subelement, and the user's eyeball sent for receiving described head-mounted display is worked as Before the key area information of display screen watched attentively;
Described video data sends subelement, and the two field picture after being processed by subregion compressed encoding sends To described head-mounted display;
Described area information receive subelement and described video data send subelement be wireless communication module or Wire communication module.
8. a method for processing video frequency, it is characterised in that the method includes:
The head of captured in real-time head-mounted display user and eyes image;
Head and the movement locus of eyes image calculating user's eyeball to user, obtains user's eyeball current Watch the key area information of display screen attentively;
Key area information is sent to the picture output device that is connected with head-mounted display so that described regard Frequently outut device carries out subregion pressure to the current frame image in video to be output according to this key area information Contracting coded treatment;And, after receiving the subregion compressed encoding process that described picture output device sends Two field picture;
Two field picture after processing the subregion compressed encoding received utilizes the inverse algorithm of compressed encoding to carry out Subregion decoding process;
Two field picture after display output decoding process so that data corresponding with key area in this two field picture High definition shows, the data reduction display that remaining region is corresponding.
9. a processing system for video, it is characterised in that this system includes: as appointed in claim 1-4 One described head-mounted display, and the picture output device as according to any one of claim 5-7;
Wired or wireless data cube computation is set up between described head-mounted display and described picture output device.
CN201610466833.6A 2016-06-23 2016-06-23 Head mount display, video output device and video processing method and system Pending CN105979224A (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201610466833.6A CN105979224A (en) 2016-06-23 2016-06-23 Head mount display, video output device and video processing method and system
PCT/CN2016/114042 WO2017219652A1 (en) 2016-06-23 2016-12-31 Head mounted display, video output device, and video processing method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610466833.6A CN105979224A (en) 2016-06-23 2016-06-23 Head mount display, video output device and video processing method and system

Publications (1)

Publication Number Publication Date
CN105979224A true CN105979224A (en) 2016-09-28

Family

ID=57020541

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610466833.6A Pending CN105979224A (en) 2016-06-23 2016-06-23 Head mount display, video output device and video processing method and system

Country Status (2)

Country Link
CN (1) CN105979224A (en)
WO (1) WO2017219652A1 (en)

Cited By (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107333119A (en) * 2017-06-09 2017-11-07 歌尔股份有限公司 The processing method and equipment of a kind of display data
WO2017219652A1 (en) * 2016-06-23 2017-12-28 青岛歌尔声学科技有限公司 Head mounted display, video output device, and video processing method and system
CN108347557A (en) * 2017-01-21 2018-07-31 盯盯拍(东莞)视觉设备有限公司 Panorama image shooting apparatus, display device, image pickup method and display methods
CN108347556A (en) * 2017-01-21 2018-07-31 盯盯拍(东莞)视觉设备有限公司 Panoramic picture image pickup method, panoramic image display method, panorama image shooting apparatus and panoramic image display device
WO2019033903A1 (en) * 2017-08-14 2019-02-21 歌尔股份有限公司 Graphics rendering method and device for virtual reality
CN110545430A (en) * 2018-05-28 2019-12-06 北京松果电子有限公司 video transmission method and device
CN112513719A (en) * 2018-07-27 2021-03-16 京瓷株式会社 Display device, display system, and movable vehicle
CN112783319A (en) * 2019-11-05 2021-05-11 宏达国际电子股份有限公司 Display system
CN112887702A (en) * 2021-01-11 2021-06-01 杭州灵伴科技有限公司 Near-eye display equipment and camera data transmission method thereof
CN113256661A (en) * 2021-06-23 2021-08-13 北京蜂巢世纪科技有限公司 Image processing method, apparatus, device, medium, and program product
CN113327489A (en) * 2021-06-03 2021-08-31 西安工业大学 Layered mechanical intelligent braille reader based on machine vision
CN114786037A (en) * 2022-03-17 2022-07-22 青岛虚拟现实研究院有限公司 Self-adaptive coding compression method facing VR projection
TWI802456B (en) * 2021-06-28 2023-05-11 見臻科技股份有限公司 Optical system with improved user experiemce and accurate eye-tracking and related method
CN116132431A (en) * 2023-04-19 2023-05-16 泰诺尔(北京)科技有限公司 Data transmission method and system
CN116737097A (en) * 2022-09-30 2023-09-12 荣耀终端有限公司 Screen projection image processing method and electronic equipment

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111225228B (en) * 2020-01-15 2022-04-12 深圳拙河科技有限公司 Video live broadcast method, device, equipment and medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103930817A (en) * 2011-06-20 2014-07-16 谷歌公司 Systems and methods for adaptive transmission of data
CN104539929A (en) * 2015-01-20 2015-04-22 刘宛平 Three-dimensional image coding method and coding device with motion prediction function
CN104767992A (en) * 2015-04-13 2015-07-08 北京集创北方科技有限公司 Head-wearing type display system and image low-bandwidth transmission method
GB2526515A (en) * 2014-03-25 2015-12-02 Jaguar Land Rover Ltd Image capture system

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105979224A (en) * 2016-06-23 2016-09-28 青岛歌尔声学科技有限公司 Head mount display, video output device and video processing method and system
CN205812229U (en) * 2016-06-23 2016-12-14 青岛歌尔声学科技有限公司 A kind of head-mounted display, picture output device and processing system for video

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103930817A (en) * 2011-06-20 2014-07-16 谷歌公司 Systems and methods for adaptive transmission of data
GB2526515A (en) * 2014-03-25 2015-12-02 Jaguar Land Rover Ltd Image capture system
CN104539929A (en) * 2015-01-20 2015-04-22 刘宛平 Three-dimensional image coding method and coding device with motion prediction function
CN104767992A (en) * 2015-04-13 2015-07-08 北京集创北方科技有限公司 Head-wearing type display system and image low-bandwidth transmission method

Cited By (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017219652A1 (en) * 2016-06-23 2017-12-28 青岛歌尔声学科技有限公司 Head mounted display, video output device, and video processing method and system
CN108347557A (en) * 2017-01-21 2018-07-31 盯盯拍(东莞)视觉设备有限公司 Panorama image shooting apparatus, display device, image pickup method and display methods
CN108347556A (en) * 2017-01-21 2018-07-31 盯盯拍(东莞)视觉设备有限公司 Panoramic picture image pickup method, panoramic image display method, panorama image shooting apparatus and panoramic image display device
US11106422B2 (en) 2017-06-09 2021-08-31 Goertek Inc. Method for processing display data
US10620903B2 (en) 2017-06-09 2020-04-14 Goertek Inc. Method and device for processing display data
CN107333119A (en) * 2017-06-09 2017-11-07 歌尔股份有限公司 The processing method and equipment of a kind of display data
WO2019033903A1 (en) * 2017-08-14 2019-02-21 歌尔股份有限公司 Graphics rendering method and device for virtual reality
US10859840B2 (en) 2017-08-14 2020-12-08 Goertek Inc. Graphics rendering method and apparatus of virtual reality
CN110545430A (en) * 2018-05-28 2019-12-06 北京松果电子有限公司 video transmission method and device
CN112513719B (en) * 2018-07-27 2023-03-31 京瓷株式会社 Display device, display system, and movable vehicle
CN112513719A (en) * 2018-07-27 2021-03-16 京瓷株式会社 Display device, display system, and movable vehicle
CN112783319A (en) * 2019-11-05 2021-05-11 宏达国际电子股份有限公司 Display system
CN112887702A (en) * 2021-01-11 2021-06-01 杭州灵伴科技有限公司 Near-eye display equipment and camera data transmission method thereof
CN113327489A (en) * 2021-06-03 2021-08-31 西安工业大学 Layered mechanical intelligent braille reader based on machine vision
CN113256661A (en) * 2021-06-23 2021-08-13 北京蜂巢世纪科技有限公司 Image processing method, apparatus, device, medium, and program product
TWI802456B (en) * 2021-06-28 2023-05-11 見臻科技股份有限公司 Optical system with improved user experiemce and accurate eye-tracking and related method
US11726564B2 (en) 2021-06-28 2023-08-15 Ganzin Technology, Inc. Optical system providing accurate eye-tracking and related method
CN114786037A (en) * 2022-03-17 2022-07-22 青岛虚拟现实研究院有限公司 Self-adaptive coding compression method facing VR projection
CN116737097A (en) * 2022-09-30 2023-09-12 荣耀终端有限公司 Screen projection image processing method and electronic equipment
CN116132431A (en) * 2023-04-19 2023-05-16 泰诺尔(北京)科技有限公司 Data transmission method and system

Also Published As

Publication number Publication date
WO2017219652A1 (en) 2017-12-28

Similar Documents

Publication Publication Date Title
CN105979224A (en) Head mount display, video output device and video processing method and system
CN111580765B (en) Screen projection method, screen projection device, storage medium, screen projection equipment and screen projection equipment
WO2018077142A1 (en) Panoramic video processing method, device and system
CN103460250B (en) Image procossing based on object of interest
CN106060582B (en) Video transmission system, method and device
CA2974104C (en) Video transmission based on independently encoded background updates
US9756328B2 (en) System, terminal, and method for dynamically adjusting video
JP3263278B2 (en) Image compression communication device
CN104539929B (en) Stereo-image coding method and code device with motion prediction
CN109417624A (en) Device and method for providing and showing content
CN106375841A (en) Wireless screen projection data processing method and device, video data display method and device, and electronic device
KR102166158B1 (en) Stereoscopic video encoding apparatus, stereoscopic video decoding apparatus and stereoscopic video display system including the same
CN107211081B (en) Video transmission based on independently coded background updates
CA3018600C (en) Method, apparatus and stream of formatting an immersive video for legacy and immersive rendering devices
CN107332977B (en) Augmented reality method and augmented reality equipment
US11310560B2 (en) Bitstream merger and extractor
CN106534882A (en) Encoding and decoding method and device for virtual reality panoramic video
CN205812229U (en) A kind of head-mounted display, picture output device and processing system for video
CN107396082B (en) Image data processing method and device
CN102843566B (en) Communication method and equipment for three-dimensional (3D) video data
US20200267199A1 (en) Method and system for transmitting video
CN101001370A (en) Vedio transmission system and method of mixed with mutiple code and decode
JP2020115299A (en) Virtual space information processing device, method and program
CN114040184A (en) Image display method, system, storage medium and computer program product
CN102780894B (en) A kind of decoding method of 3D rendering

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20160928

RJ01 Rejection of invention patent application after publication