CN109510990A - Image processing method and device, computer readable storage medium, electronic equipment - Google Patents
Image processing method and device, computer readable storage medium, electronic equipment Download PDFInfo
- Publication number
- CN109510990A CN109510990A CN201811253125.XA CN201811253125A CN109510990A CN 109510990 A CN109510990 A CN 109510990A CN 201811253125 A CN201811253125 A CN 201811253125A CN 109510990 A CN109510990 A CN 109510990A
- Authority
- CN
- China
- Prior art keywords
- frame
- described image
- video flowing
- image processing
- data packet
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/005—General purpose rendering architectures
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/186—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/643—Communication protocols
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/85—Assembly of content; Generation of multimedia applications
- H04N21/854—Content authoring
- H04N21/8547—Content authoring involving timestamps for synchronizing content
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Computer Graphics (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The present invention relates to field of computer technology, a kind of image processing method and device, computer readable storage medium, electronic equipment are provided.The image processing method includes: to generate multiple images frame by the first engine renders, and each described image frame is carried out coding compression to form video flowing；The video flowing is sent to terminal device in real time；User that the terminal device is sent in real time is received to the operation information of the video flowing, and handled the operation information to realize the control to described image frame by first engine.One aspect of the present invention can reduce the transmission quantity of data, reduce Caton, improve image quality；On the other hand facilitate operation of the user to picture, vivid sense is by virtual and real interaction, and the user experience is improved.
Description
Technical field
The present invention relates to field of computer technology, can in particular to a kind of image processing method and device, computer
Read medium and electronic equipment.
Background technique
In order to keep user experience on the spot in person, the things in three-dimensional space is really experienced, virtual reality technology has obtained extensively
General application is generated the virtual world of three-dimensional space using computer simulation, provides the various sense organ pseudo-experiences of user, work as user
When execution acts, computer carries out complex calculation, to enhance user in the sense of reality of three-dimensional world.
Common virtual reality technology is mainly used in game or virtual shooting, in order to further increase user to true
The experience of effect can be used family in game or virtual shooting using mobile device, such as mobile phone or tablet computer, but adopt
When making scene of game with game engine, powerful CPU and GPU hardware resource are needed, therefore in the process of virtual reality interaction
In, the problems such as that there are volumes of transmitted data is big, Caton, poor image quality, poor user experience.
Therefore this field needs to seek a kind of new image processing method and device.
It should be noted that information is only used for reinforcing the reason to background of the invention disclosed in above-mentioned background technology part
Solution, therefore may include the information not constituted to the prior art known to persons of ordinary skill in the art.
Summary of the invention
The purpose of the present invention is to provide a kind of image processing method and device, computer-readable medium and electronic equipment,
And then reduce volume of transmitted data, reduce card section, image quality is improved, and then promote user experience.
Other characteristics and advantages of the invention will be apparent from by the following detailed description, or partially by the present invention
Practice and acquistion.
According to the first aspect of the invention, a kind of image processing method is provided characterized by comprising
Multiple images frame is generated by the first engine renders, and each described image frame is subjected to coding compression to form video
Stream；The video flowing is sent to terminal device in real time；User that the terminal device is sent in real time is received to the video flowing
Operation information, and the operation information is handled to realize the control to described image frame by first engine.
According to the second aspect of the invention, a kind of image processing apparatus is provided characterized by comprising
Video flowing generation module, for by the first engine renders generate multiple images frame, and by each described image frame into
Row coding compression is to form video flowing；Sending module, for the video flowing to be sent to terminal device in real time；Interactive module,
For receiving user that the terminal device is sent in real time to the operation information of the video flowing, and pass through first engine pair
The operation information is handled to realize the control to described image frame.
In some embodiments of the invention, aforementioned schemes are based on, the video flowing generation module includes:
Differential pixels determination unit, for determining between other picture frames and key images frame in each described image frame
Differential pixels；First coding unit, for carrying out coding compression to the key images frame and the differential pixels, to be formed
State video flowing.
In some embodiments of the invention, described image frame is the picture frame of rgb format, is based on aforementioned schemes, described
Video flowing generation module includes:
Format conversion unit is more to obtain for the picture frame of the rgb format to be converted to the picture frame of yuv format
A target image frame；Second coding unit, for each target image frame to be carried out coding compression to form the video flowing.
In some embodiments of the invention, aforementioned schemes are based on, second coding unit includes:
Data package generating unit, for carrying out coding pressure to each target image frame respectively by pre-arranged code method
Contracting, to form image data packet corresponding with each target image frame；Video flowing generation unit, for according to each described image
Data packet forms the video flowing.
In some embodiments of the invention, aforementioned schemes are based on, the data package generating unit includes:
Writing unit, for the corresponding sequence number of described image frame and generation when forming described image data packet, to be written
The corresponding timestamp of described image data packet.
According to the third aspect of the invention we, a kind of image processing method is provided characterized by comprising
A video flowing is received, the video flowing includes multiple images data packet, and each described image data packet is by right
Multiple images frame carries out what coding compression was formed respectively；Described image data packet is decoded, to obtain each described image frame；
User is received to the operation information of described image frame, and the operation information is sent to server.
According to the fourth aspect of the invention, a kind of image processing apparatus is provided characterized by comprising
Receiving module, for receiving a video flowing, the video flowing includes multiple images data packet, and each described image number
It according to packet is formed by carrying out coding compression respectively to multiple images frame；Decoder module, for described image data packet into
Row decoding, to obtain each described image frame；Data acquisition module, for receiving user to the operation information of described image frame, and
The operation information is sent to server.
In some embodiments of the invention, described image frame is the picture frame of yuv format, is based on aforementioned schemes, described
Image processing apparatus further include:
Format converting module, for after obtaining each described image frame, the picture frame of the yuv format to be converted to RGB
The picture frame of format.
In some embodiments of the invention, aforementioned schemes are based on, described image data packet includes the sequence of described image frame
Row number timestamp corresponding with described image data packet is generated.
In some embodiments of the invention, aforementioned schemes are based on, described image processing unit includes:
Detection module carries out frame detection for the sequence number according to described image frame, and is determined according to the timestamp
Time interval between adjacent described image frame.
In some embodiments of the invention, aforementioned schemes are based on, the detection module includes:
Fall frame judging unit, for judging in the video flowing according to the sequence number of described image frame with the presence or absence of loss figure
As frame, the loss picture frame has target sequence number；Picture frame selection unit is used in the presence of judgement, will be with the mesh
The adjacent corresponding picture frame of previous sequence number of sequence number is marked as the corresponding picture frame of the target sequence number.
In some embodiments of the invention, aforementioned schemes are based on, described image processing unit includes:
Control generation module, for generating multiple functionality controls based on the second engine, and by the functionality controls to institute
Picture frame is stated to be controlled.
According to the fifth aspect of the invention, a kind of computer-readable medium is provided, computer program is stored thereon with, institute
It states and realizes such as above-mentioned image processing method as described in the examples when program is executed by processor.
According to the sixth aspect of the invention, a kind of electronic equipment is provided, comprising: one or more processors；Storage dress
It sets, for storing one or more programs, when one or more of programs are executed by one or more of processors, makes
It obtains one or more of processors and realizes such as above-mentioned image processing methods as described in the examples.
According to the image processing method in this example embodiment, multiple images frame is generated by the first engine renders first,
And multiple images frame is subjected to coding compression and forms video flowing；Then it sends terminal device in real time by video flowing, and is presented on
On the display screen of terminal device；Then user is obtained to the operation information of video flowing, and is believed by the first engine the operation
Breath is handled to realize the control to picture frame.One aspect of the present invention can reduce the transmission quantity of data, reduce Caton, improve
Image quality；On the other hand facilitate operation of the user to picture, vivid sense is improved user's body by virtual and real interaction
It tests.
The present invention is it should be understood that above general description and following detailed description is only exemplary and explanatory
, the present invention can not be limited.
Detailed description of the invention
The drawings herein are incorporated into the specification and forms part of this specification, and shows and meets implementation of the invention
Example, and be used to explain the principle of the present invention together with specification.It should be evident that the accompanying drawings in the following description is only the present invention
Some embodiments for those of ordinary skill in the art without creative efforts, can also basis
These attached drawings obtain other attached drawings.
Fig. 1 is shown can be using the image processing method of the embodiment of the present invention or the exemplary system frame of image processing apparatus
The schematic diagram of structure；
Fig. 2 shows the structural schematic diagrams of the computer system of the electronic equipment suitable for being used to realize the embodiment of the present invention；
Fig. 3 shows the flow chart of the image processing method in one embodiment of the invention；
Fig. 4 shows the flow diagram that the coding compression picture frame in one embodiment of the invention forms video flowing；
Fig. 5 shows the structural schematic diagram of the YUV420sp storage format in one embodiment of the invention；
Fig. 6 shows the flow chart of image processing method in one embodiment of the invention；
Fig. 7 shows the flow diagram for falling frame detection in one embodiment of the invention；
Fig. 8 shows the image shown in mobile terminal in one embodiment of the invention；
Fig. 9 shows the structural schematic diagram of image processing apparatus in one embodiment of the invention；
Figure 10 shows the structural schematic diagram of image processing apparatus in one embodiment of the invention；
Figure 11 shows the structural schematic diagram of image processing apparatus in one embodiment of the invention；
Figure 12 shows the structural schematic diagram of image processing apparatus in one embodiment of the invention；.
Specific embodiment
Example embodiment is described more fully with reference to the drawings.However, example embodiment can be with a variety of shapes
Formula is implemented, and is not understood as limited to example set forth herein；On the contrary, thesing embodiments are provided so that the present invention will more
Fully and completely, and by the design of example embodiment comprehensively it is communicated to those skilled in the art.
In addition, described feature, structure or characteristic can be incorporated in one or more implementations in any suitable manner
In example.In the following description, many details are provided to provide and fully understand to the embodiment of the present invention.However,
It will be appreciated by persons skilled in the art that technical solution of the present invention can be practiced without one or more in specific detail,
Or it can be using other methods, constituent element, device, step etc..In other cases, it is not shown in detail or describes known side
Method, device, realization or operation are to avoid fuzzy each aspect of the present invention.
Block diagram shown in the drawings is only functional entity, not necessarily must be corresponding with physically separate entity.
I.e., it is possible to realize these functional entitys using software form, or realized in one or more hardware modules or integrated circuit
These functional entitys, or these functional entitys are realized in heterogeneous networks and/or processor device and/or microcontroller device.
Flow chart shown in the drawings is merely illustrative, it is not necessary to including all content and operation/step,
It is not required to execute by described sequence.For example, some operation/steps can also decompose, and some operation/steps can close
And or part merge, therefore the sequence actually executed is possible to change according to the actual situation.
This field in the related technology, game engine, but usual game can be used when making game or virtual shooting
Powerful CPU and GPU hardware resource are needed when engine-operated.In virtual shooting or production of film and TV, preview link is very heavy
It wants, for the true effect of actual experience engine renders, mobile device can be used in shooting process or manufacturing process, than
Such as mobile phone or tablet computer, and by the plug-in unit in game engine, user can see game engine on mobile terminals
The picture of rendering, and the operation of user transmitting is gone back.
But the scheme that plug-in unit in the related technology uses is that every frame picture is individually compressed and transmitted, and is not considered
Continuity between picture, therefore there is a problem of that volume of transmitted data is big, there is apparent card in the case where testing LAN environment
Phenomenon；In addition in the related technology to dynamic fuzzy phenomena such as processing it is not good enough, for high-resolution picture, picture matter
Measure it is poor, if by improve resolution changing image quality, will affect the fluency of network transmission；Finally, the relevant technologies
In only provide the installation kit of mobile device, it has not been convenient to user adds (the UI control of more user interface controls in mobile device end
Part).
In view of problem present in the relevant technologies, the present invention provides a kind of image processing method and image processing apparatus.
Fig. 1 is shown can be using the image processing method of the embodiment of the present invention or the exemplary system of image processing apparatus
The schematic diagram of framework 100.
As shown in Figure 1, system architecture 100 may include terminal device 101, network 102 and server 103.Network 102 is used
To provide the medium of communication link between terminal device 101 and server 103.Network 102 may include various connection types,
Such as wired, wireless communication link or fiber optic cables etc..
It should be understood that the number of terminal device, network and server in Fig. 1 is only schematical.According to realization need
It wants, can have any number of terminal device, network and server.For example server 103 can be multiple server compositions
Server cluster etc..
User can be used terminal device 101 and be interacted by network 102 with server 103, to receive or send data etc..
Terminal device 101 can be the various electronic equipments with hard disk, including but not limited to tablet computer, smart phone, portable
Computer and desktop computer etc..
Server 103 can be to provide the server of various services.It is loaded with the first engine on server 103, passes through
Multiple images frame can be generated in one engine renders, and then the coding condensing routine in server 103 compiles multiple images frame
Code compression, generates multiple image data packets corresponding with picture frame, then forms video flowing, and the video according to image data packet
Stream is sent to terminal device 101 in real time；After terminal device 101 receives video flowing, which is decoded and is shown
Diagram picture, and present on the display screen, user can be by user interface controls button or in any position of display screen
It sets and display image is operated by the external equipments such as mouse, keyboard, screen touch pen or finger, terminal device 101 collects user
To the operation information for showing image and it is sent to server 103；After server 103 receives the operation information of user, pass through
One engine handles to realize the control to picture frame operation information.
Fig. 2 shows the structural representations of the computer system of the electronic equipment suitable for being used to realize the embodiment in the present invention
Figure.The electronic equipment is able to carry out image processing method described in the above embodiment of the present invention, to realize to picture quality
Control；Computer system shown in Fig. 2 can be adapted for terminal device 101 and/or server 103 above-mentioned.
It should be noted that Fig. 2 shows the computer system 200 of electronic equipment be only an example, should not be to this hair
The function and use scope of bright embodiment bring any restrictions.
As shown in Fig. 2, computer system 200 includes central processing unit (CPU) 201, it can be read-only according to being stored in
Program in memory (ROM) 202 or be loaded into the program in random access storage device (RAM) 203 from storage section 208 and
Execute various movements appropriate and processing.In RAM 203, it is also stored with various programs and data needed for system operatio.CPU
201, ROM 202 and RAM 203 is connected with each other by bus 204.Input/output (I/O) interface 205 is also connected to bus
204。
I/O interface 205 is connected to lower component: the importation 206 including keyboard, mouse etc.；It is penetrated including such as cathode
The output par, c 207 of spool (CRT), liquid crystal display (LCD) etc. and loudspeaker etc.；Storage section 208 including hard disk etc.；
And the communications portion 209 of the network interface card including LAN card, modem etc..Communications portion 409 via such as because
The network of spy's net executes communication process.Driver 210 is also connected to I/O interface 205 as needed.Detachable media 211, such as
Disk, CD, magneto-optic disk, semiconductor memory etc. are mounted on as needed on driver 210, in order to read from thereon
Computer program be mounted into storage section 208 as needed.
Particularly, according to an embodiment of the invention, may be implemented as computer below with reference to the process of flow chart description
Software program.For example, the embodiment of the present invention includes a kind of computer program product comprising be carried on computer-readable medium
On computer program, which includes the program code for method shown in execution flow chart.In such reality
It applies in example, which can be downloaded and installed from network by communications portion 209, and/or from detachable media
211 are mounted.When the computer program is executed by central processing unit (CPU) 201, executes and limited in system of the invention
Various functions.
It should be noted that computer-readable medium shown in the present invention can be computer-readable signal media or meter
Calculation machine readable storage medium storing program for executing either the two any combination.Computer readable storage medium for example can be --- but not
Be limited to --- electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor system, device or device, or any above combination.Meter
The more specific example of calculation machine readable storage medium storing program for executing can include but is not limited to: have the electrical connection, just of one or more conducting wires
Taking formula computer disk, hard disk, random access storage device (RAM), read-only memory (ROM), erasable type may be programmed read-only storage
Device (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), light storage device, magnetic memory device,
Or above-mentioned any appropriate combination.In the present invention, computer readable storage medium can be it is any include or storage journey
The tangible medium of sequence, the program can be commanded execution system, device or device use or in connection.And at this
In invention, computer-readable signal media may include in a base band or as carrier wave a part propagate data-signal,
Wherein carry computer-readable program code.The data-signal of this propagation can take various forms, including but unlimited
In electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media can also be that computer can
Any computer-readable medium other than storage medium is read, which can send, propagates or transmit and be used for
By the use of instruction execution system, device or device or program in connection.Include on computer-readable medium
Program code can transmit with any suitable medium, including but not limited to: wireless, electric wire, optical cable, RF etc. are above-mentioned
Any appropriate combination.
Flow chart and block diagram in attached drawing are illustrated according to the system of various embodiments of the invention, method and computer journey
The architecture, function and operation in the cards of sequence product.In this regard, each box in flowchart or block diagram can generation
A part of one module, program segment or code of table, a part of above-mentioned module, program segment or code include one or more
Executable instruction for implementing the specified logical function.It should also be noted that in some implementations as replacements, institute in box
The function of mark can also occur in a different order than that indicated in the drawings.For example, two boxes succeedingly indicated are practical
On can be basically executed in parallel, they can also be executed in the opposite order sometimes, and this depends on the function involved.Also it wants
It is noted that the combination of each box in block diagram or flow chart and the box in block diagram or flow chart, can use and execute rule
The dedicated hardware based systems of fixed functions or operations is realized, or can use the group of specialized hardware and computer instruction
It closes to realize.
Be described in the embodiment of the present invention involved unit can by by software in a manner of realize, can also pass through
The mode of hardware realizes that described unit also can be set in the processor.Wherein, the title of these units is in certain feelings
The restriction to the unit itself is not constituted under condition.
As on the other hand, the present invention also provides a kind of computer readable storage medium, the computer-readable storage mediums
Matter can be included in terminal device or server described in above-described embodiment；It is also possible to individualism, and without
It is incorporated in terminal device or server.Above-mentioned computer readable storage medium carries one or more program, when above-mentioned one
When a or multiple programs are executed by terminal device or server, so that terminal device or server are realized in as the following examples
The method.For example, terminal device or server may be implemented such as Fig. 3-each step shown in Fig. 8.
In an embodiment of the present invention, provide firstly a kind of image processing method, with to there are the problem of optimize
Processing, with specific reference to shown in Fig. 3, image processing method is at least included the following steps, specifically:
Step S310: multiple images frame is generated by the first engine renders, and each described image frame is subjected to coding compression
To form video flowing；
Step S320: the video flowing is sent to terminal device in real time；
Step S330: user that the terminal device is sent in real time is received to the operation information of the video flowing, and is passed through
First engine handles to realize the control to described image frame the operation information.
According to the image processing method in this example embodiment, the multiple images frame generated to the first engine renders is compiled
Code compression forms video flowing, and video flowing is sent to terminal device in real time；Then the user that receiving terminal apparatus is sent is to view
The operation information of frequency stream, is handled operation information by the first engine to realize the control to picture frame.A side of the invention
Face can reduce volume of transmitted data, reduce Caton；On the other hand it can improve dynamic fuzzy, improve image quality, further mention
High user experience.
In the following, the image processing method in this example embodiment is further detailed.
In step s310, multiple images frame is generated by the first engine renders, and each described image frame is encoded
Compression is to form video flowing.
In this exemplary embodiment, the first engine can be game engine, game engine refer to it is some write can
The core component of computer game system or some interactive realtime graphic application programs is edited, the game used in the present invention is drawn
It holds up and can be Unreal, Ogre, Unity, Gamebryo etc., can be rendered by the first engine and generate multiple images frame,
After obtaining multiple images frame, coding compression can be carried out to multiple images frame to form video flowing.
In this exemplary embodiment, by each picture frame carry out coding compression form video flowing when, can use preset
Video coding software, such as ffmpeg encoding and decoding software can be used, ffmpeg encoding and decoding software is a free software, can be with
Operation the making video recording, convert of audio and video multiple format, stream function, contain decoding/pressure code library of audio and video.Specifically
Ground, can by the video compression algorithm in Video coding software, such as H264, X264 video compression algorithm, to picture frame into
Row coding compression is to form video flowing, and wherein H264, X264 are compression of digital video format and a kind of coding and decoding video mark
It is quasi-.
In this exemplary embodiment, Fig. 4 shows the flow diagram that coding compression picture frame forms video flowing, such as Fig. 4
It is shown, to picture frame carry out coding compression form video flowing during, in step S401, determine described multiple images frame
In other picture frames and key images frame between differential pixels；In step S402, to the key images frame and described
Differential pixels carry out coding compression, to form the video flowing.It is worth noting that, can also first obtain key in the present invention
Then picture frame determines the differential pixels between other picture frames and key images frame again during encoding and compressing, and compiles
Code compression.Due to only having carried out first compression for identical pixel during coding compression picture frame forms video flowing,
Therefore reduce decrement, and in the related technology individually compressing every frame picture, picture frame is compressed into view
Frequency flows, and greatly reduces volume of transmitted data, saves bandwidth, further reduces the appearance of Caton phenomenon.
In step s 320, the video flowing is sent to terminal device in real time.
In this exemplary embodiment, after forming video flowing, video flowing can be sent to terminal device 101 in real time, and
Picture frame in the display screen of terminal device 101 in display of video streams, so that user watches and operates.Wherein, terminal device
It can be the mobile terminal devices such as mobile phone, tablet computer, be also possible to other common terminal devices.Server 101 can lead to
The video stream formed will be compressed to mobile terminal device by the image frame of the first engine renders by crossing wireless network, and mobile
Operation information of the user to picture frame can also be sent to server by wireless network by terminal device, for the first engine into
Row processing, realizes the interaction virtually with reality.
In this exemplary embodiment, after forming video flowing, video flowing can be transmitted to end according to preset transport protocol
In end equipment 101, which can be the real-time Data Transmissions agreement such as udp protocol, wherein udp protocol (user data package
Agreement) it is a kind of for handling the connectionless agreement of data packet, in the present invention, the type of Data Transport Protocol is not done and is had
Body limits.As a specific embodiment of the invention, can by udp protocol by video stream into terminal device 101.
In step S330, user that the terminal device is sent in real time is received to the operation information of the video flowing, and
The operation information is handled by first engine to realize the control to described image frame.
In this exemplary embodiment, video flowing is sent to terminal device 101 by server 103, and user can pass through terminal
Picture frame in the display screen viewing video flowing of equipment 101, and picture frame is operated.It is operated to picture frame
When, any position of picture frame can also be clicked by clicking the button graphic being located in picture frame triggering correspondingly event
It sets to trigger corresponding event.The user that 103 receiving terminal apparatus 101 of server is sent leads to the operation information of video flowing
The first engine is crossed to handle to realize the control to picture frame operation information.For example, the operation information of user's input can be with
To click, double-clicking, when operation information is to click, it is corresponding that the control in terminal device 101 can analyze the single-click operation
Specific event and position, and formation operation information packet, the operation information data packet can be sent to clothes by terminal device 101
Business device 103；Then the first engine loaded in server 103 can draw according to the position in operation information data packet first
It holds up and determines corresponding position in the picture frame that rendering generates, and corresponding movement is executed according to specific event, picture frame is carried out
Processing, for example image scaled can be zoomed in and out, switch picture frame etc.；Finally the picture frame obtained after processing is back to again
Terminal device 101.When operation information is to double-click, the first engine can execute corresponding movement according to double click event, such as can
To improve brightness of image, the resolution ratio etc. double-clicked at position.Certain operation information can also be other operations, the present invention couple
This is not specifically limited.
In this exemplary embodiment, picture frame is subjected to coding compression in use preset Video coding software and forms video
During stream, video quality can be ensured by adjusting the parameter of preset Video coding software, accelerates coding compression
Speed, the delay for reducing video flowing, the width of picture when wherein the parameter of Video coding software may include code rate, coding compression
With needed after height, video frequencies, frame losing how many frame can restore normal pictures, be input to from encoder decoder output delay
Frame number, Thread Count, coded format, the coding compression algorithm of coding compression etc..
Further, when carrying out coding compression to picture frame, coding compression can be carried out by single thread or multithreading,
And coding pressure can be formed using Fasync Task and FnonAbandonableTask class creation thread in the first engine
Diminishing line journey, to carry out coding compression.In the present invention, coding compression can be carried out using asynchronous mode, makes compressed encoding thread
Independently of main thread, coding compression thread independent in this way, which will not be formed main thread, to be blocked, and improves data transmission efficiency.
In this exemplary embodiment, the picture frame that the first engine renders generate is the picture frame of rgb format, by picture frame
During coding compression forms video flowing, the picture frame of rgb format can be first converted to the picture frame of yuv format, by YUV
Then target image frame is carried out coding compression again and forms video flowing by the picture frame of format as target image frame.Yuv format is
Refer to brightness parameter and the separately shown pixel format of coloration parameter, and is exactly not by the separated benefit of brightness parameter and coloration parameter
But the sample rate that avoid interfering with each other, can also reduce coloration is too big without influencing on picture quality.Due to YUV unlike
RGB requires three independent vision signal simultaneous transmissions like that, so can be with picture frame is converted to that yuv format is transmitted
Few bandwidth is occupied, vast resources is saved.Yuv format is divided into planar and packed two major classes, for the YUV lattice of planar
And then formula, the brightness of first Coutinuous store all pixels point store the color of all pixels point, are followed by all pixels point
Saturation degree；For the yuv format of packed, brightness, color and the saturation degree of each pixel are continuous interleaveds.It is right
In the picture frame of yuv format, can be stored using storage formats such as YUV422P, YUV420, YUV420sp, Fig. 5 is shown
The structural schematic diagram of YUV420sp storage format, as shown in figure 5, brightness therein (Y) information is independently stored, and color
(U) and saturation degree (V) is stored with the alternate mode of UVUV.
In this exemplary embodiment, during picture frame coding compression is formed video flowing, for each picture frame
A corresponding image data packet can be generated, video flowing then can be formed according to multiple images data packet.That is, working as root
It is substantially to transmit multiple UDP message packets when according to UDP transport protocol transmitting video-frequency flow.
Further, when forming image data packet, by the corresponding sequence number of picture frame and image data packet can be generated
Corresponding timestamp is written in picture frame, specifically, picture frame can be numbered while generating picture frame, is made every
A picture frame has corresponding sequence number；When then according to picture frame shape image data packet, corresponding timestamp is obtained, and will figure
As the sequence number and the corresponding timestamp of image data packet of frame are added in the data of image data packet.Pass through image data packet pair
The timestamp answered can determine the time interval between adjacent picture frame, while can be right according to the corresponding sequence number of picture frame
Whether picture frame, which falls frame, is detected.
In an embodiment of the present invention, additionally provide a kind of image processing method, with to there are the problem of optimize place
Reason, with specific reference to shown in Fig. 6, image processing method is at least included the following steps, specifically:
Step S610: receiving a video flowing, and the video flowing includes multiple images data packet, and each described image data packet
It is to be formed by carrying out coding compression respectively to multiple images frame.
In this exemplary embodiment, the first engine renders loaded in server 103 generate multiple images frame, by more
Re-encoding compression is after a picture frame carries out coding compression or carries out image format conversion to picture frame to generate multiple and image
The corresponding image data packet of frame, can form video flowing according to multiple images data packet.Draw to allow users to viewing first
The picture frame that rendering generates is held up, and is operated on it, video flowing can be sent to terminal device 101 by server 103.
Step S620: being decoded described image data packet, to obtain each described image frame；
It in this exemplary embodiment, can be to the image data packet in video flowing after terminal device 101 receives video flowing
It is decoded to obtain picture frame, and picture frame is presented on the display screen of terminal device 101, so that user watches and holds
Row operation.Decoding operate therein is the inverse operation that compression process is encoded in step S610, such as when using X264 Video coding
Compression algorithm picture frame formed image data packet, and according to image data packet formed video flowing when, then can correspondingly use
X264 video decoding algorithm decodes image data packet to obtain picture frame.
In this exemplary embodiment, the picture frame obtained after the decoding of image data packet can be the picture frame of yuv format, be
Picture frame is presented on the display screen of terminal device 101 and is watched for user, the picture format of picture frame can be carried out
The picture frame of yuv format, is such as converted to the picture frame of rgb format by conversion.
Step S630: user is received to the operation data of described image frame, and the operation data is sent to server.
In this exemplary embodiment, after terminal device 101 is decoded acquisition picture frame to video flowing, multiple images frame can
To be presented in the display screen of terminal device 101, user can operate picture frame after seeing picture frame.User can
To input operation information by the external input device connecting with terminal device 101, such as keyboard, mouse, screen touch pen equipment,
Operation information can also be inputted by modes such as finger touches, which can be arbitrary location information or movement letter
Breath, such as can be touch screen position, click, double-click etc., the present invention is not specifically limited in this embodiment.Terminal device 101, which obtains, to be used
After the input information at family, which is sent to server 103, so that the first engine handles input information, it is real
Now to the control of picture frame, and then realize the virtually interaction with reality.
In this exemplary embodiment, to picture frame carry out coding compression form image data packet during, can write
Enter the corresponding sequence number of picture frame and generate the timestamp of image data packet, wherein the corresponding sequence number of picture frame is for being fallen
Frame detection, timestamp are used to determine the time interval between adjacent image frame.
Further, Fig. 7 shows the flow diagram of frame detection, as shown in fig. 7, in step s 701, according to figure
As the sequence number of frame judges in video flowing that losing picture frame has target sequence number with the presence or absence of picture frame is lost.Due to sequence
Number there is continuity, therefore whether continuously can determine whether picture frame is lost by sequence number, and obtains loss picture frame
Corresponding target sequence number；In step S702, if it is determined that in the presence of, then by the previous sequence number pair adjacent with target sequence number
The picture frame answered is as the corresponding picture frame of target sequence number.If it exists the case where image frame loss, then picture frame is set in terminal
Can there are Caton and noise phenomenon when presenting in standby 101, by using the adjacent corresponding picture frame of previous sequence number as mesh
The phenomenon that marking the corresponding picture frame of sequence number, Caton and noise can be reduced.
In this exemplary embodiment, the second engine can be loaded in terminal device 101, which can be used family
Multiple functionality controls are added on the interface of terminal device 101, as (frame per second display box, electricity are aobvious for exit button, information display frame
Show frame, fall frame prompting frame) etc., picture frame is controlled by functionality controls convenient for user.Fig. 8 is shown in mobile terminal
The image of display, as shown in figure 8, the image that the rendering of the image shown in mobile terminal and server end generates is consistent, by
Function button is clicked in mobile terminal can will click on event transmission to server end, and make the first engine according to click event pair
Picture frame is handled, and realizes reality and virtual interaction.As a specific embodiment, can be used in the present invention
For Cocos2d as the second engine, Cocos2d is simple with framework, supports multilingual and multi-platform, the lesser characteristic of installation kit,
And also facilitates user to add functionality controls, and operation information of the user to picture frame is transmitted to the first engine.
In this exemplary embodiment, in order to accelerate decoding speed, reduce power consumption and with the language environment (C of Cocos2d
++) matching, a thread can be created in Cocos2d, be decoded by the decoder that terminal device 101 carries, such as
For android system, can be decoded using its included native codec.After the decoding, it can be controlled for sprite
Part writes customization tinter, and the picture frame of yuv format is converted to RGB by the image processor (GPU) in terminal device 101
The picture frame of format, and be presented in the display screen of terminal device 101.It is tested by Samsung Note5 mobile phone, final
In the case that resolution setting is 960*540, frame per second can be stablized in 30fps, and the occupancy of CPU about 15% can satisfy use
The use at continuous 3 hours of family or more, certainly for the mobile device for being equipped with stronger GPU, image quality is available further
It improves.
In image processing method of the invention, after terminal device receives video flowing, video flowing is decoded, image
Format conversion, to obtain the picture frame that the first engine renders generate, user can be in terminal device, especially mobile terminal device
Middle viewing picture frame, and it is made so that server end handles picture frame according to the operation information of input by input operation information
Corresponding variation occurs, and picture frame is back to user by treated, realizes virtual and real interaction.In addition, passing through
The second engine is loaded in terminal device, family custom feature button can be used, to be debugged.
Image processing method of the invention is illustrated with the remote interaction during game making below: firstly, logical
It crosses the game virtual engine being equipped in server and multiple game image frames is generated according to the operation of game developer rendering；By
In the format of game image frame that rendering generates be usually rgb format, and in the image of yuv format, luminance parameter and coloration ginseng
Number is separated, therefore in order to improve the compression quality of image, can be YUV lattice by the image format conversion of game image frame
Formula；Then, the game image frame of multiple yuv formats is carried out by compressed encoding by Video coding software respectively and generates multiple images
Data packet, and a video flowing is formed, wherein sequence number and generation image data packet in each image data packet comprising picture frame
Timestamp；Then, which is remotely sent to by a mobile terminal device according to transport protocol, so that user is whole by movement
End equipment carries out remote control to game image frame.Mobile terminal device is sequentially received each picture number to form the video flowing
After packet, firstly, each image data packet is decoded by Video coding software to obtain game image frame therein,
In decoding operation be generate image data packet when compressed encoding inverse operation；Then, to the picture format of game image frame
It is converted, yuv format is converted into rgb format and is shown in the display screen of mobile terminal device, so as to user's viewing
Game picture；Further, can be arranged in the user interface by another virtual engine being arranged in mobile terminal device
Multiple control buttons, so that user handles game image frame by contact action or by clicking control button with right
Game image frame triggers corresponding event can be by the operand after mobile terminal device receives the operation data of user
According to server is sent to, so that the game virtual engine in server performs corresponding processing game image frame, user is realized
Long-range control to image.Due to being the video flowing of high quality by the picture frame Real Time Compression of rendering in the embodiment of the present invention, and
It is transferred to mobile terminal device by transport protocol, the user of such mobile terminal device can see loss very little in real time
High-quality screen further improves user experience.
The device of the invention embodiment introduced below can be used for executing the above-mentioned image processing method of the present invention.For
Undisclosed details in apparatus of the present invention embodiment please refers to the embodiment of the above-mentioned image processing method of the present invention.
Fig. 9 shows a kind of structural schematic diagram of image processing apparatus, and referring to shown in Fig. 9, image processing apparatus 900 can be with
It include: video flowing generation module 901, sending module 902 and interactive module 903.
Specifically, video flowing generation module 901, for generating multiple images frame by the first engine renders, and by each institute
It states picture frame and carries out coding compression to form video flowing；Sending module 902 is set for the video flowing to be sent to terminal in real time
It is standby；Interactive module 903 for receiving user that the terminal device is sent in real time to the operation information of the video flowing, and is led to
First engine is crossed to handle to realize the control to described image frame the operation information.
In this exemplary embodiment, the video flowing generation module 901 includes differential pixels determination unit and the first coding
Unit.
Specifically, differential pixels determination unit, for determining other picture frames in described multiple images frame and crucial figure
As the differential pixels between frame；First coding unit, for carrying out coding pressure to the key images frame and the differential pixels
Contracting, to form the video flowing.
In this example embodiment, described image frame is the picture frame of rgb format, and the video flowing generation module 901 wraps
Format conversion unit is included, for the picture frame of the rgb format to be converted to the picture frame of yuv format, to obtain multiple targets
Picture frame；Second coding unit, for each target image frame to be carried out coding compression to form the video flowing.
In this exemplary embodiment, second coding unit includes data package generating unit and video flowing generation unit.
Specifically, data package generating unit, for being carried out respectively to each target image frame by pre-arranged code method
Coding compression, to form image data packet corresponding with each target image frame；Video flowing generation unit, for according to each institute
It states image data packet and forms the video flowing.
In this exemplary embodiment, the data package generating unit includes writing unit, for forming described image number
When according to packet, the corresponding sequence number of write-in described image frame timestamp corresponding with described image data packet is generated.
Figure 10 shows a kind of structural schematic diagram of image processing apparatus, referring to Fig.1 shown in 0, image processing apparatus 1000
It may include: receiving module 1001, decoder module 1002 and data acquisition module 1003.
Specifically, receiving module 1001, for receiving a video flowing, the video flowing includes multiple images data packet, and
Each described image data packet is formed by carrying out coding compression respectively to multiple images frame；Decoder module 1002, for pair
Described image data packet is decoded, to obtain each described image frame；Data acquisition module 1003, for receiving user to described
The operation information of picture frame, and the operation information is sent to server.
In this exemplary embodiment, described image frame is the picture frame of yuv format, and Figure 11 shows a kind of image procossing dress
It sets, as shown in figure 11, image processing apparatus 1000 can also include format converting module 1004, for obtaining each described image
After frame, the picture frame of the yuv format is converted to the picture frame of rgb format.
In this exemplary embodiment, described image data packet includes the sequence number and generation described image number of described image frame
According to the corresponding timestamp of packet.
Figure 12 shows a kind of image processing apparatus, and as shown in figure 12, image processing apparatus 1000 can also include detection
Module 1005 carries out frame detection for the sequence number according to described image frame, and determines adjacent institute according to the timestamp
State the time interval between picture frame.
In this exemplary embodiment, the detection module 1005 includes falling frame judging unit and picture frame selection unit.
Specifically, fall frame judging unit, for judging whether deposit in the video flowing according to the sequence number of described image frame
Picture frame is being lost, the loss picture frame has target sequence number；Picture frame selection unit is used in the presence of judgement, will
Picture frame corresponding with the adjacent previous sequence number of the target sequence number is as the corresponding picture frame of the target sequence number.
Figure 12 shows a kind of image processing apparatus, and as shown in figure 12, image processing apparatus 1000 can also include control
Generation module 1006, for generating multiple functionality controls based on the second engine, and by the functionality controls to described image frame
It is controlled.
Each functional module and above-mentioned image processing method due to the image processing apparatus of example embodiments of the present invention
Example embodiment the step of it is corresponding, therefore details are not described herein.
It should be noted that although being referred to several modules or unit of image processing apparatus in the above detailed description,
It is that this division is not enforceable.In fact, embodiment according to the present invention, two or more above-described modules or
The feature and function of person's unit can embody in a module or unit.Conversely, an above-described module or
The feature and function of unit can be to be embodied by multiple modules or unit with further division.
Those skilled in the art after considering the specification and implementing the invention disclosed here, will readily occur to of the invention its
Its embodiment.The present invention is directed to cover any variations, uses, or adaptations of the invention, these modifications, purposes or
Person's adaptive change follows general principle of the invention and including the undocumented common knowledge in the art of the present invention
Or conventional techniques.The description and examples are only to be considered as illustrative, and true scope and spirit of the invention are by appended
Claim is pointed out.
It should be understood that the present invention is not limited to the precise structure already described above and shown in the accompanying drawings, and
And various modifications and changes may be made without departing from the scope thereof.The scope of the present invention is only limited by the attached claims.
Claims (15)
1. a kind of image processing method characterized by comprising
Multiple images frame is generated by the first engine renders, and each described image frame is subjected to coding compression to form video flowing；
The video flowing is sent to terminal device in real time；
User that the terminal device is sent in real time is received to the operation information of the video flowing, and passes through first engine pair
The operation information is handled to realize the control to described image frame.
2. image processing method according to claim 1, which is characterized in that generate multiple images by the first engine renders
Frame, and each described image frame is subjected to coding compression to form video flowing, comprising:
Determine the differential pixels between the other picture frames and key images frame in each described image frame；
Coding compression is carried out to the key images frame and the differential pixels, to form the video flowing.
3. image processing method according to claim 1, which is characterized in that described image frame is the picture frame of rgb format,
Multiple images frame is generated by the first engine renders, and each described image frame is subjected to coding compression to form video flowing, comprising:
The picture frame of the rgb format is converted to the picture frame of yuv format, to obtain multiple target image frames；
Each target image frame is subjected to coding compression to form the video flowing.
4. image processing method according to claim 3, which is characterized in that each target image frame is carried out coding pressure
Contracting is to form the video flowing, comprising:
Coding compression is carried out to each target image frame respectively by pre-arranged code method, to be formed and each target image
The corresponding image data packet of frame；
The video flowing is formed according to each described image data packet.
5. image processing method according to claim 4, which is characterized in that by pre-arranged code method respectively to each described
Target image frame carries out coding compression, to form image data packet corresponding with each target image frame, comprising:
When forming described image data packet, the corresponding sequence number of write-in described image frame is corresponding with described image data packet is generated
Timestamp.
6. a kind of image processing method characterized by comprising
A video flowing is received, the video flowing includes multiple images data packet, and each described image data packet is by multiple
Picture frame carries out what coding compression was formed respectively；
Described image data packet is decoded, to obtain each described image frame；
User is received to the operation information of described image frame, and the operation information is sent to server.
7. image processing method according to claim 6, which is characterized in that described image frame is the picture frame of yuv format,
The method also includes:
After obtaining each described image frame, the picture frame of the yuv format is converted to the picture frame of rgb format.
8. image processing method according to claim 6, which is characterized in that described image data packet includes described image frame
Sequence number with generate the corresponding timestamp of described image data packet.
9. image processing method according to claim 8, which is characterized in that the method also includes:
According to the sequence number of described image frame carry out frame detection, and according to the timestamp determine adjacent described image frame it
Between time interval.
10. image processing method according to claim 9, which is characterized in that carried out according to the sequence number of described image frame
Fall frame detection, comprising:
Judge that the loss picture frame has with the presence or absence of picture frame is lost in the video flowing according to the sequence number of described image frame
There is target sequence number；
In the presence of if it is determined that, then using picture frame corresponding with the adjacent previous sequence number of the target sequence number as the target
The corresponding picture frame of sequence number.
11. image processing method according to claim 6, which is characterized in that the method also includes:
Multiple functionality controls are generated based on the second engine, and described image frame is controlled by the functionality controls.
12. a kind of image processing apparatus characterized by comprising
Video flowing generation module for generating multiple images frame by the first engine renders, and each described image frame is compiled
Code compression is to form video flowing；
Sending module, for the video flowing to be sent to terminal device in real time；
Interactive module for receiving user that the terminal device is sent in real time to the operation information of the video flowing, and passes through
First engine handles to realize the control to described image frame the operation information.
13. a kind of image processing apparatus characterized by comprising
Receiving module, for receiving a video flowing, the video flowing includes multiple images data packet, and each described image data packet
It is to be formed by carrying out coding compression respectively to multiple images frame；
Decoder module, for being decoded to described image data packet, to obtain each described image frame；
Data acquisition module is sent to clothes for receiving user to the operation information of described image frame, and by the operation information
Business device.
14. a kind of computer-readable medium, is stored thereon with computer program, which is characterized in that described program is held by processor
Such as image processing method of any of claims 1-11 is realized when row.
15. a kind of electronic equipment characterized by comprising
One or more processors；
Storage device, for storing one or more programs, when one or more of programs are by one or more of processing
When device executes, so that one or more of processors realize such as image processing method of any of claims 1-11
Method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811253125.XA CN109510990B (en) | 2018-10-25 | 2018-10-25 | Image processing method and device, computer readable storage medium and electronic device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811253125.XA CN109510990B (en) | 2018-10-25 | 2018-10-25 | Image processing method and device, computer readable storage medium and electronic device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109510990A true CN109510990A (en) | 2019-03-22 |
| CN109510990B CN109510990B (en) | 2022-03-29 |
Family
ID=65746070
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811253125.XA Active CN109510990B (en) | 2018-10-25 | 2018-10-25 | Image processing method and device, computer readable storage medium and electronic device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109510990B (en) |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110490962A (en) * | 2019-08-20 | 2019-11-22 | 武汉邦拓信息科技有限公司 | A kind of remote rendering technology based on video flowing |
| CN110634564A (en) * | 2019-09-16 | 2019-12-31 | 腾讯科技（深圳）有限公司 | Pathological information processing method, device and system, electronic equipment and storage medium |
| CN110827380A (en) * | 2019-09-19 | 2020-02-21 | 北京铂石空间科技有限公司 | Image rendering method and device, electronic equipment and computer readable medium |
| CN110865787A (en) * | 2019-11-25 | 2020-03-06 | 京东方科技集团股份有限公司 | Image processing method, server, client and image processing system |
| CN111836116A (en) * | 2020-08-06 | 2020-10-27 | 武汉大势智慧科技有限公司 | Network self-adaptive rendering video display method and system |
| CN112546633A (en) * | 2020-12-10 | 2021-03-26 | 网易（杭州）网络有限公司 | Virtual scene processing method, device, equipment and storage medium |
| CN113055729A (en) * | 2019-12-27 | 2021-06-29 | 中兴通讯股份有限公司 | Cloud set top box image differential processing method and device |
| CN113163259A (en) * | 2021-05-10 | 2021-07-23 | 宝宝巴士股份有限公司 | FFmpeg-based video node rendering method and device |
| CN113760431A (en) * | 2021-08-30 | 2021-12-07 | 百度在线网络技术（北京）有限公司 | Application program control method and device, electronic equipment and readable storage medium |
| CN113784094A (en) * | 2021-08-31 | 2021-12-10 | 上海三旺奇通信息科技有限公司 | Video data processing method, gateway, terminal device and storage medium |
| CN114445264A (en) * | 2022-01-25 | 2022-05-06 | 上海秉匠信息科技有限公司 | Texture compression method and device, electronic equipment and computer readable storage medium |
| CN114626975A (en) * | 2022-03-21 | 2022-06-14 | 北京字跳网络技术有限公司 | Data processing method, apparatus, device, storage medium and program product |
| WO2022161209A1 (en) * | 2021-01-26 | 2022-08-04 | 腾讯科技（深圳）有限公司 | Image frame loss detection method, device, storage medium, and computer program product |
| CN115225615A (en) * | 2022-06-30 | 2022-10-21 | 如你所视(北京)科技有限公司 | Illusion engine pixel streaming method and device |
| CN115379207A (en) * | 2022-08-24 | 2022-11-22 | 中国第一汽车股份有限公司 | Camera simulation method and device, electronic equipment and readable medium |
| CN116778046A (en) * | 2023-08-28 | 2023-09-19 | 乐元素科技(北京)股份有限公司 | Hair model processing method, device, equipment and medium based on multithreading |
| WO2023245495A1 (en) * | 2022-06-22 | 2023-12-28 | 云智联网络科技(北京)有限公司 | Method and apparatus for converting rendered data into video stream, and electronic device |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101340598A (en) * | 2008-08-07 | 2009-01-07 | 北京衡准科技有限公司 | Method and apparatus for implementing three-dimensional playing of media |
| CN103096128A (en) * | 2011-11-07 | 2013-05-08 | 中国移动通信集团公司 | Method capable of achieving video interaction, server, terminal and system |
| WO2016049187A1 (en) * | 2014-09-23 | 2016-03-31 | Lincolnpeak | Systems, methods, and software for processing a question relative to one or more of a plurality of population research databases |
| CN105791977A (en) * | 2016-02-26 | 2016-07-20 | 北京视博云科技有限公司 | Virtual reality data processing method and system based on cloud service and devices |
| CN107529711A (en) * | 2017-08-01 | 2018-01-02 | 杭州安恒信息技术有限公司 | The display methods and device of Streaming Media |
| CN107979763A (en) * | 2016-10-21 | 2018-05-01 | 阿里巴巴集团控股有限公司 | A kind of virtual reality device generation video, playback method, apparatus and system |
| CN108668167A (en) * | 2017-03-28 | 2018-10-16 | 中国移动通信有限公司研究院 | A kind of method and device of video reduction |
-
2018
- 2018-10-25 CN CN201811253125.XA patent/CN109510990B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101340598A (en) * | 2008-08-07 | 2009-01-07 | 北京衡准科技有限公司 | Method and apparatus for implementing three-dimensional playing of media |
| CN103096128A (en) * | 2011-11-07 | 2013-05-08 | 中国移动通信集团公司 | Method capable of achieving video interaction, server, terminal and system |
| WO2016049187A1 (en) * | 2014-09-23 | 2016-03-31 | Lincolnpeak | Systems, methods, and software for processing a question relative to one or more of a plurality of population research databases |
| CN105791977A (en) * | 2016-02-26 | 2016-07-20 | 北京视博云科技有限公司 | Virtual reality data processing method and system based on cloud service and devices |
| CN107979763A (en) * | 2016-10-21 | 2018-05-01 | 阿里巴巴集团控股有限公司 | A kind of virtual reality device generation video, playback method, apparatus and system |
| CN108668167A (en) * | 2017-03-28 | 2018-10-16 | 中国移动通信有限公司研究院 | A kind of method and device of video reduction |
| CN107529711A (en) * | 2017-08-01 | 2018-01-02 | 杭州安恒信息技术有限公司 | The display methods and device of Streaming Media |
Cited By (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110490962B (en) * | 2019-08-20 | 2023-09-15 | 武汉邦拓信息科技有限公司 | Remote rendering method based on video stream |
| CN110490962A (en) * | 2019-08-20 | 2019-11-22 | 武汉邦拓信息科技有限公司 | A kind of remote rendering technology based on video flowing |
| CN110634564A (en) * | 2019-09-16 | 2019-12-31 | 腾讯科技（深圳）有限公司 | Pathological information processing method, device and system, electronic equipment and storage medium |
| CN110827380A (en) * | 2019-09-19 | 2020-02-21 | 北京铂石空间科技有限公司 | Image rendering method and device, electronic equipment and computer readable medium |
| CN110827380B (en) * | 2019-09-19 | 2023-10-17 | 北京铂石空间科技有限公司 | Image rendering method and device, electronic equipment and computer readable medium |
| CN110865787A (en) * | 2019-11-25 | 2020-03-06 | 京东方科技集团股份有限公司 | Image processing method, server, client and image processing system |
| CN113055729A (en) * | 2019-12-27 | 2021-06-29 | 中兴通讯股份有限公司 | Cloud set top box image differential processing method and device |
| CN111836116A (en) * | 2020-08-06 | 2020-10-27 | 武汉大势智慧科技有限公司 | Network self-adaptive rendering video display method and system |
| CN112546633A (en) * | 2020-12-10 | 2021-03-26 | 网易（杭州）网络有限公司 | Virtual scene processing method, device, equipment and storage medium |
| WO2022161209A1 (en) * | 2021-01-26 | 2022-08-04 | 腾讯科技（深圳）有限公司 | Image frame loss detection method, device, storage medium, and computer program product |
| CN113163259A (en) * | 2021-05-10 | 2021-07-23 | 宝宝巴士股份有限公司 | FFmpeg-based video node rendering method and device |
| CN113760431A (en) * | 2021-08-30 | 2021-12-07 | 百度在线网络技术（北京）有限公司 | Application program control method and device, electronic equipment and readable storage medium |
| CN113760431B (en) * | 2021-08-30 | 2024-03-29 | 百度在线网络技术（北京）有限公司 | Application control method and device, electronic equipment and readable storage medium |
| CN113784094A (en) * | 2021-08-31 | 2021-12-10 | 上海三旺奇通信息科技有限公司 | Video data processing method, gateway, terminal device and storage medium |
| CN113784094B (en) * | 2021-08-31 | 2024-04-30 | 上海三旺奇通信息科技有限公司 | Video data processing method, gateway, terminal device and storage medium |
| CN114445264A (en) * | 2022-01-25 | 2022-05-06 | 上海秉匠信息科技有限公司 | Texture compression method and device, electronic equipment and computer readable storage medium |
| CN114626975A (en) * | 2022-03-21 | 2022-06-14 | 北京字跳网络技术有限公司 | Data processing method, apparatus, device, storage medium and program product |
| WO2023245495A1 (en) * | 2022-06-22 | 2023-12-28 | 云智联网络科技(北京)有限公司 | Method and apparatus for converting rendered data into video stream, and electronic device |
| CN115225615A (en) * | 2022-06-30 | 2022-10-21 | 如你所视(北京)科技有限公司 | Illusion engine pixel streaming method and device |
| CN115225615B (en) * | 2022-06-30 | 2024-02-23 | 如你所视(北京)科技有限公司 | Illusion engine pixel streaming method and device |
| CN115379207A (en) * | 2022-08-24 | 2022-11-22 | 中国第一汽车股份有限公司 | Camera simulation method and device, electronic equipment and readable medium |
| CN116778046A (en) * | 2023-08-28 | 2023-09-19 | 乐元素科技(北京)股份有限公司 | Hair model processing method, device, equipment and medium based on multithreading |
| CN116778046B (en) * | 2023-08-28 | 2023-10-27 | 乐元素科技(北京)股份有限公司 | Hair model processing method, device, equipment and medium based on multithreading |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109510990B (en) | 2022-03-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109510990A (en) | Image processing method and device, computer readable storage medium, electronic equipment | |
| CN104244088B (en) | Display controller, screen picture transmission device and screen picture transfer approach | |
| CN105791977B (en) | Virtual reality data processing method, equipment and system based on cloud service | |
| CN109309842B (en) | Live broadcast data processing method and device, computer equipment and storage medium | |
| CN102254292A (en) | Remote 3D instruction rendering system and method | |
| CN113946402A (en) | Cloud mobile phone acceleration method, system, equipment and storage medium based on rendering separation | |
| CN108235055B (en) | Method and device for realizing transparent video in AR scene | |
| CN103688240A (en) | Method for transmitting digital scene description data and transmitter and receiver scene processing device | |
| CN110070496A (en) | Generation method, device and the hardware device of image special effect | |
| US8581924B2 (en) | Method and mobile terminal for enabling animation during screen-switching | |
| CN112799891B (en) | iOS device testing method, device, system, storage medium and computer device | |
| EP3024223A1 (en) | Videoconference terminal, secondary-stream data accessing method, and computer storage medium | |
| CN110290290A (en) | Implementation method, device, computer equipment and the storage medium of the studio cloud VR | |
| WO2021244210A1 (en) | Method and apparatus for controlling cursor control, and related device | |
| CN110166796A (en) | Processing method, device, computer-readable medium and the electronic equipment of video frame | |
| CN101908033A (en) | Method and device for multiuser to remotely use multiple master computers | |
| CN106899860B (en) | Pass through the system and method for transmission of network media | |
| CN113838184A (en) | Rendering method, device and system | |
| CN105874807A (en) | Methods, systems, and media for remote rendering of web content on a television device | |
| CN105407313A (en) | Video calling method, equipment and system | |
| CN105578110A (en) | Video call method, device and system | |
| WO2024027611A1 (en) | Video live streaming method and apparatus, electronic device and storage medium | |
| CN114222185B (en) | Video playing method, terminal equipment and storage medium | |
| CN111356009B (en) | Audio data processing method and device, storage medium and terminal | |
| Altenhofen et al. | Rixels: Towards secure interactive 3d graphics in engineering clouds |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |