WO2007059670A1 - Procede de correction d'une caracteristique gamma dans un flux de codes video et unite de commande multipoint - Google Patents
Procede de correction d'une caracteristique gamma dans un flux de codes video et unite de commande multipoint Download PDFInfo
- Publication number
- WO2007059670A1 WO2007059670A1 PCT/CN2006/001671 CN2006001671W WO2007059670A1 WO 2007059670 A1 WO2007059670 A1 WO 2007059670A1 CN 2006001671 W CN2006001671 W CN 2006001671W WO 2007059670 A1 WO2007059670 A1 WO 2007059670A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- gamma
- video
- gamma characteristic
- code stream
- characteristic parameter
- Prior art date
Links
- 238000012937 correction Methods 0.000 title claims abstract description 142
- 238000000034 method Methods 0.000 title claims abstract description 109
- 230000006854 communication Effects 0.000 claims abstract description 57
- 238000004891 communication Methods 0.000 claims abstract description 51
- 230000005540 biological transmission Effects 0.000 claims abstract description 10
- 238000004364 calculation method Methods 0.000 claims description 17
- 238000012545 processing Methods 0.000 claims description 13
- 230000008569 process Effects 0.000 claims description 12
- 239000000284 extract Substances 0.000 claims description 11
- 238000000605 extraction Methods 0.000 claims description 7
- 238000010586 diagram Methods 0.000 description 15
- 230000015572 biosynthetic process Effects 0.000 description 14
- 238000003786 synthesis reaction Methods 0.000 description 14
- 230000003287 optical effect Effects 0.000 description 10
- 230000006835 compression Effects 0.000 description 7
- 238000007906 compression Methods 0.000 description 7
- 238000007429 general method Methods 0.000 description 7
- 239000002131 composite material Substances 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 230000007246 mechanism Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000005259 measurement Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 230000006837 decompression Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 239000000203 mixture Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 230000000153 supplemental effect Effects 0.000 description 2
- 208000011580 syndromic disease Diseases 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 101100236764 Caenorhabditis elegans mcu-1 gene Proteins 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 210000004899 c-terminal region Anatomy 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 230000009194 climbing Effects 0.000 description 1
- 150000001875 compounds Chemical class 0.000 description 1
- 238000002939 conjugate gradient method Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000007667 floating Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000012804 iterative process Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000004886 process control Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 239000000047 product Substances 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 238000004451 qualitative analysis Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 238000002945 steepest descent method Methods 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 230000002194 synthesizing effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/64—Circuits for processing colour signals
- H04N9/68—Circuits for processing colour signals for controlling the amplitude of colour signals, e.g. automatic chroma control circuits
- H04N9/69—Circuits for processing colour signals for controlling the amplitude of colour signals, e.g. automatic chroma control circuits for modifying the colour signals by gamma correction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/14—Picture signal circuitry for video frequency region
- H04N5/20—Circuitry for controlling amplitude response
- H04N5/202—Gamma control
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/15—Conference systems
- H04N7/152—Multipoint control units therefor
Definitions
- the present invention relates to multimedia communication, and in particular to a video code stream gamma characteristic correction method and a multi-point control unit in video communication. Background technique
- Video communication is currently being widely used with the rapid development of broadband networks.
- video conferencing and video telephony services are becoming the basic services on NGN (Next Generation Network).
- NGN Next Generation Network
- Telecom operators in various countries also attach great importance to this market opportunity.
- video communication services will become an important business growth point for operators.
- a key issue in developing such a business is to improve the End-to-end User Experience (or Quality of Experience).
- QoS drop, delay, jitter, R factor, etc.
- the methods and techniques for improving the end-to-end user experience mainly focus on pre-processing (Post-processing) related to ensuring network QoS and video compression coding, but lack of attention to the luminance change caused by Gamma characteristics. And the solution of the system, but the seriousness of the problem has attracted the attention of some major international telecom operators. France Telecom recently proposed in the ITU-T International Telecommunications Union to consider the impact of Gamma features on the communication user experience in video communications and proposes to address such issues.
- a terminal In the video communication process, in a video communication terminal (hereinafter referred to as a terminal), an optical signal from a scene (person, background, file, etc.) that needs to be transmitted enters the camera/camera, and is converted into a digital image signal by A/D. Then, after compression coding, it is transmitted and sent to the other terminal to be restored to a digital image signal by Decompression decoding, and then displayed on the display device, and finally becomes an optical signal that is perceived by the human eye.
- Image brightness signal during this process Luminance, here Is a generalized luminance signal, that is, the initial optical signal, the electrical signal, and then the digitized image luminance/gray signal.
- the signal of each phase contains the information of the luminance signal. Therefore, in a broad sense, the luminance signal passes. A number of links) have gone through multiple links.
- Fig. 1 is a schematic diagram of the model of the link Gamma characteristic.
- the Gamma characteristic is that the input/output relationship of the luminance signal of a link is not linear, but a nonlinearity.
- the effect of the non-linear link distortion of Gamma is shown in Fig. 2.
- the brightness of the above-mentioned row of gray squares is linearly increasing from 0.1 to 1.0.
- the lower line is distorted by the Gamma nonlinear link, and the brightness is increased according to the power function.
- Figure 3 shows the cascade of multiple links (Cascading or tandem).
- G CT (.) G (1) (.). G (2) (.). G (3) (.) ........ " (.) o (.)
- CT indicates Cascaded Total, which means the cascading total
- the model of the correction link is the inverse model of the Gamma characteristic equivalent model. If the equivalent model can be expressed by a function relation, the function relation of the inverse model is its inverse function. Obviously, G g (.) and G c (.) are inverse functions of each other. In general, for a function, the inverse function does not necessarily have a solution (or even if the solution exists, it cannot be obtained by calculation).
- each link has its gamma characteristic. They are cascaded. There is no general way to implement gamma correction from optical signal into camera/camera to display image. method. Therefore, there is no general method for the degradation of video quality caused by the Gamma problem. At the same time, the Gamma characteristic parameters between different terminals are unknown to each other. After the video is sent from the terminal A to the terminal B, how to implement the Gamma correction is also an unsolved problem.
- the situation is more complicated, because the MCU (Multipoint Control Unit) is used to mix video for multiple terminals, and then sent to each terminal, and the gamma characteristics of each sub-image in the multi-picture image are Different, it is more difficult to achieve Gamma correction.
- MCU Multipoint Control Unit
- the main functions of the MCU are multi-point control MC (Multipoint Control) and multi-point processing (MP Multipoint Processing).
- the MC includes communication process control, conference control, etc.
- MP includes media processing, video stream forwarding or multi-picture synthesis, audio Mix and so on.
- the main UVCP function associated with the present invention is the function of video processing in the MP.
- the MCU can Working in the following state, the first two are also called video forwarding mode:
- each terminal participating in a conference can freely choose to view the conference video of any other terminal, and the MCU is responsible for forwarding the video of the terminal being viewed to the receiving terminal.
- how many terminals can choose to watch other sites is determined by the number of free viewing sites that the multipoint control unit can support (depending on the device capabilities or the settings of the operational control system). For example, terminal A can select to view the site video of terminal B, and terminal B can select terminal C to view, and the like;
- the venue specifies the viewing (ie, the venue broadcast) status.
- the MCU is responsible for watching the video of a terminal site, that is, the video broadcast of the designated site, by the chair terminal in the conference (if any) or the conference organizer specifies that all the terminals in the conference view the video of the terminal site through the operation control system. Go out. For example, if terminal X is selected, then other terminals watch X's site video;
- the MCU combines the video of multiple terminal sites into a multi-screen, multi-screen layout (how many conference sites are included, how these site images are arranged and relative size, etc.) by the conference chair terminal (if any) or the conference organizer through the operational control system. To specify. If the selected terminal site is , X 2 , X 3 X c , then a possible layout is shown in Figure 6.
- the first type is a synthesis mode of decoding and re-encoding first.
- the MCU first decompresses and decodes the video code stream from each terminal, restores it to an uncompressed digital video format, and then assembles into a multi-screen specific layout combination. a picture image, and then compression-coding the multi-picture image to form a new multi-picture video code stream;
- the second type is the direct synthesis mode.
- the MCU combines the video code streams of various terminals into a new code stream according to a certain standard grammar. For example, the H.261, H.263, and H.264 protocols allow such combinations in syntax.
- a problem with direct synthesis The terminal needs to first reduce the resolution of the video image.
- the normal image is CIF (Common Interchange Format), and in order to synthesize a multi-picture (2x2 layout) containing 4 sub-pictures, Reduce the resolution to the original in the terminal One-fourth of the time, the QCIF (Quarter CIF) format. This situation limits that images from the terminal can only be used to synthesize multiple pictures, but cannot be viewed by other terminals at normal resolution.
- CIF Common Interchange Format
- Direct synthesis reduces the problems caused by decoding and re-encoding, such as requiring high processing power and damage images. Problems such as quality can reduce the cost of MCUs and improve communication capacity and communication quality, so they are now widely used.
- the terminal that an MCU can control is limited.
- multiple MCUs can be cascaded as shown in FIG. 7, for example,
- the uppermost MCU1 controls the following second layer of MCUs 2.1 to 2.m (total of m), while the latter controls several third layer MCUs (total n).
- Each layer of the MCU can directly control a certain number of terminals, or can indirectly control a certain number of terminals through the lower MCUs it controls.
- each MCU can be functionally internally broken down as follows: a Multipoint Controller and multiple Multipoint Processors.
- This Decomposed Model is currently very popular, which makes it more flexible in product implementation and provides more types of MCU-like telecommunications equipment. By expanding the processing power of multiple multi-point processors, it is also possible to achieve the goal of supporting a larger multimedia communication network.
- the current technology is based on the assumption that Gamma correction is performed on each terminal:
- the display device and camera/camera of each terminal are designed and manufactured according to the standard Gamma characteristics, that is, the Gamma parameter of the display is 2.2, and the Gamma parameter of the camera/camera is 0.45;
- the video stream data sent by each terminal is corrected by Gamma, and the calibration is based on the ability to cooperate with the display device of the other terminal.
- each terminal implements Gamma correction locally.
- the correction method is shown in Figure 4.
- the disadvantages of the existing calibration methods are obvious, because the three assumptions required are already in the current situation. The more you can't get it.
- high-end cameras generally provide gamma correction, but a large number of low-end cameras are not available. If the camera is capable of providing gamma correction, it means that the camera as a whole, the external gamma characteristics are given by Equation 2.
- the reality is that telecom operators are now pushing video communications to the public, and it is inevitable to provide very cheap terminals to attract the general public, so the use of cheap cameras is inevitable.
- CCD Charge Coupled Device
- the Gamma characteristic parameters of the gamma link must be used.
- the high-end device conforms to the ideal Gamma characteristic. It can use a power function such as Equation 1 or 2, including a pure power function and a power function with offset.
- the ideal state is difficult to obtain, so the representation of the Gamma property in the form of a pure power function, or a power function with an offset, may be very inaccurate in some cases.
- the camera is a cheap camera, its Gamma feature may not be a power function. In this case, the function representation is invalid.
- telecom operators are vigorously promoting video communication to the public, and it is inevitable to provide very cheap terminals to attract the general public. Therefore, the use of inexpensive cameras is inevitable.
- the formula 4 is the Gamma characteristic representation of the camera
- the formula 5 is the Gamma characteristic representation of the CRT display.
- the representation of the piecewise function is relatively accurate, it has a narrow scope of application and can only be applied to high-end (so-called broadcast) devices. It is not very good for a large number of mid-range and low-end devices, especially cameras. Summary of the invention
- the invention provides a video code stream gamma characteristic correction method and a multi-point control list in video communication Yuan to solve the gamma distortion problem of video images in existing multimedia communication.
- a method for correcting video stream gamma characteristics in video communication comprising the following steps:
- the transmitting terminal sends a video code stream to the receiving terminal, where the video code stream includes video data generated according to the video image of the transmitting terminal and gamma characteristic parameter information of the transmitting terminal;
- the receiving terminal receives the video code stream, restores the video image according to the video data, and performs gamma correction on the video image according to the local gamma characteristic parameter information and the transmitting terminal gamma characteristic parameter information.
- At least one transmitting terminal sends the video code stream to a multipoint control unit; and the multipoint control unit sends the video code stream to the receiving terminal.
- the multipoint control unit synthesizes the video code streams from the at least two transmitting terminals into one multi-picture video code stream and sends the same to the receiving terminal.
- the multi-point control unit respectively restores the corresponding video image according to the video data of each of the transmitting terminals, and respectively assembles the respective video images as a sub-screen and assembles into a multi-screen image, and then generates video data of the multi-screen image.
- the multi-point control unit separately extracts the video data in each video stream of the transmitting terminal and directly combines the video data according to the sub-picture assembling order, and then synthesizes the combined video data and the gamma characteristic parameter information corresponding to each sub-picture into a multi-picture video code stream is sent to the receiving terminal, wherein the gamma characteristic parameter information of the sub-picture is sequentially determined according to the assembling position and order of the sub-picture; and the receiving terminal extracts each sub-picture from the combined video data.
- the sub screens are assembled into a multi-screen image according to the assembly order. And indicating, in the video code stream, indication information that identifies that the video image has gamma distortion; and the receiving terminal confirms that the gamma characteristic parameter information is carried in the multi-picture video code stream according to the indication information. .
- the video code stream transmitting terminal sends a first video code stream to the multi-point control unit, where the first video code stream includes video data generated according to the video image of the transmitting terminal and gamma characteristic parameter information of the transmitting terminal;
- the multipoint control unit receives the first video code stream, restores the video image according to the video data, and performs gamma correction on the video image according to gamma characteristic parameter information of the transmitting terminal;
- the video data of the corrected video image is generated and carried in the second video stream for transmission to the receiving terminal.
- the method further includes: receiving, by the receiving terminal, the second video code stream, restoring the corrected video image, and performing the correction again according to the local gamma characteristic parameter.
- the multi-point control unit combines the first video code streams from the at least two transmitting terminals and respectively corrects the video images and assembles them into a multi-picture image; and, according to the multi-picture image, generates multi-picture image video data and carries them in the The second video stream is sent to the receiving terminal.
- the receiving terminal restores the multi-screen image according to the video data of the multi-screen image, and respectively corrects the video image of each sub-screen according to the local gamma characteristic parameter.
- the first video code stream is provided with first indication information indicating that the video image has gamma distortion; and/or, the second video code stream is provided with a second indication that the identification video image has undergone one gamma correction information.
- the multipoint control unit confirms, according to the first indication information, gamma characteristic parameter information that carries the sending terminal in the first video bitstream.
- the transmitting terminal gamma characteristic parameter information is a process in which the video image is collected, processed, and formed into a video code stream: a gamma characteristic parameter of each gamma characteristic link at the transmitting terminal, ⁇ and the cascading order of each gamma characteristic link; or the equivalent gamma characteristic parameter determined by the transmitting terminal according to all gamma links that have passed.
- the transmitting terminal carries the gamma characteristic parameter information of each frame of the video image of the terminal corresponding to the video data of the video image in the video code stream, and sends the video data to the receiving terminal; or, the sending terminal will be surely
- the gamma characteristic parameter information is carried in the video code stream and sent to the receiving terminal; or the transmitting terminal carries the initial gamma characteristic parameter information in the video code stream and sends the information to the receiving terminal at the beginning of the communication, and
- the transmitting terminal carries the initial gamma characteristic parameter information in the video code stream and sends the information to the receiving terminal at the beginning of the communication, and
- the updated gamma characteristic parameter information is carried in the video code stream and sent to the receiving terminal.
- the gamma characteristic parameter or the equivalent gamma characteristic parameter of each of the gamma links includes a set of output luminance values corresponding to each level of the input luminance signal value of the set level.
- the setting level of the input luminance signal value is 0-255, and the value of the luminance value is an integer.
- a gamma parameter information field is extended in the video code stream, and the input luminance signal value set and/or the output luminance signal value set are combined into a binary code stream and carried in the gamma parameter information field. Transfer in.
- the gamma parameter information field includes gamma parameter information and a start delimiter and an end delimiter located at both ends of the gamma parameter information, where the start delimiter and the end delimiter are used to determine the The scope of the information domain.
- the message for carrying the gamma parameter information is extended in the supplemental enhancement information SEI field of the H.264 code stream.
- the present invention also provides a multipoint control unit, including a multipoint processor, the multipoint processor including:
- a gamma characteristic parameter storage module configured to store a gamma characteristic parameter of the video image transmitting terminal; a gamma characteristic correction module, connected to the gamma characteristic parameter storage module, configured to correct a gamma characteristic parameter according to the video image transmitting terminal The gamma characteristic in the video image.
- the multipoint processor further includes:
- a video stream transceiver module configured to send and receive a video code stream, where the video code stream includes a video image Data and gamma characteristic parameter information of the video stream transmitting terminal;
- a video data encoding and decoding module connected between the video stream transceiver module and the gamma characteristic correction module, configured to decode the video data from the video stream and send it to the gamma characteristic correction module for correction; or Encoding video data according to the corrected video image;
- a gamma characteristic parameter information extraction and adding module is connected between the video stream transceiver module and the video data codec module, and is configured to extract gamma characteristic parameter information of the transmitting terminal from the received video code stream and deposit the information a gamma characteristic parameter storage module; or, extracting gamma characteristic parameter information from the gamma characteristic parameter storage module and appending to the video code stream to be sent;
- a multi-screen assembly module configured to connect the gamma characteristic correction module, to assemble the received video images of at least two terminals into a multi-picture image, and send the multi-picture image to a video data codec module or gamma Feature parameter information extraction and additional modules.
- An equivalent gamma characteristic parameter calculation submodule configured to calculate an equivalent gamma characteristic parameter according to a single gamma link characteristic parameter of the transmitting terminal and input the gamma correction generating submodule;
- a gamma correction generation sub-module for correcting gamma characteristics of the video image based on the equivalent gamma characteristic parameter.
- the multipoint control unit further includes: a multipoint controller coupled to the multipoint processor for transmitting a control signal for performing gamma characteristic correction to the multipoint processor.
- the multipoint processors are arranged in parallel in plurality.
- the method of the present invention performs the gamma characteristic parameter of the transmitting terminal in the video code stream, so that the receiving terminal corrects the received video image according to the gamma characteristic parameter of the transmitting terminal and the gamma characteristic parameter of the local terminal, or
- the video image is corrected once by the multi-point control unit according to the gamma characteristic parameter of the transmitting terminal, and then corrected by the receiving terminal according to the local gamma characteristic parameter, and the gamma distortion in the video image can be corrected at the receiving terminal.
- the present invention further provides a multi-point control unit having a gamma characteristic correction function, which can correct or re-transmit the forwarded video image according to the control instruction, and is compatible with the existing Direct forwarding function.
- Figure 1 is a general model of the link Gamma characteristics
- Figure 2 is a schematic diagram of the luminance signal distortion caused by the link Gamma characteristic bow I;
- Figure 3 is a general model of multi-link cascaded Gamma characteristics
- Figure 4 is a schematic diagram of the Gamma characteristic for correcting a single link
- Figure 5 is a schematic diagram of the Gamma characteristics for correcting a plurality of given links
- Figure 6 An example of a multi-screen layout
- Figure 7 is a schematic diagram of the MCU increasing the control function by cascading
- Figure 9 is a schematic diagram of possible correction points when multiple Gamma links are cascaded.
- FIG. 10 is a schematic diagram of a general Gamma characteristic correction method when a plurality of Gamma links are cascaded;
- FIG. 11 is a schematic diagram showing a principle of a Gamma correction method in a conference free viewing state according to the first embodiment of the present invention;
- FIG. 12 is a schematic diagram showing the principle of a Gamma correction method in a state in which a site is specified for viewing (ie, a venue broadcast) according to the second embodiment of the present invention
- FIG. 13 is a schematic diagram of a principle of decoding, re-encoding, and synthesizing a video image into a multi-picture in a multi-point control unit according to the third embodiment of the present invention
- FIG. 14 is a schematic structural diagram of a multipoint control unit having a video image gamma characteristic correction function according to Embodiment 6 of the present invention.
- FIG. 15 is a schematic diagram of a binary format of a Gamma parameter information area defined by the present invention. detailed description
- a first step is to use a Gamma path to correct multiple links.
- the Gamma path of the video signal from the acquisition to the output includes N t cascaded Gamma links. You can select the correction point between any two links.
- the passed Gamma path is divided into two sections, the correction point includes a link, and the correction point includes N p links.
- the equivalent integrated Gamma characteristics of the N a link and the N p link are respectively determined.
- the calibrator links of the equivalent Gamma characteristics are respectively constructed, and then the two syndromes are combined and inserted into the calibration points.
- the correction link is actually a combination of two syndromes, which includes the following steps:
- the correction point links are divided into the N t N a number cascaded links and links of N p cascaded after the correction point before the correction point, and
- N a ⁇ 0, N p ⁇ 0, N a + N p N t ;
- the construction method of the correction link model includes one of the following:
- Direct calculation method calculating the correction signal of the last output signal of the N a links by using the function relation of the first inverse model and the function relation of the second inverse model in real time;
- Two-step calculation method Calculate the primary correction signal of the last output signal of the N a link in real time by using the functional relationship of the first inverse model, and calculate the secondary correction signal of the primary correction signal by using the functional relationship of the second inverse model, The secondary correction signal is used as the correction signal;
- Lookup table method Calculate the correction value corresponding to the plurality of sample values in the value interval of the last output signal of the N a link according to the direct calculation method or the two-step calculation method, and protect the corresponding relationship There is a data table, and then the correction value of any value to be corrected is determined by querying the data table in real time.
- the equivalent model itself has no analytical form (for example, using the look-up table method, of course, the inverse function has no analytical form), then the inverse model is the inverse table of the data table, one table There are two columns, many rows, the left column (input column) is the sampled value of the input signal, that is, the signal value to be corrected, and the right column (output column) is the corresponding output signal value, that is, the corrected signal value, the number of rows depends on In the number of sampling points, the more the number of rows, the more accurate, the inverse table is the new data table obtained by adjusting the left and right columns. And for video data with a large amount of data, the amount of calculation for real-time calculation is large, and the table lookup is the most practical method.
- the terminal must be able to determine all the gamma features on the terminal.
- the equivalent model adopts the function representation
- the transmitting terminal or the receiving terminal respectively determines the equivalent of each gamma characteristic link of the local end by the following method.
- the present invention also describes a method for determining the equivalent model of each Gamma characteristic link and its parameters, including the following steps:
- Equation 8 Equation 8 or Equation 9 below:
- This process is actually an iterative process.
- the parameters p and ⁇ are constantly adjusted, and the function value F is decreasing.
- the function value falls below a given threshold, it is considered that the minimum point has been found.
- the corresponding parameters ⁇ and ⁇ at this time are considered to be the true parameters of the application environment model. If ⁇ ⁇ ) -(l- ) 2 is iterated over it, it still can't make
- the above method can also be used to measure the multi-link integrated Gamma characteristic model and its parameters.
- the measurement method steps are identical. It should be pointed out that, from the formal point of view, the functional relationship of multiple Gamma characteristic models cascaded by multiple Gamma links can still adopt Equation 6. And the two types shown in Equation 7, but in the first type of comprehensive characteristic model, according to the qualitative analysis results and the empirical values actually measured, the range of the index ⁇ becomes ⁇ > 0, and in the second comprehensive characteristic model, the index The range of ⁇ values becomes ⁇ >0.
- the calibration model can be determined according to the foregoing method.
- Gg can be found as long as Gg(.) satisfies certain conditions.
- Gc(.) can be found for Ga(.) and Gp (.) performs Gamma correction.
- the external optical signal enters the camera/head, and is processed by each link, and finally the display to the local display is converted into an optical signal; or the compressed coding link is transmitted to the other terminal through the communication network, and then goes through
- the compression decoding is restored into an image, and then displayed on the multi-terminal display to be converted into an optical signal; or after the optical signal enters from the camera/head, it is converted into an electrical signal, and then, after a certain process, is written to the file and saved in the hard disk.
- the path through which each video signal passes can be regarded as a Gamma path as shown in Fig. 3.
- the general method can be used to correct the Gamma characteristics.
- each terminal can detect the equivalent model of the local Gamma link and its parameters.
- the present invention proposes the following technical idea:
- Providing a mechanism for transmitting and exchanging information of a Gamma characteristic parameter in a video stream coding compression or transport bearer protocol which may be:
- the transmitting terminal carries the single-segment gamma characteristic parameter of each gamma characteristic link through which the video image passes, and the cascading order of each gamma characteristic link is carried in the video code stream; or the transmitting terminal passes all according to the video image
- the gamma characteristic link calculates the equivalent gamma characteristic parameter and then carries it in the chirp frequency stream for transmission.
- the transmitting terminal may carry corresponding gamma characteristic parameter information for each video image; or, the transmitting terminal will measure the gamma at regular intervals (the period may be measured by time, or may be measured by a certain number of image frames)
- the characteristic parameter information is carried in the video code stream and sent to the receiving terminal; or, at the beginning of the communication, the initial gamma characteristic parameter information is carried in the video code stream and sent to the receiving terminal, and in the communication process, When the end gamma characteristic parameter changes, the updated gamma characteristic parameter information is carried in the video code stream and sent to the receiving terminal.
- the above-mentioned universal correction method is used for correction, and the correction can be performed in a multipoint control unit or a multipoint communication server having similar functions, and the correction can also be performed by the video data receiving terminal. Therefore, the same applies to the case of two-point communication, or the multi-point communication situation in which the multipoint control unit does not participate.
- multi-picture mode the multi-picture mode includes: a first synthesis mode and a direct synthesis mode (commonly known as software multi-picture);
- Gamma distortion is a kind of nonlinear distortion (distortion, or distortion), which refers to the sequential action of signals through one or more Gamma links to form Gamma distortion;
- Full Gamma correction state The signal with Gamma distortion is calibrated by Gamma and reaches the same state as the distortion-free state, called the full Gamma correction state;
- Gamma Track Used in the video stream to carry the data portion of the transmitted Gamma message.
- Embodiment 1 Gamma correction in the free viewing state of the conference site
- the MCU does not process the video stream from the terminal, but only forwards it.
- the terminal A selects the video of the terminal B to be freely viewed, and the terminal B joins the video conference to send the local video code stream to the MCU, where the video code stream carries the Gamma characteristic parameter of the terminal B, assuming that the terminal B has N B Gamma links; Forwarding the video code stream from the terminal B to the terminal A;
- the terminal A receives and analyzes the video code stream from the terminal B, extracts the carried gamma characteristic parameter and the digital video data from the video code stream, and performs correction according to the gamma characteristic parameter of the terminal A itself, and the specific correction method is the aforementioned universal correction.
- Method, the principle of correction is shown in Figure 11, the calibration process includes the following steps:
- Terminal A obtains digital video data by decompressing and decoding from the video code stream of terminal B;
- Terminal A extracts the Gamma parameters of the N B Gamma links from the video code stream of the terminal B;
- Terminal A calculates the Gamma parameters of the equivalent Gamma link G BE Q by N B Gamma links according to Equation 3, where EQ represents Equivalent;
- terminal A calculates the Gamma parameter of the equivalent link GAEQ of multiple Gamma links according to formula 3;
- Terminal A calculates the correction link G c according to G BE Q and GAEQ.
- Gamma parameter of r
- Terminal A corrects the video from terminal B according to Gcor's Gamma parameter and displays it through the local display;
- the specific correction may be a direct calculation method based on a function representation or a table lookup method based on a table lookup.
- the terminal A and the terminal B directly exchange video data streams.
- Embodiment 2 Gamma correction in the state of designated meeting (ie, site broadcast)
- the MCU simply broadcasts the video code stream from the designated viewing terminal to all terminals that see the conference without any other processing.
- the Gamma parameters are extracted, and then corrected, and the calibration process is the same as in the first embodiment.
- each terminal that views the video of the X terminal is synchronously corrected and viewed, and the Gamma correction in the state of the specified viewing (ie, site broadcast) of the venue is realized.
- Embodiment 3 Multi-picture state, the Mmm works in the first decoding and re-encoding synthesis mode in the gamma correction method
- the MCU first decompresses and decodes the C pictures (the total number of sub-pictures in the multi-screen layout) from the terminal, X 2 . X c , and restores them to uncompressed numbers.
- the video format is then combined into a multi-picture image, and then encoded and compressed.
- the MCU can perform some processing and gamma characteristic correction on the uncompressed digital video format that has been restored.
- the specific # grammar is: '
- the MCU decompresses and decodes the video from the terminal Xi (l ⁇ i ⁇ C) to form an uncompressed digital video format;
- the MCU extracts the Gamma parameter information carried from the video stream from the mobile phone
- the MCU forms an equivalent link G ( XEQ according to the extracted Gamma parameter according to the extracted gamma parameter, and calculates a corresponding gamma characteristic parameter;
- the MCU performs correction according to G (i ) XEQ and the corresponding Gamma characteristic parameters, and the result of the correction forms a complete correction state. That is, the corrected video is out of the state without Gamma distortion;
- the MCU repeats the above steps 1 ⁇ 4 until the video from the C terminals is corrected;
- the MCU composes the multi-screen image by C sub-screen combination according to the specified multi-screen layout
- the MCU performs compression coding on the combined multi-picture image
- the MCU fills in the Gamma track information carrying the Gamma information for the newly generated multi-picture video stream.
- the entire multi-picture is also free of Gamma distortion. Therefore the Gamma track does not actually contain any Gamma parameters. Just need to set a correction flag in the Gamma track to indicate whether the video is without Gamma. Really. There are many ways to set the correction flag, for example: Set this correction flag to indicate that the video stream is present.
- Gamma distorted when not set, indicates no Gamma distortion; or the correction flag is set to 1 to indicate the presence of Gamma distortion, and a setting of 0 indicates no Gamma distortion. Obviously the sign should be located in Gamma
- the MCU or the terminal reads the flag, it first discriminates the correction flag. If there is no Gamma distortion, it means that there is no Gamma parameter behind, and there is no need to continue reading the data backward. Otherwise, the data should be read continuously.
- the MCU sends the newly generated multi-picture video code stream to multiple terminals.
- each receiving terminal receives a multi-picture video stream
- the processing procedure of each receiving terminal is the same, as shown in FIG. 13, and the specific # ⁇ is as follows:
- the terminal Y calculates the Gamma parameter of the equivalent link G YEQ of the plurality of Gamma links; the terminal Y first determines whether the received multi-picture video has Gamma distortion according to the correction flag, and the multi-picture received by the terminal Y in this embodiment
- the video is without Gamma distortion, so the terminal Y only needs to consider its own gamma characteristics when correcting.
- Terminal Y calculates the correction link G c according to G YEQ .
- Gamma parameter of r
- GYEQ is the equivalent Gamma characteristic of terminal Y.
- Terminal Y corrects for multi-picture video from the MCU.
- the specific correction may be a direct calculation method based on a function representation or a table lookup method based on a table lookup.
- a direct calculation method based on a function representation or a table lookup method based on a table lookup.
- a table lookup method based on a table lookup.
- Embodiment 4 Multi-picture state, the MMM works in the first decoding and re-encoding synthesis mode, the Gamma correction method 2
- the MCU After the MCU decompresses and decodes the sub-picture video from each terminal XI, X 2 . X c to the uncompressed format, the MCU does not perform the gamma correction, directly combines the multi-picture image, and then performs coding compression. A multi-picture video stream is formed. Then, after the correction flag indicating the gamma distortion is set on the gamma track of the video stream, the gamma parameter information carried by the original terminal video stream Gamma track is sequentially copied according to the arrangement order of the respective sub-pictures given by the specified multi-screen layout. A gamma track that forms a multi-picture video stream is formed at a position corresponding to the sub-picture.
- the terminal Y decompresses and decodes the multi-picture video stream to an uncompressed format, thereby obtaining an uncompressed format of each sub-picture;
- the terminal extracts Gamma parameter information corresponding to each sub-picture from the Gamma track of the multi-picture video stream;
- terminal Y calculates the Gamma parameter of the equivalent link G YEQ of its own multiple Gamma links.
- the terminal Y calculates the Gamma parameter of the correction link G COT according to G (i) XEQ and G YE Q;
- the terminal Y corrects the video of the sub-picture i according to the Gmm Gamma parameter and displays it. Terminal Y repeats steps 3 to 5 above until all sub-pictures have been corrected.
- the specific correction method may be based on a direct calculation method of a function representation or a look-up table method based on a table lookup.
- a direct calculation method of a function representation or a look-up table method based on a table lookup.
- a look-up table method based on a table lookup.
- Embodiment 5 Multi-picture state, gamma correction of MCU working in direct synthesis mode
- the MCU does not process the video code stream from the terminal Xi (l ⁇ i ⁇ C), but only conforms to the specific video compression.
- the code stream is synthesized in a manner that encodes the protocol syntax to form a composite code stream.
- the composite stream also has a Gamma track.
- the MCU is first provided with a correction flag indicating that the video stream has Gamma distortion.
- the terminal Y extracts the Gamma parameter information corresponding to each sub-picture from the Gamma track of the multi-picture video stream;
- terminal Y calculates the Gamma parameter of the equivalent link GYEQ of its own multiple Gamma links. Number
- the terminal Y locates and extracts the video stream corresponding to the sub-picture i from the composite code stream and then performs decompression decoding to restore to the uncompressed format;
- Terminal Y calculates the correction link G c according to G (i) XEQ and GYEQ. Gamma parameter of r ;
- Terminal Y is based on the calibration link Gc;.
- the Gamma parameter of ⁇ corrects the video of the sub-picture i.
- Terminal Y repeats steps 1-5 above until all sub-pictures have been corrected.
- the specific correction method may be based on a direct calculation method of a function representation or a look-up table method based on a table lookup.
- a direct calculation method of a function representation or a look-up table method based on a table lookup.
- a look-up table method based on a table lookup.
- Embodiment 6 MCU with Gamma Correction Function
- the present invention also provides an MCU having a Gamma correction function, including a multipoint controller 100 and a multipoint processor 200.
- the MCU with the Gamma correction function of the present invention is mainly improved on the multipoint processor 200.
- the multipoint processor 200 includes the following functional modules:
- a gamma characteristic parameter storage module 201 configured to store a gamma characteristic parameter of the video image transmitting terminal
- the gamma characteristic correction module 202 is connected to the gamma characteristic parameter storage module 201 for correcting the gamma characteristic in the video image according to the gamma characteristic parameter of the video image transmitting terminal.
- the video stream transceiver module 203 is configured to send and receive a video code stream, where the video code stream includes video image data and gamma characteristic parameter information of the video code stream transmitting terminal;
- the video data encoding and decoding module 204 is connected between the video stream transceiver module 203 and the gamma characteristic correction module 202, and is configured to decode the video data from the video stream and send it to the gamma characteristic correction module 202. Correction; or, encode video data based on the corrected video image.
- the gamma characteristic parameter information extraction/addition module 205 is connected between the video code stream transceiver module 203 and the video data codec module 204, and is configured to extract gamma characteristic parameter information of the transmitting terminal from the received video code stream. And stored in the gamma characteristic parameter storage module 201; or, the gamma characteristic parameter storage module 201 is extracted from the gamma characteristic parameter storage module 201 and added to the video code stream to be transmitted.
- the multi-screen assembly module 206 is configured to connect the gamma characteristic correction module 202, and assemble the received video images of at least two terminals into a multi-screen image, and send the multi-picture image to the video data codec module 204. .
- the correction module 202 sends the corrected video image directly to the video data codec module 204.
- the multi-picture assembly module 206 gamma characteristic parameter information extraction/addition module 205 is directly connected.
- the direct synthesis mode is adopted, the video code stream is not required to be coded and decoded.
- the various modes of multi-screen assembly are preset in the multi-screen assembly module 206 and the gamma characteristic parameter information extraction/addition module 205 for multi-screen assembly according to the corresponding assembly mode and determining to add each sub-picture to the video stream.
- the order of the gamma characteristic parameter information of the screen is preset in the multi-screen assembly module 206 and the gamma characteristic parameter information extraction/addition module 205 for multi-screen assembly according to the corresponding assembly mode and determining to add each sub-picture to the video stream.
- the gamma characteristic correction module 202 includes:
- the equivalent gamma characteristic parameter calculation sub-module 2021 is configured to calculate an equivalent gamma characteristic parameter according to a single gamma link characteristic parameter of the transmitting terminal and input the gamma correction generating sub-module 2022;
- the gamma correction generation sub-module 2022 is configured to correct the gamma characteristic of the video image according to the equivalent gamma characteristic parameter.
- the multipoint processor 200 is connected to the multipoint controller 100 to perform gamma characteristic correction on the received video image in accordance with an instruction from the multipoint controller 100.
- a plurality of multipoint processors 200 can be arranged in parallel, and communication between more multimedia communication terminals can be controlled.
- the video transceiver module 203 directly forwards the video stream according to the instructions of the multipoint controller.
- Gamma parameter information as Gamma characteristics as a function of its domain (L in the range) and range (L. ut value range) are Interval [0,1] or one of its subintervals, such as [0.1, 0.9].
- the key to table lookup is two sets (or sequences) ⁇ L in ( i )
- 0 ⁇ i ⁇ Nl ⁇ is the set of Discrete Values of Input Luminance
- 0 ⁇ i ⁇ Nl ⁇ is the set of discrete values of output luminance ( Set ofDiscrete Values of Output Luminance ).
- L in L.
- the value of ut must be in the interval [0,1].
- the input and output brightness of each Gamma link may not be in the [0,1] range.
- the input luminance value is first normalized during processing.
- the luminance signal with the value in the interval [0,1] is called the normalized luminance L n in , the superscript n represents Normalized "normalization", and the luminance signal with the value between 0-MaxL a in is called actual
- the actual brightness is mapped to the [0,1] interval, and the method used is as shown in Equation 10:
- MaxL a ir MaxL a . Ut 255
- the brightness level may increase, and from a technical implementation point of view, it will generally increase to an integer power of two, For example, 512 or even 1024, the general form is 2 D , and D is a natural number. Then, as shown in Equation 12 and Equation 13,
- the brightness signal level is 256 levels
- the brightness value is 0-255
- each brightness value can be represented by 8 bits (one byte).
- 0 ⁇ i ⁇ N-l ⁇ also belongs to ⁇ 0, 1 , 2, 3, 4... 254, 255 ⁇ .
- the Gamma characteristic look-up table applicable to the current video communication technology indicates that the data structure is still as shown in Table 1. Further analysis shows that the values of the left column (Left Column) of Table 1 are fixed, which must be ⁇ 0. In the order of 1, 2, 3, 4...254, 255 ⁇ , both parties to the communication know this set and order, and there is no need to transmit the left column value in the communication.
- the Gamma parameter information is transmitted.
- the general method is to define a block (block or Region) in the data area of the communication protocol allowing extension and custom content, and to store the binary stream representation of the Gamma parameter continuously. .
- the block is then encapsulated in the code stream of the protocol. This area is called the Gamma parameter information area.
- FIG. 15 There may be multiple Gamma links including a camera/camera and a display device in the multimedia communication terminal. Therefore, for a terminal to transmit all its Gamma parameter information to other communication terminals or other devices on the network, such as multi-point control units, then all of its Gamma links should be followed by their front-to-back (or reverse).
- the cascading order write its gamma characteristic parameters to the Gamma parameter information area.
- the receiving terminal or other network device can extract the Gamma parameter information in order from the information area for carrying the Gamma parameters. Therefore, the specific format for defining the Gamma parameter information area is as shown in Figure 15, which includes the following parts:
- End flag 16 bits (2 bytes), the value is OxFOOF;
- Total area length 16 bits (2 bytes), the total length of the Gamma parameter information area in bytes (including the start and end flags).
- the combination of the above three can locate the location of the Gamma information area in the code stream.
- Link 1 ⁇ T parameter sub-area There are a total of such sub-areas, corresponding to one Gamma ring Section.
- each sub-area is defined as follows:
- Sub-area length 16 bits (2 bytes), the length of the sub-area in bytes (including the transfer mode byte).
- the format of the Gamma parameter information area that does not depend on the specific bearer protocol is defined above. If the video code stream adopts the H.264 protocol coding mode, the present invention also uses the H.264 protocol to carry the Gamma parameter information by using the H.264 message extension mechanism. Methods.
- H.264 provides a variety of mechanisms for message extension.
- the supplemental enhancement information SEI Supplemental Enhancement Information
- SEI Supplemental Enhancement Information
- the data representation area of the SEI is independent of the video coding data.
- the usage method is given in the description of NAL (Network Abstraction Layer) in the H.264 protocol.
- the basic unit of H.264 code stream is NALU (NAL Unit, Network Abstraction Layer Unit).
- NALU can carry various H.264 data types, such as Sequence parameters, Picture parameters, Slice data. (ie specific image data), and SEI message data.
- the SEI is used to carry various messages and support message extension.
- SEI domain is used to transmit messages customized for a specific purpose without affecting the compatibility based on the H.264 video communication system.
- the NALU that carries the SEI message is called SEI NALU.
- An SEI NALU contains one or more SEI messages.
- Each SEI message contains variables, mainly Payload Type and Payload Size, which indicate the type and size of the message payload.
- the grammar and semantics of some commonly used H.264 SEI messages are defined in H.264 Annex D.8, D.9.
- the payload contained in the NALU is called RBSP (Raw-Byte Sequence Payload), and the SEI is a type of RBSP.
- RBSP Raw-Byte Sequence Payload
- SEI is a type of RBSP.
- Table 2 the grammar of SEI RBSP is shown in Table 2: Table 2.
- SEI RBSP in one NALU can contain multiple SEI messages.
- Table 3 The structure of an SEI message is shown in Table 3:
- payloadType + last__payload— type— byte
- payloadSize + last__payload— size— byte
- Sei_payload ( payloadType, payloadSize ) 5
- H.264 Annex D.8 defines the grammar message structure reserved for future extended SEI messages as shown in Table 4:
- each SEI field contains one or more SEI messages, which in turn consist of SEI header information and SEI payload.
- the SEI header information includes two codewords: one codeword gives the type of payload in the SEI message, and the other codeword gives the size of the payload.
- the payload type is between 0 and 255, it is represented by a byte 0x00 to OxFE.
- the type is between 256 and 511, it is represented by two bytes OxFFOO to OxFFFF.
- the type is greater than 511, the method is deduced by analogy. Users can customize any of a variety of load types.
- the type of the Gamma parameter information area to be carried can be defined as any existing SEI payload type that is not defined, because there are many possible extended message types for other purposes at present, and can be defined as OxFFFF to avoid conflicts. ( 511 ), OxFFFF is the theoretical maximum. Then, the Gamma parameter information area filled in according to the definition is directly put into the SEI payload, and the purpose of carrying and transmitting the Gamma parameter information by using the SEI message extension mechanism is realized. It should be noted that the SEI payload type is taken as OxFFFF, which is only one embodiment of the present invention. For other values, it is also within the scope of the present invention.
- the present invention provides two methods for gamma characteristic correction in multimedia communication: one is performed on the video image receiving terminal side, and the video image receiving terminal according to the gamma characteristic parameter of the local end and the gamma characteristic of the transmitting terminal The parameters are corrected once;
- the second is to perform stepwise correction in the multi-point control unit and the video receiving terminal.
- the multi-point control unit corrects the gamma distortion introduced by the gamma link of the transmitting terminal according to the gamma characteristic parameter of the transmitting terminal, and then the video receiving terminal according to the video receiving terminal.
- the gamma characteristic parameter of the local end corrects the gamma distortion introduced by the gamma link of the receiving terminal.
- the above method can be selected according to the specific scene of the multimedia communication, and can well correct the gamma distortion introduced from the collection, transmission, reception and display of the video image.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Picture Signal Circuits (AREA)
Description
一种视频码流伽玛特性校正方法及多点控制单元 技术领域
本发明涉及多媒体通信 , 特别涉及一种视频通信中视频码流伽玛特性校 正方法及多点控制单元。 背景技术
视频通信目前正在随着宽带网络的迅速发展而得到日益广泛的应用, 在 国内和国际上, 视频会议和可视电话业务正在成为 NGN ( Next Generation Network, 下一代网络)上的基本业务。 各国的电信运营商也非常重视这个市 场机会, 可以预期在未来几年中, 视频通信业务将成为运营商重要的业务增 长点。 发展此类业务的一个关键问题是提高端到端(End-to-end )的用户体验 ( User Experience , 或者叫做 Quality of Experience )。 用户体验中除了网格的 QoS (丢包, 延迟, 抖动, R因子等)参数外, 对于视频, 因为各个环节引起 的 Gamma非线性问题, 造成对于亮度信号的畸变 (Distortion ), 也是影响最 终用户体验的重要因素。 但是目前, 提高端到端用户体验的方法和技术主要 集中在保证网络 QoS 和视频压缩编码相关的前后处理 (Pre-processing, Post-processing )方面, 而对于 Gamma特性引起的亮度 ^变问题缺乏关注和 系统的解决方法, 但是该问题的严重性已经引起了一些国际大电信运营商的 关注。 法国电信(France Telecom )在国际电信联盟 ITU-T近期就提出了要在 视频通信中考虑 Gamma特性对于通信用户体验的影响, 并建议解决此类问 题。
视频通信过程中, 在一个视频通信终端 (以下简称终端) 中, 从需要被 传送,的场景(人物、 背景、文件等)的光信号进入到摄像机 /摄像头, 经过 A/D 转换成数字图像信号, 再经过压缩编码, 传送出去到达对方终端经过去压缩 ( Decompression )解码还原为数字图像信号, 然后再在显示设备上显示出来, 最终又变成光信号被人眼感知。 这个过程中图像亮度信号(Luminance, 这里
是一种广义的亮度信号, 即一开始的光信号, 到电信号, 再到数字化的图像 亮度 /灰度信号, 每个阶段的信号都含有亮度信号的信息, 因此广义来说, 亮 度信号经过了多个环节)经过了多个环节。
如图 1所示, 图 1为环节 Gamma特性的模型示意图, Gamma特性就是 一个环节的亮度信号输入-输出关系不是线性的, 而是一种非线性。 Gamma非 线性环节畸变的影响如图 2所示, 上面的一行灰度方块亮度是线性递增的, 从 0.1到 1.0, 下面一行是经过 Gamma非线性环节畸变的, 亮度是按照幂函 数规律递增的。
在实际中, Gamma非线性是由不同原因引起的,例如: CRT ( Cathode Ray Tube, 阴极射线管)显示器的 Gamma特性在理想状况下满足公式 1 :
Lou厂 Ljn (1)
而对应的摄像机 /摄像头的理想 Gamma满足公式 2:
Lcm厂 Ljn (2)
从 Gamma问题的起源来看, 起源于 CRT显示器, 因为其 Gamma值是 2.2 , 为了补偿掉这个非线性, 在摄像机中人为引入了 Gamma值 0.45。 如果在系统 中只存在两个 Gamma环节: CRT显示器和摄像机,那么可以实现完全的 Gamma 校正。 需要说明的是, 这里的输入和输出亮度信号都是在各自的坐标空间中 进行了规一化(Normalized ) , 即 0≤L。ut≤l, 0≤Lin≤l。 而其它类型的显示器, 比如液晶显示器的 Gamma函数形式或者不同、 或者虽然形式上也是幂函数但 是参数不同。
如图 3所示, 图 3为多个环节级联(Cascading或者叫做串联)起来环节
Gamma特性的模型示意图, 总的 Gamma特性等于各个环节 Gamma函数的复合
(Composition), 满足公式 3:
GCT (.) = G(1) (.)。 G(2) (.)。 G(3) (.) ........ " (.) o (.)
'-G = G(")(G("- "(GC- 2 ....... G^(G(1) ¾))))) (3)
"。 "表示函数的复合运算。 CT表示 Cascaded Total, 即级联总 Gamma的意
理想的情况是输入光信号从进入摄像头到最终在显示屏上显示输出光信 号, 输入和输出亮度信号之间存在线性关系, 即: L。ut= Lin, 这样人看到的景 物才和原来的完全一样, 用户体验最好。
要获得线性关系, 必须对于具有非线性 Gamma特性环节进行 Gamma校正 ( Gamma Correction )。 如图 4所示, 对于一个环节来说, 其 Gamma特性给定, 那么可以用另外一个校正环节和它进行级联, 来使得级联后总的 Gamma特性 称为真正的线性关系, 从而达到了补偿掉给定环节非线性的目的, 校正环节 的模型为 Gamma特性等效模型的逆模型, 如果等效模型可以用函数关系式表 示, 则逆模型的函数关系式为其反函数。 显然, Gg(.)和 Gc(.)互为反函数。 一般 情况下, 对于一个函数, 要获得其反函数不一定有解(或者即使解存在, 也 无法用计算的方法获得) 。
实际应用中更多的情况如图 5所示, 校正环节需要插入到前后两个给定环 节之间, 此时 Gc(.)情况更加复杂, Gc(.)和 Ga(.)或者 Gp(.)不再是筒单的反函数关 系。
视频通信中, 终端内部存在多个环节, 每个环节都有其 Gamma特性, 它 们之间级联起来, 目前还没有一般性的方法来实现从光信号进入摄像机 /摄像 头到显示器显示图像的 Gamma校正方法。 因此, 因为 Gamma问题引起的视频 质量下降还没有一般性的方法。 同时, 不同终端之间的 Gamma特性参数相互 是不知道的, 那么视频从终端 A发送到终端 B后, 如何实现 Gamma校正, 也是 一个没有解决的问题。 在多方视频通信中, 情况更加复杂, 因为涉及到 MCU ( Multipoint Control Unit, 多点控制单元)对于多个终端来视频进行混合, 然 后发给各个终端, 多画面图像中各个子图像的 Gamma特性都不同, 要实现 Gamma校正更力口困难。
MCU的主要功能是多点控制 MC ( Multipoint Control )和多点处理(MP Multipoint Processing ) , MC包括通信过程控制、 会议控制等, MP包括媒体处 理、 视频码流的转发或者多画面合成、 音频的混合等。 与本发明相关的主要 UVCP功能, 更严格来说, 是 MP中视频处理的功能。 在视频方面, MCU可以
. 工作在如下状态, 其中前两种也被称为视频转发模式:
1、 会场自由观看状态
在这种状态下, 参加一个会议中的每个终端都可以自由选择观看任何其 它终端的会场视频, MCU负责将被收看的终端视频转发给接收终端。 当然有 多少终端能够选择观看其它会场受到多点控制单元能够支持最大的自由观看 会场数决定的(取决于设备能力或者运营控制系统的设定)。 比如终端 A可以 选择观看终端 B的会场视频, 终端 B又可以选择终端 C来观看等;
2、 会场指定观看 (即会场广播)状态
由会议中的主席终端 (如果存在的话)或者会议组织者通过运营控制系 统指定会议中的所有终端都观看某个终端会场的视频 , MCU负责将被收看的 终端视频, 即被指定会场的视频广播出去。 比如终端 X被选择, 那么其它终端 都观看 X的会场视频;
3、 多画面状态
MCU将多个终端会场的视频合成为多画面, 多画面的布局(包含多少个 会场, 这些会场图像如何排列和相对大小等 ) 由会议主席终端 (如果存在的 话 )或者会议组织者通过运营控制系统来指定的。 如果被选择的终端会场为 、 X2、 X3 Xc, 那么一种可能的布局如图 6所示。
MCU合成多画面的主要方法有如下两类:
第一类为先解码再编码的合成模式, MCU将来自各个终端的视频码流首 先进行去压缩解码, 恢复成为未压缩的数字视频格式, 然后再按照某种多画 面的具体布局组合拼装成多画面图像, 再对多画面图像进行压缩编码形成新 的多画面视频码流;
第二类为直接合成模式, MCU将各种终端的视频码流按照一定符合标准 的语法组合成新的码流。 比如 H.261、 H.263、 H.264协议在语法上都允许这样 的组合。 一般来说, 直接合成有一个问题就是, 终端需要首先降低视频图像 的分辨率, 比如正常图像是 CIF ( Common Interchange Format ) , 而为了合成 一个包含 4个子画面的多画面 (2x2布局) , 则需要在终端把分辨率降低到原
来的 1/4, 即 QCIF ( Quarter CIF )格式。 这种情况局限了来自该终端的图像只 能用于合成多画面, 而不能以正常分辨率被其他终端观看。 当然这种限制在 具体应用环境中并不造成很大的问题, 而直接合成模式带来的好处也是显然, 直接合成减少了解码和重新编码带来的问题, 例如需要高的处理能力和损伤 图像质量等问题, 可以降低 MCU成本并提高通信容量和通信质量, 因此现在 也广泛被采用。
因为 MCU要进行处理和计算, 因此一个 MCU能够控制的终端是有限的, 为了组成更大的通信网络, 支持更多的终端,可以采用如图 7所示的多个 MCU 级联的方式,例如最上层的 MCU1控制下面的第二层的 MCU2.1至 2.m (共 m个), 而后者分别控制若干个第三层的 MCU (共 n个) 。 每一层的 MCU可以直接控 制一定数量的终端, 也可以同时通过其所控制的下层 MCU间接控制一定数量 的终端。
如图 8所示,每一个 MCU从功能上在内部可以进行如下分解: 一个多点控 制器( Multipoint Controller )和多个多点处理器( Multipoint Processor ) 。 这 种分解模型 ( Decomposed Model ) 目前非常流行, 这样可以在产品的实现上 更加灵活, 提供更多类型的 MCU类的电信设备。 通过将多个多点处理器进行 堆叠(Stack )扩大处理能力, 也能够达到支持更大多媒体通信网絡组网的目 的。
目前的技术都是基于如下假设, 在每个终端上来进行 Gamma校正:
1、 每个终端的显示设备和摄像机 /摄像头都是按照标准的 Gamma特性要 求设计和制造, 即显示器的 Gamma参数为 2.2, 而摄像机 /摄像头的 Gamma参数 为 0.45;
2、 在摄像机 /摄像头和显示设备中间没有其它的 Gamma环节;
3、每个终端发送出去的视频码流数据都经过了 Gamma校正, 并且这种校 正是基于能够和对方终端的显示设备配合来实现的。
在以上假设下, 每个终端在本地实现 Gamma校正, 校正方法如图 4所示, 现有校正方法的缺点是显然的, 因为所要求的三个假设在目前情况下已经越
来越不能成立了。 现有技术中, 高端的摄像机一般都能够提供 Gamma校正功 能, 但是大量低端的摄像头却不能提供。 如果摄像机能够提供 Gamma校正功 能, 则意味着摄像机作为一个整体, 对外的 Gamma特性由公式 2给出。 而现实 情况是目前电信运营商都在大力推动面向公众的视频通信, 必然要提供非常 便宜的终端才能吸引广大公众, 这样使用廉价的摄像头是不可避免的。 这类 廉价摄像头可能存在非线性的 Gamma特性, 但不是公式 2给出的形式, 甚至根 本不具有冪函数的形式。根据实际的测试结果,发现了很多基于 CCD ( Charge Coupled Device ) 的廉价摄像头的 Gamma特性, 最接近的幂函数是 L。ut=Lin 22 左右, 并且很多数据点偏离这条曲线, 因此很难说是一个幂函数曲线。 另夕卜, 因为在终端系统中, 完全可能存在其它的 Gamma环节, 因此即使摄像机具有 了由公式 2给出的标准的 Gamma特性, 也可能无法达到完全 Gamma校正的效 果。
如果要进行 Gamma校正, 必须使用伽玛环节的 Gamma特性参数, 高端 设备符合理想的 Gamma特性, 可以利用例如公式 1或 2的幂函数, 包括纯幂 函数和带有偏移的冪函数形式。但是大多数中端或低端设备的 Gamma特性只 能采用如表 1所示的查表 ( LUT=Look-Up Table )表示方法。
因为 Gamma函数的定义域和值域都是 [0,1]区间, 因此, 可以采用离散化 的方式来表示这种函数关系。 如表 1所示, 该表的形式是两列 N行, 左边是 Lin 的 N个离散值, 右边是对应的 L。ut的 N的离散值。 因此, 要根据 Lin的数值来计 算对应的!^^, 只要查表就可以完成了。 如果 Lin的数值不在左列中, 可以采用 插值的方法来计算对应的 L。ut值。
表 1. Gamma参数的查表表示方法
Gamma特性参数的两种方式相比来说, 有各自的优点和不足。 函数表示 简洁, 传递参数量很少。 但是计算麻烦, 尤其计算浮点数的非整数次方是非 常耗时的。 采用查表表示, 计算筒洁, 并且可以适应任意函数形式, 通用性 好, 但是需要传递参^:较多。
理想状态是很难获得的, 因此用純幂函数形式, 或者带有偏移的幂函数 形式来表示 Gamma特性, 在有些情况下可能是很不精确的。 比如如果摄像头 是廉价的摄像头, 可能其 Gamma特性就不是幂函数形式。 这种情况下, 函数 表示方式就失效了。 目前, 电信运营商都在大力推动面向公众的视频通信, 必然要提供非常便宜的终端才能吸引广大公众, 这样使用廉价的摄像头是不 可避免的。
在现有技术一中, 函数表示都不够精确, 过于简单化和理想化, 处于简 化的需要, 在对于 Gamma校正精度要求不高的情况下是可以使用。 但是, 在 一些对于质量要求很高的应用场景下, 采用了满足公式 4和公式 5分段函数模 型:
其中, 公式 4是摄像机的 Gamma特性表示, 公式 5是 CRT显示器的 Gamma 特性表示。 分段函数的表述虽然比较精确, 但是适用范围狭窄, 只能适合高 端 (所谓广播级)设备, 对于大量中端和低端的设备, 尤其是摄像头无法很 好使用。 发明内容
本发明提供一种视频通信中视频码流伽玛特性校正方法及多点控制单
元, 以解决现有多媒体通信中视频图像的伽玛失真问题。
一种视频通信中视频码流伽玛特性校正方法, 包括如下步驟:
发送终端向接收终端发送视频码流, 所述视频码流中包括根据发送终端 视频图像生成的视频数据和发送终端的伽玛特性参数信息;
接收终端接收所述视频码流, 根据所述视频数据还原所述视频图像, 并 根据本端伽玛特性参数信息和发送终端伽玛特性参数信息对该视频图像进行 伽玛校正。
所述方法中, 至少一个发送终端将所述视频码流发送给多点控制单元; 多点控制单元将所述视频码流发送给接收终端。
其中, 多点控制单元将来自至少两个发送终端的视频码流合成为一个多 画面视频码流后发送给接收终端。
其中: 多点控制单元分别根据每一个发送终端的视频数据还原出对应的 视频图像 , 将所述各个视频图像分别作为子画面并拼装成一个多画面图像, 然后生成所述多画面图像的视频数据, 并将该多画面图像的视频数据和每一 个子画面对应的伽玛特性参数信息合成为一个多画面视频码流后发送给接收 终端, 其中, 所述子画面的伽玛特性参数信息顺序根据子画面的拼装位置和 顺序确定; 以及接收终端根据所述多画面图像的视频数据还原出所述多画面 图像, 并根据本端伽玛特性参数信息和每一个子画面对应的伽玛特性参数分 别校正每一个子画面的视频图像。
其中: 多点控制单元分別提取每一个发送终端视频码流中的视频数据并 根据子画面拼装顺序直接进行复合, 然后将复合后的视频数据和每一个子画 面对应的伽玛特性参数信息合成为一个多画面视频码流后发送给接收终端, 其中, 所述子画面的伽玛特性参数信息顺序根据子画面的拼装位置和顺序确 定; 以及, 接收终端从复合后的视频数据中提取各子画面对应的视频数据并 还原出各子画面的视频图像, 然后根据本端伽玛特性参数信息和每一个子画 面对应的伽玛特性参数分别校正每一个子画面的视频图像 , 再将校正后的各 个子画面根据拼装顺序拼装为多画面图像。
在所述视频码流中还设置有标识视频图像存在伽玛失真的指示信息; 以 及, 接收终端才艮据所述指示信息确认所述多画面视频码流中携带有所述伽玛 特性参数信息。
本发明所述另一种视频通信中视频码流伽玛特性校正方法, 包括如下步 骤:
视频码流发送终端向多点控制单元发送第一视频码流, 所述第一视频码 流中包括根据发送终端视频图像生成的视频数据和发送终端的伽玛特性参数 信息;
多点控制单元接收所述第一视频码流, 根据所述视频数据还原所述视频 图像, 并根据发送终端的伽玛特性参数信息对于所述视频图像进行一次伽玛 校正; 然后
生成经过一次校正的视频图像的视频数据并携带在第二视频码流中发送 给接收终端。
所述方法还包括: 接收终端接收所述第二视频码流, 还原校正后视频图 像并根据本端伽玛特性参数再次进行校正。
其中: 多点控制单元将来自至少两个发送终端的第一视频码流并分别校 正视频图像并拼装为多画面图像; 以及, 才 据所述多画面图像生成多画面图 像视频数据并携带在所述第二视频码流中发送给接收终端。
其中: 接收终端根据所述多画面图像的视频数据还原出所述多画面图像, 并分别根据本端伽玛特性参数再次校正每一个子画面的视频图像。
所述第一视频码流中设置有标识视频图像存在伽玛失真的第一指示信 息; 和 /或, 所述第二视频码流中设置有标识视频图像已经经过一次伽玛校正 的第二指示信息。
多点控制单元根据所述第一指示信息确认所述第一视频码流中携带有发 送终端的伽玛特性参数信息。
所述的发送终端伽玛特性参数信息为所述视频图像在其采集、 处理和形 成视频码流的过程中: 在发送终端经过每一个伽玛特性环节的伽玛特性参数,
. ― 以及各伽玛特性环节的级联顺序; 或者发送终端根据经过的所有伽玛 ¾环 节确定的等效伽玛特性参数。
所述方法中: 发送终端将本终端每一帧视频图像的伽玛特性参数信息分 别对应该视频图像的视频数据携带在所述视频码流中发送给接收终端; 或者, 发送终端将每隔一定周期将伽玛特性参数信息携带在所迷视频码流中发送给 接收终端; 或者, 发送终端在通信开始时将初始伽玛特性参数信息携带在所 述视频码流中发送给接收终端, 并在通信过程中, 当本端伽玛特性参数发生 变化时, 再将更新的伽玛特性参数信息携带在所述视频码流中发送给接收终 端。
所述每一个伽玛环节的伽玛特性参数或等效伽玛特性参数包括对应设定 等级的每一级输入亮度信号值的输出亮度值集合。 其中, 所述输入亮度信号 值的设定级别是 0-255级, 亮度值的取值为整数。
根据所述方法, 在所述视频码流中扩展伽玛参数信息域, 并将所述输入 亮度信号值集合和 /或输出亮度信号值集合组成二进制码流并携带在所述伽玛 参数信息域中进行传送。
其中, 所述伽玛参数信息域分别包括伽玛参数信息和位于该伽玛参数信 息两端的起始定界符和结束定界符, 该起始定界符和结束定界符用于确定该 信息域的范围。
当所述视频码流为采用 H.264协议编码时, 在 H.264码流的补充增强信 息 SEI域中扩展用于携带所述伽玛参数信息的消息。
本发明还提供一种多点控制单元, 包括多点处理器, 所述多点处理器包 括:
伽玛特性参数存储模块, 用于存储视频图像发送终端的伽玛特性参数; 伽玛特性校正模块, 连接所述伽玛特性参数存储模块, 用于根据视频图 像发送终端的伽玛特性参数校正所述视频图像中的伽玛特性。
所述多点处理器还包括:
视频码流收发模块, 用于收发视频码流, 所述视频码流中包含视频图像
数据和该视频码流发送终端的伽玛特性参数信息;
视频数据编解码模块, 连接在所述视频码流收发模块和伽玛特性校正模 块之间, 用于从所述视频码流中解码出视频数据并送入伽玛特性校正模块进 行校正; 或者, 根据校正后的视频图像编码视频数据;
伽玛特性参数信息提取及附加模块, 连接在所述视频码流收发模块和视 频数据编解码模块之间, 用于从收到的视频码流中提取发送终端的伽玛特性 参数信息并存入伽玛特性参数存储模块; 或者, 从伽玛特性参数存储模块提 取伽玛特性参数信息并附加到待发送的视频码流中;
多画面拼装模块, 连接所述伽玛特性校正模块, 用于将收到的至少两个 终端的视频图像拼装为一个多画面图像, 并将该多画面图像送入视频数据编 解码模块或伽玛特性参数信息提取及附加模块。
其中, 所述伽玛特性校正模块中包括:
等效伽玛特性参数计算子模块, 用于根据发送终端的单伽玛环节特性参 数计算等效伽玛特性参数并输入伽玛校正发生子模块;
伽玛校正发生子模块, 用于根据等效伽玛特性参数校正视频图像的伽玛 特性。
所述多点控制单元还包括: 多点控制器, 连接所述多点处理器, 用于向 多点处理器发送进行伽玛特性校正的控制信号。
根据本发明, 所述多点处理器并行设置为多个。
本发明的有益效果如下:
本发明所述方法通过在视频码流中携带发送终端的伽玛特性参数, 使接 收终端对收到的视频图像根据发送终端的伽玛特性参数和本端的伽玛特性参 数进行一次校正, 或者, 由多点控制单元根据发送终端的伽玛特性参数对视 频图像进行一次校正后, 再由接收终端根据本地伽玛特性参数进行再次校正, 可以在接收终端校正视频图像中的伽玛失真, 本发明所述方法适用于视频通 信中的各种场景, 从而在多媒体通信中实现了伽玛校正, 提高了通信的质量 和用户体验;
为实现本发明所述方法, 本发明还提供一种具有伽玛特性校正功能的多 点控制单元, 该多点控制单元可以根据控制指令对转发的视频图像进行校正 或再转发, 并兼容现有直接转发的功能。 附图说明
图 1为环节 Gamma特性的一般模型;
图 2为环节 Gamma特性弓 I起的亮度信号畸变的示意图;
图 3为多环节级联 Gamma特性的一般模型;
图 4为校正单个环节的 Gamma特性示意图;
图 5为校正多个给定环节的 Gamma特性示意图;
图 6 多画面布局的一种示例;
图 7为 MCU通过级联增大控制功能的示意图;
图 8为现有 MCU的结构示意图;
图 9为多个 Gamma环节级联时, 可能校正点的示意图;
图 10为多个 Gamma环节级联时, 通用 Gamma特性校正方法示意图; 图 11为本发明所述实施例一, 会场自由观看状态下的 Gamma校正方法 原理示意图;
图 12 为本发明所述实施例二, 会场指定观看 (即会场广播)状态下的 Gamma校正方法原理示意;
图 13为本发明所述实施例三, 多画面状态下的 Gamma校正时, 多点控 制单元对视频图像先解码再编码合成为多画面的原理示意图;
图 14为本发明实施例六所述具有视频图像伽玛特性校正功能的多点控制 单元的结构示意图;
图 15为本发明定义的 Gamma参数信息区域二进制格式示意图。 具体实施方式
作为本发明实施的基础, 这里首先介绍一种利用 Gamma路径校正多环节
Gamma特性的通用方法, 请参阅图 9所示, 假设视频信号从采集到输出经过 的 Gamma路径包括 Nt个级联的 Gamma环节, 可以选择任意两个环节之间进行 校正点, 校正环节将图像经过的 Gamma路径分为前后两段, 校正点之前包括 个环节, 校正点之后包括 Np个环节, 如图 10所示, 先分别确定 Na 环节和 Np个环节的等效综合 Gamma特性, 然后分别构造等效 Gamma特性的校正子环 节, 然后将两个校正子环节进行复合后插入校正点, 校正环节实际上是两个 校正子环节的复合, 具体包括如下步骤:
1、 确定与信号相关的 Gamma特性环节的级联路径, 以及该级联路径包 括的环节数目为 Nt;
2、在所述路径中确定一个校正点,该校正点将所述 Nt个环节划分为级联 在该校正点之前的 Na个环节和级联在该校正点之后的 Np个环节,其中: Na≥0、 Np≥0、 Na+Np=Nt;
3、分别根据每一个环节的 Gamma特性等效模型及其参数构造所述 1^个 环节的第一综合等效模型和所述 Np个环节的第二综合等效模型;
4、 确定所述第一综合等效模型的第一逆模型和第二综合等效模型的第二 逆模型;
5、 根据所述第一逆模型和所述第二逆模型构造校正环节模型 , 利用该校 正环节模型确定所述 Na个环节最后输出信号的校正信号并将该校正信号输入 所述 Np个环节。
根据本方法, 校正环节模型的构造方法包括下列之一:
直接计算法: 实时利用第一逆模型的函数关系式和第二逆模型的函数关 系式的复合函数关系式计算 Na个环节的最后输出信号的校正信号;
两步计算法: 实时利用第一逆模型的函数关系式计算 Na个环节的最后输 出信号的一次校正信号, 利用第二逆模型的函数关系式计算该一次校正信号 的二次校正信号, 将该二次校正信号作为所述校正信号;
查表法: 预先根据所述直接计算法或两步计算法, 计算出所述 Na个环节 的最后输出信号的取值区间中的多个采样值对应的校正值, 并将对应关系保
存在一个数据表中, 然后通过实时查询该数据表确定任意待校正值的校正值。 对于采用数据表形式的模型, 等效模型本身没有解析形式(比如采用查 表方法实现的, 当然其反函数也就没有解析形式了), 那么其逆模型就是该数 据表的逆表, 一个表存在两列, 很多行, 左列 (输入列)是输入信号的采样 值, 即待校正的信号值, 右列 (输出列)是对应的输出信号值, 即校正后的 信号值, 行数取决于采样点数, 行数越多越精确, 逆表就是把左右两列对调 得到的新数据表。 并且对于数据量大的视频数据, 进行实时计算的计算量很 大, 查表是最实际的方法。
利用上述方法的前提是终端必须能够确定出本终端上所有 Gamma特性 环节, 当等效模型采用函数表示形式时, 发送终端或接收终端通过下述方法 分别确定本端每一个 Gamma特性环节的等效模型及其参数,或者直接确定级 联 ¾多个 Gamma特性环节的综合等效模型及其参数。
本发明还在此介绍一种确定每一个 Gamma特性环节等效模型及其参数 的检测方法, 包括如下步骤:
首先, 选择一组单环节 Gamma特性的通用等效模型, 例如:
第一类 Gamma模型满足公式 6: z- = pLin a + (l - p) 0 < p≤l, a≥l (6) 其中: 公式 6所示函数的定义域 (即自变量取值范围)为区间 [0,1] , 值域 (函 数值的取值范围)为区间 [(l-p),l]。 第二类 Gamma模型满足公式 7: , =(^ , + d - ^) q≥U fi≥^ (7) 其中: 公式 7所示函数的定义域 (即自变量取值范围)为区间 [l-l/q,l] , 值 域 (函数值的取值范围)为区间 [(0,1] 。
然后将其中的一个作为待测模型进行下列步骤:
1、 在输入亮度信号 1^在[0,1]区间上选择间隔均匀的 N个采样点: Lin(0)、 Lin(l)、 Lin(2)…… Lin(i)…… Lin(N-2)、 Lin(N-l);
2、 将亮度信号 N个采样值分别输入环节中, 并测量实际输出亮度信号 N 个对应的值: Lp。ut(0)、 Lp。ut(l)、 Lp out(2)…… Lp。ut(i)…… Lp。ut(N-2)、 Lp out(N-l);
3、 构造拟合的目标函数, 目标函数和实际检测的输出亮度信号与通过 Gamma特性模型确定的理论输出亮度信号之间的差值相关, 而且, 差值越小, 说明模型的等效效果越接近实际情况。
目标函数的构造方法很多, 较为常用的是下述公式 8或公式 9:
N-l
En(/?,a) = X(Lp out( - L(.„() -(1-p))2 (8)或者,
;=0
W- 1 丄
FT2(g^) =∑( p oat(i)-(qLin(i) + (l-g) )2 (9)
4、 设定目标函数值的门限 Τ和最大迭代次数 Μ, 利用数学优化法寻找最 适合的参数组; 首先对于第一类的代价函数 Fn(A") = (Lp。ut()- ) -(l-p))2 , 采用某 种数学优化技术, 例如: 爬山法、 0.618法(华罗庚优选法) 、 最速下降法或 共轭梯度法等求取其最小值;
这个过程其实是一个迭代过程,在这个过程中不断调整参数 p和 α, 函数 值 F在不断下降, 当函数值下降到小于给定门限 Τ后, 则认为已经找到了最 小点。 此时对应的参数 ρ和 α, 就认为是本次应用环境模型的真正参数。 如果对于^ ^) -(l- )2经过 Μ次迭代, 还不能使得
函数下降到门限 Τ以下, 则认为模型选择不对。 应该选择第二类模型, 于是对
W-1 丄
于 =∑(Lp out( - (gLin(i) + (1 - q)Yf重复上述步骤 4, 得到对应的模型参数 q和 β, 应当注意的是, 参数 q和 β的取值范围分别是: q≥l、 β≥1。
如果想要得到更精确的参数, 可以在目标函数值 F下降到门限 Τ以下后, 仍然再迭代几次,如果目标函数值 F持续下降,或下降后又上升,或直接上升, 不管目标函数值 F是何种变化情况, 则选择其中的最小值对应的参数作为测量 结果会在一定程度上提高参数测量的精度。
可以看到, 模型类型的确定和参数的测量是同时进行的, 实际中, 等效
模型的类型不只这两种形式, 通过上述方法可以在相关的所有等效模型通过 测量参数的方法找到最合适的一个。
同样可以利用上述方法测量多环节综合 Gamma特性模型及其参数, 测量 方法步驟完全相同, 需要指出, 从形式上看, 多个 Gamma环节级联的综合 Gamma特性模型的函数关系式仍然可以采用公式 6和公式 7所示的两类, 但是 第一类综合特性模型中, 根据定性分析结果和实际测量的经验值, 指数 α取 值范围变成 α >0, 而第二类综合特性模型中, 指数 β取值范围变成 β >0。
因此, 确定了 Gamma特性模型及其参数后, 可以才艮据前述方法进行校正 模型的确定, 如图 4所示, 对于单个给定环节, 只要 Gg(.)满足一定条件, 就可 以找到 Gc(.)对 Gg(.)进行 Gamma校正;如图 5所示,对于多给定环节,只要 Ga(.)、 Gp(.)满足一定条件 , 就可以找到 Gc(.)对 Ga(.)和 Gp(.)进行 Gamma校正。
对于一个多媒体通信终端来说, 从外部光信号进入摄像机 /头, 经过各个 环节处理, 最终到本端的显示器显示转换成光信号; 或者经过压缩编码环节 在经过通信网络传送到达对方终端, 再经过去压缩解码恢复成图像, 再到多 方终端显示器上显示出来转换成光信号;或者经过光信号从摄像机 /头进入后, 转换成电信号, 然后经过一定的处理, 被写入到文件, 保存在硬盘等存储设 备上。 每一个视频信号经过的途径多可以看作如图 3所示的 Gamma路径, 可以 利用上述通用方法进行 Gamma特性的校正。
基于上述确定每一个 Gamma特性环节等效模型及其参数的检测方法,每 一个终端都可以检测到本地各 Gamma环节的等效模型及其参数, 由此, 本发 明提出以下技术构思:
1、 在视频码流编码压缩或者传送承载协议中提供一种 Gamma特性参数 信息传递和交换的机制, 具体方式可以是:
发送终端将视频图像经过的每一个伽玛特性环节的单环节伽玛特性参 数, 以及各伽玛特性环节的级联顺序携带在视频码流中发送; 或者发送终端 根据所述视频图像经过的所有伽玛特性环节计算出等效伽玛特性参数再携带 在枧频码流中发送。
发送终端可以为每一个视频图像的携带对应的伽玛特性参数信息; 或者, 发送终端将每隔一定周期 (周期可以用时间来度量, 也可以用每隔一定图像 帧数来度量)将伽玛特性参数信息携带在所述视频码流中发送给接收终端; 或者, 在通信开始时将初始伽玛特性参数信息携带在所述视频码流中发送给 接收终端, 并在通信过程中, 当本端伽玛特性参数发生变化时, 再将更新的 伽玛特性参数信息携带在所述视频码流中发送给接收终端。
2、 基于 1 , 在交互 Gamma特性参数的基础上, 利用上述通用校正方法 进行校正, 校正可以在多点控制单元或者具有类似功能的多点通信服务器中 进行, 校正也可以由视频数据接收终端执行, 因此同样适用于两点通信的情 况, 或者没有多点控制单元参加的多点通信情况。
并适用于多点控制单元的如下两种主要工作模式:
1 )、 多画面模式, 该多画面模式包括: 先解码再编码的合成模式和直接 合成模式(俗称软件多画面);
2 )、 视频转发模式。
并适用于多个多点控制单元级联以及多点控制单元的 MC和 MP分离的 情况。
首先为描述方便, 本发明定义如下概念:
1、 Gamma失真: Gamma是一种非线性失真( Distortion,或者叫做扭曲), 指信号经过一个或者多个 Gamma环节的相继作用, 形成 Gamma失真;
2、 无 Gamma失真: 信号没有任何 Gamma失真的状态;
3、 完全 Gamma校正状态: 存在 Gamma失真的信号经过 Gamma校正后 达到了和无失真状态完全相同的状态, 叫做完全 Gamma校正状态;
4、 Gamma轨道 (Gamma Track): 在视频码流中用于携带传送 Gamma信 息的数据部分。
需要说明的是, 以上定义建立在理想状态下, 实际上对视频图像的校正 只能看作近似的无 Gamma失真或完全 Gamm 校正状态。
下面以具体实施例进行详细说明:
实施例一、 会场自由观看状态下的 Gamma校正
这种情况下, MCU并不处理来自终端的视频码流, 只是进行转发。 假设 终端 A选择自由观看终端 B的视频, 则终端 B加入视频会议后向 MCU发送 本地视频码流, 该视频码流中携带终端 B的 Gamma特性参数, 假定终端 B 有 NB个 Gamma环节; MCU将来自终端 B的视频码流转发给终端 A;
终端 A接收并分析来自终端 B的视频码流, 从视频码流中提取出所携带 的 Gamma特性参数和数字视频数据, 并结合终端 A自身的 Gamma特性参数 进行校正, 具体校正方法为前述的通用校正方法, 校正原理如图 11所示, 校 正过程包括如下步骤:
1、 终端 A从终端 B的视频码流中通过去压缩解码获得数字视频数据;
2、 终端 A从终端 B的视频码流中提取出 NB个 Gamma环节的 Gamma 参数;
3、终端 A根据公式 3计算 NB个 Gamma环节级联形成等效 Gamma环节 GBEQ的 Gamma参数, 其中 EQ表示等效( Equivalent );
同时, 终端 A根据公式 3计算出本身多个 Gamma环节的等效环节 GAEQ 的 Gamma参数;
4、 终端 A根据 GBEQ和 GAEQ计算校正环节 Gc。r的 Gamma参数;
5、终端 A根据 Gcor的 Gamma参数校正来自终端 B的视频后通过本地显 示器显示;
具体校正可以采用基于函数表示的直接计算方法或基于查表表示的查表 方法, 详细情况参见前述的通用校正方法, 这里不再赘述。
上述实施例中, 如果终端 A和终端 B之间直接通信或者当前通信系统中 没有 MCU时, 终端 A和终端 B之间直接交互视频数据流。
实施例二、 会场指定观看 (即会场广播)状态下的 Gamma校正
和实施例一相同, MCU也仅仅是对于来自被指定观看终端的视频码流向 所有参见会议的终端进行广播, 而不进行其他任何处理。
如图 12所示, 假定被广播的会场终端为 X, 其它终端收到的都是 X的视
频, 其它任何一个终端 Yl、 Υ2...... Ym, 都要根据来自 X的视频码流携带的
Gamma参数信息, 提取 Gamma参数, 然后进行校正, 其校正过程和实施例 一相同。
与实施例一不同之处在于,每个观看 X终端视频的终端同步校正并收看, 就实现了会场指定观看 (即会场广播)状态下的 Gamma校正。
实施例三、 多画面状态, MCU 工作在先解码再编码的合成模式下的 Gamma校正方法一
如图 13所示, MCU首先要对来自终端 、 X2..... Xc的 C (多画面布局 中子画面的总数) 个子画面视频进行去压缩解码, 还原成未压缩 ( Uncompressed )数字视频格式, 然后拼成多画面图像后, 再编码压缩。 在 这个过程中 , MCU可以对于已经还原出来的未压缩数字视频格式进行一些处 理和 Gamma特性校正。 具体 #文法是: '
1、 MCU对于来自终端 Xi(l≤i≤C)的视频进行去压缩解码, 形成未压缩数 字视频格式;
2、 MCU从来自 的视频码流中提取所携带的 Gamma参数信息;
3、 MCU根据所述提取出的 Gamma参数, 按照级联的法则形成等效环节 G( XEQ,计算出相应的 Gamma特性参数;
4、 MCU根据 G(i)XEQ和相应的 Gamma特性参数来进行校正,校正的结果, 形成了完全校正状态。 即经过校正的视频是出于无 Gamma失真状态的;
MCU重复以上步骤 1 ~ 4, 直到把来自 C个终端的视频都校正完毕;
5、 MCU按照指定的多画面布局由 C个子画面组合拼出多画面图像;
6、 MCU对于所述组合拼出的多画面图像进行压缩编码;
7、 MCU为新生成的多画面视频码流填写携带 Gamma信息的 Gamma轨 道信息。
因为这个时候多画面的每个子画面都是无 Gamma失真的,所以整个多画 面也是无 Gamma失真的。 因此 Gamma轨道实际上不含有任何 Gamma参数。 只是需要在 Gamma轨道中设置一个校正标志, 表明视频是否为无 Gamma失
真。 校正标志的设置方法很多, 例如: 设置该校正标志表明视频码流是存在
Gamma失真的, 不设置时表明无 Gamma失真; 或者校正标志设置为 1表明 存在 Gamma失真,设置为 0表明无 Gamma失真。显然该标志应该位于 Gamma
Track的最前面, 这样当 MCU或者终端在读到标志后, 首先判别校正标志, 如果是无 Gamma失真, 意味着后面没有 Gamma参数, 不用向后继续读数据 了, 反之则要继续读数据。
8、 MCU把新生成的多画面视频码流发送给多个终端。
在多个接收到多画面视频码流的终端中, 每一个接收终端的处理过程相 同, 仍如图 13所示, 具体 #丈法如下:
1、终端 Y计算出本身多个 Gamma环节的等效环节 GYEQ的 Gamma参数; 终端 Y首先根据校正标志判断接收到的多画面视频是否存在 Gamma失 真 , 本实施例中终端 Y收到的多画面视频为无 Gamma失真, 因此终端 Y只 需要在校正的时候考虑自身的 Gamma特性即可。
2、 终端 Y根据 GYEQ计算校正环节 Gc。r的 Gamma参数;
其中 GYEQ为终端 Y的等效 Gamma特性。
3、 终端 Y对于来自 MCU的多画面视频进行校正。
具体校正可以采用基于函数表示的直接计算方法或基于查表表示的查表 方法, 详细情况参见如前所述的伽玛特性的校正通用方法。
实施例四、 多画面状态, MCU 工作在先解码再编码的合成模式下的 Gamma校正方法二
MCU在把来自各个终端 XI、 X2..... Xc的子画面视频去压缩解码还原到未 压缩格式后, 并不进行 Gamma校正, 直接组合拼出多画面图像, 再进行编码 压缩, 形成多画面视频码流。 然后在视频码流的 Gamma轨道上设置表明 Gamma失真的校正标志后, 按照所指定多画面布局给定的各个子画面的排列 顺序, 依次将原来各个终端视频码流 Gamma轨道携带的 Gamma参数信息复 制到与子画面对应的位置, 形成多画面视频码流的 Gamma轨道。
参见图 11所示的终端处理示意图, 终端 Y接收到多画面视频后, 处理方
法如下:
1、终端 Y将多画面视频码流去压縮解码还原为未压缩格式, 从而获得各 个子画面的未压缩格式;
2、 终端 Υ从多画面视频码流的 Gamma轨道中提取每个子画面对应的 Gamma参数信息;
3、 假设取出的子画面 i(l≤i≤C)Gamma参数, 其中有 个 Gamma环节, 这些 Gamma环节级联形成一个等效 Gamma环节 G(i) XEQ, 终端 Y根据公式 2 计算出等效环节 G(i) XEQ的 Gamma参数;
同时终端 Y计算出本身多个 Gamma环节的等效环节 GYEQ的 Gamma参 数。
4、 终端 Y根据 G(i) XEQ和 GYEQ计算校正环节 GCOT的 Gamma参数;
5、 终端 Y根据 Gcor的 Gamma参数对子画面 i的视频进行校正后显示。 终端 Y重复以上步骤 3〜5 , 直到所有的子画面都校正完毕。
具体校正方法可以基于函数表示的直接计算方法或基于查表表示的查表 方法, 详细情况参见如前所述的伽玛特性的校正通用方法。
实施例五、 多画面状态, MCU工作在直接合成模式下的 Gamma校正 在直接合成模式下, MCU对于来自终端 Xi(l≤i≤C)的视频码流不进行处 理, 只是按照符合具体视频压缩编码协议语法的方式进行码流合成, 形成一 个复合码流。 复合码流同样有一个 Gamma轨道, 对于该 Gamma轨道, 首先 MCU要设置有校正标志, 校正标志表明该视频码流存在 Gamma失真。
终端 Y接收到多画面视频后, 处理方法如下:
1、 终端 Y从多画面视频码流的 Gamma轨道中提取每个子画面对应的 Gamma参数信息;
2、 假设取出的子画面 i(l≤i≤C)Gamma参数, 其中有 个 Gamma环节, 这些 Gamma环节级联形成一个等效 Gamma环节 G(i) XEQ, 终端 Y根据公式 2 计算出等效环节 G(i) XEQ的 Gamma参数;
同时终端 Y计算出本身多个 Gamma环节的等效环节 GYEQ的 Gamma参
数;
3、 终端 Y从复合码流中定位并提取出子画面 i对应的视频码流然后进行 去压缩解码还原到未压缩格式;
4、 终端 Y根据 G(i) XEQ和 GYEQ计算校正环节 Gc。r的 Gamma参数;
5、终端 Y根据校正环节 Gc;。 ^的 Gamma参数对子画面 i的视频进行校正。 终端 Y重复以上步骤 1-5, 直到所有的子画面都校正完毕。
具体校正方法可以基于函数表示的直接计算方法或基于查表表示的查表 方法, 详细情况参见如前所述的伽玛特性的校正通用方法。
实施例六、 具有 Gamma校正功能的 MCU
为实现在 MCU侧进行视频图像伽玛特性的一次校正,本发明还提供一种 具有 Gamma校正功能的 MCU, 包括多点控制器 100和多点处理器 200。 与 现有技术的 MCU结构相比, 本发明所述的具有 Gamma校正功能的 MCU主 要在多点处理器 200上进行了改进, 如图 14所示, 多点处理器 200包括如下 功能模块:
伽玛特性参数存储模块 201 , 用于存储视频图像发送终端的伽玛特性参 数;
伽玛特性校正模块 202, 连接所述伽玛特性参数存储模块 201 , 用于根据 视频图像发送终端的伽玛特性参数校正所述视频图像中的伽玛特性。
视频码流收发模块 203, 用于收发视频码流, 所述视频码流中包含视频图 像数据和该视频码流发送终端的伽玛特性参数信息;
视频数据编解码模块 204,连接在所述视频码流收发模块 203和伽玛特性 校正模块 202之间, 用于从所述视频码流中解码出视频数据并送入伽玛特性 校正模块 202进行校正; 或者, 根据校正后的视频图像编码视频数据。
伽玛特性参数信息提取 /附加模块 205,连接在所述视频码流收发模块 203 和视频数据编解码模块 204之间, 用于从收到的视频码流中提取发送终端的 伽玛特性参数信息并存入伽玛特性参数存储模块 201; 或者,从伽玛特性参数 存储模块 201提取伽玛特性参数信息并附加到待发送的视频码流中。
多画面拼装模块 206, 连接所述伽玛特性校正模块 202, 用于将收到的至 少两个终端的视频图像拼装为一个多画面图像, 并将该多画面图像送入视频 数据编解码模块 204。
如果不需要进行多画面合成, 则校正模块 202将校正后的视频图像直接 发送给视频数据编解码模块 204。
同时, 多画面拼装模块 206伽玛特性参数信息提取 /附加模块 205之间直 接连接, 当采用直接合成模式时, 不需要对视频码流进行编解码的处理。
多画面拼装的各种模式预先设定在多画面拼装模块 206和伽玛特性参数 信息提取 /附加模块 205中, 用于根据对应的拼装模式进行多画面拼装并确定 在视频码流中附加各子画面伽玛特性参数信息的顺序。
其中: 伽玛特性校正模块 202中包括:
等效伽玛特性参数计算子模块 2021 , 用于根据发送终端的单伽玛环节特 性参数计算等效伽玛特性参数并输入伽玛校正发生子模块 2022;
伽玛校正发生子模块 2022, 用于根据等效伽玛特性参数校正视频图像的 伽玛特性。
多点处理器 200连接多点控制器 100,根据多点控制器 100的指令对接收 的视频图像进行伽玛特性校正。
和现有技术相同, 在一个多点控制单元中, 可以并行设置多个多点处理 器 200, 可以控制更多的多媒体通信终端之间的通信。
如果不需要进行校正, 则视频收发模块 203根据多点控制器的指令直接 转发视频码流。
下面详细说明如何在终端之间、终端与多点控制单元之间交互 Gamma参 数信息的方法, 因为查表法的优势比较明罩, 所以本发明采用查表表示方法 交互 Gamma参数。
首先, 提供一种和具体协议无关的 Gamma参数信息表示的二进制格式, 因为 Gamma特性作为一个函数, 其定义域(Lin的取值范围)和值域(L。ut的 取值范围)都是区间 [0,1]或者其某个子区间, 比如 [0.1,0.9]。
前面已经说过,查表法的关键是两个集合(或者叫做序列){Lin( i )|0<i<N-l} 和 {L。ut ( i ) 10≤i≤N-l}, 以及它们的元素之间的对应关系。 我们定义 {Lin ( i ) |0≤i≤N-l}为输入亮度离散值集合(Set ofDiscrete Values of Input Luminance ); {Lout ( i ) |0≤i≤N-l}为输出亮度离散值集合(Set ofDiscrete Values of Output Luminance ) 。 虽然从 Gamma特性作为一个函数的定义域和值域的要求来说, Lin、 L。ut取值必须在 [0,1]区间上。 但是, 在实际应用中., 每个 Gamma环节的输 入和输出亮度的取制范围可能并不是 [0,1]区间。 这样为了保持和 Gamma定义 的一致, 实际上对于基于函数形式的 Gamma特性, 在处理的时候要首先把输 入亮度值进行规一化处理。 将取值在 [0,1]区间上的亮度信号叫做规一化亮度 Ln in,上标 n表示 Normalized"规一化 ", 而取值在 0- MaxLa in之间的亮度信号叫做 实际亮度 Lain, 上标 示 Actual"实际"。 而把实际的亮度映射到 [0,1]区间, 采 用的方法如公式 10所示:
Ln in=Lin a/MaxLa in ( 10 )
相应地, 输出的亮度信号要从规一化的值还原到实际的值(逆规一化) 。 计 算如公式 11所示:
目前摄像机 /摄像头, 显示器和中间的数字视频交换格式比如 CIF
( Common Interchange Format )采用的都是 256级亮度, 即实际的输入输出亮 度取值都是 0-255的整数, 即
MaxLa in= MaxLa 0Ut=255 , 一般情况下 MaxLa ir MaxLa。ut。 考虑到以后技术 的进步的可能, 摄像机、 显示器的发展和人类感官对于更高亮度等级分辨率 的需要, 亮度等级可能会增加, 从技术实现角度来说, 一般会增加为 2的整数 次方, 比如 512甚至 1024, —般形式是 2D, D为自然数。 那么如公式 12、 公式 13所示:
Ln in=Lin72D ( 12 )
L。ut=L。ut a/2 ( 13 )
但是在构造 {Lin ( i ) |0≤i≤N-l}和 {L。ut ( i ) |0≤i≤N-l}的时候, 必须考虑实
际的应用场景, 在当前的视频通信技术中, 亮度信号的等级为 256级, 亮度值 为 0-255, 每一个亮度值可以用 8比特表示(一个字节) 。
于是, 集合 {Lin ( i ) |0≤i≤N-l}={0, 1, 2, 3, 4......254, 255},而集合 {L。ut
( i ) |0≤i≤N-l}中的每个数值也都属于 {0, 1 , 2, 3, 4…… 254, 255}。
在以上假设下, 适用于目前视频通信技术的 Gamma特性查表表示数据结 构仍如表 1所示, 进一步分析发现, 表 1的左列 (Left Column ) 的数值都是固 定的, 必然是 {0, 1 , 2, 3, 4......254, 255}的顺序, 通信双方都知道这个集 合和顺序, 则在通信中不需要传送左列数值。
在通信协议中传送 Gamma参数信息, 不论是什么协议, 一般方法就是在 通信协议允许扩展和自定义内容的数据区内定义一个块( Block或者 Region ) , 用于连续存放 Gamma参数的二进制码流表示。 然后该块皮封装在协议的码流 中传送。 这个区域叫做 Gamma参数信息区域。
多媒体通信终端中可能有包括摄像机 /摄像头和显示设备在内的多个 Gamma环节。 因此, 对于一个终端要把自己的全部 Gamma参数信息传送给其 它通信终端或者网络上其它设备比如多点控制单元等, 那么就应该把自身的 全部 Gamma环节, 按照它们从前到后 (或反方向) 的级联顺序, 将其 Gamma 特性参数写入到 Gamma参数信息区域。 而接收的终端或者其它网络设备就可 以从用于承载 Gamma参数的信息区域中按顺序提取出 Gamma参数信息。 因此 定义 Gamma参数信息区域的具体格式如图 15所示, 包括以下部分:
开始标志: 16比特(2字节) , 取值为 OxOFFO;
结束标志: 16比特 ( 2字节) , 取值为 OxFOOF;
区域总长度: 16比特(2字节),以字节为单位的 Gamma参数信息区域的 总长度 (包含开始标志和结束标志在内) 。
以上三者结合起来, 可以定位出在码流中 Gamma信息区域的位置。
Gamma环节总数: 8比特(1字节) 以 T表示。 最多可以有 256个 Gamma环 节, 在实际应用中是足够的。
环节 1〜T参数子区域: 这样的子区域共有 Τ个, 分别对应于 Τ个 Gamma环
节。
每个子区域的结构是这样定义的:
子区域长度: 16比特(2个字节) , 以字节为单位的子区域长度(包含传 送模式字节在内) 。
连续参数: Lout ( 0 ) , Lout ( 1 ) ...... Lout ( Nr3 ) , 其中: i=l , 2...... T。 需要说明的是, 上述定义方式仅是本发明的一种实施例, 并不限定本发 明的保护范围。
上面定义了不依赖具体承载协议的 Gamma参数信息区域的格式, 如果视 频码流采用 H.264协议编码方式, 则本发明还结合 H.264协议给出利用 H.264消 息扩展机制承载 Gamma参数信息的方法。
H.264中提供了多种可以进行消息扩展的机制,其中比较适合本发明使用 的是 H.264中定义的补充增强信息 SEI( Supplement Enhancement Information ), SEI 的数据表示区域与视频编码数据独立 , 使用方法在 H.264协议中 NAL ( Network Abstraction Layer 网络抽象层) 的描述中给出。 H.264码流的基本 单位是 NALU ( NAL Unit, 即网络抽象层单元), NALU可以承载各种 H.264 数据类型, 比如视频序列参数( Sequence parameters ), 图像参数 (Picture parameters ), Slice数据 (即具体图像数据), 以及 SEI消息数据。 SEI用于传 递各种消息, 支持消息扩展。 因此 SEI域内用于传送为特定目的而自定义的 消息,而不会影响基于 H.264视频通信系统的兼容性。承载 SEI消息的 NALU 叫做 SEI NALU。一个 SEI NALU含有一个或多个 SEI消息。每个 SEI消息含 有一些变量, 主要是 Payload Type和 Payload Size, 这些变量指明了消息载 荷的类型和大小。 在 H.264 Annex D.8, D.9中定义了一些常用的 H.264 SEI 消息的文法和语意。
NALU中包含的载荷叫做 RBSP ( Raw-Byte Sequence Payload ), SEI是 RBSP的一种类型。 按照 H.264 7.3的定义, SEI RBSP的文法如表 2所示:
表 2. SEI RBSP文法
表 3.SEI消息的结构
sei— message ( ) { C Descriptor
payloadType = 0
while ( next— bits ( 8 ) = = OxFF ) {
ff— byte /* equal to OxFF */ 5 f ( 8 )
payloadType += 255
}
Iastjpayload_type_byte 5 u ( 8 )
payloadType += last__payload— type— byte
payloadSize = 0
while ( next— bits ( 8 ) = = OxFF ) {
ff_byte /* equal to OxFF */ 5 f ( 8 )
payloadSize += 255
}
last_payload_size_byte 5 u ( 8 )
payloadSize += last__payload— size— byte
sei_payload ( payloadType, payloadSize ) 5
}
H.264 Annex D.8定义了保留用于今后扩展的 SEI消息的文法消息结构如 表 4所示:
表 4.可扩展的 SEI消息结构
在本发明的描述中将 SEI的数据表示区域筒称为 SEI域, 每个 SEI域 包含一个或多个 SEI消息, 而 SEI消息又由 SEI头信息和 SEI有效载荷组成。 SEI头信息包括两个码字: 一个码字给出 SEI消息中载荷的类型, 另一个码字 给出载荷的大小。 当载荷类型在 0到 255之间时用一个字节 0x00到 OxFE表 示, 当类型在 256到 511之间时用两个字节 OxFFOO到 OxFFFF表示, 当类型 大于 511 时表示方法以此类推, 这样用户可以自定义任意多种载荷类型。 其 中类型 0到类型 18标准中已定义为特定的信息如緩存周期、 图像定时等。 对 于本发明所述技术方案,要承载 Gamma参数信息区域类型可以定义为任何现 有没有被定义的 SEI载荷类型, 因为目前很可能有很多其他目的的扩展消息 类型, 为避免冲突, 可以定义为 OxFFFF ( 511 ), OxFFFF是理论上的最大值。 然后将根据定义填写的 Gamma参数信息区域直接放入到 SEI载荷中去,就实 现了利用 SEI消息扩展机制承载和传送 Gamma参数信息的目的。
应该说明, 将 SEI载荷类型取值为 OxFFFF, 只是本发明的一个实施例, 对于其他的取值, 也在本发明保护范围内。
综上所述, 本发明提供两种多媒体通信中的伽玛特性校正方法: 其一是在视频图像接收终端侧进行, 由视频图像接收终端根据本端的伽 玛特性参数和发送终端的伽玛特性参数进行一次校正;
其二是在多点控制单元和视频接收终端分步进行校正, 先由多点控制单 元根据发送终端的伽玛特性参数校正发送终端的伽玛环节引入的伽玛失真, 然后由视频接收终端根据本端的伽玛特性参数校正接收终端的伽玛环节引入 的伽玛失真。
上述方法可以根据多媒体通信的具体场景选择使用, 可以很好的校正视 频图像从采集、 发送、 接收到显示的各个环节中引入的伽玛失真。
显然, 本领域的技术人员可以对本发明进行各种改动和变型而不脱离本 发明的精神和范围。 这样, 倘若本发明的这些修改和变型属于本发明权利要 求及其等同技术的范围之内, 则本发明也意图包含这些改动和变型在内。
Claims
1、 一种视频通信中视频码流伽玛特性校正方法, 其特征在于, 包括如下 步骤:
发送终端向接收终端发送视频码流, 所述视频码流中包括根据发送终端 视频图像生成的视频数据和发送终端的伽玛特性参数信息;
接收终端接收所述视频码流, 根据所述视频数据还原所述视频图像, 并 根据本端伽玛特性参数信息和发送终端伽玛特性参数信息对该视频图像进行 伽玛校正。
2、 如权利要求 1所述的方法, 其特征在于, 所述方法中, 至少一个发送 终端将所述视频码流发送给多点控制单元; 多点控制单元将所述视频码流发 送给接收终端。
3、 如权利要求 2所述的方法, 其特征在于, 多点控制单元将来自至少两 个发送终端的视频码流合成为一个多画面视频码流后发送给接收终端。
4、 如权利要求 3所述的方法, 其特征在于,
多点控制单元分别根据每一个发送终端的视频数据还原出对应的视频图 像, 将所述各个视频图像分别作为子画面并拼装成一个多画面图像, 然后生 成所述多画面图像的视频数据, 并将该多画面图像的视频数据和每一个子画 面对应的伽玛特性参数信息合成为一个多画面视频码流后发送给接收终端, 其中, 所述子画面的伽玛特性参数信息顺序根据子画面的拼装位置和顺序确 定; 以及
接收终端根据所述多画面图像的视频数据还原出所述多画面图像, 并根 据本端伽玛特性参数信息和每一个子画面对应的伽玛特性参数分别校正每一 个子画面的视频图像。
5、 如权利要求 3所述的方法, 其特征在于,
多点控制单元分别提取每一个发送终端视频码流中的视频数据并根据子 画面拼装顺序直接进行复合, 然后将复合后的视频数据和每一个子画面对应
的伽玛特性参数信息合成为一个多画面视频码流后发送给接收终端, 其中, 所述子画面的伽玛特性参数信息顺序根据子画面的拼装位置和顺序确定; 以 及
接收终端从复合后的视频数据中提取各子画面对应的视频数据并还原出 各子画面的视频图像, 然后根据本端伽玛特性参数信息和每一个子画面对应 的伽玛特性参数分别校正每一个子画面的视频图像, 再将校正后的各个子画 面根据拼装顺序拼装为多画面图像。
6、 如权利要求 1 ~ 5任意之一所述的方法, 其特征在于, 在所述视频码 流中还设置有标识视频图像存在伽玛失真的指示信息; 以及
接收终端根据所述指示信息确认所述多画面视频码流中携带有所述伽玛 特性参数信息。
7、 如权利要求 1 ~ 5任意之一所述的方法, 其特征在于, 所述的发送终 端伽玛特性参数信息为所述视频图像在其采集、 处理和形成视频码流的过程 中:
在发送终端经过每一个伽玛特性环节的伽玛特性参数, 以及各伽玛特性 环节的级联顺序; 或者
8、 如权利要求 7所述的方法, 其特征在于, 所述方法中:
发送终端将本终端每一帧视频图像的伽玛特性参数信息分别对应该视频 图像的视频数据携带在所述视频码流中发送给接收终端; 或者
发送终端将每隔一定周期将伽玛特性参数信息携带在所述视频码流中发 送给接收终端, 或者
发送终端在通信开始时将初始伽玛特性参数信息携带在所述视频码流中 发送给接收终端, 并在通信过程中, 当本端伽玛特性参数发生变化时, 再将 更新的伽玛特性参数信息携带在所述视频码流中发送给接收终端。
9、 如权利要求 7所述的方法, 其特征在于, 所述每一个伽玛环节的伽玛 特性参数或等效伽玛特性参数包括对应设定等级的每一级输入亮度信号值的
输出亮度值集合。
10、 如权利要求 9所述的方法, 其特征在于, 所述输入亮度信号值的设 定级别是 0-255级, 亮度值的取值为整数。
11、 如权利要求 9或 10所述的方法, 其特征在于, 在所述视频码流中扩 展伽玛参数信息域, 并将所述输入亮度信号值集合和 /或输出亮度信号值集合 组成二进制码流并携带在所述伽玛参数信息域中进行传送。
12、 如权利要求 11所述的方法, 其特征在于, 所述伽玛参数信息域分别 包括伽玛参数信息和位于该伽玛参数信息两端的起始定界符和结束定界符, 该起始定界符和结束定界符用于确定该信息域的范围。
13、 如权利要求 8、 9或 10所述的方法, 其特征在于, 当所述视频码流 为采用 H.264协议编码时, 在 H.264码流的补充增强信息 SEI域中扩展用于 携带所述伽玛参数信息的消息。
14、 一种视频通信中视频码流伽玛校正方法, 其特征在于, 包括如下步 骤:
视频码流发送终端向多点控制单元发送第一视频码流, 所述第一视频码 流中包括根据发送终端视频图像生成的视频数据和发送终端的伽玛特性参数 信息;
多点控制单元接收所述第一视频码流, 根据所述视频数据还原所述视频 图像, 并根据发送终端的伽玛特性参数信息对于所述视频图像进行一次伽玛 校正; 然后
生成经过一次校正的视频图像的视频数据并携带在第二视频码流中发送 给接收终端。
15、 如权利要求 14所述的方法, 其特征在于, 所述方法还包括: 接收终 端接收所述第二视频码流, 还原校正后视频图像并根据本端伽玛特性参数再 次进行校正。
16、 如权利要求 15所述的方法, 其特征在于,
多点控制单元将来自至少两个发送终端的第一视频码流并分别校正视频
图像并拼装为多画面图像; 以及
根据所述多画面图像生成多画面图像视频数据并携带在所述第二视频码 流中发送给接收终端。
17、 如权利要求 16所述的方法, 其特征在于, 接收终端根据所述多画面 图像的视频数据还原出所述多画面图像, 并分别根据本端伽玛特性参数再次 校正每一个子画面的视频图像。
18、 如权利要求 14 ~ 17任意之一所述的方法, 其特征在于,
所述第一视频码流中设置有标识视频图像存在伽玛失真的第一指示信 息; 和 /或
所述第二视频码流中设置有标识视频图像已经经过一次伽玛校正的第二 指示信息。
19、 如权利要求 18所述的方法, 其特征在于, 多点控制单元根据所述第 一指示信息确认所述第一视频码流中携带有发送终端的伽玛特性参数信息。
20、 如权利要求 14、 15或 16所述的方法, 其特征在于, 所述的发送终 端伽玛特性参数信息为所述视频图像在其采集、 处理和形成视频码流的过程 中:
在发送终端经过每一个伽玛特性环节的伽玛特性参数, 以及各伽玛特性 环节的级联顺序; 或者
发送终端根据经过的所有伽玛特性环节确定的等效伽玛特性参数。
21、 如权利要求 20所述的方法, 其特征在于, 所述方法中:
发送终端将每一个视频图像的伽玛特性参数信息分别对应该视频图像的 视频数据携带在所述视频码流中发送给接收终端; 或者
发送终端将每隔一定周期将伽玛特性参数信息携带在所述视频码流中发 送给接收终端, 或者
发送终端在通信开始时将初始伽玛特性参数信息携带在所述视频码流中 发送给接收终端, 并在通信过程中, 当本端伽玛特性参数发生变化时, 再将 更新的伽玛特性参数信息携带在所述视频码流中发送给接收终端。
22、 如权利要求 20所述的方法, 其特征在于, 所述每一个伽玛环节的伽 玛特性参数或等效伽玛特性参数包括对应设定等级的每一级输入亮度信号值 的输出亮度值集合。
23、 如权利要求 22所述的方法, 其特征在于, 所述输入亮度信号值的设 定级别是 0-255级, 亮度值的取值为整数。
24、 如权利要求 22或 23所述的方法, 其特征在于, 在所述视频码流中 扩展伽玛参数信息域, 并将所述输入亮度信号值集合和 /或输出亮度信号值集 合组成二进制码流并携带在所述伽玛参数信息域中进行传送。
25、 如权利要求 24所述的方法, 其特征在于, 所述伽玛参数信息域分别 包括伽玛参数信息和位于该伽玛参数信息两端的起始定界符和结束定界符, 该起始定界符和结束定界符用于确定该伽玛参数信息域的范围。
26、 如权利要求 14、 15或 16所述的方法, 其特征在于, 当所述视频码 流为采用 H.264协议编码时, 在 H.264视频码流的补充增强信息 SEI域中扩 展用于携带所述伽玛参数信息的消息。
27、 一种多点控制单元, 包括多点处理器, 其特征在于, 所述多点处理 器包括:
伽玛特性参数存储模块 , 用于存储视频图像发送终端的伽玛特性参数; 伽玛特性校正模块, 连接所述珈玛特性参数存储模块, 用于根据视频图 像发送终端的伽玛特性参数校正所述视频图像中的伽玛特性。
28、 如权利要求 27所述的多点控制单元, 其特征在于, 所述多点处理器 还包括:
视频码流收发模块, 用于收发视频码流, 所述视频码流中包含视频图像 数据和该视频码流发送终端的伽玛特性参数信息;
视频数据编解码模块, 连接在所述视频码流收发模块和伽玛特性校正模 块之间, 用于从所述视频码流中解码出视频数据并送入 玛特性校正模块进 行校正; 或者, 根据校正后的视频图像编码视频数据。
29、 如权利要求 28所述的多点控制单元, 其特征在于, 所述多点处理器
还包括: 伽玛特性参数信息提取及附加模块, 连接在所述视频码流收发模块 和视频数据编解码模块之间, 用于从收到的视频码流中提取发送终端的伽玛 特性参数信息并存入伽玛特性参数存储模块; 或者, 从伽玛特性参数存储模 块提取伽玛特性参数信息并附加到待发送的视频码流中。
30、 如权利要求 27所述的多点控制单元, 其特征在于, 所述多点处理器 还包括: 多画面拼装模块, 连接所述伽玛特性校正模块, 用于将收到的至少 两个终端的视频图像拼装为一个多画面图像, 并将该多画面图像送入视频数 据编解码模块或伽玛特性参数信息提取及附加模块。
31、 如权利要求 27所述的多点控制单元, 其特征在于, 所述伽玛特性校 正模块中包括:
等效伽玛特性参数计算子模块, 用于根据发送终端的单伽玛环节特性参 数计算等效伽玛特性参数并输入伽玛校正发生子模块;
伽玛校正发生子模块, 用于根据等效伽玛特性参数校正视频图像的伽玛 特性。
32、 如权利要求 27所述的多点控制单元, 其特征在于, 所述多点控制单 元还包括: 多点控制器, 连接所述多点处理器, 用于向多点处理器发送进行 伽玛特性校正的控制信号。
33、 如权利要求 28 ~ 32任意之一所述的多点控制单元, 其特征在于, 所 述多点处理器并行设置为多个。
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP06753139.2A EP1959691B1 (en) | 2005-11-28 | 2006-07-14 | A video code stream gamma characristic correction method and a multipoint control unit |
US12/128,287 US8111333B2 (en) | 2005-11-28 | 2008-05-28 | Video bit-stream gamma-correction method and a multipoint control unit |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN200510124080.2 | 2005-11-28 | ||
CN200510124080A CN1852414B (zh) | 2005-11-28 | 2005-11-28 | 一种视频码流伽玛特性校正方法及多点控制单元 |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/128,287 Continuation US8111333B2 (en) | 2005-11-28 | 2008-05-28 | Video bit-stream gamma-correction method and a multipoint control unit |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2007059670A1 true WO2007059670A1 (fr) | 2007-05-31 |
Family
ID=37133880
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2006/001671 WO2007059670A1 (fr) | 2005-11-28 | 2006-07-14 | Procede de correction d'une caracteristique gamma dans un flux de codes video et unite de commande multipoint |
Country Status (4)
Country | Link |
---|---|
US (1) | US8111333B2 (zh) |
EP (1) | EP1959691B1 (zh) |
CN (1) | CN1852414B (zh) |
WO (1) | WO2007059670A1 (zh) |
Families Citing this family (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1852414B (zh) | 2005-11-28 | 2010-05-05 | 华为技术有限公司 | 一种视频码流伽玛特性校正方法及多点控制单元 |
CN101047869B (zh) * | 2006-06-15 | 2011-04-27 | 华为技术有限公司 | 一种视频通信伽玛特性的校正方法和装置 |
CN101742221B (zh) * | 2009-11-09 | 2012-06-13 | 中兴通讯股份有限公司 | 一种会议电视系统中的多画面合成方法及装置 |
CN101778246B (zh) * | 2010-01-29 | 2014-04-02 | 华为终端有限公司 | 多画面视频图像处理方法和装置 |
US20110312290A1 (en) * | 2011-07-21 | 2011-12-22 | Comtech Ef Data Corp. | Method and System for Closed Loop Pre-Distortion for PSK/QAM Modulation Using Feedback from Distant End of a Link |
CN103096012B (zh) * | 2011-11-08 | 2016-08-03 | 华为技术有限公司 | 调整图像显示的方法、设备及系统 |
US9179095B2 (en) * | 2012-09-27 | 2015-11-03 | Avaya Inc. | Scalable multi-videoconferencing system |
RU2657473C2 (ru) * | 2013-04-30 | 2018-06-14 | Сони Корпорейшн | Устройство передачи, способ передачи, устройство приема и способ приема |
CN105096799A (zh) * | 2015-08-07 | 2015-11-25 | 深圳市康冠商用科技有限公司 | 将多路画面中的各路画面进行独立调节的显示方法及系统 |
KR102486398B1 (ko) * | 2015-10-14 | 2023-01-10 | 삼성디스플레이 주식회사 | 영상 신호 처리 회로 및 이를 포함하는 표시 장치 |
CN109557093A (zh) * | 2018-12-18 | 2019-04-02 | 无锡益诺华科技发展有限公司 | 一种尿液检测试纸颜色测量算法 |
US10855998B1 (en) | 2019-07-01 | 2020-12-01 | Sony Corporation | Embedded codec circuitry for block based gamma correction of a raw image block |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH06165167A (ja) | 1992-11-17 | 1994-06-10 | Canon Inc | 画像伝送システム、画像送信装置及び画像受信装置 |
JP2000067226A (ja) | 1998-08-26 | 2000-03-03 | Toshiba Corp | 画像処理装置及び医用画像観察ネットワークに用いられるワークステーション |
CN1161972C (zh) | 1998-12-21 | 2004-08-11 | 皇家菲利浦电子有限公司 | 近似线性的伽玛数字校正 |
CN1182709C (zh) | 2000-11-24 | 2004-12-29 | 索尼公司 | 数字信号处理电路、采用数字信号处理电路的显示装置和液晶投影仪 |
CN1192619C (zh) * | 2002-01-08 | 2005-03-09 | 华为技术有限公司 | 会议电视系统中多画面显示的实现方法 |
EP1906673A1 (en) | 2005-07-07 | 2008-04-02 | Huawei Technologies Co., Ltd. | Method, system and terminal for negotiating to correct the gamma characteristic |
Family Cites Families (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH01280977A (ja) * | 1988-03-31 | 1989-11-13 | Toshiba Corp | テレビジョンシステムの子画面表示方法及びその装置 |
US5473366A (en) * | 1992-11-17 | 1995-12-05 | Canon Kabushiki Kaisha | Television-telephone apparatus having a message-keeping function and an automatic response transmission function |
EP1052847B1 (en) * | 1993-07-16 | 2004-09-15 | Sony Corporation | Image editing system |
JP3325763B2 (ja) | 1996-02-20 | 2002-09-17 | シャープ株式会社 | 画像伝送装置 |
JP2001197458A (ja) | 2000-11-27 | 2001-07-19 | Canon Inc | 画像伝送システム及び画像送信装置 |
CN1852414B (zh) | 2005-11-28 | 2010-05-05 | 华为技术有限公司 | 一种视频码流伽玛特性校正方法及多点控制单元 |
-
2005
- 2005-11-28 CN CN200510124080A patent/CN1852414B/zh active Active
-
2006
- 2006-07-14 WO PCT/CN2006/001671 patent/WO2007059670A1/zh active Application Filing
- 2006-07-14 EP EP06753139.2A patent/EP1959691B1/en active Active
-
2008
- 2008-05-28 US US12/128,287 patent/US8111333B2/en active Active
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH06165167A (ja) | 1992-11-17 | 1994-06-10 | Canon Inc | 画像伝送システム、画像送信装置及び画像受信装置 |
JP2000067226A (ja) | 1998-08-26 | 2000-03-03 | Toshiba Corp | 画像処理装置及び医用画像観察ネットワークに用いられるワークステーション |
CN1161972C (zh) | 1998-12-21 | 2004-08-11 | 皇家菲利浦电子有限公司 | 近似线性的伽玛数字校正 |
CN1182709C (zh) | 2000-11-24 | 2004-12-29 | 索尼公司 | 数字信号处理电路、采用数字信号处理电路的显示装置和液晶投影仪 |
CN1192619C (zh) * | 2002-01-08 | 2005-03-09 | 华为技术有限公司 | 会议电视系统中多画面显示的实现方法 |
EP1906673A1 (en) | 2005-07-07 | 2008-04-02 | Huawei Technologies Co., Ltd. | Method, system and terminal for negotiating to correct the gamma characteristic |
Non-Patent Citations (1)
Title |
---|
See also references of EP1959691A4 |
Also Published As
Publication number | Publication date |
---|---|
EP1959691B1 (en) | 2017-07-12 |
EP1959691A4 (en) | 2008-11-26 |
US8111333B2 (en) | 2012-02-07 |
CN1852414B (zh) | 2010-05-05 |
US20080231688A1 (en) | 2008-09-25 |
CN1852414A (zh) | 2006-10-25 |
EP1959691A1 (en) | 2008-08-20 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2007059670A1 (fr) | Procede de correction d'une caracteristique gamma dans un flux de codes video et unite de commande multipoint | |
US7113200B2 (en) | Method and system for preparing video communication image for wide screen display | |
CN111316625B (zh) | 用于从第一图像生成第二图像的方法和设备 | |
US20220167019A1 (en) | Method and device for reconstructing image data from decoded image data | |
US8681197B2 (en) | Communication system, terminal apparatus and computer program | |
WO2011029402A1 (zh) | 视频图像数据处理方法、装置及视频会议系统及终端 | |
TW201026054A (en) | Method and system for motion-compensated framrate up-conversion for both compressed and decompressed video bitstreams | |
EP3367685B1 (en) | Method and device for reconstructing image data from decoded image data | |
US11388447B2 (en) | Method and apparatus for processing a medium dynamic range video signal in SL-HDR2 format | |
US11423522B2 (en) | Method and apparatus for colour correction during HDR to SDR conversion | |
TW201026056A (en) | Image capturing device and image delivery method | |
KR100311354B1 (ko) | 통신시스템에서 멀티미디어 컨텐츠의 삽입을 통한 통화 서비스방법 | |
TWI381733B (zh) | 高解析度視訊會議系統 | |
JPWO2020137766A1 (ja) | 送信装置、送信方法、符号化装置、符号化方法、受信装置および受信方法 | |
CN109104635A (zh) | 即时传送屏幕图像的方法及系统 | |
CN101340546A (zh) | 高分辨率视频会议系统 | |
WO2007137479A1 (fr) | Procédé et appareil destinés à corriger le caractère gamma d'un flux vidéo | |
JPH0846928A (ja) | 画像符号化前処理装置 | |
WO2007095811A1 (fr) | Procédé de transmission d'un paramètre de caractéristique gamma vidéo et dispositif correspondant | |
Baskaran et al. | Performance study of multiple source video streaming via wireless 5.8 GHz for intelligent traffic management system | |
KR20110062788A (ko) | 카메라, 화상회의 장치 및 화상회의 방법 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application | ||
NENP | Non-entry into the national phase |
Ref country code: DE |
|
REEP | Request for entry into the european phase |
Ref document number: 2006753139 Country of ref document: EP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2006753139 Country of ref document: EP |
|
WWP | Wipo information: published in national office |
Ref document number: 2006753139 Country of ref document: EP |