CN115396622B - Electronic equipment for low-bit-rate video reconstruction - Google Patents
Electronic equipment for low-bit-rate video reconstruction Download PDFInfo
- Publication number
- CN115396622B CN115396622B CN202211330511.0A CN202211330511A CN115396622B CN 115396622 B CN115396622 B CN 115396622B CN 202211330511 A CN202211330511 A CN 202211330511A CN 115396622 B CN115396622 B CN 115396622B
- Authority
- CN
- China
- Prior art keywords
- image
- value
- dimensional image
- score
- sequence
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/01—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/01—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level
- H04N7/0117—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level involving conversion of the spatial resolution of the incoming video signal
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/01—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level
- H04N7/0127—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level by changing the field or frame frequency of the incoming video signal, e.g. frame rate converter
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Graphics (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the technical field of video processing, and particularly discloses an electronic device for low-bit-rate video reconstruction, which comprises a cache end, a video reconstruction module and a video processing module, wherein the cache end is used for acquiring a video to be reconstructed in real time and converting the video to be reconstructed into an image sequence and audio information based on the same time axis; performing score marking on each image in the image sequence according to the audio information to obtain a score sequence; positioning a target time interval according to the fraction sequence, extracting an image of the target time interval, and converting the extracted image into a one-dimensional image according to a preset conversion model; the reconstruction end is used for acquiring the one-dimensional image generated by the cache end in real time and calculating a detail value and a change value according to the one-dimensional image; and determining the transmission code rate of each image in the image sequence according to the detail value and the change value. The code rate of the technical scheme of the invention changes in real time according to the characteristics of the video to be reconstructed, and is not preset by a worker, so that the video transmission process is greatly optimized.
Description
Technical Field
The invention relates to the technical field of video processing, in particular to electronic equipment for low-bit-rate video reconstruction.
Background
The video code rate is the number of data bits transmitted per unit time during data transmission, and generally, the unit used is kbps, namely kilobits per second. The popular understanding is that the sampling rate is higher, the higher the sampling rate in unit time is, the higher the precision is, and the closer the processed file is to the original file.
But the file volume is proportional to the sampling rate, so how to achieve the minimum distortion with the lowest code rate is an important problem in the video transmission process.
On the basis of the content, the technical characteristic of variable code rate is derived, most of the existing code rate change logics are simple, different code rates are determined based on the information high and low peaks of a video source, the code rate change logics can be understood as the stacking of fixed code rates, and the code rate change logics are not really variable code rates. Therefore, if a real-time variable code rate is generated according to the video itself, optimizing the video transmission process is a technical problem to be solved by the technical scheme of the invention.
Disclosure of Invention
The present invention is directed to an electronic device for low bit rate video reconstruction, so as to solve the problems in the background art.
In order to achieve the purpose, the invention provides the following technical scheme:
an electronic device for low-rate video reconstruction, the electronic device comprising:
the cache terminal is used for acquiring videos to be reconstructed in real time and converting the videos to be reconstructed into image sequences and audio information based on the same time axis; performing score marking on each image in the image sequence according to the audio information to obtain a score sequence; positioning a target time interval according to the fraction sequence, extracting an image of the target time interval, and converting the extracted image into a one-dimensional image according to a preset conversion model; the color value of each pixel point in the one-dimensional image is a single value;
the reconstruction end is used for acquiring the one-dimensional image generated by the cache end in real time and calculating a detail value and a change value according to the one-dimensional image; the detail values are used for representing the difference between pixel points in the one-dimensional images, and the change values are used for representing the difference of adjacent one-dimensional images; and determining the transmission code rate of each image in the image sequence according to the detail value and the change value.
As a further scheme of the invention: the cache terminal comprises:
the information extraction module is used for acquiring a video to be reconstructed in real time and converting the video to be reconstructed into an image sequence and audio information based on the same time axis;
the image scoring module is used for marking the scores of all the images in the image sequence according to the audio information to obtain a score sequence;
the content conversion module is used for positioning a target time interval according to the fraction sequence, extracting an image of the target time interval and converting the extracted image into a one-dimensional image according to a preset conversion model; and the color value of each pixel point in the one-dimensional image is a single value.
As a further scheme of the invention: the reconstruction end comprises:
the numerical value generation module is used for acquiring the one-dimensional image generated by the cache end in real time and calculating a detail value and a change value according to the one-dimensional image; the detail values are used for representing the difference between pixel points in the one-dimensional images, and the change values are used for representing the difference of adjacent one-dimensional images;
and the code rate determining module is used for determining the transmission code rate of each image in the image sequence according to the detail value and the change value.
As a further scheme of the invention: the image scoring module comprises:
the maximum amplitude difference calculating unit is used for intercepting the audio information in real time according to the dynamic time window and calculating the maximum amplitude difference; the end point time of the dynamic time window is a dynamic value;
the time window determining unit is used for comparing the maximum amplitude difference with a preset amplitude difference threshold value in real time, and determining the end point time of the dynamic time window when the maximum amplitude difference reaches the preset amplitude difference threshold value;
the cyclic execution unit is used for generating a new dynamic time window by taking the end point time of the previous dynamic time window as the starting point time, and cyclically executing the contents until the audio information is intercepted;
the time-frequency mapping unit is used for carrying out Fourier transformation on the audio information in each dynamic time window to obtain frequency domain information, and the sequence generating unit is used for carrying out score marking on each image according to the frequency domain information to obtain a score sequence.
As a further scheme of the invention: the time-frequency mapping unit comprises:
the transformation subunit is used for carrying out short-time Fourier transformation on the audio information in each dynamic time window based on a spectral function to obtain frequency domain information; the dependent variable of the frequency domain information is amplitude, and the independent variable is frequency;
the characteristic generating subunit is used for carrying out numerical analysis on the frequency domain information to obtain the distribution characteristics of each frequency band; the distribution characteristics comprise amplitude maximum values and amplitude average values;
a benchmark score generation subunit, configured to determine a level of the dynamic time window according to the distribution feature, and determine a benchmark score according to the level;
the additional score calculating subunit is used for inquiring the image information corresponding to the dynamic time window, sequentially calculating the contact ratio of each image information and the adjacent image thereof, and determining the additional score according to the contact ratio;
and the score counting subunit is used for counting the reference score and the additional score of each image to obtain a score sequence.
As a further scheme of the invention: the content conversion module includes:
the curve generating unit is used for generating a fraction change curve and a derivative curve thereof according to the fraction sequence;
the curve segment intercepting unit is used for intercepting a curve segment in the fraction change curve according to a preset fraction segment and intercepting a curve segment in the derivative curve according to a preset derivative segment;
the target time interval query unit is used for querying the time intervals corresponding to the curve segments to obtain target time intervals;
the image extraction unit is used for extracting an image in a target time period, inputting the image into a preset conversion model and obtaining a one-dimensional image;
the conversion model includes:
wherein Z is the value of the pixel point, and R, G and B are respectively red, green and blueColor values corresponding to the lanes; w 1 、W 2 And W 3 Is the correlation coefficient.
As a further scheme of the invention: the extracting of the image of the target time interval and the inputting of the image into a preset conversion model to obtain the content of the one-dimensional image comprises the following steps:
will [0,1 ]]In equal parts quantified to a predetermined number, for W 1 、W 2 And W 3 Carrying out assignment;
reading a one-dimensional image output by the conversion model in real time, and inputting the one-dimensional image into a calibration function;
selecting W according to the value of the calibration function 1 、W 2 And W 3 ；
Wherein the calibration function is:
e (g) is the value of the calibration function, (x, y) are the coordinates,
is a Gaussian distribution function>
Is (g) x -g y ），/>
Is the color contrast; selecting W according to the value of the calibration function 1 、W 2 And W 3 Is such that the value of E (g) is at a maximum.
As a further scheme of the invention: the numerical value generation module comprises:
the pixel traversing unit is used for acquiring a one-dimensional image generated by the cache end in real time and traversing the value of each pixel point in the one-dimensional image;
the array generating unit is used for calculating the mean value of the one-dimensional image according to the values obtained by traversal and generating an arithmetic array based on the mean value and the most value; each numerical value in the arithmetic progression corresponds to a grade;
the matrix generating unit is used for converting the one-dimensional image into a grade matrix according to the arithmetic progression;
the data selection unit is used for carrying out region clustering on the one-dimensional images according to the grade matrix and generating simplified images and simplified matrixes according to region clustering results;
and the parameter selection unit is used for randomly selecting at least one parameter in the grade matrix, the simplified image and the simplified matrix as the detail value of the one-dimensional image.
As a further scheme of the invention: the data selecting unit comprises:
the merging subunit is used for sequentially inquiring the row and column positions of the elements corresponding to the levels, merging the adjacent elements according to the row and column positions, and obtaining each subarea corresponding to the levels; the adjacent elements are elements with the distance smaller than a preset distance threshold;
the evaluation subunit is used for selecting the value of any pixel point from the one-dimensional image to perform full-area evaluation on the sub-area corresponding to the pixel point to obtain a simplified image;
and the matrix simplifying subunit is used for fitting each sub-area into a rectangle according to a preset fitting rule and generating a simplifying matrix according to the fitted rectangle.
As a further scheme of the invention: the numerical value generation module further includes:
the mean value comparison unit is used for reading the mean value of the one-dimensional images and the mean value of the one-dimensional images adjacent to the one-dimensional images in the time domain, and calculating a variation value according to the mean value;
and the detail value comparison unit is used for reading the detail value of the one-dimensional image and the detail value of the adjacent one-dimensional image in the time domain when the change value is zero, and calculating the change value according to the detail values.
Compared with the prior art, the invention has the beneficial effects that: the method comprises the steps of processing a video to be reconstructed in a cache end to generate an easily-recognized single-valued image sequence, carrying out time-frequency domain comprehensive analysis on each single-valued image in the reconstruction end to jointly determine the importance degree of each image and even each area of each image, and adjusting the transmission code rate in real time according to the importance degree; the code rate of the technical scheme of the invention changes in real time according to the characteristics of the video to be reconstructed, and is not preset by a worker, so that the video transmission process is greatly optimized.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention.
Fig. 1 is a block diagram of a low-bitrate video reconstruction electronic device.
Fig. 2 is a block diagram of a structure of an image scoring module in an electronic device for low-bit-rate video reconstruction.
Fig. 3 is a block diagram of a structure of a content conversion module in an electronic device for low bit rate video reconstruction.
Fig. 4 is a block diagram of a structure of a value generation module in an electronic device for low-bitrate video reconstruction.
Detailed Description
In order to make the technical problems, technical solutions and advantageous effects to be solved by the present invention more clearly apparent, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Fig. 1 is a block diagram of a composition structure of an electronic device for low-rate video reconstruction, in an embodiment of the present invention, an electronic device for low-rate video reconstruction includes:
the buffer terminal 10 is configured to obtain a video to be reconstructed in real time, and convert the video to be reconstructed into an image sequence and audio information based on the same time axis; performing score marking on each image in the image sequence according to the audio information to obtain a score sequence; positioning a target time interval according to the fraction sequence, extracting an image of the target time interval, and converting the extracted image into a one-dimensional image according to a preset conversion model; the color value of each pixel point in the one-dimensional image is a single value;
the reconstruction end 20 is used for acquiring the one-dimensional image generated by the cache end in real time and calculating a detail value and a change value according to the one-dimensional image; the detail values are used for representing the difference between pixel points in the one-dimensional images, and the change values are used for representing the difference of adjacent one-dimensional images; and determining the transmission code rate of each image in the image sequence according to the detail value and the change value.
The electronic equipment consists of two parts, namely a cache end 10 and a reconstruction end 20, wherein the cache end 10 is used for receiving a video to be reconstructed and processing the video to be reconstructed; the reconstruction terminal 20 is configured to identify the processed video to be reconstructed and change the transmission code rate in real time. The video to be reconstructed is acquired by the cache terminal 10, the acquisition process is performed in real time, after the reconstruction terminal 20 determines the transmission code rate, the electronic device sends the video to be reconstructed to other application devices based on the transmission code rate, and the cache terminal 10 and the reconstruction terminal 20 are all sub-parts of the electronic device.
Specifically, the cache terminal 10 includes:
the information extraction module 11 is configured to obtain a video to be reconstructed in real time, and convert the video to be reconstructed into an image sequence and audio information based on the same time axis;
the image scoring module 12 is configured to mark scores of the images in the image sequence according to the audio information to obtain a score sequence;
a content conversion module 13, configured to position a target time interval according to the score sequence, extract an image of the target time interval, and convert the extracted image into a one-dimensional image according to a preset conversion model; and the color value of each pixel point in the one-dimensional image is a single value.
The function of the cache terminal 10 is mainly to process the video to be reconstructed, the processing process is completed by three modules, namely an information extraction module 11, an image scoring module 12 and a content conversion module 13, the function of the information extraction module 11 is to convert the video to be reconstructed into an image sequence and audio information, and for a video file, the video file itself is composed of the image sequence and the audio information together, so that the conversion process is not difficult.
The image scoring module 12 and the content conversion module 13 are more important in function, and the purpose of the function is to determine some more "important" images, and then process the "important" images to convert the images into an image with a single color value, namely, the one-dimensional image; among them, a gray scale image is a one-dimensional image.
As for whether the image is important or not, the image is determined by the audio information, the video to be reconstructed is mostly regular video, such as a movie and the like, the video is not random video like monitoring video and can be predicted, and the audio information is very stable, so that the importance degree of each image can be judged by identifying and analyzing the audio information.
The reconstruction end 20 includes:
the numerical value generation module 21 is configured to obtain a one-dimensional image generated by the cache end in real time, and calculate a detail value and a change value according to the one-dimensional image; the detail values are used for representing the difference between pixel points in the one-dimensional images, and the change values are used for representing the difference of adjacent one-dimensional images;
and a code rate determining module 22, configured to determine a transmission code rate of each image in the image sequence according to the detail value and the change value.
The function of the reconstruction terminal 20 is simple, namely, reading the one-dimensional image, identifying the one-dimensional image, further positioning each sub-region in the one-dimensional image, and determining a detail value according to the characteristics of the sub-region; generating a change value according to the detail value corresponding to the adjacent image in the time domain; and determining the transmission code rate of each image in the image sequence according to the detail value and the change value.
For the determination process of the transmission code rate, it needs to be further explained that, for the "unimportant" pictures, a lower code rate can be used, for the "important" pictures, a higher code rate needs to be used for transmission, and on the basis that the detail value is known, even further distinction can be made, that is, high code rate transmission is performed on the "important" regions in the "important" pictures, and low code rate transmission is performed on the "unimportant" regions.
Fig. 2 is a block diagram illustrating a structure of an image scoring module in an electronic device for low-bit-rate video reconstruction, where the image scoring module 12 includes:
a maximum amplitude difference calculation unit 121, configured to intercept the audio information in real time according to the dynamic time window, and calculate a maximum amplitude difference; the end point time of the dynamic time window is a dynamic value;
a time window determining unit 122, configured to compare the maximum amplitude difference with a preset amplitude difference threshold in real time, and determine an end point time of a dynamic time window when the maximum amplitude difference reaches the preset amplitude difference threshold;
a loop execution unit 123, configured to generate a new dynamic time window by using the end time of the previous dynamic time window as the start time, and loop execute the above contents until the audio information is intercepted;
the time-frequency mapping unit 124 is configured to perform fourier transform on the audio information in each dynamic time window to obtain frequency domain information, and the sequence generating unit is configured to perform score marking on each image according to the frequency domain information to obtain a score sequence.
The above-mentioned content limits the processing process of the audio information, and firstly, the audio information is divided in time domain, because the video to be reconstructed in the technical scheme of the present invention is defaulted to be a very stable video, the characteristics of the audio information are mostly obvious, and there is no noise (each audio in the movie is meaningful), at this time, only the audio segments with the same amplitude magnitude are required to be separated in sequence, that is, the time window is the above-mentioned time window.
For audio information, time domain information can only judge the sound amplitude at each time, and it cannot judge what kind of sound is, so that it is necessary to convert the time domain information into frequency domain information, and judge what kind of sound is from the frequency of the sound, and it is conceivable that since the hearing recognition region of human ears is 20Hz to 20000Hz, the frequency distribution of audio information in a video to be reconstructed (such as a movie) is also roughly in this range.
The type of the sound is determined according to the frequency domain information, the size of the sound is determined according to the time domain information, and as a result, which moments are 'important' moments can be judged, and the corresponding images are also regarded as 'important' images.
For the process of converting time domain information into frequency domain information, the following is specific:
the time-frequency mapping unit 124 includes:
the transformation subunit is used for carrying out short-time Fourier transformation on the audio information in each dynamic time window based on a spectral function to obtain frequency domain information; the dependent variable of the frequency domain information is amplitude, and the independent variable is frequency;
the spectral function is an MATLAB function, a spectrogram of a signal is obtained by using short-time Fourier transform, and the spectrogram can be easily called in the existing software;
the characteristic generating subunit is used for carrying out numerical analysis on the frequency domain information to obtain the distribution characteristics of each frequency band; the distribution characteristics comprise amplitude maximum values and amplitude average values;
because the audio information of the technical scheme of the invention is relatively stable, the distribution characteristics of each frequency band only need to adopt the amplitude maximum value and the amplitude average value.
A benchmark score generation subunit, configured to determine a level of the dynamic time window according to the distribution feature, and determine a benchmark score according to the level;
the frequency domain information does not contain time information, but the time-frequency conversion process is respectively carried out on different time windows, so that which time window each frequency domain information corresponds to is determined, and after the sound type is determined by the distribution characteristics of the frequency domain information, the 'important' level of the corresponding time window can be determined; for example, in a martial arts movie, if the distribution characteristics of the frequency domain information corresponding to a certain time window reflect that it is the sound of weapon collision (the frequencies are obviously different), its "importance" level is higher, and if the distribution characteristics of the frequency domain information corresponding to a certain time window reflects that it is the link of personnel communication, its "importance" level is lower.
The additional score calculating subunit is used for inquiring the image information corresponding to the dynamic time window, sequentially calculating the contact ratio of each image information and the adjacent image thereof, and determining the additional score according to the contact ratio;
the "important" level corresponds to a reference score, and on this basis, the corresponding image information is inquired to determine whether the difference from the previous image is large, and an image with a large change is more important than an image without a large change. The size of the change occurrence is reflected by the additional score.
And the score counting subunit is used for counting the reference score and the additional score of each image to obtain a score sequence.
And counting the reference score and the additional score to obtain a score sequence.
Fig. 3 is a block diagram of a structure of a content conversion module in an electronic device for low bit rate video reconstruction, where the content conversion module 13 includes:
a curve generating unit 131, configured to generate a score variation curve and a derivative curve thereof according to the score sequence;
a curve segment intercepting unit 132, configured to intercept a curve segment in the fraction variation curve according to a preset fraction segment, and intercept a curve segment in the derivative curve according to a preset derivative segment;
a target time interval query unit 133, configured to query a time interval corresponding to the curve segment to obtain a target time interval;
an image extracting unit 134, configured to extract an image in a target time period, and input the image into a preset conversion model to obtain a one-dimensional image;
the conversion model includes:
wherein Z is the value of the pixel point, and R, G and B are the color values corresponding to the red channel, the green channel and the blue channel respectively; w 1 、W 2 And W 3 Is the correlation coefficient.
The score sequence corresponds to the image sequence one by one, and each image has a score; fitting a curve according to the fractional sequence, and deriving the curve to obtain a derivative curve; whether the score meets the condition (the score section) or the derivative meets the condition (the derivative section), the corresponding time section is regarded as the target time section, and then the image of the target time section is processed.
Specifically, the extracting an image in a target time period, and inputting the image into a preset conversion model to obtain the content of the one-dimensional image includes:
will [0,1 ]]In equal parts quantified to a predetermined number, for W 1 、W 2 And W 3 Carrying out assignment;
reading a one-dimensional image output by the conversion model in real time, and inputting the one-dimensional image into a calibration function;
selecting W according to the value of the calibration function 1 、W 2 And W 3 ；
Wherein the calibration function is:
e (g) is the value of the calibration function, (x, y) are the coordinates,
is a Gaussian distribution function>
Is (g) x -g y ），/>
Is the color contrast; selecting W according to the value of the calibration function 1 、W 2 And W 3 Is such that the value of E (g) is at a maximum.
For the above W 1 、W 2 And W 3 The assignment process of (a) is specified by the transformation model and W 1 、W 2 And W 3 The limitation of (A) is that W is 1 、W 2 And W 3 There are infinite combinations of values of (A), however, W 1 、W 2 And W 3 The influence of the slight change of (2) on the image itself is not large, and therefore, the above-mentioned inner partCapacitance pair [0,1]The intervals are quantized and then assigned.
For example, if the predetermined number is 10,W 1 、W 2 And W 3 The value set of (1) is {0,0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,1.0}.
Fig. 4 is a block diagram of a structure of a value generation module in an electronic device for low-bitrate video reconstruction, where the value generation module 21 includes:
the pixel traversing unit 211 is configured to obtain a one-dimensional image generated by the cache end in real time, and traverse values of each pixel point in the one-dimensional image;
a sequence generating unit 212, configured to calculate a mean value of the one-dimensional image according to the traversed values, and generate an arithmetic sequence based on the mean value and a most significant value; each numerical value in the arithmetic progression corresponds to a grade;
a matrix generating unit 213, configured to convert the one-dimensional image into a rank matrix according to the arithmetic progression;
the data selecting unit 214 is configured to perform region clustering on the one-dimensional image according to the rank matrix, and generate a simplified image and a simplified matrix according to a region clustering result;
and a parameter selecting unit 215 for arbitrarily selecting at least one parameter of the rank matrix, the reduced image and the reduced matrix as a detail value of the one-dimensional image.
The generation process of the detail value is specifically described, firstly, the mean value of the values of all the pixel points in the one-dimensional image is calculated, a classification gradient, namely the equal-difference sequence, is determined according to the mean value and two most values of all the pixel points in the one-dimensional image, and all the pixel points in the one-dimensional image can be classified according to the equal-difference sequence to obtain a grade matrix; the hierarchical matrix is analyzed to generate a simplified image and a simplified matrix.
Any one of the parameters of the rank matrix, the reduced image, and the reduced matrix may be used as or generate the detail values.
In an example of the technical solution of the present invention, the data selecting unit 214 includes:
the merging subunit is used for sequentially inquiring the row and column positions of the elements corresponding to the levels, merging the adjacent elements according to the row and column positions, and obtaining each subarea corresponding to the levels; the adjacent elements are elements with the distance smaller than a preset distance threshold;
the evaluation subunit is used for selecting the value of any pixel point in the one-dimensional image to carry out overall evaluation on the sub-area corresponding to the pixel point so as to obtain a simplified image;
the simplified image is simplified by partitioning the image according to the level matrix, and in each region, replacing the values of all other pixels with the value of one pixel, so as to obtain the simplified image.
The matrix simplification subunit is used for fitting each sub-area into a rectangle according to a preset fitting rule and generating a simplified matrix according to the fitted rectangle;
the generation principle of the simplified matrix is also simple, and the matrix is further simplified based on the thought of the partition, it is to be noted that the matrix is a rectangle, and on the basis of reading the partition result, the partition result needs to be fitted into the rectangle, so that the block matrix can be generated, and further the simplification is performed.
As a preferred embodiment of the technical solution of the present invention, the numerical value generating module 21 further includes:
a mean value comparison unit 216, configured to read a mean value of one-dimensional images and a mean value of one-dimensional images adjacent to the mean value in the time domain, and calculate a variation value according to the mean value;
and a detail value comparison unit 217 for reading the detail value of the one-dimensional image and the detail values of the one-dimensional images adjacent thereto in the time domain when the change value is zero, and calculating the change value according to the detail values.
The generation process of the variation value is easy, namely, the difference between the images adjacent in time is judged, and the judged parameter can be an average value generated in the content or a variation value generated in the content; both are progressive use-i.e. if the mean is available (not zero) then the mean is used, if the mean is not available (zero) then the detail value is used.
Specifically, after the detail value and the variation value are both determined, the mapping model for determining the transmission code rate according to the detail value and the variation value is specifically set by a worker independently in combination with hardware parameters.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.
Claims (7)
1. An electronic device, characterized in that the electronic device comprises:
the buffer terminal is used for acquiring a video to be reconstructed in real time and converting the video to be reconstructed into an image sequence and audio information based on the same time axis; performing score marking on each image in the image sequence according to the audio information to obtain a score sequence; positioning a target time interval according to the score sequence, extracting an image of the target time interval, and converting the extracted image into a one-dimensional image according to a preset conversion model; the color value of each pixel point in the one-dimensional image is a single value;
the reconstruction end is used for acquiring the one-dimensional image generated by the cache end in real time and calculating a detail value and a change value according to the one-dimensional image; the detail values are used for representing the difference between pixel points in the one-dimensional images, and the change values are used for representing the difference of adjacent one-dimensional images; determining the transmission code rate of each image in the image sequence according to the detail value and the change value;
the reconstruction end comprises:
the numerical value generation module is used for acquiring the one-dimensional image generated by the cache end in real time and calculating a detail value and a change value according to the one-dimensional image;
a code rate determining module, configured to determine a transmission code rate of each image in the image sequence according to the detail value and the change value;
the numerical value generation module comprises:
the pixel traversing unit is used for acquiring a one-dimensional image generated by the cache end in real time and traversing the value of each pixel point in the one-dimensional image;
the array generating unit is used for calculating the mean value of the one-dimensional image according to the values obtained by traversal and generating an arithmetic array based on the mean value and two most significant values of all pixel points in the one-dimensional image; each numerical value in the arithmetic progression corresponds to a grade;
the matrix generating unit is used for converting the one-dimensional image into a grade matrix according to the arithmetic progression;
the data selection unit is used for carrying out region clustering on the one-dimensional images according to the grade matrix and generating simplified images and simplified matrixes according to region clustering results;
the parameter selection unit is used for selecting at least one parameter in the grade matrix, the simplified image and the simplified matrix as a detail value of the one-dimensional image;
the data selecting unit comprises:
the merging subunit is used for sequentially inquiring the row and column positions of the elements corresponding to each grade, merging the adjacent elements according to the row and column positions, and obtaining each subarea corresponding to each grade; the adjacent elements are elements with the distance smaller than a preset distance threshold;
the evaluation subunit is used for selecting the value of any pixel point in the one-dimensional image to carry out overall evaluation on the sub-area corresponding to the pixel point so as to obtain a simplified image;
and the matrix simplifying subunit is used for fitting each sub-area into a rectangle according to a preset fitting rule and generating a simplifying matrix according to the fitted rectangle.
2. The electronic device of claim 1, wherein the cache side comprises:
the information extraction module is used for acquiring a video to be reconstructed in real time and converting the video to be reconstructed into an image sequence and audio information based on the same time axis;
the image scoring module is used for marking the scores of all the images in the image sequence according to the audio information to obtain a score sequence;
the content conversion module is used for positioning a target time interval according to the score sequence, extracting an image of the target time interval and converting the extracted image into a one-dimensional image according to a preset conversion model; and the color value of each pixel point in the one-dimensional image is a single value.
3. The electronic device of claim 2, wherein the image scoring module comprises:
the maximum amplitude difference calculating unit is used for intercepting the audio information in real time according to the dynamic time window and calculating the maximum amplitude difference; the end point time of the dynamic time window is a dynamic value;
the time window determining unit is used for comparing the maximum amplitude difference with a preset amplitude difference threshold value in real time, and determining the end point time of the dynamic time window when the maximum amplitude difference reaches the preset amplitude difference threshold value;
the cyclic execution unit is used for generating a new dynamic time window by taking the end point time of the previous dynamic time window as the starting point time, and cyclically executing the contents until the audio information is intercepted;
the time-frequency mapping unit is used for carrying out Fourier transformation on the audio information in each dynamic time window to obtain frequency domain information, and the sequence generating unit is used for carrying out score marking on each image according to the frequency domain information to obtain a score sequence.
4. The electronic device of claim 3, wherein the time-frequency mapping unit comprises:
the transformation subunit is used for carrying out short-time Fourier transformation on the audio information in each dynamic time window based on a spectral function to obtain frequency domain information; the dependent variable of the frequency domain information is amplitude, and the independent variable is frequency;
the characteristic generating subunit is used for carrying out numerical analysis on the frequency domain information to obtain the distribution characteristics of each frequency band; the distribution characteristics comprise amplitude maximum values and amplitude average values;
a benchmark score generation subunit, configured to determine the level of the dynamic time window according to the distribution feature, and determine a benchmark score according to the level;
the additional score calculating subunit is used for inquiring the image information corresponding to the dynamic time window, sequentially calculating the contact ratio of each image information and the adjacent image thereof, and determining the additional score according to the contact ratio;
and the score counting subunit is used for counting the reference score and the additional score of each image to obtain a score sequence.
5. The electronic device of claim 2, wherein the content conversion module comprises:
the curve generating unit is used for generating a fraction change curve and a derivative curve thereof according to the fraction sequence;
the curve segment intercepting unit is used for intercepting a curve segment in the fraction change curve according to a preset fraction segment and intercepting a curve segment in the derivative curve according to a preset derivative segment;
the target time period query unit is used for querying the time period corresponding to the curve segment to obtain a target time period;
the image extraction unit is used for extracting an image in a target time interval and inputting the image into a preset conversion model to obtain a one-dimensional image;
the conversion model includes:
wherein Z is the value of the pixel point, and R, G and B are the color values corresponding to the red channel, the green channel and the blue channel respectively; w 1 、W 2 And W 3 Is the correlation coefficient.
6. The electronic device of claim 5, wherein the extracting the image of the target time period and inputting the image into a preset conversion model to obtain the content of the one-dimensional image comprises:
will be within the interval [0,1]In equal parts quantified to a predetermined number, for W 1 、W 2 And W 3 Carrying out assignment;
reading a one-dimensional image output by the conversion model in real time, and inputting the one-dimensional image into a calibration function;
selecting W according to the value of the calibration function 1 、W 2 And W 3 ；
Wherein the calibration function is:
e (g) is the value of the calibration function, x and y are two points,
in order to be a function of the gaussian distribution,
is (g) x -g y ），
Is the color contrast; g is a one-dimensional image, g x And g y The values of the point x and the point y in the one-dimensional image are
Is determined in the calculation process of (1); the P is used for representing an image area; selecting W according to the value of the calibration function 1 、W 2 And W 3 Is such that the value of E (g) is at a maximum.
7. The electronic device of claim 1, wherein the value generation module further comprises:
the mean value comparison unit is used for reading the mean value of the one-dimensional images and the mean value of the one-dimensional images adjacent to the one-dimensional images in the time domain, and calculating a variation value according to the mean value;
and the detail value comparison unit is used for reading the detail value of the one-dimensional image and the detail value of the adjacent one-dimensional image in the time domain when the change value calculated according to the average value is zero, and calculating the change value according to the detail value.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211330511.0A CN115396622B (en) | 2022-10-28 | 2022-10-28 | Electronic equipment for low-bit-rate video reconstruction |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211330511.0A CN115396622B (en) | 2022-10-28 | 2022-10-28 | Electronic equipment for low-bit-rate video reconstruction |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115396622A CN115396622A (en) | 2022-11-25 |
| CN115396622B true CN115396622B (en) | 2023-03-24 |
Family
ID=84115308
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211330511.0A Active CN115396622B (en) | 2022-10-28 | 2022-10-28 | Electronic equipment for low-bit-rate video reconstruction |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115396622B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116069976B (en) * | 2023-03-06 | 2023-09-12 | 南京和电科技有限公司 | Regional video analysis method and system |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017000465A1 (en) * | 2015-07-01 | 2017-01-05 | 中国矿业大学 | Method for real-time selection of key frames when mining wireless distributed video coding |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5054608B2 (en) * | 2008-05-08 | 2012-10-24 | 日本電信電話株式会社 | Content evaluation method and apparatus, program, and computer-readable recording medium |
| US10664687B2 (en) * | 2014-06-12 | 2020-05-26 | Microsoft Technology Licensing, Llc | Rule-based video importance analysis |
| CN104240197B (en) * | 2014-08-26 | 2017-12-15 | 浙江工商大学 | A kind of erasing method for keeping contrast, colour consistency and gray-scale pixels feature |
| CN108052948B (en) * | 2017-11-14 | 2020-01-17 | 武汉科技大学 | Coding method for extracting image characteristics |
| CN111866522A (en) * | 2019-04-29 | 2020-10-30 | 杭州海康威视数字技术股份有限公司 | Video data coding method and device |
| CN111787318A (en) * | 2020-06-24 | 2020-10-16 | 浙江大华技术股份有限公司 | Video code rate control method, device, equipment and storage device |
| CN113987264A (en) * | 2021-10-28 | 2022-01-28 | 北京中科闻歌科技股份有限公司 | Video abstract generation method, device, equipment, system and medium |
-
2022
- 2022-10-28 CN CN202211330511.0A patent/CN115396622B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017000465A1 (en) * | 2015-07-01 | 2017-01-05 | 中国矿业大学 | Method for real-time selection of key frames when mining wireless distributed video coding |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115396622A (en) | 2022-11-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108090902B (en) | Non-reference image quality objective evaluation method based on multi-scale generation countermeasure network | |
| CN105208374B (en) | A kind of non-reference picture assessment method for encoding quality based on deep learning | |
| CN115396622B (en) | Electronic equipment for low-bit-rate video reconstruction | |
| CN110675395A (en) | Intelligent on-line monitoring method for power transmission line | |
| CN103841410A (en) | Half reference video QoE objective evaluation method based on image feature information | |
| TWI836607B (en) | Method and system for estimating levels of distortion | |
| CN116310882A (en) | Forestry information identification method based on high-resolution remote sensing image | |
| Wang et al. | Specific emitter identification based on deep adversarial domain adaptation | |
| CN117593193B (en) | Sheet metal image enhancement method and system based on machine learning | |
| CN117523869B (en) | Traffic signal lamp intelligent control system and method based on traffic flow | |
| Freitas et al. | Blind Image Quality Assessment Using Multiscale Local Binary Patterns. | |
| CN103578104B (en) | A kind of partial reference image method for evaluating objective quality for Gaussian Blur image | |
| CN116705059B (en) | Audio semi-supervised automatic clustering method, device, equipment and medium | |
| CN112233696A (en) | Oil field pumping unit abnormal sound detection and reporting system based on artificial intelligence and big data | |
| CN112468721A (en) | Visual acquisition method and device with automatic focusing function | |
| CN114218574B (en) | Data detection method and device, electronic equipment and storage medium | |
| Freitas et al. | Blind image quality assessment using multiscale local binary patterns | |
| CN110048977A (en) | The recognition methods of short-wave signal system and device based on the detection of gray level co-occurrence matrixes textural characteristics | |
| CN108764277A (en) | A kind of non-reference picture Fuzzy Quality evaluation algorithms | |
| Weng et al. | Message structure aided attentional convolution network for RF device fingerprinting | |
| CN110263810B (en) | LoRa signal source identification method | |
| CN118294540B (en) | Method and system for detecting defects of torsion leaf springs based on deep learning | |
| CN109215633A (en) | The recognition methods of cleft palate speech rhinorrhea gas based on recurrence map analysis | |
| CN117498992B (en) | Audio data transmission analysis method and system based on user characteristics | |
| Zeggari et al. | Perceptual image quality assessment based on gradient similarity and Ruderman operator |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |