CN102801968B - Rapid intelligent image/video retargeting method - Google Patents
Rapid intelligent image/video retargeting method Download PDFInfo
- Publication number
- CN102801968B CN102801968B CN201210201663.0A CN201210201663A CN102801968B CN 102801968 B CN102801968 B CN 102801968B CN 201210201663 A CN201210201663 A CN 201210201663A CN 102801968 B CN102801968 B CN 102801968B
- Authority
- CN
- China
- Prior art keywords
- image
- energy
- ratio value
- row
- pixel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Television Systems (AREA)
- Controls And Circuits For Display Device (AREA)
Abstract
The invention belongs to the technical field of digital images and videos, in particular to a rapid intelligent image/video retargeting method, which specifically comprises the following steps of: calculating an energy diagram of each frame of image; unifying interline energy to ensure the consistency of adjacent lines; calculating a scale value of each column according to the energy diagram, judging whether the calculated scale values are within a rational range or not to avoid image distortion, and correcting the scale values if the calculated scale values are out of a rational range; and changing the size of each frame of image according to the scale values of each column. The method has a good effect and high efficiency, and high-quality service can be provided for various related application.
Description
Technical field
The invention belongs to digital picture and video technique field, be specifically related to the quick, intelligent adaptive display packing of a kind of image and video.
Background technology
In recent years, universal along with various mobile terminals such as the develop rapidly of mobile communication technology and IPAD, mobile phone, PDA and MP4, people get more and more for the user demand of multimedia messages, and particularly digital picture (photo) and video have become people and to express thoughts and the very important digital entertainment instrument of phatic one.People need better visual experience when sharing these digital-visual information, and this demand driving multimedia information is developed rapidly.Image and video adaptation display (Image/Video Retargeting) technology are owing to having good treatment effect to image and video, obtain the concern of more and more researcher in recent years, and become a study hotspot in current multimedia signal processing field.This technology is with a wide range of applications.Specifically comprise the following aspects:
1) the intelligent convergent-divergent of digital photos: people at various mobile terminal such as: in mobile phone, IPAD, MP4 etc. browse digital photograph, display screen due to different terminals has different resolution and length-width ratio, so need to carry out convergent-divergent to original image, to adapt to the resolution requirement of display device.Traditional image scaling, mainly adopts the mode of resampling at equal intervals and image cut.All there is significant deficiency in these two kinds of methods: the former can cause the picture material after convergent-divergent unintelligible or produce sawtooth distortion; Although the latter can retain the important information required for user, if user is interested for non-conterminous region multiple in picture, it is then helpless.Image scaling based on image adaptation display based on the sensing results of user for image, can realize convergent-divergent adaptively to picture, thus retains the important information in picture to greatest extent.
2) the optimization display of different-format TV programme: the development of China's TV tech is at present in the transition stage of SD program to high-definition program.The television set of market fast sale is high definition mostly, and its length-width ratio is 16:9; And the program of most of channel remains SD form, length-width ratio is 4:3.During HDTV (High-Definition Television) display SD program, the general mode adopting geometric ratio stretching or both sides to add black surround.The former can cause programme content length-width ratio to lack of proper care, and occurs that such as personage becomes the problem such as fat; The display of the latter does not make full use of screen size, affects viewing effect.Video adaptation Display Technique can effectively solve this problem, realizes, in the process of HDTV (High-Definition Television) optimization display, effectively to keep the length-width ratio of important objects changing length-width ratio of video, thus greatly improve user watch experience.
3) bandwidth conservation in video transmitting procedure: video data is surprising due to data volume, needs effectively to compress before being transmitted, and thus the Video coding of high compression ratio is very crucial for high-quality transmission of video.Video adaptation Display Technique can realize compression further based on existing video encoder for video.Specifically, original video data, before entering video encoder, first adopts video adaptation Display Technique to realize the reduction of resolution, can not affect the subjective quality of video simultaneously, and then carry out Video coding.At receiving terminal, the later video sequence of decoders decode re-uses the raising that video adaptation Display Technique realizes resolution.Because video sequence during Video coding reduces resolution, so combine video adaptation Display Technique effectively can improve code efficiency, save transmission bandwidth.
Summary of the invention
The object of the invention is to propose a kind of efficient image and video adaptation Display Realization method, the i.e. quick, intelligent adaptive display packing of image and video fast, thus effectively for every application provides high-quality service.
The quick, intelligent adaptive display packing of the image that the present invention proposes and video, its method flow as shown in Figure 1.
Without loss of generality, we only discuss and carry out adaptation display to image or video in the horizontal direction.Adaptation for video shows, and what the present invention adopted is the mode processed frame by frame.As shown in the figure, concrete steps are:
The first step, calculates the energy diagram of each two field picture;
Second step, unifies energy in the ranks, to ensure the consistency between adjacent lines;
3rd step, calculate the ratio value of each row according to energy diagram, in order to avoid scalloping, the ratio value calculated must judge whether in the reasonable scope, if incorrect, needs to correct;
4th step, image adaptation display and pixel fusion, namely the ratio value that arranges according to each of each two field picture is varying sized.
Below each step is specifically described.
One, the calculating of energy diagram
Energy diagram and the original image of two dimensional image have identical resolution, and embody the importance of each pixel by the value between 0 to 1, and the value in energy diagram is larger just represents that this pixel is more important.In method herein, energy diagram can obtain according to the algorithm of Itti [1].
Two, in the ranks energy is unified
Energy diagram obtained in the previous step can not be used directly calculating ratio value.Image is worth in proportion flexible before, energy in the ranks must be unified, with this ensure picture size change after consistency between adjacent lines.The formula of unified energy is as follows:
Wherein
refer to the height of original image,
refer to
row
the energy value of row, unified energy value
refer to the average energy of each row.
Three, the calculating of ratio value figure and rectification
In the present invention, each pixel has a ratio value to represent its flexible degree.In image, each pixel is by as an independently base unit, image level direction flexible concerning these pixels, changes their width exactly, become ratio value, and ratio value is the decimal of a non-negative from original 1.So can expect, in every a line, the ratio value sum of all pixels is the width of target image.Propose a simple transformation equation (2) efficiently herein, ratio value figure can be calculated according to energy diagram.
Wherein,
with
the width of original image and target image respectively.
According to the different proportion value calculated by new energy diagram, the pixel in image can be stretched according to the situation of reality, and important pixel in the target image can be wider, and unessential pixel can be narrower.Therefore, important in original image region also can be kept original size in the target image as far as possible.
The rectification of ratio value.If the object of image adaptation display is the width of downscaled images
, pass through formula
calculating,
situation just probably there will be.This situation is very irrational, and can change the structure of image to a great extent.In order to address this problem, we are with formula
calculate
afterwards, must obtain and first travel through all row, find
those row, by these arrange ratio values be set to 1, here we use
represent these numbers arranged.Then remaining to all the other the new allocation proportion value of column weight, concrete formula is as follows:
Repeat this process, until the ratio value of all row is all satisfied always
.
Similarly, for the situation that the width of original image is increased
,
also be not allowed to.Similar step also illustrates in FIG, is different from formula
, the formula that the ratio value that the row remaining to all the other do is redistributed is as follows:
Four, image adaptation display and pixel fusion
After obtaining ratio value figure, target image can be obtained by the method based on pixel fusion.This side's ratio juris is such, and suppose that image is horizontal direction compression, the width of each pixel can become a decimal according to the ratio value of this pixel from original unit length.Then using the width of these pixels as weight, linearly in conjunction with these pixels, formed with this target pixel points that width is 1.The method of this pixel fusion is different from the pixel chosen in original image and puts into target image, but the pixel in original image is merged the new target pixel points of formation according to certain rules, and the benefit so done makes target image more level and smooth.
The present invention proposes a kind of efficient video adaptation Display Realization method fast.Institute's extracting method has good effect and higher efficiency, can provide high-quality service for various related application.
Leading reference
[1]L. Itti, C. Koch, and E. Niebur, “A model of saliency-based visual attention for rapid scene analysis,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, no. 11, pp. 1254 –1259, nov 1998。
Accompanying drawing explanation
Fig. 1 is the inventive method flow process diagram.
Embodiment
The PC that have employed 3.1 GHz Intel Xeon CPU and 2G RAM in experiment tests, and algorithmic language have employed Matlab.Target is that the photo resolution being 1024*800 by a resolution becomes half (512*800).According to the step of invention institute extracting method, calculate this picture energy diagram successively, energy is unified, employing formula (3) distributes the scale factor often arranged to adopt formula (1) to carry out, and finally carries out the pixel fusion of image.
Experimental result shows, algorithm of carrying has good effect, can keep the length-width ratio of image important objects.Simultaneously carry algorithm there is higher efficiency: the photo resolution being 1024*800 by a resolution becomes half (512*800) on average probably only needs 0.2s.Consider that the language Matlab efficiency that emulation uses is lower, carrying out suitable optimization according to other language will raise the efficiency further.
Claims (1)
1. a quick, intelligent adaptive display packing for image and video, adopts the mode processed frame by frame for video, it is characterized in that concrete steps are:
The first step, calculates the energy diagram of each two field picture;
Second step, unifies energy in the ranks, to ensure the consistency between adjacent lines;
3rd step, calculate the ratio value of each row according to energy diagram, in order to avoid scalloping, the ratio value calculated must judge whether in the reasonable scope, if incorrect, needs to correct;
4th step, image adaptation display and pixel fusion, namely the ratio value that arranges according to each of each two field picture is varying sized;
It is as follows that each step carries out particular content:
One, the calculating of energy diagram
Energy diagram and the original image of two dimensional image have identical resolution, and embody the importance of each pixel by the value between 0 to 1, and the value in energy diagram is larger just represents that this pixel is more important; Energy diagram obtains according to the algorithm of Itti;
Two, in the ranks energy is unified
The computing formula (1) that in the ranks energy is unified is as follows:
Wherein,
refer to the height of original image,
refer to
row
the energy value of row, unified energy value
refer to the average energy of each row;
Three, the calculating of ratio value figure and rectification
The calculating of ratio value figure, adopts equation of transformation (2):
Wherein,
with
the width of original image and target image respectively;
The rectification of ratio value
According to
with
between different relation, use formula
calculate
afterwards, first travel through all row, find
those row, will
the ratio value of row be set to 1, here, use
represent these numbers arranged; Then,
1. when
time, the column weight new allocation proportion value remaining to all the other, concrete formula is as follows:
Repeat this process, until the ratio value of all row is all satisfied always
;
2. when
time, the formula that the ratio value that the row remaining to all the other do is redistributed is as follows:
Four, image adaptation display and pixel fusion
After obtaining ratio value figure, target image is obtained by the method based on pixel fusion; This side's ratio juris is: suppose that image is horizontal direction compression, the width of each pixel becomes a decimal according to the ratio value of this pixel from original unit length; Then using the width of these pixels as weight, linearly in conjunction with these pixels, formed with this target pixel points that width is 1.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210201663.0A CN102801968B (en) | 2012-06-19 | 2012-06-19 | Rapid intelligent image/video retargeting method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210201663.0A CN102801968B (en) | 2012-06-19 | 2012-06-19 | Rapid intelligent image/video retargeting method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102801968A CN102801968A (en) | 2012-11-28 |
| CN102801968B true CN102801968B (en) | 2015-05-13 |
Family
ID=47200923
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210201663.0A Active CN102801968B (en) | 2012-06-19 | 2012-06-19 | Rapid intelligent image/video retargeting method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102801968B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104574271B (en) * | 2015-01-20 | 2018-02-23 | 复旦大学 | A kind of method of advertising logo insertion digital picture |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050094033A1 (en) * | 2003-10-30 | 2005-05-05 | Brian Schoner | Method and system for automatic detection and display of aspect ratio |
| CN1858843A (en) * | 2005-08-12 | 2006-11-08 | 深圳华为移动通信技术有限公司 | Self adaptive pantographic system and its method for image in mobile terminal |
| US20110262055A1 (en) * | 2008-12-23 | 2011-10-27 | Koninklijke Philips Electronics N.V. | Image scaling curve generation |
-
2012
- 2012-06-19 CN CN201210201663.0A patent/CN102801968B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050094033A1 (en) * | 2003-10-30 | 2005-05-05 | Brian Schoner | Method and system for automatic detection and display of aspect ratio |
| CN1858843A (en) * | 2005-08-12 | 2006-11-08 | 深圳华为移动通信技术有限公司 | Self adaptive pantographic system and its method for image in mobile terminal |
| US20110262055A1 (en) * | 2008-12-23 | 2011-10-27 | Koninklijke Philips Electronics N.V. | Image scaling curve generation |
Non-Patent Citations (1)
| Title |
|---|
| 《A model of saliency-based visual attention for rapid scene analysis》;L. Itti, C. Koch, and E. Niebur;《IEEE Transactions on Pattern Analysis and Machine Intelligence》;19981130;第20卷;1254-1259 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102801968A (en) | 2012-11-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20190141323A1 (en) | Video image encoding method and apparatus, and video image decoding method and apparatus | |
| EP3637781A1 (en) | Video processing method and apparatus | |
| CN103167284B (en) | A kind of video stream transmission method and system based on picture super-resolution | |
| CN104980697A (en) | Video transmission method for web camera | |
| CN105210374B (en) | The control of super-resolution is improved to frequency using characteristics of image | |
| CN105230033B (en) | It is converted using adaptive frequency intensity-controlled to create the details in image | |
| CN105230032B (en) | The details in image is created using adaptive frequency raising | |
| CN105144681B (en) | The details in image is created with frequency raising | |
| KR102327972B1 (en) | Projection image construction method and device | |
| CN111652830A (en) | Image processing method and device, computer readable medium and terminal equipment | |
| US11062432B2 (en) | Method and device for reconstructing an HDR image | |
| CN110996170A (en) | Video file playing method and related equipment | |
| CN105744357A (en) | Method for reducing network video bandwidth occupation based on online resolution improvement | |
| Jo et al. | Efficient Computation Offloading in Mobile Cloud Computing for Video Streaming Over 5G. | |
| CN110827380A (en) | Image rendering method and device, electronic equipment and computer readable medium | |
| Yang et al. | Fovr: Attention-based vr streaming through bandwidth-limited wireless networks | |
| CN103747269B (en) | A kind of wave filter interpolation method and wave filter | |
| CN102801968B (en) | Rapid intelligent image/video retargeting method | |
| CN101511011A (en) | Display method and device for image drop sampling quick decode | |
| CN110390636A (en) | Unmanned plane is super, the simulation Zooming method of high definition picture or video data | |
| CN102111636A (en) | Method and system for converting three-dimensional (3D) signal into two-dimensional (2D) signal | |
| CN103747251B (en) | The interpolation optimization method and wave filter of a kind of video image | |
| CN114359100A (en) | Image color enhancement method and device, storage medium and electronic equipment | |
| CN101902654A (en) | Video scaling control method applied to multi-cannel video decoding chip | |
| WO2021000650A1 (en) | Program distribution method and device, reception method, terminal apparatus, and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |