CN117389498A - Image processing method, system, device and storage medium based on screen projection - Google Patents

Image processing method, system, device and storage medium based on screen projection Download PDF

Info

Publication number
CN117389498A
CN117389498A CN202311221565.8A CN202311221565A CN117389498A CN 117389498 A CN117389498 A CN 117389498A CN 202311221565 A CN202311221565 A CN 202311221565A CN 117389498 A CN117389498 A CN 117389498A
Authority
CN
China
Prior art keywords
image
boundary
screen
column
detection
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311221565.8A
Other languages
Chinese (zh)
Inventor
曲军政
李思东
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Allwinner Technology Co Ltd
Original Assignee
Allwinner Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Allwinner Technology Co Ltd filed Critical Allwinner Technology Co Ltd
Priority to CN202311221565.8A priority Critical patent/CN117389498A/en
Publication of CN117389498A publication Critical patent/CN117389498A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1407—General aspects irrespective of display type, e.g. determination of decimal point position, display with fixed or driving decimal point, suppression of non-significant zeros
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay

Abstract

The invention discloses an image processing method, system and device based on screen projection and a storage medium. The method comprises the following steps: acquiring a first image; the first image is used for representing an image to be projected after mirror image projection coding processing; according to the height and the width of the first image, a first detection range for detecting the boundary of the first image is determined by combining a preset range of screen resolution of the first terminal; according to the first detection range, carrying out boundary detection on the first image, and determining a first boundary; the first boundary is used for representing the boundary between the image area to be displayed and the filling area in the first image; and cutting the first image according to the first boundary to obtain a cut target image, and sending the target image to a target terminal for screen projection display. According to the embodiment of the application, full-screen display under the condition of vertical screen can be realized, and the screen throwing display effect is improved. The method can be widely applied to the technical field of computers.

Description

Image processing method, system, device and storage medium based on screen projection
Technical Field
The invention relates to the technical field of computers, in particular to an image processing method, system and device based on screen projection and a storage medium.
Background
With the improvement of the hardware specification of the mobile device and the iterative upgrade of the mobile operating system, the number of multimedia scenes that can be processed by the mobile device is increasing, so that users do not meet the requirement of independently sharing multimedia content on a small screen, and are more willing to project the multimedia content on the small screen of the mobile device onto a large screen such as a television and a projector, so as to realize sharing information with other people. Data transmission between different terminals can be realized through mirror image screen (Miracast) coding processing. In general, the output resolution in the case of mirror screen-projection coding is always fixed, irrespective of whether the Source cell phone (Source cell phone) is a landscape screen or a portrait screen. And displaying a horizontal screen picture or a vertical screen picture of the mobile phone in a fixed resolution, and filling black pixels in blank places. When the Miracast is used for screen display, when the Miracast is used for screen display to a horizontal screen display, when a Source end mobile phone is used for horizontal screen or vertical screen, the display effect of a display terminal (Sink end) is good, and the user watching experience is good. However, when the Source terminal mobile phone vertical screen Miracast drops to the vertical screen display, the display image is too small to realize full screen display, and the user experience is poor.
Disclosure of Invention
The present invention aims to solve at least one of the technical problems existing in the prior art to a certain extent.
Therefore, the invention aims to provide an efficient screen-projection-based image processing method, system and device and a storage medium.
In order to achieve the technical purpose, the technical scheme adopted by the embodiment of the invention comprises the following steps:
in one aspect, an embodiment of the present invention provides a method for processing an image based on screen projection, including the following steps:
the image processing method based on screen projection, provided by the embodiment of the invention, comprises the following steps: acquiring a first image; the first image is used for representing an image to be projected after mirror image projection coding processing; according to the height and the width of the first image, a first detection range for detecting the boundary of the first image is determined by combining a preset range of screen resolution of the first terminal; the first terminal is used for representing a terminal sending the image to be projected; the first detection range is smaller than the width of the first image; according to the first detection range, carrying out boundary detection on the first image, and determining a first boundary; the first boundary is used for representing the boundary between the image area to be displayed and the filling area in the first image; and cutting the first image according to the first boundary to obtain a cut target image, and sending the target image to a target terminal for screen projection display. According to the method and the device for displaying the screen, the first boundary for image cutting is determined through the screen resolution of the first terminal and the first image, screen projection display is performed after the first image is cut, full-screen display under the condition of vertical screen is achieved, and the screen projection display effect is improved.
In addition, the image processing method based on screen projection according to the above embodiment of the present invention may further have the following additional technical features:
further, in the screen-projection-based image processing method according to the embodiment of the present invention, the preset range includes a first resolution and a second resolution, the first resolution is used for representing a left endpoint value of the preset range, and the second resolution is used for representing a right endpoint value of the preset range; the determining a first detection range for performing boundary detection on the first image according to the height and the width of the first image and combining a preset range of screen resolution of the first terminal includes:
determining a first effective width according to the height of the first image and the first resolution; the first effective width is used for representing the width of an image to be displayed in the obtained first image when the resolution of the first terminal is the first resolution;
determining a second effective width based on the height of the first image and the second resolution; the second effective width is used for representing the width of the image to be displayed in the first image obtained when the resolution of the first terminal is the second resolution;
Determining a first detection column according to the width of the first image and the first effective width;
determining a second detection column according to the width of the first image and the second effective width; the first detection column and the second detection column are used for representing two end points of the first detection range.
Further, in an embodiment of the present invention, the performing boundary detection on the first image according to the first detection range, and determining a first boundary includes:
any one first pixel point in a first detection column is acquired;
if the first pixel point is a preset color pixel, sequentially acquiring a first step in a step length queue, taking a current detection column as a starting point, taking the first step as a step length, determining the position of a next detection column, detecting the first pixel point in the next detection column until the first pixel point is a non-preset color pixel, and determining the detection column with the non-preset color pixel as a target column; the preset color pixels are used for representing pixels with the same color as the filling area;
detecting each pixel point in the target column, and determining the position corresponding to the target column as the first boundary if the number of non-preset color pixel points in the target column is greater than a first threshold; or if the non-preset color pixel point rate of the target column is greater than a second threshold, determining the position corresponding to the target column as the first boundary.
Further, in one embodiment of the present invention, the method further comprises the steps of:
if the number of the non-preset color pixels of the target column is smaller than or equal to the first threshold, detecting the next detection column of the target column by taking 1 as a step length until the number of the non-preset color pixels of the current detection column is larger than the first threshold, and determining the position corresponding to the current detection column as the first boundary;
or if the non-preset color pixel point rate of the target column is smaller than or equal to the second threshold, detecting the next detection column of the target column by taking 1 as a step length until the non-preset color pixel point rate of the current detection column is larger than the second threshold, and determining the position corresponding to the current detection column as the first boundary.
Further, in one embodiment of the present invention, the method further comprises:
receiving a first video;
acquiring a first frame image in the first video, and determining a second boundary according to the first frame image; the second boundary is used for cutting out the boundary of each frame image in the first video;
and acquiring a next frame image in the first video, and updating the second boundary according to the next frame image.
Further, in one embodiment of the present invention, the method further comprises:
if the number of the non-preset color pixel points of the first detection column is larger than a first threshold value, determining that the first image is the target image;
or if the non-preset color pixel point rate of the first detection column is greater than a second threshold value, determining that the first image is the target image.
Further, in one embodiment of the present invention, the method further comprises:
and if the target terminal is in the horizontal screen display, determining the first image as the target image.
On the other hand, an embodiment of the present invention provides an image processing system based on screen projection, including:
the first module is used for acquiring a first image; the first image is used for representing an image to be projected after mirror image projection coding processing;
the second module is used for determining a first detection range for carrying out boundary detection on the first image according to the height and the width of the first image and combining with a preset range of the screen resolution of the first terminal; the first terminal is used for representing a terminal sending the image to be projected; the first detection range is smaller than the width of the first image;
The third module is used for carrying out boundary detection on the first image according to the first detection range and determining a first boundary; the first boundary is used for representing the boundary between the image area to be displayed and the filling area in the first image;
and the fourth module is used for cutting the first image according to the first boundary to obtain a cut target image, and sending the target image to a target terminal for screen projection display.
In another aspect, an embodiment of the present invention provides an image processing apparatus based on projection, including:
at least one processor;
at least one memory for storing at least one program;
the at least one program, when executed by the at least one processor, causes the at least one processor to implement the above-described screen-projection-based image processing method.
In another aspect, an embodiment of the present invention provides a storage medium in which a processor-executable program is stored, which when executed by a processor is configured to implement the above-described screen-projection-based image processing method.
According to the method and the device for displaying the screen, the first boundary for image cutting is determined through the screen resolution of the first terminal and the first image, screen projection display is performed after the first image is cut, full-screen display under the condition of vertical screen is achieved, and the screen projection display effect is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following description is made with reference to the accompanying drawings of the embodiments of the present invention or the related technical solutions in the prior art, and it should be understood that the drawings in the following description are only for convenience and clarity of describing some embodiments in the technical solutions of the present invention, and other drawings may be obtained according to these drawings without the need of inventive labor for those skilled in the art.
FIG. 1 is an effect display diagram of an embodiment of a first image when a source terminal is a horizontal screen;
FIG. 2 is an effect display diagram of an embodiment of a first image when a source is a vertical screen;
FIG. 3 is an effect display diagram of one embodiment of a vertical screen projection to vertical screen provided by the present invention;
FIG. 4 is a schematic flow chart of an embodiment of a projection-based image processing method according to the present invention;
FIG. 5 is an effect display diagram of one embodiment of a vertical screen to horizontal screen provided by the present invention;
FIG. 6 is a schematic diagram of another embodiment of a projection-based image processing method according to the present invention;
FIG. 7 is an effect display diagram of one embodiment of a vertical screen projection to vertical screen provided by the present invention;
FIG. 8 is an effect display diagram of an embodiment of a projection-based image processing method provided by the present invention;
FIG. 9 is a schematic diagram illustrating an exemplary configuration of a projection-based image processing system according to the present invention;
fig. 10 is a schematic structural diagram of an embodiment of a projection-based image processing apparatus according to the present invention.
Detailed Description
Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative only and are not to be construed as limiting the invention. The step numbers in the following embodiments are set for convenience of illustration only, and the order between the steps is not limited in any way, and the execution order of the steps in the embodiments may be adaptively adjusted according to the understanding of those skilled in the art.
With the improvement of the hardware specification of the mobile device and the iterative upgrade of the mobile operating system, the multimedia scene which can be processed by the mobile device is increased, so that users do not only share the multimedia content on the small screen, but also can share the multimedia content on the small screen of the mobile device with others by projecting the multimedia content on a large screen such as a television and a projector. Through the external device (Sink end) with the mirror image screen projection (Miracast) function, after being connected with the mobile Source equipment, communication data of the mobile Source equipment can be decoded, and a display of the Sink end is controlled to play one path of H264 code stream format video of the mobile Source end.
In recent years, with the rise from short media videos/live broadcasts and a trend of trend, the proportion of vertical screen videos in multimedia contents watched by users is higher and higher, and in order to cater to consumers, more and more rotatable displays are presented in the market. The demands for Miracast screen throwing on a vertical screen display are increasing, however, if a direct mobile phone screen is thrown on the vertical screen display through the Miracast technology, the problem that the screen throwing to the vertical screen display is too small when the mobile phone screen is in the vertical screen direction is encountered, the mobile phone screen is not suitable for watching, and the meaning of vertically arranging the large screen display is lost. It can be appreciated that the output resolution in Miracast encoding is always fixed, independent of whether the source-side handset is a landscape or portrait screen. And displaying a horizontal screen picture and a vertical screen picture of the mobile phone in the fixed resolution, filling black pixels in blank places, and filling pixels in other colors according to requirements. Specifically, after the mirror image screen projection processing, when the mobile phone is horizontally screened, the coded picture is as shown in fig. 1, the effective picture is placed in the middle up and down, black pixels are filled up and down in the high part of the picture with insufficient resolution, and the screen projection effect is good. When the mobile phone is erected, the coded picture is centered left and right as shown in fig. 2, and black pixels are filled left and right in the width part of the picture with insufficient resolution. Therefore, when the Miracast drops to the horizontal screen display, the display effect of the Sink terminal is consistent with that of fig. 1 and 2 when the Source terminal is used for horizontally or vertically displaying the screen, and the user watching experience is good. However, when the Source terminal mobile phone is used for projecting the screen from the Miracast to the vertical screen display, the display effect is as shown in fig. 3, the display image is too small, full-screen display is not performed, and the user experience is poor.
In contrast, in the related art, an unoptimized image boundary detection algorithm is used to detect the validity of each pixel in each column of the center line of the image, and when the number of valid pixels of an image in a certain column exceeds a certain threshold value, the column is considered as an image boundary. In order to monitor whether the picture sent by the source terminal mobile phone is a horizontal screen picture or a vertical screen picture in real time, it is necessary to detect the image boundary of each frame of image and update the image boundary value used when cutting the image. If the image boundary is larger than the critical value of the judgment of the horizontal and vertical screens, the screen is considered to be the vertical screen when the source end throws the screen, and the picture needs to be cut and displayed. If the image boundary is smaller than the critical value of the judgment of the horizontal screen and the vertical screen, the image boundary is considered as the horizontal screen, and the picture does not need to be cut and displayed.
However, the above-described scheme is performed by receiving and displaying an initial image from a terminal device, the initial image including an effective image area and a peripheral area located at the periphery of the effective image area; determining the aspect ratio of Source terminal equipment and the height of an effective image area of a Sink terminal; determining the width of the effective image area according to the height and the aspect ratio of the effective image area; determining a stretching ratio according to the height of the effective image area and the height of the display device; stretching the initial image in the height direction and the width direction according to the stretching ratio to obtain a target image. The premise of this scheme is to obtain the aspect ratio of the Source terminal device, which is not available in the Miracast general scenario. In addition, the scheme cannot distinguish whether the Miracast Source terminal mobile phone is in a horizontal screen state or a vertical screen state, the horizontal screen state is that the image is not required to be cut and stretched for full screen display, and only the vertical screen state is required to cut the image and stretched for full screen display. Thus, this solution has the disadvantage of: the boundary detection operation amount of each frame of image is large, the power consumption is high, and the low delay of Miracast screen throwing is influenced. In order to detect the screen-throwing horizontal and vertical screen states, the boundary value of each frame of image is updated. When the boundary value detected by some picture scenes changes, the size of the region of the cut image changes, and the cut image is stretched to be expressed as picture jitter.
Based on the scheme, the embodiment of the invention determines the effective image area of the screen through an image edge detection algorithm, and the scheme has the following outstanding contributions:
1. reducing the image edge detection range by analyzing the main stream Miracast Source terminal equipment;
2. when the effective image judgment is carried out on the pixels in each column, the pixels are sampled at regular intervals in a sampling inspection mode, and the sampling inspection is more frequent when the sampling inspection is carried out at smaller steps when the sampling inspection is carried out closer to the central column. The final stride would drop to 1, checking the validity of each pixel. If a column of pixels finds valid pixels in the sampling process, the sampling stride of the column is immediately set to 1, and the validity of each pixel is sampled. If the effective number of pixels in the column reaches a threshold, the edge detection of the image is stopped if the boundary of the effective area of the image is considered to be detected.
3. The invention takes the average value of the image boundary values of the specific frame number detected first when the screen throwing starts as the image boundary value of the subsequent frame to prevent the detected image boundary fluctuation from causing image jitter due to the specific black image interference in the playing process.
Therefore, the embodiment of the application solves the problem of horizontal and vertical screen display of the self-adaptive Miracast Source end on the vertical screen display, adopts a single-grinding image boundary detection algorithm to cut Miracast images when the Source end is displayed in a vertical screen mode, and has the advantages of extremely small operation amount and stable images.
The following describes in detail a screen-based image processing method and system according to an embodiment of the present invention with reference to the accompanying drawings, and first, the screen-based image processing method according to the embodiment of the present invention will be described with reference to the accompanying drawings.
Referring to fig. 4, an image processing method based on screen projection is provided in the embodiment of the present invention, and the image processing method based on screen projection in the embodiment of the present invention may be applied to a terminal, a server, or software running in a terminal or a server. The terminal may be, but is not limited to, a tablet computer, a notebook computer, a desktop computer, etc. The server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, basic cloud computing services such as big data and artificial intelligent platforms. The image processing method based on screen projection in the embodiment of the invention mainly comprises the following steps:
S100: acquiring a first image; the first image is used for representing an image to be projected after mirror image projection coding processing;
s200: according to the height and the width of the first image, a first detection range for detecting the boundary of the first image is determined by combining a preset range of screen resolution of the first terminal; the first terminal is used for representing a terminal sending an image to be projected; the first detection range is smaller than the width of the first image;
s300: according to the first detection range, carrying out boundary detection on the first image, and determining a first boundary; the first boundary is used for representing the boundary between the image area to be displayed and the filling area in the first image;
s400: and cutting the first image according to the first boundary to obtain a cut target image, and sending the target image to a target terminal for screen projection display.
In some possible embodiments, for an image after screen projection processing based on mirror image, if the source end (i.e., the first terminal, for example, a mobile phone or a tablet) is displayed in a vertical screen, the target terminal after screen projection is also displayed in a vertical screen, by using the image processing method based on screen projection provided by the embodiment of the invention, the first image after screen projection processing is cut, and then the display of the target terminal is performed, so that the display effect is improved. It can be understood that the clipping logic is decided to be opened or not opened according to different screen directions according to different situations that the sink end is a horizontal screen or a vertical screen. Specifically, the image coded by Miracast is always a horizontal screen image with 1920x1080/1280x720 resolution, so that it can be known that: referring to fig. 5, when the sink end is a flat screen display, it is not necessary to cut the flat screen image transmitted from the Source end. When the sink end is a vertical screen display and the Source end is a horizontal screen display, the horizontal screen image transmitted by the Source end does not need to be cut. Referring to fig. 6, when the sink end is a vertical screen display and the Source end is a vertical screen display, a horizontal screen image transmitted from the Source end needs to be cropped. It should be noted that the sink end display (i.e., the target terminal) of the embodiment of the present application may be a large-screen display with a conventional ratio, such as a conventional resolution of a television, or a malformed display. Of course, the types of the above target terminals are examples, and the present application is not particularly limited. Likewise, corresponding to the rotary screen throwing function, the rotary screen throwing device can be used in combination with the embodiment provided by the application, and the rotary screen throwing effect is improved. Specifically, the rotation screen projection relies on a hardware module such as a GPU/G2D/decoder to rotate and display the image. When the rotating screen is a vertical screen, namely the sink end is a vertical screen display, the Source end is also a vertical screen, the GPU/G2D/decoder and other modules rotate the transverse screen image of the Miracast into a vertical screen image, and the rotated image is cut through the embodiment of the application, so that the rotating screen effect is improved.
Experiments prove that the method and the device can reduce the operation amount of image boundary detection by 99.3%, reduce the probability of image boundary fluctuation from 20% to 0%, and ensure the low delay of Miracast screen projection and the stability of the image boundary. According to the embodiment of the application, the Miracast self-adaptive display method is developed aiming at the scene, when the Source terminal mobile phone screen is projected to the screen display, the screen display can be performed in a self-adaptive mode through the method provided by the application, and the screen projection experience of a user in the screen erection mode is improved; and the operation amount of image processing and the image boundary jitter are reduced through various optimization algorithms, so that the low delay and the image stability of mirror image projection are ensured.
Optionally, in one embodiment of the present invention, the preset range includes a first resolution and a second resolution, the first resolution is used to represent a left end value of the preset range, and the second resolution is used to represent a right end value of the preset range; according to the height and width of the first image, in combination with a preset range of screen resolution of the first terminal, determining a first detection range for detecting the boundary of the first image includes:
determining a first effective width according to the height of the first image and the first resolution; the first effective width is used for representing the width of an image to be displayed in the obtained first image when the resolution of the first terminal is the first resolution;
Determining a second effective width according to the height of the first image and the second resolution; the second effective width is used for representing the width of an image to be displayed in the obtained first image when the resolution of the first terminal is the second resolution;
determining a first detection column according to the width of the first image and the first effective width;
determining a second detection column according to the width of the first image and the second effective width; the first detection column and the second detection column are used for representing two end points of the first detection range.
In some possible embodiments, since the effective area in the image encoded by the mobile phone portrait Miracast is horizontally centered (see fig. 2), the two sides are filled with black pixels (i.e., preset color pixels). Whether each pixel in each column is a black pixel may be determined from left to right in the image, and once a pixel is detected as not being an ideal network of black pixels, the column may be considered a boundary of the active image area. Illustratively, referring to fig. 7, the image boundary detection range is narrowed by analyzing the resolution aspect ratio of the mainstream Miracast Source end device. Specifically, the main stream Source end's projection device has an aspect ratio range of resolutions (i.e., first resolution) from 15:9 to 21:9 (excluding 15:9 and 21:9) and the picture is centered right and left, it is only necessary to detect the left cross-hatched area as shown in fig. 7. Of course, the specific values of the first resolution described above are exemplary, and the present application is not limited to the specific values of the first resolution and the second resolution. As shown in fig. 7, the column coordinates of the cross-hatched area (i.e., the first detection range) can be calculated as follows:
Step S201: the first resolution is 15:9, then the first effective width wmax=h×9/15; wmax represents the effective area width of the image when the source end device with the largest display aspect ratio is on screen. H represents the height of the Miracast image (i.e., the height of the first image) received by the Sink.
Step S202: second resolution 21:9, the second effective width wmin=h×9/21; wmin represents the effective area width of the image when the source end device with the smallest aspect ratio is on screen.
Step S203: bmin=w/2-Wmax/2; bmin represents the column coordinates to the left of the cross-hatched area where detection begins, i.e., the first detection column. W represents the width of Miracast image received by Sink end, namely the width of the first image.
Step S204: bmax=w/2-Wmin/2; bmax represents the column coordinates to the right of the cross-hatched area where the detection ends, i.e. the second detection column.
The column coordinate range of the cross-hatched area is [ Bmin, bmax ] obtained by the above calculation. Of course, the foregoing is exemplary, and the present application is not limited to the resolution of the first terminal.
Meanwhile, if the number of non-black pixels of a certain column is detected to exceed the threshold T (considering the case of UDP packet loss network fluctuation) within the cross-hatched area, the column is considered as the effective image boundary C. The image can be cut according to the effective image boundary and stretched to be displayed in a vertical screen and full screen. If the number of non-black pixels detected in the first column of the cross-hatched area exceeds the threshold T, the Source device is considered to be in a horizontal screen state, and the Sink device directly displays the picture transmitted by Miracast without clipping, i.e., c=0.
Optionally, in an embodiment of the present invention, performing boundary detection on the first image according to the first detection range, determining the first boundary includes:
any one first pixel point in a first detection column is acquired;
if the first pixel point is a preset color pixel, sequentially acquiring a first step in a step length queue, taking a current detection column as a starting point, taking the first step as a step length, determining the position of a next detection column, detecting the first pixel point in the next detection column until the first pixel point is a non-preset color pixel, and determining the detection column with the non-preset color pixel as a target column; the preset color pixels are used for representing pixels with the same color as the filling area;
detecting each pixel point in the target column, and determining the position corresponding to the target column as a first boundary if the number of non-preset color pixel points in the target column is greater than a first threshold; or if the non-preset color pixel point rate of the target column is greater than the second threshold, determining the position corresponding to the target column as the first boundary.
In some possible implementations, the first boundary is found first for the first image. Therefore, firstly, searching non-preset color pixels at the transverse position of the first image, and judging whether the first boundary is the first boundary according to the non-preset color pixel rate or the total number of pixels of each column detected later after finding out the non-preset color pixel columns. Specifically, when the non-preset color pixels are searched at the transverse position, the detection efficiency can be improved through a spot check mode. When judging non-black pixels of pixels in a column, a sampling inspection mode is adopted. The sampling is performed at regular intervals S by one pixel, and the sampling is performed more frequently as the step is smaller as the center column M (see fig. 6) is closer. The final step S will drop to 1 and the validity of each pixel will be checked. If a certain column of pixels find non-black effective pixels in the sampling process, the sampling step S of the column is immediately set to be 1, and the validity of each pixel in the column is sampled, so that the detection accuracy is improved. If the number of non-black pixels in the column reaches a threshold, the boundary of the image active area is considered to be detected, and the image boundary detection of the subsequent column is stopped.
In particular, the parameters in the step queue are used to characterize the step change when the detection is performed. It will be appreciated that the step between each two sense columns may be constant and may be decremented, and the application is not particularly limited.
Optionally, in one embodiment of the present invention, the method further comprises:
if the number of the non-preset color pixels of the target column is smaller than or equal to a first threshold value, detecting the next detection column of the target column by taking 1 as a step length until the number of the non-preset color pixels of the current detection column is larger than the first threshold value, and determining the position corresponding to the current detection column as a first boundary;
or if the non-preset color pixel point rate of the target column is smaller than or equal to the second threshold value, detecting the next detection column of the target column by taking 1 as a step length until the non-preset color pixel point rate of the current detection column is larger than the second threshold value, and determining the position corresponding to the current detection column as a first boundary.
In some possible embodiments, if the columns of the pixels not of the preset color appear for the first time, and do not meet the boundary requirement, the next detection column is detected in steps of 1, and the steps of the next detection column and the current detection column are 1, and when the detection columns are closer to the boundary, the steps should be smaller.
Optionally, in one embodiment of the present invention, the method further comprises:
receiving a first video;
acquiring a first frame image in a first video, and determining a second boundary according to the first frame image; the second boundary is used for cutting each frame image in the first video;
and acquiring a next frame image in the first video, and updating the second boundary according to the next frame image.
In some possible embodiments, for the video projection, the first boundary may be set to a preset value, and each frame of the subsequent video is cut according to the preset value. I.e. the switching of Source end landscape and portrait screen is monitored, the boundary value of each frame of image must be detected. The embodiment of the invention takes the average value A of the image boundary value C of the specific frame number, which is detected first when the screen projection starts, as the image boundary value C of the subsequent frame, and prevents the image jitter caused by the fluctuation of the detected image boundary C due to the interference of the specific black picture in the playing process. Of course, the embodiment of the invention can also update the second boundary in real time, thereby improving the video screen throwing effect.
Optionally, in one embodiment of the present invention, the method further comprises:
If the number of the non-preset color pixel points of the first detection column is larger than a first threshold value, determining that the first image is a target image;
or if the non-preset color pixel point rate of the first detection column is greater than the second threshold value, determining the first image as the target image.
In some possible embodiments, if the first detection column is the first boundary, the first terminal is a horizontal screen for playing, and the first image is directly transmitted to the target terminal for screen projection display without cutting.
Optionally, in one embodiment of the present invention, the method further comprises:
and if the target terminal is in the horizontal screen display, determining the first image as a target image.
In summary, by the method provided by the embodiment of the application, the operation amount of boundary detection of each frame of image is reduced by 99.3%, cpu power consumption is greatly reduced, and low delay of Miracast screen projection is ensured. The probability of image boundary fluctuation is reduced from 20% to 0%, and the cut image boundary is stable and does not fluctuate.
It can be appreciated that the embodiments of the present application narrow the image boundary detection range [ Bmin, bmax ] by analyzing the resolution aspect ratio (15:9 to 21:9) of the mainstream Miracast Source end device. The embodiment of the application also reduces the operation amount in a sampling detection mode aiming at the non-black pixel detection of a column of pixels; in the embodiment of the application, when judging the non-black pixels of the pixels in one column, a sampling inspection mode is adopted. Every other step the sampling is performed by one pixel, and the smaller the step is when approaching the center column, the more frequent the sampling is. The final stride would drop to 1, checking the validity of each pixel. If a certain column of pixels find non-black effective pixels in the sampling process, the sampling step of the column is immediately set to be 1, and the validity of each pixel in the column is sampled, so that the detection accuracy is improved. If the number of non-black pixels in the column reaches a threshold, the boundary of the image active area is considered to be detected, and the image boundary detection of the subsequent column is stopped. The embodiment of the application also detects the boundary value of each frame of image, takes the average value of the boundary values of the images of the specific frames detected first when the screen throwing starts as the boundary value of the images of the subsequent frames, and prevents the detected image boundary from fluctuating due to the specific black picture in the playing process. Referring to fig. 8, by the method provided by the embodiment of the present application, for the case that the first terminal is a vertical screen and the target terminal is a vertical screen, the image projected onto the target terminal can realize full-screen display, so that the problem that the projected image is too small and is not visible to the viewer is solved.
In summary, according to the embodiment of the application, the first boundary for cutting the image is determined through the screen resolution of the first terminal and the first image, the first image is cut and then displayed in a screen throwing mode, full-screen display under the condition of vertical screen is achieved, and the screen throwing display effect is improved.
Next, an image processing system based on projection screen according to an embodiment of the present invention will be described with reference to fig. 9.
FIG. 9 is a schematic diagram of an image processing system based on projection screen according to an embodiment of the present invention, where the system specifically includes:
a first module 910, configured to acquire a first image; the first image is used for representing an image to be projected after mirror image projection coding processing;
a second module 920, configured to determine a first detection range for performing boundary detection on the first image according to the height and the width of the first image in combination with a preset range of screen resolution of the first terminal; the first terminal is used for representing a terminal sending an image to be projected; the first detection range is smaller than the width of the first image;
a third module 930, configured to perform boundary detection on the first image according to the first detection range, and determine a first boundary; the first boundary is used for representing the boundary between the image area to be displayed and the filling area in the first image;
And a fourth module 940, configured to crop the first image according to the first boundary, obtain a cropped target image, and send the target image to the target terminal for screen projection display.
It can be seen that the content in the above method embodiment is applicable to the system embodiment, and the functions specifically implemented by the system embodiment are the same as those of the method embodiment, and the beneficial effects achieved by the method embodiment are the same as those achieved by the method embodiment.
Referring to fig. 10, an embodiment of the present invention provides an image processing apparatus based on projection, including:
at least one processor 410;
at least one memory 420 for storing at least one program;
the at least one program, when executed by the at least one processor 410, causes the at least one processor 410 to implement the screen-shot-based image processing method.
Similarly, the content in the above method embodiment is applicable to the embodiment of the present device, and the functions specifically implemented by the embodiment of the present device are the same as those of the embodiment of the above method, and the beneficial effects achieved by the embodiment of the above method are the same as those achieved by the embodiment of the above method.
The embodiment of the invention also provides a computer readable storage medium, in which a program executable by a processor is stored, the program executable by the processor is used for executing the above-mentioned image processing method based on screen projection when being executed by the processor.
Similarly, the content in the above method embodiment is applicable to the present storage medium embodiment, and the specific functions of the present storage medium embodiment are the same as those of the above method embodiment, and the achieved beneficial effects are the same as those of the above method embodiment.
In some alternative embodiments, the functions/acts noted in the block diagrams may occur out of the order noted in the operational illustrations. For example, two blocks shown in succession may in fact be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality/acts involved. Furthermore, the embodiments presented and described in the flowcharts of the present invention are provided by way of example in order to provide a more thorough understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed, and in which sub-operations described as part of a larger operation are performed independently.
Furthermore, while the invention is described in the context of functional modules, it should be appreciated that, unless otherwise indicated, one or more of the functions and/or features may be integrated in a single physical device and/or software module or may be implemented in separate physical devices or software modules. It will also be appreciated that a detailed discussion of the actual implementation of each module is not necessary to an understanding of the present invention. Rather, the actual implementation of the various functional modules in the apparatus disclosed herein will be apparent to those skilled in the art from consideration of their attributes, functions and internal relationships. Accordingly, one of ordinary skill in the art can implement the invention as set forth in the claims without undue experimentation. It is also to be understood that the specific concepts disclosed are merely illustrative and are not intended to be limiting upon the scope of the invention, which is to be defined in the appended claims and their full scope of equivalents.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in the form of a software product stored in a storage medium, including several programs for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
Logic and/or steps represented in the flowcharts or otherwise described herein, e.g., a ordered listing of executable programs for implementing logical functions, can be embodied in any computer-readable medium for use by or in connection with a program execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the programs from the program execution system, apparatus, or device and execute the programs. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the program execution system, apparatus, or device.
More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program is printed, as the program may be electronically captured, via, for instance, optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.
It is to be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable program execution system. For example, if implemented in hardware, as in another embodiment, may be implemented using any one or combination of the following techniques, as is well known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.
In the foregoing description of the present specification, reference has been made to the terms "one embodiment/example", "another embodiment/example", "certain embodiments/examples", and the like, means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
While embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that: many changes, modifications, substitutions and variations may be made to the embodiments without departing from the spirit and principles of the invention, the scope of which is defined by the claims and their equivalents.
While the preferred embodiment of the present invention has been described in detail, the present invention is not limited to the embodiments described above, and various equivalent modifications and substitutions can be made by those skilled in the art without departing from the spirit of the present invention, and these equivalent modifications and substitutions are intended to be included in the scope of the present invention as defined in the appended claims.

Claims (10)

1. An image processing method based on screen projection is characterized by comprising the following steps:
acquiring a first image; the first image is used for representing an image to be projected after mirror image projection coding processing;
according to the height and the width of the first image, a first detection range for detecting the boundary of the first image is determined by combining a preset range of screen resolution of the first terminal; the first terminal is used for representing a terminal sending the image to be projected; the first detection range is smaller than the width of the first image;
according to the first detection range, carrying out boundary detection on the first image, and determining a first boundary; the first boundary is used for representing the boundary between the image area to be displayed and the filling area in the first image;
and cutting the first image according to the first boundary to obtain a cut target image, and sending the target image to a target terminal for screen projection display.
2. The screen-based image processing method according to claim 1, wherein the preset range includes a first resolution for characterizing a left end point value of the preset range and a second resolution for characterizing a right end point value of the preset range; the determining a first detection range for performing boundary detection on the first image according to the height and the width of the first image and combining a preset range of screen resolution of the first terminal includes:
Determining a first effective width according to the height of the first image and the first resolution; the first effective width is used for representing the width of an image to be displayed in the obtained first image when the resolution of the first terminal is the first resolution;
determining a second effective width based on the height of the first image and the second resolution; the second effective width is used for representing the width of the image to be displayed in the first image obtained when the resolution of the first terminal is the second resolution;
determining a first detection column according to the width of the first image and the first effective width;
determining a second detection column according to the width of the first image and the second effective width; the first detection column and the second detection column are used for representing two end points of the first detection range.
3. The method for projection-based image processing according to claim 2, wherein the performing boundary detection on the first image according to the first detection range, and determining a first boundary, includes:
any one first pixel point in a first detection column is acquired;
If the first pixel point is a preset color pixel, sequentially acquiring a first step in a step length queue, taking a current detection column as a starting point, taking the first step as a step length, determining the position of a next detection column, detecting the first pixel point in the next detection column until the first pixel point is a non-preset color pixel, and determining the detection column with the non-preset color pixel as a target column; the preset color pixels are used for representing pixels with the same color as the filling area;
detecting each pixel point in the target column, and determining the position corresponding to the target column as the first boundary if the number of non-preset color pixel points in the target column is greater than a first threshold; or if the non-preset color pixel point rate of the target column is greater than a second threshold, determining the position corresponding to the target column as the first boundary.
4. A projection-based image processing method according to claim 3, characterized in that the method further comprises the steps of:
if the number of the non-preset color pixels of the target column is smaller than or equal to the first threshold, detecting the next detection column of the target column by taking 1 as a step length until the number of the non-preset color pixels of the current detection column is larger than the first threshold, and determining the position corresponding to the current detection column as the first boundary;
Or if the non-preset color pixel point rate of the target column is smaller than or equal to the second threshold, detecting the next detection column of the target column by taking 1 as a step length until the non-preset color pixel point rate of the current detection column is larger than the second threshold, and determining the position corresponding to the current detection column as the first boundary.
5. The projection-based image processing method of claim 1, further comprising:
receiving a first video;
acquiring a first frame image in the first video, and determining a second boundary according to the first frame image; the second boundary is used for cutting out the boundary of each frame image in the first video;
and acquiring a next frame image in the first video, and updating the second boundary according to the next frame image.
6. The projection-based image processing method of claim 2, further comprising:
if the number of the non-preset color pixel points of the first detection column is larger than a first threshold value, determining that the first image is the target image;
or if the non-preset color pixel point rate of the first detection column is greater than a second threshold value, determining that the first image is the target image.
7. The projection-based image processing method of claim 1, further comprising:
and if the target terminal is in the horizontal screen display, determining the first image as the target image.
8. An image processing system based on projection screen, comprising:
the first module is used for acquiring a first image; the first image is used for representing an image to be projected after mirror image projection coding processing;
the second module is used for determining a first detection range for carrying out boundary detection on the first image according to the height and the width of the first image and combining with a preset range of the screen resolution of the first terminal; the first terminal is used for representing a terminal sending the image to be projected; the first detection range is smaller than the width of the first image;
the third module is used for carrying out boundary detection on the first image according to the first detection range and determining a first boundary; the first boundary is used for representing the boundary between the image area to be displayed and the filling area in the first image;
and the fourth module is used for cutting the first image according to the first boundary to obtain a cut target image, and sending the target image to a target terminal for screen projection display.
9. An image processing apparatus based on projection screen, comprising:
at least one processor;
at least one memory for storing at least one program;
the at least one program, when executed by the at least one processor, causes the at least one processor to implement the projection-based image processing method of any one of claims 1 to 7.
10. A computer-readable storage medium, in which a processor-executable program is stored, characterized in that the processor-executable program is for implementing the projection-based image processing method according to any one of claims 1 to 7 when being executed by a processor.
CN202311221565.8A 2023-09-20 2023-09-20 Image processing method, system, device and storage medium based on screen projection Pending CN117389498A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311221565.8A CN117389498A (en) 2023-09-20 2023-09-20 Image processing method, system, device and storage medium based on screen projection

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311221565.8A CN117389498A (en) 2023-09-20 2023-09-20 Image processing method, system, device and storage medium based on screen projection

Publications (1)

Publication Number Publication Date
CN117389498A true CN117389498A (en) 2024-01-12

Family

ID=89438168

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311221565.8A Pending CN117389498A (en) 2023-09-20 2023-09-20 Image processing method, system, device and storage medium based on screen projection

Country Status (1)

Country Link
CN (1) CN117389498A (en)

Similar Documents

Publication Publication Date Title
US9153207B2 (en) Utilizing scrolling detection for screen content encoding
US8923611B2 (en) Automatic background identification in video images
US9396519B2 (en) Content aware video resizing
US20030177255A1 (en) Encoding and decoding system for transmitting streaming video data to wireless computing devices
US10560753B2 (en) Method and system for image alteration
US11438645B2 (en) Media information processing method, related device, and computer storage medium
CN110582012B (en) Video switching method, video processing device and storage medium
US11412311B2 (en) Methods and systems for saving data while streaming video
US11575894B2 (en) Viewport-based transcoding for immersive visual streams
CN112399123A (en) Video definition adjusting method and device, electronic equipment and storage medium
US10462200B2 (en) System for cloud streaming service, method for still image-based cloud streaming service and apparatus therefor
Ko et al. Implementation and evaluation of fast mobile VNC systems
CN117389498A (en) Image processing method, system, device and storage medium based on screen projection
CN110636372A (en) Video decoding method, video decoder initialization method and device
US20220086470A1 (en) Transcoding ultra-high-definition panoramic videos
Reznik User-adaptive mobile video streaming using MPEG-DASH
CN111885417B (en) VR video playing method, device, equipment and storage medium
CN114422866A (en) Video processing method and device, electronic equipment and storage medium
US10986337B2 (en) Systems and methods for selective transmission of media content
CN114222188A (en) Full-screen display method, device and equipment based on rotary screen and storage medium
CN108989905B (en) Media stream control method and device, computing equipment and storage medium
US20240137606A1 (en) Electronic apparatus, server apparatus and control method thereof
Lee et al. Fast modified region detection for mobile VNC systems
CN115529451A (en) Data transmission method and device, storage medium and electronic equipment
CN117412064A (en) Video decoding method, device, electronic equipment and computer storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination