WO2023142370A1

WO2023142370A1 - Screen mirroring display method and apparatus, and electronic device and readable storage medium

Info

Publication number: WO2023142370A1
Application number: PCT/CN2022/102415
Authority: WO
Inventors: 李宾; 周强
Original assignee: 深圳创维－Rgb电子有限公司
Priority date: 2022-01-30
Filing date: 2022-06-29
Publication date: 2023-08-03
Also published as: CN114463359A

Abstract

A screen mirroring display method and apparatus, and an electronic device and a readable storage medium, which relate to the technical field of screen mirroring. The screen mirroring display method comprises: acquiring screen mirroring data, wherein the screen mirroring data comprises image data; performing black-edge recognition on the image data, clipping the image data according to a black-edge recognition result, and acquiring image data which has been subjected to clipping; extracting additional information from the image data according to a preset extraction model; and integrating the additional information into the image data which has been subjected to clipping, and acquiring integrated image data. Lost additional information is displayed on a display device again, such that the information integrity of image data is ensured, and information loss caused by clipping is avoided, thereby improving the viewing experience of a user.

Description

Screen projection display method, device, electronic device and readable storage medium

related application

This application claims the priority of the Chinese patent application with application number 202210114218.4 and titled "screen projection display method, device, electronic equipment and readable storage medium" submitted to the China Patent Office on January 30, 2022.

technical field

The present application relates to the technical field of screen projection, and in particular to a screen projection display method, device, and readable storage medium.

Background technique

With the changes of the times, there are more and more demands to use the large screen on the TV side as a display or projection device. And with the continuous innovation and upgrading of mobile device functions, users have higher and higher requirements for the interaction between multiple devices, and the large-screen TV end has a better viewing experience than the mobile end, so the layout of the mobile end page can be The demand for large screens is getting stronger and stronger. However, when the screen on the mobile terminal is rearranged to a large-screen device, other information about the screen may be lost, resulting in the user being unable to obtain the core information of the screen and affecting the user's viewing experience.

Therefore, there is an urgent need for a method that can rearrange the screen on the mobile terminal to the large screen without losing relevant information of the screen.

The above content is only used to assist in understanding the technical solution of the present application, and does not mean that the above content is admitted as prior art.

technical problem

The main purpose of this application is to provide a screen projection method, which aims to solve the technical problem that other information about the screen may be lost when the screen on the mobile terminal is rearranged to a large-screen device.

technical solution

In order to achieve the above purpose, the present application provides a screen projection display method, which includes the following steps:

Obtain screen projection data, where the screen projection data includes image data;

Carrying out black border recognition on the image data, cutting the image data according to the black border recognition result, and obtaining the cut image data;

Extract additional information in the image data according to a preset extraction model;

Integrating the additional information into the clipped image data to obtain the integrated image data.

In one embodiment, the preset extraction model includes preset extraction elements, and the step of extracting additional information in the image data according to the preset extraction model includes:

extracting elements corresponding to the preset extraction elements from the image data;

The extracted elements are used as the additional information.

In one embodiment, after the step of extracting additional information in the image data according to a preset extraction model, the method further includes:

Image processing is performed on the additional information, and the image processing includes one or more of color processing, outline processing, and mask processing.

In one embodiment, the step of integrating the additional information into the trimmed image data, and obtaining the integrated image data includes:

The processed additional information is placed in a preset position of the trimmed image data, so as to obtain the integrated image data.

In one embodiment, after the step of acquiring the integrated image data, it includes:

The integrated image data is displayed, and the additional information is hidden after being displayed for a preset period of time, and the additional information is displayed again when a wake-up operation is triggered.

In one embodiment, the step of performing black border recognition on the image data, and clipping the image data according to the black border recognition result includes:

Obtain a first picture and a second picture from the image data, and respectively perform black border recognition on the first picture and the second picture, and determine a first cropping position of the first picture and the second picture. The second cropping position of the screen;

A target cropping position is generated according to the first cropping position and the second cropping position, and the image data is cropped according to the target cropping position.

In one embodiment, the step of identifying black borders on the first picture and the second picture respectively includes:

Comparing the color values of the pixels of the first picture, dividing the pixels of the first picture into first comparison units, and determining the change degree of the color values of the pixels in the first comparison units to exceed a preset threshold The position of is identified as a black border and used as the first cropping position;

Comparing the pixel point color values of the second picture, dividing the pixel points of the second picture into second comparison units, and determining the change degree of the pixel point color values in the second comparison units exceeding a preset threshold The position identified as the black border line is used as the second cropping position.

In an embodiment, obtaining the first frame and the second frame from the image data is obtaining two pictures from the image data in the projection data.

Recognizing the area of the black border in the first picture and the area of the actual play picture, using the boundary line between the black border area in the first picture and the actual play picture area as the first cropping position;

Recognize the area of the black border in the second picture and the area of the actual playing picture, and use the boundary line between the black border area in the second picture and the actual playing picture area as the second cropping position;

judging whether the first cropping position and the second cropping position are the same;

In response to the fact that the first cropping position and the second cropping position are the same, directly use the first cropping position or the second cropping position as the above-mentioned target position;

In response to the fact that the first cropping position and the second cropping position are not the same, the first cropping position and the second cropping position will be combined to generate an average cropping position, and then the black border of the image data in the projection data will be adjusted according to the target cropping position Partial crop removal.

In one embodiment, the color processing is to perform color filling on the extracted text elements and graphic elements, and the filling color can be preset or determined according to the color value of pixels in the preset area where the text elements and graphic elements are extracted;

The outline processing is when the image recognition technology cannot recognize the definite text elements and graphic elements, color-processing the identified outlines as text elements or graphic elements;

The masking process adds a masked background to the additional information text elements and graphic elements.

In an embodiment, the generating a target cropping position according to the first cropping position and the second cropping position, and cropping the image data according to the target cropping position includes:

judging whether the positions of the first line in the first trimming position and the first line in the second trimming position are the same, and judging whether the positions of the second line in the first trimming position and the second line in the second trimming position are the same;

In response to the position of the first line in the first trimming position being the same as that of the first line in the second trimming position and the position of the second line in the first trimming position being the same as the position of the second line in the second trimming position, the first trimming position or the second trimming position Any one of the clipping positions is used as the target clipping position;

In response to the position of the first line in the first trimming position being different from the position of the first line in the second trimming position and the position of the second line in the first trimming position being different from the position of the second line in the second trimming position, take the The average first line of the first line and the first line in the second trimming position, the average second line of the second line in the first trimming position and the second line in the second trimming position, the target trimming position includes the average first line and The average second line is used to crop the image data according to the average first line and the average second line of the target cropping position.

In addition, in order to achieve the above purpose, the present application also provides a projection screen display device, the projection screen display device includes:

An acquisition module, configured to acquire screen projection data, where the screen projection data includes image data;

A cropping module, configured to perform black edge recognition on the image data, clip the image data according to the black edge recognition result, and obtain the cropped image data;

An extraction module, configured to extract additional information in the image data according to a preset extraction model;

An integration module, configured to integrate the additional information into the cropped image data, and obtain the integrated image data.

In one embodiment, the preset extraction model includes a preset extraction area, and the extraction module is configured to extract an area corresponding to the preset extraction area in the image data; and use the extracted area as the additional information.

In addition, in order to achieve the above object, the present application also provides an electronic device, the electronic device includes: a memory, a processor, and a screen projection program stored in the memory and operable on the processor, the When the screen projection display program is executed by the processor, the above-mentioned steps of the screen projection display method are realized.

In addition, in order to achieve the above object, the present application also provides a readable storage medium, on which a screen projection display program is stored, and when the screen projection display program is executed by a processor, the above screen projection display is realized method steps.

Beneficial effect

In the screen projection display method proposed in the embodiment of the present application, after the screen projection data is clipped, information is extracted from the image data in the screen projection data, and the extracted additional information is integrated into the cropped graphics data , so that the relevant information of the image data is complete when displayed, avoiding the loss of core information, and improving the viewing experience of the user.

Description of drawings

FIG. 1 is a schematic diagram of a screen projection connection scene in the screen projection display method of the present application;

FIG. 2 is a schematic structural diagram of electronic equipment in the hardware operating environment involved in the solution of the embodiment of the present application;

FIG. 3 is a schematic flow chart of the first embodiment of the projection screen display method of the present application;

FIG. 4 is a schematic diagram of the display effect of the first picture on the display device before cropping in the projection screen display method of the present application;

FIG. 5 is a schematic diagram of a display screen of image data on a display device in the projection screen display method of the present application;

FIG. 6 is a schematic diagram of image data after integrating additional information in the projection screen display method of the present application;

FIG. 7 is a schematic diagram of a scene of an implementable method in the projection screen display method of the present application;

FIG. 8 is a schematic structural diagram of a device involved in the projection screen display method of the present application.

The realization, functional features and advantages of the present application will be further described in conjunction with the embodiments and with reference to the accompanying drawings.

Embodiments of the present invention

It should be understood that the specific embodiments described here are only used to explain the present application, and are not intended to limit the present application.

The main solution of the embodiment of the present application is: after the screen projection data is cropped, information is extracted from the image data in the screen projection data again, and the extracted additional information is integrated into the cropped graphic data.

When the screen projection technology rearranges the screen on the mobile terminal to a large-screen device, other information about the screen may be lost, resulting in the user being unable to obtain the core information of the screen.

The present application provides a solution, so that relevant information of image data is complete when displayed, avoiding loss of core information, and improving viewing experience of users.

Referring to FIG. 1 , FIG. 1 is a schematic diagram of a screen projection connection involved in an embodiment of the present application.

As shown in Figure 1, the screen projector includes: a screen projector transmitter and a screen projector receiver, wherein the first electronic device can be a personal computer, or a smart phone, a tablet computer, an e-book reader, a portable computer and other mobile terminal devices, the second electronic device may be a television, or a display device with a display function such as a monitor or a projector.

In some embodiments, the first electronic device is connected to the screen projector transmitter or screen projector receiver through a wired or wireless network for data transmission and interaction; the screen projector transmitter is connected to the screen projector receiver through a wireless network for data transmission and interaction, or the screen projector transmitter is connected to the second electronic device through a wireless network for data transmission and interaction; the screen projector receiver is connected to the second electronic device through a wired or wireless network for data transmission and interaction. In an optional implementation, the wired connection between the screen projector receiver and the second electronic device and between the screen projector transmitter and the first electronic device may include HDMI (High Definition Multimedia Interface, USB (Universal Serial Bus, Universal Serial Bus), DP (DisplayPort, Display Interface) and VGA (Video Graphics Array, Video Graphics Array) and other wired connections. The wireless communication methods between the first electronic device and the screen projector transmitter and the screen projector receiver, and between the second electronic device and the screen projector transmitter and the screen projector receiver may include Bluetooth, wireless Broadband (Wi-Fi), infrared communication and other wireless communications.

Therefore, possible connection methods for implementing screen projection between the first electronic device, the screen projector transmitter, the screen projector receiver and the second electronic device at least include:

(1) The screen projector transmitter is connected to the screen projector receiver, and the screen of the first electronic device is finally projected to the second electronic device through the screen projector transmitter and the screen projector receiver;

(2) The transmitter of the screen projector is wirelessly connected with the second electronic device, and the screen of the first electronic device is projected to the second electronic device through the transmitter of the screen projector. At this time, the transmitter of the screen projector and the built-in wireless The communication modules are connected through a wireless network;

(3) The first electronic device is connected to the screen projector receiver, and the screen of the first electronic device is projected to the second electronic device through the screen projector receiver.

As shown in FIG. 2 , FIG. 2 is a schematic diagram of a terminal structure of a hardware operating environment involved in the solution of the embodiment of the present application.

The electronic device in this embodiment of the present application may be a screen projector, or an electronic device with functions of data receiving, data processing, and data sending, such as a TV, a mobile phone, a PC, a tablet computer, and a portable computer.

As shown in FIG. 1 , the device may include: a processor 1001 , such as a CPU, a network interface 1004 , a user interface 1003 , a memory 1005 , and a communication bus 1002 . Wherein, the communication bus 1002 is used to realize connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. In an embodiment, the network interface 1004 may include a standard wired interface or a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001 .

In an embodiment, the device may further include a camera, an RF (Radio Frequency, radio frequency) circuit, a sensor, an audio circuit, a WiFi module, and the like. Among them, sensors such as light sensors, motion sensors and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display screen according to the brightness of the ambient light, and the proximity sensor may turn off the display screen and/or backlight. As a kind of motion sensor, the gravitational acceleration sensor can detect the magnitude of acceleration in various directions (generally three axes), and can detect the magnitude and direction of gravity when it is stationary, and can be used to identify the application of mobile terminal posture (such as horizontal and vertical screen switching, Related games, magnetometer attitude calibration), vibration recognition related functions (such as pedometer, tap), etc. Of course, mobile devices can also be equipped with other sensors such as gyroscope, barometer, hygrometer, thermometer, infrared sensor, etc. Here No longer.

Those skilled in the art can understand that the device structure shown in FIG. 1 does not constitute a limitation to the device, and may include more or less components than shown in the figure, or combine some components, or arrange different components.

As shown in FIG. 2 , the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a screen projection display program.

In the terminal shown in Figure 2, the network interface 1004 is mainly used to connect to the background server and perform data communication with the background server; the user interface 1003 is mainly used to connect to the client (client) and perform data communication with the client; and the processor 1001 can be used to call the projection screen display program stored in memory 1005, and perform the following operations:

In an embodiment, the processor 1001 may invoke the screen projection display program stored in the memory 1005, and further perform the following operations:

The preset extraction model includes a preset extraction area, and the step of extracting additional information in the image data according to the preset extraction model includes:

extracting an area corresponding to the preset extraction area in the image data;

The extracted area is used as the additional information.

The preset extraction model includes preset extraction elements, and the step of extracting additional information in the image data according to the preset extraction model includes:

The extracted elements are used as the additional information.

After the step of extracting additional information in the image data according to a preset extraction model, the method further includes:

The step of integrating the additional information into the trimmed image data and obtaining the integrated image data includes:

After the step of acquiring the integrated image data, including:

The step of performing black border recognition on the image data, and cutting the image data according to the black border recognition result includes:

The step of identifying black borders on the first picture and the second picture respectively includes:

Referring to FIG. 3 , the first embodiment of the screen projection display method of the present application, the screen projection display method includes:

Step S10, acquiring screen projection data, where the screen projection data includes image data;

In this embodiment, the subject of implementation can be an intelligent electronic device participating in the screen projection process, such as a screen projection device transmitter, a screen projection device receiver, a TV, a mobile phone, etc., and this embodiment takes the screen projection device as the optimal embodiment. illustrate. The screen projector acquires screen projection data, and the screen projection data may generally include video data or several continuous pictures, etc., and the screen projection data may also include audio data. The screen projector acquires image data from the projection data. It can be understood that if the projection data is video data, the image can be intercepted, and if it is graphic data, it can be directly received.

Step S20, performing black edge recognition on the image data, cutting the image data according to the black edge recognition result, and obtaining the cut image data;

In the actual screen projection process, if the mobile phone is used as the source device of the projection data, the image data in the projection data may have a large area of black borders, which affects the viewing experience, and then black border recognition is performed on the image data.

In an embodiment, the first picture and the second picture are obtained from the image data, and black border recognition is performed on the first picture and the second picture respectively, and the first cropping of the first picture is determined position and a second cropping position of the second frame; generating a target cropping position according to the first cropping position and the second cropping position, and cropping the image data according to the target cropping position.

Specifically, when performing screen projection, the screen projection device will obtain two pictures from the image data in the screen projection data, that is, the first picture and the second picture. Identify the area with black borders in the first picture and the area of the actual playback screen, use the boundary between the black border area in the first picture and the area of the actual playback screen as the first cropping position, and identify the area with black borders in the second picture and the actual playback area In the area of the screen, the boundary line between the black border area in the second screen and the actual playing screen area is used as the second cropping position. Determine whether the first cropping position and the second cropping position are the same, if they are the same, directly use the first cropping position or the second cropping position as the above target position, if not, combine the first cropping position and the second cropping position to generate Average crop position. Then, according to the target cropping position, the black edge part of the image data in the projection data is cropped and removed.

In one embodiment, the color values of the pixels in the first picture are compared, the pixels in the first picture are divided into first comparison units, and the color values of the pixels in the first comparison units are divided into A position whose degree of change exceeds a preset threshold is identified as a black border and used as the first cropping position; comparing the pixel color values of the second picture, and dividing the pixel points of the second picture into a second comparison A unit for identifying a position where the change degree of the color value of the pixel in the second comparison unit exceeds a preset threshold as a black boundary line and as the second cropping position.

Specifically, the first picture is taken as an example for illustration, and the processing process of the second picture is similar to that of the first picture, and details will not be described later. The first picture is composed of a large number of pixels. In this embodiment, when comparing the color value (RGB, color value, such as white color value: (255, 255, 255)) of the pixels, the pixels of the first picture Points are divided into different first comparison units according to different division rules (for example, if a row of pixels is used as a comparison unit, it is the first row comparison unit; if a column of pixels is used as a comparison unit, it is the first column comparison unit). comparison unit), compare the pixel color values of each comparison unit, for example, compare unit a with comparison unit b, if the difference between the two color values exceeds the preset threshold, then compare a The unit is used as the first cropping position, or the comparison unit b is used as the first cropping position, or the position between the comparison unit a and the comparison unit b is used as the first cropping position. It can be understood that, when performing the first comparison unit comparison on the first picture, the first comparison unit may be the first row comparison unit, the first column comparison unit, or the two comparison units respectively once. That is, when divided into row comparison units, the black borders distributed on the upper and lower sides of the screen can be identified; when divided into column comparison units, the black borders distributed on the left and right sides of the screen can be recognized, and the two comparison units are compared separately Once, you can recognize the black borders distributed on the four sides of the screen.

Take the case where there are only upper and lower black borders in the first picture as an example, and the first comparison unit is a line comparison unit, and the current comparison unit is obtained according to the first preset comparison order; the current comparison unit’s The pixel point color value is compared with the pixel point color value of the adjacent comparison unit of the current comparison unit; if the pixel point color value of the current comparison unit is compared with the pixel point color value of the adjacent comparison unit If the number of different values exceeds the preset threshold, the position between the current comparison unit and the adjacent comparison unit is used as the first cropping position; if the pixel point color of the current comparison unit is If the value differs from the color value of the pixel point of the adjacent comparison unit and does not exceed the preset threshold, the current comparison unit is reacquired until the pixel points of the first frame are traversed. Wherein, the first preset comparison order includes: an order from the upper and lower edges of the screen to the middle of the screen; or, an order from the middle of the screen to the upper and lower edges of the screen.

If the acquired first picture includes Area 1 and Area 2 as shown in Figure 4, where Figure 4 shows the display effect of the first picture on the display device before cropping, and Area 3 is the adaptive display of the first picture by the display device A black border is generated, the first line a and the second line b are the dividing line between area 1 and area 2, and area 1 is the black border and area 2 is the actual playback screen. The first comparison unit is a line comparison unit. According to the first preset comparison sequence, the pixel points of a certain row are obtained from the first picture as the current comparison row, and the color of each pixel point of the current comparison row is further extracted. Value, compare it with the color value of the pixels in the adjacent row of the current comparison row. When the ratio of the number of pixels whose color values are different from those of adjacent rows in the current comparison row to the total number of pixels in a row exceeds the preset ratio threshold, the interval line between the current comparison row and the adjacent row is used as above crop line. For example, if the total number of pixels in a row is 100, it corresponds to 100 color values. If the preset ratio threshold is 80%, then when there are 81 color values that are different between the current comparison row and the adjacent row (default The proportion threshold can be set according to the actual situation), that is, the first cropping position can be determined according to the position of the current comparison row and the position of the adjacent row, such as the first line a and the second line b in Figure 4. If the ratio of the number of pixels whose color values are different from those of adjacent rows in the current comparison row to the total number of pixels in a row does not exceed the preset ratio threshold, the current comparison sequence is reacquired according to the preset first comparison sequence. The comparison row performs pixel point color value comparison with the adjacent row of the current comparison row at this time. All pixels up to the first frame are compared in a row manner.

Correspondingly, generate the first cropping position according to the first frame, generate the second cropping position according to the second frame, and judge whether the first cropping position and the second cropping position are the same. Whether the position of the first line a in the second cutting position is the same, judge whether the position of the second line b in the first cutting position is the same as that of the second line b in the second cutting position. If they are the same, any one of the first cropping position or the second cropping position is used as the target cropping position. If not the same, take the average first line a of the first line a in the first cutting position and the first line a in the second cutting position, and take the average of the second line b in the first cutting position and the second line b in the second cutting position The average second line b, then the target cropping position includes the average first line a and the average second line b, and the image data is cropped according to the average first line a and the average second line b of the target cropping position, that is, if the image data includes As shown in Area 1 and Area 2, remove Area 1 (black border area) and leave only Area 2 (actual play area). It can be understood that the reserved actual playback area is the cropped image data.

In this embodiment, the specific number of images to be acquired, the classification of comparison units, and the comparison order can all be set according to actual conditions, and will not be repeated here.

Step S30, extracting additional information in the image data according to a preset extraction model;

In an embodiment, the preset extraction model includes a preset extraction area, and an area corresponding to the preset extraction area is extracted from the image data; the extracted area is used as the additional information.

It is understandable that after the above cropping steps, the actually cropped part of the image data may contain information related to the image data, for example, the title, author, built-in icon, introduction and other information of the image data will be lost . The preset extraction model will extract the above missing information.

Specifically, the preset extraction model may randomly select one of the first picture and the second picture as an extraction object, and may also re-acquire the picture from the image data. As shown in Figure 5, it is the image data on the display screen of the display device, including the playback area and the black border area. The playback area displays image data, and the preset 1 area and the preset 2 area in the playback area are the preset extraction models. The default extraction area for . It can be understood that the preset area 1 and the preset area 2 shown in Figure 5 do not impose specific restrictions on the position, quantity and size of the preset extraction area, and the position, number and size of the preset area can be set according to the actual situation . The pictures of the preset area 1 and the preset area 2 are further extracted and used as additional information.

In an embodiment, the preset extraction model includes preset extraction elements, elements corresponding to the preset extraction elements are extracted from the image data, and the extracted elements are used as the additional information.

The above preset extraction elements may include text elements and graphic elements. Based on the above example, the preset extraction model extracts elements from the playback area in Figure 5. The extraction of specific text elements and graphic elements can be extracted using image recognition technology, and the extracted The obtained text elements and graphic elements are used as additional information of the cropped image data.

In an embodiment, image processing is performed on the additional information, and the image processing includes one or more of color processing, outline processing, and mask processing.

Specifically, the additional information may be the aforementioned preset 1 and preset 2 areas, or text elements and graphic elements, etc., and the image processing includes one or more of color processing, outline processing, and mask processing. Such as color processing, the extracted text elements and graphic elements are filled with color, the filling color can be set in advance, and can also be determined according to the pixel color value in the preset area where the text elements and graphic elements are extracted. Contour processing is that if the image recognition technology cannot recognize certain text elements and graphic elements, the directly recognized contours are color-processed and used as text elements or graphic elements. Mask processing, adding a mask background to additional information text elements and graphic elements, such as setting a fixed color or a transparent mask, to prevent text elements and graphic elements from being distinguished from the actual playback area and affecting the user's viewing effect.

S40. Integrate the additional information into the cropped image data, and acquire the integrated image data.

In one embodiment, the processed additional information is placed in a preset position of the cropped image data, so as to obtain the integrated image data.

Specifically, the additional information is processed through the above steps, and will be placed in the preset position in the cropped image data, as shown in Figure 6, which is a schematic diagram of the image data after integrating the additional information, including the title text area, profile information area and icon area, the reserved image area is the area reserved after the image data is cut, the title text area, brief information area and icon area are the preset positions of the above-mentioned image data after cutting, if the additional information is in the text element and Graphical elements, the text elements should be placed in the title text area and the brief information area, and the graphic elements should be placed in the icon area. If the additional information is the above-mentioned preset area 1 or preset The frame of area 2 is placed at the preset position of the trimmed image data. It can be understood that, the title text area, brief information area and icon area shown in FIG. 6 do not limit the size, quantity and area of the preset positions, and can be set according to actual conditions.

In one embodiment, after the step of acquiring the integrated image data, it includes: displaying the integrated image data, and hiding the additional information after displaying for a preset duration, and triggering The additional information is redisplayed upon wakeup operation.

Specifically, when the integrated image data is displayed, the additional information will block the actual video content to a certain extent, especially when the video time is long, which will affect the user's actual viewing experience. In this regard, after the additional information is displayed for a certain period of time (such as 1 minute), the additional information is hidden, so that the actual playback content is completely displayed, improving the user's viewing experience. And when the user performs a wake-up operation, the additional information is redisplayed, such as the user performs operations such as pause, fast forward, and double-speed playback on the mobile phone, screen projector or television, and the additional information is redisplayed.

It can be understood that, in this embodiment, after the cropping operation is performed on the screen projection data, information is extracted from the image data in the screen projection data again, and the extracted additional information is integrated into the cropped graphic data, so that the image The relevant information is complete when the data is displayed, which avoids the loss of core information and improves the user's viewing experience.

In an implementable manner, referring to FIG. 7, the actual screen projection scene includes: a mobile phone 100, a screen projector transmitter 200, a screen projector receiver 300, and a television set 400. The mobile phone 100 and the projector The screen projector transmitter 200 is in communication connection, the screen projector receiver 300 is in communication connection with the screen projector transmitter 200 , and the screen projector receiver 300 is in communication connection with the TV set 400 . During screen projection, the screen projection device transmitter 200 obtains the projection data pushed by the mobile phone 100, cuts the image data in the projection data, and generates attachments and information based on the image data before cutting, and then integrates the generated attachments and information The cropped image data is sent to the screen projector receiver 300, and the screen projector receiver 300 pushes the integrated image data to the TV 400 for display.

In addition, this embodiment also provides a projection screen display device, as shown in FIG. 8 , the projection screen display device includes:

An acquisition module 10, configured to acquire screen projection data, where the screen projection data includes image data;

A cropping module 20, configured to perform black border recognition on the image data, clip the image data according to the black border recognition result, and obtain the cropped image data;

An extraction module 30, configured to extract additional information in the image data according to a preset extraction model;

The integration module 40 is configured to integrate the additional information into the cropped image data, and obtain the integrated image data.

In an embodiment, the preset extraction model includes a preset extraction area, and the extraction module 30 is further configured to:

The extracted area is used as the additional information.

In an embodiment, the preset extraction model includes preset extraction elements, and the extraction module 30 is further configured to:

The extracted elements are used as the additional information.

In one embodiment, the extracting module 30 is also used for:

In one embodiment, the integration module 40 is also used for:

In one embodiment, the clipping module 20 is also used for:

The screen projection display device provided by the present application adopts the screen projection method in the above-mentioned embodiments, and solves the technical problem that other information about the screen may be lost when the screen on the mobile terminal is rearranged to a large-screen device. Compared with the prior art, the beneficial effect of the screen projection display device provided by the embodiment of the present application is the same as that of the screen projection method provided by the above embodiment, and other technical features of the screen projection display device are the same as those of the above embodiment method The disclosed features are the same and will not be repeated here.

In addition, this embodiment also provides an electronic device, the electronic device includes: a memory, a processor, and a screen projection display program stored in the memory and operable on the processor, the screen projection display When the program is executed by the processor, the steps of the above screen projection method are realized.

The specific implementation manners of the electronic device of the present application are basically the same as the above-mentioned embodiments of the screen projection display method, and will not be repeated here.

In addition, this embodiment also provides a readable storage medium, the readable storage medium stores a screen projection display program, and when the screen projection display program is executed by a processor, the steps of the above screen projection display method are implemented .

The specific implementation manner of the medium of the present application is basically the same as that of the embodiments of the above-mentioned screen projection display method, and will not be repeated here.

It should be noted that, as used herein, the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article or system comprising a set of elements includes not only those elements, It also includes other elements not expressly listed, or elements inherent in the process, method, article, or system. Without further limitations, an element defined by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, article or system comprising that element.

The serial numbers of the above embodiments of the present application are for description only, and do not represent the advantages and disadvantages of the embodiments.

Through the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware, but in many cases the former is better implementation. Based on such an understanding, the technical solution of the present application can be embodied in the form of a software product in essence or the part that contributes to the prior art, and the computer software product is stored in a storage medium (such as ROM/RAM) as described above. , magnetic disk, optical disk), including several instructions to make a terminal device (which may be a mobile phone, computer, server, screen projector, or network device, etc.) execute the methods described in various embodiments of the present application.

The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structures or equivalent process transformations made by using the description of the application and the accompanying drawings are directly or indirectly used in other related technical fields. , are all included in the patent protection scope of the present application in the same way.

Claims

A screen projection display method, wherein the projection screen display method includes the following steps:

Obtain screen projection data, where the screen projection data includes image data;

Carrying out black border recognition on the image data, cutting the image data according to the black border recognition result, and obtaining the cut image data;

Extract additional information in the image data according to a preset extraction model;

Integrating the additional information into the clipped image data to obtain the integrated image data.
The screen projection display method according to claim 1, wherein the preset extraction model includes a preset extraction area, and the step of extracting additional information in the image data according to the preset extraction model includes:

extracting an area corresponding to the preset extraction area in the image data;

The extracted area is used as the additional information.
The screen projection display method according to claim 1, wherein the preset extraction model includes preset extraction elements, and the step of extracting additional information in the image data according to the preset extraction model includes:

extracting elements corresponding to the preset extraction elements from the image data;

The extracted elements are used as the additional information.
The screen projection display method according to claim 1, wherein, after the step of extracting additional information in the image data according to a preset extraction model, the method further comprises:

Image processing is performed on the additional information, and the image processing includes one or more of color processing, outline processing, and mask processing.
The screen projection display method according to claim 4, wherein the step of integrating the additional information into the cropped image data, and obtaining the integrated image data comprises:

The processed additional information is placed in a preset position of the trimmed image data, so as to obtain the integrated image data.
The screen projection display method according to claim 1, wherein, after the step of acquiring the integrated image data, comprising:

The integrated image data is displayed, and the additional information is hidden after being displayed for a preset period of time, and the additional information is displayed again when a wake-up operation is triggered.
The screen projection display method according to claim 1, wherein the step of performing black border recognition on the image data, and cutting the image data according to the black border recognition result comprises:

Obtain a first picture and a second picture from the image data, and respectively perform black border recognition on the first picture and the second picture, and determine a first cropping position of the first picture and the second picture. The second cropping position of the screen;

A target cropping position is generated according to the first cropping position and the second cropping position, and the image data is cropped according to the target cropping position.
The screen projection display method according to claim 7, wherein the step of identifying black borders on the first picture and the second picture respectively comprises:

Comparing the color values of the pixels of the first picture, dividing the pixels of the first picture into first comparison units, and determining the change degree of the color values of the pixels in the first comparison units to exceed a preset threshold The position of is identified as a black border and used as the first cropping position;

Comparing the pixel point color values of the second picture, dividing the pixel points of the second picture into second comparison units, and determining the change degree of the pixel point color values in the second comparison units exceeding a preset threshold The position identified as the black border line is used as the second cropping position.
The screen projection display method according to claim 7, wherein acquiring the first picture and the second picture from the image data is acquiring two pictures from the image data in the screen projection data.
The screen projection display method according to claim 7, wherein the step of performing black border recognition on the image data, and cutting the image data according to the black border recognition result comprises:

Recognizing the area of the black border in the first picture and the area of the actual play picture, using the boundary line between the black border area in the first picture and the actual play picture area as the first cropping position;

Recognize the area of the black border in the second picture and the area of the actual playing picture, and use the boundary line between the black border area in the second picture and the actual playing picture area as the second cropping position;

judging whether the first cropping position and the second cropping position are the same;

In response to the fact that the first cropping position and the second cropping position are the same, directly use the first cropping position or the second cropping position as the above-mentioned target position;

In response to the fact that the first cropping position and the second cropping position are not the same, the first cropping position and the second cropping position will be combined to generate an average cropping position, and then the black border of the image data in the projection data will be adjusted according to the target cropping position Partial crop removal.
The screen projection display method according to claim 4, wherein the color processing is color filling the extracted text elements and graphic elements, and the filling color can be preset or according to the preset when extracting text elements and graphic elements Determine the color value of the pixel point in the area;

The outline processing is when the image recognition technology cannot recognize the definite text elements and graphic elements, color-processing the identified outlines as text elements or graphic elements;

The masking process adds a masked background to the additional information text elements and graphic elements.
The screen projection display method according to claim 7, wherein, according to the first cropping position and the second cropping position, a target cropping position is generated, and the image data is cropped according to the target cropping position ,include:

judging whether the positions of the first line in the first trimming position and the first line in the second trimming position are the same, and judging whether the positions of the second line in the first trimming position and the second line in the second trimming position are the same;

In response to the position of the first line in the first trimming position being the same as that of the first line in the second trimming position and the position of the second line in the first trimming position being the same as the position of the second line in the second trimming position, the first trimming position or the second trimming position Any one of the clipping positions is used as the target clipping position;

In response to the position of the first line in the first trimming position being different from the position of the first line in the second trimming position and the position of the second line in the first trimming position being different from the position of the second line in the second trimming position, take the The average first line of the first line and the first line in the second trimming position, the average second line of the second line in the first trimming position and the second line in the second trimming position, the target trimming position includes the average first line and The average second line is used to crop the image data according to the average first line and the average second line of the target cropping position.
A projection screen display device, wherein the projection screen display device includes:

An acquisition module, configured to acquire screen projection data, where the screen projection data includes image data;

A cropping module, configured to perform black edge recognition on the image data, clip the image data according to the black edge recognition result, and obtain the cropped image data;

An extraction module, configured to extract additional information in the image data according to a preset extraction model;

An integration module, configured to integrate the additional information into the cropped image data, and obtain the integrated image data.
The projection screen display device according to claim 13, wherein the preset extraction model includes a preset extraction area, and the extraction module is configured to extract an area corresponding to the preset extraction area in the image data; And using the extracted area as the additional information.
An electronic device, wherein the electronic device includes: a memory, a processor, and a screen projection program stored on the memory and operable on the processor, and the screen projection program is executed by the processor When executed, the steps of the screen projection display method according to any one of claims 1 to 12 are realized.