CN110928509A - Display control method, display control device, storage medium, and communication terminal - Google Patents
Display control method, display control device, storage medium, and communication terminal Download PDFInfo
- Publication number
- CN110928509A CN110928509A CN201911067160.7A CN201911067160A CN110928509A CN 110928509 A CN110928509 A CN 110928509A CN 201911067160 A CN201911067160 A CN 201911067160A CN 110928509 A CN110928509 A CN 110928509A
- Authority
- CN
- China
- Prior art keywords
- display
- picture
- control operation
- control
- preview
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
- G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
- G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
- G06F3/1462—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay with means for detecting differences between the image stored in the host and the images displayed on the remote displays
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Abstract
The present disclosure relates to the field of data processing technologies, and in particular, to a display control method and apparatus, a storage medium, and a communication terminal. The method comprises the following steps: establishing communication connection with a display terminal; acquiring a display picture of the display terminal in real time through a camera module of the mobile terminal, and displaying a preview picture corresponding to the display picture; and responding to a first control operation on the preview picture, mapping the first control operation into a second control operation on the display terminal, and sending the second control operation to the display terminal to control the display picture. The display control method and the display control device can realize control of the display picture on the display terminal in a mode of operating the preview picture generated on the mobile terminal, provide a new display control mode, increase the diversity of the display control mode, and simultaneously enable the display control of the display terminal to be more convenient.
Description
Technical Field
The present disclosure relates to the field of data processing technologies, and in particular, to a display control method, a display control device, a computer-readable medium, and a communication terminal.
Background
With the advent of the information age, more and more information display forms are continuously appearing. At present, the way of presenting information by means of a display device is quite common. For example, movies, television shows, etc. may be played through a home theater in a home; the display in the computer screen can be projected by the projector to show meeting information in a meeting. In the related art, an operator usually performs an operation through a mouse, a keyboard, a remote controller, or other devices to control a display screen of a display device.
It is to be noted that the information disclosed in the above background section is only for enhancement of understanding of the background of the present disclosure, and thus may include information that does not constitute prior art known to those of ordinary skill in the art.
Disclosure of Invention
An object of the present disclosure is to provide a display control method, a display control apparatus, a computer readable medium, and a communication terminal, thereby providing a new display control manner.
Additional features and advantages of the disclosure will be set forth in the detailed description which follows, or in part will be obvious from the description, or may be learned by practice of the disclosure.
According to a first aspect of the present disclosure, there is provided a display control method applied to a mobile terminal, the method including:
establishing communication connection with a display terminal;
acquiring a display picture of the display terminal in real time through a camera module of the mobile terminal, and displaying a preview picture corresponding to the display picture;
and responding to a first control operation on the preview picture, mapping the first control operation into a second control operation on the display terminal, and sending the second control operation to the display terminal to control the display picture.
According to a second aspect of the present disclosure, there is provided a display control method applied to a display terminal, the method including:
establishing communication connection with a mobile terminal to acquire a second control operation sent by the mobile terminal; the second control operation is an operation aiming at the display terminal and obtained according to the first control operation mapping of the preview picture of the mobile terminal;
and executing the second control operation to control the display screen of the display terminal.
According to a third aspect of the present disclosure, there is provided a display control apparatus applied to a mobile terminal, the apparatus comprising:
the first connection module is used for establishing communication connection with the display terminal;
the image generation module is used for acquiring a display image of the display terminal in real time through a camera module of the mobile terminal and displaying a preview image corresponding to the display image;
and the operation conversion module is used for responding to a first control operation on the preview picture, mapping the first control operation into a second control operation on the display terminal, and sending the second control operation to the display terminal so as to control the display picture. .
According to a fourth aspect of the present disclosure, there is provided a display control apparatus applied to a display terminal, the apparatus including:
the second connection module is used for establishing communication connection with the mobile terminal so as to acquire a second control operation sent by the mobile terminal; the second control operation is an operation aiming at the display terminal and obtained according to the first control operation mapping of the preview picture of the mobile terminal;
and the display control module is used for executing the second control operation so as to control the display picture of the display terminal.
According to a fifth aspect of the present disclosure, there is provided a computer readable medium having stored thereon a computer program which, when executed by a processor, implements the above-described display control method.
According to a fourth aspect of the present disclosure, there is provided a wireless communication terminal comprising:
one or more processors;
a storage device for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to implement the above-described display control method.
In the display control method provided by an embodiment of the present disclosure, on the basis of establishing a matching connection between a mobile terminal and a display terminal, a control on a display screen on the display terminal can be implemented in a manner of mapping a first control operation performed on a preview screen displayed on the mobile terminal to a second control operation performed on the display screen of the display terminal. The technical scheme of the disclosure provides a new display control method, so that the control of the display picture can be performed through the operation of the preview picture of the mobile terminal, the diversity of display control modes is increased, and meanwhile, the display control of the display terminal can be more convenient.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure. It is to be understood that the drawings in the following description are merely exemplary of the disclosure, and that other drawings may be derived from those drawings by one of ordinary skill in the art without the exercise of inventive faculty.
Fig. 1 schematically illustrates a flow chart of a display control method in an exemplary embodiment of the present disclosure;
fig. 2 schematically illustrates a flowchart of a method of determining the preview screen and the display screen in an exemplary embodiment of the present disclosure;
fig. 3 schematically illustrates a flowchart of a method of determining whether a preview screen matches a display screen in an exemplary embodiment of the present disclosure;
fig. 4 schematically illustrates a flowchart of a method of mapping the first control operation to a second control operation for the display terminal in an exemplary embodiment of the present disclosure;
fig. 5 schematically illustrates a flowchart of a method of determining a second operation position in the display screen for a second control operation of the display terminal based on a first operation position of the first control operation in an exemplary embodiment of the present disclosure;
FIG. 6 schematically illustrates a flow chart of a method of freezing and unfreezing a preview screen in an exemplary embodiment of the present disclosure;
fig. 7 schematically illustrates a display diagram of a mobile terminal and a display terminal in an exemplary embodiment of the disclosure;
fig. 8 schematically illustrates a display diagram of another mobile terminal and display terminal in an exemplary embodiment of the present disclosure;
fig. 9 schematically illustrates a display diagram of a mobile terminal and a display terminal according to still another exemplary embodiment of the present disclosure;
FIG. 10 schematically illustrates a flow chart of a display control method in an exemplary embodiment of the disclosure;
FIG. 11 schematically illustrates a flow chart of a method of synchronized display in an exemplary embodiment of the disclosure;
fig. 12 schematically illustrates a composition diagram of a display control apparatus in an exemplary embodiment of the present disclosure;
fig. 13 schematically illustrates a composition diagram of a display control apparatus in an exemplary embodiment of the present disclosure;
fig. 14 schematically shows a structural diagram of a computer system of a wireless communication device in an exemplary embodiment of the disclosure.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
Furthermore, the drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus their repetitive description will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.
In the actual use process of the display terminal, the mode of controlling the display terminal through a keyboard, a mouse or a remote controller and other devices in the related art is often influenced by space factors in many scenes, so that the problems of low control efficiency and low accuracy rate are caused. For example, when a meeting is carried out, the computer and the projector may be far away, and since the host meeting person usually needs to rely on the screen of the projector for teaching, the display screen can be controlled only by the host person walking between the projector and the computer repeatedly or by other persons near the computer. However, the first method causes a problem of low control efficiency; the second method is to cause inaccurate control due to the difference between the controller and the speaker.
In view of the above-mentioned drawbacks and deficiencies of the prior art, the exemplary embodiment provides a new display control method, which can be applied to a mobile terminal configured with a camera module, such as a mobile phone, a tablet computer, etc., for controlling a display screen of the display terminal; the display terminal may include a computer, a television, and other display terminals equipped with a display module. Referring to fig. 1, the display control method described above may include the steps of:
s110, establishing communication connection with a display terminal;
s120, acquiring a display picture of the display terminal in real time through a camera module of the mobile terminal, and displaying a preview picture corresponding to the display picture;
s130, responding to a first control operation on the preview picture, mapping the first control operation into a second control operation on the display terminal, and sending the second control operation to the display terminal to control the display picture.
In the display control method provided by the present example embodiment, the display picture on the display terminal can be controlled by operating the preview picture on the mobile terminal, a new display control method is provided, the variety of the display control method is increased, and the display control on the display terminal can be more convenient and faster.
The following describes each step of the display control method in the present exemplary embodiment in more detail with reference to the drawings and examples.
Referring to fig. 1, in step S110, a communication connection with a display terminal is established.
In this example embodiment, the mobile terminal and the display terminal may be matched and connected through wifi, bluetooth, and the like, so that the mobile terminal and the display terminal are in the same local area network to transmit data, control information, and the like. The mobile terminal is provided with a camera module which can be a camera assembly directly assembled on the mobile terminal or an independent camera assembly connected with the mobile terminal in a wired or wireless mode and the like. The camera module can be used for shooting the current environment in real time and presenting a corresponding preview picture on a human-computer interaction interface of the mobile terminal. The display terminal refers to a display terminal capable of directly controlling a display picture. For example, a display screen on a computer can be controlled by operating the computer, and thus the computer is the display terminal according to the present disclosure.
In step S120, a display image of the display terminal is obtained in real time through a camera module of the mobile terminal, and a preview image corresponding to the display image is displayed.
In the embodiment of the present invention, after the mobile terminal and the display terminal establish a matching connection, the control mode may be activated through the control instruction, and after the control mode is activated, the display screen on the display terminal is shot in real time through the camera module, and the mobile terminal presents a preview screen corresponding to the display screen. The control instruction may be a control instruction generated by a user performing an operation to activate a control mode on the mobile terminal or the display terminal.
In this example embodiment, the display picture of the display terminal obtained by the camera module in real time may be a display picture formed by directly shooting the display terminal through the camera module, or may be another device used for presenting the display picture of the display terminal and shot by the display module. For example, when the display terminal is a computer, the camera module can obtain a display picture of the computer by directly shooting a computer screen, and present a preview picture corresponding to the display picture on the mobile terminal; for another example, when the display terminal is a computer, a projector connected to the display terminal is provided for projecting the display image of the display terminal, and at this time, the image projected by the projector can be captured by the camera module to form the display image of the display terminal, so that a preview image corresponding to the display image is presented on the mobile terminal.
In this exemplary embodiment, when the mobile terminal generates a preview screen corresponding to the display screen, referring to fig. 2, the method further includes the following steps S210 to S220:
step S210, comparing the current preview image with the display image to identify whether the preview image is matched with the display image.
In this exemplary embodiment, since a problem that a preview screen generated by the mobile terminal is not matched with a display screen due to the fact that the camera module shoots other display terminals which do not establish communication connection may occur initially, it is necessary to obtain a current display screen of the display terminal while generating the preview screen, and determine whether the preview screen on the mobile terminal is matched with the display screen on the display terminal.
It should be noted that, in the case of poor network connection or transmission delay, there may be a problem that the preview screen is not matched with the display screen, and at this time, the preview screen may be compared with the display screen corresponding to the picture tag or the time tag by adding the picture tag or the time tag to the display screen, so as to avoid the problem of picture mismatch caused by delay.
Specifically, when the preview screen includes a target area for presenting a display screen, as shown in fig. 3, the comparing the current preview screen with the display screen to identify whether the preview screen and the display screen are matched includes the following steps S310 to S340:
step S310 is to identify a target area in the preview screen, and perform size conversion on a target screen included in the target area and the display screen so that the target screen and the display screen have the same size.
In the present exemplary embodiment, after the target area in which the display screen is presented is identified in the preview screen, the size of the target screen included in the target area and the size of the display screen may be changed so that the sizes of the target screen and the display screen are the same for the purpose of comparison. For example, the size of both the target screen and the display screen may be converted to 1080 pixels by 1080 pixels.
Step S320, comparing the target picture with the display picture to calculate the similarity between the target picture and the display picture.
In this exemplary embodiment, when comparing the target picture with the display picture, the similarity between the target picture and the corresponding pixel of the display picture can be calculated in a pixel-by-pixel comparison manner, and finally, the average value of the similarities of all the pixels is taken as the similarity between the target picture and the display picture; the similarity between the target picture and the display picture can be calculated by respectively extracting the features of the target picture and the display picture, comparing the features of the target picture and the display picture and calculating the similarity between the features of the target picture and the display picture; in addition, other manners for calculating the similarity of the image may also be adopted to calculate the similarity between the target picture and the display picture, which is not limited in this disclosure.
In this other exemplary embodiment, since the camera module may have a certain angle with the display screen, the shape of the target screen may have a difference from the shape of the display screen, and at this time, the shape of the target screen may be corrected according to a correction algorithm to make the shape of the target screen consistent with the shape of the display screen, and then the target screen may be compared with the display screen.
In this other exemplary embodiment, after the size conversion is performed on the target picture and the display picture, the gray scale processing may be performed on the target picture and the display picture, and then the subsequent similarity calculation step may be performed according to the gray scale value, so as to reduce the calculation amount in calculating the similarity.
Step S330, when the similarity is greater than or equal to a preset threshold, judging that the preview picture is matched with the display picture.
Step S340, when the similarity is smaller than a preset threshold, determining that the preview image is not matched with the display image.
In the present exemplary embodiment, when the calculated similarity is greater than or equal to the preset threshold, it is determined that the preview screen matches the display screen; otherwise, when the calculated similarity is smaller than the preset threshold value, the preview picture is judged to be matched with the display picture. The preset threshold value can be set according to different definitions of preview images acquired by different camera modules, and can also be set according to the change frequency of the display images, and the disclosure does not specially limit the preset threshold value.
Continuing to refer to fig. 2, in step S220, when it is recognized that the preview screen does not match the display screen, a prompt message is displayed on the preview screen for adjusting the preview screen to match the display screen.
In the embodiment, when the preview picture is not matched with the display picture, a prompt message can be generated and displayed on the preview picture so as to prompt a user that the current preview picture does not contain a controllable display picture, and the user can adjust the camera module according to the prompt until the preview picture generated on the mobile terminal is matched with the display picture.
In step S130, in response to a first control operation on the preview screen, the first control operation is mapped to a second control operation on the display terminal, and the second control operation is sent to the display terminal to control the display screen.
In this exemplary embodiment, the control operation includes an operation position and an operation type, and as shown in fig. 4, mapping the first control operation to a second control operation for the display terminal includes the following steps S410 to S430:
step S410, determining a second operation position of a second control operation aiming at the display terminal in the display screen according to the first operation position of the first control operation.
In this exemplary embodiment, when the preview screen includes a target area in which the display screen is presented, referring to fig. 5, the determining a second operation position in the display screen of a second control operation for the display terminal based on a first operation position of the first control operation includes steps S510 to S530 as follows:
step S510, identifying a target area in the preview screen, and acquiring a target pixel corresponding to the target area in the preview screen.
In the present exemplary embodiment, a target area including a display screen is identified in a preview screen, and a pixel area of the target area in the preview screen is determined. For example, the target area is in the upper left corner of the preview screen, and the size of the target area is 40 pixels by 40 pixels, in this case, the pixel area corresponding to the target area may be determined to be a square area with (1,1), (1,40), (40,1) and (40,40) pixels as vertexes in the preview screen.
Step S520, generating a pixel mapping relationship between the target area and the display screen based on the target pixel.
In the present exemplary embodiment, after the target region is identified, the display screen is equally divided into a plurality of screen regions according to the number of pixels included in the target region in the lateral and longitudinal directions, and a one-to-one correspondence relationship is generated according to the position of the target pixel and the position of the screen region to form a pixel mapping relationship between the target pixel and the pixels included in the screen region. For example, in the above embodiment, the target area has a size of 40 pixels by 40 pixels and includes 40 pixels in the horizontal direction and the vertical direction, respectively, so that the display screen may be equally divided into 40 parts in the horizontal direction and the vertical direction, respectively, to form 160 screen areas. And finally, determining the corresponding relation between each target pixel and each display area according to the position of each pixel in the target area and the positions of 160 picture areas, for example, the target pixel (40,40) corresponds to the display area (40,40) in the display picture, and further generating a pixel mapping relation between the target pixel and the pixels contained in the corresponding display area.
Furthermore, because the camera module and the display screen may have a certain angle, the shape of the target area may have a difference from the shape of the display screen, and at this time, the target area may be corrected according to a correction algorithm, and then the step of generating the pixel mapping relationship may be performed. For example, due to the problem of the shooting angle of the camera module, the target area may be trapezoidal, and the display screen may be rectangular. At this time, the target area of the trapezoid can be corrected to be rectangular by a correction algorithm so as to generate a mapping relationship between pixels.
In addition, the attitude information of the current mobile terminal can be acquired through a sensor such as a gyroscope and the like configured on the mobile terminal and displayed on the preview picture, and a user can adjust the current attitude of the mobile terminal according to the attitude information, so that the camera module can acquire a target area with the shape consistent with that of the display picture, and the accuracy of the pixel mapping relation is improved.
Step S530, mapping the first operation position of the first control operation to a second operation position in the display screen according to the pixel mapping relationship.
In the present exemplary embodiment, after the pixel mapping relationship is determined, the first operation position of the first control operation in the preview screen may be mapped to the second operation position in the display screen, and an effect of controlling the second operation position by operating the first operation position is achieved.
In addition, the purpose of mapping the first operation position to the second operation position can be achieved by forming the mapping relation between the target picture and the display picture in other modes. For example, the display screen is filled in the target area to form a mapping relationship between the target area and each position coordinate in the display screen, thereby implementing a mapping method.
Step S420, determining a second operation type of the second control operation according to a preset mapping rule and the first operation type of the first control operation.
In this example embodiment, the preset mapping rule includes a mapping table for describing a mapping relationship between the first operation type and the second operation type. For example, when the mobile terminal is a device such as a mobile phone and a tablet computer, the first control operation type may include operation types such as clicking, long-time pressing, dragging, and the like; when the display terminal is a computer, the second operation type may include a left mouse click type, a left mouse drag type, a right mouse click type, and the like. The mapping table may be configured to describe a mapping relationship between the first operation type and the second operation type. For example, a click on a cell phone may correspond to a mouse left click on a computer. Meanwhile, when the mobile terminal and the display terminal are different devices, different settings can be performed on the mapping relationship between the operation types according to the specific devices. In addition, the preset mapping rule may also be other tools for describing the mapping relationship between the first operation type and the second operation type, and the disclosure is not limited thereto.
Step S430, generating a second control operation for the display screen based on the second operation position and the second operation type.
In the present exemplary embodiment, when the second operation position and the second operation type are determined, the second control operation for the display screen may be generated in accordance with the second operation position and the second operation type. For example, the current display frame is a video playing frame, and the user drags the progress bar in the preview frame to realize the operation of dragging the progress bar in the display frame, so as to control the display frame to display the video frame corresponding to the progress bar.
It should be noted that there is no specific sequence between step S410 and step S420 in this disclosure, and both steps may be performed simultaneously, or step S410 may be performed first and then step S420 is performed, or step S420 may be performed first and then step S410 is performed.
In the present exemplary embodiment, the target screen in the preview screen may be small due to the problem of the distance between the camera module and the display screen, and the operation may not be performed. In this case, when the preview screen matches the display screen, the method further includes: in response to a third control operation on the preview screen, controlling the preview screen to be zoomed for performing the first control operation based on the zoomed preview screen.
In this exemplary embodiment, after the preview screen is matched with the display screen, the user may zoom in or out the preview screen through the third control operation, so that the user may perform the first control operation based on the zoomed preview screen, and further control the display screen. Through when preview picture and display screen match, the mode that sets up the zoom can be convenient for the more accurate first control operation that carries on of user, avoids appearing operating position mistake scheduling problem to a certain extent.
In this exemplary embodiment, the display control method further includes: and acquiring an execution result of the second control operation on the display terminal, and calling a corresponding target application program according to the execution result.
In the present exemplary embodiment, some commonly used auxiliary applications may be required in controlling the display device. Therefore, when the display screen in the display device is controlled according to the second control operation, some application programs may be triggered, and at this time, the corresponding application program needs to be called in the mobile terminal. For example, when the second control operation triggers the input interface on the display screen, the input method program can be called in the mobile terminal to perform the input operation on the input interface.
In addition, in order to prevent the problem that the target application is interfered by the preview screen due to the movement of the camera module of the mobile terminal in the process of calling the target application, referring to fig. 6, the method further includes the following steps S610 to S620:
step S610, when the target application is called, the preview screen is frozen in the current frame.
In step S620, when the call to the target application is completed, the preview screen is unfrozen.
In the present exemplary embodiment, when the mobile terminal calls the target application, the preview screen of the mobile terminal 710 may be frozen in the current frame, and at this time, the user may operate the target application on the current frame. In this state, the display screen of the display terminal 720 can continue to be dynamically displayed, as shown with reference to fig. 7; or may be consistent with the screen of the mobile terminal 710, as shown in fig. 8; after the calling of the target application program is finished, the preview picture of the mobile terminal can be unfrozen, so that a user can conveniently perform other control operations. Through being in when the mobile terminal is in the calling state to the target application program, not only can avoid the problem that the preview picture of mobile terminal constantly changes and causes the interference to freeze the preview picture in the current frame, can also avoid simultaneously when operating the target application program, because the module of making a video recording of mobile terminal removes, causes the preview picture and shows the picture and mismatch, and then leads to the problem of control interruption.
In addition, the present exemplary embodiment also provides a display control method, which can be applied to a display terminal, such as a computer, a television, and the like. Referring to fig. 10, the display control method includes the following steps S1010 to S1020:
step S1010, establishing a communication connection with the mobile terminal to obtain a second control operation sent by the mobile terminal.
In the present exemplary embodiment, the second control operation is an operation for the display terminal obtained from the first control operation map for the preview screen of the mobile terminal. For example, when the display terminal is a computer, the first control operation may be converted into operations for the computer, such as clicking a left mouse button and clicking a right mouse button.
Step S1020, executing the second control operation for controlling a display screen of the display terminal.
In this example embodiment, after receiving the second control operation sent by the mobile terminal, the second control operation may be executed to perform corresponding control on the display screen. For example, if the operation of the second control operation is a click operation on the (30,60) pixel, the corresponding click displays the (30,60) pixel of the screen.
Further, in an example embodiment, after the establishing the communication connection with the mobile terminal, the method further comprises: and acquiring a first control operation sent by the mobile terminal, mapping the first control operation to a second control operation aiming at the display terminal, and executing the second control operation.
In this exemplary embodiment, the first control operation may be directly sent to the display terminal, the display terminal maps the first control operation for the preview screen of the mobile terminal to the second control operation for the display terminal according to the established mapping relationship, and the second control operation is executed to complete the control of the display screen.
When the mobile terminal calls the target application, the display screen of the corresponding display terminal may display the state of the target application. At this time, referring to fig. 11, the method further includes:
step S1110, sending an execution result of the second control operation to the mobile terminal, so that the mobile terminal responds to the execution result and invokes a corresponding target application program;
step S1120, synchronously displaying the display status of the target application on the preview screen.
Specifically, when the mobile terminal calls the target application program according to the execution result sent by the display terminal, the target application program is displayed in the preview screen, and the display state of the display screen in the preview screen is changed, at this time, the display state of the target application program in the mobile terminal in the preview screen can be synchronized to the display terminal. For example, as shown in fig. 9, when the mobile terminal 710 calls an input method to input in an input box, the display state of the input method in the preview screen may include an input pinyin "explaining", selectable words "camera shooting, imagine, musk, shooting, luxury", and an input word "based" in the input box. At this time, this display state may be synchronized to the display terminal 720 so that the user can see the input process through the display screen. Further, only the influence of the target application on the display screen in the preview screen may be synchronized in the display terminal. For example, in the above example, only the input result in the input box is "based on" synchronized to the display screen of the display terminal. By synchronously displaying the display state of the target application program in the preview picture on the display terminal, a user can see the control process not only on the mobile terminal, but also on the display picture, so that other users can visually see the control process of a certain user.
It is to be noted that the above-mentioned figures are only schematic illustrations of the processes involved in the method according to an exemplary embodiment of the invention, and are not intended to be limiting. It will be readily understood that the processes shown in the above figures are not intended to indicate or limit the chronological order of the processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, e.g., in multiple modules.
Further, as shown in fig. 12, in the present exemplary embodiment, there is also provided a display control apparatus 1200 including: a first connection module 1210, a screen generation module 1220, and an operation conversion module 1230. Wherein:
the first connection module 1210 may be used to establish a communication connection with a display terminal.
The image generating module 1220 may be configured to obtain a display image of the display terminal in real time through a camera module of the mobile terminal, and display a preview image corresponding to the display image.
The operation conversion module 1230 may be configured to map, in response to a first control operation on the preview screen, the first control operation to a second control operation on the display terminal, and send the second control operation to the display terminal to control the display screen.
In an example of the present disclosure, the display control module 1230 may be configured to determine a second operation position in the display screen of a second control operation for the display terminal according to the first operation position of the first control operation; determining a second operation type of the second control operation according to a preset mapping rule and the first operation type of the first control operation; generating a second control operation for the display screen based on the second operation position and the second operation type.
In an example of the present disclosure, the display control module 1230 may be configured to identify a target area in the preview screen, and acquire a corresponding target pixel of the target area in the preview screen; generating a pixel mapping relation between the target area and the display picture based on the target pixel; and mapping the first operation position of the first control operation to a second operation position in the display picture according to the pixel mapping relation.
In an example of the present disclosure, the preset mapping rule includes a mapping table for describing a mapping relationship between the first operation type and the second operation type.
In an example of the present disclosure, the screen generating module 1220 may be configured to compare the current preview screen with the display screen to identify whether the preview screen and the display screen are matched; and when the preview picture is identified to be not matched with the display picture, displaying a prompt message on the preview picture so as to adjust the preview picture to be matched with the display picture.
In an example of the present disclosure, the screen generating module 1220 may be configured to control the preview screen to be zoomed in response to a third control operation on the preview screen, so as to perform the first control operation based on the zoomed preview screen.
In an example of the present disclosure, the screen generating module 1220 may be configured to identify a target area in the preview screen, and perform size transformation on a target screen included in the target area and the display screen so that the target screen and the display screen have the same size; comparing the target picture with the display picture to calculate the similarity between the target picture and the display picture; when the similarity is larger than or equal to a preset threshold value, judging that the preview picture is matched with the display picture; or when the similarity is smaller than a preset threshold value, judging that the preview picture is not matched with the display picture.
In an example of the present disclosure, the display control module 1230 may be configured to obtain an execution result of the second control operation on the display terminal, and invoke a corresponding target application according to the execution result.
In an example of the present disclosure, the display control module 1230 may be configured to freeze the preview screen to a current frame when the target application is invoked; and unfreezing the preview picture when the calling of the target application program is finished.
Further, as shown with reference to fig. 13, an embodiment of the present example also provides a display control apparatus 1300 including: a second connection module 1310 and a display control module 1320. Wherein:
the second connection module 1310 may be configured to establish a communication connection with a mobile terminal to obtain a second control operation sent by the mobile terminal; the second control operation is an operation aiming at the display terminal and obtained according to the first control operation mapping of the preview screen of the mobile terminal.
The display control module 1320 may be configured to perform the second control operation for controlling a display screen of the display terminal.
In an example of the present disclosure, the display control module 1320 may be configured to acquire a first control operation sent by the mobile terminal, map the first control operation to a second control operation for the display terminal, and execute the second control operation.
In an example of the present disclosure, the display control module 1320 may be configured to send an execution result of the second control operation to the mobile terminal, so that the mobile terminal responds to the execution result and invokes a corresponding target application; and synchronously displaying the display state of the target application program on the preview picture.
The details of each module in the display control apparatus are described in detail in the corresponding display control method, and therefore are not described herein again.
It should be noted that although in the above detailed description several modules or units of the device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit, according to embodiments of the present disclosure. Conversely, the features and functions of one module or unit described above may be further divided into embodiments by a plurality of modules or units.
Fig. 14 illustrates a schematic block diagram of a computer system suitable for use in implementing a wireless communication device of an embodiment of the present invention.
It should be noted that the computer system 1400 of the electronic device shown in fig. 14 is only an example, and should not bring any limitation to the function and the scope of the application of the embodiment of the present invention.
As shown in fig. 14, a computer system 1400 includes a Central Processing Unit (CPU)1401 that can perform various appropriate actions and processes in accordance with a program stored in a Read-Only Memory (ROM) 1402 or a program loaded from a storage portion 1408 into a Random Access Memory (RAM) 1403. In the RAM 1403, various programs and data necessary for system operation are also stored. The CPU1401, ROM 1402, and RAM 1403 are connected to each other via a bus 1404. An Input/Output (I/O) interface 1405 is also connected to the bus 1404.
The following components are connected to the I/O interface 1405: an input portion 1406 including a keyboard, a mouse, and the like; an output portion 1407 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, a speaker, and the like; a storage portion 1408 including a hard disk and the like; and a communication section 1409 including a network interface card such as a LAN (Local area network) card, a modem, or the like. The communication section 1409 performs communication processing via a network such as the internet. The driver 1410 is also connected to the I/O interface 1405 as necessary. A removable medium 1411 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 1410 as necessary, so that a computer program read out therefrom is installed into the storage section 1408 as necessary.
In particular, according to an embodiment of the present invention, the processes described below with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the invention include a computer program product comprising a computer program embodied on a computer-readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication portion 1409 and/or installed from the removable medium 1411. When the computer program is executed by a Central Processing Unit (CPU)1401, various functions defined in the system of the present application are executed.
It should be noted that the computer readable medium shown in the embodiment of the present invention may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a Read-Only Memory (ROM), an Erasable Programmable Read-Only Memory (EPROM), a flash Memory, an optical fiber, a portable Compact Disc Read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present invention may be implemented by software, or may be implemented by hardware, and the described units may also be disposed in a processor. Wherein the names of the elements do not in some way constitute a limitation on the elements themselves.
As another aspect, the present application also provides a computer-readable medium, which may be contained in the electronic device described in the above embodiments; or may exist separately without being assembled into the electronic device. The computer readable medium carries one or more programs which, when executed by an electronic device, cause the electronic device to implement the method as described in the embodiments below. For example, the electronic device may implement the steps shown in fig. 1 to 6 or fig. 11 to 12.
Furthermore, the above-described figures are merely schematic illustrations of processes involved in methods according to exemplary embodiments of the invention, and are not intended to be limiting. It will be readily understood that the processes shown in the above figures are not intended to indicate or limit the chronological order of the processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, e.g., in multiple modules.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is to be limited only by the terms of the appended claims.
Claims (16)
1. A display control method is applied to a mobile terminal, and the method comprises the following steps:
establishing communication connection with a display terminal;
acquiring a display picture of the display terminal in real time through a camera module of the mobile terminal, and displaying a preview picture corresponding to the display picture;
and responding to a first control operation on the preview picture, mapping the first control operation into a second control operation on the display terminal, and sending the second control operation to the display terminal to control the display picture.
2. The method of claim 1, wherein the control operation comprises an operation location and an operation type;
the mapping the first control operation to a second control operation for the display terminal includes:
determining a second operation position of a second control operation aiming at the display terminal in the display screen according to the first operation position of the first control operation; and
determining a second operation type of the second control operation according to a preset mapping rule and the first operation type of the first control operation;
generating a second control operation for the display screen based on the second operation position and the second operation type.
3. The method of claim 2, wherein the preview screen includes a target area for rendering the display screen;
the determining a second operation position in the display screen of a second control operation for the display terminal based on the first operation position of the first control operation includes:
identifying a target area in the preview picture, and acquiring a target pixel corresponding to the target area in the preview picture;
generating a pixel mapping relation between the target area and the display picture based on the target pixel;
and mapping the first operation position of the first control operation to a second operation position in the display picture according to the pixel mapping relation.
4. The method of claim 2, wherein the preset mapping rule comprises a mapping table for describing a mapping relationship between the first operation type and the second operation type.
5. The method according to claim 1, wherein when the preview screen corresponding to the display screen is displayed, the method further comprises:
comparing the current preview picture with the display picture to identify whether the preview picture is matched with the display picture;
and when the preview picture is identified to be not matched with the display picture, displaying a prompt message on the preview picture so as to adjust the preview picture to be matched with the display picture.
6. The method of claim 5, wherein when the preview screen matches the display screen, the method further comprises:
in response to a third control operation on the preview screen, controlling the preview screen to be zoomed for performing the first control operation based on the zoomed preview screen.
7. The method of claim 5, wherein the preview screen comprises a target area for presenting the display screen;
the comparing the current preview picture with the display picture to identify whether the preview picture is matched with the display picture comprises:
identifying a target area in the preview picture, and carrying out size transformation on a target picture contained in the target area and the display picture so as to enable the target picture and the display picture to be the same in size;
comparing the target picture with the display picture to calculate the similarity between the target picture and the display picture;
when the similarity is larger than or equal to a preset threshold value, judging that the preview picture is matched with the display picture; or
And when the similarity is smaller than a preset threshold value, judging that the preview picture is not matched with the display picture.
8. The method of claim 1, further comprising:
and acquiring an execution result of the second control operation on the display terminal, and calling a corresponding target application program according to the execution result.
9. The method of claim 8, further comprising:
when the target application program is called, freezing the preview picture in a current frame; and
and unfreezing the preview picture when the calling of the target application program is finished.
10. A display control method is applied to a display terminal, and the method comprises the following steps:
establishing communication connection with a mobile terminal to acquire a second control operation sent by the mobile terminal; the second control operation is an operation aiming at the display terminal and obtained according to the first control operation mapping of the preview picture of the mobile terminal;
and executing the second control operation to control the display screen of the display terminal.
11. The method according to claim 10, wherein after said establishing a communication connection with a mobile terminal, the method further comprises:
and acquiring a first control operation sent by the mobile terminal, mapping the first control operation to a second control operation aiming at the display terminal, and executing the second control operation.
12. The method of claim 10, further comprising:
sending the execution result of the second control operation to the mobile terminal so that the mobile terminal can respond to the execution result and call a corresponding target application program;
and synchronously displaying the display state of the target application program on the preview picture.
13. A display control apparatus, applied to a mobile terminal, the apparatus comprising:
the first connection module is used for establishing communication connection with the display terminal;
the image generation module is used for acquiring a display image of the display terminal in real time through a camera module of the mobile terminal and displaying a preview image corresponding to the display image;
and the operation conversion module is used for responding to a first control operation on the preview picture, mapping the first control operation into a second control operation on the display terminal, and sending the second control operation to the display terminal so as to control the display picture.
14. A display control apparatus, applied to a display terminal, the apparatus comprising:
the second connection module is used for establishing communication connection with the mobile terminal so as to acquire a second control operation sent by the mobile terminal; the second control operation is an operation aiming at the display terminal and obtained according to the first control operation mapping of the preview picture of the mobile terminal;
and the display control module is used for executing the second control operation so as to control the display picture of the display terminal.
15. A computer-readable medium on which a computer program is stored, the computer program, when being executed by a processor, implementing the display control method according to any one of claims 1 to 9 or the display control method according to any one of claims 10 to 12.
16. A wireless communication terminal, comprising:
one or more processors;
a storage device to store one or more programs that, when executed by the one or more processors, cause the one or more processors to implement the display control method of any one of claims 1 to 9 or the display control method of any one of claims 10 to 12.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911067160.7A CN110928509B (en) | 2019-11-04 | 2019-11-04 | Display control method, display control device, storage medium, and communication terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911067160.7A CN110928509B (en) | 2019-11-04 | 2019-11-04 | Display control method, display control device, storage medium, and communication terminal |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110928509A true CN110928509A (en) | 2020-03-27 |
| CN110928509B CN110928509B (en) | 2023-06-27 |
Family
ID=69852291
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911067160.7A Active CN110928509B (en) | 2019-11-04 | 2019-11-04 | Display control method, display control device, storage medium, and communication terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110928509B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111405190A (en) * | 2020-04-23 | 2020-07-10 | 南京维沃软件技术有限公司 | Image processing method and device |
| CN113031843A (en) * | 2021-04-25 | 2021-06-25 | 歌尔股份有限公司 | Watch control method, display terminal and watch |
| CN113709706A (en) * | 2021-08-17 | 2021-11-26 | 中汽创智科技有限公司 | Display control method, device, system and terminal for vehicle cabin |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107147769A (en) * | 2016-03-01 | 2017-09-08 | 阿里巴巴集团控股有限公司 | Apparatus control method, device and mobile terminal based on mobile terminal |
| CN108234875A (en) * | 2018-01-15 | 2018-06-29 | 广东欧珀移动通信有限公司 | Shoot display methods, device, mobile terminal and storage medium |
| CN108986110A (en) * | 2018-07-02 | 2018-12-11 | Oppo（重庆）智能科技有限公司 | Image processing method, device, mobile terminal and storage medium |
| CN109166156A (en) * | 2018-10-15 | 2019-01-08 | Oppo广东移动通信有限公司 | A kind of generation method, mobile terminal and the storage medium of camera calibration image |
| CN109891874A (en) * | 2017-08-18 | 2019-06-14 | 华为技术有限公司 | A kind of panorama shooting method and device |
-
2019
- 2019-11-04 CN CN201911067160.7A patent/CN110928509B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107147769A (en) * | 2016-03-01 | 2017-09-08 | 阿里巴巴集团控股有限公司 | Apparatus control method, device and mobile terminal based on mobile terminal |
| CN109891874A (en) * | 2017-08-18 | 2019-06-14 | 华为技术有限公司 | A kind of panorama shooting method and device |
| CN108234875A (en) * | 2018-01-15 | 2018-06-29 | 广东欧珀移动通信有限公司 | Shoot display methods, device, mobile terminal and storage medium |
| CN108986110A (en) * | 2018-07-02 | 2018-12-11 | Oppo（重庆）智能科技有限公司 | Image processing method, device, mobile terminal and storage medium |
| CN109166156A (en) * | 2018-10-15 | 2019-01-08 | Oppo广东移动通信有限公司 | A kind of generation method, mobile terminal and the storage medium of camera calibration image |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111405190A (en) * | 2020-04-23 | 2020-07-10 | 南京维沃软件技术有限公司 | Image processing method and device |
| CN111405190B (en) * | 2020-04-23 | 2022-04-12 | 南京维沃软件技术有限公司 | Image processing method and device |
| CN113031843A (en) * | 2021-04-25 | 2021-06-25 | 歌尔股份有限公司 | Watch control method, display terminal and watch |
| CN113709706A (en) * | 2021-08-17 | 2021-11-26 | 中汽创智科技有限公司 | Display control method, device, system and terminal for vehicle cabin |
| CN113709706B (en) * | 2021-08-17 | 2024-03-26 | 中汽创智科技有限公司 | Display control method, device and system for vehicle cabin and terminal |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110928509B (en) | 2023-06-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10437545B2 (en) | Apparatus, system, and method for controlling display, and recording medium | |
| US20210166467A1 (en) | Picture rendering method and apparatus, storage medium, and electronic apparatus | |
| CN110928509B (en) | Display control method, display control device, storage medium, and communication terminal | |
| US11880999B2 (en) | Personalized scene image processing method, apparatus and storage medium | |
| CN109040524B (en) | Artifact eliminating method and device, storage medium and terminal | |
| CN109040523B (en) | Artifact eliminating method and device, storage medium and terminal | |
| US11917329B2 (en) | Display device and video communication data processing method | |
| CN108665510B (en) | Rendering method and device of continuous shooting image, storage medium and terminal | |
| CN112822529A (en) | Electronic device and control method thereof | |
| US10609305B2 (en) | Electronic apparatus and operating method thereof | |
| WO2024051639A1 (en) | Image processing method, apparatus and device, and storage medium and product | |
| CN111930233B (en) | Panoramic video image display method and display device | |
| CN113825002A (en) | Display device and focus control method | |
| CN112616078A (en) | Screen projection processing method and device, electronic equipment and storage medium | |
| CN110892361B (en) | Display apparatus, control method of display apparatus, and computer program product thereof | |
| EP3848894B1 (en) | Method and device for segmenting image, and storage medium | |
| CN113810755B (en) | Panoramic video preview method and device, electronic equipment and storage medium | |
| CN116801027A (en) | Display device and screen projection method | |
| CN108540726B (en) | Method and device for processing continuous shooting image, storage medium and terminal | |
| CN113587812B (en) | Display equipment, measuring method and device | |
| CN113645502B (en) | Method for dynamically adjusting control and display device | |
| US11334309B2 (en) | Image display method, apparatus and computer readable storage medium | |
| US20220342525A1 (en) | Pushing device and method of media resource, electronic device and storage medium | |
| CN116091628A (en) | Correction method, display device and computer readable storage medium | |
| CN114630085A (en) | Image projection method, image projection device, storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |