JP2021108187A - Method, device, electronic apparatus, and storage medium for controlling smart rearview mirror - Google Patents

Abstract

To provide a method, a device, an electronic apparatus, and a storage medium for controlling a smart rearview mirror, capable of controlling an interface projected by a smart rearview mirror by using an external apparatus on which a user is projected, being easily operated, and attaining security of driving for the user.SOLUTION: The method includes: a step (S201) for acquiring interface display information of a smart rearview mirror; a step (S202) for sending the interface display information to an external apparatus; and a step (S206) for responding to a user operation information from at least one external apparatus and controlling the smart rearview mirror to perform a corresponding operation, the user operation information being for describing an operation of the user to at least one external apparatus.SELECTED DRAWING: Figure 2

Description

The present application relates to the field of data processing, and more particularly to methods, devices, electronic devices and storage media for controlling smart rearview mirrors in vehicle networking technology.

With the development of intelligent vehicles, the number of in-vehicle electronic devices has gradually increased. The rear-view mirror has evolved from the initial rear-view function to a smart rear-view mirror with an auxiliary inversion function, and application programs may be installed in the conventional smart rear-view mirror, and many functions such as music playback and map navigation can be installed. Make it happen. However,
When operating and observing the smart rear-view mirror while driving, the user needs to shift his or her attention to the top of the head, which is inconvenient and harms the user's driving security.

In order to solve the above problem, the prior art provides a technical solution of projecting the interface of the smart rearview mirror onto the in-vehicle infotainment terminal, and the user sees the in-vehicle infotainment terminal. The content displayed on the smart rear-view mirror can be acquired with, avoiding the problem of the impact on driving security caused by the user looking at the smart rear-view mirror. The smart rearview mirror keeps taking screenshots, that is, it is a screen recording, recording what needs to be projected as a video, encoding the video into a video stream and transmitting it to the in-vehicle infotainment terminal. The in-vehicle infotainment terminal decodes the video stream to display the smart rear-view mirror interface, and thus realizes a solution in which the smart rear-view mirror interface is projected onto the in-vehicle infotainment terminal.

However, in the prior art, when the user needs to operate the projected interface after projecting the interface of the smart rearview mirror on the in-vehicle infotainment terminal, the operation is inconvenient because it can be operated only by the smart rearview mirror. , There is also the problem of harming the user's driving security.

The present application provides a method, an apparatus, an electronic device and a storage medium for controlling a smart rear-view mirror, and can realize that a user controls an application program projected on the smart rear-view mirror with an external device.

In a first aspect, the present application provides a method for controlling a smart rear-view mirror, in which a step of acquiring interface display information of the smart rear-view mirror and the interface display information are communicably connected to the smart rear-view mirror. A step of transmitting to at least one external device and a step of controlling the smart rearview mirror to perform a corresponding operation in response to user operation information from the at least one external device. The information includes steps used to describe the user's operation on the at least one external device.

In a second aspect, the present application provides a method for controlling the smart rearview mirror, the step of receiving the interface display information from the projection control device, and the projection by the smart rearview mirror based on the interface display information. It includes a step of displaying the interface, a step of generating user operation information when a user's operation on the interface is detected, and a step of transmitting the user operation information to the projection control device.

In a third aspect, the present application provides a device for controlling a smart rearview mirror.
A processing module for acquiring the interface display information of the smart rearview mirror,
Includes a transmit / receive module for transmitting the interface display information to at least one external device communicably connected to the smart rearview mirror.
The processing module is also used to respond to user operation information from the at least one external device and control the smart rearview mirror to perform the corresponding operation, and the user operation information is the at least one. It is used to describe the user's operation on one external device.

In a fourth aspect, the present application provides a device for controlling a smart rearview mirror.
A transmission / reception module for receiving interface display information from the projection control device,
Based on the interface display information, a display module for displaying the projected interface in the smart rear-view mirror, and
When a user operation on the interface is detected, a processing module for generating user operation information and a processing module
The transmission / reception module for transmitting the user operation information to the projection control device is included.

In a fifth aspect, the present application provides an electronic device, comprising at least one processor, a memory communicatively connected to the at least one processor, the memory storing computer execution instructions and said at least one processor. Executes the computer execution instruction stored in the memory, and as a result, the electronic device executes the method for controlling the smart back mirror according to the first aspect and the second aspect, and outputs the output. The device is for displaying the interface of the electronic device.

In a sixth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer execution instruction, and when the computer execution instruction is executed by a processor, the first aspect described above. The method for controlling the smart back mirror according to the first aspect and the second aspect is realized.

In a seventh aspect, the present application provides a computer program, the computer program being stored in a computer readable storage medium, and at least one processor of the electronic device reading the computer program from the computer readable storage medium. The at least one processor can cause the electronic device to execute the method for controlling the smart back mirror according to the first aspect and the second aspect by executing the computer program.

For the beneficial effects of each of the above embodiments, the beneficial effects described in the specific embodiments below can be specifically referred to.

With the method, device, electronic device and storage medium for controlling the smart rear-view mirror according to the present application, the specific realization means is to acquire the interface display information of the smart rear-view mirror and to communicate the interface display information with the smart rear-view mirror. Sending to at least one connectable external device and responding to user operation information from at least one external device, the smart rearview mirror controls to perform the corresponding operation, and the user operation information is at least Includes that it is intended to describe a user's operation on one external device. In the technical solution disclosed in the present application, it is possible for the user to control the projected interface in the smart rear-view mirror with the projected external device, the operation is convenient, and the user's driving security can be ensured.

The drawings are used to better understand the present solution and do not constitute a limitation to the present application.
It is an exemplary system architecture diagram of the method for controlling the smart rearview mirror according to the embodiment of this application. It is a flowchart of the method for controlling a smart rearview mirror which concerns on this application. FIG. 1 is a schematic view of a change in the interface of the smart rear-view mirror according to the present application. FIG. 2 is a schematic view of a change in the interface of the first external device according to the present application. FIG. 2 is a schematic view of a change in the interface of the smart rear-view mirror according to the present application. FIG. 2 is a schematic view of a change in the interface of the first external device according to the present application. It is the schematic of the interface of the 1st external device which concerns on this application. It is the schematic of the interface of the smart rearview mirror which concerns on this application. FIG. 3 is a schematic view of a change in the interface of the first external device according to the present application. It is the schematic of the scene to which the method for controlling a smart rearview mirror which concerns on this application is applied. FIG. 1 is a schematic structure diagram 1 of a device for controlling a smart rear-view mirror according to the present application. FIG. 2 is a schematic structure diagram 2 of a device for controlling a smart rear-view mirror according to the present application. It is a structural schematic diagram of the electronic device which concerns on this application.

Hereinafter, exemplary embodiments of the present application will be described with reference to the drawings. Although various details of the embodiments of the present application are included to contribute to understanding, these should be considered to be merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications to the embodiments described herein are possible and do not deviate from the scope and purpose of the present application. Similarly, for clarity and conciseness, the following description will omit the description of well-known functions and structures.

FIG. 1 is an exemplary system architecture diagram of a method for controlling a smart rearview mirror according to an embodiment of the present application. As shown in FIG. 1, the method relates to a smart rearview mirror and at least one external device. The smart rear-view mirror in this embodiment is installed in the vehicle, and the smart rear-view mirror can be installed at a position near the center of the windshield inside the vehicle. The smart rear-view mirror in this embodiment has only a rear view function. It should be understood that the rear-view mirror may be a rear-view mirror, a smart rear-view mirror with an auxiliary reversal function, or a smart rear-view mirror with other functions such as navigation and music playback, but is not limited thereto.

The external device in this embodiment may be an in-vehicle infotainment terminal in the vehicle or a user's terminal device in the vehicle. The in-vehicle infotainment terminal in this embodiment can be mounted in the center console of the vehicle, and the screen of the in-vehicle infotainment terminal is installed on the surface of the center console to display various information to the user. The smart rear-view mirror and the in-vehicle infotainment terminal are connected to the in-vehicle infotainment terminal by a wired connection, for example, a universal serial bus (abbreviated as USB), or a wireless connection, for example, Bluetooth (abbreviated as USB). It may be connected by a registered trademark) or by wireless fidelity (abbreviated as Wi-Fi). The smart rear-view mirror in this embodiment can assist the vehicle inversion, and an application program may be installed in the smart rear-view mirror to realize many functions such as music playback and map navigation. The terminal device of the user in this embodiment may be a mobile phone, a personal digital assistant (abbreviated as PDA), a tablet computer, a portable device (for example, a portable computer, a pocket computer, or a handheld computer). , Not limited to these. It should be understood that FIG. 1 illustrates one external device, which is an in-vehicle infotainment terminal, as an example.

As shown in FIG. 1, the smart rear-view mirror is often installed near the center of the windshield inside the vehicle, so when the user operates the interface of the smart rear-view mirror and looks at it, pay attention to the top of the head. It needs to be moved, inconvenient to operate, and harms the user's driving security. In the prior art, in order to solve the problem, a technical solution means for projecting an interface of a smart rearview mirror onto an external device, and the external device is, for example, an in-vehicle infotainment terminal, and as a result, a user can use the device. By looking at the in-vehicle infotainment terminal, the contents displayed on the smart rear-view mirror can be acquired, and the problem of the influence on driving security caused by the user looking at the smart rear-view mirror can be avoided.

However, in the prior art, when the user needs to operate the projected interface after projecting the interface of the smart rearview mirror on the in-vehicle infotainment terminal, the user can operate only with the smart rearview mirror, so that the smart rearview mirror can be used. Since the control of the interface cannot be realized by the in-vehicle infotainment terminal, the problem that the operation is inconvenient and the user's driving security is harmed after the conventional smart rear-view mirror interface is projected on the in-vehicle infotainment terminal is the same. Exists in.

To solve the above problems, the present application provides a method for controlling a smart rearview mirror, in which the user projects the interface of the smart rearview mirror onto an external device and then projects it onto the external device. By operating the interface, control of the smart rear-view mirror interface can be realized, the operation is convenient, and the user's driving security can be ensured.

Hereinafter, a method for controlling the smart rear-view mirror according to the present application will be described with reference to specific examples. FIG. 2 is a flowchart of a method for controlling the smart rear-view mirror according to the present application. The execution body of the method flow as shown in FIG. 2 may be a device for controlling the smart rear-view mirror, and the device for controlling the smart rear-view mirror is a smart rear-view mirror or projection control in the smart rear-view mirror. It may be a device, the projection control device may be a processor, a chip, etc., and it should be understood that the projection control device can be implemented by any software and / or hardware. Hereinafter, a case where the control device of the smart rear-view mirror is a projection control device will be described.

As shown in FIG. 2, the method for controlling the smart rear-view mirror according to the present embodiment includes the following steps.
S201, the projection control device acquires the interface display information of the smart rearview mirror,
S202, the projection control device transmits the interface display information to at least one external device communicably connected to the smart rearview mirror.
S203, the first external device displays the projected interface in the smart rearview mirror based on the interface display information.
S204, when the first external device detects the user's operation on the interface, the user operation information is generated.
S205, the first external device transmits user operation information to the projection control device,
S206, the projection control device responds to the user operation information from at least one external device, the smart rearview mirror controls to execute the corresponding operation, and the user operation information describes the user's operation on at least one external device. It is for doing.

In S201 described above, the projection control device can acquire the interface display information of the smart rear-view mirror when the interface of the smart rear-view mirror is projected. In the present embodiment, the projection control device can record the screen on the interface displayed on the smart rear-view mirror, and thus obtain the interface display information of the smart rear-view mirror. Alternatively, the projection control device can also acquire the display information of the current interface of the smart rearview mirror and use the display information of the current interface as the interface display information. It may be a structured layout, information displayed on the interface, and so on.

In S202 above, at least one external device in this embodiment can refer to the related description above and will not be repeated herein. When the smart rear-view mirror is projected onto at least one external device, the interface display information of the smart rear-view mirror is transmitted to the at least one external device, and each of the individual external devices is based on the interface display information of the smart rear-view mirror. The interface in the smart rear-view mirror is fixed, and by extension, the projected interface in the smart rear-view mirror can be displayed on an external device.

In S203 described above, in the present embodiment, the first external device is any of the above-mentioned at least one external device, and hereinafter, the first external device is taken as an example in the at least one external device. The operation process of each external device will be described. It should be understood that the operating process of each external device is the same as the operating process of the first external device.

In this embodiment, the first external device can display the projected interface on the smart rearview mirror based on the interface display information from the projection control device. The interface display information of the smart rear-view mirror may be the main interface display information of the smart rear-view mirror, or the interface display information of the application program in the smart rear-view mirror. It should be understood that the main interface of the smart rearview mirror may be the interface of the smart rearview mirror before entering the application program.

Alternatively, the projection control can be displayed on the interface of the smart rearview mirror in this embodiment, and after the user selects the projection control, the interface of the smart rearview mirror is projected onto at least one external device by the projection control device. Control to be done.

Illustratively, FIG. 3 is a schematic view of a change in the interface of the smart rearview mirror according to the present application. As shown in the interface 301 in FIG. 3, the interface 301 is the main interface of the smart rear-view mirror, and information such as the icon, the date and time, and the setting of the application program in the smart rear-view mirror is displayed on the main interface. .. A projection control is also displayed on the interface 301, and when the user clicks on the projection control, the interface 301 jumps to the interface 302 and a sign of the external device projected onto the interface 302, for example, "external device". 1 ”is displayed, and information on the interface 301 is also displayed. Optionally, the interface 302 does not display information about the interface 301 and is displayed and projected with external device signs such as "external device 1" and "the main interface has been projected onto the external device". Caution information may be displayed. Before projecting the interface of the smart rearview mirror onto an external device, it should be understood that the smart rearview mirror has established a wired or wireless connection with the external device.

After the projection control device controls the smart rear-view mirror interface to be projected onto the external device, the external device can be displayed in association with the projected interface on the smart rear-view mirror.

Illustratively, FIG. 4 is a schematic view 2 of an interface change of the first external device according to the present application. As shown in the interface 401 in FIG. 4, the interface 401 is an interface before being projected onto the first external device, and the interface may be the main interface of the first external device, for example, the interface 401. Information such as the date and time and settings may be displayed on the interface. After controlling the interface of the smart rearview mirror to be projected onto the first external device, in the first external device, the interface 401 jumps to the interface 402, and the interface 402 and the interface 301 in FIG. 3 described above. It may be the same, but the projection control is not displayed on the interface 402.

In the above S204 and the above S205, the first external device can detect the user's operation on the projection interface displayed on the first external device and generate the user operation information. It should be understood that the user operation information is for describing the user operation information for the first external device. The user operation information can describe the user's operation on the machine control installed in the first external device or the user's operation on the projection interface displayed on the first external device. In this embodiment, the first external device also transmits the user operation information to the projection control device, and the user operation information instructs the projection control device to control the smart rearview mirror, and as a result, The projection control device controls the smart rear-view mirror to perform the corresponding operation.

Illustratively, when the user clicks a setting control on the interface 402, the first external device can detect the user's operation on the interface 402 and generate the user operation information. It is for describing that the user clicked the setting control.

In S206 described above, the projection control device can confirm the user operation on the projection interface of the external device after receiving the user operation information. That is, the projection control device in this embodiment can be controlled so that the smart rear-view mirror executes the corresponding operation in response to the user operation information from at least one external device.

Illustratively, if the user operation information is meant to describe that the user clicked on a configuration control, the projection control device controls and responds to the smart rearview mirror to perform an operation that enters the configuration interface. Then, the interface 302 of the smart rear-view mirror jumps to the interface 303, and the interface 303 is a setting interface of the smart rear-view mirror. For example, the interface 303 may display settings such as a connection method, such as Bluetooth connection, more connections, and settings such as interface brightness and wallpaper. Since the interface of the smart rearview mirror is projected onto the first external device, the interface 402 of the external device jumps to the interface 403 accordingly, and the interface 403 is the same as the interface 303 described above.

The method for controlling the smart rear-view mirror according to the present embodiment includes a step of acquiring the interface display information of the smart rear-view mirror and transmitting the interface display information to at least one external device communicably connected to the smart rear-view mirror. And the control step so that the smart rearview mirror performs the corresponding operation in response to the user operation information from at least one external device, and the user operation information performs the user's operation on at least one external device. Includes steps that are meant to be described. In this embodiment, the external device can detect the user's operation on the interface projected on the external device, and can transmit the user's operation information to the projection control device of the smart rear-view mirror, and the interface by the smart rear-view mirror. The control is realized, that is, in the present embodiment, the user can realize the control for the interface of the smart rear-view mirror projection by the external device, the operation is convenient, and the driving security of the user is ensured.

Based on the above embodiment, the projection control process of the application program in the smart rearview mirror will be described in the following embodiment, and the method for controlling the smart rearview mirror in the following embodiment will be described in the above embodiment. Should also be understood to apply. The above-mentioned interface of the smart rear-view mirror is an interface of the application program in the smart rear-view mirror, and the application program in the smart rear-view mirror is, for example, a music application program, a navigation application program, a weather application program, etc. as shown in the above interface 301. There may be. Correspondingly, the user's operation on the interface projected on the external device is, that is, the operation on the interface of the application program projected on the external device, and the projection control device is a smart back mirror based on the user operation information. Application programs can be controlled to perform the corresponding operations.

Hereinafter, a method for controlling the smart rear-view mirror according to the present application will be described in more detail with reference to FIG. FIG. 4 is a flowchart of the second embodiment of the method for controlling the smart rear-view mirror according to the present application. As shown in FIG. 4, the method for controlling the smart rear-view mirror according to the present embodiment includes the following steps.
S401, the projection control device receives the projection instruction output by the user, records the interface of the application program on the smart back mirror, obtains the video to be projected, and the projection instruction makes the interface of the application program at least one. It is intended to instruct one external device to project
S402, the projection control device transmits the video to be projected to at least one external device,
S403, the first external device displays the interface of the application program in the smart rearview mirror based on the video to be projected.
S404, when the first external device detects the user's operation on the interface, the user operation information is generated.
S405, the first external device transmits the user operation information to the projection control equipment,
S406, the projection control device generates a trigger event based on the interface layout of the application program and the user operation information, and the trigger event instructs the application program to perform the corresponding operation.
Based on S407, the trigger event, the application program in the smart rearview mirror is controlled to execute the corresponding operation.

In S401 above, when the interface of the smart rear-view mirror is the interface of the application program in the smart rear-view mirror, the projection control can be displayed on the interface of the application program, and when the user clicks the projection control, the projection instruction is smart-backed. It can be triggered to input to the projection control device in the mirror. In this embodiment, it should be understood that the user can also input projection instructions using voice or action commands.

Illustratively, FIG. 5 is a schematic view of a change in the interface of the smart rearview mirror according to the present application. As shown in the interface 501 in FIG. 5, what is displayed on the interface 501 is an interface of a music application program of a smart rearview mirror projected on at least one external device, and the interface is currently an interface for playing a song. Is displayed.

In the present embodiment, in the method of projecting the interface of the application program, the projection control device records the interface of the application program with the smart back mirror, obtains the video to be projected, and by extension, at least one of the videos to be projected. It is to send to one external device so that each external device projects the interface of the application program.

In S402 described above, the projection control device can encode the projection target video after obtaining the projection target video and transmit the encoded projection target video to at least one external device.

Correspondingly, in the above S403, the first external device decodes the projection target video after receiving the projection target video from the projection control device, and makes the interface of the application program the first external device. It can be displayed on the device. Illustratively, FIG. 6 is a schematic view of a change in the interface of the first external device according to the present application. As shown in the interface 601 in FIG. 6, what is displayed on the interface 601 is the interface of the music application program in the smart rear-view mirror, and the interface for playing a song is currently displayed on the interface.

In S404 described above, for example, the user can perform operations such as cutting a song, adjusting the volume, and fast-forwarding on the interface 601. The user can perform operations on the interface for different application programs. It should be understood that may be different. When the first external device detects the user's operation on the interface, it generates the user operation information. The user's operation information can include the coordinate position of the user's operation on the interface that the user is currently displaying on the first external device. As shown in the interface 601. Assuming that the A position of the first external device is the origin and the horizontal and vertical directions are the X-axis and the Y-axis, respectively, the user at the interface currently displayed on the first external device The coordinate position of the operation should be understood to be the position in the coordinate system. Optionally, the user's operational information can also include the user's operational actions. The user's operation may be, for example, a single click, a double click, a slide, or the like.

Illustratively, in this embodiment, when the user clicks the pause control on the interface 601 the user's operation information can include the coordinate position clicked by the user and the operation operation of a single click.

In S406 described above, the projection control device can generate a trigger event based on the interface layout of the application program and the user operation information after receiving the user operation information from the first external device. Trigger events direct the application program to perform the appropriate operation.

Hereinafter, in this embodiment, a possible embodiment for generating a trigger event will be described.

The user operation information in the first type method and the present embodiment includes the coordinate position of the user operation in the current interface of the first external device. The projection control device can convert the coordinate position of the user operation on the current interface of the first external device to the coordinate position on the current interface of the smart rearview mirror.

In such a method, when the interface of the application program projected by all the interfaces of the first external device is displayed as shown in the interface 601 above, the smart back mirror is displayed as shown in the interface 501 above. When the interface of the application program projected by all of the interfaces of is displayed, in this embodiment, the coordinate position of the user operation on the current interface of the first external device and the application displayed on the first external device. Based on the size of the interface of the program, the relative position of the user operation on the interface of the application program displayed on the first external device can be determined.

Illustratively, when the user clicks the pause control on interface 601 the user's operational information can include the coordinate position clicked by the user. In this embodiment, the relative position of the coordinate position with respect to the interface of the application program is determined based on the coordinate position and the size of the interface of the application program displayed on the first external device. For example, the coordinate position is at a position at the lower left of the interface of the application program.

In this embodiment, the coordinate position of the smart rearview mirror at the current interface can be determined based on the relative position as described above and the size of the interface of the application program displayed on the smart rearview mirror. Illustratively, when the coordinate position is at a lower left position of the application program interface, the position clicked by the user is the application in the smart rearview mirror, based on the size of the application program interface in the smart rearview mirror. It can be determined to be in a certain position at the bottom left of the program's interface.

In such a method, as shown in FIG. 7, when the interface of the application program projected by a part of the interface of the first external device is displayed, information such as time is displayed in the other part. As shown in 8, when the interface of the application program projected by a part of the interface of the smart rearview mirror is displayed, information such as time is displayed in the other part. FIG. 7 is a schematic view of the interface of the first external device according to the present application. FIG. 8 is a schematic view of the interface of the smart rearview mirror according to the present application. In this embodiment, the first external device is based on the coordinate position of the user operation on the current interface of the first external device and the coordinate position and size of the interface of the application program displayed on the first external device. The relative position of the user operation on the interface of the displayed application program can be determined. Since not all of the interfaces of the first external device are displayed on the interface of the application program, when determining the relative position of the user operation on the interface of the application program displayed on the first external device, the first external device is displayed. It should be understood that it is necessary to determine the coordinate position of the interface of the displayed application program. As shown in the coordinate position on the left side of the first external device in FIG. 7, the coordinate position of the interface of the application program displayed on the first external device is the position of the interface of the application program on the interface of the first external device. Should be understood as a set of.

Illustratively, when the user clicks the pause control on interface 601 the user's operational information can include the coordinate position clicked by the user. In this embodiment, the relative position of the coordinate position with respect to the interface of the application program can be determined based on the coordinate position and the coordinate position and size of the interface of the application program displayed on the first external device. .. For example, the coordinate position is at a position at the lower left of the interface of the application program.

Correspondingly, the projection control device can determine the coordinate position of the smart rearview mirror at the current interface based on the relative position and the coordinate position and size of the interface of the application program displayed on the smart rearview mirror. Illustratively, when the coordinate position is at a lower left position of the application program interface, the position clicked by the user is the smart rearview mirror based on the coordinate position and size of the application program interface in the smart rearview mirror. It can be confirmed that it is in a certain position at the lower left of the interface of the application program in. It should be understood that the above two methods of converting the coordinate position of the user operation on the current interface of the first external device to the coordinate position of the smart rearview mirror on the current interface can be used in combination. Is.

In this embodiment, the projection control device converts the coordinate position of the user operation on the current interface of the external device to the coordinate position on the current interface of the smart rearview mirror, and then the interface layout of the application program and the current state of the smart rearview mirror. Trigger events can be generated based on the coordinate position in the interface.

Based on the interface layout of the application program, the operation control corresponding to the coordinate position in the current interface of the smart rearview mirror can be determined, and the trigger event can be generated based on the operation control. Illustratively, based on the interface layout of the application program, if the operation control corresponding to the coordinate position in the current interface of the smart rearview mirror is determined to be the pause control, a trigger event is generated based on the pause control. , The trigger event instructs the music application program to pause playback.

The second type method, the information of the user operation includes the coordinate position of the user operation in the current interface of the first external device and the operation operation of the user. In this embodiment, the trigger event can be generated based on the interface layout of the application program, the coordinate position of the smart rearview mirror on the current interface, and the operation operation of the user.

In this embodiment, the method of converting the coordinate position of the user operation on the current interface of the first external device to the coordinate position of the current interface of the smart rearview mirror is specifically described in relation to the above-mentioned first type method. It should be understood that it can be referenced. In this embodiment, after determining the coordinate position of the smart rear-view mirror on the current interface, the operation position of the user in the application program is determined based on the interface layout of the application program and the coordinate position of the smart rear-view mirror on the current interface. As a result, the trigger event can be generated based on the operation behavior of the user. For example, a single click by the user indicates to pause playback, a double click indicates to cut a song, and correspondingly, in this embodiment, the generated trigger event is played back in the music application program. Control to pause, or instruct the music application program to control to cut the song.

Correspondingly, in S407 described above, the projection control device can control the application program in the smart rearview mirror to perform the corresponding operation based on the trigger event. Illustratively, when the user clicks on the pause control on the first external device, the projection control device controls the music application program to pause playback, correspondingly to the above smart back mirror interface 501. Jumps to interface 502, the interface 502 is an interface on which the music application program pauses playback, and for example, the interface 502 can display a music pause indicator. Since the interface of the smart rearview mirror is projected onto the first external device, the interface 601 of the first external device jumps to the interface 602, and the interface 602 is the same as the interface 502 described above.

In the present embodiment, the projection control device determines the intention of the user operation based on the coordinate position of the user operation at the current interface of the first external device in the user operation information and the operation operation of the user, and thus determines the intention of the user operation. Corresponding trigger events can be generated, which in turn can provide control over the application program in the smart rearview mirror.

In the above embodiment, one application program in the smart rear-view mirror is projected onto the first external device to realize reverse control (that is, control of the application program in the smart rear-view mirror is realized by the first external device. The process was explained. It should be noted that the method for controlling the smart rearview mirror in the present application can realize simultaneous projection of a multi-application program. Correspondingly, when the interface of the multi-application program is simultaneously projected onto the first external device, the target application program displayed on the current interface of the first external device is determined, and by extension, the interface layout of the target application program. Trigger events can be generated based on user operation information. In this embodiment, the method of generating the trigger event based on the interface layout and the user operation information of the target application program is specifically described in the above-mentioned "Based on the interface layout and the user operation information of the application program" of the second embodiment. It should be understood that the related description of "Generate trigger event" can be referred to. The projection control device can determine the application program displayed on the current interface of the smart rear-view mirror and set the application program displayed on the current interface of the smart rear-view mirror as the target application program.

In the scene where the interface of the multi-application program is simultaneously projected on the first external device, the switching control is also displayed on the current interface of the first external device, and the switching control is displayed on the current interface of the first external device. Instructs to switch the targeted application program. Optionally, the switching control may be a marker for a projected application program other than the target application program.

Correspondingly, when the user clicks the switching control, the user operation information generated by the first external device is the coordinate position of the switching control in the current interface of the first external device and the indicator of the switched application program. The user operation information including, or generated by the first external device, includes the coordinate position of the switching control in the current interface of the first external device, the user operation operation, and the indicator of the switched application program. .. Correspondingly, the projection control device instructs the generated trigger event to switch from the target application program to the switched application program based on the operation operation of the user. Correspondingly, the projection control device controls the application program in the smart rearview mirror to switch to the switched application program based on the trigger event.

Illustratively, FIG. 9 is a schematic view of a change in the interface of the first external device according to the present application. As shown in FIG. 9, the interface 901 is an interface for projecting a music application program, and a switching control is displayed on the interface 901. For example, the switching control is a sign for a navigation application program or a sign for a weather application program. After the user clicks the indicator of the navigation application program, the interface 901 jumps to the interface 902, and what is displayed on the interface 902 is the interface of the navigation application program. Corresponding to FIG. 9, it should be understood that the interface change of the smart rearview mirror is the same as the interface change in FIG.

In order to explain that the projection of a multi-application program can be realized by the method for controlling the smart rear-view mirror in this embodiment, first, the method for controlling the smart rear-view mirror in the prior art will be introduced.

In the projection method in the prior art, when a projection application program is installed in a smart back mirror and an interface of a certain application program is projected onto an external device, it is necessary to start the projection application program, and the interface of the application program is projected. It's a way to make it happen. The projection application program integrates the business logic code of the application program in the smart rear-view mirror, and when projecting, it is necessary to call and realize the business logic code and the projection code of the application program. When adding an application program to the smart rear-view mirror, it is necessary to copy the business logic code of the added application program to the projection application program, and specifically to the installation package apk of the projection application program. In the prior art, the projection of each application program is realized in one process (ie, the projection application program), and in the prior art, only the projection of the interface of one application program can be allowed at the same time. In the prior art, when a user projects an application program, it is necessary to exit the currently projected application program and then project another application program, which means that the multi-application program is projected onto an external device at the same time. It cannot be realized.

In this embodiment, a software development tool (Software Development Kit, SDK) is installed in each application program in the smart back mirror, and the reverse control business logic is realized in the SDK, and operations corresponding to the execution of each application program are performed. The operations for controlling are integrated. In this embodiment, when adding an application program, it is not necessary to copy the business logic code of the added application program, it is only necessary to integrate the SDK in the added application program, saving events, and succeeding. It is advantageous for realizing rapid projection reverse control extension of other application programs.

Correspondingly, in the present embodiment, a method for controlling the smart rear-view mirror according to the present application will be further described with reference to FIG. FIG. 10 is a schematic view of a scene to which the method for controlling the smart rear-view mirror according to the present application is applied, and as shown in FIG. 10, the scene includes a first external device and a smart rear-view mirror. .. The smart rear-view mirror includes a projection control device and a plurality of application programs such as a music application program, a navigation application program and a weather application program. The first external device is for performing the operation performed by the first external device in the above embodiment, and the projection control device is for performing the operation performed by the projection control device. belongs to.

After the projection control device receives the projection instruction, the application program that needs to be projected must first be registered in the projection control device, which creates the projection process for each projected application program. As a result, the interface of the application program is controlled so as to be projected onto the first external device, and the state of the application program is saved as the projected state.

After receiving the user operation information from the first external device, the projection control device generates a trigger event MotionEvent based on the user operation information and the interface layout of the application program, and sets the trigger event as the target application program (first). Distribute to the projection process of the application program (application program displayed on the current interface of the external device of 1), and specifically, distribute to the event input window (window.inject Input Event (event)) in the projection process of the target application program. As a result, the target application program can be controlled to realize the reverse control for the target application program by using the reverse control business logic in the SDK.

Correspondingly, at the end of projection of the application program, the projection control device can correct the projection state of the application program to the end state.

According to the method for controlling the smart back mirror according to the present embodiment, it is possible to simultaneously project a plurality of application programs of the smart back mirror, and it is possible to switch between a plurality of projected application programs. When the user projects an application program, the problem of terminating the currently projected application program and then projecting another application program is avoided, and the user experience is improved.

FIG. 11 is a schematic structure diagram 1 of a device for controlling the smart rear-view mirror according to the present application. The device for controlling the smart rear-view mirror is the projection control device in the above embodiment. As shown in FIG. 11, the device 1100 for controlling the smart rear-view mirror includes a processing module 1101 and a transmission / reception module 1102.

The processing module 1101 is used to acquire the interface display information of the smart rearview mirror.

The transmission / reception module 1102 is used to transmit the interface display information to at least one external device communicably connected to the smart rearview mirror.

Correspondingly, the processing module 1101 is also used to respond to user operation information from at least one external device and control the smart rearview mirror to perform the corresponding operation, and the user operation information is at least one. It is for describing the user's operation on an external device.

In one possible embodiment, the interface display information of the smart rear-view mirror is the interface display information of the application program in the smart rear-view mirror.

Correspondingly, the processing module 1101 is specifically used to control the application program in the smart rearview mirror to perform the corresponding operation based on the user operation information.

In one possible embodiment, the processing module 1101 specifically generates a trigger event based on the interface layout and user operation information of the application program so that the application program performs the corresponding operation. It is used to direct and control the application program in the smart back mirror to perform the corresponding operation based on the trigger event.

In one possible embodiment, the user operation information includes the coordinate position of the user operation on the current interface of the first external device, and the first external device is any device of at least one external device.

Correspondingly, the processing module 1101 specifically converts the coordinate position of the user operation in the current interface of the first external device into the coordinate position in the current interface of the smart rearview mirror, and the interface layout of the application program and the interface layout. Used to generate a trigger event based on the coordinate position of the smart rearview mirror in the current interface.

In one possible embodiment, the processing module 1101 specifically determines the operational controls corresponding to the coordinate positions in the current interface of the smart rearview mirror, based on the interface layout of the application program, and based on the operational controls. Used to generate a trigger event.

In one possible embodiment, the user operation information includes a user operation operation.

Correspondingly, the processing module 1101 is used to generate a trigger event specifically based on the interface layout of the application program, the coordinate position of the smart rearview mirror on the current interface, and the user's operation behavior.

In one possible embodiment, the processing module 1101 is specifically based on the coordinate position of the user operation on the current interface of the first external device and the size of the interface of the application program displayed on the first external device. Then, the relative position of the user operation on the interface of the application program displayed on the first external device is determined, and the smart back mirror is based on the relative position and the size of the interface of the application program displayed on the smart back mirror. Used to determine the coordinate position of the current interface of.

In one possible embodiment, the processing module 1101 specifically includes the coordinate position of the user operation on the current interface of the first external device and the coordinate position of the interface of the application program displayed on the first external device. Based on the size, determine the relative position of the user operation on the interface of the application program displayed on the first external device, and to the relative position and the coordinate position and size of the interface of the application program displayed on the smart back mirror. Based on, it is used to determine the coordinate position of the smart back mirror on the current interface.

In one possible embodiment, there are multiple application programs.

Correspondingly, the processing module 1101 specifically determines the target application program displayed on the current interface of the first external device, and generates a trigger event based on the interface layout of the target application program and the user operation information. Used to do.

In one possible embodiment, the processing module 1101 is specifically used to target the application program displayed in the current interface of the smart rearview mirror.

In one possible embodiment, the switching control is also displayed on the current interface of the first external device, which directs the switching of the target application program displayed on the current interface of the first external device. Correspondingly, the user operation information includes the coordinate position of the switching control on the current interface of the first external device and the indicator of the switched application program, and the trigger event switches from the target application program to the switched application program. Instruct.

Correspondingly, the processing module 1101 is specifically used to control the target application program in the smart rearview mirror to switch to the switched application program based on the trigger event.

In one possible embodiment, processing module 1101 is further used to create a projection process for the application program.

In one possible embodiment, the processing module 1101 specifically receives the projection instruction output by the user, records the interface of the application program on the smart back mirror, obtains the video to be projected, and projects it. The instruction is for instructing the interface of the application program to be projected to the first external device, and is used to transmit the video to be projected to the first external device.

In one possible embodiment, the processing module 1101 further stores the state of the application program as a projected state, and at the end of the projection of the application program, modifies the projected state of the application program to the finished state, the projection process of the application program. Used to terminate.

FIG. 12 is a schematic structure diagram 2 of a device for controlling the smart rear-view mirror according to the present application. The device for controlling the smart rear-view mirror is the first external device in the above embodiment. As shown in FIG. 12, the device 1200 for controlling the smart rear-view mirror includes a display module 1201, a processing module 1202, and a transmission / reception module 1203.

The transmission / reception module 1203 is used to receive the interface display information from the projection control device.

The display module 1201 is used to display the projected interface in the smart rearview mirror based on the interface display information.

The processing module 1202 is used to generate user operation information when it detects a user operation on the interface.

The transmission / reception module 1203 is also used to transmit user operation information to the projection control device.

In one possible embodiment, the interface display information of the smart rear-view mirror is the interface display information of the application program in the smart rear-view mirror.

In one possible embodiment, the processing module 1202 is specifically used to detect the coordinate position of the user operation on the current interface of the first external device and generate the user operation information.

In one possible embodiment, the processing module 1202 is specifically used to detect the coordinate position and operation operation of the user operation on the current interface of the first external device and generate the user operation information.

In one possible embodiment, the application program is plural, the switching control is also displayed on the current interface of the first external device, and the switching control is the target application displayed on the current interface of the first external device. Instructing to switch the program, the user operation information includes the coordinate position of the switching control on the current interface of the first external device and the indicator of the switched application program.

In one possible embodiment, the transmit / receive module 1203 also receives the projected video from the projection control device after being connected to the projection control device, and the projected video is obtained by recording the interface of the application program. This is a video.

Correspondingly, the display module 1201 is used specifically to display the interface of the application program in the smart rearview mirror based on the video to be projected.

Since the realization principle and technical effect of the device for controlling the smart rear-view mirror according to the present embodiment are similar to the realization principle and technical effect of the method for controlling the smart rear-view mirror described above, it is repeated here. I will not explain.

According to the embodiments of the present application, the present application further provides electronic devices and readable storage media. FIG. 13 is a schematic structural diagram of the electronic device according to the present application, and the electronic device may be the smart rear-view mirror in the above embodiment. The components described herein, their connections and relationships, and their functionality are merely examples and are not intended to limit the realization of the present application described and / or required herein.

As shown in FIG. 13, the electronic device includes one or more processors 1301, memory 1302, and an interface for connecting each component, and the interface includes a high-speed interface and a low-speed interface. The individual components are connected to each other using different buses and can be installed on the public mainboard or otherwise installed as needed. The processor can process instructions executed within the electronic device and is in memory or in memory for displaying GUI graphics information on an external input / output device (such as a display device coupled to an interface). Contains the instructions stored above. In other embodiments, multiple processors and / or multiple buses can be used with multiple memories, if desired. Similarly, a plurality of electronic devices can be connected, and each device can provide some necessary operations (eg, a server array, a group of blade servers, or a multiprocessor system). In FIG. 13, one processor 1301 is taken as an example.

The memory 1302 is a non-temporary computer-readable storage medium according to the present application. Instructions that can be executed by at least one processor are stored in the memory so that the at least one processor executes the method for controlling the smart rearview mirror according to the present application. The non-temporary computer-readable storage medium of the present application stores computer instructions, and the computer instructions are for causing a computer to execute a method for controlling a smart rear-view mirror according to the present application.

The memory 1302 is a non-temporary computer-readable storage medium, such as a non-temporary software program such as a program instruction / module corresponding to a method for controlling a smart back mirror in an embodiment of the present application, or a non-temporary computer execution. It may be used to store possible programs and modules. Processor 1301 executes various functional applications and data processing of the server by executing non-temporary software programs, instructions and modules stored in memory 1302, ie the smartback in the embodiment of the above method. Realize a method for controlling the mirror.

The memory 1302 can include a program storage area and a data storage area, the program storage area can store an operating system, and an application program required for at least one function, and the data storage area is a smart back mirror. It is possible to store data created by the use of electronic devices to carry out methods for controlling the operating system. Memory 1302 can also include fast random access memory and also includes non-temporary memory such as at least one magnetic disk storage device, flash memory device, or other non-temporary solid-state storage device. You can also do it. In some embodiments, the memory 1302 can selectively include memory located remotely to the processor 1301, which remote memory is for controlling the smart rearview mirror over the network. It can be connected to an electronic device to carry out the method. Examples of the above networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

The electronic device of the method for controlling the smart rearview mirror can further include an input device 1303 and an output device 1304. The processor 1301, the memory 1302, the input device 1303, and the output device 1304 may be connected by a bus or other method, and FIG. 13 shows an example of a method in which the processor 1301, the memory 1302, the input device 1303, and the output device 1304 are connected by a bus.

The input device 1303 receives the input numerical or character information and generates an input of a key signal related to user setting of an electronic device for executing a method for controlling a smart back mirror, and a function control. These include input devices such as touch screens, keypads, mice, trackpads, touchpads, indicator rods, one or more mouse buttons, trackballs, and control handles. The output device 1304 can include a display device, an auxiliary lighting device (LED or the like), a tactile feedback device (vibration motor or the like), or the like. The display device can include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display, or a plasma display. In some embodiments, the display device may be a touch screen.

According to an embodiment of the present application, the present application further provides a computer program, the computer program is stored in a computer readable storage medium, and at least one processor of the electronic device stores the computer program from the computer readable storage medium. The at least one processor, which can be read, causes the electronic device to perform the means according to any of the above embodiments by executing the computer program.

The systems and various technical embodiments described herein are realized by digital electronic circuits, integrated circuit systems, application specific ASICs (ASICs), computer hardware, firmware, software, and / or combinations thereof. May be done. The various embodiments thereof are implemented in one or more computer programs, the one or more computer programs which can be executed and / or interpreted in a programmable system including at least one programmable processor. The programmable processor may be an application-specific or general purpose programmable processor, receiving data and instructions from a storage system, at least one input device, and at least one output device, and data and instructions. Can be transmitted to the storage system, the at least one input device, and the at least one output device.

These computer programs (also called programs, software, software applications, or code) include programmable processor machine instructions in high-level process and / or object-oriented programming languages and / or assembly / machine languages. These computer programs can be implemented. As used herein, the terms "machine readable medium" and "computer readable medium" are any computer program products, devices for providing machine instructions and / or data to a programmable processor. And / or devices (eg, magnetic disks, optical disks, memories, programmable logic devices (PLDs)), including machine-readable media for receiving machine instructions, which are machine-readable signals. The term "machine readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

In order to provide interaction with the user, the systems and techniques described herein can be implemented on the computer, which computer is a display device for displaying information to the user (eg, a CRT (eg, CRT). It has a cathode line tube) or LCD (liquid crystal display) monitor) and a keyboard and a pointing device (for example, a mouse or a trackball), and the user can provide input to the computer by the keyboard and the pointing device. become. Other types of devices can also provide interaction with the user, eg, the feedback provided to the user is any form of sensing feedback (eg, visual feedback, auditory feedback, or tactile feedback). And can receive input from the user in any form (including sound input, voice input, or tactile input).

The systems and technologies described herein are computing systems that include back-end components (eg, data servers), or computing systems that include middleware components (eg, application servers), or computing systems that include front-end components (eg, application servers). For example, a user computer having a graphical user interface or WEB browser, through which the user can interact with embodiments of the systems and techniques described herein), or. It can be implemented in any combination computing system including such back-end components, middleware components, or front-end components. The components of the system can be interconnected via digital data communication of any form or medium (eg, a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

A computer system can include a client and a server. Clients and servers are usually far apart from each other and typically interact with each other over a communication network. The client-server relationship is generated by a computer program that runs on the corresponding computer and has a client-server relationship with each other.

Steps can be rearranged, added, and deleted using the various forms of flow described above. For example, each step described in the present application may be performed in parallel, sequentially, or in a different order, as long as the proposed technology disclosed in the present application can achieve the desired result. Well, this specification does not limit it.

The specific embodiments described above do not constitute a limitation on the scope of patent protection of the present application. As will be apparent to those skilled in the art, various modifications, combinations, sub-combinations, and substitutions can be made, depending on design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present application should be included in the scope of patent protection of the present application.

Claims (25)

A way to control smart rearview mirrors
Steps to get the interface display information of the smart rearview mirror,
A step of transmitting the interface display information to at least one external device communicably connected to the smart rear-view mirror.
It is a step of responding to the user operation information from the at least one external device and controlling the smart rearview mirror to perform the corresponding operation, and the user operation information is the user's operation information for the at least one external device. A method for controlling a smart rearview mirror, characterized in that it contains steps that are intended to describe the operation. The interface display information of the smart rear-view mirror is the interface display information of the application program in the smart rear-view mirror, and the step of controlling the smart rear-view mirror to perform a corresponding operation is
The method according to claim 1, further comprising a step of controlling the application program in the smart rear-view mirror to perform a corresponding operation based on the user operation information. The step of controlling the application program in the smart rear-view mirror to perform a corresponding operation based on the user operation information is
A step of generating a trigger event based on the interface layout of the application program and the user operation information, and instructing the application program to perform a corresponding operation by the trigger event.
The method of claim 2, comprising: controlling the application program in the smart rearview mirror to perform a corresponding operation based on the trigger event. The user operation information includes the coordinate position of the user's operation on the current interface of the first external device, and the first external device is any device of the at least one external device.
The step of generating a trigger event based on the interface layout of the application program and the user operation information is
A step of converting the coordinate position of the user's operation on the current interface of the first external device to the coordinate position of the smart rearview mirror on the current interface.
The method of claim 3, comprising: generating the trigger event based on the interface layout of the application program and the coordinate position of the smart rearview mirror in the current interface. The step of generating the trigger event is based on the interface layout of the application program and the coordinate position of the smart rearview mirror on the current interface.
Based on the interface layout of the application program, the step of determining the operation control corresponding to the coordinate position of the smart rearview mirror in the current interface, and
The method according to claim 4, further comprising a step of generating the trigger event based on the operation control. The user operation information also includes the operation operation of the user, and the step of generating the trigger event based on the interface layout of the application program and the coordinate position of the smart rearview mirror at the current interface of the smart rearview mirror.
The method according to claim 4, further comprising a step of generating the trigger event based on the interface layout of the application program, the coordinate position of the smart rearview mirror at the current interface, and the operation operation of the user. .. The step of converting the coordinate position of the user's operation on the current interface of the first external device to the coordinate position of the smart rearview mirror on the current interface is
Displayed on the first external device based on the coordinate position of the user's work on the current interface of the first external device and the size of the interface of the application program displayed on the first external device. Steps to determine the relative position of the user's operation on the interface of the application program
4. The fourth aspect of the present invention includes a step of determining a coordinate position of the smart rear-view mirror at the current interface based on the relative position and the size of the interface of the application program displayed on the smart rear-view mirror. The method according to any one of claims 6. The step of determining the relative position of the user's operation on the interface of the application program displayed on the first external device is
The first external device is based on the coordinate position of the user's operation on the current interface of the first external device and the coordinate position and size of the interface of the application program displayed on the first external device. Including the step of determining the relative position of the user's operation on the interface of the application program displayed in.
Correspondingly, the step of determining the coordinate position of the smart rearview mirror on the current interface is
A claim comprising the step of determining the coordinate position of the smart rearview mirror at the current interface based on the relative position and the coordinate position and size of the interface of the application program displayed on the smart rearview mirror. 7. The method according to 7. The application program is plural, and the step of generating a trigger event based on the interface layout of the application program and the user operation information is described.
The steps to determine the target application program displayed on the current interface of the first external device,
The method according to any one of claims 3 to 8, further comprising a step of generating the trigger event based on the interface layout of the target application program and the user operation information. The step of determining the target application program displayed on the current interface of the first external device is
The method according to claim 9, wherein the method includes a step of setting the application program displayed on the current interface of the smart rearview mirror as the target application program. A switching control is also displayed on the current interface of the first external device, which instructing and correspondingly switching the target application program displayed on the current interface of the first external device. The user operation information includes the coordinate position of the switching control on the current interface of the first external device and the indicator of the switched application program, and the trigger event is the switched application program from the target application program. Instruct to switch to
The step of controlling the application program in the smart rearview mirror to perform a corresponding operation based on the trigger event is
The method according to claim 9 or 10, further comprising a step of controlling the target application program in the smart rearview mirror to switch to the switched application program based on the trigger event. Before the step of getting the interface display information of the smart rearview mirror, also
The method according to any one of claims 2 to 8, wherein the method includes a step of creating a projection process of the application program. Before the step of getting the interface display information of the smart rearview mirror,
Including the step of receiving the projection instruction output by the user
The interface display information of the application program is to record a video to be projected by the application program, and the step of acquiring the interface display information of the smart rearview mirror is
12. The method of claim 12, comprising recording the interface of the application program in the smart rearview mirror and obtaining the video to be projected. Before the step of recording the interface of the application program in the smart rearview mirror, also
Including the step of saving the state of the application program as a projected state.
The method also
13. The method of claim 13, wherein when the projection of the application program ends, the projection state of the application program is modified to the end state to end the projection process of the application program. A way to control smart rearview mirrors
Steps to receive interface display information from the projection control device,
Based on the interface display information, the step of displaying the projected interface in the smart rearview mirror and
When a user operation on the interface is detected, a step of generating user operation information and a step
A method for controlling a smart rearview mirror, comprising: transmitting the user operation information to the projection control device. The method according to claim 15, wherein the interface display information of the smart rear-view mirror is interface display information of an application program in the smart rear-view mirror. If the user's operation on the interface is detected, the step of generating the user operation information is
The method according to claim 15 or 16, wherein the method includes a step of detecting a coordinate position of the user's operation on the current interface of the first external device and generating the user operation information. If the user's operation on the interface is detected, the step of generating the user operation information is
The method according to claim 15 or 16, wherein the method includes a step of detecting a coordinate position and an operation operation of the user's operation on the current interface of the first external device and generating the user operation information. .. The application program is plural, and a switching control is also displayed on the current interface of the first external device, and the switching control switches the target application program displayed on the current interface of the first external device. 16. The method of claim 16, wherein the user operation information includes a coordinate position of the switching control on the current interface of the first external device and a indicator of the switched application program. The method also
A step of receiving a video of a projection target from the projection control device after being connected to the projection control device, wherein the projection target video is a video obtained by recording an interface of the application program. When,
16. The method of claim 16, comprising displaying an interface of an application program in the smart rearview mirror based on the projected video. A device for controlling smart rearview mirrors
A processing module for acquiring the interface display information of the smart rearview mirror,
Includes a transmit / receive module for transmitting the interface display information to at least one external device communicably connected to the smart rearview mirror.
The processing module is also used to respond to user operation information from the at least one external device and control the smart rearview mirror to perform the corresponding operation, and the user operation information is the at least one. A device for controlling a smart rearview mirror, which is intended to describe a user's operation on two external devices. A device for controlling smart rearview mirrors
A transmission / reception module for receiving interface display information from the projection control device,
Based on the interface display information, a display module for displaying the projected interface in the smart rear-view mirror, and
When a user operation on the interface is detected, a processing module for generating user operation information is included.
The transmission / reception module is a device for controlling a smart rear-view mirror, which is used for transmitting the user operation information to the projection control device. It ’s an electronic device,
Includes at least one processor and a memory and output device communicatively connected to the at least one processor.
An instruction that can be executed by the at least one processor is stored in the memory, and when the instruction is executed by the at least one processor, the electronic device has any one of claims 1 to 20. An electronic device capable of performing the method according to paragraph 1, wherein the output device is for displaying an interface of the electronic device. A non-temporary computer-readable storage medium in which computer instructions are stored, wherein the computer instructions are for causing a computer to perform the method according to any one of claims 1 to 20. A non-temporary computer-readable storage medium characterized by. A computer program, characterized in that, when executed by a processor, the computer program causes a computer to execute the method according to any one of claims 1 to 20.

Images