JP2021101356A - Equipment interaction method, device, equipment, system and medium - Google Patents

Abstract

To provide a main equipment interaction method, a device, equipment, a system and a medium.SOLUTION: A equipment interaction method includes: receiving an operation command sent by main control equipment and generating operation command by responding to touch operation of a user on a touch component, the operation command generation being performed by the main control equipment; writing an input command corresponding to the operation command into a command space of input equipment corresponding to the operation command; accessing the input command in the command space through driving of the input equipment so as to control an accessory equipment and responding to the input command. The accessory equipment is controlled through the touch operation in the main control equipment. In the command space of the input equipment corresponding to the control command, distinction with respect to different input equipment and input commands is materialized, a foundation is laid for generalized input of accessory equipment, the input command is accessed and the input command is responded in the command space through driving of the input equipment, thereby, control of the accessory equipment is materialized.SELECTED DRAWING: Figure 1

Description

The present application relates to communication technology, particularly to the field of in-vehicle device control, and specifically to device interaction methods, devices, devices, systems and media.

With the continuous increase in the demand for intelligence, it is usually necessary to fix an accessory device in one device to enrich the functions of the device. In general, the user is very inconvenient to operate the accessories because there is no screen or the screen is small.

In the prior art, the main control device, which is usually easy to operate, inputs a voice or gesture to generate an operation command, and the main control device sends it to an accessory device, whereby the accessory device responds to the operation command. Perform the operation to be performed.

However, the above-described embodiment is applied only when there is only one operable area, and when the accessory device includes a plurality of operable areas, the area currently required to be operated cannot be specified. Will fail.

The embodiments of the present application provide device interaction methods, devices, devices, systems and media to achieve generalized operation of ancillary devices in the process of interaction between the main control device and the ancillary devices.

According to a first aspect, an embodiment of the present application provides a device interaction method performed by an accessory, which method is a step of receiving an operational command transmitted from a main control device, said said. The operation command includes a step of writing the input command corresponding to the operation command in the command space of the input device corresponding to the operation command, which is generated when the main control device responds to the user's touch operation on the touch component. It comprises a step of accessing and responding to the input command in the command space by driving the input device so as to control the accessory.

In the embodiment of the present application, the accessory device receives an operation command generated by the main control device in response to the user's touch operation on the touch component, and corresponds to the operation command in the command space of the input device corresponding to the operation command. To access and respond to input commands in the command space by driving the input device, as if writing an input command and controlling ancillary equipment. The above technical proposal is to receive an operation command transmitted from the main control device by the accessory device and respond to the operation command, thereby performing a touch operation on the main control device in the process of interaction between the main control device and the accessory device. The operation control for the accessory device is realized, and at the same time, the accessory device writes the input command corresponding to the operation command in the command space of the input device corresponding to the operation command in the process of responding to the operation command. By realizing the distinction of input commands, laying the foundation for generalized input of accessory devices, and accessing and responding to input commands in the command space by driving the input devices, generalized control of the accessory devices can be achieved. Realize.

The step of accessing and responding to the input command in the command space by driving the input device optionally accesses the input command in the command space by driving the input device so as to control the accessory device. The step of analyzing the input command and the analysis result of the input command by driving the input device are delivered to the corresponding accessory application via the operating system of the accessory device and made to respond to the accessory application. Includes steps to control.

One optional embodiment in the above application is to access an input command access response operation by driving an input device, access the input command in the command space, analyze the input command, and analyze it via the operating system. The result is subdivided so that it is delivered to the corresponding attached application and responded, thereby realizing control over the attached application and improving the access response mechanism of the input command.

Optionally, the touch component comprises a hardware touch component and / or a software touch component, the operation command includes an operation event and / or input text, and correspondingly, the drive of the input device is a hardware touch component drive. , And / or include text input drive.

One optional embodiment in the above application is to manipulate events for ancillary devices via hardware touch components and / or software touch components by subdividing the drive of touch components, operating commands and input devices. At the same time as realizing control and / or text input control, the interaction form and interaction content between the main control device and the accessory device are enriched.

The step of accessing and responding to the input command in the command space by driving the input device so as to optionally control the accessory is the operation event from the command space by driving the hardware touch component. The text character of the input text from the command space by the step of acquiring the control character command of and updating the current focus on the display screen of the attached application based on the control character command, and / or by the text input drive. It includes a step of acquiring a command and controlling the attached application to display the input text from the current focus based on the text character command.

One optional practice in the above application is to perform an access response operation to an input command by driving a hardware touch component to update the focus position on the display screen of ancillary applications and / or by driving a text input. It is subdivided to realize control for text input in the attached application, and the control mechanism when the main control device controls the attached device is further improved.

Optionally, the step of receiving the operation command transmitted from the main control device includes the step of receiving the operation command transmitted from the transmission application of the main control device by the receiving application.

One optional embodiment in the above application is to receive an operation command transmitted from a transmission application of the main control device by a receiving application in the accessory device, thereby receiving a data interaction mode between the main control device and the accessory device. To improve.

After receiving the operation command transmitted from the transmission application of the main control device by the receiving application, before writing the input command corresponding to the operation command in the command space of the input device corresponding to the operation command. In addition, the method further includes the step of analyzing the operation command based on the Android® open accessory AOA protocol.

One optional embodiment in the above application is a communication mechanism between the main control device and the accessory device by analyzing the operation command by the AOA protocol and determining the corresponding input device based on the analysis result. To improve.

Arbitrarily, the step of writing the input command corresponding to the operation command in the command space of the input device corresponding to the operation command is a step of determining the corresponding input device based on the operation command and the operation. It includes a step of constructing an input command based on the command and a step of writing the input command in the command space of the input device.

One optional embodiment in the above application is to write an input command in the command space, determine the corresponding input device based on the operation command, construct the input command, and input it into the command space of the input device. By subdividing to write commands, it improves the input device determination mechanism and the input command writing mechanism, and provides a guarantee to distinguish different input commands from different input devices.

Optionally, the main control device is an in-vehicle infotainment system and the accessory device is an in-vehicle device.

One optional embodiment of the above application is to subdivide the main control device into an in-vehicle infotainment system and subdivide the accessory device into an in-vehicle device, thereby generalizing the in-vehicle infotainment system to the in-vehicle device. It is possible to realize generalized control over the in-vehicle device by realizing the control and providing convenience for the user to control the in-vehicle device, and at the same time, by operating the touch component in the in-vehicle infotainment system. Improve the user's operability.

According to a second aspect, the embodiments of the present application further provide a device interaction method performed by the main control device, which generates operational commands in response to a user's touch operation on the touch component. And the accessory device writes the input command corresponding to the operation command in the command space of the input device corresponding to the operation command, and accesses and responds to the input command in the command space by driving the input device. By this means, the step of transmitting the operation command to the accessory device so as to control the accessory device is included.

According to a third aspect, an embodiment of the present application further provides a device interaction device arranged in an accessory device, which device is a receiving module for receiving an operation command transmitted from a main control device. The operation command corresponds to the operation command in the receiving module generated when the main control device responds to the user's touch operation on the touch component and the command space of the input device corresponding to the operation command. It includes an input command writing module for writing an input command, and an input command response module for accessing and responding to the input command in the command space so as to control the accessory device.

According to a fourth aspect, an embodiment of the present application further provides a device interaction device located in a main control device, which device generates operation commands in response to a user's touch operation on a touch component. The operation command generation module for the operation and the accessory device write the input command corresponding to the operation command in the command space of the input device corresponding to the operation command, and the input device drives the input command in the command space. It includes an operation command transmission module for transmitting the operation command to the accessory so as to control the accessory by accessing and responding.

According to a fifth aspect, an embodiment of the present application further provides an accessory, the accessory comprising at least one processor and a memory communicatively connected to the at least one processor. Of these, the memory stores commands that can be executed by the at least one processor, and the commands can execute the device interaction method provided by the at least one processor in the embodiment of the first embodiment. As described above, it is executed by at least one processor.

According to a sixth aspect, an embodiment of the present application further provides a main control device, comprising at least one processor and a memory communicatively connected to the at least one processor, of which the said. The memory stores commands that can be executed by the at least one processor, and the commands are such that the at least one processor can execute the device interaction method provided in the embodiment of the second embodiment. Runs on at least one processor.

According to a seventh aspect, an embodiment of the present application further provides an interaction system, which comprises a main control device and ancillary devices, wherein the main control device is a user's touch operation on a touch component. In response to, the operation command is generated and the operation command is transmitted to the accessory device, and the accessory device writes the input command corresponding to the operation command in the command space of the input device corresponding to the operation command. In addition, the input device is driven to access and respond to the input command in the command space so as to control the accessory device.

According to an eighth aspect, the embodiments of the present application further provide a non-temporary computer-readable storage medium in which computer commands are stored, and when the computer commands are executed, the first aspect. The examples or the examples in the second aspect are carried out.
According to a ninth aspect, the embodiment of the present application further provides a computer program, and when an instruction in the computer program is executed, the embodiment in the first aspect or the embodiment in the second aspect is executed. To.

Other effects of the above optional form will be described below in combination with specific examples.

The drawings are used to better understand the proposed technology and are not intended to limit the application.
It is a flowchart of the device interaction method which concerns on Example 1 of this application. It is a flowchart of the device interaction method which concerns on Example 2 of this application. It is a flowchart of the device interaction method which concerns on Example 3 of this application. It is a flowchart of the device interaction method which concerns on Example 4 of this application. It is a flowchart of the device interaction method which concerns on Example 5 of this application. It is a schematic diagram of the operation interaction between the vehicle-mounted infotainment system and the vehicle-mounted device according to the fifth embodiment of the present application. It is a block diagram of the device interaction apparatus which concerns on Example 6 of this application. It is a block diagram of the device interaction apparatus which concerns on Example 7 of this application. It is a block diagram of the electronic device for realizing the device interaction method which concerns on embodiment of this application. It is a block diagram of the device interaction system which concerns on Example 9 of this application.

Hereinafter, exemplary examples of the present application are described in combination with the drawings, which include various details of the examples of the present application for ease of understanding, which are merely exemplary. Should be considered. Accordingly, one of ordinary skill in the art can make various changes and amendments to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and brevity, the following description omits the description of well-known functions and structures.

FIG. 1 is a flowchart of the device interaction method according to the first embodiment of the present application. The embodiments of the present application apply in the process of interaction between the main control device and the accessory device when the accessory device is controlled by the main control device, the method is executed by the device interaction device, and the device is software and / or It is realized by hardware and is specifically placed in the accessory device.

The device interaction method shown in FIG. 1 includes S101, S102, and S103.

S101, receives the operation command transmitted from the main control device. The operation command is generated when the main control device responds to the user's touch operation on the touch component.

Here, the touch component can be understood as an input device in the main control device, for example, a hardware touch component such as a key, a knob, or a joystick, or a software touch component such as a virtual key in a soft keyboard or a touch panel.

Optionally, the operational command includes an operational event that controls ancillary equipment. Alternatively, the operation command optionally includes input text for inputting text to the accessory application on the accessory device.

As an example, when the user performs a touch operation on a touch component in the input device of the main control device, the main control device generates an operation command and issues an operation command in response to the user's touch operation on the touch component. Send to the attached device. Correspondingly, the accessory device receives the operation command.

Optionally, the operating command sent from the main control device by the operating system of the accessory may be directly received. Or, optionally, to distinguish between the operation command generated by the user directly touching the accessory device and the operation command sent from the main control device so as to realize the direct reuse of the existing accessory device. In addition, a receiving application may be set in the accessory device, and an operation command transmitted from the main control device by the receiving application may be received.

S102, the input command corresponding to the operation command is written in the command space of the input device corresponding to the operation command.

As an example, the corresponding input device may be determined based on the operation command, the input command may be constructed based on the operation command, and the input command may be written in the command space of the input device.

Specifically, based on the operation command, an input device capable of executing the operation command is determined, and based on the operation command, an input command recognizable by the determined input device is constructed, and the determined command space is used. , Writes the constructed input commands, thereby distinguishing the input commands that can be processed by different input devices, realizing the sequential storage of the input commands, and laying the foundation for generalized control of the accessory devices by the main control device.

Here, the input device corresponding to the operation command can be understood as a hardware input device such as a key, a knob and a joystick in the accessory device, or a software input device such as a virtual key on a touch panel and a soft keyboard.

In addition, the operation command related to the input device of the main control device is sent to the accessory device, and the input command corresponding to the operation command is written in the command space of the input device of the accessory device corresponding to the operation command by the accessory device. This makes it possible to map the input operation of the input device of the main control device to the input device of the accessory device, and achieve the effect that the input device of the main control device replaces the input device of the accessory device in the same manner.

S103, the input device is driven to access and respond to the input command in the command space so as to control the accessory device.

As an example, the input device is driven to access the input command in the command space and the input command is analyzed, and the input device is driven to deliver the analysis result of the input command to the corresponding accessory application via the operating system of the accessory device. Control the attached application by making it respond.

Specifically, the input command is accessed in the command space by driving the input device, the input command is analyzed to obtain the corresponding command and content, and the command and content obtained by the analysis are used as the operating system of the attached device. The command and content obtained by analyzing by the operating system of the accessory device are delivered to the attached application, and the attached application receives and responds to the command and content sent from the operating system.

Optionally, the drive of the input device may be a hardware touch component drive corresponding to the hardware touch component, or optionally, the drive of the input device may further be a text input drive corresponding to the software touch component, etc. It may be included.

In one optional embodiment of the embodiments of the present application, when the drive of the input device includes a hardware touch component drive, the drive of the input device causes the input in the command space to control the accessory device. The step of accessing and responding to a command is to get the control character command of the operation event from the command space and update the current focus on the display screen of the attached application based on the control character command by driving the hardware touch component. There may be. If the above mode is adopted, the current focus on the display screen of the attached application in the attached device is updated by the touch operation executed in the hardware input device such as the key, knob or joystick in the main control device, and the current focus is changed accordingly. Operation control can be performed on the attached device at the focus position.

In another optional embodiment of the embodiment of the present application, when the drive of the input device includes a text input drive, the drive of the input device causes the input command in the command space to control the accessory. The step to access and respond to is driven by text input to get the text character command of the input text from the command space and control the attached application to display the input text from the current focus based on the text character command. It may be something to do. Here, the current focus is the focus corresponding to the current time on the display screen of the attached application. The focus may be a predetermined focus, a focus updated by a touch operation performed by the user on the accessory device, or a focus updated by a touch operation performed by the user on the touch component in the main control device. You may.

In the embodiment of the present application, the accessory device receives an operation command generated by the main control device in response to the user's touch operation on the touch component, and corresponds to the operation command in the command space of the input device corresponding to the operation command. To access and respond to input commands in the command space by driving the input device, as if writing an input command and controlling ancillary equipment. The above technical proposal is to receive an operation command transmitted from the main control device by the accessory device and respond to the operation command, thereby performing a touch operation on the main control device in the process of interaction between the main control device and the accessory device. The operation control for the accessory device is realized, and at the same time, the accessory device writes the input command corresponding to the operation command in the command space of the input device corresponding to the operation command in the process of responding to the operation command. By realizing the distinction of input commands, laying the foundation for generalized input of accessory devices, and accessing and responding to input commands in the command space by driving the input devices, generalized control of the accessory devices can be achieved. Realize.

Based on the technical proposal of each of the above embodiments, the main control device may be an in-vehicle infotainment system, and the accessory device may be an in-vehicle device, thereby between the in-vehicle infotainment system and the in-vehicle device. By realizing the interaction operation, the touch operation on the touch component in the in-vehicle infotainment system realizes generalized control over the in-vehicle device, providing convenience for the user to control the in-vehicle device, and at the same time, the user can use the in-vehicle infotainment. By operating the touch component of the infotainment system, generalized control over the in-vehicle device can be realized, and the user's operability is improved.

FIG. 2 is a flowchart of the device interaction method according to the second embodiment of the present application, and the embodiment of the present application is optimized and improved based on the technical proposal of each of the above embodiments.

Further, in order to improve the form of data interaction between the main control device and the accessory device, the operation of "receiving the operation command transmitted from the main control device" is "transmitted from the main control device by the receiving application". It is subdivided into "Receive operation commands".

The device interaction method shown in FIG. 2 includes S201, S202, and S203.

S201, the receiving application receives the operation command transmitted from the transmission application of the main control device. Here, the operation command is generated when the main control device responds to the user's touch operation on the touch component.

As an example, when the user performs a touch operation on a touch component in the input device of the main control device, the main control device generates an operation command in response to the user's touch operation on the touch component, and the main control device The operation command is transmitted to the corresponding transmission application by driving, and the operation command is transmitted to the accessory device by the transmission application.

Correspondingly, in order to distinguish between the operation command generated by the user directly touching the accessory and the operation command sent from the main control device so as to realize the direct reuse of the existing accessory. Further, the receiving application may be set in the accessory device, and the operation command transmitted from the transmission application of the main control device may be received by the receiving application.

Data can be optionally transmitted between the main control device and the accessory device in the form of USB (Universal Serial Bus) or a local network.

As an example, when the main control device and the accessory device communicate based on the AOA (Android (registered trademark) Open Access Protocol, Android (registered trademark) open accessory) protocol, the main control device sends an operation command to the accessory device. , The operation command is sent to the corresponding transmission application by driving the main control device, the operation command is acquired from the operating system of the main control device by the transmission application, the operation command is packaged based on the AOA protocol, and the data transmission channel. The package result is transmitted to the accessory device via. Correspondingly, the accessory device receives the operation command transmitted from the transmission application of the main control device by the receiving application, and analyzes the operation command based on the AOA protocol.

When the main control device and the accessory device communicate based on the AOA protocol, the accessory mode is entered, in which the main control device functions as a USB slave, and the accessory device functions as a USB host and supplies power to the bus. .. At this time, the accessory device is an android (registered trademark) accessory.

S202, the input command corresponding to the operation command is written in the command space of the input device corresponding to the operation command.

As an example, the corresponding input device is determined based on the analysis result of the operation command, the input command recognizable by the input device is constructed based on the analysis result of the operation command, and the input command is written in the command space of the input device.

S203, the input device is driven to access and respond to the input command in the command space so as to control the accessory device.

An embodiment of the present application subdivides receiving an operation command transmitted from a main control device into receiving an operation command transmitted from a transmission application of the main control device by a receiving application. At the same time as improving the reception mode of, by setting the receiving application and receiving the operation command sent from the main control device, it is possible to distinguish between the operation command generated by the accessory device itself and the operation command generated by the main control device. Realize, thereby realizing the reuse of accessories.

FIG. 3 is a flowchart of the device interaction method according to the third embodiment of the present application, and the embodiment of the present application is applied when the accessory device is controlled by the main control device in the process of interaction between the main control device and the accessory device. However, the method is performed in a device interaction device, which is implemented in software and / or hardware and specifically located in the main control device.

The device interaction method shown in FIG. 3 includes S301, S302, and S303.

S301, an operation command is generated in response to a user's touch operation on the touch component.

Here, the touch component can be understood as an input device in the main control device, for example, a hardware touch component such as a key, a knob, or a joystick, or a software touch component such as a virtual key in a soft keyboard or a touch panel.

Optionally, the operational command includes an operational event that controls ancillary equipment. Alternatively, the operation command optionally includes input text for inputting text to the accessory application on the accessory device.

As an example, if the touch component includes a hardware touch component, the step of generating an operation command in response to the user's touch operation on the hardware touch component is a control electrical signal in response to the user's touch operation on the hardware touch component. By converting the control electrical signal into a touch digital signal, converting the touch digital signal into an operation event based on a preset protocol, and generating an operation command including the operation event. There may be.

Specifically, after the user touch-operates the hardware touch component, the touch operation message is transmitted to the main control device by the CAN (Control Area Network) bus protocol, and here, the transmission message is transmitted by the CAN bus protocol. Is divided into data frames, remote frames, error frames, overload frames, etc. In each frame, the frame identification distinguishes each type of event. For example, when an electronic control unit such as an engine control system, an anti-lock system, and a seat adjustment system in an automobile is connected by a CAN bus in a vehicle, a light signal, a knob signal, and the like can be distinguished by frame identification. Here, the frame identification of the light signal is 0X555, and the frame identification of the knob signal is 0X566.

After receiving the electric signal of the CAN bus, the main control device converts the control electric signal into a touch digital signal, converts the digital signal into an analog event of the control system based on a preset protocol, and includes an operation event. Generate an operation command. Here, the analog event is an operation event, and may be at least one of a knob event, a key event, and the like.

As an example, when the touch component includes a software touch component, the step of generating an operation command in response to a touch operation on the user's software touch component is an input text in response to an input operation on the user's software touch component. It may generate an operation command including.

S302, when the accessory device writes an input command corresponding to the operation command in the command space of the input device corresponding to the operation command, and accesses and responds to the input command in the command space by driving the input device. , The operation command is transmitted to the accessory device so as to control the accessory device.

Optionally, the main control device may send operational commands to ancillary devices via its operating system.

Alternatively, optionally, in order to distinguish between different functions of the main control device, and further, based on the user's operation on the main control device, data is transmitted to the operating system of the main control device, or the main control device and the accessory device. The transmission application used to perform data transmission with may be set in the main control device. Correspondingly, the main control device sends an operation command to the accessory device by the transmission application.

In the embodiment of the present application, the main control device generates an operation command in response to the user's touch operation on the touch component, and the accessory device inputs the input command corresponding to the operation command in the command space of the input device corresponding to the operation command. By writing and driving the input device to access and respond to the input command in the command space, the operation command is sent to the accessory to control the accessory. The above technical proposal is in the process of interaction between the main control device and the accessory device by the user generating an operation command by touch operation on the touch component in the main control device and further controlling the accessory device based on the operation command. , The operation control for the accessory device is realized by the touch operation in the main control device, and at the same time, the accessory device is in the process of responding to the operation command, and the input command corresponding to the operation command is input to the command space of the input device corresponding to the operation command. By writing, thereby achieving the distinction between different input devices and input commands, laying the foundation for the generalized input of ancillary devices, and driving the input devices to access and respond to the input commands in the command space. Achieve generalized control over accessory devices.

Based on the technical proposal of each of the above embodiments, the main control device may be an in-vehicle infotainment system, and the accessory device may be an in-vehicle device, thereby between the in-vehicle infotainment system and the in-vehicle device. By realizing the interaction operation, the touch operation on the touch component in the in-vehicle infotainment system realizes generalized control over the in-vehicle device, providing convenience for the user to control the in-vehicle device, and at the same time, the user can use the in-vehicle infotainment. By operating the touch component of the infotainment system, generalized control over the in-vehicle device can be realized, and the user's operability is improved.

FIG. 4 is a flowchart of the device interaction method according to the fourth embodiment of the present application, and the embodiment of the present application is optimized and improved based on the technical proposal of each of the above embodiments.

Further, in order to improve the form of data interaction between the main control device and the accessory device, the operation of "sending the operation command to the accessory device" is changed to "a transmission application corresponding to the operation command by driving the main control device". And the operation command is transmitted to the accessory device by the transmission application. ”

The device interaction method shown in FIG. 4 includes S401, S402, and S403.

S401, an operation command is generated in response to a user's touch operation on the touch component.

The operation command is transmitted to the corresponding transmission application by driving S402, the main control device.

S403, when the accessory device writes an input command corresponding to the operation command in the command space of the input device corresponding to the operation command, and accesses and responds to the input command in the command space by driving the input device. , The operation command is transmitted to the accessory device via the transmission application so as to control the accessory device.

Optionally, the transmission application may be an additionally installed application program to facilitate data transmission between the main control device and the accessory device.

In the conventional main control device, when the user controls the operation of the main control device via the input device of the main control device, the user usually responds to the user's input operation according to the application program set in the main control device. By transmitting data to the operating system of the main control device, the unique function of the input device of the main control device can be realized, for example, the light can be controlled by operating a knob in a vehicle. Correspondingly, the transmission application realizes operation control for the accessory device by realizing data transmission between the main control device and the accessory device by reusing the application program set in itself.

When the transmission application is reused, the two can be distinguished by the operation mode of the transmission application in order to avoid confusion between the operation control for the main control device itself and the operation control for the accessory device.

As an example, when the transmission application is in the foreground operation mode, the operation command is transmitted to the corresponding transmission application by driving the main control device, whereby the operation command is transmitted to the accessory device by the transmission application. When the transmission application is in the background operating mode, the transmission application realizes its own unique function and does not realize data transmission between the main control device and the accessory device.

Optionally, the step of transmitting operational commands to ancillary devices by a transmission application can be implemented in the form of USB or local network transmission.

As an example, when transmitting data in the form of USB, the transmission application obtains the operation command from the operating system of the main control device, packages the operation command based on the AOA protocol, and outputs the package result via the data transmission channel. It can be transmitted to attached equipment. Correspondingly, the accessory device receives the operation command transmitted from the transmission application of the main control device by the receiving application, and analyzes the operation command based on the AOA protocol. Here, the data transmission channel may be an AOA connection channel, and is obtained by being constructed between the main control device and the accessory device after the main control device and the accessory device are communicably connected via USB.

When the main control device and the accessory device communicate based on the AOA protocol, the accessory mode is entered, in which the main control device functions as a USB slave, and the accessory device functions as a USB host and supplies power to the bus. .. At this time, the accessory device is an android (registered trademark) accessory.

In the embodiment of the present application, the operation command transmission process is subdivided so that the operation command is transmitted to the corresponding transmission application by driving the main control device, and the operation command is transmitted to the accessory device by the transmission application, whereby the main control is performed. The data transmission mode between the device and the accessory device is improved, the operation control for the main control device itself and the operation control for the accessory device are distinguished, and the phenomenon that the control is confused is avoided.

FIG. 5A is a flowchart of the device interaction method according to the fifth embodiment of the present application, and the embodiment of the present application is described in the technical proposal of each of the above embodiments so as to realize the control of the in-vehicle device by the in-vehicle infotainment system. Based on this, a preferred embodiment is provided. The description will be exemplified with reference to the schematic diagram of the operation interaction between the in-vehicle infotainment system and the in-vehicle device shown in FIG. 5B.

The device interaction method shown in FIG. 5A includes S501 to S517.

S501, an operation signal is generated in response to a knob operation performed by the user on the vehicle.

Here, the knob of the in-vehicle infotainment system is a touch component of the in-vehicle infotainment system and functions as an input device of the in-vehicle infotainment system.

S502, the knob control system converts the operation signal into a knob signal frame and transmits it in the form of an electric signal to the in-vehicle infotainment system via the CAN bus.

Here, the electrical signal is a voltage or current that changes with time, the operating signal generated by knob operation is a non-electrical signal, and such a signal is converted into an electrical signal by the sensor, time and high and low levels. By combining with, an electric signal can be formed to transmit a non-electrophysical quantity.

Here, the CAN bus is an ISO international standardized serial communication protocol and is used to connect each electric control unit in the vehicle (for example, an engine control system, an anti-lock system, a seat adjustment system, etc.). In the CAN bus protocol, transmission messages are divided into data frames, remote frames, error frames, overload frames, and the like. In each frame, each type of event is distinguished based on the frame identification (ID), for example, the frame ID of the write signal is 0X555, the frame ID of the knob signal is 0X566, and the like.

S503, After the in-vehicle infotainment system receives the electric signal of the CAN bus, the electric signal is converted into a digital signal.

Based on S504, the vehicle-mounted infotainment system protocol, the digital signal is converted into an analog event of the vehicle-mounted infotainment system.

Here, analog events are grouped into up, down, left, and right commands and used to characterize different operational events, such as knob events and key events.

S505, the input text is generated in response to the keyboard input operation.

S506, the in-vehicle infotainment system receives the input text from the keyboard of the in-vehicle infotainment system.

Here, the keyboard of the in-vehicle infotainment system is a touch component of the car and functions as an input device of the in-vehicle infotainment system. Here, the transmission application is in the foreground operating mode.

Here, the keyboard of the in-vehicle infotainment system may be a hardware keyboard or a software keyboard.

Here, S505 to S506 may be performed before S501 to S505, or after S501 to S505.

S507, drive and transfer the operating system of the in-vehicle infotainment system to deliver operational commands, including analog events and / or input text, to specific transmission applications.

S508, the transmission application of the in-vehicle infotainment system constructs an AOA connection channel between the in-vehicle infotainment system and the in-vehicle device.

Here, the in-vehicle device may be a device such as a smart speaker box and a smart rear-view mirror.

Here, in S508, when the in-vehicle infotainment system and the in-vehicle device are connected via USB, the AOA connection channel is constructed, and the operation is performed at any timing before data transmission is performed based on the AOA connection channel. Can be executed with.

S509, the transmission application acquires the operation command from the operating system and packages it in the AOA protocol data.

The packaged operation command is transmitted to the in-vehicle device via the S510 and AOA connection channels.

S511, the receiving application of the in-vehicle device analyzes the operation command based on the AOA protocol.

S512, the character device node in the in-vehicle device is determined based on the analysis result.

Here, the character device node includes hardware touch component drive and text input drive. For example, an operation command including a knob event is generated by a knob operation, and the character device node is a knob drive "/ dev / input / event1" correspondingly.

S513, Build a character command that can be recognized by the character device node based on the analysis result.

For example, the character command "1 330 1" corresponding to the operation command of the knob event can be mentioned.

S514, Write a character command in the command space of the character device node.

For example, the character command "1 330 1" is written in the knob drive "/ dev / input / event 1", and "/ dev / input / event 1 1 330 1" is acquired.

S515, the character device drive analyzes the command and the content in the character command.

Here, the analysis result includes a focus update command corresponding to the knob operation and an input text corresponding to the text input operation.

S516, the analysis result is delivered to the in-vehicle application via the operating system of the in-vehicle device.

S517, the in-vehicle application changes the current focus on the display screen of the in-vehicle application based on the received focus update command, and / or displays the text content from the current focus based on the received input text.

FIG. 6 is a block diagram of the device interaction device according to the sixth embodiment of the present application. The embodiments of the present application apply to the case where the main control device controls the accessory device in the process of interaction between the main control device and the accessory device, and the device is realized by software and / or hardware, and specifically. It is placed in the accessory equipment.

The device interaction device 600 shown in FIG. 6 includes an operation command receiving module 601, an input command writing module 602, and an input command response module 603.

Here, the operation command receiving module 601 is used to receive the operation command transmitted from the main control device, and the operation command is generated when the main control device responds to the user's touch operation on the touch component. , The input command writing module 602 is used to write the input command corresponding to the operation command in the command space of the input device corresponding to the operation command, and the input command response module 603 controls the accessory device. Is used to access and respond to the input command in the command space.

In the embodiment of the present application, the operation command is received by the main control device in response to the user's touch operation by the operation command receiving module, and the operation command is entered in the command space of the input device corresponding to the operation command by the input command writing module. Write the corresponding input command and access and respond to the input command in the command space with the input command response module to control the accessory. The above technical proposal is to receive an operation command transmitted from the main control device by the accessory device and respond to the operation command, thereby performing a touch operation on the main control device in the process of interaction between the main control device and the accessory device. The operation control for the accessory device is realized, and at the same time, the accessory device writes the input command corresponding to the operation command in the command space of the input device corresponding to the operation command in the process of responding to the operation command. By realizing the distinction of input commands, laying the foundation for generalized input of accessory devices, and accessing and responding to input commands in the command space by driving the input devices, generalized control of the accessory devices can be achieved. Realize.

Further, the input command response module 603 specifically accesses the input command in the command space, analyzes the input command, and outputs the analysis result of the input command to the corresponding accessory application via the operating system of the accessory device. It is used to control the attached application by delivering and responding to.

Further, the touch component comprises a hardware touch component and / or a software touch component, the operation command includes an operation event and / or an input text, and correspondingly, an input command response module 603 is driven by a hardware touch component and / or an input text. / Or includes text input drive.

Further, the input command response module 603 specifically acquires the control character command of the operation event from the command space, and updates the current focus on the display screen of the attached application based on the control character command. And / or used to obtain a text character command of the input text from the command space and, based on the text character command, control the ancillary application to display the input text from the current focus.

Further, the operation command receiving module 601 is specifically used to receive the operation command transmitted from the transmission application of the main control device.

Further, in the apparatus, after the operation command receiving module 601 receives the operation command transmitted from the transmission application of the main control device, the input command writing module 602 enters the command space of the input device corresponding to the operation command. An operation command analysis module for analyzing the operation command is further provided based on the Android (registered trademark) open accessory AOA protocol before writing the input command corresponding to the operation command.

Further, the input command writing module 602 specifically determines a corresponding input device based on the operation command, constructs an input command based on the operation command, and inputs the input to the command space of the input device. Used to write commands.

Further, the main control device is an in-vehicle infotainment system, and the accessory device is an in-vehicle device.

The device interaction device can execute the device interaction method provided in any embodiment of the present application, and has a beneficial effect with a functional module corresponding to the execution of the device interaction method.

FIG. 7 is a configuration diagram of the device interaction device according to the seventh embodiment of the present application. The embodiments of the present application apply to the case where the main control device controls the accessory device in the process of interaction between the main control device and the accessory device, and the device is realized by software and / or hardware, and specifically. It is placed in the accessory equipment.

The device interaction device 700 shown in FIG. 7 includes an operation command generation module 701 and an operation command transmission module 702.

Here, the operation command generation module 701 is used to generate an operation command in response to a user's touch operation on the touch component, and the operation command transmission module 702 is an input in which the accessory device corresponds to the operation command. The operation command so as to control the accessory device by writing the input command corresponding to the operation command in the command space of the device and accessing and responding to the input command in the command space by driving the input device. Is used to send to ancillary equipment.

In the embodiment of the present application, the operation command generation module generates the operation command in response to the user's touch operation on the touch component, and the attached device generates the operation command in the command space of the input device corresponding to the operation command. Is written, and the operation command is transmitted to the accessory device via the operation command transmission module so as to control the accessory device by accessing and responding to the input command in the command space by driving the input device. The above technical proposal is in the process of interaction between the main control device and the accessory device by the user generating an operation command by touch operation on the touch component in the main control device and further controlling the accessory device based on the operation command. , The operation control for the accessory device is realized by the touch operation in the main control device, and at the same time, the accessory device is in the process of responding to the operation command, and the input command corresponding to the operation command is input to the command space of the input device corresponding to the operation command. By writing, thereby achieving the distinction between different input devices and input commands, laying the foundation for the generalized input of ancillary devices, and driving the input devices to access and respond to the input commands in the command space. Achieve generalized control over accessory devices.

Further, the operation command transmission module 702 is specifically used to transmit the operation command to the corresponding transmission application by driving the main control device, and to transmit the operation command to the accessory device by the transmission application.

Further, when the operation command transmission module 702 executes transmission of the operation command to the accessory device by the transmission application, specifically, the transmission application acquires the operation command from the operating system of the accessory device. Android® Open Accessory Based on the AOA protocol, it is used to package the operating commands and transmit the packaged results to the ancillary devices via a data transmission channel.

Further, the device further includes a data transmission channel construction module for constructing a data transmission channel between the main control device and the accessory device before transmitting the package result to the accessory device via the data transmission channel. ..

Further, when the operation command transmission module 702 executes transmission of the operation command to the corresponding transmission application by driving the main control device, specifically, when the transmission application is in the foreground operation mode, the main control device. It is used to send the operation command to the corresponding transmission application by driving.

Further, the touch component includes a hardware touch component and / or a software touch component.

Further, when the touch component includes a hardware touch component, the operation command generation module 701 specifically generates a control electric signal in response to a touch operation on the user's hardware touch component, and generates the control electric signal. It is used to convert to a touch digital signal, convert the touch digital signal into an operation event based on a preset protocol, and generate an operation command including the operation event.

Further, the operation event includes a knob event and / or a key event.

Further, when the touch component includes a software touch component, the operation command generation module 701 is specifically used to generate an operation command including input text in response to an input operation on the software touch component of the user. ..

Further, the main control device is an in-vehicle infotainment system, and the accessory device is an in-vehicle device.

The device interaction device can execute the device interaction method provided in any embodiment of the present application, and has a beneficial effect with a functional module corresponding to the execution of the device interaction method.

Based on the examples of this application, this application further provides electronic devices and readable storage media. Here, the electronic device may be a main control device, or may be an accessory device.

As shown in FIG. 8, it is a block diagram of an electronic device for realizing the device interaction method according to the embodiment of the present application. Electronic devices are intended to represent various forms of digital computers such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices can also represent various forms of mobile devices such as personal digital assistants, mobile phones, smartphones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the realization of the present application as described and / or required herein.

As shown in FIG. 8, the electronic device includes one or more processors 801 and memory 802, and an interface including a high-speed interface and a low-speed interface for connecting each member. The members are connected to each other by different buses and may be mounted on a common motherboard or, if necessary, in other forms. The processor can process commands executed within the electronic device, which is a memory for displaying GUI graphic information on an external input / output device (for example, a display device coupled to an interface). Contains commands stored in or in memory. In other embodiments, a plurality of processors and / or a plurality of buses may be used in combination with a plurality of memories and a plurality of memories, if necessary. Similarly, a plurality of electronic devices can be connected, and each device can provide some necessary operations (eg, a server array, a group of blade servers, or a multiprocessor system). In FIG. 8, one processor 801 is taken as an example.

Memory 802 is a non-temporary computer-readable storage medium provided in this application. Here, the memory stores a command that can be executed by the at least one processor so that the at least one processor executes the device interaction method provided in the present application. The non-temporary computer-readable storage medium of the present application stores computer commands, which are used by the computer to perform the device interaction methods provided in the present application.

The memory 802 is a non-temporary computer-readable storage medium such as a non-temporary software program, a non-temporary computer-executable program and module, for example, a program command / module corresponding to the device interaction method according to the embodiment of the present application. For example, it is used to store the operation command receiving module 601 shown in FIG. 6, the input command writing module 602, the input command response module 603, or the operation command generation module 701 and the operation command transmission module 702 shown in FIG. The processor 801 executes various types of functional applications and data processing of the server by executing non-temporary software programs, commands and modules stored in the memory 802, that is, the device interaction method according to the embodiment of the above method. Realize.

The memory 802 can include a program storage area and a data storage area, where the program storage area can store an operating system, an application program required for at least one function, and the data storage area is a data storage area. It is possible to store data created by using an electronic device that realizes a device interaction method. The memory 802 may also include high speed random access memory and may further include non-temporary memory, eg, at least one disk storage device, flash memory, or other non-temporary solid state storage device. Can be mentioned. In some embodiments, the memory 802 optionally comprises memory provided remotely to the processor 801 and these remote memories are connected via a network to an electronic device that implements a device interaction method. be able to. Examples of the above networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks and combinations thereof.

The electronic device that realizes the device interaction method can further include an input device 803 and an output device 804. The processor 801 and the memory 802, the input device 803, and the output device 804 can be connected by a bus or another form, and in FIG. 8, the connection by the bus is taken as an example.

The input device 803 can receive the input number or character information and generate a key signal input related to user setting and function control of the electronic device that realizes the device interaction method, for example, a touch screen, a keyboard, and a mouse. , Trackpads, touch panels, pointing sticks, one or more mouse buttons, trackballs, joysticks and other input devices. The output device 804 can include a display device, an auxiliary lighting device (for example, an LED), a tactile feedback device (for example, a vibration motor), and the like. The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display, and a plasma display. In some embodiments, the display device may be a touch screen.
According to the examples of the present application, a computer program is provided. When an instruction in the computer program is executed, the device interaction method of the embodiment of the present application is executed.

Various embodiments of the systems and techniques described herein include digital electronic circuit systems, integrated circuit systems, application specific ASICs (ASICs), computer hardware, firmware, software, and / or them. It can be realized by the combination of. These various embodiments include performing in one or more computer programs, the one or more computer programs being executed and / or interpreted in a programmable system comprising at least one programmable processor. The programmable processor may be a dedicated or general purpose programmable processor, receiving data and commands from a storage system, at least one input device, and at least one output device, and delivering the data and commands to the storage system. It can be transmitted to the at least one input device and the at least one output device.

These computing programs (also called programs, software, software applications, or code) include machine commands for programmable processors, and these computing in advanced process and / or object-oriented programming languages and / or assembly / machine languages. The program can be implemented. As used herein, the terms "machine readable medium" and "computer readable medium" are any computer program products, devices for providing machine commands and / or data to programmable processors. And / or devices (eg, magnetic disks, optical disks, memories, programmable logic devices (PLDs)), including machine-readable media that receive machine commands, which are machine-readable signals. The term "machine readable signal" refers to any signal for providing machine commands and / or data to a programmable processor.

In order to provide interaction with the user, the systems and techniques described herein can be implemented on the computer, which computer is a display device for displaying information to the user (eg, CRT). It has a tube) or LCD (liquid crystal display) monitor) and a keyboard and pointing device (eg, mouse or trackball), and the user can provide input to the computer by the keyboard and the pointing device. Other types of devices can also provide interaction with the user, eg, the feedback provided to the user is any form of sensing feedback (eg, visual feedback, auditory feedback, or tactile feedback). It is also possible to receive input from the user in any form (including acoustic input, voice input, and tactile input).

The systems and techniques described herein are computing systems with back-end components (eg, data servers), or computing systems with middleware components (eg, application servers), or computing with front-end components. A system (eg, a user computer having a graphical user interface or web browser, and the user interacts with embodiments of the system and technology described herein by the graphical user interface or web browser), or such back. It can be implemented in a computing system that has any combination of end components, middleware components, and front end components. The components of the system can be interconnected by any form or medium of digital data communication (eg, a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), the Internet, and blockchain networks.

A computer system can include a client and a server. Clients and servers are generally separated from each other and typically interact over a communication network. A client-server relationship is created by a computer program that runs on the corresponding computer and has a client-server relationship with each other.

In the embodiment of the present application, the accessory device receives an operation command generated by the main control device in response to the user's touch operation on the touch component, and corresponds to the operation command in the command space of the input device corresponding to the operation command. To access and respond to input commands in the command space by driving the input device, as if writing an input command and controlling ancillary equipment. The above technical proposal is to receive an operation command transmitted from the main control device by the accessory device and respond to the operation command, thereby performing a touch operation on the main control device in the process of interaction between the main control device and the accessory device. The operation control for the accessory device is realized, and at the same time, the accessory device writes the input command corresponding to the operation command in the command space of the input device corresponding to the operation command in the process of responding to the operation command. By realizing the distinction of input commands, laying the foundation for generalized input of accessory devices, and accessing and responding to input commands in the command space by driving the input devices, generalized control of the accessory devices can be achieved. Realize.

FIG. 9 is a schematic configuration diagram of an equipment interaction system according to a ninth embodiment of the present application. The system includes a main control device 901 and an accessory device 902, and the main control device 901 is a user for a touch component. In response to the touch operation, an operation command is generated and the operation command is transmitted to the accessory device 902, and the accessory device 902 enters the input corresponding to the operation command in the command space of the input device corresponding to the operation command. The input device is driven to access and respond to the input command in the command space so as to write the command and control the accessory device 902.

Further, the main control device 901 can execute the device interaction method provided in the embodiment of the present application, and the accessory device 902 can execute the other device interaction method provided in the embodiment of the present application. Can be done.

It should be noted that the various forms of flow shown above can be used to newly order, add, or delete steps. For example, the steps described in this application may be performed in parallel, sequentially, or in a different order, but the techniques disclosed in this application. The present specification is not limited to this, as long as the desired result of the proposal can be achieved.

The specific embodiments described above do not limit the scope of protection of the present application. Those skilled in the art should understand that various modifications, combinations, subcombinations, and alternatives can be made depending on design requirements and other factors. Any amendments, equivalent substitutions, and improvements made within the spirit and principles of this application should all be within the scope of protection of this application.

Claims (25)

Performed by ancillary equipment,
A step of receiving an operation command transmitted from the main control device, wherein the operation command is a step generated by the main control device responding to a user's touch operation on a touch component.
A step of writing an input command corresponding to the operation command in the command space of the input device corresponding to the operation command, and a step of writing the input command corresponding to the operation command.
A step of accessing and responding to the input command in the command space by driving the input device so as to control the accessory.
Equipment interaction methods including. A step of accessing and responding to the input command in the command space by driving the input device so as to control the accessory.
A step of accessing an input command and analyzing the input command in the command space by driving the input device, and
A step of controlling the accessory application by delivering the analysis result of the input command to the corresponding accessory application via the operating system of the accessory device and causing the response by driving the input device.
The device interaction method according to claim 1. The touch component comprises a hardware touch component and / or a software touch component.
The operation command contains an operation event and / or input text.
Correspondingly, the device interaction method according to claim 1, wherein the drive of the input device includes a hardware touch component drive and / or a text input drive. A step of accessing and responding to the input command in the command space by driving the input device so as to control the accessory.
By driving the hardware touch component, a step of acquiring a control character command of the operation event from the command space and updating the current focus on the display screen of the attached application based on the control character command, and / or the text input. A step of driving to obtain a text character command of the input text from the command space and controlling the attached application to display the input text from the current focus based on the text character command.
3. The device interaction method according to claim 3. The device interaction method according to claim 1, wherein the step of receiving the operation command transmitted from the main control device includes the step of receiving the operation command transmitted from the transmission application of the main control device by the receiving application. After receiving the operation command transmitted from the transmission application of the main control device by the receiving application, and before writing the input command corresponding to the operation command in the command space of the input device corresponding to the operation command, the android (register). The device interaction method according to claim 5, wherein the operation command is analyzed based on the open accessory AOA protocol. The step of writing the input command corresponding to the operation command in the command space of the input device corresponding to the operation command is
Steps to determine the corresponding input device based on the operation command,
Steps to build input commands based on the above operation commands,
A step of writing the input command in the command space of the input device,
The device interaction method according to claim 1. The device interaction method according to any one of claims 1 to 7, wherein the main control device is an in-vehicle infotainment system, and the accessory device is an in-vehicle device. Performed by the main control device,
A step to generate an operation command in response to a user's touch operation on a touch component,
The accessory device writes the input command corresponding to the operation command in the command space of the input device corresponding to the operation command, and accesses and responds to the input command in the command space by driving the input device. A step of transmitting the operation command to the accessory device so as to control the device, and
Equipment interaction methods including. The step of sending the operation command to the attached device is
A step of transmitting the operation command to the corresponding transmission application by driving the main control device, and
The step of transmitting the operation command to the accessory device by the transmission application, and
9. The device interaction method according to claim 9. The step of transmitting the operation command to the accessory device by the transmission application is
The step of acquiring the operation command from the operating system of the main control device by the transmission application, and
Android® Open Accessory Based on the AOA protocol, the steps to package the operation commands and
The step of transmitting the package result to the accessory device via the data transmission channel,
10. The device interaction method according to claim 10. The device interaction method according to claim 11, further comprising a step of constructing a data transmission channel between the main control device and the accessory device before transmitting the package result to the accessory device via the data transmission channel. The step of transmitting the operation command to the corresponding transmission application by driving the main control device is a step of transmitting the operation command to the corresponding transmission application by driving the main control device when the transmission application is in the foreground operation mode. The device interaction method according to claim 10, which includes. The device interaction method according to claim 9, wherein the touch component includes a hardware touch component and / or a software touch component. When the touch module includes a hardware touch module, a step of generating an operation command in response to a user's touch operation on the hardware touch module is performed.
A step of generating a control electrical signal in response to a user's touch operation on the hardware touch module and converting the control electrical signal into a touch digital signal.
A step of converting the touch digital signal into an operation event and generating an operation command including the operation event based on a preset protocol.
The device interaction method according to claim 14. The device interaction method according to claim 15, wherein the operation event includes a knob event and / or a key event. When the touch component includes a software touch component, an operation in which a step of generating an operation command in response to a user's touch operation on the software touch component includes an input text in response to a user's input operation on the software touch component. The device interaction method according to claim 14, which comprises a step of generating a command. The device interaction method according to any one of claims 9 to 17, wherein the main control device is an in-vehicle infotainment system and the accessory device is an in-vehicle device. Placed in the accessory device,
A receiving module for receiving an operation command transmitted from the main control device, wherein the operation command includes a receiving module generated when the main control device responds to a user's touch operation on a touch component.
An input command writing module for writing an input command corresponding to the operation command in the command space of the input device corresponding to the operation command, and an input command writing module.
An input command response module for accessing and responding to the input command in the command space to control the accessory.
Equipment interaction device equipped with. Placed in the main control device,
An operation command generation module for generating operation commands in response to a user's touch operation on a touch component,
The accessory device writes the input command corresponding to the operation command in the command space of the input device corresponding to the operation command, and accesses and responds to the input command in the command space by driving the input device. An operation command transmission module for transmitting the operation command to the accessory device so as to control the device,
Equipment interaction device equipped with. With at least one processor
A memory that is communicatively connected to the at least one processor.
The memory stores a command that can be executed by the at least one processor, and when the command is executed by the at least one processor, the at least one processor according to any one of claims 1 to 8. Ancillary equipment that performs the described device interaction method. With at least one processor
A memory that is communicatively connected to the at least one processor.
The memory stores a command that can be executed by the at least one processor, and when the command is executed by the at least one processor, the at least one processor according to any one of claims 9 to 18. The main control device that performs the device interaction method described. Equipped with main control equipment and accessory equipment,
The main control device generates an operation command in response to the user's touch operation on the touch component, and sends the operation command to the accessory device.
The input in the command space is driven by the input device so that the accessory device writes the input command corresponding to the operation command in the command space of the input device corresponding to the operation command and controls the accessory device. An interaction system that accesses and responds to commands. A non-temporary computer-readable storage medium that stores computer commands.
When the computer command is executed, the computer command in which the device interaction method according to any one of claims 1 to 8 or the device interaction method according to any one of claims 9 to 18 is executed is executed. A stored non-temporary computer-readable storage medium. A computer program in which, when an instruction is executed, the device interaction method according to any one of claims 1 to 8 or the device interaction method according to any one of claims 9 to 18 is executed.

Images