WO2022095303A1

WO2022095303A1 - Method, computing device, and computer storage medium used for vehicle interaction

Info

Publication number: WO2022095303A1
Application number: PCT/CN2021/075741
Authority: WO
Inventors: 王庆坛; 龙涛; 俞铭
Original assignee: 博泰车联网(南京)有限公司
Priority date: 2020-11-04
Filing date: 2021-02-07
Publication date: 2022-05-12
Also published as: CN112040468A; CN112887954A; CN112887954B; CN112040468B

Abstract

The present disclosure relates to a method, computing device, and computer storage medium used for vehicle interaction. The method comprises: monitoring for an interconnection signal between a vehicle-mounted device of a vehicle and a user terminal in order to determine that a wireless communication connection has been established between the vehicle-mounted device and the user terminal; in response to detecting an operation action on an object mapped to a touch screen by the user terminal, obtaining information about the device attributes of the user terminal; if it is determined that the information about the device attributes of the user terminal belongs to a predetermined set, then generating first indication information; if it is determined that the information about the device attributes of the user terminal does not belong to a predetermined set, then sending a first test signal by way of an application program interface of the user terminal; if it is determined that no response signal is received from the user terminal for the first test signal within a first predetermined time interval, then generating and presenting at least one of the first indication information and second indication information. The present disclosure is capable of identifying a user terminal which does not support vehicle-mounted device reverse-control.

Description

Method, computing device, and computer storage medium for vehicle interaction

technical field

The present disclosure relates generally to information processing, and in particular, to methods, computing devices, and computer storage media for vehicle interaction.

Background technique

With the development of vehicle interconnection technology, the user terminal can be interconnected with the vehicle equipment, and the display information of the user terminal can be projected onto the touch screen of the vehicle equipment, so that the user can browse the screen display of the user terminal by using the touch screen of the vehicle equipment. . Sometimes, the user needs to operate the touch screen of the in-vehicle device to control the user terminal connected to the in-vehicle device. In the solution of the traditional method for vehicle interaction, there is a situation that the user terminal does not actually support reverse control by the vehicle device due to hardware configuration or software settings, but the vehicle device cannot recognize that it can be normally interconnected with the vehicle device, but does not A user terminal that supports reverse control of in-vehicle equipment.

Therefore, in the traditional solution for vehicle interaction, the in-vehicle device cannot identify the user terminal that does not support the anti-control of the in-vehicle device, and cannot assist the user to realize the interconnection method that supports the anti-control of the in-vehicle device.

SUMMARY OF THE INVENTION

The present disclosure provides a method, computing device and computer storage medium for vehicle interaction, which can identify user terminals that do not support reverse control of on-board equipment, and assist users to confirm whether the user terminal has an interconnection mode that supports reverse control of on-board equipment.

According to a first aspect of the present disclosure, there is provided a method for vehicle interaction. The method includes: monitoring the interconnection signal between the on-board device of the vehicle and the user terminal, so as to determine that the on-board device and the user terminal have established a wireless communication connection; The operation action of the object on the touch screen obtains the device attribute information of the user terminal; determines whether the device attribute information of the user terminal belongs to a predetermined set; and generates first indication information in response to determining that the device attribute information of the user terminal belongs to the predetermined set, the first The indication information is used to indicate that the user terminal does not support wireless manipulation by the touch screen; in response to determining that the device attribute information of the user terminal does not belong to the predetermined set, a first test signal is sent via the application program interface of the user terminal, the first test signal and the operation The action is associated; in response to determining that a response signal from the user terminal to the first test signal is not received within the first predetermined time interval, at least one of the first indication information and the second indication information is generated, and the second indication information is used to indicate The user terminal is to be connected to the in-vehicle device via a Universal Serial Bus (Universal Serial Bus, USB) connection; and at least one of the first indication information and the second indication information is presented at the touch screen of the in-vehicle device.

According to a second aspect of the present disclosure, there is provided a computing device comprising: a memory configured to store one or more computer programs; and a processor coupled to the memory and configured to execute the one or more programs to cause computing The apparatus performs the method of the first aspect of the present disclosure.

According to a third aspect of the present disclosure, there is also provided a non-transitory computer storage medium. The non-transitory computer storage medium has stored thereon machine-executable instructions that, when executed, cause a machine to perform the method of the first aspect of the present disclosure.

In some embodiments, generating at least one of the first indication information and the second indication information in response to determining that a response signal from the user terminal to the first test signal is not received within the first predetermined time interval includes: A response signal from the user terminal to the first test signal is not received within a predetermined time interval, determining whether the number of historical Bluetooth paired devices indicated in the Bluetooth pairing list of the user terminal is less than a predetermined number threshold; and in response to determining the Bluetooth When the number of historical Bluetooth paired devices indicated in the pairing list is less than a predetermined number threshold, at least one of the first indication information and the second indication information is generated.

In some embodiments, for the vehicle interaction comprises: in response to determining that the number of historical Bluetooth paired devices is greater than or equal to a predetermined number threshold, presenting information indicative of reducing the historical Bluetooth paired devices in the Bluetooth connection list; and in response to determining the history The number of Bluetooth paired devices is reduced to less than a predetermined number threshold, and the first test signal is sent again via the application program interface of the user terminal to determine whether a response from the user terminal to the first test signal is received within the first predetermined time interval Signal.

In some embodiments, the predetermined number threshold is seven.

In some embodiments, for vehicle interaction includes presenting a response signal in response to determining that a response signal from the user terminal to the first test signal is received within a first predetermined time interval.

In some embodiments, for vehicle interaction includes: in response to determining that the user terminal is connected to the vehicle-mounted device via a USB connection, generating a second test signal for transmission to the user terminal, the second test signal and the user terminal via a touch screen Control the association in a USB connection; and in response to determining that a response signal from the user terminal to the second test signal is not received within the second predetermined time interval, acquiring an image displayed on the touch screen of the in-vehicle device for determining what the image contains The indicated target object.

In some embodiments, the method for vehicle interaction includes: in response to determining that the target object indicated by the image meets a predetermined condition, generating a third indication signal for indicating that the user terminal cannot be operated via the touch screen in a USB connection; and in the vehicle The third indication information is presented on the touch screen of the device.

In some embodiments, determining that the target object indicated by the image meets a predetermined condition includes one of the following: determining that the matching degree of the target object indicated by the image and a predetermined template is greater than a predetermined matching degree threshold, and the predetermined template and the cursor Image correlation; a target object recognition model trained on multiple samples predicts that the class of target object indicated by the image is a cursor.

In some embodiments, for vehicle interaction includes: in response to determining that at least one of the first indication signal and the third indication signal is generated, adding device attribute information of the user terminal to a predetermined set, the predetermined set being obtained from the cloud server. This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary section is not intended to identify key features or essential features of the disclosure, nor is it intended to limit the scope of the disclosure.

Description of drawings

FIG. 1 shows a schematic diagram of a system for a method of vehicle interaction according to an embodiment of the present disclosure.

FIG. 2 shows a flowchart of a method for vehicle interaction according to an embodiment of the present disclosure.

FIG. 3 schematically shows a flowchart of a method for generating at least one of first indication information and second indication information according to an embodiment of the present disclosure.

FIG. 4 schematically shows a flowchart of a method 400 for presenting a reduced history Bluetooth paired device according to an embodiment of the present disclosure.

FIG. 5 schematically shows a flowchart of a method for identifying a user terminal supported by a USB connection in a reverse control manner according to an embodiment of the present disclosure.

FIG. 6 schematically shows a schematic structural diagram of a computing device for implementing the method of an embodiment of the present disclosure.

In the various figures, the same or corresponding reference numerals designate the same or corresponding parts.

Detailed ways

Preferred embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While preferred embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

As used herein, the term "including" and variations thereof mean open-ended inclusion, ie, "including but not limited to". The term "or" means "and/or" unless specifically stated otherwise. The term "based on" means "based at least in part on". The terms "one example embodiment" and "one embodiment" mean "at least one example embodiment." The term "another embodiment" means "at least one additional embodiment." The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

As described above, in the solution of the traditional method for vehicle interaction, there is a situation that the user terminal does not actually support the reverse control by the in-vehicle device due to hardware configuration or software settings, etc., but the in-vehicle device cannot recognize that it can normally communicate with A user terminal that is connected to in-vehicle equipment but does not support anti-control of in-vehicle equipment cannot help users to confirm whether there is an interconnection method that supports anti-control of in-vehicle equipment.

User Terminal To at least partially address one or more of the above problems and other potential problems, example embodiments of the present disclosure propose a solution for vehicle interaction. The solution includes: monitoring the interconnection signal between the in-vehicle device and the user terminal of the vehicle, so as to determine that the in-vehicle device and the user terminal have established a wireless communication connection; The operation action of the object on the screen, obtain the device attribute information of the user terminal; determine whether the device attribute information of the user terminal belongs to a predetermined set; in response to determining that the device attribute information of the user terminal belongs to the predetermined set, generate first indication information, the first indication The information is used to indicate that the user terminal does not support wireless manipulation by the touch screen; in response to determining that the device attribute information of the user terminal does not belong to the predetermined set, a first test signal is sent via the application program interface of the user terminal, the first test signal and the operation action associated; in response to determining that a response signal from the user terminal to the first test signal is not received within the first predetermined time interval, generating at least one of first indication information and second indication information, where the second indication information is used to indicate the user The terminal is to be connected to the in-vehicle device via a USB connection; at least one of the first indication information and the second indication information is presented at the touch screen of the in-vehicle device.

In the above solution, when it is determined that the in-vehicle device and the user terminal have been interconnected and an operation action on the touch screen of the in-vehicle device is detected for the screen-casting object of the user terminal, if it is determined that the acquired device attribute information of the user terminal belongs to a predetermined set, a first An indication message to indicate that the user terminal does not support wireless reverse control by the touch screen. The present disclosure can quickly identify user terminals that have been recorded and do not support vehicle-machine anti-control and indicate to the user. In addition, if it is determined that the device attribute information does not belong to the predetermined set, the first test signal associated with the operation action is sent via the application program interface of the user terminal; if it is determined that no response to the first test signal is received within the first predetermined time interval signal, generating and presenting information indicating that the user terminal does not support wireless manipulation by the touch screen, and/or information indicating that the user terminal is to be connected to the in-vehicle device via a USB connection. Therefore, even for a new type of user terminal that is not recorded, the present disclosure can accurately identify the user terminal that does not support the wireless reverse control of the in-vehicle device, and instruct the user terminal to try the USB reverse control. Therefore, the present disclosure can identify the user terminal that does not support the anti-control of the in-vehicle device, and assist the user to confirm whether the user terminal has an interconnection mode that supports the anti-control of the in-vehicle device.

FIG. 1 shows a schematic diagram of a system 100 for a method of vehicle interaction according to an embodiment of the present disclosure. As shown in FIG. 1 , the system 100 includes a vehicle 110 and its in-vehicle equipment 130 , a user terminal 120 , a cloud server 160 , a base station 150 , and a network 170 . In some embodiments, the vehicle 110 , the user terminal 120 , and the cloud server 160 may perform data interaction via, for example, the base station 150 and the network 170 .

Regarding the cloud server 160 , for example, it is used to store and update a predetermined set, and the predetermined set includes device attribute information of user terminals that do not support reverse control of each vehicle-mounted device 130 . The cloud server 160 may receive the device attribute information of the newly added user terminal that does not support reverse control sent by the in-vehicle device 130, and add the device attribute information of the newly added user terminal to a predetermined set. In some embodiments, the cloud server may have one or more processing units including, for example, a Graphics Processing Unit (GPU), a Field Programmable Gate Array (FPGA), and an Application-Specific Integrated Circuit (Application Specific Integrated Circuit, ASIC), etc., and general-purpose processing units such as CPUs. In addition, one or more virtual machines can be running on each server.

Regarding the vehicle 110 , it includes, for example, at least an in-vehicle device 130 (eg, an in-vehicle device) and its touch screen, an in-vehicle Bluetooth device, and the like.

Regarding the in-vehicle device 130, it is used for data interaction and sharing with the user terminal 120 through wireless or wired interconnection methods such as Bluetooth and USB. The in-vehicle device 130 is also configured with a high definition multimedia interface (High Definition Multimedia Interface, HDMI) interface, which can display content on the interface of the user terminal 120, for example, transmit the played video signal to the display of the in-vehicle device 130. Specifically, the in-vehicle device 130 can be used to monitor the interconnection signal between the in-vehicle device 130 and the user terminal 120, so as to determine that the two have established a wireless communication connection; When mapping to the operation action of the object on the touch screen, determine whether the acquired device attribute information of the user terminal belongs to the predetermined device set; if it belongs to the predetermined device set, it indicates that the user terminal does not support wireless manipulation by the touch screen; if it is determined If the user terminal does not belong to the predetermined equipment set, the first test signal associated with the operation action is sent via the application program interface of the user terminal, and if the corresponding response signal is not received within the first predetermined time interval, it indicates that the user terminal does not support the The touch screen controls wirelessly or indicates the in-vehicle device can be connected via a USB connection. The in-vehicle device 130 includes, for example, at least a wireless communication connection confirmation unit 132 , a device attribute information acquisition unit 134 , a predetermined set determination unit 136 , a first indication information generation unit 138 , a first test signal transmission unit 140 , and a response signal confirmation unit 142 .

Regarding the wireless communication connection confirmation unit 132 , it is used to monitor the interconnection signal between the in-vehicle device of the vehicle and the user terminal, so as to determine that the in-vehicle device and the user terminal have established a wireless communication connection.

Regarding the device attribute information acquisition unit 134, it is used to confirm whether an operation action applied to the touch screen of the in-vehicle device for an object mapped to the touch screen by the user terminal is detected; For the operation actions of the user terminal mapped to the object on the touch screen, the device attribute information of the user terminal is acquired.

Regarding the predetermined set determining unit 136, it is used to determine whether the device attribute information of the user terminal belongs to the predetermined device set.

Regarding the first indication information generating unit 138, if it is determined that the device attribute information of the user terminal does not belong to the predetermined device set, send a first test signal via the application program interface of the user terminal, the first test signal is associated with the operation action, and the first test signal is associated with the operation action. an indication message is used to indicate that the user terminal does not support manipulation by the touch screen in a wireless manner; and if it is determined that a response signal from the user terminal to the first test signal is not received within the first predetermined time interval, generating the first indication information and the second indication message At least one of the indication information, and the second indication information is used to indicate that the user terminal is to be connected to the in-vehicle device via a USB connection.

Regarding the first test signal sending unit 140, if it is determined that the user terminal does not belong to the predetermined device set, the first test signal is sent via the application program interface of the user terminal, and the first test signal is associated with the operation action.

Regarding the response signal confirmation unit 142, it is configured to determine whether a response signal of the user terminal to the first test signal is not received within the first predetermined time interval.

Regarding the user terminal 120, it is, for example, but not limited to, a mobile phone. The mobile device 120 may directly interact and share data with the in-vehicle device 130 through wireless or wired interconnection such as Bluetooth and USB. Data interaction with the server 160 may also be performed via the base station 150 and the network 170 . The user terminal 120 can project the displayed content onto the touch screen of the vehicle-mounted device 130 . When the user terminal 120 is a mobile phone, it may include an application program interface 122, a Bluetooth module 124, a video playing unit 126 and other units not shown. The user terminal 120 may be implemented by the computing device 600 shown in FIG. 6 .

Regarding the application program interface 122, it is, for example and not limited to, data interaction with in-vehicle devices. Specifically, the in-vehicle device 130 can, for example, obtain the device attribute information of the user terminal through the application program interface 122; and obtain the number of historical Bluetooth paired devices in the Bluetooth pairing list of the user terminal 120 through the application program interface 122.

Regarding the Bluetooth module 124, it is used for the user terminal 120 and the in-vehicle device 130 to perform Bluetooth connection and data interaction. The Bluetooth module 124 includes, for example, at least a protocol layer and underlying hardware modules. The protocol layer is composed of, for example, a Bluetooth-specific protocol formulated by the Bluetooth Special Interest Group (SIG). The underlying hardware modules include, for example, a radio frequency module, a baseband module, a link control module, a link management module, and a host controller interface. The radio frequency module (Radio) is used for data reception and transmission. The baseband module (BaseBand) is used for wireless media access conventions, providing synchronous connection-oriented physical links and asynchronous connectionless physical links. The Link Control (LC) module is used for encoding and decoding of Bluetooth data packets. The Link Management (LM) module is used to create, modify and publish logical links and update physical link parameters between devices. The Host Controller Interface (HCI) is used to provide a command interface for the baseband controller, connection manager, hardware status and control registers. For example, pairing and interconnection with the in-vehicle Bluetooth device of the in-vehicle device 130 is realized via the Bluetooth module 124 . The in-vehicle device 130 can obtain the identification of the user terminal 120 and the Bluetooth pairing password; and then configure the data connection and personal hotspot function of the user terminal 120 to be enabled, so that the in-vehicle device 130 can scan the user terminal 120 for interconnection.

Regarding the video playing unit 126, for example, it is used to play video files stored locally or obtained online by the user terminal. For example, the user terminal 120 supports the high-definition video and audio standard interface (Mobile High-Definition Link, MHL) function of the mobile terminal, and via the MHL function, the video signal being played by the user terminal 120 can be transmitted to the touch screen of the in-vehicle device 130. For example, the MHL line can be connected to the USB interface of the user terminal 120 and the HDMI interface of the in-vehicle device 130 at the same time to realize the screen mapping of the video signal played by the user terminal 120 to the touch screen of the in-vehicle device 130 .

A method for vehicle interaction according to an embodiment of the present disclosure will be described below with reference to FIG. 2 . FIG. 2 shows a flowchart of a method 200 for vehicle interaction according to an embodiment of the present disclosure. It should be appreciated that the method 200 may be performed, for example, at the computing device 600 depicted in FIG. 6 . It may also be performed at the in-vehicle device 130 described in FIG. 1 . It should be understood that the method 200 may also include additional actions not shown and/or the actions shown may be omitted, and the scope of the present disclosure is not limited in this regard.

At step 202, the in-vehicle device 130 monitors the interconnection signal between the in-vehicle device of the vehicle and the user terminal, so as to determine that the in-vehicle device and the user terminal have established a wireless communication connection.

At step 204 , the in-vehicle device 130 confirms whether an operation action applied to the touch screen of the in-vehicle device for an object mapped on the touch screen by the user terminal is detected.

Regarding the object mapped by the user terminal to the touch screen, it is, for example, but not limited to, the video player and the video being played by the user terminal 120 and projected to the touch screen of the in-vehicle device 130 .

Regarding the operation action, it is, for example, but not limited to, dragging the playback progress bar of the video player projected to the touch screen of the in-vehicle device 130 . The operation action may also be a click operation for an icon in the display interface of the user terminal 120 that is projected onto the touch screen of the in-vehicle device 130 .

At step 206 , if the in-vehicle device 130 detects an operation action applied to the touch screen of the in-vehicle device for the object mapped to the touch screen by the user terminal, the device attribute information of the user terminal is obtained.

Regarding the way of acquiring the device attribute information of the user terminal, it includes, for example: the in-vehicle device 130 acquires the record model of the user terminal via the user terminal application programming interface (Application Programming Interface, API).

At step 208, the in-vehicle device 130 determines whether the device attribute information of the user terminal belongs to a predetermined set.

Regarding the predetermined device set, it includes, for example, a set of device attribute information that does not support user terminals that are wirelessly controlled by each in-vehicle device. The device attribute information is, for example, the record model of the user terminal.

At step 210, if the in-vehicle device 130 determines that the device attribute information of the user terminal belongs to a predetermined set, first indication information is generated, and the first indication information is used to indicate that the user terminal does not support wireless manipulation by the touch screen.

At step 212, if the in-vehicle device 130 determines that the device attribute information of the user terminal does not belong to the predetermined set, a first test signal is sent via the application program interface of the user terminal, and the first test signal is associated with the operation action.

At step 214, the in-vehicle device 130 determines whether a response signal from the user terminal to the first test signal has not been received within the first predetermined time interval. If the in-vehicle device 130 determines that a response signal from the user terminal to the first test signal is received within the first predetermined time interval, it jumps to step 220 to present the response signal.

Regarding the response signal to the first test signal, it is, for example, a processing result of an operation action or an associated interaction signal. Research shows that the feedback information of the application program interface of the user terminal that does not support wirelessly controlled by various in-vehicle devices may indicate that the in-vehicle device 130 and the user terminal 120 are compatible and can normally perform wireless interconnection. However, the feedback information of the application program interface cannot reflect whether the user terminal does not support reverse control by the in-vehicle device. Studies have shown that the Bluetooth module of the user terminal 120 may not be able to support wireless control by various in-vehicle devices due to the incompatibility of the underlying hardware modules; In-vehicle devices are controlled wirelessly. Therefore, the in-vehicle device 130 can detect whether the processing result related to the operation action or the associated interaction signal is not received within the first predetermined time interval, and then can accurately determine whether the current user terminal 120 is in a state that does not support wireless manipulation by the in-vehicle device. state.

At step 216, if the in-vehicle device 130 determines that a response signal from the user terminal to the first test signal is not received within the first predetermined time interval, at least one of the first indication information and the second indication information is generated, and the second indication information It is used to indicate that the user terminal is to be connected to the in-vehicle device via the USB connection.

If the in-vehicle device 130 determines that a response signal from the user terminal to the first test signal is not received within the first predetermined time interval. The in-vehicle device 130 may determine that the current user terminal 120 is in a state that does not support wireless manipulation by various in-vehicle devices, and generate indication information (such as an indicator, or an indication icon), and/or generate indication information (for example, an indication word or an indication icon) for instructing to replace the user terminal 120 with a USB connection. The purpose of generating the instruction information for instructing to replace the user terminal 120 with a USB connection is that some user terminals 120 do not support being controlled by the in-vehicle device in a wireless manner, but support being controlled by the in-vehicle device in a USB connection.

By adopting the above-mentioned means, the present disclosure can remind the user to try whether to support the reverse control of the USB connection method when it is confirmed that the user terminal does not support the reverse control of the wireless method.

At step 218, at least one of the first indication information and the second indication information is presented at the touch screen of the in-vehicle device.

In some embodiments, the method 200 further includes: if the in-vehicle device 130 determines that at least one of the first indication signal and the third indication signal is generated, adding the device attribute information of the user terminal to the predetermined device set, and obtaining the predetermined device set from cloud server. Thus, the cloud server can update a predetermined set of device attribute information about user terminals that do not support reverse control by each vehicle device 130 at any time based on information fed back by multiple vehicle devices, so that each vehicle device can identify user terminals that do not support reverse control.

In some embodiments, the method 200 further includes a method 300 for generating at least one of the first indication information and the second indication information. FIG. 3 shows a flowchart of a method 300 for generating at least one of first indication information and second indication information according to an embodiment of the present disclosure. It should be understood that the method 300 may be performed, for example, at the computing device 600 depicted in FIG. 6 . It may also be performed at the in-vehicle device 130 described in FIG. 1 . It should be understood that the method 300 may also include additional actions not shown and/or the actions shown may be omitted, and the scope of the present disclosure is not limited in this regard.

At block 302, the in-vehicle device 130 determines that a response signal from the user terminal to the first test signal has not been received within a first predetermined time interval.

At block 304, the in-vehicle device 130 determines whether the number of historical Bluetooth paired devices indicated in the user terminal's Bluetooth pairing list is less than a predetermined number threshold.

After research, it is found that the reasons that cause the user terminal not to support the anti-control of in-vehicle equipment include hardware configuration and software configuration. The reasons for software configuration include, for example, if the user terminal has too many historical Bluetooth paired devices, even if the historical Bluetooth paired device is not currently connected, the function of the in-vehicle device to reversely control the user terminal cannot be realized. Therefore, if the in-vehicle device 130 detects that the number of historical Bluetooth paired devices exceeds a predetermined number threshold, the user can be prompted to delete the number of historical Bluetooth paired devices, so as to troubleshoot and avoid the situation that the reverse control is not supported due to software configuration reasons. . Regarding the predetermined number threshold, it is, for example, seven.

At block 306, if the in-vehicle device 130 determines that the number of historical Bluetooth paired devices indicated in the user terminal's Bluetooth pairing list is less than a predetermined number threshold, at least one of first indication information and second indication information is generated. If it is detected that the number of historical Bluetooth paired devices does not exceed the predetermined number threshold, the software configuration can be excluded, and the hardware of the user terminal can be determined and instructed that the in-vehicle device does not support the reverse control of the in-vehicle device by Bluetooth, or the user can be instructed at the same time to The user terminal is interconnected with the in-vehicle device in a USB way, so as to identify whether the hardware of the user terminal supports the in-vehicle device to perform reverse control in the way of USB connection.

In the above solution, the present disclosure can quickly identify and check whether there are user terminals that do not support reverse control of in-vehicle devices due to too many historical Bluetooth paired devices.

FIG. 4 shows a flowchart of a method 400 for presenting a reduced history Bluetooth paired device according to an embodiment of the present disclosure. It should be understood that the method 400 may be performed, for example, at the computing device 600 depicted in FIG. 6 . It may also be performed at the in-vehicle device 130 described in FIG. 1 . It should be understood that the method 400 may also include additional actions not shown and/or the actions shown may be omitted, and the scope of the present disclosure is not limited in this regard.

At step 402, if the in-vehicle device 130 determines that the number of historical Bluetooth paired devices is greater than or equal to a predetermined number threshold, indication information about reducing the historical Bluetooth paired devices in the Bluetooth connection list is presented.

The research found that if the user terminal has too many historical Bluetooth pairing devices, even if the historical Bluetooth pairing device is not currently connected, the function of the in-vehicle device to counter-control the user terminal cannot be realized. Therefore, if the in-vehicle device 130 determines that the number of historical Bluetooth paired devices is greater than or equal to 7, an indication message is presented on the touch screen of the in-vehicle device 130 to remind the user to partially delete the historical Bluetooth paired devices from the Bluetooth pairing list.

At step 404, the in-vehicle device 130 determines whether the number of historical Bluetooth paired devices has been reduced to less than a predetermined number threshold.

At step 406, if the in-vehicle device 130 determines that the number of historical Bluetooth paired devices is reduced to less than the predetermined number threshold, the first test signal is re-sent via the application program interface of the user terminal for determining whether within the first predetermined time interval A response signal of the user terminal to the first test signal is received.

For example, if the in-vehicle device 130 determines that the number of historical Bluetooth paired devices of the user terminal is less than 7, the first test signal associated with the aforementioned operation signal is sent again, so as to test again whether the user terminal whose Bluetooth pairing list has been adjusted supports the in-vehicle device reverse operation. control. If the in-vehicle device 130 determines that it still does not receive a response signal from the user terminal to the re-sent first test signal within the first predetermined time interval, it indicates that the user terminal belongs to hardware that does not support in-vehicle device anti-control.

In the above solution, the present disclosure can prompt the user to confirm whether it belongs to the user terminal that supports the anti-control of the in-vehicle device on the hardware by reducing the historical Bluetooth paired devices.

FIG. 5 shows a flowchart of a method 500 for identifying a user terminal supported by a USB connection in a reverse control manner according to an embodiment of the present disclosure. It should be appreciated that the method 500 may be performed, for example, at the computing device 600 depicted in FIG. 6 . It may also be performed at the in-vehicle device 130 described in FIG. 1 . It should be understood that the method 500 may also include additional actions not shown and/or the actions shown may be omitted, and the scope of the present disclosure is not limited in this regard.

At step 502, the in-vehicle device 130 determines whether the user terminal is connected to the in-vehicle device via a USB connection.

In some embodiments, if the in-vehicle device 130 determines that the user terminal is not connected to the in-vehicle device via a USB connection, step 502 is continued until it is determined that the user terminal has been connected to the in-vehicle device by a USB connection.

At step 504, if the in-vehicle device 130 determines that the user terminal is connected to the in-vehicle device via a USB connection, a second test signal for sending to the user terminal is generated, and the second test signal is related to the user terminal being operated via the touch screen in a USB connection. link.

Regarding the second test signal, it is associated with the operation action on the touch screen of the in-vehicle device mentioned in the aforementioned step 204 for the object mapped on the touch screen by the user terminal. The second test signal is different from the first test signal used for testing wireless reverse control.

At step 506, the in-vehicle device 130 determines whether a response signal from the user terminal to the second test signal has not been received within the second predetermined time interval.

At step 508, if the in-vehicle device 130 determines that a response signal from the user terminal to the second test signal has not been received within the second predetermined time interval, the image displayed on the touch screen of the in-vehicle device is acquired to determine the target.

Research has shown that if the user terminal cannot be manipulated via the touch screen with a USB connection, a cursor is usually displayed on the touch screen. Therefore, the in-vehicle device 130 can obtain the image displayed on the touch screen by means of screen copying or the like, and then identify the target object indicated by the image to determine whether the user terminal cannot be controlled by the USB connection via the touch screen.

At step 510, the in-vehicle device 130 determines whether the target object indicated by the image meets a predetermined condition. For example, determining that the target object indicated by the image meets the predetermined condition includes one of the following: determining that the matching degree of the target object indicated by the image and the predetermined template is greater than a predetermined matching degree threshold, and the predetermined template is associated with the cursor image; The target object recognition model predicts that the category of the target object indicated by the image is a cursor.

Regarding the predetermined condition, it is, for example, that the predicted class of the target object is a cursor. The predetermined condition may also be that the image including the target object matches a predetermined template (eg, a cursor image).

Regarding the way of determining the target object indicated by the image, it is determined whether the matching degree of the target object indicated by the image and the predetermined template is greater than a predetermined matching degree threshold through template matching, for example. The way to determine the target object indicated by the image may also be to predict whether the category of the target object is a cursor through a target object recognition model trained with multiple samples. The target object recognition model is constructed based on, for example, a neural network model (such as, but not limited to, a Convolutional Neural Network (CNN) model).

In some embodiments, the in-vehicle device 130 determines whether the matching degree of the target object indicated by the image and the predetermined template is greater than a predetermined matching degree threshold, for example, by template matching. Specifically, the in-vehicle device 130 may move the predetermined template pixel by pixel from left to right and from top to bottom on the acquired image to be recognized, and calculate the degree of matching between the predetermined template image and the overlapping sub-images of the image to be recognized. , the greater the matching degree, the greater the possibility that the predetermined template is the same as the overlapping sub-image.

Regarding the calculation method of the matching degree, the in-vehicle device 130 may use the squared difference to calculate the matching degree.

The method for calculating the matching degree by using the squared difference will be described below with reference to formula (1).

R(x,y)=∑ _x′,y′ (T(x′,y′)-I(x+x′,y+y′)) ² (1)

In the above formula (1), I(x+x', y+y') represents the acquired parameters of the touch screen display image to be recognized, and the parameters are, for example, 8-bit integers or 32-bit floating point numbers. x, y represent the pixels of the touch screen display image to be identified. T(x', y') represents parameters of a predetermined template (eg, a cursor template image). The predetermined template is smaller than the touch screen display image to be recognized and has the same data type. x', y' represent the pixels of the predetermined template image. R(x,y) represents the matching result.

In some embodiments, the in-vehicle device 130 may also calculate the matching degree through correlation matching calculation or standard correlation matching calculation through the multiplication operation between the predetermined template and the touch screen display image to be recognized.

At step 512, if the in-vehicle device 130 determines that the target object indicated by the image meets the predetermined condition, a third indication signal is generated for indicating that the user terminal cannot be manipulated via the touch screen in a USB connection manner.

At step 514, the in-vehicle device 130 presents the third indication information at the touch screen of the in-vehicle device.

In the above solution, the present disclosure can further identify whether the user terminal can be reverse-controlled by the vehicle-mounted device via the USB connection when it is confirmed that the user terminal cannot be reverse-controlled by the vehicle-mounted device via the Bluetooth method.

6 schematically illustrates a block diagram of a computing device 600 suitable for implementing embodiments of the present disclosure. The computing device 600 may be a device for implementing the methods 200 to 500 shown in FIGS. 2 to 5 . As shown in FIG. 6, the computing device 600 includes a central processing unit (Central Processing Unit, CPU) 601, which can be loaded according to computer program instructions stored in a read-only memory (Read-Only Memory, ROM) 602 or from a storage unit 608 Computer program instructions into random access memory (RAM) 603 to perform various appropriate actions and processes. In RAM 603, various programs and data required for the operation of computing device 600 may also be stored. The CPU 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. Input/output (I/O) interface 605 is also connected to bus 604 .

A number of components in the computing device 600 are connected to the I/O interface 605, including: an input unit 606, an output unit 607, and a storage unit 608, and the CPU 601 executes the various methods and processes described above, such as methods 200 to 500. For example, in some embodiments, methods 200 - 500 may be implemented as a computer software program stored on a machine-readable medium, such as storage unit 608 . In some embodiments, part or all of the computer program may be loaded and/or installed on computing device 600 via ROM 602 and/or communication unit 609 . When a computer program is loaded into RAM 603 and executed by CPU 601, one or more of the operations of methods 200 to 500 described above may be performed. Alternatively, in other embodiments, CPU 601 may be configured to perform one or more actions of methods 200-500 by any other suitable means (eg, by means of firmware).

It should be further stated that the present disclosure may be a method, an apparatus, a system and/or a computer program product. The computer program product may include a non-transitory computer storage medium having computer readable program instructions loaded thereon for carrying out various aspects of the present disclosure.

Non-transitory computer storage media may also be referred to as non-transitory computer readable storage media, or computer storage media. A computer storage medium may be a tangible device that can hold and store instructions for use by the instruction execution device. Computer storage media may be, for example, but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (non-exhaustive list) of computer storage media include: portable computer disks, hard disks, RAM, ROM, Erasable Programmable Read-Only Memory (EPROM or flash memory), static random access Access Memory (Static Random-Access Memory, SRAM), Portable Compact Disc Read-Only Memory (CD-ROM), Digital Video Disc (DVD), Memory Stick, Floppy Disk, Mechanical Encoding devices, such as punched cards or raised structures in grooves on which instructions are stored, and any suitable combination of the foregoing. Computer storage media, as used herein, are not to be interpreted as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (eg, light pulses through fiber optic cables), or transmitted through electrical wires electric signal.

The computer readable program instructions described herein may be downloaded to various computing/processing devices from computer storage media, or to external computers or external storage devices over a network, such as the Internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers. A network adapter card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer storage medium in the respective computing/processing device.

The computer program instructions for carrying out the operations of the present disclosure may be assembly instructions, Instruction Set Architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or in one or more Source or object code written in any combination of programming languages, including object-oriented programming languages such as Smalltalk, C++, etc., and conventional procedural programming languages such as the "C" language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server implement. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or, may be connected to an external computer (eg, use an internet service provider to connect via the internet). In some embodiments, electronic circuits, such as programmable logic circuits, FPGAs, or Programmable Logic Arrays (PLAs), that can execute computer-readable Program instructions are read to implement various aspects of the present disclosure.

Aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions.

These computer readable program instructions may be provided to a processor in a voice interaction device, a general purpose computer, a special purpose computer, or a processing unit of other programmable data processing devices, thereby producing a machine in which the instructions are processed by a computer or other programmable The processing elements of the data processing apparatus, when executed, produce means for implementing the functions/acts specified in one or more blocks of the flowcharts and/or block diagrams. These computer readable program instructions can also be stored in a computer storage medium, these instructions cause a computer, programmable data processing apparatus and/or other equipment to operate in a specific manner, whereby the computer readable medium storing the instructions includes a manufacturing Articles comprising instructions for implementing various aspects of the functions/acts specified in one or more blocks of the flowchart and/or block diagrams.

Computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other equipment to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other equipment to produce a computer-implemented process , thereby causing instructions executing on a computer, other programmable data processing apparatus, or other device to implement the functions/acts specified in one or more blocks of the flowcharts and/or block diagrams.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which contains one or more oper- ables for implementing the specified logical function(s) Execute the instruction. In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented in dedicated hardware-based systems that perform the specified functions or actions , or can be implemented in a combination of dedicated hardware and computer instructions.

Various embodiments of the present disclosure have been described above, and the foregoing descriptions are exemplary, not exhaustive, and not limiting of the disclosed embodiments. Numerous modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the embodiments, the practical application or technical improvement in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

The above are only optional embodiments of the present disclosure, and are not intended to limit the present disclosure. For those skilled in the art, the present disclosure may have various modifications and changes. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present disclosure should be included within the protection scope of the present disclosure.

Claims

A method for vehicle interaction, comprising:

Monitor the interconnection signal between the vehicle's on-board equipment and the user terminal, so as to determine that the on-board equipment and the user terminal have established a wireless communication connection;

Acquiring device attribute information of the user terminal in response to detecting an operation action applied to the touch screen of the in-vehicle device for an object mapped to the touch screen by the user terminal;

determining whether the device attribute information of the user terminal belongs to a predetermined set;

In response to determining that the device attribute information of the user terminal belongs to a predetermined set, generating first indication information, where the first indication information is used to indicate that the user terminal does not support wireless manipulation by the touch screen;

In response to determining that the device attribute information of the user terminal does not belong to a predetermined set, sending a first test signal via an application program interface of the user terminal, the first test signal being associated with the operation action;

In response to determining that a response signal from the user terminal to the first test signal is not received within the first predetermined time interval, at least one of first indication information and second indication information is generated, and the second indication information is used instructing the user terminal to be connected to the in-vehicle device via a USB connection; and

At least one of the first indication information and the second indication information is presented at the touch screen of the in-vehicle device.
The method of claim 1, wherein at least one of the first indication information and the second indication information is generated in response to determining that a response signal from the user terminal to the first test signal is not received within a first predetermined time interval One includes:

In response to determining that a response signal from the user terminal to the first test signal is not received within a first predetermined time interval, determining whether the number of historical Bluetooth paired devices indicated in the Bluetooth pairing list of the user terminal is less than a predetermined number Quantity thresholds; and

In response to determining that the number of historical Bluetooth paired devices indicated in the Bluetooth pairing list of the user terminal is less than a predetermined number threshold, at least one of the first indication information and the second indication information is generated.
The method of claim 2, further comprising:

In response to determining that the number of historical Bluetooth paired devices is greater than or equal to the predetermined number threshold, presenting information indicative of reducing the number of historical Bluetooth paired devices in the Bluetooth connection list; and

In response to determining that the number of historical Bluetooth paired devices is reduced to less than the predetermined number threshold, retransmitting the first test signal via the application program interface of the user terminal for determining that within a first predetermined time interval Whether a response signal from the user terminal to the first test signal is received.
3. The method of claim 2 or 3, wherein the predetermined number threshold is seven.
The method of any one of claims 1-4, further comprising:

The response signal is presented in response to determining that a response signal from the user terminal to the first test signal is received within a first predetermined time interval.
The method according to any one of claims 1-5, further comprising:

In response to determining that the user terminal is connected to the in-vehicle device via a USB connection, a second test signal for transmission to the user terminal is generated, the second test signal is connected to the user terminal via the touch screen by USB the manner in which the manipulation is associated; and

In response to determining that a response signal from the user terminal to the second test signal is not received within a second predetermined time interval, acquiring an image displayed on the touch screen of the vehicle-mounted device for determining a target indicated by the image object.
The method of claim 6, further comprising:

In response to determining that the target object indicated by the image meets a predetermined condition, generating a third indication signal for indicating that the user terminal cannot be manipulated via the touch screen in a USB connection; and

The third indication information is presented at the touch screen of the in-vehicle device.
The method of claim 7, wherein determining that the target object indicated by the image meets a predetermined condition comprises one of the following:

determining that the matching degree of the target object indicated by the image and a predetermined template is greater than a predetermined matching degree threshold, and the predetermined template is associated with the cursor image;

Through a target object recognition model trained with multiple samples, it is predicted that the category of the target object indicated by the image is a cursor.
The method according to claim 7 or 8, further comprising:

In response to determining that at least one of the first indication signal and the third indication signal is generated, the device attribute information of the user terminal is added to the predetermined set, the predetermined set being obtained from a cloud server.
A computing device comprising:

memory configured to store one or more computer programs; and

A processor, coupled to the memory and configured to execute the one or more programs, causes the computing device to perform the method of any of claims 1-9.
A non-transitory computer storage medium having stored thereon machine-executable instructions that, when executed, cause a machine to perform the steps of the method of any of claims 1-9.