WO2024002276A1

WO2024002276A1 - Method and apparatus for determining script sequence, and electronic device and vehicle

Info

Publication number: WO2024002276A1
Application number: PCT/CN2023/104091
Authority: WO
Inventors: 李谦
Original assignee: 华人运通(江苏)技术有限公司
Priority date: 2021-11-01
Filing date: 2023-06-29
Publication date: 2024-01-04
Also published as: WO2024002297A1; CN116061829A; CN116061170A; CN116061168A; CN116061820A; CN116061824A; CN116061169A; CN116061828A; WO2024002273A1; CN116061827A; CN116061821A; CN116061819A; CN116061825A; CN116061167A; CN116061826A; CN116061822A; CN116061823A; WO2024002303A1

Abstract

A method and apparatus for determining a script sequence, and an electronic device and a vehicle, which relate to the field of artificial intelligence. The method for determining a script sequence comprises: for each control script in a first script sequence, sequentially detecting a reachable condition of a vehicle-mounted robot arm during the process of executing a preset action according to the first script sequence, wherein the control script is a script for controlling the vehicle-mounted robot arm to execute the preset action, and the reachable condition is used for indicating whether the vehicle-mounted robot arm exceeds a preset reachable space; and according to the reachable condition and using the first script sequence, determining a target script sequence for controlling the vehicle-mounted robot arm to execute the preset action.

Description

Method, device, electronic equipment and vehicle for determining script sequence

This application requests the priority of the Chinese patent application submitted to the State Intellectual Property Office on June 30, 2022, with application number 202210767008.5 and titled "Method, device, electronic equipment and vehicle for determining script sequence", and its entire content is approved This reference is incorporated into this application.

This application claims priority to the Chinese patent application submitted to the State Intellectual Property Office on June 30, 2022, with application number 202210765976.2 and titled "Vehicle-mounted robotic arm, and methods and devices for controlling the same", the entire content of which is incorporated by reference. incorporated in this application.

This application requests the priority of the Chinese patent application submitted to the State Intellectual Property Office on June 30, 2022, with application number 202210766060.9 and titled "Control method, device, electronic equipment and vehicle for vehicle-mounted robotic arm", and its entire contents incorporated herein by reference.

Technical field

This application relates to the field of artificial intelligence, and in particular to the determination of script sequences, control methods and devices for vehicle-mounted robotic arms, vehicle-mounted display equipment vehicles, and electronic equipment.

Background technique

As an automatic control device that imitates the functions of a human arm and can complete various tasks, the vehicle-mounted robotic arm has been widely used in industrial manufacturing, medical rescue, aerospace and other fields. However, installing vehicle-mounted robotic arms in the vehicle cockpit to provide intelligent driving services to vehicle occupants is an application field of vehicle-mounted robotic arms that few people have touched upon.

And if you want to install a vehicle-mounted robotic arm in the vehicle cockpit to provide intelligent driving services to the people on the vehicle, how to achieve personalized control of the vehicle-mounted robotic arm has become a problem that you have to face.

Contents of the invention

Embodiments of the present application provide script sequence determination, vehicle-mounted robotic arm control methods and devices, vehicle-mounted display equipment vehicles, and electronic devices to solve problems existing in related technologies.

In a first aspect, embodiments of the present application provide a method for determining a script sequence, which includes: for each control script in the first script sequence, sequentially detecting the feasibility of a vehicle-mounted robotic arm in executing a preset action according to the first script sequence. The control script is a script used to control the vehicle-mounted manipulator to perform preset actions. The reachability situation is used to indicate whether the vehicle-mounted manipulator exceeds the preset reachable space. According to the reachability situation, the first script sequence is used to determine the user. Target script sequence used to control the vehicle-mounted robotic arm to perform preset actions.

In the second aspect, embodiments of the present application provide a method for controlling a vehicle-mounted robotic arm, which includes: determining a target script sequence based on a control script set. The control script set includes multiple control scripts. The control script is used to control the execution of the vehicle-mounted robotic arm. Scripts for preset actions; control the vehicle-mounted manipulator to perform the corresponding preset actions in the preset reachable space according to the target script sequence; wherein, based on the control script set, the target script sequence is determined, including: in response to the user's request in the control script set The selected control script generates a first script sequence; for each control script in the first script sequence, the reachability of the vehicle-mounted manipulator when executing the corresponding preset action according to the first script sequence is detected in turn. The reachability is used to represent the vehicle-mounted robot arm. Whether the robotic arm exceeds the preset reachable space; based on the reachability situation, use the first script sequence to determine the target script sequence.

In a third aspect, embodiments of the present application provide a method for controlling a vehicle-mounted robotic arm, which includes: generating a first control instruction sequence for vehicle-mounted controllable components including the vehicle-mounted robotic arm according to the first trigger information; the first trigger The information is determined based on the instruction information of different people in the vehicle and/or the environmental information of the current vehicle location; when the vehicle-mounted controllable component executes the first control instruction sequence and receives the second trigger information, it is generated The second control instruction sequence of the vehicle-mounted controllable components including the vehicle-mounted manipulator; the second trigger information is determined based on the instruction information of different people in the vehicle and/or the environmental information of the current vehicle location; in the first control instruction When there is a conflict between the sequence and the second control instruction sequence, a conflict resolution strategy is determined to resolve the conflict; the conflict situation includes that the control objects of the first control instruction sequence and the second control instruction sequence both include the vehicle-mounted robotic arm; where, The first control instruction sequence is at least obtained based on script parsing of the received target script sequence containing the vehicle-mounted manipulator control instruction. The target script sequence is determined by executing the script sequence determination method provided by the embodiment of the present application.

In the fourth aspect, embodiments of the present application provide a device for determining a script sequence, including: an accessibility detection unit, configured to sequentially detect, for each control script in the first script sequence, whether the vehicle-mounted robotic arm is in accordance with the first script sequence. The reachability situation during the execution of the preset action. The control script is a script used to control the vehicle-mounted robotic arm to perform the preset action. The reachability situation is used to indicate whether the vehicle-mounted robotic arm exceeds the preset reachable space; the target script sequence determination unit , used to use the first script sequence to determine the target script sequence for controlling the vehicle-mounted manipulator to perform the preset action according to the reachability situation.

In the fifth aspect, embodiments of the present application provide a vehicle-mounted display device, including: a control unit, a vehicle-mounted robotic arm, and a vehicle-mounted screen; the control unit is used to execute the script sequence determination method provided by the embodiment of the application to determine the target. The script sequence, and based on the target script sequence, controls the vehicle-mounted robotic arm; or includes the script sequence determining device provided by the embodiment of the present application, and the control unit is configured to use the script sequence determining device to determine the target script sequence, and based on the target script sequence, Control the vehicle-mounted robotic arm; the vehicle-mounted robotic arm is used to drive the vehicle-mounted screen to complete at least one preset action; the vehicle-mounted screen is connected to the vehicle-mounted robotic arm.

In a sixth aspect, embodiments of the present application provide a vehicle, including: a control unit, a vehicle-mounted robotic arm, and a vehicle-mounted screen; the control unit is used to execute the script sequence determination method provided by the embodiment of the application to determine the target script sequence. , and control the vehicle-mounted robotic arm based on the target script sequence; or include a script sequence determination device provided by an embodiment of the present application, the control unit is configured to determine the target script sequence using the script sequence determination device, and control the vehicle-mounted robot arm based on the target script sequence. A robotic arm; a vehicle-mounted robotic arm, used to drive the vehicle-mounted screen to complete at least one preset action; the vehicle-mounted screen is connected to the vehicle-mounted robotic arm.

In a seventh aspect, embodiments of the present application provide a control device for a vehicle-mounted robotic arm, including: a target script sequence determination unit, configured to determine the target script sequence based on a control script set, where the control script set includes multiple control scripts. The control script is a script used to control the vehicle-mounted manipulator to perform preset actions; the vehicle-mounted manipulator control unit is used to control the vehicle-mounted manipulator to perform the corresponding preset actions in the preset reachable space according to the target script sequence; wherein, the target script sequence The determining unit includes: a first script sequence generating subunit, used to generate a first script sequence in response to a control script selected by the user in the control script set; a first reachable situation detection subunit, used to target the first script sequence Each control script sequentially detects the reachability of the vehicle-mounted manipulator when executing the corresponding preset action according to the first script sequence. The reachability is used to indicate whether the vehicle-mounted manipulator exceeds the preset reachable space; the first determiner of the target script sequence The unit is used to determine the target script sequence using the first script sequence based on the reachability situation.

In an eighth aspect, embodiments of the present application provide another vehicle-mounted display device, including: a control unit, a vehicle-mounted robotic arm, and a vehicle-mounted screen; the control unit is used to perform control of the first vehicle-mounted robotic arm provided by the embodiment of the application. Method; or a control device including the first vehicle-mounted robotic arm provided by the embodiment of the present application; the vehicle-mounted robotic arm is used to drive the vehicle-mounted screen to complete at least one preset action; the vehicle-mounted screen is connected to the vehicle-mounted robotic arm.

In a ninth aspect, embodiments of the present application provide a vehicle, including: a control unit, a vehicle-mounted robotic arm, and a vehicle-mounted screen; the control unit is used to perform control of the first vehicle-mounted robotic arm provided by the embodiment of the present application. Method; or a control device including the first vehicle-mounted robotic arm provided by the embodiment of the present application. The vehicle-mounted robotic arm is used to drive the vehicle-mounted screen to complete at least one preset action; the vehicle-mounted screen is connected to the vehicle-mounted robotic arm.

In the tenth aspect, embodiments of the present application provide another control device for a vehicle-mounted robotic arm, including: a first control instruction sequence generation module, configured to generate a controllable vehicle-mounted robot arm including the vehicle-mounted robotic arm based on the first trigger information. The first control instruction sequence of the component; the first trigger information is determined based on the instruction information of different people in the vehicle and/or the environmental information of the current vehicle location; the second control instruction sequence generation module is used to generate controllable components on the vehicle. During the execution of the first control instruction sequence, when the second trigger information is received, a second control instruction sequence for the vehicle-mounted controllable components including the vehicle-mounted robotic arm is generated; the second trigger information is based on the information of different people in the vehicle. The instruction information and/or the environmental information of the current vehicle location are determined; the conflict resolution strategy determination module is used to determine the conflict resolution strategy to resolve the conflict when there is a conflict between the first control instruction sequence and the second control instruction sequence. Resolve; the conflict situation includes that the control objects of the first control instruction sequence and the second control instruction sequence both include the vehicle-mounted robotic arm; wherein the first control instruction sequence is at least based on the received target script sequence containing the vehicle-mounted robotic arm control instructions. Script parsing is performed to obtain the target script sequence, and the target script sequence is determined by executing the script sequence determination method provided in the embodiment of this application.

In an eleventh aspect, embodiments of the present application provide a vehicle-mounted display device, including: a control unit configured to execute the second control method of a vehicle-mounted robotic arm provided by the embodiment of the application, or include a second vehicle-mounted robotic arm. Control device; a display module composed of a vehicle-mounted robotic arm and a vehicle-mounted screen. The vehicle-mounted robotic arm is used to drive the vehicle-mounted screen to complete at least one target action.

In a twelfth aspect, embodiments of the present application provide a vehicle, including: a control unit for executing the second method of controlling a vehicle-mounted robotic arm provided by an embodiment of the present application, or a second control device for a vehicle-mounted robotic arm. ;Composed of vehicle-mounted robotic arm and vehicle-mounted screen The display module is completed, and the vehicle-mounted robotic arm is used to drive the vehicle-mounted screen to complete at least one target action.

In a thirteenth aspect, embodiments of the present application provide an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores information that can be executed by the at least one processor. instructions to enable at least one processor to execute any method provided by the embodiments of this application.

The above summary is for illustration purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments and features described above, further aspects, embodiments and features of the present disclosure will be readily apparent by reference to the drawings and the following detailed description.

Description of drawings

In the drawings, unless otherwise specified, the same reference numbers refer to the same or similar parts or elements throughout the several figures. The drawings are not necessarily to scale. It should be understood that these drawings depict only some embodiments in accordance with the disclosure and are not to be considered limiting of the scope of the disclosure.

Figure 1 shows a flow chart of a method for determining a script sequence provided in Embodiment 1 of the present application.

Figure 2 shows a schematic diagram of an accessible space provided in Embodiment 1 of the present application.

FIG. 3 shows a schematic diagram of the coordinate system of a vehicle-mounted robotic arm provided in Embodiment 1 of the present application.

FIG. 4 shows a schematic diagram of a script sequence determining device provided in Embodiment 2 of the present application.

Figure 5 shows a flow chart of a control method for a vehicle-mounted robotic arm provided in Embodiment 5 of the present application.

FIG. 6 shows a schematic diagram of a control device for a vehicle-mounted robotic arm provided in Embodiment 6 of the present application.

Figure 7 shows a flow chart of a control method for a vehicle-mounted robotic arm provided in Embodiment 9 of the present application.

FIG. 8 shows a flow chart for determining a conflict resolution strategy to resolve conflicts provided in Embodiment 9 of the present application.

Figure 9 shows a schematic diagram of a control device for a vehicle-mounted robotic arm provided in Embodiment 10 of the present application.

FIG. 10 shows a block diagram of an electronic device used to implement the method for determining a script sequence provided by an embodiment of the present application.

Figure 11 shows an overall schematic diagram of a vehicle-mounted robotic arm according to an embodiment of the present application.

Figure 12 shows a schematic diagram of the guide rail of the vehicle-mounted robotic arm according to the embodiment of the present application.

Figure 13 shows a schematic diagram of the rotation mechanism of the vehicle-mounted robotic arm according to the embodiment of the present application.

Figure 14 shows a schematic diagram of another installation method of the linear motion unit of the vehicle-mounted robotic arm according to the embodiment of the present application.

Figure 15 shows a schematic diagram of the vehicle screen flipping action of the vehicle-mounted robotic arm according to the embodiment of the present application.

Figure 16 shows a schematic diagram of the vehicle screen translation action of the vehicle-mounted robotic arm according to the embodiment of the present application.

Figure 17 shows a schematic diagram of the vehicle screen rotation action of the vehicle-mounted robotic arm according to the embodiment of the present application.

Figure 18 shows a schematic diagram of the forward and backward movement of the vehicle screen of the vehicle-mounted robotic arm according to the embodiment of the present application.

Figure 19 shows a schematic diagram of the action of the rotating member of the vehicle-mounted robotic arm according to the embodiment of the present application.

Detailed ways

In the following, only certain exemplary embodiments are briefly described. As those skilled in the art would realize, the described embodiments may be modified in various different ways without departing from the spirit or scope of the disclosure. Accordingly, the drawings and description are to be regarded as illustrative in nature and not restrictive.

Embodiment 1

The method for determining a script sequence provided in Embodiment 1 of the present application is shown in Figure 1 . Figure 1 is a flow chart of a method for determining a script sequence provided in Embodiment 1 of the present application. The method for determining the script sequence may include the following steps.

Step S101: For each control script in the first script sequence, sequentially detect the reachability of the vehicle-mounted robotic arm during the execution of the preset action according to the first script sequence. The control script is used to control the vehicle-mounted robotic arm to perform the preset action. Script, the reachability condition is used to indicate whether the vehicle-mounted manipulator exceeds the preset reachable space.

Step S102: Based on the reachability situation, use the first script sequence to determine the target script sequence for controlling the vehicle-mounted manipulator to perform the preset action.

The script sequence determination method provided in Embodiment 1 of the present application can sequentially detect the reachability of the vehicle-mounted robotic arm during the execution of the preset action according to the first script sequence, and use the first script according to the corresponding reachability. sequence, determined for controlling the car Load the target script sequence for the robot arm to perform preset actions. Since the target script sequence is a script sequence used to control the vehicle-mounted robot arm to perform preset actions, the target script sequence can be used to achieve personalized control of the vehicle-mounted robot arm, so that the vehicle-mounted robot arm can control each of the first script sequences. All preset actions are executed.

The execution subject of the script sequence determination method provided in Embodiment 1 of the present application is generally the server, and may also be the client.

The so-called server can be a cloud server or cloud server cluster that provides services such as data processing, storage, and forwarding, or it can be a traditional server or traditional server cluster that provides services such as data processing, storage, and forwarding. Wherein, the traditional server is generally implemented as a computing device.

The so-called client is an application (Application, APP), application or software that at least has a script sequence determination function. The client can be deployed and run on a vehicle-mounted electronic device, on an application electronic device, or on a browser web page (web). Among them, the client deployed and run on the vehicle-mounted electronic device is the vehicle-mounted client, the client deployed and run on the mobile electronic device is the mobile client, and the client deployed and run on the browser web page is the web client.

A common mobile client is a mobile client.

It should be noted that when the execution subject of the script sequence determination method provided in Embodiment 1 of the present application is a client, the first script sequence is generally determined in the following manner: the user triggers the client to generate the first script sequence through the human-computer interaction interface. script sequence. At this time, the client needs to have both the script sequence processing function and the script sequence editing function. Taking the execution subject as a vehicle-mounted client as an example, the process in which the user triggers the client to generate the first script sequence through the human-computer interaction interface is as follows:

First, the vehicle-mounted client responds to the script sequence editing request triggered by the user through the human-computer interaction interface and displays the pre-designed control script on the designated page.

Then, the vehicle-mounted client selects the corresponding control script in response to the selection operation triggered by the user through the human-computer interaction interface for the control script, and uses the selected control script to generate the first script sequence according to the order in which the user selects the control script.

In the case where the execution subject of the script sequence determination method provided in Embodiment 1 of the present application is a client, the first script sequence may also be generated by the target application, program or software in response to the user through the human-computer interaction interface of the electronic device. Generated. The so-called target application, program or software is an application, program or software that at least has a script editing function. The so-called electronic devices run target applications, programs or software, and the specific implementation methods include but are not limited to mobile phones, computers and vehicle-mounted electronic devices.

Specifically, when the execution subject is a vehicle-mounted client, implementation methods of electronic devices include but are not limited to mobile phones and computers. When the execution subject is a mobile terminal, the so-called electronic devices include but are not limited to mobile phones, computers and vehicle-mounted electronic devices.

Specifically, the first script sequence may also be generated by other clients in response to the user's triggering of the human-computer interaction interface. In this case, the other clients also need to be notified of having the script sequence editing function. That is to say, when the target application, program or software is an application, program or software that has both a script sequence editing function and a script sequence processing function, the target application, program or software is another client.

The following is a detailed description of the determination process of the first script sequence, taking the execution subject as the vehicle client and the other clients as other mobile clients as an example: First, the mobile client responds to the script sequence triggered by the user through the human-computer interaction interface. Edit the request to display the preconfigured control script on the specified page. Secondly, the mobile client responds to the user's selection operation triggered by the control script through the human-computer interaction interface, selects the corresponding control script, and uses the selected control script to generate the first script sequence according to the order in which the user selects the control script. Thirdly, in response to the first script sequence determination operation triggered by the user through the human-computer interaction interface, the mobile phone client sends the first script sequence forwarding request to the server. After receiving the first script sequence forwarding request, the server will send a target script sequence determination request to the vehicle-mounted client. Finally, after receiving the target script sequence determination request, the vehicle-mounted client will parse the target script sequence determination request and obtain the first script sequence carried in the target script sequence determination request.

In the first embodiment of the present application, when the execution subject of the script sequence determination method is the server, the so-called first script sequence acquisition process can be as follows: first, the client responds to the script sequence editing triggered by the user through the human-computer interaction interface Request to display the preconfigured control script on the specified page. Secondly, the client responds to the selection operation triggered by the user for the control script through the human-computer interaction interface, selects the corresponding control script, and uses the selected control script to generate the first script sequence according to the order in which the user selects the control script. Again, in response to the first script sequence determination operation triggered by the user through the human-computer interaction interface, the client sends a target script sequence determination request to the server. Finally, after receiving the target script sequence determination request, the server parses the target script sequence determination request and obtains the first script sequence carried in the target script sequence determination request.

It should be noted that since the selection order is determined according to the order in which the user selects control scripts, and the first script sequence Column is a sequence of scripts generated for the control scripts selected by the user based on this selection order. Therefore, the first script sequence can reflect the user's personalized needs for the vehicle-mounted robotic arm to perform specified preset actions in a specified execution order.

In addition, since the first script sequence is generated in response to the control script selected by the user in the control script collection, the first script sequence can meet the user's control requirements for the vehicle-mounted robotic arm. And because the target script sequence is determined using the first script sequence, the target script sequence can also meet the user's control requirements for the vehicle-mounted robotic arm.

In addition, after the first script sequence is generated, the target script sequence will be further determined based on the first script sequence according to the reachability situation, so as to control the vehicle-mounted manipulator to execute the corresponding response within the preset reachable space according to the target script sequence. preset actions. Therefore, during the process of generating the first script sequence, the user does not need to pay attention to whether the selected control script will cause the vehicle-mounted manipulator to exceed the reachable space, thereby improving the user experience.

It should be noted that the first script sequence in Embodiment 1 of the present application can not only be obtained through the above method, but also can be obtained through the following method: first, the merchant writes according to the preset action requirements through specific programming software. First script sequence. Then, after the first script sequence is written, it is uploaded to the server through the merchant. When the execution subject is the server, the server determines the script sequence for the first script sequence sent by the merchant. When the execution subject is a client, the server forwards the first script sequence sent by the merchant to the corresponding client, so that the corresponding client determines the script sequence for the first script sequence.

That is to say, in the first embodiment of the present application, there is no specific limitation on the method of obtaining the first script sequence. However, for first script sequences obtained through different channels, it is often necessary to unify the formats after obtaining the first script sequence.

In Embodiment 1 of the present application, the so-called vehicle-mounted robotic arm is a robotic arm installed in the vehicle cabin and used to provide intelligent driving services to people on the vehicle.

It should be noted that the number of vehicle-mounted robotic arms may be one or multiple. When there are multiple vehicle-mounted robotic arms, the script sequence can be determined separately for the multiple vehicle-mounted robotic arms, or the script sequence can be determined for any one of the multiple vehicle-mounted robotic arms. The method for determining the script sequence provided in Embodiment 1 of the present application will be described in detail below, taking only one vehicle-mounted robotic arm as an example.

The vehicle-mounted robotic arm can independently perform corresponding preset actions to provide intelligent driving services for vehicle personnel. Specifically, the vehicle-mounted robotic arm can be used alone to control the movement of the display screen, such as controlling the preset action of moving the vehicle display back and forth or adjusting the angle, or independently performing the preset action of swinging left and right.

In addition, the vehicle-mounted robotic arm can also cooperate with the vehicle infotainment system, instrument panel, head-up display, streaming rearview mirror, ambient light, smart door, smart speaker, etc. in the vehicle cockpit to complete corresponding tasks in preset scenarios. Default action. For example, perform left and right swinging actions in conjunction with the flashing of the ambient light.

In Embodiment 1 of this application, the vehicle-mounted client realizes control of the vehicle-mounted robotic arm by parsing and executing the control script. Specifically, the vehicle-mounted client can parse the control script and control the vehicle-mounted robotic arm according to the preset actions determined by the control script.

The so-called preset actions are preset execution actions for the vehicle-mounted robotic arm. During the execution of the preset actions, the vehicle-mounted robotic arm is within the reachable space. The preset action may specifically be a separate basic action, such as upward movement, downward movement, or left movement, etc. Preset actions can also be complex actions composed of basic actions, such as swinging left and right, moving up and down, shaking your head or waving your hands, etc.

In the first embodiment of the present application, when using the first script sequence to determine the target script sequence according to the reachability situation, the target control script can be obtained first when the reachability situation is detected, including the first reachability situation. The situation is used to indicate that the vehicle-mounted robot arm exceeds the reachable space, and the target control script is the control script that causes the vehicle-mounted robot arm to exceed the reachable space. Secondly, insert the reset script into the first script sequence as the previous control script of the target control script to obtain the second script sequence. The reset script is a script used to control the vehicle-mounted manipulator to perform a reset action. Thirdly, for each control script in the second script sequence, the reachability of the vehicle-mounted robotic arm during execution of the preset action according to the second script sequence is detected in turn. Finally, according to the reachability situation, the second script sequence is used to determine the target script sequence.

In the first embodiment of the present application, when it is detected that the vehicle-mounted robotic arm exceeds the reachable space due to the presence in the first script sequence, it indicates that the vehicle-mounted robotic arm can no longer continue to perform the corresponding preset action according to the first script sequence. At this time, the detection of the remaining control scripts in the first script sequence can be stopped, and the control script that causes the vehicle-mounted manipulator to exceed the reachable space needs to be determined as the target control script. After determining the target control script, a reset script needs to be inserted into the first script sequence as the previous control script of the target control script to Get the second script sequence.

In the process of the vehicle-mounted manipulator executing the corresponding preset action according to the second script sequence, the reset action will be performed before executing the preset action corresponding to the target control script. In this case, executing the preset action corresponding to the target control script is to further execute the preset action corresponding to the target control script on the basis of the reset of the vehicle-mounted manipulator, thereby ensuring that the vehicle-mounted manipulator can respond to the target control script. The preset action is executed.

Although it is guaranteed that the vehicle-mounted manipulator can execute the preset actions corresponding to the target control script, it is still unclear whether the vehicle-mounted manipulator can execute the preset actions corresponding to the control scripts after the target control script. Sure. Therefore, after obtaining the second script sequence, it is still necessary to sequentially detect the reachability of the vehicle-mounted robotic arm during the execution of the preset action according to the second script sequence for each control script in the second script sequence, and further determine the reachability according to the reachability In this case, the second script sequence is used to determine the target script sequence.

In the first embodiment of the present application, the generation process of the second script sequence is specifically taken as an example to describe the process of generating the next script sequence from the previous script sequence. The generation process of the second script sequence will be described in detail below with reference to Figure 2. Figure 2 is a schematic diagram of an accessible space provided in Embodiment 1 of the present application. 201 in Figure 2 is used to represent the vehicle-mounted manipulator, and 202 is used to represent the space range defined by the accessible space. In the first embodiment of the present application, the generation process of the second script sequence is as follows: for example, there are 4 control scripts in the first script sequence in sequence, which are respectively recorded as the first control script, the second control script, and the third control script. and a fourth control script. After obtaining the first control script sequence, the compiler deployed on the server or client to compile the control script will sequentially detect the reachability of the vehicle-mounted manipulator during the execution of the preset action according to the first script sequence.

If the compiler compiles and detects the first control script and detects that the vehicle-mounted robotic arm 201 is within the reachable space 202 during the execution of the preset actions corresponding to the first control script, it proves that the vehicle-mounted robotic arm can Execute a preset action corresponding to the first script sequence according to the first script sequence. At this time, it can be further detected whether the vehicle-mounted robotic arm can perform the preset action corresponding to the second script sequence according to the first script sequence.

Specifically, during the process of executing the preset action corresponding to the second control script, the vehicle-mounted manipulator 201 needs to further execute the second control on the basis of multiple corresponding stop positions after executing the preset action corresponding to the first control script. The preset action corresponding to the script. At this time, the compiler will detect the second control script.

If it is detected that the vehicle-mounted robot arm 201 is in the process of further executing the preset action corresponding to the second control script, there may be a situation where the vehicle-mounted robot arm 201 exceeds the accessible space 202 . At this time, it is necessary to stop detecting the remaining control scripts in the first script sequence. That is, further detection of the third control script and the fourth control script is stopped, and the second control script is determined as the target control script.

After the target control script is determined, a reset script needs to be inserted into the first script sequence as a previous control script of the target control script to obtain the second script sequence. At this time, the script execution order in the second script sequence is the first control script (the first control script to be executed in the second script sequence), the reset script (the second control script to be executed in the second script sequence). script), the second control script (the third control script to be executed in the second script sequence), the third control script (the fourth control script to be executed in the second script sequence), and the fourth control script ( The fifth executed control script is required in the second script sequence).

The second script control sequence not only includes all the control scripts in the first script, but also because before executing the second control script according to the second control script sequence, the vehicle-mounted manipulator 201 needs to execute the reset script first to achieve reset. When the vehicle-mounted manipulator 201 is reset, it can be ensured that the second control script will not exceed the reachable space 202 during execution.

In the first embodiment of the present application, the step of using the second script sequence to determine the target script sequence according to the reachability situation includes: determining the second script sequence as the target when the reachability situations are all second reachability situations. Script sequence, the second reachable situation is used to indicate that the vehicle-mounted manipulator does not exceed the reachable space.

If the vehicle-mounted manipulator is executing the preset action according to the second script sequence, and the reachability conditions corresponding to the vehicle-mounted manipulator are all the second reachability conditions, it proves that the vehicle-mounted manipulator is executing the preset action according to the second script sequence. The accessible space will not be exceeded during the process. At this time, determining the second script sequence as the target script sequence can ensure that the target script sequence can realize personalized control of the vehicle-mounted robotic arm, so that the vehicle-mounted robotic arm performs all preset actions in the first script sequence. implement.

In the first embodiment of the present application, the step of determining the target script sequence using the second script sequence according to the reachability situation also includes: first, when the reachability situation is detected including the first reachability situation, obtaining the target control script . Insert the reset script in the second script sequence as the previous control script of the target control script to obtain the third script sequence, and so on until the target script sequence is determined.

That is to say, as long as the vehicle-mounted manipulator executes the corresponding preset actions according to the newly generated script sequence, and there are still preset actions that cannot be continued, it is necessary to continuously determine the target control script and execute it in the current script sequence. Insert the reset script as the previous control script of the target control script to obtain a new script sequence. It is not until the vehicle-mounted robotic arm is within the reachable space when the vehicle-mounted robotic arm performs corresponding preset actions according to a certain script sequence that the script sequence is determined as the target script sequence. In this way, it can be ensured that the target script sequence can be generated, and the target script sequence can realize personalized control of the vehicle-mounted robotic arm, so that the vehicle-mounted robotic arm executes each preset action in the first script sequence.

In the first embodiment of the present application, using the first script sequence according to the reachability situation, the step of determining the target script sequence may also include: when the reachability situations are all second reachability situations, determining the first script sequence as Target script sequence, the second reachable situation is used to indicate that the vehicle-mounted manipulator does not exceed the reachable space.

That is to say, if the vehicle-mounted robotic arm performs the preset action according to the first script sequence, and the reachability conditions corresponding to the vehicle-mounted robot arm are all the first reachability conditions, it proves that the vehicle-mounted robot arm is executing the preset action according to the first script sequence. The accessible space will not be exceeded during the execution of preset actions. At this time, determining the first script sequence as the target script sequence can ensure that the target script sequence can realize personalized control of the vehicle-mounted robotic arm, so that the vehicle-mounted robotic arm can execute the specified preset action in accordance with the specified execution sequence.

In the first embodiment of the present application, the steps of determining the target script sequence using the first script sequence according to the reachability situation can also be as follows: Step 1, first detect whether the vehicle-mounted robotic arm performs the preset action according to the first script sequence. There is a first reachable situation. Step 2: If there is a first reachable situation, obtain the target control script, insert the reset script into the first script sequence as the previous control script of the target control script, and re-execute step 1. Step 3: If there is no first reachable situation, determine the first script sequence as the target script sequence.

Specifically, the so-called sequential detection of whether there is a first reachable situation when the vehicle-mounted robotic arm performs the preset action according to the first script sequence is: sequentially detecting whether the vehicle-mounted robotic arm performs the preset action according to the first script sequence. Is there any situation where the vehicle-mounted robotic arm exceeds the accessible space during the process?

It is said that if there is a first reachable situation, the target control script is obtained, and a reset script is inserted into the first script sequence as the previous control script of the target control script. The specific implementation method includes: if there is a situation where the vehicle-mounted robotic arm exceeds the reachable space , then insert the reset script into the first script sequence as the previous control script of the target control script.

The so-called specific implementation method of determining the first script sequence as the target script sequence if there is no first reachable situation means: if the vehicle-mounted manipulator does not exceed the reachable space during the execution of the preset action according to the first script sequence. case, the first script sequence is determined as the target script sequence.

In order to accurately detect the reachability situation in real time, in the first embodiment of the present application, the detection method of the reachability situation is as follows: first, determine the spatial range limited by the reachability space. Then, based on the currently detected control script, the real-time position of the vehicle-mounted robotic arm during execution of the corresponding preset action is predicted. Finally, the spatial range and real-time location are used to detect reachability.

In the first embodiment of the present application, the specific implementation method of determining the spatial range defined by the accessible space is: first, using the installation position of the vehicle-mounted robotic arm in the vehicle cockpit as the coordinate origin, a three-dimensional coordinate system for the vehicle-mounted robotic arm is constructed. As shown in Figure 3, Figure 3 is a schematic diagram of the coordinate system of a vehicle-mounted robotic arm provided in Embodiment 1 of the present application. The x-axis of the three-dimensional coordinate system shown in Figure 3 points to the rear of the car, and the z-axis points to the roof. Then, determine the set of coordinate points that the vehicle-mounted manipulator can reach in a certain direction in the cockpit space of the vehicle cabin, and use the set of coordinate points to determine the spatial range. The spatial range can be represented by the pitch angle range, yaw angle range and spatial distance of the vehicle-mounted manipulator arm.

The specific implementation method of predicting the real-time position of the vehicle-mounted robotic arm during the execution of the corresponding preset action is: based on the stop position, based on the currently detected control script, based on the relative position of the vehicle-mounted robotic arm during the execution of the corresponding preset action. The spatial position changes that occur at the installation position are used to predict the real-time position of the vehicle-mounted manipulator during the execution of the corresponding preset action.

Specifically, for each preset action, the spatial position change of the vehicle-mounted manipulator relative to the installation position during the execution of the preset action will be configured in advance, and the process of the vehicle-mounted manipulator performing the corresponding preset action will be predicted. When the real-time position in the vehicle is determined, the real-time position of the vehicle-mounted robot arm during the execution of the corresponding preset action can be predicted based on the stop position and the spatial position change of the vehicle-mounted robot arm relative to the installation position during the execution of the preset action.

It should be noted that in Embodiment 1 of the present application, the vehicle-mounted robotic arm may be composed of at least one joint or connecting rod. In the case where the vehicle-mounted robotic arm is composed of two or more joints or connecting rods, each joint or link The pole may exceed the accessible space during the execution of the preset action. Therefore, when predicting the real-time position of the vehicle-mounted manipulator during execution of the corresponding preset action, It is necessary to predict the real-time position of each joint or link in the vehicle-mounted manipulator during the execution of the corresponding preset action, and based on the real-time position of each joint or link in the vehicle-mounted manipulator during the execution of the corresponding preset action. location to detect reachability.

When the execution subject of the script sequence determination method provided in Embodiment 1 of the present application is the server, in order to enable the vehicle-mounted client to realize personalized control of the vehicle-mounted robotic arm through the target script sequence, so that the vehicle-mounted robotic arm can Each preset action in a script sequence is executed. After determining the target script program, the target script sequence needs to be further sent to the vehicle-mounted client for calling and parsing the target script sequence, so that the vehicle-mounted client can follow the The target script sequence controls the vehicle-mounted manipulator to perform preset actions.

When the execution subject of the method for determining the script sequence provided in Embodiment 1 of the present application is a vehicle-mounted client, in order to achieve personalized control of the vehicle-mounted robotic arm through the target script sequence, so that the vehicle-mounted robotic arm controls the first script sequence Each preset action in is executed. After the target script program is determined, the target script sequence needs to be called and parsed first, and based on the parsed target script sequence, the vehicle-mounted manipulator is controlled to execute the preset according to the target script sequence. action.

Embodiment 2

Corresponding to the first embodiment of the present application, the second embodiment of the present application also provides a device for determining a script sequence. Please refer to Figure 4 for details. Figure 4 is a diagram of a script sequence provided in the second embodiment of the present application. Determine the schematic diagram of the device. The device may include: an reachability detection unit 401, configured to sequentially detect the reachability of the vehicle-mounted manipulator during the process of executing the preset action according to the first script sequence for each control script in the first script sequence. The control script is: A script used to control the vehicle-mounted robot arm to perform preset actions. The reachability condition is used to indicate whether the vehicle-mounted robot arm exceeds the preset reachable space; the target script sequence determination unit 402 is used to use the first script sequence according to the reachability condition. , determine the target script sequence used to control the vehicle-mounted manipulator to perform preset actions.

In one embodiment, the target script sequence determining unit 402 may include: a first subunit for obtaining the target control script, configured to obtain the target control script when detecting that the reachable situation includes the first reachable situation. The reach situation is used to indicate that the vehicle-mounted robot arm exceeds the reachable space, and the target control script is the control script that causes the vehicle-mounted robot arm to exceed the reachable space; the second script sequence obtains a sub-unit, which is used to insert a reset script as a target in the first script sequence The previous control script of the control script is used to obtain the second script sequence. The reset script is a script used to control the vehicle-mounted robotic arm to perform a reset action; the reachability detection subunit is used to target each control script in the second script sequence, The reachability of the vehicle-mounted manipulator is detected in sequence while performing preset actions according to the second script sequence; the target script sequence determines the first subunit, which is used to determine the target script sequence according to the reachability using the second script sequence.

In one embodiment, the target script sequence determines the first subunit, which may include: the target script sequence determines the second subunit, which is used to convert the second script sequence into the second reachable situation when all the reachable situations are the second reachable situation. Determined as the target script sequence, the second reachability condition is used to indicate that the vehicle-mounted manipulator does not exceed the reachable space.

In one embodiment, the target script sequence determines the first subunit, and may further include: the target control script obtains the second subunit, which is used to obtain the target control when the reachable situation is detected including the first reachable situation. Script; the third script sequence obtains the sub-unit, which is used to insert the reset script in the second script sequence as the previous control script of the target control script to obtain the third script sequence, and so on until the target script sequence is determined.

In one embodiment, the target script sequence determining unit 402 may include: a third target script sequence determining subunit, configured to determine the first script sequence as the second reachable situation when all reachable situations are: Target script sequence, the second reachable situation is used to indicate that the vehicle-mounted manipulator does not exceed the reachable space.

In one embodiment, the reachability detection unit 401 may include: a spatial range determination subunit, used to determine the spatial range defined by the reachable space; a real-time position prediction subunit, used to control the script for the current detection, Predict the real-time position of the vehicle-mounted robotic arm during the execution of the corresponding preset action; use the spatial range and real-time position to detect reachability.

In one embodiment, the device may further include: a target script sequence sending unit, configured to send the target script sequence to the vehicle-mounted client for calling and parsing the target script sequence, so that the vehicle-mounted client controls according to the target script sequence. The vehicle-mounted robotic arm performs preset actions.

In one embodiment, the device may also include: a target script sequence parsing unit, used to call and parse the target script sequence; and a vehicle-mounted robotic arm control unit, configured to control the vehicle-mounted robotic arm according to the target script sequence according to the parsed target script sequence. Script sequences perform preset actions.

For the functions of each unit in each device of the embodiment of the present application, please refer to the corresponding description in the above method, and will not be described again here.

Embodiment 3

The third embodiment of the present application also provides a vehicle-mounted display device, which may include a control unit, the robotic arm and the vehicle-mounted screen provided in the first and second embodiments of the present application, wherein the control unit is used to execute the first embodiment of the present application. A script sequence determination method is provided to determine the target script sequence, and control the vehicle-mounted robotic arm based on the target script sequence; or include the script sequence determination device provided in Embodiment 2 of the present application, and the control unit is configured to utilize the determination of the script sequence. The device determines the target script sequence and controls the vehicle-mounted robotic arm based on the target script sequence; the vehicle-mounted robotic arm is used to drive the vehicle-mounted screen to complete at least one preset action; the vehicle-mounted screen is connected to the vehicle-mounted robotic arm.

In addition, the vehicle-mounted robotic arm is used to drive the vehicle-mounted screen to complete at least one preset action; the vehicle-mounted screen is connected to the vehicle-mounted robotic arm.

Embodiment 4

The fourth embodiment of the present application also provides a vehicle, which may include a control unit, the robotic arm and the vehicle-mounted screen in the first and second embodiments of the present application, wherein the control unit is used to execute the script sequence provided in the first embodiment of the present application. Determination method to determine the target script sequence, and control the vehicle-mounted robotic arm based on the target script sequence; or the robotic arm control unit may include the script sequence determination device in Embodiment 1 and Embodiment 2 of the present application, and be used to utilize the above script A sequence determination device determines a target script sequence, and controls the vehicle-mounted robotic arm based on the target script sequence.

Embodiment 5

And if you want to install a vehicle-mounted robotic arm in the vehicle cockpit to provide intelligent driving services to the people on the vehicle, how to achieve personalized control of the vehicle-mounted robotic arm has become an urgent technical problem that needs to be solved.

In order to solve the problems existing in related technologies, Embodiment 5 of the present application provides a method for controlling a vehicle-mounted robotic arm, as shown in Figure 5. Figure 5 is a method for controlling a vehicle-mounted robotic arm provided in Embodiment 5 of the present application. flow chart. The control method of the vehicle-mounted robotic arm may include the following steps.

Step S501: Determine the target script sequence based on the control script set. The control script set includes multiple control scripts. The control script is a script used to control the vehicle-mounted robotic arm to perform preset actions; wherein, the target script sequence is determined based on the control script set, It includes: generating a first script sequence in response to the control script selected by the user in the control script set; for each control script in the first script sequence, sequentially detecting the reachability of the vehicle-mounted robotic arm when executing the corresponding preset action according to the first script sequence. situation, the reachability situation is used to indicate whether the vehicle-mounted manipulator exceeds the preset reachable space; according to the reachability situation, the first script sequence is used to determine the target script sequence.

Step S502: Control the vehicle-mounted manipulator to perform the corresponding preset action in the preset reachable space according to the target script sequence.

The control method of the vehicle-mounted robotic arm provided in Implementation Five of this application can generate a target script sequence based on the control script set to control the vehicle-mounted robotic arm to perform the corresponding preset actions in the preset reachable space according to the target script sequence, thereby achieving It provides personalized control of the vehicle-mounted robotic arm so that the vehicle-mounted robotic arm can perform corresponding preset actions in the preset accessible space according to the target script sequence.

In addition, when determining the target script sequence, the reachability of the vehicle-mounted manipulator during the execution of the preset action according to the first script sequence can be detected sequentially, and based on the corresponding reachability, the first script sequence is used to determine the target. script sequence. Since the first script sequence is generated in response to the control script selected by the user in the control script set, the first script sequence can meet the user's control requirements for the vehicle-mounted robotic arm. And because the target script sequence is determined using the first script sequence, the target script sequence can also meet the user's control requirements for the vehicle-mounted robotic arm.

The execution subject of the control method of the vehicle-mounted robotic arm provided in Implementation Five of this application is the vehicle-mounted client. The so-called vehicle client generally refers to the target application, program or software deployed and run on the vehicle electronic device. The target application, program or software is an application, program or software that at least has script sequence generation and vehicle-mounted robotic arm control functions.

In the fifth implementation of this application, the so-called vehicle-mounted robotic arm is a robotic arm installed in the vehicle cockpit and used to provide intelligent driving services to the people on the vehicle.

It should be noted that the number of vehicle-mounted robotic arms may be one or multiple. When there are multiple vehicle-mounted robotic arms, the script sequence can be determined separately for the multiple vehicle-mounted robotic arms, or the vehicle-mounted robotic arm can be controlled for any one of the multiple vehicle-mounted robotic arms. The following is a detailed description of the control method of the vehicle-mounted robotic arm provided in Implementation 5 of this application, taking only one vehicle-mounted robotic arm as an example.

In the fifth implementation of this application, the vehicle-mounted client realizes control of the vehicle-mounted robotic arm by parsing and executing the control script. Specifically, the vehicle-mounted client can parse the control script and control the vehicle-mounted robotic arm according to the preset actions determined by the control script.

In the fifth implementation of this application, the control script set includes at least two preset actions. The so-called preset actions are preset execution actions for the vehicle-mounted robotic arm. During the execution of the preset actions, the vehicle-mounted robotic arm is within the reachable space. The preset action may specifically be a separate basic action, such as upward movement, downward movement, or left movement, etc. Preset actions can also be complex actions composed of basic actions, such as swinging left and right, moving up and down, shaking your head or waving your hands, etc.

In Embodiment 5 of the present application, in response to the control script selected by the user in the control script set, the process of generating the first script sequence may be: first, the vehicle-mounted client responds to the script sequence editing request triggered by the user through the human-computer interaction interface. Display the control scripts in the control script set on the specified page. Then, the vehicle-mounted client selects the corresponding control script in response to the selection operation triggered by the user through the human-computer interaction interface for the control script, and uses the selected control script to generate the first script sequence according to the order in which the user selects the control script. Because the selection sequence is determined according to the order in which the user selects the control script, and the first script sequence is a script sequence generated for the control script selected by the user based on the selection sequence. Therefore, the first script sequence can reflect the user's personalized needs for the vehicle-mounted robotic arm to perform specified preset actions in a specified execution order.

In the fifth implementation of this application, the process of determining the current script sequence using the first script sequence is an iterative process. The specific iterative process may include the following steps: Step 1: Obtain the script sequence to be detected. Step 2: For each control script in the script sequence to be detected, the reachability of the vehicle-mounted robotic arm when executing the corresponding preset action according to the script sequence to be detected is detected in turn. Step 3: Determine whether the first reachable situation is included in the reachable situation. Step 4: When a reachable situation including a first reachable situation is detected, the target control script is obtained, and a reset script is inserted into the first script sequence as the previous control script of the target control script to obtain a candidate script sequence. Step five: determine the candidate script sequence as the script sequence to be detected, and perform step one. Step 6: When the reachability situation does not include the first reachability situation, determine the script sequence to be detected as the target script sequence.

Specifically, when using the first script sequence to determine the target script sequence according to the reachability situation, the first script sequence may first be determined as the script sequence to be detected. Then, for each control script in the first script sequence, the reachability of the vehicle-mounted robotic arm when executing the corresponding preset action according to the first script sequence is detected in turn. Afterwards, it is determined whether the first reachable situation is included in the reachable situations. Finally, if all the reachable situations are the second reachable situation, the first script sequence is determined as the target script sequence.

However, if the detected reachability situation includes the first reachability situation, it is necessary to obtain the target control script and insert the reset script into the first script sequence as the previous control script of the target control script to obtain the second script sequence.

After obtaining the second script sequence, it is also necessary to first determine the second script sequence as a candidate script sequence, and determine the candidate script sequence as a script sequence to be detected. That is, the second script sequence is determined as the script sequence to be detected. Then, for each control script in the second script sequence, the reachability of the vehicle-mounted robotic arm when executing the corresponding preset action according to the second script sequence is detected in turn. Finally, according to the reachability situation, the second script sequence is used to determine the target script sequence.

When determining the target script sequence using the first script sequence according to the reachability situation, it may also be determined first whether the first reachability situation is included in the reachability situation. Then, if all the reachable situations are the second reachable situation, the second script sequence is determined as the target script sequence.

However, if the detected reachability situation includes the first reachability situation, it is necessary to obtain the target control script and insert the reset script into the second script sequence as the previous control script of the target control script to obtain the third script sequence.

After obtaining the third script sequence, it is also necessary to first determine the third script sequence as a candidate script sequence, and assign the candidate script sequence to Columns are identified as script sequences to be detected. That is, the third script sequence is determined as the script sequence to be detected. Then, for each control script in the third script sequence, the reachability of the vehicle-mounted robotic arm when executing the corresponding preset action according to the third script sequence is detected in turn. Finally, according to the reachability situation, the third script sequence is used to determine the target script sequence.

The above-mentioned process of determining the target script sequence using the first script sequence according to the reachability is performed in sequence until the target script program is determined. In other words, the iteration stop condition for determining the iteration process of the current script sequence is: the determination of the target script program.

It should be noted that in the fifth implementation of this application, the reachability condition is used to indicate whether the vehicle-mounted robotic arm exceeds the preset reachable space. Among them, the first reachability condition is used to indicate that the vehicle-mounted manipulator arm exceeds the reachable space; the second reachability condition is used to indicate that the vehicle-mounted manipulator arm does not exceed the reachable space.

In addition, in the fifth implementation of this application, the target control script is a control script that causes the vehicle-mounted robotic arm to exceed the reachable space, and the reset script is a script used to control the vehicle-mounted robotic arm to perform a reset action.

In the fifth implementation of this application, if the vehicle-mounted robot arm performs the preset action according to the first script sequence, and the reachability conditions corresponding to the vehicle-mounted robot arm are all the first reachability conditions, it proves that the vehicle-mounted robot arm performs the preset action according to the first script sequence. The script sequence will not exceed the reachable space when executing the preset actions. At this time, determining the first script sequence as the target script sequence can ensure that the target script sequence can realize personalized control of the vehicle-mounted robotic arm, so that the vehicle-mounted robotic arm can execute the specified preset action in accordance with the specified execution sequence.

In addition, in the fifth implementation of the present application, when it is detected that the vehicle-mounted robot arm exceeds the reachable space due to the presence in the first script sequence, it indicates that the vehicle-mounted robot arm can no longer continue to perform the corresponding preset action according to the first script sequence. At this time, the detection of the remaining control scripts in the first script sequence can be stopped, and the control script that causes the vehicle-mounted manipulator to exceed the reachable space needs to be determined as the target control script. After the target control script is determined, a reset script needs to be inserted into the first script sequence as a previous control script of the target control script to obtain the second script sequence.

In the fifth implementation of this application, the generation process of the second script sequence is specifically taken as an example to illustrate the process of generating the next script sequence from the previous script sequence. The generation process of the second script sequence will be described in detail below with reference to Figure 2. Figure 2 is a schematic diagram of an accessible space provided in Implementation 5 of the present application. 201 in Figure 2 is used to represent the vehicle-mounted manipulator, and 202 is used to represent the space range defined by the accessible space.

In an example, there are four control scripts in sequence in the first script sequence, which are respectively recorded as the first control script, the second control script, the third control script and the fourth control script. In this case, the second script sequence is generated as follows.

After obtaining the first control script sequence, the compiler deployed on the vehicle-mounted client for compiling the control script will sequentially detect the reachability of the vehicle-mounted manipulator during the execution of the preset action according to the first script sequence.

If the compiler compiles and detects the first control script and detects that the vehicle-mounted robotic arm 201 is within the reachable space 202 during the execution of the preset actions corresponding to the first control script, it proves that the vehicle-mounted robotic arm can Execute a preset action corresponding to the first script sequence according to the first script sequence. At this time, it can be further detected whether the vehicle-mounted manipulator can perform the preset action corresponding to the second script sequence according to the first script sequence.

In order to accurately detect the reachability situation in real time, in Implementation Five of this application, the detection method of the reachability situation is as follows: first, determine the spatial range limited by the reachability space. Then, based on the currently detected control script, the real-time position of the vehicle-mounted robotic arm during execution of the corresponding preset action is predicted. Finally, the spatial range and real-time location are used to detect reachability.

In the fifth implementation of this application, the specific implementation method of determining the spatial range limited by the accessible space is: first, using the installation position of the vehicle-mounted robotic arm in the vehicle cockpit as the coordinate origin, construct a three-dimensional coordinate system for the vehicle-mounted robotic arm. As shown in Figure 3, Figure 3 is a schematic diagram of the coordinate system of a vehicle-mounted robotic arm provided in Implementation 5 of the present application. The x-axis of the three-dimensional coordinate system shown in Figure 3 points to the rear of the car, and the z-axis points to the roof. Then, determine the set of coordinate points that the vehicle-mounted manipulator can reach in a certain direction in the cabin space of the vehicle cabin, and use the set of coordinate points to determine the spatial range. The spatial range can be represented by the pitch angle range, yaw angle range and spatial distance of the vehicle-mounted manipulator arm.

It should be noted that in the fifth implementation of the present application, the vehicle-mounted robotic arm may be composed of at least one joint or connecting rod. In the case where the vehicle-mounted robotic arm is composed of two or more joints or connecting rods, each joint or connecting rod It is possible that the reachable space may be exceeded during the execution of the preset action. Therefore, when predicting the real-time position of the vehicle-mounted manipulator during the execution of the corresponding preset action, it is necessary to predict the real-time position of each joint or link in the vehicle-mounted manipulator during the execution of the corresponding preset action, and according to the vehicle-mounted The real-time position of each joint or link in the robotic arm during the execution of the corresponding preset action is used to detect reachability.

In the fifth implementation of this application, based on the control script set, the implementation method of determining the target script sequence may include the following steps: Step 1, in response to the selection operation triggered by the user for the control script, determine the control script currently selected by the user. Step 2: Determine the next control script that can be selected in the control script set. The control script that can be selected is to control the vehicle-mounted robotic arm in the preset state after the vehicle-mounted robotic arm completes the preset action corresponding to the currently selected control script. The control script corresponding to the preset action is further executed within the set reachable space. Step 3: Determine the control script that can be selected as the next control script that can be selected by the user. Step 4: Set the control script available for selection by the user to a state available for selection by the user, and display the control script available for selection by the user, so that the user can trigger a selection operation on the control script available for selection by the user. Step 5: Determine whether to generate the target script sequence. If not, perform step 1. Step six, if yes, determine the target script sequence.

The control method of the vehicle-mounted robotic arm provided by the embodiment of the present application can automatically determine the selectable control script as the next available control script whenever the user selects a control script during the process of the user editing the script sequence using the control script set. A control script for user selection.

Because the control script that can be selected is a control script that can control the vehicle-mounted robotic arm to further execute the corresponding preset action within the preset reachable space after the vehicle-mounted robotic arm completes the preset action corresponding to the currently selected control script. Therefore, when the control script that can be selected is determined as the next control script that can be selected by the user, it can be ensured that the next control script selected by the user after the currently selected control script must be after the on-board manipulator is executed and After the preset action corresponding to the currently selected control script is performed, the vehicle-mounted manipulator can be controlled to further execute the control script corresponding to the preset action within the preset accessible space.

The control method of the vehicle-mounted robotic arm provided by the embodiment of the present application can enable the vehicle-mounted robotic arm to operate in sequence according to the edited script sequence. During the execution of the preset actions corresponding to each script sequence, all can be within the reachable space. Therefore, personalized control of the vehicle-mounted robotic arm can be achieved through the edited script sequence, so that the vehicle-mounted robotic arm can sequentially execute the preset actions corresponding to each script sequence according to the edited script sequence.

The above method of determining the target script sequence based on the control script set is to determine the target script sequence in an iterative manner. Specifically, in the process of determining the target script sequence based on the control script set, the following steps are performed in sequence: first, in response to the selection operation triggered by the user for the control script, the control script currently selected by the user is determined. Secondly, determine the next control script that can be selected in the control script set. Again, the control script that can be selected is determined as the next control script that can be selected by the user. Finally, the control script available for selection by the user is set to a state available for selection by the user, and the control script available for selection by the user is displayed so that the user can trigger a selection operation on the control script available for selection by the user.

In the fifth implementation of this application, when determining the next control script that can be selected in the control script set for the control script currently selected by the user, it can first be predicted that the vehicle-mounted robotic arm will execute the preset action corresponding to the currently selected control script. the final stopping position. Then, for each control script in the control script set, the real-time position of the vehicle-mounted robotic arm during further execution of the corresponding preset action is predicted based on the stopping position. Finally, determine the control script that can be selected based on the real-time location.

In the fifth implementation of this application, the control script that can be selected is determined based on the real-time position of the vehicle-mounted robot arm during its further execution of the corresponding preset action based on the stop position, which can ensure that the control script that can be selected is the control script that can be selected by the vehicle-mounted robot arm. Control scripts that are within the reachable space during the execution of corresponding preset actions. Thus, it can be avoided that there is a situation in the control script that can be selected. Although the stop position of the vehicle-mounted robot arm can be within the reachable space after the vehicle-mounted robot arm is controlled to perform the corresponding preset action, the vehicle-mounted robot arm is controlled to perform the corresponding preset action. The process will cause the vehicle-mounted robotic arm to exceed the control script of the accessible space.

Because when the control script that can be selected is determined as the next control script that can be selected by the user, it can be ensured that the next control script selected by the user after the currently selected control script must be the same as the currently selected control script after the vehicle-mounted manipulator is executed. After selecting the preset action corresponding to the control script, the vehicle-mounted manipulator can be controlled to further execute the control script corresponding to the preset action within the preset accessible space. Therefore, the real-time position is used to determine the control script that can be selected, so that the vehicle-mounted robotic arm can be in the accessible space during the process of executing the preset actions corresponding to each script sequence in sequence according to the edited script sequence.

In other words, personalized control of the vehicle-mounted robotic arm can be achieved through the edited script sequence, so that the vehicle-mounted robotic arm can sequentially execute the preset actions corresponding to each script sequence according to the edited script sequence.

Specifically, taking the currently selected control script as the second control script in the script sequence as an example, the process of determining the next selectable control script in the control script set for the control script currently selected by the user will be described in detail. : First, after the user selects the second control script, for the second control script, predict the stopping position of the vehicle-mounted robotic arm after executing the corresponding preset action; then, traverse each control script in the control script set, The real-time position of the vehicle-mounted manipulator during further execution of the corresponding preset action is predicted based on the stopping position. Finally, the real-time position corresponding to each control script is used to determine the selectable control script in the control script set that can be selected as the third control script in the script sequence.

It should be noted that the so-called next control script that can be selected includes at least one control script. And if for the control script currently selected by the user, any control script in the control script set cannot be determined as the next control script that can be selected, it proves that the processing of the script sequence cannot continue. At this point, it is necessary to stop the script sequence processing and generate the target script sequence.

In the fifth implementation of this application, the specific process of determining the control script that can be selected based on the real-time location is as follows: First, determine the spatial range limited by the accessible space. Then, for each control script, detect whether the real-time position is within the spatial range. Finally, the control script whose real-time position is within the spatial range is determined as a control script that can be selected.

Determining the control script whose real-time position is within the spatial range as a control script that can be selected can enable the vehicle-mounted manipulator to perform the preset actions corresponding to each script sequence in sequence according to the edited script sequence. within accessible space. Therefore, personalized control of the vehicle-mounted robotic arm can be achieved through the edited script sequence, so that the vehicle-mounted robotic arm can sequentially execute the preset actions corresponding to each script sequence according to the edited script sequence.

In the fifth implementation of this application, in order to improve the visibility of the control scripts available for users to select and improve the user experience, in the process of displaying the control scripts available for users to select, the control scripts available for users to select can be first Set to highlight display mode, and then highlight the control scripts that can be selected by the user.

In addition, in order to make the visualization effect more obvious, the control script that can be selected by the user can also be set to be selectable by the user. mode, and after setting the control script that is not available for user selection to the mode that is not available for user selection, further setting the control script that is available for user selection to the highlighted mode, and setting the control script that is not available for user selection to Grayed out mode.

In addition, other methods can be used to further improve the user's visibility of control scripts available for selection by the user. For example, control scripts that are not available for selection by the user can also be set to a mode that cannot be displayed.

In the fifth implementation of this application, in order to achieve personalized control of the vehicle-mounted robotic arm through the target script sequence, so as to control the vehicle-mounted robotic arm to perform corresponding preset actions in the preset reachable space according to the target script sequence, after determining the target After the script program is generated, the target script sequence needs to be called and parsed first, and based on the parsed target script sequence, the vehicle-mounted manipulator is controlled to perform preset actions according to the target script sequence.

Embodiment 6

In Embodiment 6 of the present application, a control device for a vehicle-mounted robotic arm is also provided. For details, please refer to 6. FIG. 6 is a schematic diagram of a control device for a vehicle-mounted robotic arm provided in Embodiment 6 of the present application. The device may include:

The target script sequence determining unit 601 is used to determine the target script sequence based on a control script set. The control script set includes multiple control scripts. The control script is a script used to control the vehicle-mounted robotic arm to perform preset actions;

The vehicle-mounted robotic arm control unit 602 is used to control the vehicle-mounted robotic arm to perform corresponding preset actions in the preset accessible space according to the target script sequence;

The target script sequence determining unit 601 includes: a first script sequence generating subunit, used to generate the first script sequence in response to the control script selected by the user in the control script set; and a first reachable situation detection subunit, used to target Each control script in the first script sequence sequentially detects the reachability of the vehicle-mounted robotic arm when performing the corresponding preset action according to the first script sequence. The reachability condition is used to indicate whether the vehicle-mounted robotic arm exceeds the preset reachable space; target The first script sequence determination subunit is used to determine the target script sequence using the first script sequence according to the reachability situation.

In a sixth embodiment, the first determination subunit of the target script sequence may include: a first acquisition subunit of the target control script, used to obtain the target control script when detecting that the reachability situation includes the first reachability situation, The first reachable situation is used to indicate that the vehicle-mounted robot arm exceeds the reachable space. The target control script is the control script that causes the vehicle-mounted robot arm to exceed the reachable space. The first insertion subunit of the reset script is used to insert reset in the first script sequence. The script is used as the previous control script of the target control script to obtain the second script sequence. The reset script is a script used to control the vehicle-mounted robotic arm to perform the reset action; the second detection subunit of reachability is used to target the second script sequence. Each control script of the vehicle-mounted manipulator sequentially detects the reachability of the vehicle-mounted manipulator during the execution of the preset action according to the second script sequence; the second determination subunit of the target script sequence is used to determine the reachability of the vehicle-mounted manipulator using the second script sequence according to the reachability. Target script sequence.

In a sixth embodiment, the second determination subunit of the target script sequence may include: the third determination subunit of the target script sequence determines the second script sequence as Target script sequence, the second reachable situation is used to indicate that the vehicle-mounted manipulator does not exceed the reachable space.

In a sixth embodiment, the second target script sequence determination subunit may also include: a second target control script obtaining subunit, configured to obtain the target when the reachable situation is detected including the first reachable situation. Control script; the second insertion subunit of the reset script is used to insert the reset script in the second script sequence as the previous control script of the target control script to obtain the third script sequence, and so on until the target script sequence is determined.

In a sixth embodiment, the first determination subunit of the target script sequence may include: a fourth determination subunit of the target script sequence, used to determine the first script when all reachable situations are second reachable situations. The sequence is determined as the target script sequence, and the second reachability condition is used to indicate that the vehicle-mounted manipulator does not exceed the reachable space.

In a sixth embodiment, the target script sequence determining unit 601 may include: a fifth target script sequence determining subunit, and the fifth target script sequence determining subunit is specifically configured to: in response to a selection operation triggered by the user for the control script, Determine the control script currently selected by the user; determine the next control script that can be selected in the control script set. The control script that can be selected is the control script that can be controlled after the vehicle-mounted manipulator completes the preset action corresponding to the currently selected control script. The vehicle-mounted manipulator further executes the control script corresponding to the preset action within the preset reachable space; the control script that can be selected is determined as the next control script that can be selected by the user; the control script that can be selected by the user is set to The state is available for user selection, and the control scripts available for user selection are displayed so that the user can trigger the selection operation on the control scripts available for user selection; and so on, until the target script sequence is determined.

In a sixth embodiment, the fifth target script sequence determination subunit may include: a stop position determination subunit for predetermining Measure the stopping position of the vehicle-mounted robotic arm after executing the preset action corresponding to the currently selected control script; the real-time position prediction subunit is used to predict the stopping position of the vehicle-mounted robotic arm for each control script in the control script set. On the basis, the real-time position during the corresponding preset action is further executed; the first determining subunit of the selectable script is used to determine the control script that can be selected based on the real-time position.

In a sixth embodiment, the first determination subunit of the selectable script may include: a spatial range determination subunit, used to determine the spatial range defined by the accessible space; a real-time position detection subunit, used to determine each control The script detects whether the real-time position is within the spatial range; the second determination subunit of the selectable script is used to determine the control script whose real-time position is within the spatial range as a selectable control script.

In a sixth embodiment, the fifth target script sequence determination subunit may include: a display mode setting subunit, used to set the control scripts available for user selection to a highlight display mode; a highlight display subunit , used to highlight the control scripts available for users to select.

Embodiment 7

The seventh embodiment of the present application also provides a vehicle-mounted display device, including: a control unit, a vehicle-mounted robotic arm, and a vehicle-mounted screen; the control unit is used to execute the control method of the vehicle-mounted robotic arm provided in the fifth embodiment of the present application; or includes the present invention. The control device for a vehicle-mounted robotic arm provided in the sixth embodiment of the application.

Embodiment 8

Embodiment 8 of the present application also provides a vehicle, which may include a control unit, the robotic arm and the vehicle-mounted screen provided in Embodiments 5 and 6 of the present application, wherein the control unit is used to execute the third step provided in Embodiment 5 of the present application. A method for controlling a vehicle-mounted robotic arm; or a control device for a vehicle-mounted robotic arm provided in Embodiment 6 of the present application.

Embodiment 9

The main components of the smart cockpit include in-vehicle infotainment systems, instrument panels, heads-up displays, streaming rearview mirrors, ambient lights, smart doors, and smart speakers. Various functions in the smart cockpit can be combined to provide more personalized driving services. Robotic arms have been widely used in automation scenarios, but there are few examples of combining robotic arms with smart cockpits. How to combine the two to provide drivers and passengers with more diverse intelligent services while ensuring driving safety? , taking into account the control instructions for the vehicle-mounted robotic arm under different circumstances has become an urgent problem to be solved.

In order to solve the problems existing in related technologies, Embodiment 9 of the present application provides a method for controlling a vehicle-mounted robotic arm. The method may include the following steps.

S701: Based on the first trigger information, generate a first control instruction sequence for the vehicle-mounted controllable components including the vehicle-mounted robotic arm; the first trigger information is based on the instruction information of different people in the vehicle and/or the current location of the vehicle. Environmental information is determined;

S702: When the vehicle-mounted controllable component executes the first control instruction sequence and the second trigger information is received, generate a second control instruction sequence for the vehicle-mounted controllable component including the vehicle-mounted manipulator; second trigger information It is determined based on the instruction information of different people in the vehicle and/or the environmental information of the current vehicle location;

S703: When there is a conflict between the first control instruction sequence and the second control instruction sequence, determine a conflict resolution strategy to resolve the conflict; the situation where the conflict exists includes that the control objects of the first control instruction sequence and the second control instruction sequence both Including a vehicle-mounted robotic arm; wherein the first control instruction sequence is obtained based on at least script parsing of a received target script sequence containing vehicle-mounted robotic arm control instructions, and the target script sequence is obtained by executing the script sequence determination method provided in the embodiment of the present application. Sure.

The execution subject of the above process of this application may be the vehicle terminal. Vehicle-mounted controllable components can include vehicle-mounted robotic arms, doors, speakers, ambient lights, etc. The control principles of each vehicle-mounted controllable component are the same. In the current implementation, a vehicle-mounted manipulator is used as an example for detailed description.

Among them, the vehicle-mounted robotic arm can be a bracket for the vehicle display screen. Through the movement of the vehicle-mounted robotic arm, it can match the display content of the vehicle display screen. For example, it can cooperate with the vehicle display screen to swing back and forth at a certain angle, left and right, etc. The initial position of the vehicle-mounted manipulator can be embedded in the vehicle console. After receiving the control instruction, it can move based on the control instruction. For example, the movement may be tilting toward the driver, moving to the corresponding position of the rear passenger, etc. Alternatively, the vehicle-mounted robotic arm can also be an independent vehicle-mounted component, for example, it can be installed at the arm rest of the car. The initial position of the vehicle-mounted robotic arm can serve as a support The hand rest, after receiving the control command, can move based on the control command. Not only that, the vehicle-mounted robotic arm can also perform different actions in conjunction with the music played by the speakers and the different colors of the ambient lights, so as to meet the personalized needs of users. In this application, the specific location or function of the vehicle-mounted robotic arm is not limited.

The (first or second) trigger information can be issued by different people in the vehicle. For example, the first triggering command is issued by the driver, and the second triggering command is issued by the passenger sitting behind the driver. Information that triggers control instructions for controlling controllable components on the vehicle. Alternatively, the trigger information may also be information detected by a vehicle-side sensor.

Taking the trigger information issued by different people in the car as an example, the trigger information can be a control instruction for the vehicle's controllable components issued by the driver or passenger through voice, movement or touch.

Among them, the control instruction actions issued to the vehicle-mounted controllable components may include the actions issued through gesture adjustment. As shown in Figure 8, the standard direction (up, down, left, right) is symmetrically expanded to a 90-degree sector composed of 45 degrees as the judgment execution interval. When the gesture recognition system is working normally and is turned on, the vehicle-mounted robotic arm can be rotated left and right and tilted up and down through gesture recognition. For example, the user's adjustment instructions through fist gestures can be detected in real time. When the gesture recognition system is kept in front of the gesture recognition system for a certain period of time (for example, 2 seconds), the gesture adjustment can be activated. After activation, feedback sound effects can be emitted, and the central control screen displays the corresponding content that the gesture has been activated, indicating that the gesture adjustment function has been activated. The image acquisition device in the gesture recognition system collects the movement direction of the user's gesture (such as making a fist), and the image analysis module in the gesture recognition system determines the up, down, left, and right movements of the gesture, and ultimately can control the left and right rotation or up and down pitch movements of the vehicle-mounted robotic arm. When the user stops moving gesture is detected, the vehicle-mounted robotic arm can be controlled to stop rotating. In addition, during the detection of the user's movement (to the left), the user stays briefly (for example, not more than 1 second) and then moves in the other direction (to the right). In this case, you can continue to determine the direction and control the vehicle-mounted manipulator to move in a new direction (to the right). When it is detected that the user moves their fist away from the sensing area, or it is detected that the user changes the gesture, resulting in unrecognizability, or it is detected that the user maintains the fist posture and remains stationary for more than 1 second, the gesture adjustment mode can be controlled to exit. After exiting the gesture adjustment mode, the central control screen displays the corresponding content that the gesture adjustment has exited. In addition, for gesture control, the following gestures can also be included. For example, the gesture may include multiple repetitions after detecting that the user's five fingers are combined, the palm is facing down, the five fingers are bent, and the five finger tips are toward/away from the palm. For example, each repetition may represent the position of the vehicle-mounted robotic arm toward the controller. move. Alternatively, the gesture may also include detecting that the user's five fingers are merged, the palm is facing upward, the five fingers are bent, and the five finger tips are repeated multiple times toward/away from the palm direction, etc.

Further, touch control is used as an example for further elaboration. The trigger information can be generated by the user editing through the front-end device. For example, the front-end device may be a smartphone terminal (APP), a car terminal (APP), or a Web terminal (editing page or APP), etc. Take the front-end device as an APP as an example. The visual programming interface can be prefabricated in the APP on the smartphone, car or web. Users (car owners) can enter the visual editing page by opening the APP. In the visual programming interface, you can use the icon drag-and-drop editing method to edit the peripheral atomization function blocks corresponding to the control command icons such as the movement of the vehicle's robotic arm, the sound effects of the speakers, the door opening status, and the ambient light changes. For example, the visual programming interface is provided with control instruction icons for a vehicle-mounted robotic arm, a control instruction icon for a speaker, a control instruction icon for an ambient light, a control instruction icon for a car door, etc. Taking the control of a vehicle-mounted robotic arm as an example, when receiving the user's drag command for the control instruction icon of the vehicle-mounted robotic arm, the control instruction icon of the vehicle-mounted robotic arm can be set to editable, and the control instructions of other vehicle-mounted components can be set to editable. The icon is made uneditable. Non-editable can be grayed out, loading non-editable layers, etc. For editable control instruction icons, submenus can be displayed. For example, the submenus can display prefabricated actions of the vehicle-mounted robotic arm, such as swinging left and right (shaking head), swinging up and down (nodding), etc. For another example, the submenu can display a single control action, such as moving forward one space, up one space, etc., turning 5° to the left, etc. The dragging instruction may be to select the control instruction icon of a single vehicle-mounted component, or it may be to select the control instruction icons of multiple vehicle-mounted components in succession.

Furthermore, in the case where the control instruction icons of multiple vehicle-mounted components are selected one after another, the execution sequence of the multiple vehicle-mounted components can also be controlled in conjunction with the timeline. For example, the order of execution can be serial or parallel, etc. Upon receiving the user's instruction to complete the editing, the trigger information can be obtained, thereby triggering the control instruction for controlling the vehicle-mounted controllable components.

Convert the received different script formats containing vehicle-mounted robot arm control instructions into a normalized format. For example, it can be converted to .json format. After converting to .json format, the control instructions can be parsed and obtained based on the .json format file during the subsequent script parsing process.

In addition, the trigger information may also be information detected by the vehicle-side sensor. For example, trigger information can be generated when it is detected that the braking amplitude of the vehicle exceeds the corresponding threshold, or when it is detected that an obstacle in front of the vehicle actively triggers braking. During vehicle braking In this case, the trigger information can be to control the vehicle-mounted robotic arm to execute a reset command to ensure safety. Alternatively, when it is detected that the distance between the driver and the vehicle is less than the corresponding threshold, trigger information can be generated to trigger a control instruction that controls the vehicle-mounted robotic arm to perform a reset action.

The first trigger information may be trigger information generated first in time series. According to the first trigger information, a first control instruction sequence for vehicle-mounted controllable components including the vehicle-mounted robotic arm can be generated. For example, the first control instruction sequence generated using the first trigger information may be directed only to the vehicle-mounted robotic arm, or may be directed to multiple vehicle-mounted controllable components such as the vehicle-mounted robotic arm, the door, the speaker, and the ambient light. In the same way, the second control instruction sequence generated using the second trigger information can also be targeted at the vehicle-mounted robotic arm, or can also be targeted at multiple vehicle-mounted controllable components such as car doors, speakers, and ambient lights.

When the vehicle-mounted controllable component executes the first control instruction sequence and receives a second control instruction sequence, a conflict may exist.

Therefore, it is necessary to determine conflict resolution strategies to resolve conflicts. For example, the conflict resolution strategy may be determined based on the time when the first trigger information and the second trigger information are received; it may also be determined based on a predetermined priority. For example, the first control instruction sequence is a command issued by a rear passenger to control the vehicle-mounted robotic arm to move toward the rear seat. During the movement, a second control instruction sequence corresponding to the second trigger information issued by the driver is detected, which controls the vehicle-mounted robotic arm to move toward the driver; or, during the movement, the second trigger information is detected The corresponding second control instruction sequence is generated in the event of emergency braking of the vehicle. In the above situation, both the first control instruction sequence and the second control instruction sequence include control of the vehicle-mounted robotic arm, and the priority of the later second control instruction sequence is higher than the priority of the first control instruction sequence. Therefore, there may be a conflict between the first control instruction sequence and the second control instruction sequence.

On the contrary, if the first control instruction sequence is issued by the driver, the second control instruction sequence is issued by the rear passenger. In the above situation, the priority of the first control instruction sequence is higher than that of the second control instruction sequence. Therefore, there will be no conflict between the first control instruction sequence and the second control instruction sequence. If there is no conflict, the controllable device can be controlled and executed in the order in which the control instruction sequence is received.

Through the above process, when the detection result is that there is a conflict, a preset conflict resolution strategy is executed to adjust the first control instruction sequence and the second control instruction sequence. For example, the first control instruction sequence and the second control instruction sequence can be adjusted according to the priority of the control instruction sequence as a conflict resolution strategy. This supports the concurrency of multiple control instruction sequences, and the controllable components perform their respective functions according to the conflict resolution strategy, improving the intelligence of the vehicle's controllable components.

In one implementation, when determining a conflict resolution strategy to resolve conflicts, the types of the first trigger information and the second trigger information may be determined first. Then, if the specified type exists, or the priority of the trigger information of the specified type is higher than the priority of another trigger information, the other trigger information is ignored.

Types of the first trigger information and the second trigger information may include security or non-security. The safety category can be triggered by vehicle braking. For example, the automatic emergency braking signal (AEB) sent by the Auto-driving Domain Controller Module (ADCM) can correspond to safety trigger information. For another example, the control signals sent through the Infotainment Domain Controller Module (IDCM) and the control signals sent through the Body Domain Controller Module (BDCM) can be used as non-safety trigger information.

The specified type can be a safe class. That is, the priority of the security-type trigger information may be the highest level. For example, when the (first) control instruction sequence corresponding to the non-safety trigger information is interrupted by the (second) control instruction sequence corresponding to the safety trigger information, and there is a conflict between the two, due to the safety trigger The (second) control instruction sequence corresponding to the information corresponds to a higher level. In this case, even if the (first) control instruction sequence corresponding to the non-safety trigger information is earlier, it can still be ignored.

In addition, for the control signals sent by the IDCM and the control signals sent by the BDCM, the priority of each signal can be determined in advance. For example, the priority of control signals corresponding to high-frequency, fixed-function scenarios sent through BDCM may be higher than the priority of control signals corresponding to user-initiated adjustment scenarios.

Among them, the control signals corresponding to high-frequency and fixed-function scenarios can include welcome mode, co-pilot mode, power-off return mode, track mode, rear space enjoyment mode, etc. The welcome mode may be to perform a multimedia display when it is detected that the user is beyond a predetermined distance from the vehicle. For example, when a user (carrying a key) approaches the vehicle, the vehicle display screen can display words such as "Welcome back" or images such as fireworks. In addition, in the welcome mode, the vehicle-mounted robotic arm can shake left and right to express waves, etc. The co-pilot mode can include that after detecting that the co-pilot has entered the car, the vehicle-mounted robotic arm rotates left and right so that the vehicle display screen can face the The passenger side is convenient for the passenger to get in and operate the car. The power-off return mode can be the reset of the vehicle-mounted manipulator after power-off. Track mode can be when the vehicle starts racing, the on-board robotic arm resets, or it tilts at a certain angle in conjunction with the vehicle's steering, etc. The triggering condition for the rear-seat free space mode can be the parking state, or the driving speed is 0 km/h. In this case, it is also necessary to ensure that there is no occupation signal for the front seats for 5 seconds. The main and passenger seats move toward the front of the car to their closest position and fold forward. The vehicle-mounted robotic arm (carrying the display device) moves toward the rear of the vehicle to the optimal position (for example, centered and extended forward), and the ambient light performs dim light display, etc. Normal interruptions of the rear row space mode can include: the user ends the scene independently. This situation will no longer trigger the scene (this time the back row free space mode ends), and other untriggered situations can still trigger the scene. Abnormal interruptions of the rear row enjoyment space mode may include: a higher priority scene is executed (the rear row enjoyment space mode is suspended this time), and the scene will be retriggered when the trigger conditions are met again.

User active adjustment scenarios can include manual power adjustment, steering wheel control adjustment, voice adjustment, motion adjustment, etc. The user's active adjustment scene may include a control instruction sequence (jog command) created by the user in a personalized manner; the user's active adjustment scene may also include an existing control instruction sequence selected by the user, which is referred to here as the control sequence corresponding to the calling scene card. To call the control sequence corresponding to the scene card, you need to use the scene engine, and the scene card is stored in the scene engine. The scene card provides two forms: preset scenes (created at the factory) and user-created scenes (customized by the user to compile and save to meet rationality requirements). The scene card itself is a public resource, which can be used to call the vehicle-mounted robotic arm controller through script pre-embedding, UI interface human-computer interaction, voice, steering wheel control, etc., and then control the vehicle-mounted robotic arm controller.

The trigger signal of the jog command directly calls the vehicle-mounted robotic arm service (Bot Service), thereby controlling the vehicle-mounted robotic arm controller. Conflict resolution strategies can be adjusted during this process. The jog command has a certain degree of randomness. For example, it can be a control command issued randomly by the driver or passenger during the ride. For example, it can be "move a little to the left", "come closer to me", etc. uttered by the driver through voice.

Through the above process, the conflict resolution strategy is determined by determining the priority of the first control instruction sequence and the second control instruction sequence. If they have the same priority, they can be executed on a first-come, first-served basis, or they can be executed in parallel if there is no conflict with the controlled objects.

It is easy to understand that if there is no conflict, the first control instruction sequence and the second control instruction sequence can be executed in parallel, or they can be executed sequentially according to the reception time.

In one implementation, the method further includes the following process: directly sending the control instruction sequence corresponding to the specified type of trigger information to the vehicle-mounted controllable component.

In the current implementation, the specified type of trigger information may be the aforementioned safety trigger information, that is, the trigger information corresponding to the automatic emergency braking signal sent by the ADCM; or, it may also be the trigger information corresponding to the control signal sent through the BDCM. Trigger information.

Taking the safety trigger information as an example, after receiving the trigger information corresponding to the automatic emergency braking signal sent by the ADCM, a control command sequence is required to reset the vehicle-mounted robotic arm to its original posture. In this case, the AEB signal passes through the main control chip (MCU) of the infotainment domain controller (IDCM) and directly controls the vehicle-mounted robotic arm. That is, when both safety and non-safety classes exist, the non-safety control instruction sequence can be ignored.

In the same way, after receiving the trigger information corresponding to the control signal sent through the BDCM, the vehicle-mounted robotic arm can also be directly controlled through the MCU of the IDCM.

Therefore, by directly controlling the vehicle-mounted robotic arm, the control signal transmission path can be reduced, ensuring that the vehicle-mounted robotic arm can be accurately controlled in the first time.

In one implementation, when determining a conflict resolution strategy to resolve conflicts, the types of the first trigger information and the second trigger information may also be determined first. Then, if the types are the same, the attributes of the first control instruction sequence and the second control instruction sequence are determined, and the attributes include template class attributes or customized class attributes. Finally, if the attributes are different, or if the attributes are the same and belong to the customized category, the vehicle-mounted robotic arm is controlled to pause the action. When a new control instruction sequence is received, the vehicle-mounted robotic arm is controlled to execute the new control instruction sequence.

In the current implementation, the same types may correspond to non-safety classes. Determining the type of trigger information may be based on the source channel of the trigger information. For example, the trigger information corresponding to the control signal sent through BDCM or through IDCM can correspond to the same type (non-safety type).

In the case where the types of the first trigger information and the second trigger information are both non-security types, it can be further determined that the first trigger information The corresponding attributes of the first control instruction sequence, and the attributes of the second control instruction sequence corresponding to the second trigger information.

Properties can include template class properties or custom class properties. For example, the previously mentioned welcome mode, co-pilot convenience mode, power-off return mode, track mode, etc. can correspond to template attributes. That is, each of the above modes is a complete set of preset control sequences, which can be executed according to the corresponding sequence when triggered.

Customized attributes can be attributes corresponding to fine-tuning control instructions triggered by the user through voice, gesture, or touch instructions. For example, "a little higher", "a little closer to me", "turn 10° towards me", etc. The above control instruction sequence does not have a complete control sequence, but a control instruction or control instruction sequence randomly issued by the user. In this regard, it can be divided into custom class attributes.

The attributes of the first control instruction sequence corresponding to the first trigger information and the attributes of the second control instruction sequence corresponding to the second trigger information are different from each other (one is a template class attribute and the other is a custom class attribute), or the When the attributes of the first control instruction sequence corresponding to one trigger information and the attributes of the second control instruction sequence corresponding to the second trigger information both belong to customized attributes, the conflict resolution strategy can compete for the strategy.

For example, the vehicle-mounted robotic arm is executing the control instruction sequence (first control instruction sequence, template attribute) corresponding to the passenger convenience. At this time, the driver issues instructions through side control, voice, gestures, etc. to generate the second trigger information. In the case where the second control instruction sequence corresponding to the second trigger information conflicts with the first control instruction sequence corresponding to the first trigger information, if the vehicle-mounted manipulator is in a running state when it receives a new jog control command (at this time The jog control command of the second control command sequence is not cached), and the vehicle-mounted manipulator is controlled to stop and enter a state ready to receive new control commands. For subsequent action execution, a first-come, first-served conflict resolution strategy is implemented. That is, after the vehicle-mounted manipulator is paused, the subsequent control command sequence shall be determined first. That is, the received new control instruction sequence may be a newly received first control instruction sequence and a second control instruction sequence, or may be a third control instruction sequence different from the first control instruction sequence and the second control instruction sequence.

For another example, when the types are the same and the attributes controlling the instruction sequence are all customized class attributes, the conflict resolution strategy can also be obtained in the same way.

Through the above process, the determination of conflict resolution strategies can be achieved.

In one implementation, when determining a conflict resolution strategy to resolve conflicts, the priorities of the first control instruction sequence and the second control instruction sequence can be compared first when the attributes are the same and belong to customized attributes. Then, based on the comparison results, the vehicle-mounted manipulator is controlled to execute a control instruction sequence with high priority.

When the attributes of the first control instruction sequence corresponding to the first trigger information and the attributes of the second control instruction sequence corresponding to the second trigger information are the same and both belong to customized attributes, the priority of the two can first be compared. class.

As mentioned before, in the non-safety type, the priority of the control signal corresponding to the high-frequency, fixed-function scene sent through the BDCM can be higher than the priority of the control signal corresponding to the user-initiated adjustment operation sent by the user using the steering wheel.

That is, the priority of the control signal corresponding to the high-frequency and fixed-function scenario can be medium (the priority of the safety category is high).

In addition, you can also have the following scenarios, and the priority of each scenario is detailed as follows:

The priority of control signals in user co-creation scenarios can be medium or low. User co-created scenes can be scenes composed of user-defined action control instructions. Since it is a scenario that has been created by the user and passed the rationality test, it can also be used as a template type for user co-created scenarios.

The priority of the control signal corresponding to the user's active adjustment scene sent by the user using the steering wheel may be low.

The priority of the control signal corresponding to the human-computer interaction scenario may be low. Human-computer interaction scenes can be in several prefabricated modes. For example, it can include a safety steward mode (such as safety management, autonomous driving, etc.), a test drive introduction mode (such as vehicle function introduction), an awakening mode (such as interacting with the driver), and an intelligent volume adjustment mode (such as intelligent noise reduction). , mute when answering calls or talking), KTV mode, smart weather broadcast mode, New Year lion dance mode (multimedia playback on specific holidays), children's mode (playing cartoons or cartoon songs), low battery mode, etc.

The priority of the control signal corresponding to the automatic adjustment scene can be low. The follow-up automatic adjustment scene can be a sequence of control instructions of the accompanying type. For example, the color change of the ambient light can be automatically adjusted as the music in the speaker changes. Follow-up automatic adjustment scenes can correspond to template types.

By comparing the priorities, the normal progress of the control instruction sequence with high priority can be ensured. In addition, for control instructions with the same priority, the conflict resolution strategy can be obtained by continuing the first-come-first-served approach in the previous steps.

In one implementation, the following steps may also be included: first, detect the status of the vehicle-mounted controllable components in real time, and the status includes normal normal state or abnormal state. Then, the executability of the first control instruction sequence and/or the second control instruction sequence is determined based on the detection result of the status of the vehicle-mounted controllable component.

The status of the vehicle-mounted controllable components may include normal status or abnormal status (abnormal status). For the vehicle-mounted manipulator, its status can be shown in Table 1.

Table 1

Through the trigger information corresponding to the automatic emergency braking signal sent by the aforementioned ADCM and the trigger information corresponding to the control signal sent through the BDCM, the vehicle-mounted robotic arm can be directly controlled through the MCU of the IDCM. The above information transmission process can use the CAN protocol. This ensures the stability and timeliness of control. For the trigger information corresponding to the control signal sent by the IDCM, the vehicle-mounted robotic arm can be controlled based on the vehicle-mounted robotic arm service (Bot Service). For example, Bot Service can implement functions such as communication, scene encapsulation, vehicle-mounted robotic arm status query, and vehicle-mounted robotic arm driving. The communication function principle includes: exchanging data with external data interfaces (such as Open API) and receiving control instructions in .json format from the upper scene engine. Scene encapsulation may refer to encapsulating the aforementioned different scenes to obtain an encapsulated control instruction sequence to control the vehicle-mounted robotic arm.

For the encapsulated control instruction sequence, the current status, target status, and trigger conditions of the vehicle-mounted controllable components can be combined to comprehensively determine whether the execution of the control instruction sequence is satisfied. If it is not satisfied, the corresponding return value will be returned. For example, the return value may contain error code and other information. If satisfied, the control instruction sequence is executed.

In one implementation, when there is no conflict between the first control instruction sequence and the second control instruction sequence, the vehicle-mounted controllable component is controlled to execute the first control instruction sequence and the second control instruction sequence in parallel.

The first control instruction sequence and the second control instruction sequence that do not conflict may be executed in parallel. It can realize personalized experiences and services for the driver, co-driver, and rear passengers. In summary, by combining safety, user experience, service priority, and triggering timing, different services can be provided to the main driver, co-driver, and rear passengers at the same time. When there is a conflict between vehicle-mounted controllable components, it is necessary to consider the conflicting vehicle-mounted controllable components and those related to the vehicle-mounted controllable components from the perspectives of safety and experience, combined with conflict resolution strategies. Priority arbitration is performed between connected components.

Embodiment 10

As shown in Figure 9, Embodiment 10 of the present application relates to a control device for a vehicle-mounted robotic arm. The device may include: a first control instruction sequence generation module 901, configured to generate a pair of control instructions including a vehicle-mounted robotic arm based on the first trigger information. The first control instruction sequence of the vehicle-mounted controllable components in the vehicle; the first trigger information is determined based on the instruction information of different people in the vehicle and/or the environmental information of the current vehicle location; the second control instruction sequence generation module 902 uses When the second trigger information is received during the execution of the first control instruction sequence by the vehicle-mounted controllable component, a second control instruction sequence of the vehicle-mounted controllable component including the vehicle-mounted manipulator is generated; the second trigger information is Determined based on the instruction information of different people in the vehicle and/or the environmental information of the current vehicle location; the conflict resolution strategy determination module 903 is used to determine when there is a conflict between the first control instruction sequence and the second control instruction sequence. Conflict resolution strategy to resolve the conflict; the conflict situation includes that the control objects of the first control instruction sequence and the second control instruction sequence both include the vehicle-mounted mechanical arm; wherein the first control instruction sequence is at least based on the received control object including the vehicle-mounted mechanical arm. The target script sequence of the arm control instruction is obtained through script analysis, and the target script sequence is determined by executing the method for determining the script sequence provided in Embodiment 1 of the present application.

In one implementation, the conflict resolution strategy determination module 903 may further include: a type determination sub-module, used to determine the types of the first trigger information and the second trigger information; a conflict resolution strategy determination execution sub-module, used to determine the type of the first trigger information and the second trigger information when there is If the specified type or the priority of the specified type of trigger information is higher than the priority of another trigger information, the other trigger information will be ignored.

In one embodiment, the control device of the vehicle-mounted robotic arm may further include: a sending module, specifically configured to directly send the control instruction sequence corresponding to the specified type of trigger information to the vehicle-mounted controllable component.

In one implementation, the conflict resolution strategy determination module 903 may further include: a type determination sub-module, used to determine the types of the first trigger information and the second trigger information; an attribute determination sub-module, used in the case of the same type Next, determine the attributes of the first control instruction sequence and the second control instruction sequence. The attributes include template class attributes or customized class attributes; the conflict resolution strategy determines the execution sub-module, which is used when the attributes are different, or the attributes are the same and belong to the customized class. Under the condition, the vehicle-mounted robotic arm is controlled to pause, and when a new control instruction sequence is received, the vehicle-mounted robotic arm is controlled to execute the new control instruction sequence.

In one implementation, the conflict resolution strategy determination module 903 may further include: a priority comparison submodule, used to compare the first control instruction sequence and the second control sequence when the attributes are the same and belong to customized attributes. The priority of the instruction sequence; the conflict resolution strategy determination execution sub-module is also used to control the vehicle-mounted manipulator to execute the control instruction sequence with high priority based on the comparison results.

In one embodiment, the control device of the vehicle-mounted robotic arm may also include: a state detection module for detecting the state of the vehicle-mounted controllable components in real time, where the state includes a normal state or an abnormal state; and an executability determination module for detecting the state of the vehicle-mounted controllable component according to The detection result of the status of the vehicle-mounted controllable component determines the executability of the first control instruction sequence and/or the second control instruction sequence.

In one embodiment, the control device of the vehicle-mounted manipulator may further include: an execution control module, configured to control the vehicle-mounted controllable component to execute the first control component in parallel when there is no conflict between the first control instruction sequence and the second control instruction sequence. a control instruction sequence and a second control instruction sequence.

In the technical solution of this application, the acquisition, storage and application of user personal information involved are in compliance with relevant laws and regulations and do not violate public order and good customs.

Embodiment 11

Embodiment 11 of the present application provides a vehicle-mounted display device, including: a control unit for executing the control method of a vehicle-mounted robotic arm provided in Embodiment 9 of the present application, or including a vehicle-mounted mechanical arm provided in Embodiment 10 of the present application. Arm control device; a display module composed of a vehicle-mounted robotic arm and a vehicle-mounted screen. The vehicle-mounted robotic arm is used to drive the vehicle-mounted screen to complete at least one target action.

Embodiment 12

Embodiment 12 of the present application provides a vehicle, including: a control unit configured to execute the control method of the vehicle-mounted robotic arm provided in Embodiment 9 of the present application, or including the vehicle-mounted robotic arm provided in Embodiment 10 of the present application. Control device; a display module composed of a vehicle-mounted robotic arm and a vehicle-mounted screen. The vehicle-mounted robotic arm is used to drive the vehicle-mounted screen to complete at least one target action.

Embodiment 13

Figure 10 shows a structural block diagram of an electronic device according to an embodiment of the present application. As shown in Figure 10, the electronic device includes: a memory 1001 and a processor 1002. The memory 1001 stores instructions that can be run on the processor 1002. When the processor 1002 executes this instruction, the method for determining the script sequence in the above embodiment is implemented. The number of memory 1001 and processor 1002 may be one or more. This electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, Personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are examples only and are not intended to limit the implementation of the present application as described and/or claimed herein.

The electronic device may also include a communication interface 1003 for communicating with external devices for interactive data transmission. The individual devices are connected to each other using different buses and can be installed on a common motherboard or otherwise installed as needed. The processor 1002 may process instructions executed within the electronic device, including instructions stored in or on memory to display graphical information of a GUI on an external input/output device, such as a display device coupled to an interface. In other embodiments, multiple processors and/or multiple buses may be used with multiple memories and multiple memories, if desired. Likewise, multiple electronic devices can be connected, each device providing part of the necessary operation (eg, as a server array, a set of blade servers, or a multi-processor system). The bus can be divided into address bus, data bus, control bus, etc. For ease of presentation, only one thick line is used in Figure 10, but it does not mean that there is only one bus or one type of bus.

Optionally, in terms of specific implementation, if the memory 1001, the processor 1002 and the communication interface 1003 are integrated on one chip, the memory 1001, the processor 1002 and the communication interface 1003 can communicate with each other through the internal interface.

It should be understood that the above-mentioned processor can be a central processing unit (Central Processing Unit, CPU), or other general-purpose processor, digital signal processor (Digital Signal Processing, DSP), application specific integrated circuit (Application Specific Integrated Circuit, ASIC), Field Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor, etc. It is worth noting that the processor may be a processor that supports Advanced RISC Machines (ARM) architecture.

Embodiments of the present application provide a computer-readable storage medium (such as the above-mentioned memory 1001), which stores computer instructions. When the program is executed by a processor, the method provided in the embodiment of the present application is implemented.

Optionally, the memory 1001 may include a stored program area and a stored data area, wherein the stored program area may store an operating system and an application program required for at least one function; the stored data area may store programs created according to the use of XXX's electronic device. Data etc. In addition, the memory 1001 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 1001 optionally includes memories remotely located relative to the processor 1002, and these remote memories can be connected to XXX's electronic device through a network. Examples of the above-mentioned networks include but are not limited to the Internet, intranets, local area networks, mobile communication networks and combinations thereof.

An embodiment of the present application also provides a vehicle-mounted display device, which may include the electronic device provided by the embodiment of the present application and the vehicle-mounted robotic arm and vehicle-mounted screen of any embodiment of the present application.

Embodiments of the present application also provide a vehicle-mounted display device, which may include a control unit, the robotic arm and the vehicle-mounted screen of any embodiment of the present application, wherein the control unit is used to execute the script sequence determination method of any embodiment of the present application, to determine the target script sequence, and control the vehicle-mounted robotic arm based on the target script sequence; or the robotic arm control unit may include the script sequence determining device of any embodiment of the present application, and be used to determine the target script using the script sequence determining device. Sequence, and based on the target script sequence, control the vehicle-mounted robotic arm.

An embodiment of the present application also provides a vehicle, which may include the vehicle-mounted robotic arm and the vehicle-mounted screen provided by the embodiment of the present application and any embodiment of the present application.

Embodiments of the present application also provide a vehicle, which may include a control unit and the robotic arm and vehicle-mounted screen of any embodiment of the present application, wherein the control unit is used to execute the script sequence determination method of any embodiment of the present application to determine The target script sequence, and based on the target script sequence, controls the vehicle-mounted robotic arm; or the robotic arm control unit may include the script sequence determining device of any embodiment of the present application, and be used to determine the target script sequence using the above script sequence determining device, And based on the target script sequence, control the vehicle-mounted robotic arm.

For example, the electronic device may be a Body Domain Control Module (BDCM), an Infotainment Domain Control Module (IDCM), or a Vehicle Domain Control module. Module (VDCM), an automated driving domain control module (Automated-driving Domain Control Module, ADCM), and a robotic arm control unit (Robotic Arm Controller, RAC).

For example, the vehicle in this embodiment can be a vehicle driven by any power such as a fuel vehicle, an electric vehicle, a solar vehicle, etc. For example, the vehicle in this embodiment may be an autonomous vehicle.

Other components of the vehicle in this embodiment, such as the frame and the car.

It should be noted that the vehicle-mounted robotic arm provided in the embodiment of the present application is specifically shown in Figures 11 to 112. The following describes the vehicle-mounted robotic arm according to the embodiment of the present application with reference to Figures 11 to 112.

As shown in Figure 11, an optional embodiment of a vehicle-mounted robotic arm is shown, including: a multi-degree-of-freedom adjustment mechanism fixed on the back of the vehicle-mounted screen 3, and multiple telescopic units installed on the multi-degree-of-freedom adjustment mechanism; Among them, the vehicle-mounted robotic arm is used to drive the vehicle-mounted screen 3 to complete any one or more of the following four actions. The four actions include: vehicle-mounted screen translation action, vehicle-mounted screen flipping action, vehicle-mounted screen rotation action, and vehicle-mounted screen forward and backward movement. . Further, assuming that the vehicle-mounted screen 3 is in a state where no action is taking place as the initial state, when the vehicle-mounted screen 3 is in the initial state, it has an initial axis in space, and the initial axis is perpendicular to the plane where the vehicle-mounted screen 3 is located in the initial state. ; Then the above four actions have the following specific explanations: Vehicle screen translation action: As shown in Figure 16, the front of the vehicle screen 3 moves in translation, and the front of the vehicle screen 3 moves at any angle in a plane perpendicular to the initial axis. translation; vehicle screen flip action: as shown in Figure 15, the front of the vehicle screen 3 is flipped, and there is an angle between the front of the vehicle screen 3 and the initial axis after the flip; vehicle screen rotation action: as shown in Figure 17 As shown in Figure 111, the front of the vehicle screen 3 rotates around the initial axis or an axis parallel to the initial axis; the vehicle screen moves forward and backward: as shown in Figure 111, the front of the vehicle screen 3 moves forward and backward, and the front of the vehicle screen 3 moves forward and backward. The direction of movement is set parallel to the initial axis. In other words, multiple telescopic units are used to drive the vehicle-mounted screen 3 to flip up, down, left, and right, and a multi-degree-of-freedom adjustment mechanism is used to drive the vehicle-mounted screen 3 to rotate and translate. Up, down, left, and right refer to when the above-mentioned vehicle-mounted screen 3 is facing the user. In the vertical state of the vehicle, relative to the initial position, the vehicle-mounted screen 3 performs movements such as the upper part tilting to the rear, the lower part tilting to the rear, the left part tilting to the rear, and the right part tilting to the rear.

Furthermore, as an optional embodiment, the vehicle-mounted central control screen adjustment mechanism involved in this application may not be provided with the above-mentioned multi-degree-of-freedom adjustment mechanism, and directly connect the vehicle-mounted screen 3 with several telescopic units 13, thereby realizing that the control screen can It only swings up, down, left and right according to the usage requirements.

Furthermore, as an optional embodiment, the vehicle-mounted central control screen adjustment mechanism involved in this application can also directly connect the vehicle-mounted screen 3 with the multi-degree-of-freedom adjustment mechanism without providing the above-mentioned telescopic units, thereby realizing the control screen itself. Rotational and translational sliding.

In another optional embodiment, the movement of each telescopic unit is connected to a multi-degree-of-freedom adjustment mechanism, and the driving end of each telescopic unit is connected to a driving part.

Further, as an optional embodiment, the driving part is a center console inside the car.

Furthermore, as an optional embodiment, a corresponding control system is provided in the center console, and the control system is used to control the telescopic actions of several telescopic units and the movement of the multi-degree-of-freedom adjustment mechanism.

Furthermore, as an optional embodiment, the telescopic unit can also be a bendable rod with a ball head structure, and the telescopic rod interferes with the side of the ball head and the multi-degree-of-freedom adjustment mechanism away from the vehicle screen 3 Extruded installation. Furthermore, the user can manually apply force on the vehicle-mounted screen 3, so that the multi-degree-of-freedom adjustment mechanism exerts force on the ball head structure as part of the force transmission, and when the ball head structure is made to swing to a certain angle, the ball head structure and the multi-degree-of-freedom adjustment mechanism are Sufficient friction is generated between the adjustment mechanisms so that the vehicle screen 3 maintains its current position.

Further, as an optional embodiment, the moving end of each telescopic unit includes: a linear motion unit 14 and a multi-degree of freedom connector. One end of the linear motion unit 14 is connected to the multi-degree of freedom connector, and the multi-degree of freedom connector is connected. The device is installed on a multi-degree-of-freedom adjustment mechanism.

Further, as an optional embodiment, the multi-degree-of-freedom connector is a ball joint structure or a universal joint structure.

Further, as shown in Figure 12, as an optional embodiment, the ball joint structure includes: a ball joint 15 and a ball socket slider 16. The ball joint 15 is fixedly connected to the linear motion unit 14, and each ball joint 15 is It is installed in a ball socket slide block 16, and each ball socket slide block 16 is installed on a multi-degree-of-freedom adjustment mechanism.

Furthermore, as an optional embodiment, each ball socket slider 16 has a spherical recess that matches the ball joint 15 .

Further, as an optional embodiment, the universal joint joint structure includes: a first rotating part, a second rotating part and a hinge part connecting the first and second rotating parts, one end of the first rotating part is fixed to the telescopic unit The other end of the first rotating part is connected to one end of the hinge part, the other end of the hinge part is rotatably connected to one end of the second rotating part, and the other end of the second rotating part is fixedly connected to the multi-degree-of-freedom adjustment mechanism.

Further, as an optional embodiment, the linear motion unit 14 is an electric push rod or a manual push rod. Furthermore, when the linear motion unit 14 is a manual push rod, the user can manually push the vehicle-mounted screen 3 to make the vehicle-mounted screen 3 make corresponding actions; when the electric push rod of the vehicle-mounted mechanical arm is in a power-off state, the electric push rod The user should be allowed to manually drive the electric push rod to expand and contract accordingly to complete the movement of the vehicle screen 3.

Furthermore, as an optional embodiment, the contact surfaces between the ball joint 15, the linear motion unit 14, the multi-degree-of-freedom adjustment mechanism and other movable parts of the vehicle-mounted robotic arm have a certain degree of friction with the corresponding connection parts. Resistance, frictional resistance is used to maintain the stability of the current posture during the vehicle's form.

Further, as an optional embodiment, the electric push rod or the vehicle-mounted screen 3 is provided with a force sensing part. The force sensing part is used to obtain information about the external force at the corresponding position. The force sensing part passes The information of the external force determines the target of the force: when the target of the force is a passenger, that is, when the passenger pushes the vehicle screen 3, the force sensing part analyzes the information of the external force into action information, and causes the vehicle-mounted robotic arm to Carry out corresponding actions according to the action information to provide assistance to the passengers in the process of pushing the vehicle-mounted screen, so that the passengers can easily drive the vehicle-mounted screen 3 to complete the corresponding actions; when the target of the force is an external force that is not the passenger's intention to push, that is, When the vehicle encounters bumps or a passenger performs a touch operation on the vehicle screen, the vehicle-mounted robotic arm does not move or drives the corresponding driving part to drive in reverse to control the vehicle-mounted screen 3 to maintain the current state.

Further, as an optional embodiment, it also includes: a vehicle collision detection system. The vehicle collision detection system is installed on the car. The vehicle collision system is used to detect the driving information of the vehicle in real time. When the vehicle is about to collide or has already collided, The vehicle-mounted robotic arm immediately drives the vehicle-mounted screen 3 to quickly move away from the passengers, so as to prevent the passengers from colliding with the vehicle-mounted screen 3 due to the inertia of the collision and causing injuries.

Furthermore, as an optional embodiment, the linear motion unit 14 is an unpowered telescopic rod.

Further, as an optional embodiment, the linear motion unit 14 is a hydraulic push rod.

Furthermore, as an optional embodiment, it also includes: a plurality of guide rails 17, each guide rail 17 is installed on a multi-degree-of-freedom adjustment mechanism, and each ball socket slider 16 is slidably installed on a guide rail 17.

Based on the above, this application also has the following implementation modes:

In an optional embodiment of the present application, the vehicle-mounted robotic arm includes three telescopic units.

In an optional embodiment of the present application, the number of guide rails 17 , ball joints 15 , ball socket sliders 16 and linear motion units 14 are each three.

In an optional embodiment of the present application, the three guide rails 17 are arranged at intervals of 150 degrees in pairs. Furthermore, that is, the extension lines of the three guide rails 17 converge at one point after intersecting, and the two adjacent extension lines are spaced 150 degrees apart.

In an optional embodiment of the present application, the multi-degree-of-freedom adjustment mechanism includes: a sliding mechanism 5 and a rotating mechanism 2, the rotating mechanism 2 is connected to the sliding mechanism 5, and one of the rotating mechanism 2 and the sliding mechanism 5 is connected to the vehicle screen 3, The other one of the rotating mechanism 2 and the sliding mechanism 5 is connected to the telescopic unit.

As shown in Figure 13, in the optional embodiment of the present application, the rotating mechanism 2 includes: a support part, a motor 21, a worm 22, a turbine 23 and a sector gear 24. The motor 21, the worm 22 and the turbine 23 are all installed on the support part. On the vehicle, the motor 21 is connected to the worm 22, the worm 22 is drivingly connected to the turbine 23, the turbine 23 is meshed with the sector gear 24, and the sector gear 24 is installed on the vehicle screen 3 or the sliding mechanism 5. Further, the support part is connected with several telescopic units or connected with the sliding mechanism 5 . The support part has a shell-type structure, and the above-mentioned shell-type structure accommodates the motor 21, the worm 22, the turbine 23 and the sector gear 24 in the support part. The outer edge of one end of the sector gear 24 protrudes radially outward to form an arcuate portion. An arcuate rack is provided on the arcuate portion. The tooth tips of the rack are arranged radially inward. The rack is drivingly connected to the turbine 23 .

The optional embodiment of the present application also includes: two rotation stop blocks 25, the two rotation stop blocks 25 are installed on the support part, and the two rotation stop blocks 25 are respectively operable with both ends of the sector gear 24. Offset settings.

In an optional embodiment of the present application, the rotating mechanism 2 also includes: a rotating shaft, one end of the rotating shaft is fixedly installed on the support part, and the other end of the rotating shaft is rotatably installed on the back of the vehicle screen 3 or slides through a bearing or the like. Agency 5 on.

In an optional embodiment of the present application, the sliding mechanism 5 includes: a first sliding part, a second sliding part and a sliding driving device. The first sliding part is slidably connected to the second sliding part, and the sliding direction is consistent with the direction of the rotating mechanism 2 The rotation axis is arranged vertically, and the sliding driving device is installed between the first sliding part and the second sliding part. The sliding driving device is used to drive the relative sliding between the first sliding part and the second sliding part.

Optional embodiments of the present application also include: a visual sensing device, the visual sensor is installed on the front of the vehicle screen 3, the visual sensor is used to detect the position of the user's eyes, and the visual sensor is connected to the control system. Furthermore, with the help of the angle adjustment mechanism and the multi-degree-of-freedom adjustment mechanism, the front of the vehicle screen 3 is set as far as possible toward the driver through the visual sensor and the control system, where the visual sensor is a smart camera or a human body position sensor. In another optional embodiment, the visual sensing device includes a plurality of visual sensors, and the plurality of visual sensors are arranged on the front of the vehicle screen 3 and/or at any position in the cab of the automobile. In a specific application of the visual sensing device, as a method of using the visual sensing device, the visual sensor is used to recognize the specified gestures of the passengers of the car, and based on the recognized gestures, the vehicle-mounted robotic arm The vehicle-mounted screen 3 is controlled to perform matching actions based on the gesture action information obtained by the visual sensor. For example, gestures can control the car screen to move forward and backward, or trigger the car screen to face the user according to a certain application scenario.

Optional embodiments of the present application also include: a mechanism controller, which is used to control the vehicle-mounted robotic arm. The mechanism controller can be used to collect information about passengers in the vehicle. The information includes but is not limited to the corresponding The passenger's personal information such as height information, weight information or gender information is collected. At the same time, the mechanism controller is also used to collect the posture information of the corresponding member's seat. By processing the passenger's personal information and the posture information of the seat, automatic control The vehicle-mounted mechanical arm or the seat posture adjustment mechanism makes the front of the vehicle-mounted screen 3 face the passenger. At the same time, the mechanism controller also collects relative position information with the car's steering wheel at all times, and limits the action range of the vehicle screen 3 by calculating a safe distance between the vehicle screen 3 and the steering wheel. That is, the mechanism controller controls the movement of the vehicle-mounted robotic arm. The distance between the vehicle screen 3 and the steering wheel is controlled to be always greater than or equal to the above-mentioned safe distance.

Optional embodiments of the present application also include: a sound sensing device. The sound sensing device includes several sound receivers. The plurality of sound receivers are arranged on the outer edge of the vehicle screen 3 or in the cab of the car. The sound sensing device The device is connected to the control system. Furthermore, the sound sensing device is used to detect the user's speaking position, thereby adjusting the orientation position of the vehicle-mounted screen 3 .

As shown in Figures 14 and 112, in an optional embodiment of the present application, unlike the above technical solution that uses the matching of the ball and socket slider 16 and the slide rail to adapt to the displacement of the end of the telescopic unit, the present application Another technical solution for providing the above-mentioned displacement is also provided, specifically as follows: the driving end of each telescopic unit of the present application also has a rotating member 4, and each rotating member 4 is installed on the driving part. The moving ends are all rotatably connected to a rotating member 4. That is, the adaptive displacement that originally occurred on the guide rail 17 is transferred to the rotation of the telescopic unit itself to match the displacement of the ball joint 15 of the telescopic unit.

In an optional embodiment of the present application, corresponding to the above-mentioned embodiment using the rotating member 4, the multi-degree-of-freedom connector of the telescopic unit is no longer connected to the guide rail 17, but is directly connected to the multi-degree-of-freedom adjustment mechanism.

In an optional embodiment of the present application, the ball socket slider 16 is directly fixed on the multi-degree-of-freedom adjustment mechanism, and the ball joint 15 is rotatably installed on the ball socket slider 16 .

In an optional embodiment of the present application, the rotating member 4 is rotationally connected to the middle part of the linear motion unit 14 .

In an optional embodiment of the present application, the rotating member 4 is arranged in a shaft-shaped structure.

In an optional embodiment of the present application, the driving part has a shell structure, and the three rotating parts 4 are all fixedly installed on the shell.

In an optional embodiment of the present application, the axes of the three rotating members 4 intersect and are arranged at intervals of 150 degrees.

As an optional embodiment, the vehicle-mounted central control screen includes a vehicle-mounted screen 3 and a vehicle-mounted mechanical arm of any one of the above, that is, the corresponding vehicle-mounted screen 3 is a central control screen, and the central control screen is arranged in the front cabin of the car. At the console, several telescopic units and multi-degree-of-freedom adjustment mechanisms work together to drive the central control screen to complete the translational movement of the vehicle screen 3, the flipping movement of the vehicle screen 3, the rotation movement of the vehicle screen 3 and the forward and backward movement of the vehicle screen 3 in the small space inside the car. action. In addition to the application scenario examples mentioned above, the individual implementation or combined implementation based on the above actions can also form the presentation of various other application scenarios, such as turning and/or rotating to the user (driver or passenger in the car) Say hello, specific flipping actions in certain car-machine interaction scenarios (showing rocking effect or shaking head effect or head-tilt effect), specific flipping actions when over-the-air upgrade (OTA) is successful, and facing users when triggered by a certain car-machine interaction scene (such as using the car screen as a makeup mirror), triggering the car screen to move forward and backward with gesture operations or other motion capture, triggering the car screen rotation with specific content or motion capture, adjusting the amount of movement of each of the above single actions or action combinations with voice, etc.

The embodiments of the present application also provide a vehicle, including a control device according to the above embodiments of the present application, a vehicle control system according to the above embodiments of the present application, a vehicle-mounted robotic arm according to the above embodiments of the present application, and a vehicle-mounted robotic arm according to the above embodiments of the present application. Embodiments of vehicle-mounted At least one item in the central control screen.

In the description of this specification, reference to the terms "one embodiment," "some embodiments," "an example," "specific examples," or "some examples" or the like means that specific features are described in connection with the embodiment or example. , structures, materials or features are included in at least one embodiment or example of the present application. Furthermore, the specific features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, those skilled in the art may combine and combine different embodiments or examples and features of different embodiments or examples described in this specification unless they are inconsistent with each other.

In addition, the terms “first” and “second” are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of indicated technical features. Thus, features defined as "first" and "second" may explicitly or implicitly include at least one of these features. In the description of this application, "plurality" means two or more than two, unless otherwise explicitly and specifically limited.

Any process or method description in a flowchart or otherwise described herein may be understood to represent a representation that includes one or more (two or more) executable instructions for implementing the specified logical functions or steps of the process. A module, fragment, or portion of code. And the scope of the preferred embodiments of the present application includes additional implementations in which functions may be performed out of the order shown or discussed, including in a substantially concurrent manner or in the reverse order, depending on the functionality involved.

The logic and/or steps represented in the flowcharts or otherwise described herein, for example, may be considered a sequenced list of executable instructions for implementing the logical functions, and may be embodied in any computer-readable medium, For use by, or in combination with, instruction execution systems, devices or devices (such as computer-based systems, systems including processors or other systems that can fetch instructions from and execute instructions from the instruction execution system, device or device) or equipment.

It should be understood that various parts of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the method in the above embodiment can be completed by instructing relevant hardware through a program. The program can be stored in a computer-readable storage medium. When executed, the program includes one of the steps of the method embodiment or other steps. combination.

In addition, each functional unit in various embodiments of the present application can be integrated into a processing module, or each unit can exist physically alone, or two or more units can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software function modules. If the above integrated modules are implemented in the form of software function modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a magnetic disk or an optical disk, etc.

The above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person familiar with the technical field can easily think of various changes or modifications within the technical scope disclosed in the present application. Replacements, these should all be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

A method for determining script sequence, including:

For each control script in the first script sequence, the reachability of the vehicle-mounted robotic arm during the execution of the preset action according to the first script sequence is sequentially detected. The control script is used to control the vehicle-mounted robotic arm to perform the preset actions. The script of the preset action, the reachability condition is used to indicate whether the vehicle-mounted robotic arm exceeds the preset reachable space;

According to the reachability situation, the first script sequence is used to determine a target script sequence for controlling the vehicle-mounted manipulator to perform the preset action.
The method according to claim 1, wherein the step of using the first script sequence to determine a target script sequence for controlling the vehicle-mounted manipulator to perform the preset action according to the reachability condition includes:

When it is detected that the reachable situation includes a first reachable situation, a target control script is obtained. The first reachable situation is used to indicate that the vehicle-mounted manipulator exceeds the reachable space. The target control script is to cause The control script for the vehicle-mounted robotic arm exceeding the accessible space;

Insert a reset script into the first script sequence as a previous control script of the target control script to obtain a second script sequence, where the reset script is a script used to control the vehicle-mounted manipulator to perform a reset action;

For each of the control scripts in the second script sequence, sequentially detect the reachability of the vehicle-mounted robotic arm during the execution of the preset action according to the second script sequence;

According to the reachability situation, the second script sequence is used to determine the target script sequence.
The method according to claim 2, wherein determining the target script sequence using the second script sequence according to the reachability condition includes:

When the reachability situations are all second reachability situations, the second script sequence is determined as the target script sequence, and the second reachability situation is used to indicate that the vehicle-mounted manipulator does not exceed the target script sequence. Describe the accessible space.
The method according to claim 3, wherein the determining the target script sequence using the second script sequence according to the reachability situation further includes:

Obtaining the target control script when detecting that the reachability situation includes a first reachability situation;

The reset script is inserted into the second script sequence as the previous control script of the target control script to obtain a third script sequence, and so on until the target script sequence is determined.
The method according to claim 1, wherein the step of using the first script sequence to determine a target script sequence for controlling the vehicle-mounted manipulator to perform the preset action according to the reachability condition includes:

When the reachability situations are all second reachability situations, the first script sequence is determined as the target script sequence, and the second reachability situation is used to indicate that the vehicle-mounted manipulator does not exceed the target script sequence. Describe the accessible space.
The method according to any one of claims 1-5, wherein the detection method of the reachability condition includes:

Determine the spatial range defined by the accessible space;

For the currently detected control script, predict the real-time position of the vehicle-mounted robotic arm during execution corresponding to the preset action;

The reachability situation is detected using the spatial range and the real-time location.
The method according to any one of claims 1-5, further comprising:

The target script sequence is sent to the vehicle-mounted client for calling and parsing the target script sequence, so that the vehicle-mounted client controls the vehicle-mounted manipulator to perform the preset action according to the target script sequence.
The method according to any one of claims 1-5, further comprising:

Call and parse the target script sequence;

According to the parsed target script sequence, the vehicle-mounted robotic arm is controlled to perform the preset action according to the target script sequence.
A control method for a vehicle-mounted robotic arm, which includes:

Determine a target script sequence based on a control script set, where the control script set includes multiple control scripts, where the control scripts are scripts used to control the vehicle-mounted robotic arm to perform preset actions;

Control the vehicle-mounted manipulator to perform the corresponding preset action in the preset accessible space according to the target script sequence;

Wherein, determining the target script sequence based on the control script set includes: generating a first script sequence in response to the control script selected by the user in the control script set; for each of the first script sequence The control script sequentially detects the reachability of the vehicle-mounted manipulator when executing the preset action according to the first script sequence. The reachability is used to indicate whether the vehicle-mounted manipulator exceeds the preset reachability. Space; according to the reachability situation, use the first script sequence to determine the target script sequence.
The method according to claim 9, wherein determining the target script sequence using the first script sequence according to the reachability condition includes:

When it is detected that the reachable situation includes a first reachable situation, a target control script is obtained. The first reachable situation is used to indicate that the vehicle-mounted manipulator exceeds the reachable space. The target control script is to cause The control script for the vehicle-mounted robotic arm exceeding the accessible space;

Insert a reset script into the first script sequence as a previous control script of the target control script to obtain a second script sequence, where the reset script is a script used to control the vehicle-mounted manipulator to perform a reset action;

For each of the control scripts in the second script sequence, sequentially detect the reachability of the vehicle-mounted robotic arm during the execution of the preset action according to the second script sequence;

According to the reachability situation, the second script sequence is used to determine the target script sequence.
The method according to claim 10, wherein the determining the target script sequence using the second script sequence according to the reachability condition includes:

When the reachability situations are all second reachability situations, the second script sequence is determined as the target script sequence, and the second reachability situation is used to indicate that the vehicle-mounted manipulator does not exceed the target script sequence. Describe the accessible space.
The method according to claim 11, wherein the determining the target script sequence using the second script sequence according to the reachability situation further includes:

Obtaining the target control script when detecting that the reachability situation includes a first reachability situation;

The reset script is inserted into the second script sequence as the previous control script of the target control script to obtain a third script sequence, and so on until the target script sequence is determined.
The method according to claim 9, wherein the step of using the first script sequence to determine a target script sequence for controlling the vehicle-mounted manipulator to perform the preset action according to the reachability condition includes:

When the reachability situations are all second reachability situations, the first script sequence is determined as the target script sequence, and the second reachability situation is used to indicate that the vehicle-mounted manipulator does not exceed the target script sequence. Describe the accessible space.
The method of claim 9, wherein determining the target script sequence based on the control script set further includes:

In response to a selection operation triggered by the user for the control script, determine the control script currently selected by the user;

Determine the next control script that can be selected in the control script set. The control script that can be selected is after the vehicle-mounted manipulator completes the preset action corresponding to the currently selected control script. The vehicle-mounted robotic arm can be controlled to further execute the control script corresponding to the preset action within a preset accessible space;

Determine the control script that can be selected as the next control script that can be selected by the user;

The user-selectable control script is set to a user-selectable state, and the user-selectable control script is displayed for the user to trigger the selection of the user-selectable control script. operate;

And so on until the target script sequence is determined.
The method according to claim 14, wherein determining the next selectable control script in the control script set includes:

Predict the stopping position of the vehicle-mounted robotic arm after executing the preset action corresponding to the currently selected control script;

For each control script in the control script set, predict the real-time position of the vehicle-mounted robotic arm during further execution of the corresponding preset action based on the stop position;

The selectable control script is determined according to the real-time location.
The method of claim 15, wherein determining the selectable control script according to the real-time location includes:

Determine the spatial range defined by the accessible space;

For each control script, detect whether the real-time position is within the spatial range;

The control script whose real-time position is within the spatial range is determined as the selectable control script.
The method according to claim 14, wherein the displaying the control script available for selection by the user includes:

Set the control script available for user selection to a highlighted display mode;

The control scripts available for selection by the user are highlighted.
A control method for a vehicle-mounted robotic arm, which includes:

According to the first trigger information, a first control instruction sequence for the vehicle-mounted controllable components including the vehicle-mounted robotic arm is generated; the first trigger information is based on the instruction information of different people in the vehicle and/or the current location of the vehicle. Environmental information is determined;

When the second trigger information is received during the execution of the first control instruction sequence by the vehicle-mounted controllable component, generate a second control instruction sequence for the vehicle-mounted controllable component including the vehicle-mounted manipulator; The second trigger information is determined based on the instruction information of different people in the vehicle and/or the environmental information of the current vehicle location;

When there is a conflict between the first control instruction sequence and the second control instruction sequence, determine a conflict resolution strategy to resolve the conflict; the conflict situation includes the first control instruction sequence and the second control instruction sequence. The control objects of the second control instruction sequence all include the vehicle-mounted robotic arm;

Wherein, the first control instruction sequence is obtained based on at least script parsing of a received target script sequence including vehicle-mounted manipulator control instructions, and the target script sequence is obtained by executing the script sequence described in any one of claims 1 to 8 The determination method is determined.
The method according to claim 18, wherein, when there is a conflict between the first control instruction sequence and the second control instruction sequence, determining a conflict resolution strategy to resolve the conflict includes:

Determine the types of the first trigger information and the second trigger information;

In the case where a specified type exists, or the priority of trigger information of the specified type is higher than the priority of another trigger information, the other trigger information is ignored.
The method of claim 19, further comprising:

The control instruction sequence corresponding to the specified type of trigger information is directly sent to the vehicle-mounted controllable component.
The method according to claim 18, wherein, when there is a conflict between the first control instruction sequence and the second control instruction sequence, determining a conflict resolution strategy to resolve the conflict includes:

Determine the types of the first trigger information and the second trigger information;

In the case of the same type, determine the attributes of the first control instruction sequence and the second control instruction sequence, where the attributes include template class attributes or customized class attributes;

If the attributes are different, or if the attributes are the same and belong to the customized category, the vehicle-mounted robotic arm is controlled to pause the action. When a new control instruction sequence is received, the vehicle-mounted robotic arm is controlled to execute the new control instruction sequence.
The method according to claim 21, wherein said determining a conflict resolution strategy to resolve the conflict when there is a conflict between the first control instruction sequence and the second control instruction sequence further includes:

If the attributes are the same and belong to customized attributes, compare the priorities of the first control instruction sequence and the second control instruction sequence;

According to the comparison result, the vehicle-mounted manipulator is controlled to execute a control instruction sequence with a high priority.
The method of claim 18, further comprising:

Detect the status of the vehicle-mounted controllable components in real time, and the status includes a normal state or an abnormal state;

The executability of the first control instruction sequence and/or the second control instruction sequence is determined according to the detection result of the status of the vehicle-mounted controllable component.
The method of claim 18, further comprising:

When there is no conflict between the first control instruction sequence and the second control instruction sequence, the vehicle-mounted controllable component is controlled to execute the first control instruction sequence and the second control instruction sequence in parallel.
A device for determining a script sequence, which includes:

The reachability detection unit is used to sequentially detect the vehicle-mounted robotic arm according to the required control scripts in the first script sequence. The reachability situation during the execution of the preset action by the first script sequence. The control script is a script used to control the vehicle-mounted mechanical arm to perform the preset action. The reachability condition is used to represent the vehicle-mounted machinery. Whether the arm exceeds the preset accessible space;

A target script sequence determination unit is configured to use the first script sequence to determine a target script sequence for controlling the vehicle-mounted manipulator to perform the preset action according to the reachability situation.
A vehicle-mounted display device, which includes: a control unit, a vehicle-mounted robotic arm, and a vehicle-mounted screen;

The control unit is configured to execute the method for determining a script sequence according to any one of claims 1 to 8 to determine a target script sequence, and to control the vehicle-mounted robotic arm based on the target script sequence; or include the right The script sequence determining device of claim 25, the control unit is configured to determine a target script sequence using the script sequence determining device, and control the vehicle-mounted robotic arm based on the target script sequence;

The vehicle-mounted robotic arm is used to drive the vehicle-mounted screen to complete at least one preset action;

The vehicle-mounted screen is connected to the vehicle-mounted robotic arm.
A vehicle, which includes: a control unit, a vehicle-mounted robotic arm, and a vehicle-mounted screen;

The control unit is configured to execute the method for determining a script sequence according to any one of claims 1 to 8 to determine a target script sequence, and to control the vehicle-mounted robotic arm based on the target script sequence; or include the right The script sequence determining device of claim 25, the control unit is configured to determine a target script sequence using the script sequence determining device, and control the vehicle-mounted robotic arm based on the target script sequence;

The vehicle-mounted robotic arm is used to drive the vehicle-mounted screen to complete at least one preset action;

The vehicle-mounted screen is connected to the vehicle-mounted robotic arm.
A control device for a vehicle-mounted robotic arm, which includes:

A target script sequence determination unit, configured to determine a target script sequence based on a control script set, where the control script set includes a plurality of control scripts, and the control scripts are scripts used to control the vehicle-mounted robotic arm to perform preset actions;

A vehicle-mounted manipulator control unit, used to control the vehicle-mounted manipulator to perform the corresponding preset action in the preset accessible space according to the target script sequence;

Wherein, the target script sequence determining unit includes:

The first script sequence generating subunit is used to generate the first script sequence in response to the control script selected by the user in the control script set;

The first detection subunit of reachability is used to detect the reachability of the vehicle-mounted manipulator in sequence for each control script in the first script sequence when executing the corresponding preset action according to the first script sequence. The reachability condition is used to indicate the reachability of the vehicle-mounted manipulator. Whether the robotic arm exceeds the preset accessible space;

The first determination subunit of the target script sequence is used to determine the target script sequence using the first script sequence according to the reachability situation.
A vehicle-mounted display device, which includes: a control unit, a vehicle-mounted robotic arm, and a vehicle-mounted screen;

The control unit is used to execute the control method of the vehicle-mounted robotic arm according to any one of claims 9-17; or includes the control device of the vehicle-mounted robotic arm according to claim 28;

The vehicle-mounted robotic arm is used to drive the vehicle-mounted screen to complete at least one preset action;

The vehicle-mounted screen is connected to the vehicle-mounted robotic arm.
A vehicle, which includes: a control unit, a vehicle-mounted robotic arm, and a vehicle-mounted screen;

The control unit is used to execute the control method of the vehicle-mounted robotic arm according to any one of claims 9-17; or includes the control device of the vehicle-mounted robotic arm according to claim 28;

The vehicle-mounted robotic arm is used to drive the vehicle-mounted screen to complete at least one preset action;

The vehicle-mounted screen is connected to the vehicle-mounted robotic arm.
A control device for a vehicle-mounted robotic arm, which includes:

A first control instruction sequence generation module, configured to generate a first control instruction sequence for vehicle-mounted controllable components including a vehicle-mounted robotic arm based on first trigger information; the first trigger information is based on instructions from different people in the vehicle information, and/or determined by environmental information of the current vehicle location;

A second control instruction sequence generating module, configured to generate a vehicle-mounted control sequence including the vehicle-mounted manipulator when the second trigger information is received during the execution of the first control instruction sequence by the vehicle-mounted controllable component. The second control instruction sequence of the controllable component column; the second trigger information is determined based on the instruction information of different people in the vehicle and/or the environmental information of the current vehicle location;

A conflict resolution strategy determination module, configured to determine a conflict resolution strategy to resolve the conflict when there is a conflict between the first control instruction sequence and the second control instruction sequence; the conflict situation includes: The control objects of the first control instruction sequence and the second control instruction sequence both include the vehicle-mounted robotic arm;

Wherein, the first control instruction sequence is obtained based on at least script parsing of a received target script sequence including vehicle-mounted manipulator control instructions, and the target script sequence is obtained by executing the script sequence described in any one of claims 1 to 8 The determination method is determined.
A vehicle-mounted display device, including:

A control unit, used to execute the control method according to any one of claims 18 to 24, or a control device including the vehicle-mounted robotic arm according to claim 23;

A display module is composed of a vehicle-mounted robotic arm and a vehicle-mounted screen. The vehicle-mounted robotic arm is used to drive the vehicle-mounted screen to complete at least one target action.
A vehicle including:

A control unit, used to execute the control method according to any one of claims 18 to 24, or a control device including the vehicle-mounted robotic arm according to claim 31;

A display module is composed of a vehicle-mounted robotic arm and a vehicle-mounted screen. The vehicle-mounted robotic arm is used to drive the vehicle-mounted screen to complete at least one target action.
An electronic device, including:

at least one processor; and

a memory communicatively connected to the at least one processor; wherein,

The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform any one of claims 1-24. Methods.