WO2023036230A1 - 一种执行指令确定方法、装置、设备及存储介质 - Google Patents

一种执行指令确定方法、装置、设备及存储介质 Download PDF

Info

Publication number
WO2023036230A1
WO2023036230A1 PCT/CN2022/117769 CN2022117769W WO2023036230A1 WO 2023036230 A1 WO2023036230 A1 WO 2023036230A1 CN 2022117769 W CN2022117769 W CN 2022117769W WO 2023036230 A1 WO2023036230 A1 WO 2023036230A1
Authority
WO
WIPO (PCT)
Prior art keywords
execution instruction
sample
information
target
model
Prior art date
Application number
PCT/CN2022/117769
Other languages
English (en)
French (fr)
Inventor
郑红丽
吴明哲
刘朝阳
蔡旭
樊永友
Original Assignee
中国第一汽车股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中国第一汽车股份有限公司 filed Critical 中国第一汽车股份有限公司
Priority to EP22866692.1A priority Critical patent/EP4369185A1/en
Publication of WO2023036230A1 publication Critical patent/WO2023036230A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/448—Execution paradigms, e.g. implementations of programming paradigms
    • G06F9/4482—Procedural
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/451—Execution arrangements for user interfaces

Definitions

  • the embodiments of the present application relate to the technical field of vehicles, for example, to a method, device, device, and storage medium for determining an execution instruction.
  • modality refers to the way things happen or exist
  • multimodal refers to the combination of two or more modalities in various forms.
  • Each source or form of information can be called a modality.
  • Multimodal fusion technology in deep learning is the process by which models process different forms of data when completing analysis and recognition tasks.
  • the fusion of multimodal data can provide more information for model decision-making, thereby improving the accuracy of the overall result of decision-making.
  • the purpose is to establish a model that can process and correlate information from multiple modalities.
  • the reason for the fusion of modalities is that different modalities have different expressions and look at things from different angles, so there are some intersections, which means that there is information redundancy, and complementarity is a phenomenon that is better than single features. There may even be a variety of different information interactions between modalities. If multi-modal information can be processed reasonably, rich feature information can be obtained.
  • the related technical solutions can be divided into two types.
  • One is the method without multi-mode fusion, that is, the operation is performed immediately after receiving the input from the input source.
  • the phenomenon of misoperation after information is to only use simple judgment and analysis, which makes the interactive experience worse.
  • Embodiments of the present application provide a method, device, device, and storage medium for determining an execution instruction, so as to improve interactive experience.
  • the embodiment of the present application provides a method for determining an execution instruction, including:
  • the input information includes: at least one first execution instruction and at least one modal information;
  • a target execution instruction is determined based on the at least one first execution instruction and the at least one second execution instruction.
  • the embodiment of the present application also provides an apparatus for determining an execution instruction, which includes:
  • the input information acquisition module is configured to acquire input information, wherein the input information includes: at least one first execution instruction and at least one modal information;
  • the second execution instruction determination module is configured to determine at least one second execution instruction according to the at least one modality information
  • the target execution instruction determination module is configured to determine the target execution instruction according to the at least one first execution instruction and the at least one second execution instruction.
  • the embodiment of the present application also provides an electronic device, including:
  • the processor When the program is executed by the processor, the processor is made to implement the method for determining an execution instruction as described in any one of the embodiments of the present application.
  • the embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements any one of the embodiments of the present application. Execution instruction determination method.
  • FIG. 1 is a flowchart of a method for determining an execution instruction in an embodiment of the present application
  • Fig. 1a is a flow chart of the training sample acquisition method in the embodiment of the present application.
  • FIG. 2 is a schematic structural diagram of an execution instruction determining device in an embodiment of the present application
  • FIG. 3 is a schematic structural diagram of an electronic device in an embodiment of the present application.
  • Fig. 4 is a schematic structural diagram of a computer-readable storage medium containing a computer program in an embodiment of the present application.
  • Fig. 1 is a flowchart of a method for determining an execution instruction provided by an embodiment of the present application. This embodiment is applicable to the situation of determining an execution instruction, and the method can be executed by the apparatus for determining an execution instruction in the embodiment of the application.
  • the device can be implemented in the form of software and/or hardware, as shown in Figure 1, the method includes the following steps:
  • the acquired input information may be modal information or instruction information.
  • the input information is obtained by the manager.
  • the manager can be a controller equipped with an Electronic Control Unit (ECU) on the car, it can be an infotainment host, controllers for the body, chassis, engine, etc., and controllers related to automatic driving.
  • ECU Electronic Control Unit
  • the input information may be information input through screens of controllers such as infotainment systems and air-conditioning systems, information input through voice, information input through cameras, hard keys, buttons or Information entered through smart surface buttons, etc., information entered through external connected devices, and information entered through built-in applications.
  • controllers such as infotainment systems and air-conditioning systems, information input through voice, information input through cameras, hard keys, buttons or Information entered through smart surface buttons, etc., information entered through external connected devices, and information entered through built-in applications.
  • the external connection equipment may include:
  • USB Universal Serial Bus
  • WIFI wireless communication technology
  • Bluetooth Bluetooth
  • I/O input/output
  • the information input through the built-in application may include:
  • the input information includes: at least one first execution instruction and at least one modal information;
  • the first execution instruction may be an instruction generated by pressing a window button, or an instruction generated by pressing an air-conditioning button, or may be an instruction generated by clicking a preset area on the touch screen, It may also be an instruction generated by clicking a switch of the vehicle-mounted terminal.
  • the modality information can be obtained through sensor collection or user input.
  • the modality information can be:
  • Personal identity information face, fingerprint, login account, etc.
  • Information media voice, video, text, graphics and pictures
  • Sensors such as radar, infrared, navigation and positioning information, temperature and humidity, air quality, odor sensors, lights, etc.;
  • Historical interaction information occupant operations and other interaction information within a certain period of time.
  • S120 Determine at least one second execution instruction according to the at least one modality information.
  • the second execution instruction may be an instruction for the window button, for example, to open all the windows, and the second execution instruction may also be an instruction for the air conditioner, for example, it may be to turn the air conditioner The temperature is adjusted to 27 degrees, and the second execution instruction may also be an instruction for the vehicle terminal, for example, it may be to adjust the sound to 35 decibels.
  • the manner of determining at least one second execution instruction according to the at least one modality information may be: establishing a table in advance, the table stores the correspondence between the modality information and the second execution instruction, and according to the at least one modality modal information lookup table to obtain at least one second execution instruction corresponding to at least one modal information; the method of determining at least one second execution instruction according to the at least one modal information may also be: inputting the at least one modal information
  • An artificial intelligence (AI) model is obtained by obtaining at least one second execution instruction, wherein the AI model is obtained by iteratively training a neural network model with a target sample set, and the target sample set includes: scene samples and corresponding execution instructions of the scene samples
  • the instruction sample and the scene sample include: at least one modal information sample.
  • the neural network model is described based on the mathematical model of neurons, and is a complex network system formed by a large number of simple processing units widely connected to each other.
  • the neural network model may be a convolutional neural network model, or may be a deconvolutional neural network, or may be a recurrent neural network.
  • determining at least one second execution instruction according to the at least one modality information may include:
  • the execution instruction samples, the scene samples include: at least one modal information sample.
  • the AI model is obtained by iteratively training the neural network model through the target sample set, and the target sample set is constantly updated.
  • the method before inputting at least one modal information into the AI model and obtaining at least one second execution instruction, the method further includes: iteratively training the neural network model through the target sample set. Iteratively train the neural network model through the target sample set, which can include:
  • the manner of inputting the at least one modal information sample into the neural network model to obtain the predictive execution instruction may be: input at least one modal information sample into the neural network model, and the output of the neural network model is Predictively execute instructions.
  • the method of obtaining the first scene sample corresponding to the at least one modality information sample is: pre-establishing a scene database, querying the scene database according to the at least one modality information sample, and obtaining the at least one modality The first scene sample corresponding to the information sample.
  • the manner of obtaining the first execution instruction sample corresponding to the first scene sample is: querying the scene library according to the first scene sample, and obtaining the first execution instruction sample corresponding to the first scene sample.
  • the objective function may be a loss function of a neural network model.
  • FIG. 1a it is a flow chart of the method for obtaining training samples in the embodiment of the present application.
  • the scene library contains multiple scene samples, each scene sample contains multiple modal information samples, and each scene sample corresponds to multiple execution instruction samples.
  • the database is updated according to the first operation after the first operation is acquired within a preset time.
  • input information is acquired, wherein the input information includes: at least one first execution instruction and at least one modality information, and at least one second execution instruction is determined according to the at least one modality information, and then based on the The at least one first execution instruction and the at least one second execution instruction are determined to determine a target execution instruction. Effectively avoid the phenomenon that the execution instruction is determined incorrectly due to misoperation after receiving the input information. Compared with the simple judgment and analysis method in the related art, this embodiment needs to combine the information input by the user and the modal information to generate Instructions are comprehensively judged, thereby better enhancing the interactive experience.
  • the following steps may be performed:
  • the executor can be:
  • Information entertainment system display screen instrument screen, central control screen, co-pilot screen, rear screen, etc.;
  • HUD Head Up Display
  • the target sample set is updated according to the execution instruction to be added.
  • the method of judging whether the first operation satisfies the preset condition may be as follows: the user pre-establishes an operation list, and judges whether the first operation exists in the operation list, and if the first operation exists in the operation list , the first operation satisfies the preset condition, and if the first operation does not exist in the operation list, the first operation does not satisfy the preset condition.
  • the method of judging whether the first operation satisfies the preset condition may be: establishing a judgment model in advance, inputting the first operation into the judgment model, and if the output result is yes, then determining that the first operation satisfies the preset condition , if the output result is negative, it is determined that the first operation does not satisfy the preset condition.
  • the target sample set is a sample set obtained by collecting the actual operation of the user in different scenarios.
  • the AI model is obtained by iteratively training the neural network model through the target sample set. Some user operations within a certain period of time update the target sample set and optimize the target sample set.
  • the AI model that inputs the modal information obtains the second execution instruction, and then determines the target execution instruction according to the first execution instruction and the second execution instruction. If within a period of time after the execution of the target execution instruction, the first execution instruction When an operation is triggered, information will be collected for this operation. If this operation is in a list preset by the user, the execution instruction to be added will be determined according to this operation, and then the target sample set will be updated according to the execution instruction to be added.
  • the execution instruction to be added may be directly added to the instruction corresponding to the modality, or the instruction corresponding to the modality may be replaced by the execution instruction to be added.
  • the instruction corresponding to the original modal 1 is recorded as A instruction
  • the execution instruction to be added is recorded as B instruction.
  • the execution instruction B to be added can be added directly In the instruction corresponding to modal 1
  • instruction A can also be replaced with the execution instruction B to be added; if there is no instruction sample corresponding to modal 1, it is not only necessary to add the execution instruction B to be added to modal 1, It also needs to be added to the command corresponding to modal 1.
  • FIG. 2 is a schematic structural diagram of an apparatus for determining an execution instruction provided by an embodiment of the present application. This embodiment can be applied to the situation of determining the execution instruction, the device can be realized by software and/or hardware, and the device can be integrated in a device providing computer functions, as shown in Figure 2, the execution instruction is determined
  • the apparatus may include: an information acquisition module 210 , a second execution instruction determination module 220 and a target execution instruction determination module 230 .
  • the information obtaining module 210 is configured to obtain input information, wherein the input information includes: at least one first execution instruction and at least one modal information;
  • the second execution instruction determining module 220 is configured to determine at least one second execution instruction according to the at least one modality information
  • the target execution instruction determining module 230 is configured to determine the target execution instruction according to the at least one first execution instruction and the at least one second execution instruction.
  • a pair of second execution instructions is determined according to the obtained plurality of modality information, and a target execution instruction is determined based on the first execution instruction and the second execution instruction , to achieve a more friendly interaction and get a better interactive experience.
  • the second execution instruction determining module 220 includes:
  • the second execution instruction acquisition unit is configured to input the at least one modal information into the AI model to obtain at least one second execution instruction, wherein the AI model is obtained by iteratively training a neural network model with a target sample set, and the target
  • the sample set includes: scene samples and execution instruction samples corresponding to the scene samples, and the scene samples include: at least one modal information sample.
  • the device for executing instruction determination also includes an AI model acquisition module, and the AI model acquisition module is configured to:
  • the device for performing instruction determination also includes:
  • the execution module is configured to execute the target execution instruction after determining the target execution instruction according to the at least one first execution instruction and the at least one second execution instruction.
  • the device for performing instruction determination also includes:
  • the instruction determination module is configured to, after executing the target execution instruction, based on the judgment result that the first operation is triggered within a preset time and the first operation satisfies a preset condition, determine according to the first operation Add execution instructions;
  • An update module configured to update the target sample set according to the execution instruction to be added.
  • input information is acquired, wherein the input information includes: at least one first execution instruction and at least one modality information, and at least one second execution instruction is determined according to the at least one modality information, and then based on the The at least one first execution instruction and the at least one second execution instruction are determined to determine a target execution instruction. Effectively avoid the phenomenon that the execution instruction is determined incorrectly due to misoperation after receiving the input information. Compared with the simple judgment and analysis method in the related art, this embodiment needs to combine the information input by the user and the modal information to generate Instructions are comprehensively judged, thereby better enhancing the interactive experience.
  • FIG. 3 is a schematic structural diagram of an electronic device provided in Embodiment 3 of the present application.
  • FIG. 3 shows a block diagram of an electronic device 312 suitable for implementing embodiments of the present application.
  • Electronic device 312 shown in FIG. 3 is only one example.
  • Device 312 is typically a computing device for trajectory fitting functions.
  • electronic device 312 takes the form of a general-purpose computing device.
  • the components of the electronic device 312 may include: a processor 316, a storage device 328, and a bus 318 connecting different system components (including the storage device 328 and the processor 316).
  • Bus 318 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus structures.
  • bus structures include, for example, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MCA) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) ) Local bus and Peripheral Component Interconnect (PCI) bus.
  • ISA Industry Standard Architecture
  • MCA Micro Channel Architecture
  • VESA Video Electronics Standards Association
  • PCI Peripheral Component Interconnect
  • Electronic device 312 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 312 and include both volatile and nonvolatile media, removable and non-removable media.
  • Storage device 328 may include computer system readable media in the form of volatile memory, such as random access memory (Random Access Memory, RAM) 330 and/or cache memory 332 .
  • Electronic device 312 may include other removable/non-removable, volatile/nonvolatile computer system storage media.
  • storage system 334 may be used to read and write to non-removable, non-volatile magnetic media (commonly referred to as a "hard drive”), Read-write disk drives, and removable non-volatile discs (such as Compact Disc-Read Only Memory (CD-ROM), Digital Video Disc-Read Only Memory, DVD-ROM) or other optical media) read and write optical disc drives.
  • CD-ROM Compact Disc-Read Only Memory
  • DVD-ROM Digital Video Disc-Read Only Memory
  • each drive may be connected to bus 318 through one or more data media interfaces.
  • the storage device 328 may include at least one program product having a set (for example, at least one) of program modules configured to execute the functions of the various embodiments of the present application.
  • a program 336 having a set (at least one) of program modules 326 may be stored, for example, in storage device 328, such program modules 326 including an operating system, one or more application programs, other program modules, and program data, in these examples Each or a combination may include implementations of network environments.
  • the program modules 326 generally perform the functions and/or methods of the embodiments described herein.
  • the electronic device 312 may also communicate with one or more external devices 314 (such as keyboards, pointing devices, cameras, displays 324, etc.), and may also communicate with one or more devices that enable a user to interact with the electronic device 312, and/or Or communicate with any device (eg, network card, modem, etc.) that enables the electronic device 312 to communicate with one or more other computing devices. Such communication may occur through input/output (I/O) interface 322 .
  • the electronic device 312 can also communicate with one or more networks (such as a local area network (Local Area Network, LAN), a wide area network, Wide Area Network, WAN) and/or a public network, such as the Internet, through the network adapter 320.
  • networks such as a local area network (Local Area Network, LAN), a wide area network, Wide Area Network, WAN) and/or a public network, such as the Internet
  • network adapter 320 communicates with other modules of electronic device 312 via bus 318 .
  • other hardware and/or software modules may be used in conjunction with electronic device 312, including: microcode, device drivers, redundant processing units, external disk drive arrays, Redundant Arrays of Independent Disks (RAID) systems, tape drives And data backup storage system, etc.
  • the processor 316 executes various functional applications and data processing by running the program stored in the storage device 328, for example, implementing the execution instruction determination method provided by the above-mentioned embodiments of the present application, the method includes:
  • the input information includes: at least one first execution instruction and at least one modal information;
  • a target execution instruction is determined according to the at least one first execution instruction and the at least one second execution instruction.
  • Fig. 4 is a schematic structural diagram of a computer-readable storage medium containing a computer program in an embodiment of the present application.
  • the embodiment of the present application provides a computer-readable storage medium 41, on which a computer program 410 is stored.
  • the execution instruction determination method provided in all the application embodiments of the present application is realized. Methods include:
  • the input information includes: at least one first execution instruction and at least one modal information;
  • a target execution instruction is determined according to the at least one first execution instruction and the at least one second execution instruction.
  • a computer readable medium may be a computer readable signal medium or a computer readable storage medium or any combination of the two.
  • a computer-readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof.
  • a computer readable storage medium may include: an electrical connection having one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable Read memory (Erasable Programmable Read-Only Memory, EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.
  • a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
  • a computer readable signal medium may include a data signal carrying computer readable program code in baseband or as part of a carrier wave. Such propagated data signals may take many forms, including electromagnetic signals, optical signals, or any suitable combination of the foregoing.
  • a computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. .
  • Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
  • the client and the server can communicate using any currently known or future-developed network protocols such as Hyper Text Transfer Protocol (Hyper Text Transfer Protocol, HTTP), and can communicate with any form or medium of digital Data communication (eg, communication network) interconnections.
  • HTTP Hyper Text Transfer Protocol
  • Examples of communication networks include local area networks (LANs), wide area networks (WANs), internetworks (eg, the Internet), and peer-to-peer networks (eg, ad hoc peer-to-peer networks), as well as any currently known or future developed networks.
  • the above-mentioned computer-readable medium may be included in the above-mentioned electronic device, or may exist independently without being incorporated into the electronic device.
  • Computer program code for carrying out the operations of the present application may be written in one or more programming languages or combinations thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional process programming language - such as "C" or a similar programming language.
  • the program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server.
  • the remote computer may be connected to the user computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g. via the Internet using an Internet Service Provider). .
  • LAN local area network
  • WAN wide area network
  • Internet Service Provider e.g. via the Internet using an Internet Service Provider.
  • each block in a flowchart or block diagram may represent a module, program segment, or portion of code that contains one or more logical functions for implementing specified executable instructions.
  • the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved.
  • each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations can be implemented by a dedicated hardware-based system that performs the specified functions or operations , or may be implemented by a combination of dedicated hardware and computer instructions.
  • the units involved in the embodiments described in the present disclosure may be implemented by software or by hardware.
  • FPGA Field Programmable Gate Array
  • ASIC Application Specific Integrated Circuit
  • ASSP Application Specific Standard Parts
  • SOC System on Chip
  • Complex Programmable Logic Device Complex Programmable Logic Device, CPLD
  • a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device.
  • a machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium.
  • a machine-readable medium may comprise an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing.
  • a machine-readable storage medium may include one or more wire-based electrical connections, a portable computer disk, a hard disk, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash flash memory), optical fiber, compact disc read only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination of the foregoing.
  • RAM random access memory
  • ROM read only memory
  • EPROM or flash flash memory erasable programmable read only memory
  • CD-ROM compact disc read only memory
  • magnetic storage or any suitable combination of the foregoing.

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • User Interface Of Digital Computer (AREA)
  • Machine Translation (AREA)

Abstract

本申请公开了一种执行指令确定方法、装置、设备及存储介质。该方法包括：获取输入信息，其中，所述输入信息包括：至少一个第一执行指令和至少一个模态信息，根据所述至少一个模态信息确定至少一个第二执行指令，根据所述至少一个第一执行指令和所述至少一个第二执行指令确定目标执行指令。

Description

一种执行指令确定方法、装置、设备及存储介质
本申请要求在2021年09月10日提交中国专利局、申请号为202111060589.0的中国专利申请的优先权，该申请的全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及车辆技术领域，例如涉及一种执行指令确定方法、装置、设备及存储介质。
背景技术
一般来说，模态是指事物发生或存在的方式，多模态是指两个或者两个以上的模态的各种形式的组合。对每一种信息的来源或者形式，都可以称为一种模态。深度学习中的多模态融合技术是模型在完成分析和识别任务时处理不同形式的数据的过程。多模态数据的融合可以为模型决策提供更多的信息，从而提高了决策总体结果的准确率，目的是建立能够处理和关联来自多种模态信息的模型。之所以要对模态进行融合，是因为不同模态的表现方式不一样，看待事物的角度也会不一样，所以存在一些交叉即存在信息冗余，互补即是比单特征更优的现象，甚至模态间可能还存在多种不同的信息交互，如果能合理的处理多模态信息，就能得到丰富特征信息。
相关技术方案分可为两种，一种是未进行多模融合的方法，即收到输入源的输入后，立即执行操作，由于未采用多模融合的方法可造成当收到输入源的输入信息后产生误操作的现象。另一种是只通过简单的判断分析，使得交互体验感更差。
发明内容
本申请实施例提供一种执行指令确定方法、装置、设备及存储介质，提升交互体验。
第一方面，本申请实施例提供了一种执行指令确定方法，包括：
获取输入信息，其中，所述输入信息包括：至少一个第一执行指令和至少一个模态信息；
根据所述至少一个模态信息确定至少一个第二执行指令；
根据所述至少一个第一执行指令和所述至少一个第二执行指令确定目标执 行指令。
第二方面，本申请实施例还提供了一种执行指令确定装置，该装置包括：
输入信息获取模块，被设置为获取输入信息，其中，所述输入信息包括：至少一个第一执行指令和至少一个模态信息；
第二执行指令确定模块，被设置为根据所述至少一个模态信息确定至少一个第二执行指令；
目标执行指令确定模块，被设置为根据所述至少一个第一执行指令和所述至少一个第二执行指令确定目标执行指令。
第三方面，本申请实施例还提供了一种电子设备，包括：
处理器；
存储器，用于存储程序；
当所述程序被所述处理器执行时，使得所述处理器实现如本申请实施例中任一所述的执行指令确定方法。
第四方面，本申请实施例还提供了一种计算机可读存储介质，所述存储介质上存储有计算机程序，所述计算机程序被处理器执行时实现如本申请实施例中任一所述的执行指令确定方法。
附图说明
下面将对实施例中所需要使用的附图作简单地介绍，应当理解，以下附图仅示出了本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他相关的附图。
图1是本申请实施例中的一种执行指令确定方法的流程图；
图1a是本申请实施例中的训练样本获取方法的流程图；
图2是本申请实施例中的一种执行指令确定装置的结构示意图；
图3是本申请实施例中的一种电子设备的结构示意图；
图4是本申请实施例中的一种包含计算机程序的计算机可读存储介质的结构示意图。
具体实施方式
下面结合附图和实施例对本申请作详细说明。可以理解的是，此处所描述的实施例仅仅用于解释本申请。另外还需要说明的是，为了便于描述，附图中仅示出了与本申请相关的部分而非全部结构。此外，在不冲突的情况下，本申请中的实施例及实施例中的特征可以相互组合。
应当提到的是，一些示例性实施例被描述成作为流程图描绘的处理或方法。虽然流程图将各项操作(或步骤)描述成顺序的处理，但是其中的许多操作可以被并行地、并发地或者同时实施。此外，各项操作的顺序可以被重新安排。当其操作完成时所述处理可以被终止，但是还可以具有未包括在附图中的附加步骤。所述处理可以对应于方法、函数、规程、子例程、子程序等等。此外，在不冲突的情况下，本申请中的实施例及实施例中的特征可以相互组合。
本申请使用的术语“包括”及其变形是开放性包括，即“包括但不限于”。术语“基于”是“至少部分地基于”。术语“一个实施例”表示“至少一个实施例”。
应注意到：相似的标号和字母在下面的附图中表示类似项，因此，一旦一项在一个附图中被定义，则在随后的附图中不需要对其进行定义和解释。同时，在本申请的描述中，术语“第一”、“第二”等仅用于区分描述，而不能理解为指示或暗示相对重要性。
实施例一
图1为本申请实施例提供的一种执行指令确定方法的流程图，本实施例可适用于对执行指令进行确定的情况，该方法可以由本申请实施例中的执行指令确定装置来执行，该装置可采用软件和/或硬件的方式实现，如图1所示，该方法包括如下步骤：
S110、获取输入信息。
本实施例中，所获取的输入信息可以是模态信息，也可以是指令信息。
在一实施例中，输入信息是由管理器获取得到的。其中，管理器可以是汽车上一个电子控制单元(Electronic Control Unit，ECU)搭载的控制器，可以是信息娱乐主机，车身、底盘、发动机等控制器以及自动驾驶相关控制器。
本实施例中，输入信息可以是通过信息娱乐系统、空调系统等控制器的屏幕输入的信息、通过语音输入的信息、通过摄像头输入的信息、通过方向盘、空调、车窗的硬按键、按钮或智能表面按键等输入的信息、通过外置连接设备输入的信息以及通过内置应用输入的信息。
本实施例中，外置连接设备可以包括：
通过通用串行总线(Universal Serial Bus，USB)、无线通信技术(WIFI)、蓝牙连接到信息娱乐系统的手机、平板电脑等；
通过整车网络连接(比如：控制器域网(Controller Area Network，CAN)网络、以太网等)连接到整车的外围设备；
通过硬线输入/输出(Input/Output，I/O)连接正常的其他类型的控制器、传感器等；
通过车载以太网连接到整车的云服务器等。
本实施例中，通过内置应用输入的信息可以包括：
整车控制器提供的导航定位信息；
云端传输到本地的道路信息及其他导航信息；
云端传输到本地的天气信息；
云端传输到本地的智慧城市信息。
可选的，所述输入信息包括：至少一个第一执行指令和至少一个模态信息；
本实施例中，所述第一执行指令可以是通过按压车窗按钮生成的指令，也可以是通过按压空调按钮生成的指令，或者可以是通过点击触控屏上的预设区域生成的指令，还可以为通过点击车载终端的开关生成的指令。
本实施例中，所述模态信息可以通过传感器采集得到，或者用户输入得到。
在一实施例中，模态信息可以是：
人的五觉：触觉、听觉、视觉、嗅觉、味觉；
人的身份信息：人脸、指纹、登录账号等；
人的各种情绪：高兴、生气、难过、委屈、哭泣、兴奋等；
信息的媒介：语音、视频、文字、图形图画；
传感器：如雷达、红外、导航定位信息、温湿度、空气质量、气味传感器、灯光等；
公共信息：时间、时区、天气、气温等；
个人喜好：喜欢空调25℃、喜欢流行风音乐、厌恶嘈杂环境、厌恶堵车等；
历史交互信息：乘员在一定时间段内的操作和其他交互信息等。
S120、根据所述至少一个模态信息确定至少一个第二执行指令。
本实施例中，所述第二执行指令可以为针对车窗按钮的指令，例如可以是，将车窗全部打开，所述第二执行指令还可以为针对空调的指令，例如可以是，将空调温度调整至27度，所述第二执行指令还可以为针对车载终端的指令，例如可以是，将声音调整至35分贝。
在一实施例中，根据所述至少一个模态信息确定至少一个第二执行指令的方式可以为：预先建立表格，表格中存储有模态信息和第二执行指令的对应关系，根据至少一个模态信息查找表格，得到至少一个模态信息对应的至少一个第二执行指令；根据所述至少一个模态信息确定至少一个第二执行指令的方式还可以为：将所述至少一个模态信息输入人工智能(Artificial Intelligence，AI)模型，得到至少一个第二执行指令，其中，所述AI模型通过目标样本集迭代训练神经网络模型得到，所述目标样本集包括：场景样本和场景样本对应的执行指令样本，场景样本包括：至少一个模态信息样本。
本实施例中，神经网络模型是以神经元的数学模型为基础来描述的，是由大量的、简单的处理单元广泛地互相连接而形成的复杂网络系统。
本实施例中，所述神经网络模型可以为卷积神经网络模型，或者可以为反卷积神经网络，也可以为循环神经网络。
可选的，根据所述至少一个模态信息确定至少一个第二执行指令可以包括：
将所述至少一个模态信息输入AI模型，得到至少一个第二执行指令，其中，所述AI模型通过目标样本集迭代训练神经网络模型得到，所述目标样本集包括：场景样本和场景样本对应的执行指令样本，场景样本包括：至少一个模态信息样本。
需要说明的是，AI模型是通过目标样本集迭代训练神经网络模型得到，所述目标样本集是不断更新的。
可选的，将至少一个模态信息输入AI模型，得到至少一个第二执行指令之前，该方法还包括：通过目标样本集迭代训练神经网络模型。通过目标样本集迭代训练神经网络模型，可以包括：
建立神经网络模型；
将所述至少一个模态信息样本输入所述神经网络模型得到预测执行指令；
获取所述至少一个模态信息样本对应的第一场景样本；
获取所述第一场景样本对应的第一执行指令样本；
根据所述预测执行指令和所述第一执行指令样本形成的目标函数训练所述神经网络模型的参数；
返回执行将所述至少一个模态信息样本输入所述神经网络模型得到预测执行指令的操作，直至得到AI模型。
在一实施例中，将所述至少一个模态信息样本输入所述神经网络模型得到预测执行指令的方式可以为：将至少一个模态信息样本输入神经网络模型，所述神经网络模型的输出为预测执行指令。
在一实施例中，获取所述至少一个模态信息样本对应的第一场景样本的方式为：预先建立场景库，根据所述至少一个模态信息样本查询场景库，得到所述至少一个模态信息样本对应的第一场景样本。
在一实施例中，获取所述第一场景样本对应的第一执行指令样本的方式为：根据所述第一场景样本查询场景库，得到所述第一场景样本对应的第一执行指令样本。
本实施例中，所述目标函数可以为神经网络模型的损失函数。
S130、根据所述至少一个第一执行指令和所述至少一个第二执行指令确定目标执行指令。
示例性的，如图1a所示，为本申请实施例中的训练样本获取方法的流程图。场景库中包含有多个场景样本，每个场景样本包含多个模态信息样本，每个场景样本对应多个执行指令样本。执行目标执行指令后，在预设时间内，获取到第一操作后，根据第一操作对数据库进行更新。
本实施例通过获取输入信息，其中，所述输入信息包括：至少一个第一执行指令和至少一个模态信息，并根据所述至少一个模态信息以确定至少一个第二执行指令，然后基于所述至少一个第一执行指令和所述至少一个第二执行指令以确定目标执行指令。有效避免了因收到输入信息后误操作导致的执行指令确定有误的现象，相较于相关技术中简单的判断分析的方法，本实施例需要结合用户输入的信息以及模态信息分别所产生的指令，综合进行判断，从而更好的提升了交互体验。
可选的，可以在根据所述至少一个第一执行指令和所述至少一个第二执行指令确定目标执行指令之后，执行下述步骤：
执行所述目标执行指令。
在一实施例中，确定目标执行指令之后，通过执行器执行相应指令。其中，执行器可以为：
信息娱乐系统显示屏幕：仪表屏、中控屏、副驾屏、后排屏幕等；
搭载空调、抬头显示(Head Up Display，HUD)、倒车环视影像、透明A 柱、流媒体后视镜、行车记录仪的显示屏幕；
语音反馈信息；
整车提示信息或提示音等；
方向盘、空调、车窗等硬按键、按钮或智能表面按键的状态灯；
车窗、车门等控制器。
可选的，还可以在执行所述目标执行指令之后，执行如下步骤：
基于在预设时间内第一操作被触发，且所述第一操作满足预设条件的判定结果，根据所述第一操作确定待添加执行指令；
根据所述待添加执行指令更新所述目标样本集。
本实施例中，判断所述第一操作是否满足预设条件的方式可以为：用户预先建立操作列表，判断操作列表中是否存在所述第一操作，若在操作列表中存在所述第一操作，则所述第一操作满足预设条件，若在操作列表中不存在所述第一操作，则所述第一操作不满足预设条件。判断所述第一操作是否满足预设条件的方式可以为：预先建立判断模型，将所述第一操作输入所述判断模型，若输出结果为是，则确定所述第一操作满足预设条件，若输出结果为否，则确定所述第一操作不满足预设条件。
在本实施例中，目标样本集是在处于不同场景下采集用户的实际操作得到的样本集，通过目标样本集迭代训练神经网络模型得到AI模型，在执行目标执行指令之后，还可以为根据一定时间内用户的一些操作，对目标样本集进行更新，优化目标样本集。
在一实施例中，将模态信息输入的AI模型得到第二执行指令，再根据第一执行指令与第二执行指令确定目标执行指令，若在执行目标执行指令之后的一段时间内，第一操作被触发，则对此操作进行信息采集，如果这个操作在用户预先设置的一个列表中，则根据此操作确定待添加执行指令，然后根据待添加执行指令更新目标样本集。
本实施例中，可以直接将待添加执行指令添加至模态对应的指令中，或者可以将模态对应的指令替换为所述待添加执行指令。
示例性的，若存在模态1对应的指令样本，则将原有模态1中对应的指令记为A指令，待添加执行指令记为B指令，此时可以将待添加执行指令B直接添加至模态1对应的指令中，也可以将指令A替换为待添加执行指令B；若不存在模态1对应的指令样本，则不仅需要将该待添加执行指令B添加至模态1中，还需要添加至模态1对应的指令中。
实施例二
图2为本申请实施例提供的一种执行指令确定装置的结构示意图。本实施例可适用于对执行指令确定的情况，该装置可采用软件和/或硬件的方式实现，该装置可集成在提供计算机的功能的设备中，如图2所示，所述执行指令确定的装置可以包括：信息获取模块210、第二执行指令确定模块220和目标执行指令确定模块230。
其中，信息获取模块210，被设置为获取输入信息，其中，所述输入信息包括：至少一个第一执行指令和至少一个模态信息；
第二执行指令确定模块220，被设置为根据所述至少一个模态信息确定至少一个第二执行指令；
目标执行指令确定模块230，被设置为根据所述至少一个第一执行指令和所述至少一个第二执行指令确定目标执行指令。
本实施例通过获取至少一个第一执行指令和至少一个模态信息，以根据获取的多个模态信息确定对个第二执行指令，基于第一执行指令和第二执行指令以确定目标执行指令，实现了更加友好的交互方式，并得到更好的交互体验。
可选的，所述第二执行指令确定模块220包括：
第二执行指令获取单元，被设置为将所述至少一个模态信息输入AI模型，得到至少一个第二执行指令，其中，所述AI模型通过目标样本集迭代训练神经网络模型得到，所述目标样本集包括：场景样本和场景样本对应的执行指令样本，场景样本包括：至少一个模态信息样本。
可选的，所述执行指令确定的装置还包括AI模型获取模块，所述AI模型获取模块被设置为：
建立神经网络模型；
将所述至少一个模态信息样本输入所述神经网络模型得到预测执行指令；
获取所述至少一个模态信息样本对应的第一场景样本；
获取所述第一场景样本对应的第一执行指令样本；
根据所述预测执行指令和所述第一执行指令样本形成的目标函数训练所述神经网络模型的参数；
返回执行将所述至少一个模态信息样本输入所述神经网络模型得到预测执行指令的操作，直至得到AI模型。
可选的，所述执行指令确定的装置还包括：
执行模块，被设置为在根据所述至少一个第一执行指令和所述至少一个第二执行指令确定目标执行指令之后，执行所述目标执行指令。
可选的，所述执行指令确定的装置还包括：
指令确定模块，被设置为在执行所述目标执行指令之后，基于在预设时间内第一操作被触发，且所述第一操作满足预设条件的判定结果，根据所述第一操作确定待添加执行指令；
更新模块，被设置为根据所述待添加执行指令更新所述目标样本集。
本实施例通过获取输入信息，其中，所述输入信息包括：至少一个第一执行指令和至少一个模态信息，并根据所述至少一个模态信息以确定至少一个第二执行指令，然后基于所述至少一个第一执行指令和所述至少一个第二执行指令以确定目标执行指令。有效避免了因收到输入信息后误操作导致的执行指令确定有误的现象，相较于相关技术中简单的判断分析的方法，本实施例需要结合用户输入的信息以及模态信息分别所产生的指令，综合进行判断，从而更好的提升了交互体验。
实施例三
图3为本申请实施例三提供的一种电子设备的结构示意图。图3示出了适于用来实现本申请实施方式的电子设备312的框图。图3显示的电子设备312仅仅是一个示例。设备312是典型的轨迹拟合功能的计算设备。
如图3所示，电子设备312以通用计算设备的形式表现。电子设备312的组件可以包括：处理器316，存储装置328，连接不同系统组件(包括存储装置328和处理器316)的总线318。
总线318表示几类总线结构中的一种或多种，包括存储器总线或者存储器控制器，外围总线，图形加速端口，处理器或者使用多种总线结构中的任意总线结构的局域总线。举例来说，这些体系结构包括工业标准体系结构(Industry Standard Architecture，ISA)总线，微通道体系结构(Micro Channel Architecture，MCA)总线，增强型ISA总线、视频电子标准协会(Video Electronics Standards Association，VESA)局域总线以及外围组件互连(Peripheral Component Interconnect，PCI)总线。
电子设备312典型地包括多种计算机系统可读介质。这些介质可以是任何能够被电子设备312访问的可用介质，包括易失性和非易失性介质，可移动的 和不可移动的介质。
存储装置328可以包括易失性存储器形式的计算机系统可读介质，例如随机存取存储器(Random Access Memory，RAM)330和/或高速缓存存储器332。电子设备312可以包括其它可移动/不可移动的、易失性/非易失性计算机系统存储介质。仅作为举例，存储系统334可以用于读写不可移动的、非易失性磁介质(通常称为“硬盘驱动器”)，可以提供用于对可移动非易失性磁盘(例如“软盘”)读写的磁盘驱动器，以及对可移动非易失性光盘(例如只读光盘(Compact Disc-Read Only Memory，CD-ROM)、数字视盘(Digital Video Disc-Read Only Memory，DVD-ROM)或者其它光介质)读写的光盘驱动器。在这些情况下，每个驱动器可以通过一个或者多个数据介质接口与总线318相连。存储装置328可以包括至少一个程序产品，该程序产品具有一组(例如至少一个)程序模块，这些程序模块被配置以执行本申请各实施例的功能。
具有一组(至少一个)程序模块326的程序336，可以存储在例如存储装置328中，这样的程序模块326包括操作系统、一个或者多个应用程序、其它程序模块以及程序数据，这些示例中的每一个或一种组合中可能包括网络环境的实现。程序模块326通常执行本申请所描述的实施例中的功能和/或方法。
电子设备312也可以与一个或多个外部设备314(例如键盘、指向设备、摄像头、显示器324等)通信，还可与一个或者多个使得用户能与该电子设备312交互的设备通信，和/或与使得该电子设备312能与一个或多个其它计算设备进行通信的任何设备(例如网卡，调制解调器等等)通信。这种通信可以通过输入/输出(I/O)接口322进行。并且，电子设备312还可以通过网络适配器320与一个或者多个网络(例如局域网(Local Area Network，LAN)，广域网Wide Area Network，WAN)和/或公共网络，例如因特网)通信。如图所示，网络适配器320通过总线318与电子设备312的其它模块通信。应当明白，可以结合电子设备312使用其它硬件和/或软件模块，包括：微代码、设备驱动器、冗余处理单元、外部磁盘驱动阵列、磁盘阵列(Redundant Arrays of Independent Disks，RAID)系统、磁带驱动器以及数据备份存储系统等。
处理器316通过运行存储在存储装置328中的程序，从而执行各种功能应用以及数据处理，例如实现本申请上述实施例所提供的执行指令确定方法，该方法包括：
获取输入信息，其中，所述输入信息包括：至少一个第一执行指令和至少一个模态信息；
根据所述至少一个模态信息确定至少一个第二执行指令；
根据所述至少一个第一执行指令和所述至少一个第二执行指令确定目标执行指令。
实施例四
图4为本申请实施例中的一种包含计算机程序的计算机可读存储介质的结构示意图。本申请实施例提供了一种计算机可读存储介质41，其上存储有计算机程序410，该程序被一个或多个处理器执行时实现如本申请所有申请实施例提供的执行指令确定方法，该方法包括：
获取输入信息，其中，所述输入信息包括：至少一个第一执行指令和至少一个模态信息；
根据所述至少一个模态信息确定至少一个第二执行指令；
根据所述至少一个第一执行指令和所述至少一个第二执行指令确定目标执行指令。
可以采用一个或多个计算机可读的介质的任意组合。计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质可以包括：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机存取存储器(RAM)、只读存储器(Read-Only Memory，ROM)、可擦式可编程只读存储器(Erasable Programmable Read-Only Memory，EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本文件中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。
计算机可读的信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括电磁信号、光信号或上述的任意合适的组合。计算机可读的信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。
计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括无线、电线、光缆、RF等等，或者上述的任意合适的组合。
在一些实施方式中，客户端、服务器可以利用诸如超文本传输协议(Hyper Text Transfer Protocol，HTTP)之类的任何当前已知或未来研发的网络协议进行 通信，并且可以与任意形式或介质的数字数据通信(例如，通信网络)互连。通信网络的示例包括局域网(LAN)，广域网(WAN)，网际网(例如，互联网)以及端对端网络(例如，ad hoc端对端网络)，以及任何当前已知或未来研发的网络。
上述计算机可读介质可以是上述电子设备中所包含的；也可以是单独存在，而未装配入该电子设备中。
可以以一种或多种程序设计语言或其组合来编写用于执行本申请操作的计算机程序代码，所述程序设计语言包括面向对象的程序设计语言诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言-诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络包括局域网(LAN)或广域网(WAN)连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。
附图中的流程图和框图，图示了按照本公开各种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，该模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
描述于本公开实施例中所涉及到的单元可以通过软件的方式实现，也可以通过硬件的方式来实现。
本文中以上描述的功能可以至少部分地由一个或多个硬件逻辑部件来执行。例如，可以使用的示范类型的硬件逻辑部件包括：现场可编程门阵列(Field Programmable Gate Array，FPGA)、专用集成电路(Application Specific Integrated Circuit，ASIC)、专用标准产品(Application Specific Standard Parts，ASSP)、片上系统(System on Chip，SOC)、复杂可编程逻辑设备(Complex Programmable Logic Device，CPLD)等等。
在本公开的上下文中，机器可读介质可以是有形的介质，其可以包含或存 储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备结合地使用的程序。机器可读介质可以是机器可读信号介质或机器可读储存介质。机器可读介质可以包括电子的、磁性的、光学的、电磁的、红外的、或半导体系统、装置或设备，或者上述内容的任何合适组合。机器可读存储介质可以包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦除可编程只读存储器(EPROM或快闪存储器)、光纤、便捷式紧凑盘只读存储器(CD-ROM)、光学储存设备、磁储存设备、或上述内容的任何合适组合。

Claims (10)

  1. 一种执行指令确定方法，包括：
    获取输入信息，其中，所述输入信息包括：至少一个第一执行指令和至少一个模态信息；
    根据所述至少一个模态信息确定至少一个第二执行指令；
    根据所述至少一个第一执行指令和所述至少一个第二执行指令确定目标执行指令。
  2. 根据权利要求1所述的方法，其中，根据所述至少一个模态信息确定至少一个第二执行指令，包括：
    将所述至少一个模态信息输入人工智能AI模型，得到至少一个第二执行指令，其中，所述AI模型通过目标样本集迭代训练神经网络模型得到，所述目标样本包括：场景样本和场景样本对应的执行指令样本，所述场景样本包括：至少一个模态信息样本。
  3. 根据权利要求2所述的方法，其中，将所述至少一个模态信息输入AI模型，得到至少一个第二执行指令之前，所述方法还包括：
    建立神经网络模型；
    将所述至少一个模态信息样本输入所述神经网络模型得到预测执行指令；
    获取所述至少一个模态信息样本对应的第一场景样本；
    获取所述第一场景样本对应的第一执行指令样本；
    根据所述预测执行指令和所述第一执行指令样本形成的目标函数训练所述神经网络模型的参数；
    返回执行将所述至少一个模态信息样本输入所述神经网络模型得到预测执行指令的操作，直至得到AI模型。
  4. 根据权利要求1所述的方法，在根据所述至少一个第一执行指令和所述至少一个第二执行指令确定目标执行指令之后，所述方法还包括：
    执行所述目标执行指令。
  5. 根据权利要求4所述的方法，在执行所述目标执行指令之后，所述方法还包括：
    基于在预设时间内第一操作被触发，且所述第一操作满足预设条件的判定结果，根据所述第一操作确定待添加执行指令；
    根据所述待添加执行指令更新所述目标样本集。
  6. 一种执行指令确定装置，包括：
    信息获取模块，被设置为获取输入信息，其中，所述输入信息包括：至少一个第一执行指令和至少一个模态信息；
    第二执行指令确定模块，被设置为根据所述至少一个模态信息确定至少一个第二执行指令；
    目标执行指令确定模块，被设置为根据所述至少一个第一执行指令和所述至少一个第二执行指令确定目标执行指令。
  7. 根据权利要求6所述的装置，其中，所述第二执行指令确定模块包括：
    第二执行指令获取单元，被设置为将所述至少一个模态信息输入AI模型，得到至少一个第二执行指令，其中，所述AI模型通过目标样本集迭代训练神经网络模型得到，所述目标样本集包括：场景样本和场景样本对应的执行指令样本，所述场景样本包括：至少一个模态信息样本。
  8. 根据权利要求7所述的装置，所述装置还包括AI模型获取模块，所述AI模型获取模块被设置为：
    建立神经网络模型；
    将所述至少一个模态信息样本输入所述神经网络模型得到预测执行指令；
    获取所述至少一个模态信息样本对应的第一场景样本；
    获取所述第一场景样本对应的第一执行指令样本；
    根据所述预测执行指令和所述第一执行指令样本形成的目标函数训练所述神经网络模型的参数；
    返回执行将所述至少一个模态信息样本输入所述神经网络模型得到预测执行指令的操作，直至得到AI模型。
  9. 一种电子设备，包括：
    处理器；
    存储器，用于存储程序；
    当所述程序被所述处理器执行时，使得所述处理器实现如权利要求1-5中任一所述的方法。
  10. 一种计算机可读存储介质，所述存储介质上存储有计算机程序，所述计算机程序被处理器执行时实现如权利要求1-5中任一所述的方法。
PCT/CN2022/117769 2021-09-10 2022-09-08 一种执行指令确定方法、装置、设备及存储介质 WO2023036230A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP22866692.1A EP4369185A1 (en) 2021-09-10 2022-09-08 Execution instruction determination method and apparatus, device, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111060589.0A CN113791841A (zh) 2021-09-10 2021-09-10 一种执行指令确定方法、装置、设备及存储介质
CN202111060589.0 2021-09-10

Publications (1)

Publication Number Publication Date
WO2023036230A1 true WO2023036230A1 (zh) 2023-03-16

Family

ID=78879932

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/117769 WO2023036230A1 (zh) 2021-09-10 2022-09-08 一种执行指令确定方法、装置、设备及存储介质

Country Status (3)

Country Link
EP (1) EP4369185A1 (zh)
CN (1) CN113791841A (zh)
WO (1) WO2023036230A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113791841A (zh) * 2021-09-10 2021-12-14 中国第一汽车股份有限公司 一种执行指令确定方法、装置、设备及存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100299044A1 (en) * 2007-11-26 2010-11-25 Equos Research Co., Ltd. Vehicle control device
CN109688565A (zh) * 2018-12-11 2019-04-26 广州小鹏汽车科技有限公司 车辆控制方法、装置、服务器及车辆控制系统
CN111431782A (zh) * 2020-03-23 2020-07-17 深圳市雪锐科技有限公司 车辆交互控制方法、装置及计算机可读存储介质
CN113791841A (zh) * 2021-09-10 2021-12-14 中国第一汽车股份有限公司 一种执行指令确定方法、装置、设备及存储介质
CN113792059A (zh) * 2021-09-10 2021-12-14 中国第一汽车股份有限公司 一种场景库更新方法、装置、设备及存储介质
CN113805698A (zh) * 2021-09-10 2021-12-17 中国第一汽车股份有限公司 一种执行指令确定方法、装置、设备及存储介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100299044A1 (en) * 2007-11-26 2010-11-25 Equos Research Co., Ltd. Vehicle control device
CN109688565A (zh) * 2018-12-11 2019-04-26 广州小鹏汽车科技有限公司 车辆控制方法、装置、服务器及车辆控制系统
CN111431782A (zh) * 2020-03-23 2020-07-17 深圳市雪锐科技有限公司 车辆交互控制方法、装置及计算机可读存储介质
CN113791841A (zh) * 2021-09-10 2021-12-14 中国第一汽车股份有限公司 一种执行指令确定方法、装置、设备及存储介质
CN113792059A (zh) * 2021-09-10 2021-12-14 中国第一汽车股份有限公司 一种场景库更新方法、装置、设备及存储介质
CN113805698A (zh) * 2021-09-10 2021-12-17 中国第一汽车股份有限公司 一种执行指令确定方法、装置、设备及存储介质

Also Published As

Publication number Publication date
EP4369185A1 (en) 2024-05-15
CN113791841A (zh) 2021-12-14

Similar Documents

Publication Publication Date Title
EP3482344B1 (en) Portable personalization
US10913463B2 (en) Gesture based control of autonomous vehicles
US9485474B2 (en) System and method for learning driving information in vehicle
US9613459B2 (en) System and method for in-vehicle interaction
US10764536B2 (en) System and method for a dynamic human machine interface for video conferencing in a vehicle
CN113302664A (zh) 运载工具的多模态用户接口
JP6386618B2 (ja) ジェスチャ用のインテリジェントなチュートリアル
JP6604151B2 (ja) 音声認識制御システム
EP3647914B1 (en) Electronic apparatus and controlling method thereof
WO2023036230A1 (zh) 一种执行指令确定方法、装置、设备及存储介质
WO2024104045A1 (zh) 基于座舱区域获取操作指示的方法、显示方法及相关设备
WO2023036234A1 (zh) 一种管理系统、管理方法、装置、设备及存储介质
WO2023036233A1 (zh) 一种控制方法、装置、设备及存储介质
CN113539265A (zh) 一种控制方法、装置、设备及存储介质
CN113805698B (zh) 一种执行指令确定方法、装置、设备及存储介质
WO2023272629A1 (zh) 界面的控制方法、装置和系统
CN113791842A (zh) 一种管理方法、装置、设备及存储介质
CN116101205A (zh) 基于车内摄像头的智能座舱车内智能感知系统
CN113792059A (zh) 一种场景库更新方法、装置、设备及存储介质
CN113791843A (zh) 一种执行方法、装置、设备及存储介质
US11315335B1 (en) Mixed-reality interaction with touch device
WO2023272635A1 (zh) 一种目标位置的确定方法、确定装置及确定系统
US11983328B2 (en) Apparatus for recognizing gesture in vehicle and method thereof
KR102463581B1 (ko) 대화 시스템 및 이를 포함하는 차량
CN112506125A (zh) 一种多模态控制方法、装置和系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22866692

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2022866692

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2022866692

Country of ref document: EP

Effective date: 20240208

NENP Non-entry into the national phase

Ref country code: DE