JP2008191074A - Output arbitration program and device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To efficiently and quickly provide necessary information to a user. <P>SOLUTION: An arbitration method includes a determination step, in which before an output of first information regarding first action registered in an action queue is completed, it is determined whether a second action registered next in the action queue satisfies prescribed conditions, including the condition that a specific output function used for outputting the second information regarding the second action is in an open state or is a function without interruption control; and a step, in which when it is determined that the second action satisfies the prescribed conditions, a specific output function is requested to output the second information, regarding the second action irrespective of whether the output of the first information is completed. For example, if the output of audio that requires a long output time and the output of an image with a different output function are performed in parallel, necessary information can be provided efficiently and quickly to the user. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a technique for mediating output of information.

  For example, using vehicle-mounted terminals such as car navigation systems, such as outputting road congestion using VICS (Vehicle Information and Communication System) information distributed over the air, or outputting surrounding images acquired by an in-vehicle camera In addition, the driver is notified of information inside and outside the vehicle. In order to improve the safety of the running state, it is desired to notify the driver quickly and efficiently, as well as providing various information received by the in-vehicle terminal in an easily understandable form.

Therefore, for example, Japanese Patent Laid-Open No. 10-104003 discloses a navigation device that can provide necessary information accurately without excessively providing information. Specifically, a navigation device that provides information to a driver is selected by means for selecting information having a higher rank according to a preset priority when there are a plurality of pieces of information to be provided. Means for providing to the driver in order. However, the detailed output arbitration of the present invention as described below is impossible.
Japanese Patent Laid-Open No. 10-104003

  Accordingly, an object of the present invention is to provide a novel technique for enabling a user to efficiently provide necessary information in a short time.

  In the output arbitration method according to the first aspect of the present invention, before the output of the first information related to the first action registered in the action queue has been completed, The second action satisfies a predetermined condition including a condition that a specific output function used for outputting the second information related to the second action is a vacant state or a function without interrupt control. A determination step for determining whether or not the second action satisfies a predetermined condition, the second information is output regardless of whether or not the output of the first information is completed. Requesting the specific output function to output the second information related to the action.

  For example, if the output of sound that takes time to output and the output of images having different output functions are performed in parallel, the necessary information can be efficiently provided to the user in a short time. Note that, even if output from the same speaker, the speech synthesis output and the beep sound output are performed in parallel if the above-described conditions are satisfied.

  Further, the predetermined condition described above may include a condition that the importance of the second action is equal to or more than a threshold value linked to a load state of a user who perceives the first information and the second information. . As a result, when the load state of the user is high, information with relatively low importance is not output.

  Further, the determination step described above includes a step of searching a management table for managing the interrupt type and state for each output function with a specific output function used for outputting the second information related to the second action. You may make it. By using such a management table, processing can be performed efficiently.

  Note that when the priority of the first action is higher than the priority of the second action, the interrupt is not interrupted and the parallel output as described above is performed.

  Further, in the first aspect of the present invention, when it is determined that the second action does not satisfy the predetermined condition, a step of setting the execution state of the second action to a standby state may be further included. If it cannot be output in parallel, it waits until the output of the first information is completed.

  The output arbitration method according to the second aspect of the present invention has a higher priority than the first action in the action queue and the first action during the output of the first information related to the first action registered in the action queue. A stop step of stopping the output of the first information by the specific output function when the second action is registered with the same specific output function used for the output of the first information related to the first action; Requesting the output of the second information related to the action to the specific output function, and the first information registered in the action queue after the output of the second information related to the second action is completed. When it is the order to process the actions again, the step of determining whether the output of the first information related to the first action has been completed up to a predetermined portion, and the first action That when the output of the first information has been completed to the predetermined portion includes a step of canceling the output of the first information.

  If the output can be performed to the extent that the contents can be transmitted in the first output, information can be efficiently transmitted in a short time if similar output is not performed again after the interruption.

  In addition, the above-described stop step searches the management table for managing the interrupt type for each output function with a specific output function, and determines whether the specific output function is an interruptible output function. If the output function is an interruptible output function, the specific output function may include a step of stopping the output of the first information. In this way, it is confirmed that the output function is interruptible.

  Note that information having a higher priority than the second action, such as emergency information, may be interrupted during the output of the second information.

  Note that the predetermined portion may be specified by the output ratio of the first information. Further, it may be determined based on the position of a specific keyword.

  Furthermore, the specific output function described above may further include a step of registering the output ratio of the first information in the variable designated by the first action.

  Further, when the output of the first information related to the first action is not completed up to a predetermined portion, a step of requesting the specific output function to output the first information again may be further included. This is to inform the user again of the information that has become unclear.

  A program for causing a computer to execute this method can be created, and this program is stored in a storage medium or storage device such as a flexible disk, a CD-ROM, a magneto-optical disk, a semiconductor memory, or a hard disk. . Moreover, it may be distributed as a digital signal via a network or the like. The intermediate processing result is temporarily stored in a storage device such as a main memory.

  According to the present invention, necessary information can be efficiently provided to a user in a short time.

  FIG. 1 shows an architecture of an agent framework according to an embodiment of the present invention. A system in which the agent framework is implemented will be described assuming an example of an in-vehicle terminal, but is not necessarily an in-vehicle terminal.

  The agent framework includes an agent framework runtime 1 that is a runtime engine and an agent framework SDK 9 that is an SDK (Software Development Kit) for creating a service application. The architecture includes an application layer 3 that uses the agent framework / runtime 1, a base middleware layer 5 that cooperates with the agent framework / runtime 1, and a device layer 7 that includes various devices mounted on an in-vehicle terminal, for example. Further included.

  The agent framework SDK 9 includes a development API (Application Program Interface) library 94. The development API library 94 includes a logic development interface including a scenario logic interface and an event aggregation logic interface, and an interface definition for the base middleware layer. As will be described later, the scenario logic 32 in the application layer 3 is a C ++ function that can be dynamically called from the scenario XML. There are three types of scenario logic 32: state logic associated with a state, transition logic associated with a transition, and activation and termination logic called in all states and all transitions at the time of activation and termination. The scenario logic interface defines an interface specification for enabling these logics to be loaded from the scenario engine 11 described later. The event aggregation logic interface defines an interface specification that allows the event aggregator 122 described later to load and execute the event aggregation logic. The interface definition for the infrastructure middleware layer is a gateway component that enables the device of the application system (for example, in-vehicle terminal) to be controlled from the agent framework, and should be developed for each device according to the requirements of the application system Is. A device control function interface and a device status update interface are defined so that the mounted components in the base middleware layer 5 can interact with the action conductor 13 and the event engine 12. The device control function interface defines a command request interface and a command cancel interface as callback interfaces in order to implement an execution request / cancellation request from the action conductor 13. The device status update interface provides an interface for updating data when the status of the device changes with respect to the event data table 34 managed by the event engine 12. Although such an agent framework SDK 9 provides other functions necessary for the agent framework, it is not a main part of the present embodiment and will not be described further.

  The agent framework runtime 1 includes a scenario engine 11, an event engine 12, and an action conductor 13. The scenario engine 11 includes a scenario manager 111 and a scenario interpreter 112. The event engine 12 includes an event manager 121 and an event aggregator 122. The action conductor 13 includes an action control engine 131 and a multimodal engine 132.

  The scenario engine 11 inputs / deletes the scenario 31 describing the interaction procedure of the service, and when executing the scenario, each scenario is detected according to the event detected from the input of the service user (user) or the surrounding situation change. It is a scenario execution component that controls the progress of interaction by requesting state management and output actions. The user can use a plurality of services on an ad hoc basis while switching according to the situation. The scenario engine 11 has a function of controlling the progress of service use interaction for the purpose of having a user use the service conveniently, comfortably and safely. Manages the status of multiple services used in the system, determines the output action and the next event request corresponding to the user input event or sensor input change event according to the scenario 31 describing the interaction progress of the service, and the user And the interaction between the service and the service. In addition, when a plurality of services are operated simultaneously, they can be switched and used without a sense of incongruity.

  The scenario manager 111 of the scenario engine 11 is a module that controls the activation of each module in the scenario engine 11. When the scenario engine 11 is activated, the setting information is read and an instance of each module is generated. The module generated by this module includes a scenario interpreter 112. The scenario manager 111 generates as many scenario interpreters 112 as the number of scenarios 31 to be executed, and reads and registers a scenario definition file.

  The scenario interpreter 112 is a module that controls the progress of the interaction according to the definition of the scenario 31. In accordance with the input event (notified from the event engine 12), the output action defined in the scenario 31 is determined, and the next necessary input event is determined. The determined action request and event request are sent to the action conductor 13 and the event engine 12 for processing. The scenario definition is read by the scenario manager 111 and registered when the scenario interpreter 112 is generated.

  The scenario engine 11 provides a service user with a service according to the scenario 31 represented by the state transition model. The scenario 31 describes an interaction procedure for receiving a service, and determines a response according to an input event such as a user input or a sensor change.

  The state transition model is expressed by several “states” and “transitions” from state to state. This representation has a network structure in which the state is a node and the transition is an arc. In the state transition scenario, when the condition defining the transition is true, the state changes to the state that is the transition destination. In the scenario XML used in this framework, the state element corresponds to a node, and the transition element corresponds to an arc. For example, the state in the state transition scenario is to perform an external action (action) such as an action to the user, such as display on the screen or voice utterance, or a control command to the connected device, etc. Means. The transition in the state scenario means accepting an input from the device such as a button press, key operation, or speech by the user, or an event from the device.

  In the state in this system, output to the outside and internal processing are described. The output to the outside is called an action, and describes voice, expression for an HTML (Hyper Text Markup Language) browser, a device control command for a car navigation system, and the like. The internal process is defined in the scenario logic 32 described in a program language, and is called at a specific timing during execution.

  An action is a set of commands for controlling external output, and is a command for each modality to be controlled (in this embodiment, a modality is a device / middleware grouped into one input / output function unit). Describe. As described below, an action can be set with an importance level and a priority level, and normally, an importance level and a priority level given to a service are used. Therefore, it is not necessary to specify them explicitly. The importance level is used in the action conductor 13 for the control of automatically suppressing or restarting the output according to the importance level threshold given from the outside. With this function, for example, it is possible to realize an application that changes the threshold of importance in conjunction with the load state of the user and outputs only an important action when the load is high. On the other hand, the priority is used in the action conductor 13 to arbitrate an action to be output when there are conflicting output requests of a plurality of actions. With this function, even when a low-priority action is being output, a high-priority action is interrupted and output, or vice versa. You can wait for the output of a low-priced action.

  Transition in this system defines a transition relationship from state to state. Transitions that extend from the “currently executing state (also called current context)” are effective. An event that causes a state transition is described under the transition. The scenario interpreter 112 requests the event engine 12 to be notified when an event described in a valid transition occurs.

  An input to this system is called an event. The event determines an input to be detected in the scenario 31, and is described under the transition. In the case where a plurality of events are described in one transition, an OR connection is formed, and if any one of the events occurs, a state transition occurs.

Events include user-defined events that are user-defined events, predefined events that are predefined in the base middleware, such as sensor inputs, and special events that are predefined events associated with system operation. is there. The user-defined event accepts an event defined by the user in the scenario 31, and is used when, for example, receiving a recognition result and notifying the occurrence of an event when voice recognition occurs. The predefined event is an event that is defined in advance in a place different from the scenario 31 and corresponds to sensor input. The scenario developer will write a scenario 31 using this on the premise that the scenario developer has knowledge about a predefined event. The special event is an event related to the operation of the system, and is a predefined event. Includes events for notifying the output status of actions. In this framework, the following six types of events related to output states are prepared as special events.
Wait: Notify when action output request cannot be started for some reason.
Start: Notify when the action output request is actually started.
Interruption: Notify when the action being output is interrupted for some reason.
Resume: Notify when an interrupted action is started again.
Complete: Notify when action output is complete (including timeout).
Error: Notify when action output results in an error.

  By describing special event definitions that specify these, it is possible to describe a scenario 31 corresponding to a change in the output state.

  When it is desired to handle inputs from a plurality of modalities as the same event, the event write destination specified by the command is set to be the same for each modality. For example, if the voice input is entered as “Yes” and the “Yes” button on the screen is identified, the input result is written to the same place with the voice and the command to the modality of the screen. specify.

  In addition, a function executed at a specific timing in the state and transition processing of the scenario 31 is called scenario logic. These functions have default behaviors that are called if nothing is specified. When the user defines the required processing, the default behavior is overridden and user-defined processing is performed.

  In the scenario logic 32, complicated internal processing that cannot be described in the scenario 31, operations for external applications controlled by outbound, and the like are described.

  The timing for calling the scenario logic 32 is shown as an example in which a state transition occurs from the state a through the transition t to the state b. First, when an event of transition t occurs when the current is in state a, the transition determination logic of transition t is executed as part of the determination process of whether to accept the event. If the result of determination is that transition is possible, the state end logic of state a is executed, and the transition execution logic of transition t is subsequently executed. Then, after transitioning to state b, the action request logic of state b is executed. Thereafter, an action request is made to the action conductor 13, but the command confirmation logic in the state b is executed at a timing immediately before this action is actually executed. Thereafter, the same processing is repeated when the next event occurs.

  The event engine 12 collects and manages various states such as the service user's own state, the state around the service user, and the execution state of the service in the event data table 34, and an event occurs for each event monitoring source. This is an event management component for realizing an event-driven service that provides a service when an event occurs by notifying the occurrence of an event when the event occurs.

  The event engine 12 has an event management function (function by the event manager 121), an event notification function (function by the event manager 121), and an event aggregation function (function of the event aggregator 122). The event management function is a function for storing and managing events in the event data table 34. Provides functions for registering, updating, deleting, and searching for events. The event notification function is a function for notifying the event monitoring source of the occurrence of the event. The event monitoring source can request event registration (Notify) from the event engine 12 when an event to be subscribed occurs by requesting monitoring registration (Subscribe) in advance for the event to be monitored. The event monitoring source can filter unnecessary event notifications by designating an event notification condition for receiving notification at the time of Subscribe (conditional Subscribe). The event aggregation function provides a function for processing detailed event data obtained from various devices into event data that can be easily handled by an application by an external logic defined by the developer (hereinafter, event aggregation logic). For example, the driver's busy degree (low to medium to high) is calculated as new event data by combining the values of the winker, steering wheel steering angle, accelerator opening, and brake strength obtained from the in-vehicle sensor. The event aggregation logic is registered in advance in the shared library and loaded from the event engine 12.

  The event logic 33 processes the event data table 34 in the application layer 3 and is defined by the developer.

  The action conductor 13 includes an action control engine 131 and a multimodal engine 132. The action control engine 131 is a module that manages an action request from a service, mediates an output order, suppresses output, and notifies an action output status. Output arbitration function that receives action requests asynchronously from multiple services and changes the output order according to their priority, output suppression function that executes only actions with high importance according to load, action output status Provides an output status notification function that updates the event type according to. The multimodal engine 132 is a module that controls a plurality of modalities managed by the action conductor 13 (sends commands to the modalities and receives responses from the modalities) according to the action description contents of the scenario engine 11.

  The action is what the service defines the command contents for each modality, and is composed of the control command, priority / importance of each modality, and the place where the processing result is written. The action is generated by a request from the service and is managed by an action queue described later in the action conductor 13, but is discarded by the action conductor 13 when the action is canceled from the service or when the action is completed.

  A data structure related to the action is shown in FIG. FIG. 2 shows the data structure for one action. The action request information structure includes an action ID, action request source ID, action execution state, action priority, action importance, command structure array start address, number of commands, command confirmation callback function address, command confirmation parameter Includes address, event write destination address, etc. One action can include a plurality of commands, and the number is defined by the number of commands. The start address of the command structure array indicates the start address of the command information structure related to the action. The command information structure includes identification information of the instruction destination input / output middleware, a command, and a command parameter. The event write destination information address indicates the start address of the event write destination information structure. The event write destination information structure includes the event type name of the completion event write destination, the event type name of the standby event write destination, the event type name of the start event write destination, the event type name of the interrupt event write destination, and the event of the resume event write destination Contains the type name and event type name to which the error event is written. The command is a control command for the modality. In the command information structure, the identification information of the instruction destination input / output middleware is used by the action conductor 13, but the other contents are interpreted by the modality of the output destination. The importance is used for output suppression control according to the load. The priority is used for output arbitration of the action.

  Note that the timing at which an action is requested may be greatly different from the timing at which actual output is performed due to the output suppression function and the output order arbitration function. In such a case, if the control command at the time of requesting an action is used as it is, the output contents may become inappropriate. In order to avoid this, there are cases where it is desired to confirm the contents of the control command immediately before output. When the command confirmation callback function is specified, it is called at the timing immediately before the actual output, and the output is performed after the command content is confirmed. The command confirmation parameter is a parameter used in the command confirmation callback function. The command confirmation callback function will be described in detail later.

  The action request source ID is designated to control to continuously output actions from the same request source when the priorities are the same.

The action request is processed in the following flow.
(1) Action request The service registers an action output request for the action conductor 13. The action conductor 13 determines whether or not the request can be output based on the importance level and the priority level. Even when the action can be output immediately, (2) command confirmation processing is performed at the timing when the action is actually output. Action requests may be made asynchronously from multiple services.

  The action conductor 13 stores the action that could not be output, that is, the suspended action, and determines the possibility of output again at the next output determination timing. When a suspended action can be newly output, a start event is notified as necessary.

(2) Command Confirmation The action conductor 13 calls a callback function for confirming the command content at the timing when the action is actually output. The service replaces commands as necessary in the called callback function. Note that the modality specified at the time of request cannot be changed. If a callback function for command confirmation is not registered in the action at the time of request, these processes are not performed and the command at the time of requesting an action is used as it is. If neither a command nor a callback function is specified when an action is requested, it is processed as an error when requested. The output may be started multiple times due to the interruption / resumption of the action. In this case, the callback function is called each time.

(3) Canceling an action An action request whose output has been completed is discarded by the action conductor 13. On the other hand, for actions that are no longer necessary, the service must explicitly issue a cancel. The cancellation target is specified using the action ID returned when the action is requested. The action requested to be canceled is deleted regardless of the output status. If output is in progress, output is stopped at that point.

  The actual action cancellation request is made simultaneously with the timing of the action request (request for the action next to the canceled action). This is to prevent another action that should not be output if a time lag occurs between the action cancel process and the next action request, and this is to be prevented.

  The output arbitration function of the action control engine 131 has an output arbitration function.

  In the action queue, as shown in FIG. 3, the addresses to the action request information structures are arranged in order of priority. In the example of FIG. 3, when the action A for audio output, the action B for icon output, and the action C for audio output exist in this order of priority, the address to the action request information structure A, An address to the action request information structure B and an address to the action request information structure C are arranged in order.

  Since the action cancellation and the request are performed at the same timing, this process is performed when the action to be canceled is being executed or after completion and there is a next action.

Next, the output arbitration function will be described. The action conductor 13 has a function of adjusting the order of outputting actions according to the priority of the action request. When multiple services are operating asynchronously, requests for action output may conflict. For example, the following cases.
[1] While a service is outputting an action, another service requests an action output.
[2] Different services simultaneously request action output.

  The action output arbitration function provides a function of arbitrating so that important actions are preferentially output when such action request contention occurs.

As described above, the priority is defined for the action, and the following mediation is performed using this value.
[1] An action request having a higher priority than the action currently being output is output after the current output is interrupted.
[2] An action request having a priority lower than or equal to the action currently being output is waited until the current output is completed.

  Note that even when a plurality of action requests are generated at the same time, the actual processing is serialized, and thus the same processing as in the case where requests are made in order.

  The action conductor 13 also has an action output suppressing function that suppresses action output in accordance with the magnitude of the load. The threshold of importance corresponding to the magnitude of the load is prepared as an event type, and is set so that the action conductor 13 receives an event according to a change in the value.

  The action conductor 13 accepts an event and internally holds a change in importance threshold. The threshold value is compared with the importance of the requested action or the action being executed, and output suppression control is performed. Specifically, different output suppression control is performed in the following two cases.

[A] When requesting an action When requesting an action, the importance of the action is compared with a threshold value, and if the importance of the action is higher or equal, output is possible. Otherwise, the output is suppressed and the action request is deferred.
[B] When the threshold value changes When the importance level threshold value changes, the importance levels are compared for all action requests including those during output, and only those whose importance level is higher than or equal to the threshold value are collected, and output arbitration processing To implement. As a result, the action being output may be interrupted by another action. Further, when the importance of the action being output is low and cannot be output, the action is interrupted even if no other action output is output.

  Actions that could not be output by the output suppression function are notified of a standby event as necessary. Further, when the action that has been waiting can be output because the load is reduced (that is, the importance threshold is lowered), a start event is notified as necessary. Actions interrupted due to a high load during output are notified of an interrupt event if necessary, and if the load is reduced and output is started again (from the beginning), a restart event is notified if necessary. The

  The action conductor 13 has an action output state notification function that manages the output status of each action and performs notification as necessary. The output status of the action is managed in the “action execution state” in the action request information structure shown in FIG. One of the states shown in FIG. 4 is set as the action execution state.

  As shown in FIG. 4, the action is started, the “registered” state in which action data is registered (initial state), the “standby” state in which the action cannot be output immediately, and is in a wait state. "Start" state that is the state, "Suspend" state where the action being output has been interrupted for some reason, "Resume" state where the interrupted action is output again, and action output “Completed” state is defined, and “Error” state is defined in which the action output is in error.

  The “registered” state is a state immediately after an action request is registered in the action conductor 13 and is a state in which action output suppression / output arbitration is not performed. Specifically, the “standby” state occurs when the requested action is waited for output by the output arbitration function or when the output is waited for by the output suppression function. The “start” state occurs when the requested action is output for the first time. Even when the action is not waiting for output, it occurs at the timing when output is started for the first time. The “suspended” state occurs when the action being output is stopped when it is not completed. Specifically, it occurs when output is interrupted by output arbitration by a request action of another service and when output is suppressed by the output suppression function. The “restart” state occurs at the timing when the action in the suspended state can be executed again. Specifically, it occurs when another service is completed and the interrupted action can be executed again, or when an action that has been suppressed by the output suppression function is released. The “completed” state occurs when the action being output is completed. The completion of the action differs depending on the modality. For example, in the case of speech synthesis, the action is completed when the designated utterance content and the timeout time have elapsed. Normally, the completion is notified from the modality to the action conductor 13, and the multimodal engine 132 integrates the completion states of a plurality of modalities and integrates the completions to determine completion. The “error” state occurs when an action to be output or being output becomes an error for some reason. An example of an error occurrence is an action execution error in the modality control function of the multimodal engine 132.

  The states of “standby”, “start”, “registered”, “completed”, and “error” occur only once through the action output, but the events of “interrupt” and “resume” may occur a plurality of times.

  In addition, although it is described as “notification” here for simplification, in actuality, the same event is requested by requesting an event type name and event notification with a service using the Subscribe function provided by the event engine 12. Notification is performed by writing a value to the type name using a write function.

  Each action is designated with a notification location (the event type name of the write destination in the event write destination information structure in FIG. 2) corresponding to these states. If no event notification is required, no writing process is performed because there is no designation.

  The multimodal engine 132 has a multimodal function. The multimodal function provides a function of controlling a plurality of modalities managed by the action conductor 13 (sending commands to the modalities and receiving responses from the modalities) according to the action description contents of the scenario engine 11. Specifically, the multimodal function has a modality management function and a modality control function.

  The modality management function includes a new modality registration function, a modality control suspension function, a modality control restart function, and a modality registration deletion function.

  The new modality registration function is a function for newly registering modality information when the modality is activated because the modality of the base middleware layer 5 is controlled by the action conductor 13. The modality control pause function is used when the modality registered by the new modality registration function temporarily becomes uncontrollable for some reason (for example, when the car is running and it is necessary to suppress screen output) ) To declare that the action conductor 13 is temporarily controlled. The modality control resuming function is a function for declaring resumption of control when the modality that has been suspended by the modality control pausing function becomes controllable due to a change in the situation. The modality registration deletion function is a function for deleting registration when the modality registered by the new modality registration function ends the processing due to the system termination or the like.

  The modality control function has a modality state management function and a modality output arbitration function.

  The modality state management function monitors the command request and response to each modality described in the action, and whether each modality is empty or busy (if it is busy, the priority of the action Is also a function that always manages

  The modality state management function manages an input / output middleware (modality) management table as shown in FIG. In the example of FIG. 5, the modality ID, modality name, interrupt type, and execution action information are managed. The execution action information includes data related to a busy or standby state. As for the interrupt type, no interrupts are permitted, such as “interrupt enabled” where interrupts are performed by other high-priority actions and “no interrupt control” allowing no parallel interrupt control. There is no "no interrupt" etc.

  The modality output arbitration function determines whether individual commands to each modality described in the action request can be output from a plurality of action requests passed from the action control engine 131 (refer to the modality management function and the modality state management function). , Whether or not a modality exists, and if so, whether or not the modality is busy) and arbitrates the conflict of action requests.

  Specifically, a check result as shown in FIG. 6 is obtained according to the state of the modality corresponding to the input command described in the action request, and further shown in FIG. 7 according to the state of the modality corresponding to the output command. Get a check result like this: Further, an action execution determination as shown in FIG. 8 is performed from the check result. The input / output command is checked as an input command, and further checked as an output command.

  The correspondence table of input commands and check results in FIG. 6 includes unregistered modalities, registered and control suspension modalities, registered and busy modalities, and registered and empty modalities. The check results in the case where a part of the modalities related to the input command are applicable or in the case where all of them are applicable are summarized. The modality of “no interrupt control” is always checked as “free”.

  Specifically, if all of the modality states related to the input command are not registered, it is determined as “error”, if all of the modalities are suspended, it is determined as “hold”, and if some modalities are busy, “hold” is determined. Is determined as “hold” when a part of the modality is unregistered or temporarily suspended, and “executable” is determined otherwise.

  Similarly, the output command and check result correspondence table of FIG. 7 includes an unregistered modality, a registered and control pause modality, a registered and busy modality, and a registered and empty state. As for the modalities, the check results in the case where all or all of the modalities related to the output command are applicable are summarized. The modality of “no interrupt control” is always checked as “free”.

  Specifically, if all of the modality states related to the output command are not registered, it is determined as “error”, if it is all suspended, it is determined as “hold”, and if some modalities are busy, “hold” Is determined as “hold” when a part of the modality is unregistered or temporarily suspended, and “executable” is determined otherwise.

  FIG. 8 is a table showing the result of action execution determination by combining the check result of the input command group and the check result of the output command group. Specifically, if any of the check result of the input command group and the check result of the output command group is “error”, it is determined as “error”, and the check result of the input command group or the check result of the output command group is If it is “hold”, it is determined as “hold”, and if both the check result of the input command group and the check result of the output command group are “executable”, it is determined as “executable”. If any check result is an error, a warning log is output as necessary.

  As described above, the base middleware layer 5 includes the input middleware 1 (51), the input middleware 2 (52),. . . . , Output middleware 1 (53), output middleware 2 (54),. . . Etc. In this manner, event input middleware that detects events from various sensors 71 and microphones 73 included in the device layer 7 and outputs them to the event engine 12, and provides commands to the display 72 and speakers 74 included in the device layer 7. It includes output modality middleware that performs the corresponding output processing.

  The device layer 7 includes various sensors such as in-vehicle terminals and output devices.

  Next, main processing according to the present embodiment will be described with reference to FIGS. The action control engine 131 performs the following processing at the action request or cancellation from the scenario engine 11, the occurrence of the importance threshold update event, and the action execution state update timing.

  First, a start instruction is waited until an event as described above occurs and an action output suppression / arbitration start instruction is received (step S1). When the action output suppression / arbitration start instruction is received, the current importance level threshold value is compared with the importance level defined for the action being executed in the action queue (step S3). The action importance level included in the action request information structure shown in FIG. 2 is compared with the current importance level threshold held in the internal structure. When a plurality of actions are registered in the action queue, the action execution state included in the action request information structure shown in FIG.

  As a result of the comparison, if there is an action being executed that has an importance level <threshold value, all the actions being executed that satisfy this condition are suspended (step S5). That is, the action execution state included in the action request information structure shown in FIG. Then, the process proceeds to step S7. On the other hand, as a result of the comparison, if there is no action being executed with importance <threshold, the process proceeds to step S7.

  Thereafter, output arbitration is performed in the order of priority included in the action request information structure shown in FIG. 2 for all action information currently queued in the action queue, and action information (action request information structure in priority order) And the information linked to it) are acquired from the action queue (step S7). Then, the current importance threshold is compared with the importance of the action included in the action information acquired this time (step S9). If importance <threshold, the process returns to step S7 in order to shift to the next priority action process. On the other hand, if threshold value ≦ importance level, action execution state determination is performed from the action information acquired this time (step S11). It is determined whether the action execution state included in the action information is “start”, “resume”, “complete”, or “error”. In such a case, since it is not necessary to perform the action execution determination again, the process returns to step S7 to shift to the action processing of the next priority.

  On the other hand, if the action execution state included in the action information is “registered”, “waiting”, or “interrupted”, the multimodal engine 132 makes an action execution determination (step S13). If possible, execute at this stage. This process will be described with reference to FIG. If there is an action for which output arbitration is not performed in the action queue, the process returns to step S7. On the other hand, if there is no action for which output arbitration is not performed in the action queue, the process returns to step S1.

  Next, action execution determination by the multimodal engine 132 will be described with reference to FIG. First, for the input command and input / output command specified from the command information structure linked from the action request information structure, check the status of the execution target modality of the command, referring to the input / output middleware management table shown in FIG. (Step S21). The state of the execution target modality is checked from the execution action information in the input / output middleware management table shown in FIG. As described above, the interrupt type is also related. If the modality is “no interrupt control”, the interrupt type is always specified as “free”. If interrupt is impossible and busy, execution is suspended.

  For the output command and input / output command specified from the command information structure linked from the action request information structure, check the status of the execution target modality of the command, and refer to the input / output middleware management table shown in FIG. (Step S23). The state of the execution target modality is checked from the execution action information in the input / output middleware management table shown in FIG. If interrupt is impossible and busy, execution is suspended.

  Then, as described with reference to FIG. 6, the execution determination of the input command and the input / output command is performed (step S25). Here, either “error”, “hold” or “executable” is specified. Similarly, as described with reference to FIG. 7, the execution determination of the output command and the input / output command is executed (step S27). As in step S25, either “error”, “hold” or “executable” is specified.

  Thereafter, as described with reference to FIG. 8, an action execution determination is performed (step S29). As a result, either “error”, “hold” or “executable” is specified.

  When “error” is specified, “completion” is registered in the action execution state of FIG. 2 as a completion event occurrence (step S31). Then, the process returns to the original process.

  When “Hold” is specified, the action execution state determination is performed as in Step S11 (Step S33). If it is in the registered state, “standby” is registered in the action execution state of FIG. 2 as a standby event occurrence (step S35). On the other hand, if the state is other than “registered”, the process returns to the original process.

  When “executable” is specified, the action execution state determination is performed (step S37) as in step S11. As a result, “registered”, “waiting”, “suspended”, “completed”, “error”, “start”, or “resume” is specified. If it is “registered” or “standby”, “start” is registered in the action execution state of FIG. 2 as a start event occurrence (step S39). Then, control goes to a step S43. On the other hand, if it is other than “Registered” or “Standby”, “Resume” is registered in the action execution state of FIG. 2 as a resume event occurrence (Step S41). Then, control goes to a step S43.

  If the command confirmation callback function is registered in the action request information structure of FIG. 2, command confirmation processing is performed (step S43). Since the processing of the command confirmation callback function will be described in detail below, the command confirmation callback function will be described here as not registered. In this case, step S43 is skipped.

  Then, the status of the command execution target modality is checked by referring to the input / output middleware management table of FIG. When the command is being executed, “suspend” is registered in the action execution state of FIG. Basically, the same check as in step S21 or S23 is performed.

  If the command execution modality state is not “error” or “busy”, the action is executed (step S47). Then, the process returns to the original process.

  By performing the processing as described above, processing as shown in FIG. 11 becomes possible. For example, it is assumed that an action of voice output as an action of the first priority and an action of icon output as the action of the second priority are registered in the action queue. Here, it is assumed that the importance of each action is higher than the threshold of importance. In such a case, output arbitration of the first priority action is performed according to the flow of FIGS. 9 and 10 (step S51). If the text speech output modality (TextToSpeech modality) is not busy, the first priority action output processing is performed, and speech output is started by the speech synthesis modality (step S53).

  In parallel with step S53, output arbitration of the second priority action is performed according to the flow of FIGS. 9 and 10 (step S55). When the modality related to the action of the second priority is in an executable state (ie, “empty”) or “no interrupt control” (step S57: Yes route), the output of the second priority is started. (Step S59). That is, since the icon display modality is “no interrupt control”, the voice and the icon are output in duplicate. On the other hand, when the modality related to the second priority action is not in the executable state and is not “no interrupt control” (step S57: No route), the state of “waiting” is set for the second priority action. Set (step S61).

  For example, even if the modality related to the action of the second priority is not “icon display”, duplicate output is similarly performed even if the image is displayed on the display device or the beep sound is output.

  Thus, since it is the structure which enables duplication output flexibly, much information can be provided to a user in a short time.

  Next, the command confirmation callback function will be described with reference to FIGS. The case where the command confirmation callback function is not defined in step S43 in FIG. 10 is shown above. Here, in scenario 1 (311) of FIG. 12, as shown in FIG. 13, action action1 in state state1 is shown. Is defined as using a logic group prefixed with a character string “st1” designated by the logical attribute. As a result, in the case of the command confirmation logic, the function “st1_action (...)” Is designated.

  Based on such scenario 1 (311), the scenario engine 11 outputs the action request 1 shown in the lower left in FIG. 12 to the action conductor 13 (step (101)). The action request 1 includes an event data variable name (event type name) “$ action01_start” to which the start event is written, an event data variable name (event type name) “$ action01_complete” to which the completion event is written. Further, the action request 1 includes command information shown in the lower center of FIG. This command information includes data such as an instruction destination input / output middleware “TTS (TextToSpeech modality)”, a command “text reading out”, an output content “Today's weather is sunny”, and a progress notification destination “$ weather_progress”. .

  The action conductor 13 performs output arbitration and execution determination of the action described in FIGS. 9 and 10 (step (102)), and when the action request 1 is determined to be executed, it is designated as a start event write destination. The event data variable “$ action01_start” in the data management table of the event engine 12 is changed from FALSE to TRUE (step (103)). Then, the command confirmation callback function is called in step S43 in FIG. 10 (step (104)). The command confirmation logic 322 designated as the command confirmation callback function is 0% with reference to the event data variable “$ weather_progress” in the data management table of the event engine 12 designated by the command information. (Step (105)). Then, the command determination logic 322 determines that there is no need to change the current command information (step (106)). Therefore, the action conductor 13 outputs the command information to the TextToSpeech middleware 55 (step (107)). The TextToSpeech middleware 55 synthesizes and outputs the voice data “Today's weather is sunny” via the speaker 74 in accordance with the command information. Further, the event variable “$ weather_progress” in the data management table of the event engine 12 specified in the command information is updated to output the voice data up to 80%, and the above-described event variable is output. Has been changed to 80% (step (108)).

  At this stage, as shown in FIG. 14, it is assumed that the scenario engine 11 outputs the action request 2 to the action conductor 13 according to the scenario 2 (312) (step (201)). Note that the priority of the action request 2 is higher than the priority of the action request 1. Then, the action conductor 13 performs the output arbitration shown in FIGS. 9 and 10, and interrupts the output of the command for the action request 1 (step (202)). The TextToSpeech middleware 55, which is the output destination of the command, is interrupted because it can be interrupted. If interrupt is impossible, action request 2 cannot be interrupted. Then, the action conductor 13 outputs the command information 2 for the action request 2 to the TextToSpec middleware 55 (step (203)). In this example, voice data “Please pay attention to the car ahead” is synthesized and output from the speaker 74.

  When such output is completed, as shown in FIG. 15, it becomes a resumption trigger (step (301)), the action request 1 stored in the action queue is again subject to output arbitration, and the action conductor 13 And execution determination is implemented (step (302)). Then, the command confirmation callback function is called again in step S43 of FIG. 10 (step (303)). The command confirmation logic 322 specified as the command confirmation callback function refers to the event data variable “$ weather_progress” in the data management table of the event engine 12 specified in the command information, and outputs the data It is determined whether the ratio is 80% or more (step (304)). If it is less than 80%, the command information is not changed, but in this example, it is 80% or more, so the output content of the command information is changed to “NULL” (step (305)). Then, the action conductor 13 outputs the changed command information to the TextToSpeech middleware 55 (step (306)). The TextToSpeech middleware 55 tries to output the output content of the command information, but since it is “NULL”, there is nothing to output, and the event data variable “in the data management table of the event engine 12 that is the progress status notification destination immediately” “$ Weather_progress” is changed from 80% to 100% (step (307)). Then, the TextToSpeech middleware 55 notifies the end of the process and returns to the process of the action conductor 13 (step (308)). Then, the write event data variable “$ action01_complete” of the completion event specified in the action request 1 is changed from FALSE to TRUE (step (309)). In this way, the process is completed.

  When performing the processes as shown in FIGS. 12 to 15, the command determination logic 322 performs the process as shown in FIG. 16. That is, the command information of action 1 (311) is interpreted, and the event data variable to be referred to is specified (step S71). If $ weather_progress <80%, the process ends without performing any particular processing. On the other hand, if $ weather_progress ≧ 80%, the output content of the command information, here, the utterance of the weather is invalidated (step S73). Then, the process ends.

  Further, the TextToSpeech middleware 55 performs a process as shown in FIG. First, the output content included in the command information is referred to (step S81). If the output content is “NULL”, the process returns to the original process without performing the process. On the other hand, when the text to be output (TEXT) is included in the output content, speech synthesis processing is performed and stored in the output buffer (step S83). Here, PCM (Pulse Code Modulation) data is generated from the text. When the end of the output buffer is reached, the original processing is restored. On the other hand, if there is a remaining in the output buffer, an audio buffer reproduction process is performed, and audio is output via the speaker 74 (step S85). Then, the reproduced data amount / audio data as a whole is stored in $ weather_progress designated by the command information (step S87). Such processing is repeated until the end of the output buffer is reached.

  80% is an example, and may be changed for each command. For example, you may make it designate the ratio until a specific keyword is output.

  The processing shown in FIGS. 12 to 17 can be summarized as shown in the processing flow of FIG. First, the first information related to the first action is output (step S91). Thereafter, an interrupt is generated before the output of the first information is completed (step S93). Here, assuming that the priority of the second action related to the interrupt is higher, the second information related to the second action is output (step S95). However, it is determined whether or not the second information is to be output after confirming whether the modality is interruptible. When the output of the second information is completed, the output arbitration of the first information is performed again, and the re-output of the first information is determined (step S97). And it is judged whether the output content of 1st information is outputting more than the predetermined ratio (step S99). When the output of the first information has been completed for a predetermined ratio or more, the completion of the output of the first information is registered (step S101). Then, the process ends without outputting the first information. On the other hand, when the output content of the first information is less than a predetermined ratio, the first information is output again (step S103).

  As described above, if a sufficient amount of information has been output, a large amount of information can be output in a short time by suppressing re-output and outputting other information even when resuming output. Will be able to.

  Although the embodiment of the present invention has been described above, the present invention is not limited to this. For example, the module configuration is not limited to that shown in FIG. 1, and other configurations may be used as long as the processing described above can be performed.

  The vehicle-mounted terminal described above is a computer device, and as shown in FIG. 19, the computer device displays a memory 2501 (storage unit), a CPU 2503 (processing unit), and a hard disk drive (HDD) 2505. A display control unit 2507 connected to the device 2509, a drive device 2513 for the removable disk 2511, an input device 2515, and a communication control unit 2517 for connecting to a network are connected by a bus 2519. Application programs including an operating system (OS) and a Web browser are stored in the HDD 2505, and are read from the HDD 2505 to the memory 2501 when executed by the CPU 2503. If necessary, the CPU 2503 controls the display control unit 2507, the communication control unit 2517, and the drive device 2513 to perform necessary operations. Further, data in the middle of processing is stored in the memory 2501 and stored in the HDD 2505 if necessary. Such a computer realizes various functions as described above by organically cooperating hardware such as the CPU 2503 and the memory 2501 described above with the OS and necessary application programs. However, the HDD 2505 may be replaced by another storage device. In some cases, the drive device 2513 and the communication control unit 2517 are not provided. The in-vehicle terminal also includes various sensors and output devices shown in FIG. 1, but not shown in FIG.

(Appendix 1)
Before the output of the first information related to the first action registered in the action queue is completed, the second action registered in the action queue after the next is the second action related to the second action. A determination step for determining whether or not a specific output function used for outputting the information of 2 satisfies a predetermined condition including a condition that the specific output function is a vacant state or a function without interrupt control;
When it is determined that the second action satisfies the predetermined condition, the second action related to the second action is performed regardless of whether or not the output of the first information is completed. Requesting the specific output function to output information;
Output arbitration program that causes a computer to execute

(Appendix 2)
The predetermined condition is
The output arbitration program according to supplementary note 1, including a condition that an importance level of the second action is equal to or greater than a threshold linked to a load state of a user who perceives the first information and the second information.

(Appendix 3)
The determination step includes
The output arbitration program according to supplementary note 1, including a step of searching a management table for managing an interrupt type and a state for each output function by using the specific output function used for outputting the second information related to the second action.

(Appendix 4)
The output arbitration program according to appendix 1, wherein the priority of the first action is higher than the priority of the second action.

(Appendix 5)
The output arbitration program according to supplementary note 1, further causing the computer to execute a step of setting the execution state of the second action to a standby state when it is determined that the second action does not satisfy the predetermined condition. .

(Appendix 6)
During the output of the first information related to the first action registered in the action queue, the first information related to the first action that has a higher priority than the first action is output to the action queue. When the second action having the same specific output function is registered, a stop step of stopping the output of the first information by the specific output function;
Requesting the specific output function to output second information related to the second action;
After the output of the second information related to the second action is completed, and when the first action registered in the action queue is processed again, the first action is related Determining whether the output of the first information has been completed to a predetermined portion;
Canceling the output of the first information when the output of the first information related to the first action has been completed up to a predetermined portion;
Output arbitration program that causes a computer to execute

(Appendix 7)
The stopping step comprises:
Searching the management table for managing the interrupt type for each output function with the specific output function, and determining whether the specific output function is an interruptable output function;
If the specific output function is an interruptible output function, the specific output function stops outputting the first information;
The output arbitration program according to appendix 6, including

(Appendix 8)
The output arbitration program according to appendix 6, wherein the predetermined portion is specified by an output ratio of the first information.

(Appendix 9)
The output arbitration program according to appendix 8, wherein the specific output function further causes the computer to execute a step of registering an output ratio of the first information in a variable designated by the first action.

(Appendix 10)
If the output of the first information related to the first action is not completed up to a predetermined portion, the computer further executes a step of requesting the specific output function to output the first information again. The output mediation program according to appendix 6 for causing

(Appendix 11)
Before the output of the first information related to the first action registered in the action queue is completed, the second action registered in the action queue after the next is the second action related to the second action. A determination step for determining whether or not a specific output function used for outputting the information of 2 satisfies a predetermined condition including a condition that the specific output function is a vacant state or a function without interrupt control;
When it is determined that the second action satisfies the predetermined condition, the second action related to the second action is performed regardless of whether or not the output of the first information is completed. Requesting the specific output function to output information;
An output arbitration method that is executed by a computer.

(Appendix 12)
The predetermined condition is
The output arbitration method according to appendix 11, including a condition that the degree of importance of the second action is equal to or greater than a threshold linked to a load state of a user who perceives the first information and the second information.

(Appendix 13)
The determination step includes
The output arbitration method according to appendix 11, including a step of searching a management table for managing an interrupt type and a state for each output function with the specific output function used for outputting the second information related to the second action.

(Appendix 14)
The output arbitration method according to claim 11, wherein the priority of the first action is higher than the priority of the second action.

(Appendix 15)
The output arbitration method according to claim 11, further comprising a step of setting the execution state of the second action to a standby state when it is determined that the second action does not satisfy the predetermined condition.

(Appendix 16)
During the output of the first information related to the first action registered in the action queue, the first information related to the first action that has a higher priority than the first action is output to the action queue. When the second action having the same specific output function is registered, a stop step of stopping the output of the first information by the specific output function;
Requesting the specific output function to output second information related to the second action;
After the output of the second information related to the second action is completed, and when the first action registered in the action queue is processed again, the first action is related Determining whether the output of the first information has been completed to a predetermined portion;
Canceling the output of the first information when the output of the first information related to the first action has been completed up to a predetermined portion;
An output arbitration method that is executed by a computer.

(Appendix 17)
The stopping step comprises:
Searching the management table for managing the interrupt type for each output function with the specific output function, and determining whether the specific output function is an interruptable output function;
If the specific output function is an interruptible output function, the specific output function stops outputting the first information;
The output arbitration method according to appendix 16, including:

(Appendix 18)
The output arbitration method according to claim 16, wherein the predetermined part is specified by an output ratio of the first information.

(Appendix 19)
The output arbitration method according to appendix 18, further comprising the step of the specific output function registering an output ratio of the first information in a variable designated by the first action.

(Appendix 20)
The supplementary note 16 further comprising: requesting the specific output function to output the first information again when the output of the first information related to the first action is not completed up to a predetermined portion. The output arbitration method described.

(Appendix 21)
Before the output of the first information related to the first action registered in the action queue is completed, the second action registered next in the action queue is the second action related to the second action. Determining means for determining whether or not a specific output function used for outputting the information satisfies a predetermined condition including a condition that the specific output function is idle or a function without interrupt control;
When it is determined that the second action satisfies the predetermined condition, the second action related to the second action is performed regardless of whether or not the output of the first information is completed. Means for requesting the specific output function to output information;
An output arbitration device.

(Appendix 22)
The predetermined condition is
The output arbitration apparatus according to appendix 21, including a condition that the importance of the second action is equal to or greater than a threshold value linked to a load state of a user who perceives the first information and the second information.

(Appendix 23)
The determination means is
The output arbitration apparatus according to appendix 21, further comprising means for searching a management table for managing an interrupt type and a state for each output function by using the specific output function used for outputting the second information related to the second action.

(Appendix 24)
The output arbitration apparatus according to appendix 21, wherein the priority of the first action is higher than the priority of the second action.

(Appendix 25)
The output arbitration apparatus according to appendix 21, further comprising means for setting the execution state of the second action to a standby state when it is determined that the second action does not satisfy the predetermined condition.

(Appendix 26)
During the output of the first information related to the first action registered in the action queue, the first information related to the first action that has a higher priority than the first action is output to the action queue. Stop means for stopping the output of the first information by the specific output function when a second action having the same specific output function is registered,
Means for requesting the specific output function to output second information related to the second action;
After the output of the second information related to the second action is completed and the order of processing the first action registered in the action queue is reached again, the first action relates to the first action Means for determining whether the output of the first information has been completed up to a predetermined portion;
Means for canceling the output of the first information when the output of the first information related to the first action has been completed up to a predetermined portion;
An output arbitration device.

(Appendix 27)
The stopping means is
Means for searching a management table for managing an interrupt type for each output function with the specific output function, and determining whether the specific output function is an interruptable output function;
When the specific output function is an interruptible output function, the specific output function causes the specific output function to stop outputting the first information;
27. The output arbitration device according to appendix 26.

(Appendix 28)
27. The output arbitration apparatus according to appendix 26, wherein the predetermined portion is specified by an output ratio of the first information.

(Appendix 29)
29. The output arbitration apparatus according to appendix 28, wherein the specific output function further includes means for registering an output ratio of the first information in a variable designated by the first action.

(Appendix 30)
(Supplementary note 26) further comprising means for requesting the specific output function to output the first information again when the output of the first information related to the first action has not been completed up to a predetermined portion. The output arbitration device described.

It is a figure which shows the architecture of the agent framework which concerns on embodiment of this invention. It is a figure which shows an example of the information structure for action requests. It is a figure for demonstrating the action queue of an action conductor. It is a figure for demonstrating the classification of the output state of an action. It is a figure which shows an example of an input / output middleware management table. It is a figure which shows the example of the correspondence table of an input command and a check result. It is a figure which shows the example of the correspondence table of an output command and a check result. It is a figure which shows the example of the corresponding | compatible table of a check result and action execution content. It is an activity diagram for action output suppression / output arbitration. It is an activity diagram of action execution determination. It is a figure for demonstrating the 1st point in embodiment of this invention. It is a figure for demonstrating the 2nd point in embodiment of this invention. It is a figure for demonstrating the relationship between a scenario and command confirmation logic. It is a figure for demonstrating the 2nd point in embodiment of this invention. It is a figure for demonstrating the 2nd point in embodiment of this invention. It is an activity diagram of command confirmation logic. It is an activity diagram of text voice output middleware. It is a figure for demonstrating the 2nd point in embodiment of this invention. It is a functional block diagram of a computer.

Explanation of symbols

1 Agent framework runtime 3 Application layer 5 Platform middleware layer 7 Device layer 9 Agent framework SDK
11 Scenario Engine 12 Event Engine 13 Action Conductor 31 Scenario 32 Scenario Logic 33 Event Logic 34 Event Data Table 94 Development API / Library 111 Scenario Manager 112 Scenario Interpreter 121 Event Manager 122 Event Aggregator 131 Action Control Engine 132 Multimodal Engine

Claims (8)

Before the output of the first information related to the first action registered in the action queue is completed, the second action registered in the action queue after the next is the second action related to the second action. A determination step for determining whether or not a specific output function used for outputting the information of 2 satisfies a predetermined condition including a condition that the specific output function is a vacant state or a function without interrupt control;
When it is determined that the second action satisfies the predetermined condition, the second action related to the second action is performed regardless of whether or not the output of the first information is completed. Requesting the specific output function to output information;
Output arbitration program that causes a computer to execute The predetermined condition is
The output arbitration program according to claim 1, further comprising: a condition that the importance of the second action is equal to or greater than a threshold linked to a load state of a user who perceives the first information and the second information. The determination step includes
The output arbitration program according to claim 1, further comprising: searching for a management table for managing an interrupt type and a state for each output function by using the specific output function used for outputting the second information related to the second action. . While outputting the first information related to the first action registered in the action queue, the first information related to the first action is output to the action queue having a higher priority than the first action. When a second action having the same specific output function is registered, a stop step of stopping the output of the first information by the specific output function;
Requesting the specific output function to output second information related to the second action;
After the output of the second information related to the second action is completed and the order of processing the first action registered in the action queue is reached again, the first action relates to the first action Determining whether the output of the first information has been completed to a predetermined portion;
Canceling the output of the first information if the output of the first information related to the first action has been completed up to a predetermined portion;
Output arbitration program that causes a computer to execute The stopping step comprises:
Searching the management table for managing the interrupt type for each output function with the specific output function, and determining whether the specific output function is an interruptable output function;
If the specific output function is an interruptible output function, the specific output function stops outputting the first information;
The output arbitration program according to claim 4 including: The output arbitration program according to claim 4, wherein the predetermined part is specified by an output ratio of the first information. Before the output of the first information related to the first action registered in the action queue is completed, the second action registered next in the action queue is the second action related to the second action. Determining means for determining whether or not a specific output function used for outputting the information satisfies a predetermined condition including a condition that the specific output function is idle or a function without interrupt control;
When it is determined that the second action satisfies the predetermined condition, the second action related to the second action is performed regardless of whether or not the output of the first information is completed. Means for requesting the specific output function to output information;
An output arbitration device. During the output of the first information related to the first action registered in the action queue, the first information related to the first action that has a higher priority than the first action is output to the action queue. Stop means for stopping the output of the first information by the specific output function when a second action having the same specific output function is registered,
Means for requesting the specific output function to output second information related to the second action;
After the output of the second information related to the second action is completed, and when the first action registered in the action queue is processed again, the first action is related Means for determining whether the output of the first information has been completed up to a predetermined portion;
Means for canceling the output of the first information when the output of the first information related to the first action has been completed up to a predetermined portion;
An output arbitration device.

Images