JP2008040755A - Data structure of hmi definition information, and hmi definition information management system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an HMI definition information management system for managing HMI definition information describing the scenario of an integral application program where a plurality of application programs are integrated. <P>SOLUTION: The HMI definition information management system for managing HMI definition information 10 of the integrated application program P for integrating individual application programs P1, P2, P3 has an HMI-executable means 20 for setting the HMI of the integrated application program P to be in an executable state, based on HMI definition information 10 comprising an image configuration information definition file 1000 described in metadata, a voice configuration information definition file 1002, and a state transition information definition file 1001 for relating both the files. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a data structure of HMI (Human Machine Interface) definition information and an HMI definition information management system for managing the HMI definition information, and in particular, HMI definition information describing a scenario of an integrated application program in which a plurality of application programs are integrated. The present invention relates to an HMI definition information management system for managing the data structure and HMI definition information.

  2. Description of the Related Art Conventionally, a scenario creation device that generates a scenario of multimedia content that combines material information such as video, audio, and sub-picture, and includes a logical structure of multimedia content, attributes of programs constituting the multimedia content, and material information Scenario objects that define attributes (hereinafter referred to as “scenario information”), user interface objects that set scenario information, and scenario information set in the scenario object when the specified conditions that define scenario information are met There is known a scenario creation device that includes an observer object to be searched, and before the user interface object changes each attribute set in the scenario object, the observer object determines permission / non-permission of the change. Are (e.g., see Patent Document 1.).

Since the scenario creation device permits the change after confirming that the changed value is within the changeable range when the attribute value is changed by the user, the user who is not familiar with the content standard Can also create content that conforms to the standard.
JP 2000-99754 A

  However, the scenario creation device described in Patent Document 1 is for managing a multimedia content scenario configured by a single application, and a multimedia content scenario configured by integrating a plurality of applications. Can not manage.

  In view of the above-described problems, the present invention provides a data structure of HMI definition information describing a scenario of an integrated application program obtained by integrating a plurality of application programs, and an HMI definition information management system for managing the HMI definition information. And

  To achieve the above object, an HMI definition information management system according to a first aspect of the present invention is an HMI definition information management system for managing HMI definition information of an integrated application program that integrates a plurality of individual application programs. An HMI that makes the HMI of the integrated application program executable based on HMI definition information composed of an image configuration information definition file and audio configuration information definition file described in data and a state transition information definition file that associates both files. An HMI definition information management system comprising an executable means.

  A data structure of HMI definition information according to the second invention is a data structure of HMI definition information in an integrated application program that integrates a plurality of individual application programs, and includes an image configuration information definition file described in metadata, and Individual application HMI definition information composed of an audio configuration information definition file and a state transition information definition file associating both files, and an image configuration information definition file and audio configuration information definition file described in metadata, associating both files A transition information definition file, and each definition file refers to a corresponding image configuration information definition file, audio configuration information definition file, and state transition information definition file corresponding to a plurality of individual application HMI definition information. Characterized in that it comprises the application HMI definition information.

  By the means described above, the present invention can provide a data structure of HMI definition information that describes a scenario of an integrated application program in which a plurality of application programs are integrated, and an HMI definition information management system that manages the HMI definition information.

  Hereinafter, the best mode for carrying out the present invention will be described with reference to the drawings.

  FIG. 1 is a diagram illustrating a configuration example of a car navigation device S using an HMI (Human Machine Interface) definition information management system. The car navigation device S includes a nonvolatile storage device 1, a control device 2, and an image output device 3. And an audio output device 4 and a volatile storage device 5.

  The non-volatile storage device 1 is a device that stores data to be retained even when the car navigation device S is turned off, and includes, for example, a storage medium such as a hard disk, a DVD ROM, a CD ROM, or a flash memory.

  The nonvolatile storage device 1 stores the application program in a state of being separated into the HMI definition information 10 and the sequence unit 11.

  “HMI” refers to an interface between an operator and an application program executed by the car navigation device S. For example, a GUI (Graphical User Interface) of the application program, a voice input / output interface, and a state transition of the application program It is a concept including a scenario to be defined (referred to as a scenario (scenario) that defines execution of a predetermined action (action) according to a predetermined event (event)).

  The HMI definition information 10 is a component of the application program and is information that defines the HMI of the application program. For example, the HMI definition information 10 is described in XML (eXtensible Markup Language), and defines the visual interface (display language, screen, etc.) It includes definitions relating to auditory interfaces (speakers, microphones, etc.) and definitions relating to state transitions of application programs that change in response to operations via those interfaces.

  The HMI definition information 10 includes, for example, information relating to images or information used by an application program as metadata (hereinafter referred to as “image configuration information definition file” and “audio configuration information definition file”) as metadata. It is defined and defined as one file (hereinafter referred to as “state transition information definition file”) in the form of a response (method) to an event (event) that has generated information regarding state transition.

  “Metadata” is data describing information about data. For example, assuming a software button in the GUI, “name” when defining a button name such as “return”, “OK”, “erase”, etc. "Language" when defining the type of language used for display such as "", "Japanese", "English", "Chinese", etc. corresponds to the metadata.

  The state transition information definition file specifies image data or audio data to be used in each state of the application program by generating a link to the image configuration information definition file or the audio configuration information definition file.

  The sequence unit 11 is a component of the application program, and is a part other than the HMI definition information 10 of the application program. The sequence unit 11 stores contents of methods (functions) and the like of the application program that are sequentially executed while referring to the HMI definition information 10. It is a part to describe.

  The control device 2 is a computer equipped with a CPU (Central Processing Unit), and a program module corresponding to the HMI enabling means 20 stored in the nonvolatile storage device 1 is developed on the volatile storage device 5 to cope with it. CPU is caused to execute the process.

  The HMI executable means 20 is means for expanding HMI data on the volatile storage device 5 based on the information defined in the HMI definition information 10 to materialize the HMI and make it executable. The HMI definition information 10 that is an XML document stored in the nonvolatile storage device 1 is analyzed by software such as a processor or an XML parser, and HTML (Hyper Text Markup Language) data is instantiated on the volatile storage device 5 (substance) To display on the browser. The HMI executable means 20 may be a browser incorporating software such as an XML parser.

  Here, the “instance” is an entity of information to which a predetermined memory space is allocated, which is generated on the volatile storage device 5 based on information defined in the definition file.

  The image output device 3 is a device for outputting an image, such as a liquid crystal display or an organic EL (Electro-Luminescence) display.

  The sound output device 4 is a device for outputting sound, and includes an in-vehicle speaker.

  The volatile storage device 5 is a device for holding program codes and data temporarily necessary for executing an application program, such as a RAM (Random Access Memory).

  Next, the flow of processing that enables the car navigation apparatus S to execute HMI based on the HMI definition information 10 will be described with reference to FIGS. 2 and 3.

  FIG. 2 shows a display screen in a case where the integrated application program P in which the three individual application programs P1, P2, and P3 are integrated is executed by the car navigation device S. The individual application programs are an operator application P1, an in-vehicle explanation application P2, and a markup application P3, respectively. The integrated application program P is an application program that simultaneously executes these three individual applications P1, P2, and P3.

  FIG. 3 is a diagram for explaining a configuration example of the HMI definition information 10 of the integrated application program P and a flow of processing for making the HMI executable based on the HMI definition information 10.

  The HMI definition information 10 includes integrated application HMI definition information 100 corresponding to the integrated application program, and individual application HMI definition information 101, 102, and 103 for each of the three individual application programs.

  The integrated application HMI definition information 100 is composed of an integrated image configuration information definition file 1000, an integrated state transition information definition file 1001, and an integrated audio configuration information definition file 1002 described in XML. The individual application HMI definition information 101, 102, 103 includes individual image configuration information definition files 1010, 1020, 1030, individual state transition information definition files 1011, 1021, 1031 and individual audio configuration information definition files 1012, 1022, 1032.

  The individual image configuration information definition files 1010, 1020, and 1030 define image configuration information, design information, language information, and the like. For example, the position of a bitmap file or AVI file used for a background image as image configuration information in the hard disk ( Address) or data “Japanese” is set in the metadata “screen display language” as language information.

  The individual image configuration information definition files 1010, 1020, and 1030 set the size, display time, display timing, and the like of the display window of the application program as design information.

  Individual state transition information definition files 1011, 1021, and 1031 define state transition information, event definition information, a link to an image configuration information definition file, and a link to an audio configuration information definition file. For example, a predetermined software button is pressed Set a condition for transitioning from one state to another, setting a method (function) to be called when transitioning to a predetermined state, or pasting a link to the image configuration information definition file An image file to be reproduced when the state is changed to is designated, or a sound file to be reproduced is designated when a link is pasted to the audio configuration information definition file and the state is changed to a predetermined state.

  The individual audio configuration information definition files 1012, 1022, and 1032 define audio configuration information, design information, dialogue utterance information, and the like. For example, the position of the audio file used for BGM (Back Ground Music) as audio configuration information in the hard disk (Address) is specified, or data “English” is set in the metadata “sound reproduction language”.

  In addition, the individual audio configuration information definition files 1012, 1022, and 1032 define which speaker of an in-vehicle speaker system configured by a plurality of speakers is used as design information, or input from a microphone as dialog utterance information. A method for handling instructions by the operator's voice (processing when there is no response by voice within a predetermined time) is defined.

  In addition, when the car navigation device S has a voice reading application, a character string such as “Good morning” or “Good morning” read by the voice reading application is designated as data for the metadata “playback character string”. Also good.

  The integrated image configuration information definition file 1000 is a file that defines image configuration information, design information, language information, and the like. The integrated image configuration information definition file 1000 is positioned above the individual image configuration information definition files 1010, 1020, and 1030, and duplicate definitions in a plurality of layers. Definitions that are effective when multiple definition files are integrated while referring to each definition file (for example, by definition in individual definition files such as image file playback priority and definition related to simultaneous playback) Is a definition that is suitable to be integrated and defined.)

  Similarly, the integrated state transition information definition file 1001 and the integrated voice configuration information definition file 1002 separately describe definitions to be set when a plurality of definition files are integrated with reference to lower definition files.

  The navigation device S makes the HMI executable on the volatile storage device 5 by the HMI enabling means 20 based on the HMI definition information 10 having the above-described configuration.

  The HMI of the integrated application program has three definition files (1010, 1020, 1030), (1011, 1021, 1031), and (1012, 1022, 1032) below the three definition files 1000, 1001, and 1002, respectively. Based on the definition information 10, while maintaining the hierarchical structure of the HMI definition information 10, the image management engine 200, the state transition engine 201, and the voice management engine 202 constituting the HMI executable means 20 are instantiated in the volatile storage device 5. Is realized (substantiated).

  The image management engine 200 is software such as an XML parser that materializes an HMI related to an image based on an image configuration information definition file. Similarly, the state transition engine 201 and the voice management engine 202 are also a state transition information definition file, a voice configuration, and the like. This is software that materializes an HMI related to state transition and voice configuration based on an information definition file.

  The materialization of the HMI by the HMI enabling means 20 is performed every time the car navigation device S is turned on. By replacing a part of the file constituting the HMI definition information 10, the English environment and the Japanese environment are changed. The change may be made when the HMI of the integrated application program is changed, such as switching between the beginner mode and the expert mode in which the contents of the display screen and the voice message are different.

  The HMI 50 is an instance of the HMI that is materialized in the volatile storage device 5 and includes an integrated image instance 500, an integrated state transition instance 501, and an integrated audio instance 502. The integrated image instance 500 includes application software P1, P2, The image instances 510, 520, and 530 of P3 are referenced while being subordinately subordinate, and the integrated state transition instance 501 and the integrated audio instance 502 are similarly referred to as the state transition instances 511, 521, 531, and the audio instances 512, 522, 532, respectively. It is configured to refer to the subordinates.

  Next, the flow of processing in the application program P that advances the processing via the HMI 50 that has become executable by the HMI enabling means 20 will be described with reference to FIG.

  First, as shown in FIG. 4A, when the integrated state transition instance 501 detects that an event such as a touch operation on the touch panel, an input of a voice command, or an ignition switch is generated, a method for the detected event is detected. Is present in the integrated state transition instance 501.

  For example, the method for preventing the operator application P1 from being displayed only switches on / off the output without changing the image or sound of the operator application P1, and is therefore arranged at the level (higher level) of the integrated state transition instance 501. On the other hand, since the method for switching the voice output from the operator application P1 from Japanese to English requires a detailed setting to change the voice type of the operator application P1, a state transition instance of the operator application P1 is required. It is arranged at level 511 (lower level).

  When the method for the detected event exists in the integrated state transition instance 501, the integrated state transition instance 501 refers to the integrated image instance 500 or the integrated audio instance 502 as illustrated in FIG. Get necessary information about.

  When the method for the detected event does not exist in the integrated state transition instance 501, the integrated state transition instance 501 has information on the event that has occurred in each of the state transition instances 511, 521, and 531 as shown in FIG. To be notified. Each of the state transition instances 511, 521, and 531 determines whether a method for the notified event exists in the state transition instances 511, 521, and 531.

  When there is a method for the detected event (in this case, it is assumed that it exists in the state transition instance 521), the state transition instance 521 has an image instance 520 or an audio instance 522 as shown in FIG. To obtain necessary information on image or sound.

  When there is no method for the detected event (in this case, it is assumed that the method does not exist in the state transition instances 511 and 531), the state transition instances 511 and 531 end the processing.

  With the above configuration, the HMI definition information management system converts the HMI definition information and sequence part of the integrated application program P and the individual application programs P1, P2, and P3 operating on the navigation device S into parts and manages them separately. Therefore, it is possible to reduce the amount of work (load) at the time of modifying the application program such as changing the application program for the US market to the application program for the Japanese market. This is because the scenario of the application program can be easily and quickly changed by changing a part of the affected definition file or replacing only the affected definition file.

  The HMI definition information management system can generate various integrated application programs P by flexibly integrating various functions of the individual application programs.

  Further, the HMI definition information management system can improve the reusability of the sequence part or definition file of the individual application program.

  Further, since the HMI definition information management system materializes the necessary instances on the volatile storage device 5 only when necessary, the capacity of the volatile storage device 5 can be reduced.

  The preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited to the above-described embodiments, and various modifications and substitutions can be made to the above-described embodiments without departing from the scope of the present invention. Can be added.

  For example, in the above-described embodiment, the HMI is dynamically materialized while holding the HMI definition information 10 in the navigation device S. However, the developer of the integrated application program converts the definition file into a component by the HMI definition information management system. The integrated application program P is compiled while being held in another development computer or the like, and the HMI is made executable (binary file) and then stored statically on the nonvolatile storage device 1 of the navigation device S. Good. With this configuration, the navigation device S advances the processing of the integrated application program via the HMI that is statically stored on the nonvolatile storage device 1.

  Even in this case, since the HMI definition information management system converts the HMI definition information 10 into parts, the reusability of the HMI definition information 10 can be improved and the flexibility of the integrated application program can be improved.

  The HMI definition information management system may dynamically change the content, structure, and appearance of the HMI using a DOM (Document Object Model) mechanism.

It is a figure which shows the structural example of the car navigation apparatus using an HMI definition information management system. It is a figure which shows the display screen at the time of making an integrated application program which integrated three separate application programs run with a car navigation apparatus. It is a figure explaining the example of a structure of the HMI definition information of an integrated application program, and the flow of the process which makes HMI executable based on the HMI definition information. It is a figure explaining the flow of a process in the application program which advances a process via HMI made into the executable state by the HMI executable enabling means.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Nonvolatile memory | storage device 2 Control apparatus 3 Image output device 4 Audio | voice output device 5 Volatile memory device 10 HMI definition information 11 Sequence part 20 HMI execution enabling means 50 HMI
100 Integrated application HMI definition information 101, 102, 103 Individual application HMI definition information 200 Image management engine 201 State transition engine 202 Voice management engine 500 Integrated image instance 501 Integrated state transition instance 502 Integrated voice instance 510, 520, 530 Image instance 511, 521, 531 State transition instance 512, 522, 532 Audio instance 1000 Integrated image configuration information definition file 1001 Integrated state transition information definition file 1002 Integrated audio configuration information definition file 1010, 1020, 1030 Individual image configuration information definition file 1011, 1021, 1031 Individual state transition information definition file 1012, 1022, 1032 Individual voice configuration information definition file P Application program P1, P2, P3 individual application program S car navigation system

Claims (2)

An HMI definition information management system for managing HMI definition information of an integrated application program that integrates a plurality of individual application programs,
Based on the HMI definition information composed of the image configuration information definition file and the audio configuration information definition file described in the metadata and the state transition information definition file that associates both files, the HMI of the integrated application program is made executable. HMI enabling means,
An HMI definition information management system comprising: A data structure of HMI definition information in an integrated application program that integrates a plurality of individual application programs,
Individual application HMI definition information composed of an image configuration information definition file and an audio configuration information definition file described in metadata, and a state transition information definition file associating both files;
An image configuration information definition file and an audio configuration information definition file described in metadata, and a state transition information definition file for associating both files, each definition file corresponding to a plurality of individual application HMI definition information Integrated application HMI definition information that refers to the information definition file, the audio configuration information definition file, and the state transition information definition file;
A data structure of HMI definition information, comprising:

Images