JP2015049346A - Control device, electronic control system, control method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To confirm whether a user recognizes information provided by a system by using a natural action of the user.SOLUTION: A control device comprises: a display control unit for causing a display unit to display a display object moving in a display screen of the display unit; a visual field direction detection unit for calculating index data indicating a direction of a view field of a user; a determination unit for determining whether at least a movement direction of the display object in the display screen is matched with the index data calculated by the view field direction detection unit; and a processing unit for performing processing different depending on whether the determination unit determines they are matched.

Description

  The present invention relates to a control device, an electronic control system, a control method, and a program.

  Two or one pupil is detected from the face image by the combination of near-infrared light source and camera, and according to the spatial position of the pupil moving in the plane almost parallel to the personal computer screen by the movement of the head, especially the rotation, A pointing device that moves a cursor on a personal computer screen is known (for example, see Patent Document 1).

  There is also known a control device that identifies a user's head movement from information detected by the head movement detection unit and executes a process desired by the user (such as image switching) corresponding to the angular velocity ( For example, see Patent Document 2). In this apparatus, the user's head movement is not reflected in the process, so that the operation based on the user's head movement is executed more accurately.

JP 2005-182247 A JP 2012-123812 A

  The conventional technology focuses on how to detect the result of the operation performed by the user, and there are cases where it is not possible to properly confirm whether the user has recognized the information provided by the device.

  The present invention has been made in view of such circumstances, and an object of the present invention is to confirm whether or not the user has recognized the information provided by the system by utilizing the natural operation of the user. I will.

One aspect of the present invention is a display control unit that causes the display unit to display a display object that moves within the display surface of the display unit, a visual field direction detection unit that calculates index data representing a direction in which the user's visual field is directed, A determination unit that determines whether at least the moving direction of the display target in the display surface and the index data calculated by the visual field direction detection unit are compatible, and the determination unit determines that the display object is compatible. And a processing unit that performs different processing based on whether or not.
In one aspect of the present invention, the visual field direction detection unit refers to a detection result of the behavior detection unit that detects the behavior of the display unit, and is a moving speed vector of the display unit and parallel to the display surface of the display unit. A moving speed vector in any direction may be calculated as the index data.
In the aspect of the invention, the determination unit may be configured such that an angle formed by the moving speed vector of the display target and the moving speed vector of the display unit in the display surface is equal to or less than a reference angle, and the display target When a difference in magnitude between vectors when one or both of the moving speed vector and the moving speed vector of the display unit is mapped in one direction is equal to or less than a reference speed, at least the display target in the display surface The moving direction and the index data calculated by the visual field direction detection unit may be determined to be compatible.
In the aspect of the invention, the processing unit may perform different processing according to a time during which the state determined to be compatible by the determination unit continues.
Moreover, 1 aspect of this invention WHEREIN: The said process part is good also as what changes the threshold value with respect to the said continuous time according to the information content which the said display object has.
In the aspect of the present invention, the display control unit may move the display target in a direction opposite to the initial movement direction when the determination unit determines that the display unit is suitable.
In one aspect of the present invention, when the display control unit determines that the display unit is suitable, the display control unit converts the display target moving speed vector from the display target moving speed vector. It may be moved in the subtracted direction.
In one aspect of the present invention, when the display control unit determines that the display unit is suitable, the direction of the moving speed vector of the display unit matches the direction in which the display target moves. The display object may be moved in the direction opposite to the moving speed vector of the display unit only when the display object is present.
In the aspect of the invention, the display control unit may stop the position of the display target when it is determined by the determination unit that the display control unit is suitable.
In the aspect of the invention, the display control unit may randomly determine the moving direction of the display target.
In one embodiment of the present invention, a detection unit that detects a moving direction of a landscape in a direction in which a user's visual field is facing is provided, and the display control unit detects the moving direction of the display target by the detecting unit. It may be determined in a direction different from the moving direction of the landscape.
In the aspect of the invention, the detection unit includes an imaging unit that captures a landscape in a direction in which a user's field of view is directed, and performs optical flow processing on the image captured by the imaging unit. The moving direction of the scenery may be detected.
In the aspect of the invention, the detection unit includes a position detection unit that detects a position of the display unit, and the moving direction of the landscape is based on the position of the display unit detected by the position detection unit. May be detected.
Another aspect of the present invention refers to a detection result of a display control unit that causes the display unit to display a display object that moves within the display surface of the display unit, and a behavior detection unit that detects the behavior of the display unit, and A calculating unit that calculates a moving speed vector of the display unit in a direction parallel to the display surface of the display unit, at least the moving direction of the display object in the display surface, and the visual field direction detecting unit A determination unit that determines whether or not the calculated index data is compatible, and when the display control unit determines that the display unit is compatible, the display target is moved up to that time. It is a control device that moves in a direction opposite to the direction.
Another aspect of the present invention refers to a detection result of a display control unit that displays on the display unit a display object that moves within the display surface of the display unit, and a behavior detection unit that detects the behavior of the display unit. A calculating unit that calculates a moving speed vector of the display unit in a direction parallel to the display surface of the display unit; at least a moving direction of the display object in the display surface; and the visual field direction detection A determination unit that determines whether or not the index data calculated by the unit is compatible, and the display control unit displays the display target when the determination is determined by the determination unit It is a control device that moves in the direction based on the moving speed vector of the part.
In another aspect of the present invention, the control computer of the control device causes the display unit to display a display target that moves within the display surface of the display unit, and calculates index data that represents a direction in which the user's visual field is directed. Determining whether or not the calculated index data is compatible with at least the moving direction of the display object in the display surface, and based on whether or not it is determined that the determination unit is compatible, An electronic control method comprising: performing different processes.
According to another aspect of the present invention, the control computer of the control device causes the display unit to display a display object that moves within the display surface of the display unit, and calculates index data representing a direction in which the user's visual field is directed. , At least the moving direction of the display object in the display surface and whether or not the calculated index data is matched, and based on whether or not it is determined by the determination unit, It is a program that performs different processing.

  According to one aspect of the present invention, it is possible to check whether a user has recognized information provided by the system by using a natural operation of the user.

It is a figure which shows typically a mode that the electronic control system 1 which concerns on 1st Embodiment of this invention is utilized. It is a figure which defines an XYZ coordinate system. 3 is a diagram illustrating an example of a functional configuration of a control device 30. FIG. It is a figure which shows an example of a mode that presentation information OD moves the display surface 12A of the see-through type display apparatus 12. FIG. It is a figure which shows the relationship of presentation speed vector-> Vi, moving speed vector-> Vd, and mapping vector-> tVi. It is a figure which shows an example of the change of the display position of presentation information OD when a user visually recognizes presentation information OD and OK determination is made. It is a figure which shows the other example of the change of the display position of presentation information OD when a user visually recognizes presentation information OD and the OK determination is made. It is a figure which shows the other example of the change of the display position of presentation information OD when a user visually recognizes presentation information OD and the OK determination is made. It is a figure which shows a mode that presentation information OD moves to the direction different from the moving direction of the landscape detected by the landscape direction detection part 36A. It is an example of the flowchart which shows the flow of the process performed by the electronic control system 1 of this embodiment. It is another example of the flowchart which shows the flow of the process performed by the electronic control system 1 of this embodiment. It is a figure which shows an example of the screen which the electronic control system 2 of 2nd Embodiment displays on a see-through type display apparatus. It is a figure which shows the other example of the screen which the electronic control system 2 of 2nd Embodiment displays on the see-through type display apparatus.

  Hereinafter, embodiments of a control device, an electronic control system, a control method, and a program according to the present invention will be described with reference to the drawings.

<First Embodiment>
[Hardware configuration]
FIG. 1 is a diagram schematically showing how the electronic control system 1 according to the first embodiment of the present invention is used. The electronic control system 1 has, for example, a glasses-like shape, and includes a head mounted display 10 that is used by being mounted on the user U's head, and a control device 30.
The head mounted display 10 transmits light incident on the user's eyes to some extent, and displays a see-through display device 12 capable of displaying an image on a surface facing the user's eyes, a sensor unit 14, and landscape photography. A camera 16. The see-through display device 12 includes, for example, a light transmissive LCD (Liquid Crystal Display) or an organic EL (Electroluminescence) display device. The see-through display device 12 displays various images based on the image data provided from the control device 30.

  The sensor unit 14 includes, for example, an acceleration sensor and a gyro sensor, and detects acceleration, angular velocity, and the like, which are behaviors of the head mounted display 10 including the see-through display device 12. The camera 16 is a camera having a solid-state imaging device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The detection value of the sensor unit 14 and the image captured by the camera 16 are transmitted to the control device 30.

  The control device 30 is housed in, for example, a housing that can be held by a user, and communicates with the head mounted display 10 by a communication cable or wirelessly to transmit and receive images and sensor detection values. For example, the control device 30 is a device in which dedicated software is installed in a portable terminal such as a mobile phone or a tablet terminal. In addition, the control device 30 is not limited thereto, and may be a dedicated device for the electronic control system 1. Further, the control device 30 may be built in the head mounted display 10.

  Hereinafter, functions of the electronic control system 1 will be described using an XYZ coordinate system as appropriate. FIG. 2 is a diagram for defining an XYZ coordinate system.

[Function configuration and basic processing]
FIG. 3 is a diagram illustrating an example of a functional configuration of the control device 30. The control device 30 includes, for example, an input unit 32 such as a touch panel and various buttons, a processing unit 34, a display control unit 36, a visual field direction detection unit 38, a determination unit 40, and a storage unit 50. For example, a program 52 and content data 54 are stored in the storage unit 50. The control device 30 includes a processor (not shown) such as a CPU (Central Processing Unit). The processing unit 34, the display control unit 36, the visual field direction detection unit 38, and the determination unit 40 are software function units that function when the processor executes the program 52, for example. Some or all of these functional units may be hardware functional units such as LSI (Large Scale Integration) and ASIC (Application Specific Integrated Circuit). The program 52 and the content data 54 are downloaded from, for example, a server device on the Internet when the control device 30 connects to the Internet. The program 52 and the content data 54 may be stored in various portable storage media and installed in the storage unit 50, or may be stored in the storage unit 50 in advance when the control device 30 is shipped. As the storage unit 50, for example, a flash memory, a RAM (Random Access Memory), a register, an HDD, an SD card, or the like may be used.

  For example, the processing unit 34 performs processing for displaying an image based on the content data 54 on the see-through display device 12 and providing the image to the user. In this embodiment, the electronic control system 1 provides a user with learning content for memorizing English words, for example. In the content data 54, English words and translated words to be memorized are stored, for example, for each level. The processing unit 34 reads out desired level English words and translated text data from the content data 54 and outputs them to the display control unit 36 in accordance with a user operation performed on the input unit 32.

  The display control unit 36 causes the see-through display device 12 to display the English word and the translated text data input from the processing unit 34 in a predetermined direction at a predetermined speed. Hereinafter, the information presented in this way will be described as presentation information OD. The presentation information OD is an example of “display target”. Further, a predetermined direction and a predetermined speed in which the presentation information OD moves are expressed as a presentation speed vector → Vi. Hereinafter, “→” indicates that the following code is a vector. Presentation speed vector → Vi is a movement speed vector of the presentation information OD.

  FIG. 4 is a diagram illustrating an example of how the presentation information OD moves on the display surface 12 </ b> A of the see-through display device 12. As shown in FIG. 4, the presentation information OD is controlled so as to move to the end portion 12 </ b> Aa of the display surface 12 </ b> A and disappear if the user does not take a special action.

  The visual field direction detection unit 38 detects the direction in which the user's visual field is directed based on the detection value input from the sensor unit 14. In the present embodiment, the “direction in which the user's visual field is directed” means a direction in which the user views the landscape via the head mounted display 10 (Z direction in FIG. 2). The field-of-view direction detection unit 38 is, for example, index data indicating the direction in which the user's field of view is directed, for example, in the direction parallel to the display surface 12A (direction on the XY plane in FIG. 2) of the see-through display device 12 itself. The movement speed vector → Vd is calculated and output to the determination unit 40. When the user tilts the head and changes the direction of the field of view, the see-through display device 12 inevitably rotates about the head, so the moving speed vector → Vd is the direction in which the user's field of view is oriented. It becomes index data showing.

  The determination unit 40 determines whether or not the direction in which the presentation information OD moves within the display surface 12A (the direction of the presentation speed vector → Vi) matches the movement speed vector → Vd calculated by the visual field direction detection unit 38. Determine. First, the determination unit 40 determines whether or not the angle θ (the absolute value is considered without regard to the direction) between the moving speed vector → Vd and the presentation speed vector → Vi is equal to or less than the reference angle θ1. Next, the determination unit 40 calculates a mapping vector → tVi obtained by mapping the moving speed vector → Vd to the presentation speed vector → Vi, and a difference in magnitude between the presentation speed vector → Vi and the mapping vector → tVi | Vi−tVi. It is determined whether or not | is equal to or less than the reference speed V1. Then, when the angle θ is equal to or less than the reference angle θ1 and the difference | Vi−tVi | is equal to or less than the reference speed V1, the determination unit 40 determines the direction in which the presentation information OD moves within the display surface 12A and the viewing direction. It is determined that the moving speed vector → Vd calculated by the detection unit 38 is compatible. The result of determination by the determination unit 40 is output to the processing unit 34 and the display control unit 36. FIG. 5 is a diagram showing the relationship of the presentation speed vector → Vi, the movement speed vector → Vd, and the mapping vector → tVi.

[Processing based on judgment results]
The processing unit 34 determines whether or not the determination unit 40 determines that the direction in which the presentation information OD moves within the display surface 12A and the moving speed vector → Vd calculated by the visual field direction detection unit 38 are compatible. Different processing is performed accordingly. Hereinafter, the determination of conformity is referred to as OK determination, and the determination of nonconformity is referred to as NG determination.

  If the OK determination continues for a predetermined time, for example, the processing unit 34 determines that the determination user has sufficiently recognized the presentation information OD, and, for example, instructs the display control unit 36 to stop displaying the presentation information OD. At the same time, a confirmed flag is given to the original data of the presentation information OD in the content data 54. The original data to which the confirmed flag is assigned is not read by the processing unit 34 unless there is a special operation by the user. In addition, when the OK determination continues for a predetermined time, the processing unit 34 may add a predetermined value to the score that is internally counted and accumulate it, and present it to the user at a desired timing. . Here, the “predetermined time” may be a fixed time (for example, about 3 [sec]) or may be changed according to the information amount (for example, the number of characters) of the presentation information OD. That is, when the English word or the translated word is a long word, the predetermined time may be set longer. The predetermined time may be defined according to, for example, the level of the presentation information OD. Further, “continue for a predetermined time” does not mean that the OK determination continues without a gap, but may mean that the number of times the OK determination is made reaches a predetermined number of times.

  On the other hand, when the NG determination continues, for example, until the presentation information OD disappears from the display surface 12A, the processing unit 34 determines that the user has not recognized the presentation information OD. After the time has elapsed, the display control unit 36 is instructed to display again.

  As described above, in the electronic control system 1 according to the present embodiment, the user sufficiently recognizes the presentation information OD when the presentation speed vector → Vi and the movement speed vector → Vd continue to be close in both direction and size. Judge. When the user visually recognizes the presentation information OD, the head is unconsciously rotated in order to follow the presentation information OD. For this reason, when the angle θ is equal to or smaller than the reference angle θ1 and the difference | Vi−tVi | is equal to or smaller than the reference speed V1, it can be determined that the user has recognized the presentation information OD. It can be determined that it was not recognized. Since the electronic control system 1 according to the present embodiment performs the above-described control based on such consideration, it is confirmed using the user's natural operation whether the user has recognized the information provided by the system. can do. In addition, since the user's movement is recognized after the direction is clearly indicated, the possibility of erroneous determination due to the movement of the user's head unrelated to the system can be reduced.

  When the OK determination is made (may be continued for a predetermined time), the display control unit 36 changes the presentation information OD to, for example, a certain distance (mapping vector → tVi, or presentation speed vector → Vi. When the difference from the vector → tVi is equal to or less than the reference speed V1, the presentation speed vector → the magnitude of Vi) is returned to the direction opposite to the initial moving direction and displayed on the see-through display device 12. “Initial” means that the display of the presentation information OD is started. FIG. 6 is a diagram illustrating an example of a change in the display position of the presentation information OD when the user visually recognizes the presentation information OD and makes an OK determination. Further, when an OK determination is made (same as above), the display control unit 36 may move the presentation information OD back to the initial position at the start of display and then move it again in the same direction as at the beginning, or at the start of display. It may be returned to the initial position and stopped, or may be stopped on the spot. FIG. 7 is a diagram illustrating another example of a change in the display position of the presentation information OD when the user visually recognizes the presentation information OD and makes an OK determination. Through these processes, the user can obtain a feeling that the user has recognized the presentation information OD and has been able to improve the feeling of use.

  In addition, when the OK determination is made (may be continued for a predetermined time), the display control unit 36 moves the presentation information OD in the direction obtained by subtracting the movement speed vector → Vd from the presentation speed vector → Vi. Also good. FIG. 8 is a diagram illustrating another example of a change in the display position of the presentation information OD when the user visually recognizes the presentation information OD and makes an OK determination. In this case, the amount by which the presentation information OD is moved is, for example, an amount in which an appropriate upper limit value is provided for the difference obtained by subtracting the magnitude of the moving speed vector → Vd from the magnitude of the presentation speed vector → Vi. It is determined.

  Here, when the information can be recognized without moving the head, it may be determined that the information is not recognized although the information is recognized. Therefore, the initial display position of the presentation information is set to a position that partially protrudes from the display surface 12A as shown in the right diagram of FIG. 4, and when the OK determination is made, control is performed so as to return to the position that fits on the display surface 12A. May be.

[Control of the direction in which the presentation information OD is moved]
Here, a process in which the display control unit 36 determines the initial moving direction of the presentation information OD will be described. The display control unit 36 may determine the moving direction of the presentation information OD at random, but it is preferable to determine the moving direction of the presentation information OD by performing the process described below. The display control unit 36 may include a landscape direction detection unit 36A. The landscape direction detection unit 36A detects the moving direction of the landscape in the direction in which the user's visual field is facing. The moving direction of the landscape is clearly manifested as the same direction in a scene where the user is on a vehicle such as a train and is looking at the landscape from the window. Also, in a scene where the user is walking looking straight ahead, the landscape appears in the direction of flowing left and right at the left and right ends of the direction of view. Then, the display control unit 36 moves the presentation information OD in a direction different from the moving direction of the landscape detected by the landscape direction detecting unit 36A (for example, in a direction orthogonal to the moving direction of the landscape).

  The landscape direction detection unit 36A detects the moving direction of the landscape, for example, by analyzing an image captured by the camera 16 in time series and performing, for example, an optical flow process. Since the optical flow process is a known technique, a detailed description thereof is omitted, but a process that employs an average value excluding an abnormal value may be performed. For example, when the control device 30 has a GPS (Global Positioning System) function, the landscape direction detection unit 36A detects the position obtained by GPS calculation (same as the position of the see-through display device 12) and the detection of the sensor unit 14. The moving direction of the landscape may be detected by comparing the direction obtained from the value. The landscape direction detection unit 36A calculates the moving direction of the landscape as data represented by a vector on the XYZ coordinate system shown in FIG.

  The display control unit 36 holds, for example, a table or a map in which the moving direction of the landscape and the moving information of the presentation information OD are associated with each other in the storage unit 50. When the moving direction of the landscape is input, the display control unit 36 The map is searched to acquire and determine the movement information of the presentation information OD.

  FIG. 9 is a diagram illustrating how the presentation information OD moves in a direction different from the moving direction of the landscape detected by the landscape direction detection unit 36A. In such a scene, the scenery is tracked with the eyes, which may cause a false determination that the presentation information OD is recognized or not recognized. On the other hand, in the electronic control system 1 of this embodiment, since the presentation information OD is moved in a direction different from the moving direction of the landscape, the possibility of erroneous determination can be reduced. In addition, when such a process is not performed, the camera 16 can be omitted.

[flowchart]
FIG. 10 is an example of a flowchart showing a flow of processing executed by the electronic control system 1 of the present embodiment. First, the processing unit 34 determines whether or not the display start timing of the presentation information OD has arrived (step S100). The display start timing of the presentation information OD may be determined randomly, or may be determined according to sequence data that defines the timing.

  When the display start timing of the presentation information OD arrives, the processing unit 34 and the display control unit 36 display the display information OD on the see-through display device 12 while moving the presentation information OD (step S102). Next, the visual field direction detection unit 38 calculates the moving speed vector → Vd (step S104). Next, the determination unit 40 determines whether or not the difference between the presentation speed vector → Vi and the mapping vector → tVi is equal to or less than the reference speed V1, and whether the angle θ is equal to or less than the reference angle θ1, that is, within the display surface 12A. It is determined whether or not the direction in which the presentation information OD moves matches the moving speed vector → Vd calculated by the visual field direction detection unit 38 (step S106).

  If the determination unit 40 determines that it is matched in step S106, the determination unit 40 outputs an OK determination (step S108). When the OK determination is output, the display control unit 36 returns the presentation information OD to the direction illustrated in FIGS. 6 and 8, for example (step S110). Instead of this, the display control unit 36 may stop the presentation information OD. Further, the display control unit 36 may perform the process of returning (or stopping) the presentation information OD before the determination in step S106. That is, the display control unit 36 may return or stop the presentation information OD based on the relationship between the presentation speed vector → Vi and the movement speed vector → Vd regardless of whether or not it is “conforming”. Then, the processing unit 34 determines whether or not a predetermined time has elapsed in a state determined to be “conforming” (step S112). When the predetermined time has elapsed, the user recognizes the presentation information OD. (Step S114) and the above-described processing is performed. Then, the display control unit 36 erases the presentation information OD from the display surface 12A (Step S116). When an OK determination is output, the process of step S116 may be omitted and the presentation information OD may be controlled to remain on the display surface 12A. If the predetermined time has not elapsed, the process returns to step S102.

  On the other hand, when it determines with it not adapting in step S106, the determination part 40 outputs NG determination (step S118). When the NG determination is output, the display control unit 36 determines whether or not the presentation information OD has moved to the end of the display surface 12A (step S120). When the presentation information OD has not moved to the end of the display surface 12A, the process returns to step S102, and when the presentation information OD has moved to the end of the display surface 12A, the presentation information is deleted (step S116).

  Here, in the above description, the determination unit 40 determines only whether or not it is compatible, and the processing unit 34 has been described as determining whether or not the user has recognized the presentation information. 40 may determine whether the user has recognized the presentation information. FIG. 11 is a diagram for explaining the flow of processing in this case, and is another example of a flowchart showing the flow of processing executed by the electronic control system 1 of the present embodiment.

  First, the processing unit 34 determines whether or not the display start timing of the presentation information OD has arrived (step S200). When the display start timing of the presentation information OD arrives, the processing unit 34 and the display control unit 36 display the display information OD on the see-through display device 12 while moving the presentation information OD (step S202). Next, the visual field direction detection unit 38 calculates the moving speed vector → Vd (step S204). Next, the determination unit 40 determines whether or not the difference between the presentation speed vector → Vi and the mapping vector → tVi is equal to or less than the reference speed V1, and whether the angle θ is equal to or less than the reference angle θ1, that is, within the display surface 12A. It is determined whether or not the direction in which the presentation information OD moves matches the moving speed vector → Vd calculated by the visual field direction detection unit 38 (step S206).

  When the determination unit 40 determines that “it conforms”, the display control unit 36 returns the presentation information OD to the direction illustrated in FIGS. 6 and 8, for example (step S208). Instead of this, the display control unit 36 may stop the presentation information OD. Further, the display control unit 36 may perform the process of returning (or stopping) the presentation information OD before the determination in step S206. That is, the display control unit 36 may return or stop the presentation information OD based on the relationship between the presentation speed vector → Vi and the movement speed vector → Vd regardless of whether or not it is “conforming”. Then, the determination unit 40 determines whether or not a predetermined time has elapsed in a state where it is determined to be “conforming” (step S210). If the predetermined time has elapsed, an OK determination is output (step S212). . When the OK determination is output, the processing unit 34 determines that the user has recognized the presentation information OD (step S214), and performs the above-described processing. Then, the display control unit 36 erases the presentation information OD from the display surface 12A (Step S216). Note that when the OK determination is output, the process of step S216 may be omitted and the presentation information OD may be controlled to remain on the display surface 12A. If the predetermined time has not elapsed, the process returns to step S202.

  On the other hand, when it determines with it not being suitable in step S206, the determination part 40 determines whether presentation information OD moved to the end of 12 A of display surfaces (step S218). If the presentation information OD has not moved to the end of the display surface 12A, the process returns to step S202. If the presentation information OD has moved to the end of the display surface 12A, the determination unit 40 outputs an NG determination (step S220). Thereafter, the presentation information OD is erased from the display surface 12A (step S216).

[Summary]
According to the electronic control system 1 and the control device 30 of the present embodiment described above, when the presentation speed vector → Vi and the movement speed vector → Vd are close in both direction and size, the user can provide the presentation information OD. Therefore, it is possible to check whether the user has recognized the information provided by the system by using the natural operation of the user. In addition, since the user's movement is recognized after the direction is clearly indicated, the possibility of erroneous determination due to the movement of the user's head unrelated to the system can be reduced.

  Further, according to the electronic control system 1 and the control device 30 of the present embodiment, when the determination unit 40 makes an OK determination, the movement direction is changed in the direction opposite to the previous movement direction, or the movement speed vector → Vd and the presentation speed vector → Since the display is returned to the direction calculated from Vi, the user can feel that he / she has recognized the presentation information OD has been transmitted to the system, and the usability can be improved.

  Further, according to the electronic control system 1 and the control device 30 of the present embodiment, since the presentation information OD is moved in a direction different from the moving direction of the landscape, the possibility of erroneous determination can be reduced.

Second Embodiment
Hereinafter, the electronic control system 2 according to the second embodiment will be described. The electronic control system 1 of the first embodiment has been described by taking a learning system for English words or the like as an example, but the electronic control system 2 of the second embodiment accepts various inputs by the user when viewing the user's field of view. The input is determined by the direction of the. In addition, about the whole structure and a function structure, FIG. 1, 3 in 1st Embodiment is used and description for the second time is abbreviate | omitted.

  FIG. 12 is a diagram illustrating an example of a screen displayed on the see-through display device 12 by the electronic control system 2 according to the second embodiment. The electronic control system 2 according to the second embodiment displays, for example, the presentation information OD1 displayed as “YES” while moving with the presentation speed vector → Vi1 when accepting any two-choice input, and “NO”. The displayed presentation information OD2 is displayed while being moved by the presentation speed vector → Vi2. Then, when the user follows any of the presentation information and the movement speed vector → Vd is detected, it is determined that the presentation information that matches the movement speed vector → Vd is selected. In the example of FIG. 12, when the user tilts his / her head in the direction of the presentation speed vector → Vi1 (right direction), and the right movement speed vector → Vd is detected, “YES” is selected. If the user moves his / her head in the direction of the presentation speed vector → Vi2 (left direction) and, as a result, a movement speed vector → Vd in the left direction is detected, it is determined that “NO” is selected. To do.

  FIG. 13 is a diagram illustrating another example of a screen displayed on the see-through display device 12 by the electronic control system 2 according to the second embodiment. For example, when receiving an electronic mail addressed to the user, the electronic control system 2 of the second embodiment displays the presentation information OD3 notifying the arrival of the electronic mail while moving the presentation speed vector → Vi3. When the user follows the presentation information OD3 and the movement speed vector → Vd matching the presentation speed vector → Vi3 is detected, it is determined that the user has recognized the presentation information OD3, and the electronic mail Display the main text.

  According to the electronic control system 2 and the control device 30 of the second embodiment described above, the user's natural operation is used to determine whether the user has recognized the information provided by the system, as in the first embodiment. Can be confirmed. In addition, since the user's movement is recognized after the direction is clearly indicated, the possibility of erroneous determination due to the movement of the user's head unrelated to the system can be reduced.

  As mentioned above, although the form for implementing this invention was demonstrated using embodiment, this invention is not limited to such embodiment at all, In the range which does not deviate from the summary of this invention, various deformation | transformation and substitution Can be added.

  For example, the entire electronic control system may be realized by a portable terminal having a display unit, not a combination of the head mounted display 10 and the control device 30. Also in this case, it is considered that the user unconsciously moves the mobile terminal in the direction of the presentation speed vector → Vi when following the presentation information OD. For the control method in this case, the methods of the above embodiments can be used.

  In each of the above embodiments, whether or not the user has recognized the presentation information is determined based on whether or not the direction and speed in which the presentation information moves matches the direction and speed in which the head mounted display 10 moves. However, the determination may be made by paying attention only to the direction without considering the speed.

  In each of the above embodiments, the direction of the user's visual field is detected based on the behavior of the display unit. However, the sensor includes a sensor that detects the user's line of sight, and based on the output of the sensor, The direction of the visual field may be detected.

DESCRIPTION OF SYMBOLS 1 Electronic control system 10 Head mounted display 12 See-through display apparatus 12A Display surface 14 Sensor part 16 Camera 30 Control apparatus 32 Input part 34 Processing part 36 Display control part 38 View direction detection part 40 Determination part 50 Storage part 52 Program 54 Content data

Claims (17)

A display control unit that causes the display unit to display a display object that moves within the display surface of the display unit;
A visual field direction detection unit for calculating index data representing the direction in which the user's visual field is directed;
A determination unit for determining whether at least the moving direction of the display target in the display surface is compatible with the index data calculated by the visual field direction detection unit;
A processing unit that performs different processing based on whether or not the determination unit determines that it is compatible; and
A control device comprising: The control device according to claim 1,
The visual field direction detection unit refers to the detection result of the behavior detection unit that detects the behavior of the display unit, and determines a movement speed vector of the display unit in a direction parallel to the display surface of the display unit. , Calculated as the index data,
Control device. The control device according to claim 2,
The determination unit is configured such that an angle formed between the moving speed vector of the display target and the moving speed vector of the display unit within the display surface is a reference angle or less, and the moving speed vector of the display target and the movement of the display unit When the difference in magnitude between vectors when one or both of the velocity vectors is mapped in one direction is equal to or less than a reference velocity, at least the moving direction of the display object in the display surface and the visual field direction detection unit It is determined that the index data calculated by
Control device. The control device according to any one of claims 1 to 3,
The processing unit performs different processing according to a time during which the state determined to be compatible by the determination unit continues.
Control device. The control device according to claim 4,
The processing unit changes a threshold value for the continuous time according to an information amount of the display target.
Control device. The control device according to any one of claims 1 to 5,
The display control unit moves the display object in a direction opposite to the initial movement direction when it is determined by the determination unit to be compatible.
Control device. The control device according to claim 2,
The display control unit moves the display object in a direction obtained by subtracting the moving speed vector of the display unit from the moving speed vector of the display object when it is determined by the determining unit that the display control unit is adapted.
Control device. The control device according to any one of claims 1 to 7,
The display control unit stops the position of the display target when it is determined by the determination unit that the display control unit is adapted.
Control device. The control device according to any one of claims 1 to 8,
The display control unit randomly determines a moving direction of the display target.
Control device. The control device according to any one of claims 1 to 9,
A detection unit that detects the moving direction of the landscape in the direction in which the user's field of view is facing,
The display control unit determines the moving direction of the display target to be a direction different from the moving direction of the landscape detected by the detection unit;
Control device. The control device according to claim 10, comprising:
The detection unit includes an imaging unit that captures a landscape in a direction in which a user's visual field is facing, and detects a moving direction of the landscape by performing an optical flow process on an image captured by the imaging unit.
Control device. The control device according to claim 10 or 11,
The detection unit includes a position detection unit that detects a position of the display unit, and detects a moving direction of the landscape based on the position of the display unit detected by the position detection unit.
Control device. A display control unit that causes the display unit to display a display object that moves within the display surface of the display unit;
A visual field direction detection unit that refers to a detection result of a behavior detection unit that detects a behavior of the display unit, and calculates a movement speed vector of the display unit in a direction parallel to the display surface of the display unit; ,
A determination unit for determining whether at least the moving direction of the display target in the display surface is compatible with the index data calculated by the visual field direction detection unit;
With
The display control unit moves the display object in a direction opposite to the moving direction so far when it is determined by the determination unit to be compatible.
Control device. A display control unit that causes the display unit to display a display object that moves within the display surface of the display unit;
A visual field direction detection unit that refers to a detection result of a behavior detection unit that detects a behavior of the display unit, and calculates a movement speed vector of the display unit in a direction parallel to the display surface of the display unit; ,
A determination unit for determining whether at least the moving direction of the display target in the display surface is compatible with the index data calculated by the visual field direction detection unit;
With
The display control unit moves the display object in a direction based on a moving speed vector of the display unit when the determination unit determines that the display unit is adapted.
Control device. A control device according to any one of claims 1 to 14,
The display unit controlled by the control device;
An electronic control system. Computer
Display the display object that moves within the display surface of the display unit on the display unit,
Calculate index data representing the direction in which the user's field of view is facing,
Determining whether or not the calculated index data is compatible with at least the moving direction of the display object in the display surface;
Depending on whether or not it is determined to be compatible, different processing is performed.
A control method comprising: On the computer,
Display the display object that moves within the display surface of the display unit on the display unit,
Calculate the index data indicating the direction in which the user's field of view is facing,
Determining whether or not the calculated index data is compatible with at least the movement direction of the display object in the display surface;
Depending on whether or not it is determined to be compatible, different processing is performed.
program.

Images