WO2016185541A1

WO2016185541A1 - Display control device and image display device

Info

Publication number: WO2016185541A1
Application number: PCT/JP2015/064211
Authority: WO
Inventors: 博彦樋口; 貴弘大塚
Original assignee: 三菱電機株式会社
Priority date: 2015-05-18
Filing date: 2015-05-18
Publication date: 2016-11-24
Also published as: JP6275332B2; TW201642084A; JPWO2016185541A1

Abstract

Provided is a display control device, comprising: an entry direction acquisition unit (5) which, when a display position of a pointer which moves in accordance with a user operation overlaps with a display position of a movable region which guides the movement of the pointer, acquires a direction at which the pointer has entered the movable region; a verification unit (6) which acquires display information based on the acquired entry direction; a guide generating unit (8) which generates guide regions which guide the pointer and the movable region on the basis of the acquired display information; and a display control unit (3) which carries out a control which displays the pointer, the movable region, and the guide regions.

Description

Display control apparatus and image display apparatus

The present invention relates to a technique for controlling display contents based on a user's gesture operation.

In recent years, head-mounted display devices (hereinafter referred to as head-mounted displays), which are display devices mounted on the head, have been miniaturized, and products that allow users to move while wearing the head-mounted display have been developed. Yes.
Normally, the operation of the display screen of the head mounted display is executed by operating a button arranged on the head mounted display or a touch pad or a button of a controller connected to the head mounted display. In any operation on the display screen, the user needs to attach his / her finger to the head mounted display or hold the controller.

As an operation method that does not take away the freedom of the user's fingers, there is a technique in which the user's fingers are imaged using an imaging means to recognize the shape of the fingers, and a command is executed by a gesture operation with the recognized fingers. As a method of executing a command by a gesture operation of a finger, there are a method of assigning a command to the shape of the user's finger, a method of recognizing the intention of executing the command by holding the finger on the operation screen for a certain time. In Patent Document 1, a user is imaged to generate stereoscopic image data, a virtual operation surface is formed and displayed with a shape and position based on the user's image and position read by the three-dimensional imaging means, and displayed. An image recognition apparatus is disclosed that reads a user's movement with respect to a displayed virtual operation surface with a three-dimensional imaging means and outputs a predetermined signal when it is determined that the movement is an operation.

However, there is a problem that the user needs to memorize in advance the command assigned to the shape of the finger when executing the command by the gesture operation of the finger. Also, the method of recognizing the intention of executing a command by holding the finger still has the problem that it is difficult to hold the finger in the space for a certain period of time. Further, the image recognition apparatus of Patent Document 1 requires a three-dimensional imaging unit, which causes an increase in the size of the apparatus, and causes a problem that the load on the CPU increases due to computation in the depth direction.

For example, Japanese Patent Application Laid-Open No. 2004-228620 detects the user's finger by analyzing the image signal of the image captured by the sensor, and displays a pointer that operates in conjunction with the detected finger in the virtual image display area. When the pointer moves to the vicinity of the boundary between the virtual image display area and the area other than the virtual image display area in accordance with the operation of the finger, the display device shifts to the command reception mode and displays the command assigned to the vicinity of the boundary Is disclosed. As a result, the user can make a transition to the command reception mode only by moving the finger.

JP 2011-175617 A JP 2014-106698 A

However, in the technique disclosed in Patent Document 2 described above, there is a problem in that buttons cannot be arranged in the vicinity of the boundary between the virtual image display area and the area other than the virtual image display area, and the button arrangement is restricted. . Further, in the instruction acceptance mode, it is necessary to store the command assigned to the boundary, and the position of the displayed command is separated from the pointer, which makes it difficult to execute the command smoothly. It was.

The present invention has been made to solve the above-described problems, and allows a command to be executed only by moving the pointer in the space, without restricting the display position of display information such as buttons and images. The purpose is to execute the command smoothly.

In the display control device according to the present invention, when the display position of the pointer that moves in accordance with a user operation and the display position of the movable area that guides the movement of the pointer overlap, the direction in which the pointer enters the movable area is determined. An incident direction acquisition unit to be acquired, a verification unit that acquires display information based on the incident direction acquired by the incident direction acquisition unit, and a guide region that guides the pointer and the movable region based on the display information acquired by the verification unit A guide generation unit to be generated and a display control unit that performs control to display a pointer, a movable region, and a guide region generated by the guide generation unit.

According to the present invention, the command can be executed only by moving the pointer in the space, and the command can be smoothly executed without restricting the display position of the display information on the display screen.

2 is an explanatory diagram illustrating an overview of a display control device according to Embodiment 1. FIG. 1 is a block diagram illustrating a configuration of a display control device according to a first embodiment. 4 is an explanatory diagram illustrating a configuration of a command database of the display control device according to Embodiment 1. FIG. 2 is a diagram illustrating a hardware configuration of a display control apparatus according to Embodiment 1. FIG. 3 is a flowchart showing an operation of the display control apparatus according to the first embodiment. 6 is a diagram showing a display example of the display control apparatus according to Embodiment 1. FIG. FIG. 6 is an explanatory diagram illustrating an effect of the display control device according to the first embodiment. FIG. 6 is an explanatory diagram illustrating an effect of the display control device according to the first embodiment. FIG. 6 is an explanatory diagram illustrating an effect of the display control device according to the first embodiment. FIG. 6 is an explanatory diagram illustrating an effect of the display control device according to the first embodiment. It is a figure which shows the example of a display at the time of applying the display control apparatus which concerns on Embodiment 1 to a list display. It is a figure which shows the case where the movable area | region of the display control apparatus which concerns on Embodiment 1 is comprised with an octagon. 6 is a diagram illustrating a display example when the display control apparatus according to the first embodiment is applied to moving display of an image. FIG. FIG. 10 is an explanatory diagram illustrating an overview of a display control device according to a second embodiment. 6 is a block diagram illustrating a configuration of a display control device according to Embodiment 2. FIG. 10 is a flowchart illustrating an operation of the display control apparatus according to the second embodiment. It is a figure which shows the example of a display at the time of applying the display control apparatus which concerns on Embodiment 2 to a numerical input.

Hereinafter, in order to explain the present invention in more detail, modes for carrying out the present invention will be described with reference to the accompanying drawings.
Embodiment 1 FIG.
First, an outline of control contents of the display control apparatus 100 will be described with reference to FIG. FIG. 1 is a diagram illustrating a display example in which the display control apparatus 100 according to Embodiment 1 performs display control. In FIG. 1, the display control apparatus 100 is applied to a head mounted display (HMD) 200, and the screen of the head mounted display 200 is operated by movement of the user's hand 201 in space (hereinafter referred to as gesture operation). Explained as an example. Note that the display control apparatus 100 is not limited to the gesture operation, and can be applied to a touch panel operation or a mouse operation.

First, as shown in FIG. 1A, the user moves the hand 201 in the space and performs an operation for moving the pointer displayed on the screen of the head mounted display 200. The user's gesture operation is imaged using the imaging unit 202 and the image signal of the acquired image is input to the display control device 100. Although FIG. 1A shows an example in which imaging is performed by one imaging unit 202, a plurality of imaging units may be provided. Any detection method can be used as long as it can detect the user's gesture operation. Further, a plurality of detection methods may be combined.

A display example of the screen of the head mounted display 200 is shown in FIGS. 1 (b) and 1 (c). In the initial screen shown in FIG. 1B, a pointer 301 and a movable area 302 are arranged. The pointer 301 is an instruction display that moves following the user's gesture operation, and the movable area 302 is an area that triggers command execution. In FIG. 1B, when the pointer 301 moves in the direction of the movable region 302 and the pointer 301 enters the movable region, the screen shown in FIG. 1C is displayed.

In the screen of FIG. 1C, a pointer 301, a guidance area 303 that guides the movable area 302, a guide frame 304 that indicates the outer periphery of the guidance area 303, and a command area 305 that is arranged along the guide frame 304. , 306 are displayed. The user performs a gesture operation while looking at the screen of FIG. 1C to move the pointer 301 to the intended

command areas

305 and 306. The pointer 301 and the movable area 302 move in conjunction with each other, and when the movable area 302 moves into the command area 305, the command set in the command area 305 is executed.

Next, the configuration of the display control apparatus 100 according to Embodiment 1 that realizes the display control described above will be described.
FIG. 2 is a block diagram illustrating a configuration of the display control apparatus 100 according to the first embodiment.
The display control apparatus 100 includes an input unit 1, a position information acquisition unit 2, a display control unit 3, an overlap determination unit 4, an incident direction acquisition unit 5, a collation unit 6, a command database storage unit 7, a guide generation unit 8, and a linkage processing unit. 9, the contact determination part 10, the command execution part 11, the cancellation | release determination part 12, the guide display cancellation | release part 13, and the display part 14 are comprised.

The input unit 1 acquires a signal of an image of a user's gesture operation captured by the imaging unit 202. The position information acquisition unit 2 detects the user's hand 201 from the input information received by the input unit 1 and acquires the detected position information of the hand 201 as the pointer position information. Further, the position information acquisition unit 2 refers to the display information on the display unit 14 and acquires the position information of the movable region displayed on the display unit 14. It is assumed that the movable area is arranged at a predetermined position on the initial screen.

The position information acquired by the position information acquisition unit 2 is a coordinate value indicating a display position on the display unit 14, for example. The coordinate values of the entire screen of the head mounted display that is the display unit 14 are acquired in advance, and the coordinate values where the pointer is located and the coordinate values where the movable region is located are obtained. The display control unit 3 performs display control for displaying the pointer and the movable area on the display unit 14 based on the position information of the pointer and the position information of the movable area acquired by the position information acquisition unit 2. Details of the display control unit 3 will be described later.

The overlap determination unit 4 refers to the position information of the pointer and the position information of the movable area acquired by the position information acquisition unit 2, and determines whether the pointer and the movable area overlap. The incident direction acquisition unit 5 stores a buffer (not shown) that temporarily stores the history of position information acquired by the position information acquisition unit 2 when the overlap determination unit 4 determines that the pointer and the movable region overlap. Reference is made to obtain an incident direction indicating from which direction of the movable region the pointer is incident. In the case of the example shown in FIG. 1C, the direction of incidence of the pointer 301 on the movable area 302 is indicated by an arrow A, indicating that the pointer 301 has entered the movable area 302 from the top to the bottom of the page. Show.

The collation unit 6 collates the incident direction acquired by the incident direction acquisition unit 5 with the command database stored in the command database storage unit 7, and acquires command information corresponding to the incident direction. As the command information, information indicating the display content of the command, the allocation position of the command area, the allocation position of the cancellation area, and whether the movable area is expanded in a specific direction is acquired. Here, the cancel region is a cancel boundary composed of, for example, a line segment. The command execution process is canceled when the pointer passes the cancel boundary and moves out of the guidance area.

The command database storage unit 7 is a storage area for storing a command database.
FIG. 3 is a diagram illustrating a configuration example of a command database stored in the display control apparatus 100 according to the first embodiment. FIG. 3A shows the setting contents of the command database, and FIG. 3B shows a display example when the display control is performed based on the setting contents of FIG.
The command database includes, for example, “command”, “angle from the incident direction of the pointer”, and “movable region extension processing”. “Command” indicates the display content of the command, “OK” and “NG” indicate that command areas for executing the “OK” command and “NG” command are arranged, and “Cancel” indicates command execution processing This indicates that a cancel area for canceling is arranged.

“Angle from the incident direction of the pointer” indicates a position assigned to the command area. When the incident direction of the pointer is indicated by an arrow A in FIG. 3B, an area in the angle direction of “0 °” from the incident direction. The “OK” command area is arranged in 401, the “NG” command area is arranged in the area 402 at an angle direction of “90 °” from the incident direction, and “Cancel” is arranged in the area 403 in the angle direction of “180 °” from the incident direction. ”Area is arranged, and the“ cancel ”area is arranged in the area 404 in the angle direction of“ −90 ° ”from the incident direction. Hereinafter, an area to which an “OK” command and an “NG” command are assigned will be described as a command area.

“Moving region extension processing” indicates whether or not the moving region is extended and displayed in a specific direction. FIG. 3A shows that the movable area is extended only in the direction in which the “OK” command area is assigned. Comparing the movable area 302 on the initial screen shown in FIG. 1B with the movable area 302a shown in FIG. 3B, the movable area 302a is in the direction in which the “OK” command area is assigned, that is, on the paper surface. It is displayed expanded from top to bottom.

The guide generation unit 8 generates pointer guidance information based on the command information acquired by the verification unit 6. The guidance information to be generated includes a guide display, a command area, and a cancel area. The generated guidance information is output to the display control unit 3. When the overlap determination unit 4 determines that the pointer and the movable region overlap, the interlocking processing unit 9 instructs the display control unit 3 to display the pointer movement and the movable region in conjunction with each other. Output.

The contact determination unit 10 determines contact between the guide frame of the guidance area displayed on the display unit 14 and the outer periphery of the movable area. Further, the contact between the guide frame to which the command area is assigned and the outer periphery of the movable area is determined. When the command execution unit 11 determines that the outer periphery of the movable region is in contact with a part of the guide frame to which the command region is assigned (hereinafter referred to as a guide frame to which the command region is assigned). In addition, the execution of the command set in the command area is controlled. The cancellation determination unit 12 determines whether the pointer has moved out of the guidance area. The guide display canceling unit 13 outputs an instruction to cancel the display of the guidance area and the guide frame, the display of the command area, and the extended display of the movable area to the display control unit 3. The display unit 14 includes, for example, a head-mounted display screen, a touch panel, and a display, and performs screen display based on the control of the display control unit 3.

FIG. 4 is a diagram illustrating a hardware configuration of the display control apparatus 100 according to the first embodiment.
An input unit 1, a position information acquisition unit 2, a display control unit 3, an overlap determination unit 4, an incident direction acquisition unit 5, a collation unit 6, a guide generation unit 8, an interlock processing unit 9, a contact determination unit 10, The command execution unit 11, the cancellation determination unit 12, the guide display cancellation unit 13, and the display unit 14 are realized when the processor 101 executes a program stored in the memory 102. The command database storage unit 7 constitutes the memory 102. Further, a plurality of processors 101 and a plurality of memories 102 may be configured to cooperate to execute the above-described functions.

Next, details of the display control unit 3 will be described with reference to FIG.
The display control unit 3 includes a pointer display control unit 31, a movable area display control unit 32, and a guide display control unit 33. The pointer display control unit 31 performs control to display a pointer at a specified position based on the position information acquired by the position information acquisition unit 2. The movable area display control unit 32 performs control to display in the movable area at the designated position based on the position information acquired by the position information acquisition unit 2. Moreover, based on the collation result of the collation part 6, control which expands a movable area to a specific direction and displays it is performed.

The pointer display control unit 31 and the movable region display control unit 32 display the pointer and the movable region based on an instruction to display the movement of the pointer and the movement of the movable region in conjunction with each other input from the interlocking processing unit 9. Perform display control to be linked. For example, when an interlock instruction is input from the interlock processing unit 9, the pointer and the movable area are displayed at the designated position based only on the pointer position information. The guide display control unit 33 performs control to display the guide region, the guide frame, the command region, and the cancel region at a specified position based on the guide information generated by the guide generation unit 8. When an instruction to cancel the display interlocking between the pointer and the movable area is input from the interlock processing unit 9, the display interlocking between the pointer and the movable area is canceled.

The pointer display control unit 31, the movable region display control unit 32, and the guide display control unit 33 are input from the guide display release unit 13 with an instruction to cancel the guidance region and guide frame display, the command region display, and the movable region extension display. Then, control is performed to cancel the display of the guide area and the guide frame, the display of the command area, and the extended display of the movable area.

Next, the operation of the display control apparatus 100 will be described with reference to FIG.
FIG. 5 is a flowchart showing the operation of the display control apparatus 100 according to the first embodiment.
The operation of the display control apparatus 100 will be described by dividing it into three processes (1) guide display process, (2) command execution process, and (3) guide non-display process. In the three processes, it is assumed that the input unit 1, the position information acquisition unit 2, the display control unit 3, and the overlap determination unit 4 are always operating, and the pointer display control unit 31 and the movable area display control unit 32 of the display control unit 3 are used. Is assumed to perform control to display the pointer and the movable region based on the position information acquired by the position information acquisition unit 2. Thereby, the pointer and the movable area are displayed on the display unit 14.

(1) Guide Display Processing First, the guide display processing will be described with reference to the processing from step ST1 to step ST9 in the flowchart of FIG.
The overlap determination unit 4 refers to the pointer and the position information of the movable area acquired by the position information acquisition unit 2, and determines whether or not the pointer and the movable area overlap (step ST1). If the pointer and the movable area do not overlap (step ST1; NO), the process returns to step ST1. On the other hand, when the pointer and the movable region overlap (step ST1; YES), the incident direction acquisition unit 5 refers to the history of the position information of the pointer acquired by the position information acquisition unit 2 stored in the buffer, and the pointer is An incident direction indicating from which direction of the movable region the incident is obtained is acquired (step ST2).

The collation unit 6 collates the incident direction of the pointer acquired in step ST2 with the command database stored in the command database storage unit 7, and acquires command information corresponding to the incident direction (step ST3). The guide generating unit 8 generates a guide area, a command area, and a cancel area based on the command information acquired in step ST3 (step ST4).

The movable region display control unit 32 of the display control unit 3 refers to the information indicating the expansion of the movable region in the command information acquired in step ST3, and determines whether or not information indicating the expansion of the movable region in a specific direction is described. Is performed (step ST5). When information instructing extension of the movable area is described in the specific direction (step ST5; YES), the movable area display control unit 32 performs display control for extending the movable area in the specific direction (step ST6). On the other hand, when the information for instructing the extension of the movable region in the specific direction is not described (step ST5; NO), the process proceeds to step ST7. Based on the guide area, the command area, and the cancellation area generated in step ST4, the guide display control unit 33 guides the guide area and the guide frame (hereinafter, when both the guide area and the guide frame are indicated) Display control for displaying the command area and the cancel area (step ST7).

The interlocking processing unit 9 refers to the determination result of the overlap determination unit 4 and determines that the pointer and the movable region overlap (step ST1; YES), the pointer display control unit 31 and the movable region display control unit 32 is instructed to link the pointer and the display of the movable area (step ST8). The pointer display control unit 31 and the movable area display control unit 32 perform display control for interlocking the display of the pointer and the movable area based on the instruction in step ST8 (step ST9).

(2) Command Execution Processing Next, command execution processing will be described with reference to the processing from step ST10 to step ST16 in the flowchart of FIG.
In the command execution process, the pointer display control unit 31 and the movable area display control unit 32 of the display control unit 3 move in conjunction with the pointer and the movable area based on the position information acquired by the position information acquisition unit 2. Display control is performed, and the display unit 14 displays a pointer that moves in conjunction and a movable region.
Following the processing of step ST9, the contact determination unit 10 is based on the position information of the movable area where the movable area display control unit 32 performs display control and the position information of the guide frame where the guide display control unit 33 performs display control. It is determined whether or not the outer periphery of the movable region is in contact with the guide frame (step ST10). When the outer periphery of the movable region is not in contact with the guide frame (step ST10; NO), the process returns to step ST9.

On the other hand, when the outer periphery of the movable region contacts the guide frame (step ST10; YES), the contact determination unit 10 further determines whether the outer periphery of the movable region contacts the guide frame to which the command region is assigned in the guide frame. A determination is made (step ST11). When the outer periphery of the movable area contacts the guide frame to which the command area is assigned (step ST11; YES), the command execution unit 11 executes the command assigned to the contacted guide frame (step ST12). When the command is executed in step ST12, the guide display release unit 13 outputs a guide display release instruction to the pointer display control unit 31, the movable area display control unit 32, and the guide display control unit 33 (step ST13). Specifically, the guide display release instruction is an instruction to release display of the guide area, the command area, and the extension display of the movable area, and an instruction to release the linkage between the pointer and the display of the movable area.

The guide display control unit 33 performs display control for hiding the guide area and the command area based on the guide area and command area display cancellation instruction that is the guide display cancellation instruction input in step ST13 (step ST14). . The movable area display control unit 32 performs display control for returning the movable area to the size before extension based on the instruction to cancel the extension display of the movable area, which is the guide display release instruction input in step ST13 (step ST15). Further, the pointer display control unit 31 and the movable region display control unit 32 perform display of the pointer and the movable region based on the instruction for canceling the link between the pointer and the movable region, which is the guide display release instruction input in step ST13. Display control for releasing the interlock is performed (step ST16). Then, it returns to the process of step ST1 and repeats the process mentioned above.

(3) Guide non-display processing Finally, the guide non-display processing will be described with reference to the processing from step ST17 to step ST20 and the processing from step ST13 to step ST16 in the flowchart of FIG.
The guide non-display process is performed when it is determined in the determination process of step ST11 described above that the movable area is not in contact with the guide frame to which the command area is assigned (step ST11; NO). This is processing when the movable area comes into contact with a cancel area to which no command area is assigned.

When it is determined that the movable region is not in contact with the guide frame to which the command region is assigned (step ST11; NO), the interlocking processing unit 9 determines whether the pointer is moving toward the cancel region (step ST17). ). If the pointer has not moved in the direction of the cancellation area (step ST17; NO), the process returns to step ST10. On the other hand, when the pointer is moving in the direction of the cancel area (step ST17; YES), the pointer display control section 31 and the movable area display control section 32 are instructed to perform display control for releasing the linkage between the display of the pointer and the movable area ( Step ST18). The pointer display control unit 31 and the movable area display control unit 32 individually control the display of the pointer and the movable area based on the position information acquired by the position information acquisition unit 2 in accordance with the instruction of step ST18 (step ST19).

Next, the release determination unit 12 determines whether or not the pointer has moved out of the guidance area based on the determination results of the overlap determination unit 4 and the contact determination unit 10 (step ST20). When the pointer goes out of the guidance area (step ST20; YES), the process proceeds to step ST13. On the other hand, when the pointer does not go outside the guidance area (step ST20; NO), the process returns to step ST11 and the above-described process is repeated.

Next, the flowchart of FIG. 5 will be described with reference to a specific example of FIG.
FIG. 6 is a diagram illustrating a screen display example of the display control apparatus 100 according to the first embodiment. In the description referring to the specific example, the description will be divided into three processes (1) guide display process, (2) command execution process, and (3) guide non-display process.
First, an initial screen shown in FIG. 6A is displayed on the display unit 14 before the guide display process is started. Based on the display control of the pointer display control unit 31 and the movable region display control unit 32, the pointer 501 and the movable region 502 are displayed on the initial screen.

(1) Guide display process When the screen changes from the initial screen shown in FIG. 6A to the screen shown in FIG. 6B, the overlap determination unit 4 determines whether the pointer 501 and the movable region 502 overlap as step ST1. In the case of FIG. 6B, the incident direction acquisition unit 5 determines that the pointer has moved in the arrow A direction from the upper side to the lower side of the page, and sets the incident direction to the movable region of the pointer as “downward” in step ST <b> 2. get.

In step ST3 and step ST4, the collation unit 6 collates the incident direction “downward” with the command database shown in FIG. 3A and sets the angle ‘0 °’ to the incident direction of the pointer 501. “OK” command region, “NG” command region at an angle “90 °” described in FIG. 3B with respect to the incident direction, an angle “180 °” described in FIG. 3B with respect to the incident angle, and Information for assigning a cancel region to the position of the angle “−90 °” shown in FIG. 3B with respect to the incident direction and information for extending the movable region in the direction in which the “OK” command region is assigned are acquired. In step ST5, the movable area display control unit 32 refers to the extension information acquired by the collation unit 6, and determines whether or not the movable area 502 is extended. In the case of FIG. 3A, as a determination result in step ST6, the movable area 502 is expanded in the allocation direction of the “OK” command area to be a movable area 502a.

As shown in FIG. 6C, the guide generation unit 8 displays an “OK” command area 504 in an area adjacent to the guide area 503 and the guide frame 503a in step ST7, and “ NG "command area 505 is displayed. Further, the guide frames 503c and 503d are also cancel boundaries that are cancel areas. The interlock processing unit 9, the pointer display control unit 31, and the movable area display control unit 32 perform display control for interlocking the display of the pointer 501 and the movable area 502 in FIG. 6C as step ST8 and step ST9.

(2) Command Execution Processing When the pointer 501 and the movable area 502a move on the screen shown in FIG. 6C and change to the display screen shown in FIG. 6D or 6E, the contact determination unit 10 determines whether the outer periphery of the movable region 502a is in contact with the guide frame of the guide region 503 as step ST10. Further, in step ST11, it is determined whether the outer periphery of the movable area 502a has contacted the guide frames 503a and 503b to which the “OK” command area 504 and the “NG” command area 505 are assigned.

In the example shown in FIG. 6D, it is determined that the pointer 501 follows the movement locus indicated by the arrow B, and the outer periphery of the movable area 502a has come into contact with the guide frame 503a to which the “OK” command area 504 is assigned. . In the example shown in FIG. 6E, it is determined that the pointer 501 follows the movement locus indicated by the arrow C, and the outer periphery of the movable area 502 is in contact with the guide frame 503b to which the “NG” command area 505 is assigned. The

In step ST12, the command execution unit 11 executes an “OK” command in the case of FIG. 6D, and executes an “NG” command in the case of FIG. When the guide display cancellation unit 13 outputs a guide display cancellation instruction as step ST13, the pointer display control unit 31, the movable area display control unit 32, and the guide display control unit 33 perform steps ST14 to ST16 as shown in FIG. As shown in FIG. 6G, the display of the guide area 503, the display of the “OK” command area 504 and the “NG” command area 505, and the extension display of the movable area 502a are canceled, and only the pointer 501 and the movable area 502 are displayed. Display. Note that the contents of the executed command may be displayed in the movable area 502.

(3) Guide non-display processing When the pointer 501 moves and changes to the display screen shown in FIG. 6H on the screen shown in FIG. 6C, the contact determination unit 10 moves the movable region 502 as step ST10. It is determined whether or not the outer periphery of the contact has come into contact with the guide frame in the guide region 503. Further, in step ST11, it is determined whether or not the outer periphery of the movable area 502 is in contact with the guide frames 503a and 503b to which the command area is assigned. In the example shown in FIG. 6H, the pointer follows the movement locus indicated by the arrow D, and the guide frame 503c to which the command area is not assigned to the outer periphery of the movable area 502a, that is, the guide frame 503c to which the cancel area is assigned. It is determined that the contact has been made.

In step ST17, in the case of FIG. 6 (h), the interlocking processing unit 9 determines that the pointer is moving in the direction of the guide frames 503c and 503d in which the cancel area is set, and instructs the cancellation of interlocking in step ST18. In step ST19, the pointer display control unit 31 and the movable region display control unit 32 move only the pointer 501 without moving the movable region 502 in the case of FIG. Display control for individually moving the pointer 501 and the movable area 502 is performed only by moving in the directions of the guide frames 503c and 503d in which the cancel area is set.

In step ST20, the release determination unit 12 determines that the pointer 501 has moved out of the guidance area in the case of FIG. 6 (h). Thereafter, the process proceeds to step ST13. As shown in FIG. 6 (i), the guide display cancellation unit 13, the pointer display control unit 31, and the movable region display control unit 32 perform a guide region 503, an “OK” command region 504, and “NG” as shown in FIG. The display of the command area 505 and the expanded display of the movable area 502a are canceled, and only the pointer 501 and the movable area 502 are displayed.

Next, the effect when the display control apparatus 100 according to the first embodiment is applied will be described with reference to FIGS.
7 to 10 are diagrams showing effects obtained when the display control apparatus 100 according to the first embodiment is applied.
FIG. 7A shows a case where the movement locus indicated by the arrow E is traced and the pointer 601 overlaps the movable area 602. FIG. 7B follows the movement locus indicated by the arrow F, and the pointer 601 is movable. The case where it overlaps with the area | region 602 is shown. In both FIG. 7A and FIG. 7B, the “OK”

command areas

603a and 603b are positions where the angle of the pointer 601 from the incident direction (arrows E and F) is “0 °”. This is because the incident direction acquisition unit 5 acquires the direction in which the pointer 601 is incident on the movable region 602, and the collation unit 6 acquires command information corresponding to the incident direction, so that the pointer 601 does not depend on the movement locus of the pointer 601. The command region allocation position is the same with respect to the incident direction to the movable region 602. As a result, as shown in FIGS. 7A and 7B, the “OK”

command areas

603 a and 603 b can be arranged on the extension line of the movement locus of the pointer 601, and the pointer 601 is moved to the movable area 602. Commands can be continuously input only by the incident operation. In addition, the command can be continuously executed regardless of the movement locus of the pointer 601.

FIG. 8 shows a case where the command area 603c is not assigned to the position where the angle from the arrow G that is the incident direction of the pointer 601 is “0 °”, that is, the extension line in the incident direction. In the command database shown in FIG. 3A, the command area 603c is set in advance so as not to be assigned to the angle “0 °” from the incident direction of the pointer 601. Based on the incident direction of the pointer 601 and the command database, the collation unit 6 assigns the command area 603c to a position at an angle “−90 °” from the incident direction of the pointer 601 as shown in FIG. Accordingly, it is possible to set so that the “OK” command is not executed even when the pointer 601 erroneously goes straight in the direction of the arrow G in the direction of the arrow G and goes out of the guidance area. In this way, erroneous input by the user can be suppressed by considering the allocation position of the command area.

FIG. 9 is a diagram showing the interlocking between the display of the pointer 601 and the movable area 602. The display of the pointer 601 and the movable area 602 is interlocked, and the command set in the command area 603 is executed via the guide area 604. It is a configuration. In the contact determination unit 10, it is determined that the outer periphery of the movable region 602 has come into contact with the guide frames 604 a and 604 b to which the cancel region is assigned, and in the interlocking processing unit 9, the guide frames 604 a and 604 a to which the cancel region is assigned. When it is determined to move in the direction 604b, the interlock processing unit 9, the pointer display control unit 31, and the movable area display control unit 32 release the interlocking of the display of the pointer 601 and the movable area 602, and the pointer 601 and the movable area A display in which 602 moves individually is performed.

Thus, when the pointer 601 and the movable area 602 overlap each other, the user can recognize that the pointer 601 and the movable area 602 move in conjunction with the direction in which the command area is assigned. For this reason, as shown in FIG. 9B, when the movement locus of the pointer 601 meanders and the user moves the pointer 601 in the direction of the cancel region unintentionally, the display of the pointer 601 and the movable region 602 is linked. It is canceled and the user can recognize that the command area 603 is moving in a direction not assigned. The user can change the moving direction of the pointer 601 and smoothly guide the movable area 602 to the area to which the command area 603 is assigned.

FIG. 10 is a diagram illustrating a case where the pointer moves outside the guidance area.
The initial screen in which the pointer 601 and the movable area 602 shown in FIG. 10A do not overlap each other changes to the state where the pointer 601 and the movable area 602 shown in FIG. 10B1 overlap. When the extension process of the movable area 602 is set in the command database stored in the command database storage unit 7, the movable area is set in the defined direction as shown in FIGS. 10 (b1) and 10 (b2). Elongate. The pointer 601 follows the movement trajectories Ia, Ib, and Ic via the extended movable region 602a and exits from the guide region 604a where the cancel region is set. As a result, even if the buttons that are the movable regions 602 are arranged adjacent to each other as shown in FIG. 10A, the movement trajectories Ia, Ib, and Ic can be followed to cancel the command execution. .

On the other hand, FIG. 10 (c <b> 1) and FIG. 10 (c <b> 2) illustrate a case where the movable region 602 is not extended and the area of the guide region of the guide region 604 approximates the area of the movable region 602. In this case, in order to return to the initial screen of FIG. 10A, when the pointer 601 is moved by following the movement trajectory of the arrows Ja and Jb, the pointer 601 overlaps the adjacent movable area 605, and the movable area 605 is newly added. Corresponding command area 606 and guide area 607 are displayed, and the initial state shown in FIG.

In the example shown in FIG. 10D1 to FIG. 10D3, the movable region 602 is not expanded, and only the guide region of the guide region 604 is expanded. In this case, as indicated by arrows Ka, Kb, and Kc, the moving distance by the pointer 601 becomes long, and the hand shake of the user who operates the pointer 601 occurs. Therefore, the user may move the pointer 601 in an unintended direction.
In the display control apparatus 100, as shown in FIG. 10B, the overlap determination unit 4 determines whether the pointer 601 and the movable area 602 overlap, and sets the movable area 602 appropriately in advance in the command database. Accordingly, the movable area display control unit 32 can appropriately expand and display the movable area based on the information based on the command database, and the operability of the display unit 14 can be improved.

Next, application examples of the display control apparatus 100 according to the first embodiment will be described with reference to FIGS. 11 to 13.
FIG. 11 is an explanatory diagram illustrating a case where the display control apparatus 100 according to the first embodiment is applied to display of a list menu.
In FIG. 11A, a plurality of

movable areas

702a, 702b, 702c, and 702d are arranged side by side on the display unit 14 to constitute a list menu. For example, the movable area 701a constitutes the menu “Item A”, and the movable area 701b constitutes the menu “Item B”. FIG. 11A shows an initial screen, and the pointer 701 does not overlap any of the

movable areas

702a, 702b, 702c, and 702d. When the pointer 701 is moved in the direction of the arrow L on the initial screen, the screen transitions to the screen shown in FIGS. 11B, 11C, 11E, and 11F according to the amount of movement.

11B, when the overlap determination unit 4 determines that the pointer 701 has overlapped the movable area 702a, a command area 703a and a guide area 704 are displayed, and the movable area 702a is expanded in the direction in which the command area 703a is arranged. In the case of the example of FIG. 11B, the command database stored in the command database storage unit 7 has the “OK” command area allocated in the direction of the angle “90 °” from the incident direction of the pointer. It is set to “execute” the extension process of the movable region in the direction. In addition, the cancel area is assigned to the directions of the angles “0 °”, “180 °”, and “−90 °” from the incident direction of the pointer, and the extension process of the movable area is set to “not”.

11C, when the overlap determination unit 4 determines that the pointer 701 has overlapped the movable area 702b, a command area 703b and a guide area 704 are displayed, and the movable area 702b is expanded in the direction in which the command area 703b is assigned. Furthermore, as shown in FIG. 11D, when the pointer 701 moves in the direction of the arrow La, the contact determination unit 10 causes the outer periphery of the movable region 702b to contact the guide frame of the guide region 704, and the command region 703b is assigned. The command execution unit 11 selects the menu “Item B” set in the command area 703b.

In FIG. 11 (f), when the overlap determination unit 4 determines that the pointer 701 has overlapped the movable area 702d,

command areas

703d and 703e and a guide area 704 are displayed, and the movable area 702d is arranged in the direction in which the

command areas

703d and 703e are assigned. It is stretched. Further, as shown in FIG. 11G, when the pointer 701 moves in the direction of the arrow Lb, the contact determination unit 10 causes the outer periphery of the movable region 702d to contact the guide frame of the guide region 704, and the command region 703e is assigned. The command execution unit 11 executes the command “Next” assigned to the command area 703e. With this execution, the display from “Item A” to “Item C” on page “1/5” is switched to the display from “Item D” to “Item F” on page “2/5”.

12 and 13 are explanatory diagrams illustrating a case where the display control apparatus 100 according to the first embodiment is applied to moving display of an image.
FIG. 12A shows a display example when the movable area 802 is configured with an octagon, and FIG. 12B shows a configuration example of a command database when the movable area 802 is configured with an octagon.
The movable region 802 is not limited to a rectangle, and can be configured by an octagon or another polygon shown in FIG. When the movable region 802 is an octagon, the incident direction acquisition unit 5 specifies the incident direction of the pointer 801 in eight directions. The collation unit 6 collates the incident direction acquired by the incident direction acquisition unit 5 with the command database stored in the command database storage unit 7 and acquires command information corresponding to the eight incident directions.

The command database is composed of “command”, “angle from the direction of incidence of the pointer” and “extension processing of movable region” as in FIG. The “move arrow in the entry direction” command is placed at an angle “0 °” from the incident direction of the pointer, and the “move arrow in the opposite direction” command is an angle of “180 °” from the incident direction of the pointer. It is defined to be placed at a position. Here, the “move arrow in the entry direction” command is a command for moving the background image in the direction in which the pointer has entered the movable area, and the “move arrow in the opposite direction” command is for the pointer to enter the movable area. This command is used to move the background image in the opposite direction. “Cancel” command should be placed at the angle of 45 °, “90 °”, “135 °”, “−45 °”, “−90 °”, “−135 °” from the incident direction of the pointer. Is defined. The extension process of the movable area is defined as “No” for all commands.

FIG. 12A shows a collation result between the command database shown in FIG. 12B and the incident direction acquired by the incident direction acquisition unit 5, and the pointer 801 enters the movable region 802 from the direction of arrow M. Is assigned a command area 803a displaying a “moving arrow in the approach direction” at a position of an angle “0 °” from the arrow M direction, and a “non-entry direction” at a position of an angle “180 °” from the arrow M direction. A command area 803b displaying "Move arrow to" is assigned.

FIG. 13A to FIG. 13F are explanatory diagrams showing the movement of the image according to the incident direction of the pointer.
In FIG. 13A and FIG. 13B, the pointer 801 enters the movable area 802 from the direction of the arrow M, and based on the command information corresponding to the incident direction, the guide display control unit 33 displays the

command areas

803a and 803b. A guide area 804 and a cancel area (not shown) are displayed. Further, the movable region 802 moves in the direction of arrow M, the outer periphery of the movable region 802 contacts the guide frame of the guide region 804 to which the “moving arrow in the entry direction” command region is assigned, and the contact determination unit 10 moves the movable region 802. Is determined to have contacted the guide frame of the guide area 804 to which the command area 803a is assigned, the command execution unit 11 executes the command assigned to the command area 803a, and the image X moves in the entry direction of the pointer 801 The image Xa is obtained.

In FIG. 13C and FIG. 13D, the pointer 801 enters the movable area 802 from the direction of the arrow N, and the guide display control unit 33 determines the

command areas

803c and 803d based on the command information corresponding to the incident direction. A guide area 804 and a cancel area (not shown) are displayed. Further, the movable region 802 moves in the direction of arrow N, the outer periphery of the movable region 802 contacts the guide frame of the guide region 804 to which the “moving arrow in the entry direction” command region is assigned, and the contact determination unit 10 moves the movable region 802. Is determined to have contacted the guide frame of the guide area 804 to which the command area 803c is assigned, the command execution unit 11 executes the command assigned to the command area 803c, and the image X moves in the entry direction of the pointer 801 The image Xb is obtained.

13 (e) and 13 (f), the pointer display 801 enters the movable area 802 from the direction of the arrow O, and the guide display control unit 33 uses the

command area

803e, 803f, a guide area 804, and a cancel area (not shown) are displayed. Further, the movable area 802 moves in the direction of arrow O, the outer periphery of the movable area 802 contacts the guide frame of the guide area 804 to which the “moving arrow in the entry direction” command area is assigned, and the contact determination unit 10 moves the movable area 802. Is determined to have touched the guide frame to which the command area 803e is assigned, the command execution unit 11 executes the command assigned to the command area 803e, and the image X is moved in the entry direction of the pointer 801 to become the image Xc. .

As described above, according to the first embodiment, when the overlap determination unit 4 determines that the pointer and the movable area overlap, it is determined from which direction of the movable area the pointer has entered the movable area. An incident direction acquisition unit 5 that acquires an incident direction to be shown, a verification unit 6 that acquires command information by comparing the acquired incident direction with a command database, and a guide that generates guidance information from the command information acquired by the verification unit 6 Since the generation unit 8 and the guide display control unit 33 that performs display control of the guide area, the command area, and the cancellation area based on the generated guidance information are provided, the movable area that guides the pointer and the movable area are guided. The guide area and the command area can be arranged anywhere on the display unit 14.
In addition, since the command area allocation position is determined based on the direction in which the pointer is incident from the movable area, the command can be executed without depending on the movement trajectory of the pointer. In addition, commands can be easily executed continuously.
Furthermore, since the pointer moves through the movable area and the guide area, it is possible to absorb the hand shake of the user who operates the pointer and smoothly lead to command execution. Further, without providing depth detection means for the user's finger, the user can execute the command only by moving the finger in the space without requiring the user to stop the finger.

Further, according to the first embodiment, when the command database describes a setting for expanding and displaying the movable area in the specific direction, the movable area display control unit 32 sets the movable area in the specific direction. Since the display control to be extended is performed, the moving distance of the pointer can be shortened, and the shaking of the user who operates the pointer can be absorbed.

Further, according to the first embodiment, when the contact determination unit 10 determines that the outer periphery of the movable region is in contact with the guide frame to which the command region is not assigned, the interlock processing unit 9 sets the cancel region Since the linkage between the display of the pointer and the movable area is released only in the movement in the set guide frame direction, the user linked the pointer and the movable area in the guide frame direction to which the command area is assigned. The user can recognize that the pointer has moved, and the user can recognize that the pointer has moved in the direction of the canceling area when the pointer and the movable area are not moving together. Thereby, when the user moves the pointer in the direction of the cancel area unintentionally, the moving direction of the pointer can be changed.

Further, according to the first embodiment, the collation unit 6 is configured to refer to the command database in which the command area allocation position is set based on the incident direction of the pointer, so the command area allocation position is considered. As a result, it is possible to prevent the user from selecting a command by mistake.

Further, according to the first embodiment, it is possible to apply to various user interfaces such as a list display by simultaneously displaying a plurality of movable regions. *

Further, according to the first embodiment, the movable region can be configured in any shape, and can be applied to various user interfaces such as direction input. *

Embodiment 2. FIG.
In the second embodiment, a case where a plurality of movable areas and a plurality of command areas are displayed on the display unit 14 will be described.
FIG. 14 is a diagram illustrating a display example as a result of display control performed by the display control apparatus 100a according to the second embodiment.
When the pointer 901 enters the command display area 902, display control is performed to display a plurality of movable areas 903, a plurality of guide areas, a plurality of guide frames (hereinafter referred to as guide areas) 904, and a plurality of command areas 905. . In the state shown in FIG. 14, when the pointer 901 moves and the overlap determination unit 4 determines that the pointer 901 and any movable region 903 overlap, the display control process described in the first embodiment is performed.

FIG. 15 is a block diagram illustrating a configuration of the display control apparatus 100a according to the second embodiment. In the following description, the same or corresponding parts as those of the display control device 100 according to the first embodiment are denoted by the same reference numerals as those used in the first embodiment, and the description thereof is omitted or simplified. .
The display control apparatus 100a includes a pointer position determination unit 15 and a display information storage unit 16 in place of the incident direction acquisition unit 5, the collation unit 6, the command database storage unit 7, and the guide generation unit 8 described in the first embodiment. Yes.

The pointer position determination unit 15 refers to the position information of the pointer acquired by the position information acquisition unit 2, and determines whether the pointer is positioned in the command display area. The command display area is an area set in advance within the display area of the display unit 14, and a command is displayed when the pointer enters the area. When the pointer is positioned in the command display area, the pointer position determination unit 15 instructs the movable area display control unit 32a and the guide display control unit 33a to display the movable area, the guide area, and the command area. The movable area display control unit 32 a refers to the display information stored in the display information storage unit 16 and performs display control for displaying a plurality of movable areas on the display unit 14. The guide display control unit 33 a refers to the display information stored in the display information storage unit 16 and performs control to display the guide area on the display unit 14.

The display information storage unit 16 displays display information indicating the display size and display position of the movable area, the display size and display position of the guide area and each guide frame, the display content of the command, the arrangement size of the command area, and the arrangement position of the command area. I remember it. The overlap determination unit 4 outputs the determination result to the interlocking processing unit 9 and the release determination unit 12. When the command execution unit 11 controls execution of the command, the command execution unit 11 notifies the release determination unit 12a. The cancellation determination unit 12a refers to the determination results of the overlap determination unit 4 and the contact determination unit 10, and determines whether it is determined that the pointer has moved out of the command display area. The guide display cancellation unit 13a outputs an instruction to cancel the display of the guide area to the display control unit 3 only when the cancellation determination unit 12a determines that the pointer has moved out of the command display area.

Next, the operation of the display control apparatus 100a will be described with reference to FIG.
FIG. 16 is a flowchart showing the operation of the display control apparatus 100a according to the second embodiment. In the following, the same steps as those of the display control apparatus 100 according to the first embodiment are denoted by the same reference numerals as those used in FIG. 5, and the description thereof is omitted or simplified.
The operation of the display control apparatus 100a will also be described by dividing it into three processes (1) guide display process, (2) command execution process, and (3) guide non-display process. In the three processes, it is assumed that the input unit 1, the position information acquisition unit 2, and the display control unit 3 are always in operation, a command display area is displayed in advance on the display unit 14, and the pointer display control unit of the display control unit 3 It is assumed that 31 performs control to display a pointer based on the position information acquired by the position information acquisition unit 2.

(1) Guide Display Process First, the guide display process will be described with reference to the processes in steps ST31 to ST33, step ST8, and step ST9 in the flowchart of FIG.
The pointer position determination unit 15 refers to the position information of the pointer acquired by the position information acquisition unit 2, and determines whether or not the pointer is positioned in the command display area (step ST31). If the pointer is not located in the command display area (step ST31; NO), the process returns to step ST31. On the other hand, when the pointer is located in the command display area (step ST31; YES), the movable area display control unit 32a and the guide display control unit 33a refer to the display information stored in the display information storage unit 16 and perform a plurality of movements. Display control for displaying the area, the plurality of guide areas, and the plurality of command areas is performed (step ST32).

The overlap determination unit 4 refers to the pointer and the position information of the movable area acquired by the position information acquisition unit 2, and determines whether or not the pointer and the movable area overlap (step ST33). If the pointer does not overlap the movable area (step ST33; NO), the process returns to step ST33. On the other hand, when the pointer and the movable region overlap (step ST33; YES), the interlocking processing unit 9 instructs the pointer display control unit 31 and the movable region display control unit 32a to interlock the display of the pointer and the movable region. (Step ST8). The pointer display control unit 31 and the movable area display control unit 32a perform display control for linking the pointer displayed on the display unit 14 and the display of the movable area based on the instruction in step ST8 (step ST9).

(2) Command Execution Processing Next, command execution processing will be described with reference to the processing from step ST10 to step ST12 and step ST34 to step ST37 in the flowchart of FIG.
When the processing from step ST10 to step ST12 is performed and the command is executed, the cancellation determination unit 12a refers to the determination results of the overlap determination unit 4 and the contact determination unit 10, and whether or not the pointer has moved out of the command display area. Is determined (step ST34). If the pointer does not go outside the command display area (step ST34; NO), the process returns to step ST33. On the other hand, when the pointer goes out of the command display area (step ST34; YES), the guide display cancellation unit 13a instructs the pointer display control unit 31, the movable region display control unit 32a, and the guide display control unit 33a to cancel the guide display. Is output (step ST35). The guide display release instruction is specifically a non-display instruction for the guide area, the command area, and the movable area.

The guide display control unit 33a performs display control for hiding the guide area and the command area based on the guide area and command area non-display instruction that is the guide display cancellation instruction input in step ST35 (step ST36). . The movable area display control unit 32a performs display control to hide the movable area based on the instruction to hide the movable area input in step ST35 (step ST37). Then, it returns to the process of step ST31 and repeats the process mentioned above.

(3) Guide non-display processing Finally, the guide non-display processing will be described with reference to the processing from step ST17 to step ST19 and the processing from step ST34 to step ST37 in the flowchart of FIG.
The guide non-display process is performed when it is determined in the determination process of step ST11 described above that the movable area is not in contact with the guide frame to which the command area is assigned (step ST11; NO). This is processing when the movable area comes into contact with a cancel area to which no command area is assigned.

When it is determined that the movable region is not in contact with the guide frame to which the command region is assigned (step ST11; NO), the interlocking processing unit 9 determines whether the pointer is moving toward the cancel region (step ST17). ). If it has not moved in the cancellation area direction (step ST17; NO), the process returns to step ST10. On the other hand, when moving toward the cancel region (step ST17; YES), the pointer display control unit 31 and the movable region display control unit 32a are instructed to perform display control for releasing the linkage between the display of the pointer and the movable region (step ST18). ). The pointer display control unit 31 and the movable area display control unit 32a individually control the display of the pointer and the movable area based on the position information acquired by the position information acquisition unit 2 based on the instruction in step ST18 (step ST19). Thereafter, the process proceeds to the determination process of step ST34.

Next, the flowchart of FIG. 16 will be described with reference to a specific example of FIG.
FIG. 17 is a diagram illustrating a screen display example of the display control device 100a according to the second embodiment, and is an explanatory diagram illustrating a case where the display control device 100a is applied to numerical input.
In step ST31, the pointer position determination unit 15 determines whether the pointer 901 is located in the command display area 902. As shown in FIG. 17, when the pointer 901a is located in the command display area 902, the movable area display control unit 32a and the guide display control unit 33a perform a plurality of movable areas 903, a plurality of guide areas 904, and Display control for displaying a plurality of command areas 905 is performed (step ST32).

The overlap determination unit 4 determines the overlap between the pointer 901 and the movable area 903 as step ST33. As shown in FIG. 17, the pointer 901b and the movable area 903a overlap. The interlock processing unit 9, the pointer display control unit 31, and the movable region display control unit 32a perform display control for interlocking the display of the pointer 901b and the movable region 903a in FIG. 17 as step ST8 and step ST9.

The contact determination unit 10 determines whether or not the movable region 903a has contacted the guide frame of the guide region 904 and the command region 905 has been allocated as the step ST10 and step ST11. In the case of the example illustrated in FIG. 17, the contact determination unit 10 determines that the movable region 903a has contacted the guide frame of the guide region 904a to which the command region 905a is assigned. The command execution part 11 performs selection of command "6" as step ST12. The cancellation determination unit 12a determines whether the pointer 901 has moved out of the command display area 902 as step ST34. In the case of the example shown in FIG. 17, the pointer 901a does not go out of the command display area 902, but repeats the processing from step ST33 and step ST8 to step ST12, executes the selection of the command “0”, and selects the command “Ent”. Execute.

Thereafter, when the cancellation determination unit 12a determines that the pointer 901 has moved out of the command display area 902 in step ST34, the guide display cancellation unit 13a outputs a guide display cancellation instruction in step ST35. In step ST36, the guide display control unit 33a performs display control to hide the guide area 904 and the command area 905. The movable area display control part 32a performs display control which hides the movable area 903 as step ST37.
As shown in FIG. 17, the display control unit 3 may be configured to display a selection result of the command executed by the command execution unit 11, for example, a numerical value 906 indicating “600” on the display unit 14. . Further, when the display control for hiding the guide area 904 and the command area 905 is performed, the display of the numerical value 906 may be left.

As described above, according to the second embodiment, when the pointer position determination unit 15 determines whether the pointer is located in the command display area, and when it is determined that the pointer is located in the command display area, the display information Since the display information stored in the storage unit 16 is referred to, the movable region display control unit 32a and the guide display control unit 33a that perform display control to display the movable region, the guide frame, and the command are provided. A movable region, a plurality of guide frames, and a plurality of commands can be displayed side by side. Thereby, it is possible to apply the display control device 100a to a user interface that continuously executes a plurality of arranged commands. For example, the display control device 100a can be applied to numeric input, and numeric input can be performed by a user's gesture operation.

In addition to the above, within the scope of the present invention, the present invention can be freely combined with each embodiment, modified any component of each embodiment, or omitted any component in each embodiment. Is possible.

The display control device according to the present invention can provide a display screen that allows a command to be executed only by a user's gesture operation, and is a wearable image display device such as a head-mounted display, and an image of a television, a projector, or the like. It can be applied to a display device.

1 input unit, 2 position information acquisition unit, 3 display control unit, 4 overlap determination unit, 5 incidence direction acquisition unit, 6 collation unit, 7 command database storage unit, 8 guide generation unit, 9 interlocking processing unit, 10 contact determination unit 11, command execution unit, 12, 12a cancellation determination unit, 13, 13a guide display cancellation unit, 14 display unit, 15 pointer position determination unit, 16 display information storage unit, 31 pointer display control unit, 32, 32a movable area display control Unit, 33, 33a guide display control unit, 100, 100a display control device, 101 processor, 102 memory, 200 head mounted display.

Claims

An incident direction acquisition unit that acquires a direction in which the pointer is incident on the movable area when a display position of the pointer that moves in accordance with a user operation overlaps with a display position of the movable area that guides the movement of the pointer. When,
A verification unit that acquires display information based on the incident direction acquired by the incident direction acquisition unit;
Based on the display information acquired by the verification unit, a guide generation unit that generates a guide region for guiding the pointer and the movable region;
A display control apparatus comprising: a display control unit that performs control to display the pointer, the movable region, and the guide region generated by the guide generation unit.
The guide generation unit is an area to be allocated to the guide area according to the incident direction based on the display information acquired by the verification unit, and is set in the area when the pointer enters the area. A command area where the command is executed, a guidance area for guiding the pointer and the movable area to the command area, and a release area for releasing the guidance of the pointer and the movable area to the command area The display control apparatus according to claim 1.
When the display position of the pointer and the display position of the movable area overlap, the display control unit includes an interlocking processing unit that interlocks the display of the pointer and the movable area,
The display control apparatus according to claim 1, wherein the display control unit performs control to display the pointer and the movable region in association with each other based on an interlock instruction from the interlock processing unit.
The collation unit acquires the display information that causes the movable region to expand and display in a specific direction with respect to the incident direction,
The display control apparatus according to claim 1, wherein the display control unit performs control to display the movable region by extending in the specific direction based on the display information acquired by the collation unit.
A contact determination unit that determines contact between the movable region and the guide region displayed based on the control of the display control unit;
When the contact determination unit determines that the movable region and the guide region are in contact, the command set in the command region assigned to the region in which the movable region and the guide region are in contact is executed. The display control apparatus according to claim 2, further comprising a command execution unit that performs the operation.
When the contact determination unit determines that the movable region is in contact with the guide region to which the release region is assigned, the interlock processing unit is configured to move the pointer in the allocation direction of the release region. The display control apparatus according to claim 3, wherein the interlocking of the display of the movable area is canceled.
A display for displaying an image;
An imaging unit for imaging a user operation;
A position information acquisition unit that analyzes an image indicating the user operation captured by the imaging unit and acquires a display position of a pointer that moves in accordance with the user operation and a display position of a movable region that guides the movement of the pointer When,
An incident direction acquisition unit that acquires a direction in which the pointer is incident on the movable region when the display position of the pointer acquired by the position information acquisition unit and the display position of the movable region overlap;
A verification unit that acquires display information based on the incident direction acquired by the incident direction acquisition unit;
Based on the display information acquired by the verification unit, a guide generation unit that generates a guide region for guiding the pointer and the movable region;
An image display apparatus comprising: a display control unit that performs control to display the pointer, the movable region, and the guide region generated by the guide generation unit on the display unit.