JP2016015078A - Display control device, display control method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a display control device that can change the display size of a display image to be operated freely as intended by a user.SOLUTION: A display control device includes: operation means for operating a display image according to the input of the amount of displacement of coordinates; and control means for controlling a display attribute of the display image and a conversion factor for converting the amount of displacement of coordinates into the amount of displacement of the display position of the display image according to the amount of displacement of coordinates input by the operation means.

Description

  The present invention relates to a display control device, a display control method, and a program.

  Conventionally, regarding an operation on an information processing apparatus such as a personal computer (hereinafter referred to as “PC”), an operation of moving a cursor to a desired position on a screen using a pointing device such as a mouse has been widely performed. For example, move the cursor over an icon displayed on the screen, select the icon by clicking the mouse, move the icon by dragging and dropping with the mouse, or specify the character input position with the cursor. ing.

  By the way, in the operation of moving the cursor with the mouse, the user often loses sight of the cursor. This is because, for example, in a large-screen and high-resolution display device such as horizontal 2048 pixels × vertical 2048 pixels, the displayed cursor is relatively small, and thus the visibility is deteriorated.

  On the other hand, for example, in Patent Document 1, not only the position of the cursor but also the moving speed is obtained from the amount of displacement of the mouse, and when the user wants to grasp the position of the cursor, the mouse is quickly moved as a cueing operation to the computer. A cursor display position control device that performs display emphasis processing such as enlarging a cursor image when an operation is performed and it is detected that the moving speed exceeds a predetermined speed threshold is disclosed.

JP-A-10-207441

  According to Patent Document 1, it is easy to find a cursor that has been lost. However, in Patent Document 1, the cursor size is merely enlarged to a certain size such as a size about twice the normal size, and the cursor size cannot be freely changed.

  The present invention has been made in view of such circumstances, and an object of the present invention is to provide a display control device that can freely change the display size of a display image to be operated as desired by the user. And

  In order to solve the above problems, a display control apparatus according to the present invention includes an operation unit that operates a display image according to an input of a coordinate displacement amount, and a coordinate displacement that is input according to the coordinate displacement amount that is input by the operation unit. And a control means for controlling a conversion coefficient for converting the amount into a displacement amount of the display position of the display image and a display attribute of the display image.

  According to the present invention, it is possible to freely change the display size of the display image to be operated as desired by the user.

It is operation | movement explanatory schematic using the personal computer as an example of the display control apparatus in embodiment of this invention. It is a schematic diagram which shows the example of a display screen which concerns on the display control process in embodiment of this invention. It is a figure which shows the hardware structural example of the display control apparatus in embodiment of this invention. It is a functional block diagram of a display control device in an embodiment of the present invention. It is a schematic diagram explaining an example of the display control process in the embodiment of the present invention. It is a flowchart which shows an example of the display control processing procedure in embodiment of this invention.

  A display control apparatus according to an embodiment of the present invention will be described below with reference to the drawings. In addition, in each figure, the same code | symbol is attached | subjected to the part which is the same or it corresponds, The duplication description is simplified thru | or abbreviate | omitted suitably. In the following description, a notebook personal computer (hereinafter referred to as “PC”) is used as an example in describing the display control apparatus according to the present embodiment. However, the present invention is not limited to this. PCs, multifunctional mobile phones (smartphones), PDAs (Personal Digital Assistants), mobile terminals such as mobile phones, televisions (TV) having information processing functions, or refrigerators having information processing functions, for example, refrigerators The display control device according to the present embodiment may be applied to household appliances such as a washing machine or a private car.

  In the present embodiment, for example, the operation contents used to change the size of a cursor for selecting a display image such as an icon displayed on a display screen such as a display, etc., are the operating states of the operating means used for the operation. Is recognized by a predetermined method. The operation means may be a human finger for executing a desired function by a so-called gesture operation, or may be a pointing device such as a mouse or a touch pad. In the present embodiment, a finger of a person who performs a gesture operation will be mainly described below as an example of an operation unit. Note that the recognized gesture is not limited to a finger and may be any gesture that uses at least a part of the body. For example, any gesture using the arms, legs, face, or the whole body may be recognized, and various processes may be realized based on the recognized gestures.

  An outline of a notebook PC (hereinafter simply referred to as “PC”) 1 which is an example of a display control apparatus according to the present embodiment will be described with reference to FIG. For example, the PC 1 includes a camera 11 for recognizing a gesture, a display 12, a keyboard 13, a touch pad 14, a mouse 15, and the like.

  For example, the camera 11 captures a gesture R as shown in FIG. 1 and captures the captured image in the apparatus. The PC 1 analyzes the captured captured image and executes a desired process based on the analysis result. In this figure, the camera 11 is provided above the display 12 of the PC 1. However, the present invention is not limited to this, and the camera 11 may be provided anywhere as long as it can capture a gesture or the like. In addition, the camera 11 may be rotated by a rotation mechanism (not shown) to the back side of the apparatus housing with the attachment position as a base point to photograph a gesture or the like on the back side.

  The keyboard 13, the touch pad 14, and the mouse 15 are pointing devices that input operation control signals generated by the keyboard 13, the touch pad 14, and the mouse 15 to a CPU (Central Processing Unit) mounted in the PC 1.

  As an index for explaining the three-dimensional space in FIG. 1, the horizontal direction is the X axis, the vertical direction is the Y axis, and the depth direction is the Z axis. The camera 11 in this embodiment is not only a camera that captures a planar image composed of the X axis and the Y axis, but also an IR camera that images the temperature distribution of an object, for example, in order to recognize the length in the Z axis direction. It is preferable to use in combination. Further, a 3D camera such as a so-called stereo camera may be used. Thus, the gesture can be accurately recognized regardless of the direction of the gesture.

  Next, an operation example of the display screen according to the display control process in the present embodiment will be described with reference to FIG. This figure has shown the example of the display screen displayed on the display 12 in FIG. 1, for example. An icon A, a selected image B, and a cursor C are displayed on the display 12.

  For example, the user moves the cursor C to the icon A or the selected image B by a gesture operation using fingers. At this time, for example, the user widens or narrows the distance between the index finger and the thumb. When the camera 11 captures such a gesture operation by the user, the PC 1 grasps the position of the hand and analyzes the captured image of the finger to recognize the distance between the index finger and the thumb, that is, the distance. This distance is, for example, a distance from the coordinate position of the thumb to the coordinate position of the index finger when the position of the thumb is set as the reference coordinate in the captured image. Then, the PC 1 performs processing for changing, for example, the size and moving speed of the cursor C according to the recognized interval size. Thereby, for example, as shown in the figure, when the icon A is selected, the cursor C is displayed in a small size, and when the selected image B is selected, the cursor C is displayed in a large size. Further, the moving speed of the cursor may be changed together with the size changing process or separately. For example, when selecting a display image that is relatively small with respect to the entire screen such as the icon A, the moving speed of the cursor is decreased, and a display image that is relatively large with respect to the entire screen such as the selection image B is selected. When selecting, increase the cursor movement speed. When selecting a display image with a small size, a careful operation is required. Therefore, it is preferable to move the cursor slowly. When selecting a display image with a large size, it is easy to select, so the cursor movement speed is fast. Is preferred.

  Next, a hardware configuration of the display control apparatus according to the present embodiment will be described with reference to FIG. The display control apparatus according to the present embodiment includes, for example, a CPU 21, a RAM (Random Access Memory) 22, a ROM (Read Only Memory) 23, an HDD (Hard Disk Drive) 24, an imaging unit 25, a display unit 26, and the like. The input unit 27 and the output unit 28 are provided.

  The CPU 21 performs overall control of the display control device. In addition, the CPU 21 controls each of the above-described devices in order to realize a process of recognizing the distance between the user's fingers from the recognized gesture and changing the cursor size according to the recognized distance. To do.

  For example, the RAM 22 may be used as a work area for the CPU 21 and as a temporary storage area for various data. The ROM 23 only needs to store various data necessary for control in addition to a control program such as a BIOS (Basic Input Output System) executed by the CPU 21.

  The HDD 24 is a large-capacity storage device that stores, for example, distance data between fingers that serves as a reference for gesture recognition. In the present embodiment, for example, the HDD 24 is built in the PC 1, but the HDD 24 may be externally attached. Further, distance data between fingers may be stored in an external medium such as an SD card, and the data may be read from the SD card or the like. Further, distance data between fingers is stored in an external PC or the like connected via a network (not shown), and is read into the PC 1 via the network when gesture recognition is necessary. For example, a cloud or the like It is good also as corresponding to.

  The photographing unit 25 is photographing means such as a digital camera that photographs a human finger like the gesture R shown in FIG. As described above, in this embodiment, since it is preferable to measure the distance in the three-axis direction of an object such as a gesture, for example, an image in the two-axis direction, that is, the XY axis plane in FIG. In addition to the camera, it is preferable to include an IR (infrared) camera that measures the distance in the depth direction of the object, that is, the distance in the Z-axis direction in FIG. In addition, as long as the imaging | photography part 25 can acquire 3D image | video, parallax cameras, such as a stereo camera, may be sufficient, for example. In the following description, cameras that obtain captured images for measuring the distances in the three-axis directions are collectively referred to as “3D cameras”.

  The display unit 26 is a display device such as a liquid crystal display or an organic EL display that displays a display image such as the icon A or the selection image B shown in FIG. 2, and corresponds to the display 12 shown in FIG. . In the present embodiment, for example, when the user interface such as the icon A is displayed on the display unit 26 and the icon A is an operation key for realizing a predetermined function mounted on the PC 1, the user displays While observing the icon A displayed on the unit 26, it is possible to experience a feeling as if the icon A was directly operated by a gesture operation.

  The input unit 27 is, for example, a pointing device such as the keyboard 13, the touch pad 14, and the mouse 15 illustrated in FIG. For example, when the mouse 15 is used as the operation means in the present invention, an outline of the action recognition process using the mouse 15 will be described. When the mouse 15 is moved on the table, for example, an encoder in the mouse generates a number of pulses corresponding to the amount of rotation of the ball provided on the lower surface of the mouse. A mouse interface (not shown) provided in the PC 1 counts this pulse, for example, converts the count number between them to digital data at regular time intervals, and transmits it to the CPU 21 as the coordinate displacement amount of the mouse. This coordinate displacement amount is a two-dimensional displacement amount and is represented by two components, ie, a coordinate displacement amount in the X direction corresponding to the horizontal and vertical directions of the screen and a coordinate displacement amount in the Y direction. When only one of the horizontal direction and the vertical direction is displaced as the coordinate displacement amount of the mouse, a single coordinate displacement amount in the X direction or the Y direction may be used. In the present embodiment, these coordinate displacement amounts are used as operation control signals for manipulating the display image, and these coordinate displacement amounts as the operation control signals are compared with distance data stored in, for example, the HDD 24 of the PC 1. To do. As a result, by determining whether the distance data calculated from these coordinate displacement amounts matches the stored distance data, or whether the distance data has become shorter or longer with reference to the stored distance data, Recognize 15 movement distances. The touch pad 14 can also function in the same manner as the mouse 15 except for its structure and pulse generation principle. The touch pad 14 may be either electrostatic or pressure sensitive. Further, a so-called touch panel superimposed on the display 12 or the like may be used instead of the touch pad 14.

  The output unit 28 is, for example, a sound output unit such as a speaker that outputs sound, or a light output unit such as an LED that outputs light. For example, the output unit 28 outputs sound, light, or the like in order to notify the user of the processing state such as the completion of processing or the processing being performed.

  Next, functional blocks of the display control apparatus in the present embodiment will be described with reference to FIG. The display control apparatus according to the present embodiment includes a display unit 26, an operation unit 31, and a control unit 32 as functional blocks. Since the display unit 26 has been described with reference to FIG. 3, the description thereof is omitted here.

  The operation unit 31 is an operation unit that operates the display image such as the cursor C illustrated in FIG. 2 according to the input of the coordinate displacement amount, such as the gesture and the mouse described above with reference to FIG. For convenience, the gesture is a pointing device that outputs coordinate information based on the position of the first part of the user in the image including the user's body acquired by the 3D camera.

The control unit 32 uses the conversion coefficient k for converting the coordinate displacement amount input according to the coordinate displacement amount input by the operation unit 31 into the displacement amount of the display position of the display image, and the display attribute p of the display image. For example, the function is executed by the CPU 21 described with reference to FIG. For example, the control unit 32 calculates the displacement amount of the display position using the following formula (1), and determines the size of the display image and the like using the following formula (2). The display position of the display image means, for example, a position where the cursor C shown in FIG. 2 is displayed.
[Displacement amount of display position] = k × [input coordinate displacement amount] (1)
[Size of display image] = p × [input coordinate displacement amount] (2)

  For example, when the gesture is the operation unit 31, the position of the thumb as the first part in the captured image of the gesture is set as the reference coordinate position. The control unit 32 controls the conversion coefficient and the display attribute according to the distance in the three-dimensional space with, for example, the index finger as a second part different from the thumb. The display attribute is, for example, a parameter for determining the size, color, shape, etc. of the cursor C shown in FIG.

  More specifically, the control unit 32 controls the conversion coefficient so that if the input coordinate displacement amount per unit time is small, the display position of the display image is displaced with respect to the same input coordinate displacement amount. Further, the control unit 32 controls the display attributes so that the display image becomes smaller when the input coordinate displacement amount per unit time is small. Details of the control by the control unit 32 will be described later with reference to FIG.

  Next, an example of the display control process in this embodiment will be described with reference to FIG. 5 together with the processes by the operation unit 31 and the control unit 32 shown in FIG. Hereinafter, the operation unit 31 will be described as a gesture using fingers.

  In FIG. 5, an example of a gesture using a finger photographed by the photographing unit 25 is shown in the upper part. From left to right in the figure, gestures that are “with your hands fully open”, gestures that are “with your hands slightly open”, and gestures that are “not completely but with your fingertips closed and open slightly” , A gesture which is “a state where the fingertips of the hand are closed” is shown. In addition, as “cursor size”, “large”, “medium”, “small”, and “minimal” are shown in order from the left in the figure corresponding to the distance between fingers. As for “cursor movement speed”, “fast” is shown on the left side of the figure and “slow” is shown on the right side of the figure. By showing.

  For example, when the distance between the thumb and the index finger is short, the control unit 32 controls the conversion coefficient so that the display position of the cursor C is displaced with respect to the same input coordinate displacement amount. For example, the control unit 32 controls the display attribute so that the display image becomes smaller when the distance between the thumb and the index finger is short.

  In the present embodiment, for example, the distance between the thumb and the index finger in the gesture “in a state where the hand is fully opened” is “1”, and the distance between the thumb and the index finger in the gesture in which the “hand is slightly opened” is The distance between the thumb and forefinger in a gesture of “0.5 to 0.8”, “not completely but with the fingertips of the hand closed and open open” is “0.2 to 0.4” The distance between the thumb and the index finger in the gesture “with the fingertip closed” is “0”, and the input coordinate displacement amount is shown. In addition, about these distances, it is an example and is not limited to the shown numerical value. Further, any numerical value is set as the distance, and therefore, it is not limited to a specific unit. Therefore, the unit notation is omitted. Furthermore, numerical values “0.1” and “0.9” are shown as gestures, and gestures in the middle of a change indicating the process of movement change of each gesture shown in the figure are not shown, but for convenience in the illustration. These are simply omitted, and are not excluded. That is, the control part 32 in this embodiment can recognize the distance between fingers linearly.

  For example, when the distance between the thumb and the index finger is short, the control unit 32 controls the conversion coefficient k so that the display position of the cursor C is displaced, for example, slowly moved. Specifically, as illustrated in FIG. 5, for example, when the distance between the thumb and the index finger is “0”, the control unit 32 controls the conversion coefficient k so that the moving speed of the cursor becomes slow. Although only “fast” and “slow” are described as the moving speeds, it is needless to say that “slightly fast”, “slightly slow”, etc. may be included between them.

  Further, as illustrated in FIG. 5, for example, when the distance between the thumb and the index finger is short, the control unit 32 controls the display attribute so that the size of the cursor C is reduced. In the present embodiment, for example, the change of the cursor size has been described. However, the present invention is not limited to this. For example, the color or shape of the cursor may be changed. Thereby, since the variation of the display mode of the displayed cursor increases, it becomes possible to realize an operation reflecting the user's free intention. It should be noted that the shape of the cursor may be changed to a circle, an arrow, a quadrangle, or other shapes.

  Next, the display control processing procedure in the present embodiment will be described with reference to FIG. 6 using the example of FIG. First, for example, when the user's finger is photographed by the photographing unit 25, the display control apparatus starts a gesture recognition process using the input of the photographed image as a key (step S1). Note that this is an example, and the trigger of the gesture recognition process is not limited to the input of the captured image, but the voice input using the microphone or the like can be performed using the operation input signal by the user via the input unit 27 such as the keyboard 13 as a key. Even if the input signal is used as a key, any other signal may be used so long as the apparatus can recognize some trigger using a device provided in the PC 1.

  Next, the control unit 32 recognizes, for example, the distance between the index finger and the thumb from the captured image (step S2). When the recognized distance is “0” (step S2, distance: 0), the control unit 32 controls the display attribute p so that the cursor size is changed to “small”, and the moving speed of the cursor is slow. The conversion coefficient k is controlled so as to be (step S3). In this embodiment, the control when the input coordinate displacement amount is small is described. However, the present invention is not limited to this, and if the input coordinate displacement amount is large, the control is performed for the same input coordinate displacement amount. It goes without saying that the conversion coefficient k may be controlled so that the display position of the display image is largely displaced, and the display attribute p may be controlled so that the display image becomes large.

  Each of the above-described embodiments is a preferred embodiment of the present invention, and various modifications can be made without departing from the scope of the present invention. For example, each process in the display control apparatus of the present embodiment described above can be executed using hardware, software, or a combined configuration of both.

  When executing processing using software, a program in which a processing sequence is recorded can be installed and executed in a storage unit such as a memory in a computer incorporated in dedicated hardware. . Alternatively, the program can be installed and executed on a general-purpose computer capable of executing various processes.

1 Notebook PC
11 Camera 12 Display 13 Keyboard 14 Touchpad 15 Mouse 21 CPU
22 RAM
23 ROM
24 HDD
25 Image capturing unit 26 Display unit 27 Input unit 28 Output unit 31 Operation unit 32 Control unit

Claims (8)

Display means for displaying a display image to be operated;
An operation means for operating the display image in response to an input of a coordinate displacement amount;
Control means for controlling the conversion coefficient for converting the input coordinate displacement amount into the displacement amount of the display position of the display image and the display attribute of the display image according to the coordinate displacement amount input by the operation means. When,
A display control apparatus comprising:   The control means controls the conversion coefficient such that when the input coordinate displacement amount per unit time is small, the display position of the display image is displaced with respect to the same input coordinate displacement amount. The display control apparatus according to claim 1.   The display control apparatus according to claim 1, wherein the control unit controls the display attribute so that the display image becomes small when the input coordinate displacement amount per unit time is small. The operation means is a pointing device that outputs coordinate information based on the position of the first part of the user in the image including the user's body acquired by the 3D camera,
The control means controls a conversion coefficient and a display attribute according to a distance in a three-dimensional space between the first part and a second part different from the first part. 4. The display control device according to any one of items 3.   The control means controls the conversion coefficient so that the display position of the display image is displaced with respect to the same input coordinate displacement amount when the distance between the first part and the second part is short. The display control device according to claim 4, characterized in that:   The display control apparatus according to claim 4, wherein the control unit controls display attributes so that a display image becomes smaller when the distance between the first part and the second part is short. A step of inputting a coordinate displacement amount in accordance with an operation by a predetermined operation means for a display image displayed on the predetermined display means;
Controlling a conversion coefficient for converting the input coordinate displacement amount into a displacement amount of a display position of the display image according to the input coordinate displacement amount, and a display attribute of the display image;
A display control method comprising: A process of inputting a coordinate displacement amount in accordance with an operation by a predetermined operation unit with respect to a display image displayed on the predetermined display unit;
A process for controlling a conversion coefficient for converting the input coordinate displacement amount into a displacement amount of a display position of the display image according to the input coordinate displacement amount, and a display attribute of the display image;
A program that causes a computer to execute.

Images