JP2022053121A - Information processing device and information processing method - Google Patents

Abstract

To provide an information processing device and an information processing method capable of improving the usability of a touch screen.SOLUTION: An information processing device includes a determination unit configured to determine whether a user is looking inside or outside a first area of a touch screen based on the user's line of sight, and a mode switching unit configured to switch an operation mode of the first area, based on the result of the determination by the determination unit, into a first mode for causing the first area to function so that an operation object displayed in the first area is operable by an operation input into the first area, or a second mode for causing the first area to function so that an operation object displayed in a second area, which is different from the first area, in the touch screen or a display device different from the touch screen, is operable by an operation input into the first area.SELECTED DRAWING: Figure 5

Description

The present invention relates to an information processing apparatus and an information processing method.

In recent years, various techniques have been proposed in which a predetermined area of a touch screen functions as a virtual touch pad (hereinafter, also referred to as "virtual touch pad"). By touching the virtual touchpad, the user can operate, for example, an object (for example, a mouse cursor) displayed outside the area that functions as the virtual touchpad.

The following Patent Document 1 discloses, for example, a technique for making a predetermined area on a touch screen function as a virtual touch pad when an input satisfying a predetermined condition (for example, touch or hovering) is performed on the touch screen. ing. Specifically, when the user inputs an input satisfying a predetermined condition to the touch screen, a menu related to the virtual touch pad is displayed. When the user selects the virtual touchpad mode from the menu, the virtual touchpad mode in which a predetermined area on the touch screen functions as a virtual touchpad is started. On the other hand, when the user makes an input to end the virtual touchpad mode, the virtual touchpad mode ends.

Japanese Unexamined Patent Publication No. 2017-21827

However, as in the technique of Patent Document 1, it is troublesome for the user to input an operation for starting or ending the virtual touchpad mode each time the virtual touchpad is used. Therefore, it is desired that the user can use the virtual touchpad without inputting an operation for using the virtual touchpad.

In view of the above problems, an object of the present invention is to provide an information processing apparatus and an information processing method capable of improving the usability of a touch screen.

In order to solve the above-mentioned problems, in the information processing apparatus according to one aspect of the present invention, whether the user is looking inside or outside the first area of the touch screen based on the user's line of sight. The operation mode of the first area is operated by the operation of inputting the operation target displayed in the first area to the first area based on the determination unit for determining the above and the result of the determination by the determination unit. A first mode that allows the first area to function, or an operation displayed in the second area of the touch screen or a display device that is different from the first area and different from the touch screen. A mode switching unit for switching the target to a second mode in which the first area is operably functioned by an operation input to the first area is provided.

In the information processing method according to one aspect of the present invention, the determination unit determines whether the user is looking inside or outside the first region of the touch screen based on the user's line of sight. , The mode switching unit can operate the operation mode of the first area based on the result of the determination by the determination unit by the operation of inputting the operation target displayed in the first area to the first area. The operation target displayed in the first mode in which the first area is made to function, or in the second area of the touch screen or a display device different from the first area but different from the first area. Includes switching to a second mode in which the first area is operably functioned by an operation input to the first area.

The information processing device according to one aspect of the present invention is an information processing device including a mode switching unit for switching the operation mode of the first region of the touch screen, and the mode switching unit is an operation target in the first region. Is displayed, the operation target displayed in the first area can be operated by the operation input to the first area, and the mode is switched to the first mode in which the first area functions. When the operation target is displayed in at least one second area of the touch screen or a display device different from the touch screen, which is a region different from the first region, an input is made in the first region. By the operation performed, the operation target displayed in the second area is switched to the second mode in which the first area is operated so that the operation target can be operated.

The information processing method according to one aspect of the present invention is an information processing method including the mode switching unit switching the operation mode of the first region on the touch screen, and the mode switching unit is the first region. In the first mode in which the operation target displayed in the first area can be operated by the operation input to the first area when the operation target is displayed in the first area. When the operation target is displayed in at least one second area of the touch screen or a display device different from the touch screen, which is a region different from the first region, the first region is switched. The operation target displayed in the second area is switched to the second mode in which the operation target displayed in the second area can be operated by the operation input to the area.

According to the present invention, the usability of the touch screen can be improved.

It is a figure which shows an example of the external configuration of the information processing apparatus which concerns on this embodiment. It is a figure which shows an example of the operation in the direct operation mode which concerns on this embodiment. It is a figure which shows an example of the operation in the indirect operation mode which concerns on this embodiment. It is a figure which shows an example of the hardware composition of the information processing apparatus which concerns on this embodiment. It is a figure which shows an example of the functional structure of the information processing apparatus which concerns on this embodiment. It is a figure which shows an example of the case where it is determined that the user is directly looking at the operation area R1 based on the viewpoint position and the operation position of the user according to this embodiment. It is a figure which shows an example of the case where it is determined that the user is looking at the indirect operation area R2 based on the viewpoint position and the operation position of the user according to this embodiment. It is a flowchart which shows the flow of processing when the direct operation mode is set in the direct operation area which concerns on this embodiment. It is a flowchart which shows the flow of processing when the indirect operation mode is set in the direct operation area which concerns on this embodiment.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings.

<1. External configuration>
First, with reference to FIG. 1, the external configuration of the information processing apparatus according to the present embodiment will be described. FIG. 1 is a diagram showing an example of an external configuration of the information processing apparatus according to the present embodiment. FIG. 1 shows a dual display device 1 as an example of an information processing device. The dual display device 1 according to the present embodiment is a notebook PC (Personal Computer) type terminal having two displays (Dual Display) and one of the two displays is at least a touch screen.

As shown in FIG. 1, the dual display device 1 includes a housing 2A and a housing 2B. The shapes of the housing 2A and the housing 2B are both substantially rectangular parallelepipeds. The housing 2A and the housing 2B are connected by a connecting portion 3 at their respective ends. The connecting portion 3 is, for example, a hinge and supports the housing 2A and the housing 2B so as to be openable and closable.

The housing 2A includes a touch screen 10A, and the housing 2B includes a touch screen 10B. In the following description, when the touch screens 10A and 10B are distinguished, either A or B is added to the end of the reference numeral, and when the touch screens 10A and 10B are not distinguished, A and B are omitted. Further, in the following, the screen display on the touch screen 10 is an example of a horizontal screen display in which one of the two long sides constituting the touch screen 10 is in the upper direction of the screen display and the other is in the lower direction of the screen display. Will be explained.

The touch screen according to the present embodiment is an input / output device including a display screen capable of displaying various information and a touch panel capable of detecting an operation (for example, a touch operation) on the touch screen. The user can operate the operation target by directly touching the operation target displayed on the touch screen 10. For example, the user can directly touch the icon displayed on the touch screen 10 to start an application or open a folder or file.

Further, on the touch screen 10, input by gesture operation is also possible. For example, when two fingers are brought into contact with the surface of the touch screen 10 and a gesture of sliding the two fingers in a direction parallel to the surface is performed, the screen is scrolled. Further, when the surface of the touch screen 10 is brought into contact with three fingers and a gesture of swiping the three fingers to the left or right is performed, the display of the window or the display of the application is switched. Further, when two fingers are brought into contact with the surface of the touch screen 10 and a gesture of releasing the two fingers (pinch out) is performed, the display is enlarged, and when a gesture of bringing the two fingers closer (pinch in) is performed, the display is displayed. It will be reduced. In the gesture operation, the number of fingers in contact with the touch screen and the type of gesture are not limited to these examples. Further, the combination of the number of fingers, the type of gesture, and the action content by the gesture operation (hereinafter, also referred to as “gesture definition”) is not limited to such an example. The gesture definition may be set according to the OS (operating system: Operating System), the type of application, and the operation mode described later.

In the dual display device 1 according to the present embodiment, an operation mode for controlling the operation in a predetermined area of the touch screen 10 is set. The predetermined area is, for example, an area on the touch screen 10 where an operation by the user is input and an area where the operation target can be displayed (first area). The operation target displayed in the first area is operated according to the operation input to the first area by the user. That is, the operation target displayed in the first area is directly operated. Hereinafter, the first area is also referred to as a “direct operation area”. In the present embodiment, the user treats the touch screen 10A of the housing 2A as a device for operation input. Therefore, in the present embodiment, the area R1 of the touch screen 10A is assumed to be a direct operation area. Hereinafter, the area R1 is also referred to as a direct operation area R1.
The operation mode includes, for example, a direct operation mode (first mode) and an indirect operation mode (second mode).

The direct operation mode is a mode in which the direct operation area in the touch panel area where the touch panel of the touch screen 10 is formed functions as a touch screen. As a result, the direct operation mode makes it possible to directly operate the operation target displayed in the direct operation area. The direct operation area that functions as a touch screen is, for example, the entire area of the touch panel area.

When the direct operation mode is set as the operation mode of the touch screen 10A, the direct operation area R1 functions as the touch screen. In the direct operation mode of the present embodiment, it is possible to operate the operation target displayed in the direct operation area R1 by operating the direct operation area R1. Therefore, the user can directly operate the operation target displayed in the direct operation area R1 by performing the operation on the direct operation area R1 of the touch screen 10A in which the direct operation mode is set.

The indirect operation mode is a mode in which the direct operation area in the touch panel area where the touch panel of the touch screen 10 is formed virtually functions as a normal touch pad or a high-precision touch pad (PTP: Precision Touch Pad). Hereinafter, "Virtual Touch Pad (VTP)" is used as a general term for a virtual normal touch pad and a virtual high-precision touch pad. In the present embodiment, VTP is a virtual high-precision touch pad. An example will be described. The indirect operation mode enables the operation target displayed in an area different from the direct operation area to be indirectly operated by the direct operation area functioning as a VTP. The direct operation area functions as a VTP. The operation area is, for example, the entire area of the touch pad area.

When the indirect operation mode is set as the operation mode of the touch screen 10A, the direct operation area R1 functions as VTP. In the indirect operation mode of the present embodiment, the operation of the operation target displayed in the area different from the direct operation area R1 is enabled by the operation on the direct operation area R1.

Here, the region different from the direct operation region R1 is, for example, at least one region (second region) in the touch screen 10 or a display device different from the touch screen 10 which is a region in which the operation target can be displayed. .. The operation target displayed in the second area is operated according to the operation directly input to the operation area by the user. That is, the operation target displayed in the second area is indirectly operated. Hereinafter, the second area is also referred to as an “indirect operation area”. In the present embodiment, the user treats the touch screen 10B of the housing 2B as a device on which an operation target to be indirectly operated is displayed. Therefore, in the present embodiment, the area R2 of the touch screen 10B is assumed to be an indirect operation area. Hereinafter, the area R2 is also referred to as an indirect operation area R2.

The VTP according to the present embodiment is a touch pad capable of performing gesture operations. In VTP, in addition to operating the mouse cursor, gesture operations similar to those on the touch screen are also possible. In the indirect operation mode, the touch screen 10 may function as a mere touch pad.

The switching of the operation mode is controlled by, for example, the control program (firmware) of the touch screen 10. The control program is firmware executed by an EC (embedded controller) described later, and is also referred to as "EC firmware" below. The gesture operation by the user in each mode is recognized by the OS. For example, the OS recognizes the gesture operation based on the signal transmitted from the EC firmware according to the set operation mode.

When the touch to the touch screen 10 is detected, the EC firmware generates a signal corresponding to the operation mode set in the touch screen 10 in which the touch is detected and outputs the signal to the OS. At this time, the EC firmware generates a signal including definition information according to the operation mode of the touch screen 10 and outputs the signal to the OS.

Here, the definition information is information including information about hardware (hereinafter, also referred to as "hardware information"). The definition information may include various hardware information, for example, touch screen hardware information and PTP hardware information. The hardware information includes, for example, information such as the type and size of the hardware. The information included in the definition information is not limited to this example. For example, the hardware information may include information related to virtual display such as size information indicating a virtual display size, position information indicating a virtual display position, and display layout information indicating a layout including other display targets. Based on the information related to the virtual display, the virtual hardware can be displayed on the touch screen 10 by the resident program on the OS instructed by the EC. In this embodiment, an example in which the virtual hardware is not displayed on the touch screen 10 will be described.

When a touch is detected on the touch screen 10 in which the direct operation mode is set, the EC firmware generates a touch screen signal (first signal) indicating that the touch screen 10 has been touched as a touch screen. Output to the OS. The touch screen signal includes, for example, touch screen definition information and coordinate information indicating a touch position.
When a touch is detected on the touch screen 10 in which the indirect operation mode is set, the EC firmware generates a VTP signal (second signal) indicating that the touch screen 10 is touched as VTP and outputs it to the OS. do. The VTP signal includes, for example, PTP definition information and coordinate information indicating a touch position.

The OS that received the signal from the EC firmware recognizes that the hardware indicated by the definition information included in the received signal has been touched. At this time, the OS recognizes that the hardware indicated by the definition information is connected to the dual display device 1. For example, when receiving a touch screen signal, the OS recognizes that the touch screen 10 is connected to the dual display device 1. On the other hand, when the VTP signal is received, the OS recognizes that the VTP is connected to the dual display device 1. In this embodiment, the touch screen 10 is physically connected to the dual display device 1. However, VTP is not physically connected to the dual display device 1. That is, the OS can recognize the hardware that is not physically connected to the dual display device 1 based on the definition information.

Then, the OS operates according to the coordinate position where the touch is detected by the recognized hardware. When the touch screen 10 is recognized, the OS performs an operation related to the touch screen 10. On the other hand, when VTP is recognized, the OS performs an operation related to VTP. In this way, the EC firmware can control the operation of the OS by generating a signal corresponding to the operation mode set on the touch screen 10 in which the touch is detected and outputting the signal to the OS.

Switching between the direct operation mode and the indirect operation mode is performed based on the user's line of sight. The user's line of sight is detected by, for example, a sensor device 33 provided in the housing 2B. The dual display device 1 determines an object (for example, direct operation area R1 or indirect operation area R2) that the user is looking at from the detected line of sight of the user. Then, the dual display device 1 switches the operation mode to the direct operation mode or the indirect operation mode based on the target indicated by the determination result.

Here, an example of operation in the direct operation mode will be described with reference to FIG. FIG. 2 is a diagram showing an example of operation in the direct operation mode according to the present embodiment. In FIG. 2, when it is determined that the operation mode of the touch screen 10A is the direct operation mode and the user U1 is looking at the direct operation area R1 of the touch screen 10A, the user U1 with respect to the direct operation area R1. The operation when the pinch-out operation is performed is shown. The left figure of FIG. 2 shows the state before the pinch out, and the right figure of FIG. 2 shows the state after the pinch out.
As shown in the left figure of FIG. 2, the text "Hello" is displayed in the direct operation area R1 of the touch screen 10A. It is assumed that the user U1 performs a pinch-out operation in the vicinity of the display position of the text in the direct operation area R1. In this case, as shown in the right figure of FIG. 2, "Hello" directly displayed in the operation area R1 is enlarged and displayed.

Further, an example of the operation in the indirect operation mode will be described with reference to FIG. FIG. 3 is a diagram showing an example of operation in the indirect operation mode according to the present embodiment. In FIG. 3, when it is determined that the operation mode of the touch screen 10A is the indirect operation mode and the user U1 is looking at the indirect operation area R2 of the touch screen 10B, the user U1 with respect to the direct operation area R1. The operation when the pinch-in operation is performed is shown. The left figure of FIG. 3 shows the state before pinch-in, and the right figure of FIG. 3 shows the state after pinch-in.
As shown in the left figure of FIG. 3, the text "Hello" is displayed in the indirect operation area R2 of the touch screen 10B. It is assumed that the user U1 directly performs a pinch-in operation in the operation area R1. The position where the user U1 performs the pinch-in operation in the direct operation area R1 is, for example, near the position corresponding to the position where the text is displayed in the indirect operation area R2. In this case, as shown in the right figure of FIG. 3, "Hello" displayed in the indirect operation area R2 is reduced and displayed.

<2. Hardware configuration>
The outline of the embodiment of the present invention has been described above. Subsequently, with reference to FIG. 4, the hardware configuration of the dual display device 1 according to the present embodiment will be described. FIG. 4 is a diagram showing an example of the hardware configuration of the dual display device 1 according to the present embodiment.

As shown in FIG. 4, the dual display device 1 includes a touch screen 10A, a touch screen 10B, a CPU (Central Processing Unit) 15, a main memory 16, a GPU (Graphic Processing Unit) 17, and a chipset 21. Further, the dual display device 1 includes a BIOS (Basic Input Output System) memory 22, an HDD (Hard Disk Drive) 23, an audio system 24, a communication device 25, an EC31, an input device 32, a sensor device 33, and a power supply circuit 34. ..

The touch screen 10 includes a display screen 11 and a touch panel 12. The touch screen 10 displays various information corresponding to the display data converted into video signals on the display screen 11, and also detects touch by an operation medium such as a user's finger or a pen and the proximity of the operation medium by the touch panel 12. By doing so, the operation input by the operation medium is accepted.

The display screen 11 is, for example, a display device such as an OLED display (organic EL display). The display screen 11 may be configured to be foldable (foldable).

The touch panel 12 is arranged so as to overlap the display surface of the display screen 11. The touch panel 12 detects an operation position (specifically, a touch position). The touch panel 12 may be integrally configured with the display screen 11 and may be foldable (foldable) in the same manner as the display screen 11.

The CPU 15 executes various arithmetic processes by program control and controls the entire dual display device 1.

The main memory 16 is a writable memory used as a read area for the execution program of the CPU 15 or as a work area for writing the processing data of the execution program. The main memory 16 is composed of, for example, a plurality of DRAM (Dynamic Random Access Memory) chips. This execution program includes an OS, various drivers for operating peripheral devices by hardware, various services / utilities, application programs, and the like.

The GPU 17 executes image processing based on the control of the CPU 15 to generate display data. The GPU 17 is connected to the display screen 11 and outputs the generated display data to the display screen 11.

The chip set 21 includes USB (Universal Serial Bus), serial ATA (AT Attachment), SPI (Serial Peripheral Interface) bus, PCI (Peripheral Component Interconnect) bus, PCI-Express bus, LPC (Low Pin Count) bus, and the like. It has a controller and multiple devices are connected. In FIG. 4, as an example of the device, the BIOS memory 22, the HDD 23, the audio system 24, the communication device 25, and the EC 31 are connected to the chipset 21.

The BIOS memory 22 is composed of, for example, an electrically rewritable non-volatile memory such as an EEPROM (Electrically Erasable Programmable Read Only Memory) or a flash ROM (Read Only Memory). The BIOS memory 22 stores the BIOS, the system firmware for controlling the EC31 and the like, and the like. The system firmware is the firmware executed by the CPU 15, and is different from the EC firmware executed by the EC 31.

The HDD (Hard Disk Drive) 23 (an example of a non-volatile storage device) stores an OS, various drivers, various services / utilities, application programs, and various data.

The audio system 24 records, reproduces, and outputs sound data. For example, a microphone or a speaker is connected to the audio system 24.

The communication device 25 is communicably connected to another device via a wireless or wired communication network, and transmits and receives various data. For example, the communication device 25 includes a wired LAN interface such as Ethernet (registered trademark) and a wireless LAN interface such as Wi-Fi (registered trademark). The communication device 25 may be configured to include a USB interface and a Bluetooth (registered trademark) interface.

The EC31 is a one-chip microcomputer that monitors and controls various devices (peripheral devices, sensors, etc.) regardless of the system state of the dual display device 1. The EC 31 includes a CPU (Read only Memory) and a RAM (Random Access memory) (not shown). The EC 31 operates independently of the CPU 15 and mainly functions as a control unit that manages the internal operating environment of the dual display device 1. The EC 31 reads a control program (EC firmware) stored in ROM in advance, executes a process instructed by various instructions described in the read control program, and realizes various functions. Further, the EC 31 includes a plurality of channels of A / D input terminals, D / A output terminals, timers, digital input / output terminals, and the like. For example, an input device 32, a sensor device 33, a power supply circuit 34, and the like are connected to the EC 31 via their input / output terminals, and the EC 31 controls these operations.

The input device 32 is, for example, an input device that inputs an input such as a power switch or a function switch.

The sensor device 33 includes, for example, a sensor device capable of detecting a user's line of sight and a viewpoint position. The sensor device is equipped with, for example, an image pickup device such as a camera, an infrared LED (light emitting diode) light source capable of outputting near infrared rays, and a program including an algorithm for detecting the line of sight and the viewpoint position. The algorithm detects the user's line of sight based on the image of the user's eye captured by the image pickup device. The image of the user's eye includes a reflection pattern when the light output by the infrared LED light source is reflected by the user's eye. Further, the algorithm detects the user's viewpoint position on the touch screen 10 based on the detected line of sight.

The power supply circuit 34 includes, for example, a DC / DC converter, a charge / discharge unit, a battery unit, an AC / DC adapter, and the DC voltage supplied from the AC / DC adapter or the battery unit is used for the dual display device 1. Convert to multiple voltages required for operation. Further, the power supply circuit 34 supplies electric power to each part of the dual display device 1 based on the control from the EC 31.

<3. Function configuration>
The example of the hardware configuration of the dual display device 1 according to this embodiment has been described above. Subsequently, an example of the functional configuration of the dual display device 1 according to the present embodiment will be described with reference to FIG. FIG. 5 is a block diagram showing an example of the functional configuration of the dual display device 1 according to the present embodiment.

As shown in FIG. 5, the dual display device 1 includes a communication unit 110, an input unit 120, a control unit 130, a storage unit 140, and a display unit 150.

(1) Communication unit 110
The communication unit 110 has a function of transmitting and receiving various information. The function is realized by, for example, the communication device 25 described with reference to FIG.

(2) Input unit 120
The input unit 120 has a function of accepting input of various information. The function is realized by, for example, the touch screen 10, the input device 32, the sensor device 33, and the like described with reference to FIG. The input unit 120 receives, for example, input of information input by the user to the touch screen 10. Further, the input unit 120 receives input of operation position information indicating a position (operation position) in which the user has operated on the touch screen 10 (specifically, the direct operation area R1). Further, the input unit 120 accepts input of information to be input by the user by operating the input device 32. Further, the input unit 120 receives input of information indicating the user's line of sight and viewpoint position detected by the sensor device 33 (hereinafter, also referred to as "line of sight information"). The input unit 120 outputs the information that has received the input to the control unit 130.

(3) Control unit 130
The control unit 130 has a function of controlling the overall operation of the dual display device 1. The function is realized by, for example, the CPU 15 and the chipset 21 described with reference to FIG. As shown in FIG. 5, the control unit 130 includes a line-of-sight information acquisition unit 131, an operation position information acquisition unit 132, a determination unit 133, a mode switching unit 134, a signal processing unit 135, and a display processing unit 136.

(3-1) Line-of-sight information acquisition unit 131
The line-of-sight information acquisition unit 131 has a function of acquiring line-of-sight information. For example, the line-of-sight information acquisition unit 131 acquires the line-of-sight information received by the input unit 120. The line-of-sight information acquisition unit 131 outputs the acquired line-of-sight information to the determination unit 133.

(3-2) Operation position information acquisition unit 132
The operation position information acquisition unit 132 has a function of acquiring operation position information. For example, the operation position information acquisition unit 132 acquires the operation position information for which the input unit 120 has received the input. The operation position information acquisition unit 132 outputs the acquired operation position information to the determination unit 133.

(3-3) Judgment unit 133
The determination unit 133 has a function of determining an object that the user is looking at based on the line-of-sight information. The determination unit 133 outputs the determination result to the mode switching unit 134.
For example, the determination unit 133 determines whether the user is looking in the direct operation area R1 or the indirect operation area R2 based on the information indicating the user's line of sight among the line-of-sight information input from the line-of-sight information acquisition unit 131. Is determined. For example, the determination unit 133 determines the area the user is looking at based on the angle and direction of the line of sight. Further, the determination unit 133 is based on the information indicating the viewpoint position acquired based on the line of sight among the line-of-sight information input from the line-of-sight information acquisition unit 131, and the user can use either the direct operation area R1 or the indirect operation area R2. You may determine if you are looking inside. Specifically, when the viewpoint position is on the direct operation area R1, the determination unit 133 determines that the user is looking inside the area of the direct operation area R1, and when the viewpoint position is on the indirect operation area R2. It is determined that the user is looking inside the indirect operation area R2.

When the area at the time of the previous determination and the area at the time of the current determination are different, the determination unit 133 determines whether or not the line of sight of the user is detected in the area at the time of the current determination for a predetermined time (for example, 100 msec) or more. As a result, the determination unit 133 can determine whether or not the user has intentionally moved his / her line of sight from the area at the time of the previous determination to the area at the time of the current determination. Here, the movement of the line of sight unintentional by the user may occur, for example, when the line of sight is blurred due to the unconscious movement of the eyeball.
For example, when the user's line of sight is detected in the area at the time of this determination for a predetermined time or longer, the determination unit 133 determines that the user has intentionally moved the line of sight from the area at the time of the previous determination to the area at the time of this determination. do. On the other hand, if the user's line of sight is not detected in the area at the time of this determination for a predetermined time or longer, it is determined that the movement of the user's line of sight from the area at the time of the previous determination to the area at the time of this determination was not intentional.

The mode switching unit 134, which will be described later, switches the operation mode based on the determination result. For example, when the determination result indicates that the user has intentionally moved the line of sight, the mode switching unit 134 executes switching of the operation mode. On the other hand, when the determination result indicates that the user has not intentionally moved the line of sight, the mode switching unit 134 does not execute the operation mode switching.
This makes it possible to prevent the user from unintentionally switching the operation mode when the user unintentionally moves the line of sight.

When determining the area that the user is looking at, the determination unit 133 determines whether or not it is clear whether the user is looking in the direct operation area R1 or the indirect operation area R2. The position ahead of the user's line of sight is not necessarily detected at a point and can be detected in a predetermined area. When the entire predetermined area is contained in either the direct operation area R1 or the indirect operation area R2, the determination unit 133 allows the user to enter either the direct operation area R1 or the indirect operation area R2. You can clearly judge whether you are watching. However, when the user's line of sight is near the boundary between the direct operation area R1 and the indirect operation area R2, the area indicating the position ahead of the user's line of sight is partially a region of both the direct operation area R1 and the indirect operation area R2. May be included in. In this case, it is difficult for the determination unit 133 to clearly determine whether the user is looking in the direct operation area R1 or the indirect operation area R2.

When it is difficult to clearly determine whether the user is looking in the direct operation area R1 or the indirect operation area R2, the determination unit 133 may use the determination unit 133, for example, based on the user's viewpoint position and operation position. Determines whether is looking in the direct operation area R1 or the indirect operation area R2. Here, the operation position is information included in the operation position information input from the operation position information acquisition unit 132. Specifically, the determination unit 133 determines whether the user is looking in the direct operation area R1 or the indirect operation area R2 based on whether or not the viewpoint position and the operation position substantially match. Here, the substantially match refers to a state in which at least a part of the operation position is included in the predetermined area, for example, when the position ahead of the user's line of sight is indicated in the predetermined area. Further, substantially the same means that when the position ahead of the user's line of sight is indicated by a point, the position of the point and the operation position may be in agreement.

When the viewpoint position and the operation position substantially match, the determination unit 133 determines that the user is directly looking inside the operation area R1. The fact that the viewpoint position substantially coincides with the operation position means that the user is looking at the area where the operation is input. In the present embodiment, the area where the user inputs an operation is the direct operation area R1. Therefore, the determination unit 133 can determine that the user is directly looking inside the operation area R1 when the viewpoint position and the operation position substantially match. On the other hand, when the viewpoint position and the operation position do not substantially match, the determination unit 133 determines that the user is looking inside the indirect operation area R2. The fact that the viewpoint position and the operation position do not substantially match means that the user is not looking in the area of the direct operation area R1 in which the operation is input. Therefore, the determination unit 133 can determine that the user is looking inside the indirect operation area R2 when the viewpoint position and the operation position do not substantially match. As described above, the determination unit 133 allows the user to see which area based on the viewpoint position and the operation position even when the user's line of sight is near the boundary between the direct operation area R1 and the indirect operation area R2. It can be clearly determined whether or not it is.

Here, with reference to FIGS. 6 and 7, an example of determining the area viewed by the user based on the viewpoint position and the operation position of the user will be described. FIG. 6 is a diagram showing an example of a case where it is determined that the user is directly looking in the area of the operation area R1 based on the viewpoint position and the operation position of the user according to the present embodiment. FIG. 7 is a diagram showing an example of a case where it is determined that the user is looking inside the indirect operation area R2 based on the user's viewpoint position and operation position according to the present embodiment. In the examples shown in FIGS. 6 and 7, an example in which the line-of-sight position VP is a region is shown.

In the example shown in FIG. 6, the line-of-sight position VP is near the boundary between the direct operation region R1 on the touch screen 10A and the indirect operation region R2 on the touch screen 10B. The operation position OP is included in the area of the line-of-sight position VP. Therefore, the determination unit 133 determines that the user is directly looking inside the operation area R1.
Similarly, in the example shown in FIG. 7, the line-of-sight position VP is near the boundary between the direct operation region R1 on the touch screen 10A and the indirect operation region R2 on the touch screen 10B. However, the operation position OP is located outside the region of the line-of-sight position VP. Therefore, the determination unit 133 determines that the user is looking inside the indirect operation area R2.

(3-4) Mode switching unit 134
The mode switching unit 134 has a function of switching the operation mode. For example, the mode switching unit 134 switches the operation mode of the direct operation area R1 to the direct operation mode or the indirect operation mode based on the result of the determination by the determination unit 133.

The mode switching unit 134 switches the operation mode to the indirect operation mode when the determination unit 133 determines that the user is looking inside the indirect operation area R2 when the operation mode is the direct operation mode. Further, the mode switching unit 134 switches the operation mode to the direct operation mode when the determination unit 133 determines that the user is looking in the area of the direct operation area R1 when the operation mode is the indirect operation mode. With such a configuration, the mode switching unit 134 can switch the operation mode of the direct operation area R1 to the direct operation mode or the indirect operation mode according to the area viewed by the user. As a result, the user does not need to input an operation for switching the operation mode to the dual display device 1. That is, the dual display device 1 can reduce the time and effort of the operation when the user uses VTP.

As described above, the mode switching unit 134 may control the switching of the operation mode based on whether or not the user intentionally moves the line of sight. For example, the mode switching unit 134 executes operation mode switching when it is determined by the determination unit 133 that the user has intentionally moved the line of sight. On the other hand, when the determination unit 133 determines that the user has not intentionally moved the line of sight, the mode switching unit 134 does not execute the operation mode switching.
This makes it possible to prevent the user from unintentionally switching the operation mode when the user unintentionally moves the line of sight.

The mode switching unit 134 may perform control not to execute the switching of the operation mode. For example, when the user inputs to the indirect operation area R2 using the screen keyboard activated in the direct operation area R1, the user's line of sight frequently moves between the direct operation area R1 and the indirect operation area R2. obtain. At this time, for example, if the mode switching unit 134 switches the operation mode to the indirect operation mode, the direct operation area R1 functions as VTP. Therefore, the user cannot input with the on-screen keyboard. Therefore, the mode switching unit 134 does not execute the operation mode switching when the screen keyboard is activated in the direct operation area R1. As a result, the mode switching unit 134 can prevent the user from becoming unable to use the screen keyboard due to the switching of the operation mode.

(3-5) Signal processing unit 135
The signal processing unit 135 has a function of generating a signal for controlling the operation of the OS and outputting it to the OS. For example, the signal processing unit 135 directly generates a signal including definition information according to the operation mode of the operation area R1 and outputs the signal to the OS.

When the operation mode of the direct operation area R1 is the direct operation mode, the signal processing unit 135 generates a touch screen signal including the definition information of the touch screen and outputs it to the OS. As a result, the signal processing unit 135 can make the OS recognize that the touch screen 10 is operated as a touch screen and perform an operation related to the touch screen.

On the other hand, when the operation mode of the direct operation area R1 is the indirect operation mode, the signal processing unit 135 generates a VTP signal including PTP definition information and outputs it to the OS. As a result, the signal processing unit 135 can make the OS recognize that the touch screen 10 is operated as VTP and perform an operation related to VTP.

At the time of signal generation, the signal processing unit 135 changes the signal to be output to the OS, for example, by changing the definition information included in the signal. Specifically, it is assumed that the operation mode is switched to the indirect operation mode after the touch screen signal is output in the direct operation mode. In this case, the signal processing unit 135 changes the touch screen definition information included in the touch screen signal to the PTP definition information. As a result, the touch screen signal is changed to a VTP signal. On the other hand, it is assumed that the operation mode is switched to the direct operation mode after the VTP signal is output in the indirect operation mode. In this case, the signal processing unit 135 changes the PTP definition information included in the VTP signal to the touch screen definition information. As a result, the VTP signal is changed to a touch screen signal.

The touch screen signal and the VTP signal are generated by the EC firmware of the touch screen 10 and output to the OS. As a result, the operation of the touch screen 10 in the OS is controlled by the EC firmware of the touch screen 10.

(3-6) Display processing unit 136
The display processing unit 136 has a function of controlling the display on the display unit 150 when the operation mode is switched.

For example, the display processing unit 136 displays (that is, the user sees) whether the operation target displayed in the direct operation area R1 or the indirect operation area R2 can be operated by the user according to the operation mode after switching. Controls the display) that indicates the area in which it is located. As an example, the display processing unit 136 causes the display unit 150 to display a frame indicating the area of the direct operation area R1 or the indirect operation area R2 that the user is viewing. The frame is, for example, a frame showing the shape of a region. The shape, color, thickness, etc. of the frame are not particularly limited.
When the operation mode is switched from the indirect operation mode to the direct operation mode, the display processing unit 136 causes the display unit 150 to display the frame of the direct operation area R1. On the other hand, when the operation mode is switched from the direct operation mode to the indirect operation mode, the display processing unit 136 causes the display unit 150 to display the frame of the indirect operation area R2.

Further, for example, the display processing unit 136 controls the display of the cursor according to the operation mode after switching. As an example, when the operation mode is switched from the direct operation mode to the indirect operation mode, the display processing unit 136 moves the cursor displayed in the direct operation area R1 to the indirect operation area R2 and causes the display unit 150 to display the cursor. .. On the other hand, when the operation mode is switched from the indirect operation mode to the direct operation mode, the display processing unit 136 moves the cursor displayed in the indirect operation area R2 to the direct operation area R1 and causes the display unit 150 to display the cursor. As a result, the user can smoothly start the operation after the mode is switched.
The position where the cursor is displayed after the mode switching is, for example, the center of the direct operation area R1 or the indirect operation area R2. The position where the cursor is displayed after the mode is switched is not limited to this example.

(4) Storage unit 140
The storage unit 140 has a function of storing various types of information. The storage unit 140 includes a main memory 16, a BIOS memory 22, an HDD 23, a ROM and a RAM included in the EC 31, and the like. The storage unit 140 stores, for example, software such as an OS and various applications, various firmware such as system firmware and EC firmware, and definition information.

(5) Display unit 150
The display unit 150 has a function of displaying various information.
The display unit 150 is realized by, for example, a display provided as hardware in the dual display device 1.

<4. Processing flow>
The example of the functional configuration of the dual display device 1 according to this embodiment has been described above. Subsequently, an example of the processing flow in the dual display device 1 according to the present embodiment will be described with reference to FIGS. 8 and 9.

(1) When the Direct Operation Mode is Set in the Direct Operation Area R1 With reference to FIG. 8, an example of the processing flow when the direct operation mode is set in the direct operation area R1 will be described. FIG. 8 is a flowchart showing a processing flow when the direct operation mode is set in the direct operation area R1 according to the present embodiment. In the flowchart, it is assumed that the process starts while the user is directly looking inside the operation area R1.

As shown in FIG. 8, first, the determination unit 133 of the control unit 130 determines whether or not the user's line of sight is directly outside the operation region R1 based on the line-of-sight information detected by the sensor device 33 (S100). ). When it is determined that the line of sight of the user is directly within the operation area R1 (S100 / NO), the determination unit 133 repeats the process of S100. On the other hand, when it is determined that the line of sight of the user is directly outside the operation area R1 (S100 / YES), the determination unit 133 executes the process of S102.

The determination unit 133 determines whether or not the user's line of sight is detected directly outside the operation area R1 for a predetermined time or longer (S102). When the determination unit 133 does not determine that the user's line of sight is detected directly outside the operation area R1 for a predetermined time or longer (S102 / NO), the control unit 130 ends the process. On the other hand, when it is determined that the line of sight of the user is detected outside the operation area R1 for a predetermined time or longer (S102 / YES), the determination unit 133 executes the process of S104.

The determination unit 133 determines whether or not it is clear that the user's line of sight is directly outside the operation area R1 (S104). When the determination unit 133 determines that it is clear that the user's line of sight is directly outside the operation area R1 (S104 / YES), the mode switching unit 134 of the control unit 130 executes the process of S106. On the other hand, when it is determined that it is not clear that the user's line of sight is directly outside the operation area R1 (S104 / NO), the determination unit 133 executes the process of S108.

The mode switching unit 134 switches the operation mode of the direct operation area R1 from the direct operation mode to the indirect operation mode (S106). After switching the operation mode, the control unit 130 ends the process.

The determination unit 133 determines whether or not the viewpoint position of the user and the operation position substantially match (S108). When the determination unit 133 determines that the user's viewpoint position and the operation position do not substantially match (S108 / NO), the mode switching unit 134 executes the process of S106. On the other hand, when the determination unit 133 determines that the user's viewpoint position and the operation position substantially match (S108 / YES), the control unit 130 ends the process.

(2) When the Indirect Operation Mode is Set in the Direct Operation Area R1 With reference to FIG. 9, an example of the processing flow when the indirect operation mode is set in the direct operation area R1 will be described. FIG. 9 is a flowchart showing a processing flow when the indirect operation mode is set in the direct operation area R1 according to the present embodiment. In the flowchart, it is assumed that the process starts while the user is looking at the indirect operation area R2.

As shown in FIG. 9, first, the determination unit 133 of the control unit 130 determines whether or not the user's line of sight is directly in the vicinity of the operation region R1 based on the line-of-sight information detected by the sensor device 33 (S200). .. When it is determined that the user's line of sight is not directly in the vicinity of the operation area R1 (S200 / NO), the determination unit 133 repeats the process of S200. On the other hand, when it is determined that the line of sight of the user is directly in the vicinity of the operation area R1 (S200 / YES), the determination unit 133 executes the process of S202.

The determination unit 133 determines whether or not the user's line of sight is detected in the vicinity of the direct operation area R1 for a predetermined time or longer (S202). When the determination unit 133 does not determine that the user's line of sight is detected in the vicinity of the direct operation area R1 for a predetermined time or longer (S202 / NO), the control unit 130 ends the process. On the other hand, when it is determined that the line of sight of the user is detected in the vicinity of the direct operation area R1 for a predetermined time or longer (S202 / YES), the determination unit 133 executes the process of S204.

The determination unit 133 determines whether or not it is clear that the user's line of sight is directly within the operation area R1 (S204). When the determination unit 133 determines that it is clear that the user's line of sight is directly within the operation area R1 (S204 / YES), the mode switching unit 134 of the control unit 130 executes the process of S206. On the other hand, when it is determined that it is not clear that the user's line of sight is directly within the operation area R1 (S204 / NO), the determination unit 133 executes the process of S208.

The mode switching unit 134 switches the operation mode of the direct operation area R1 from the indirect operation mode to the direct operation mode (S206). After switching the operation mode, the control unit 130 ends the process.

The determination unit 133 determines whether or not the viewpoint position of the user and the operation position substantially match (S208). When the determination unit 133 determines that the user's viewpoint position and the operation position substantially match (S208 / YES), the mode switching unit 134 executes the process of S206. On the other hand, when the determination unit 133 determines that the user's viewpoint position and the operation position do not substantially match (S208 / NO), the control unit 130 ends the process.

As described above, the dual display device 1 according to the present embodiment determines which area of the direct operation area and at least one indirect operation area is viewed by the user based on the line of sight of the user.
Further, the dual display device 1 switches the operation mode of the direct operation area to the direct operation mode or the indirect operation mode based on the result of the determination.

With such a configuration, the dual display device 1 according to the present embodiment switches the operation mode of the direct operation area to the direct operation mode or the indirect operation mode according to the area viewed by the user. As a result, the user does not need to input an operation for switching the operation mode to the dual display device 1. That is, the dual display device 1 can reduce the time and effort of the operation when the user uses VTP.

Therefore, the dual display device 1 according to the present embodiment can improve the usability of the touch screen.

<5. Modification example>
The embodiment of the present invention has been described above. Subsequently, a modified example of the embodiment of the present invention will be described. In addition, each modification described below may be applied to the embodiment of the present invention alone, or may be applied to the embodiment of the present invention in combination. Further, each modification may be applied in place of the configuration described in the embodiment of the present invention, or may be additionally applied to the configuration described in each embodiment of the present invention.

In the above-described embodiment, an example in which the information processing device is a dual display device has been described, but the present invention is not limited to such an example. For example, the information processing device may be a terminal having at least one touch screen. Specifically, the information processing device may be a terminal such as a notebook PC (Personal Computer), a tablet terminal, a smartphone, or a game machine having at least one touch screen. Further, the information processing device may be realized by a foldable terminal (Foldable Device) capable of folding the touch screen.

In the above-described embodiment, the determination unit 133 determines whether the user is looking at the direct operation area or the indirect operation area, and the mode switching unit 134 switches the operation mode based on the determination result. Not limited. For example, the determination unit 133 determines whether the user is looking inside the direct operation area or outside the direct operation area, and the mode switching unit 134 determines whether the user is looking inside the direct operation area or outside the direct operation area. The operation mode may be switched accordingly. That is, as long as the target exists outside the direct operation area, the target seen by the user when using the indirect operation mode is not limited to the indirect operation area. Specifically, when the determination unit 133 determines that the user is looking directly outside the operation area when the operation mode is the direct operation mode, the mode switching unit 134 switches the operation mode to the indirect operation mode. On the other hand, when the determination unit 133 determines that the user is looking directly into the operation area when the operation mode is the indirect operation mode, the mode switching unit 134 switches the operation mode to the direct operation mode.

Such a configuration is particularly effective when, for example, when the user operates the direct operation area in which the indirect operation mode is set, the user sees the direct operation area without looking at the indirect operation area. An example of a specific scene is a scene in which a presentation is given. For example, it is assumed that the presentation material is displayed in the indirect operation area at the time of presentation. When advancing the page of the material during the presentation, the user may input the operation directly into the operation area while looking at the material displayed on the listener of the presentation or the projector, instead of the indirect operation area. At this time, if the control is such that the indirect operation mode is set only when the indirect operation area is viewed, the user operates the material displayed in the indirect operation area while looking at the material displayed on the listener or the projector. You will not be able to do it. Therefore, by not limiting the target to be viewed by the user when using the indirect operation mode as in this modification, the material displayed in the indirect operation area while viewing the material displayed on the listener or the projector. It is possible to perform the operation of.

Further, in the above-described embodiment, an example will be described in which the touch panel areas of the two touch screens (touch screen 10A and touch screen 10B) of the information processing device (dual display device 1) are set as a direct operation area or an indirect operation area, respectively. However, it is not limited to such an example. For example, when the information processing device is a terminal having only one touch screen, the touch panel area in the touch screen may be divided into two areas, and each area may be set as a direct operation area or an indirect operation area. When the information processing terminal is a foldable terminal, for example, the touch panel area may be divided into two areas by folding the touch screen.

Further, the touch panel area of the touch screen of the information processing device may be used as the direct operation area, and the display area of the display device externally connected to the information processing device may be used as the indirect operation area. The number of display devices externally connected to the information processing device is not particularly limited. For example, when a plurality of display devices are externally connected to an information processing device, there are a plurality of indirect operation areas. When the operation mode is switched to the indirect operation mode when there are multiple indirect operation areas, the cursor is set to, for example, the indirect operation area seen by the user in the multiple indirect operation areas, the primary in the OS. It can be displayed in an indirect operating area, or always in a particular indirect operating area.

In the above-described embodiment, an example in which the mode switching unit 134 switches the operation mode based on the line of sight of the user has been described, but the present invention is not limited to this example. The mode switching unit 134 switches the operation mode to the direct operation mode when the operation target is displayed in the direct operation area, and changes the operation mode to the indirect operation mode when the operation target is displayed in the indirect operation area. It may be switched.

In the above-described embodiment, an example in which the user's line of sight and the viewpoint position are detected by the sensor device 33 has been described, but the present invention is not limited to such an example. For example, the line-of-sight and the viewpoint position may be detected by the line-of-sight information acquisition unit 131 based on the information sensed by the sensor device 33. As an example, when the sensor device 33 is an image pickup device such as a camera, the line-of-sight information acquisition unit 131 detects the user's line of sight based on the image captured by the image pickup device. The captured image is specifically an image of the user's eyes or an image of the user's head. When the captured image is an image of the user's eye, the line-of-sight information acquisition unit 131 detects the user's line of sight based on, for example, a change in the movement of the user's eyelids. When the captured image is an image of the user's head, the line-of-sight information acquisition unit 131 detects the user's line of sight based on, for example, a change in the movement of the user's head (for example, a direction or an angle). Further, the line-of-sight information acquisition unit 131 detects the viewpoint position based on the line of sight of the detected user.

The embodiment of the present invention has been described above.
Each configuration included in the dual display device 1 described above has a computer system inside. Then, a program for realizing the functions of each configuration included in the above-mentioned dual display device 1 is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read and executed by the computer system. Therefore, the processing in each configuration included in the dual display device 1 described above may be performed. Here, "loading and executing a program recorded on a recording medium into a computer system" includes installing the program in the computer system. The term "computer system" as used herein includes hardware such as an OS and peripheral devices.
Further, the "computer system" may include a plurality of computer devices connected via a network including a communication line such as the Internet, WAN, LAN, and a dedicated line. Further, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, or a CD-ROM, and a storage device such as a hard disk built in a computer system. As described above, the recording medium in which the program is stored may be a non-transient recording medium such as a CD-ROM.

The recording medium also includes an internal or external recording medium accessible from the distribution server for distributing the program. It should be noted that the program may be divided into a plurality of parts, downloaded at different timings, and then combined with each configuration provided in the notebook PC 1, or the distribution server for distributing each of the divided programs may be different.
Furthermore, a "computer-readable recording medium" is a volatile memory (RAM) inside a computer system that serves as a server or client when a program is transmitted via a network, and holds the program for a certain period of time. It shall include things. Further, the above program may be for realizing a part of the above-mentioned functions. Further, a so-called difference file (difference program) may be used, which can realize the above-mentioned function in combination with a program already recorded in the computer system.

Further, a part or all of the above-mentioned functions may be realized as an integrated circuit such as an LSI (Large Scale Integration). Each of the above-mentioned functions may be made into a processor individually, or a part or all of them may be integrated into a processor. Further, the method of making an integrated circuit is not limited to the LSI, and may be realized by a dedicated circuit or a general-purpose processor. Further, when an integrated circuit technology that replaces an LSI appears due to advances in semiconductor technology, an integrated circuit based on this technology may be used.

Although the embodiments of the present invention have been described in detail with reference to the drawings, the specific configuration is not limited to the above, and various design changes and the like can be made without departing from the gist of the present invention. It is possible to do.

1 Dual display device 2 Housing 3 Connection part 10 Touch screen 11 Display screen 12 Touch panel 15 CPU
16 main memory 17 GPU
21 Chipset 22 BIOS Memory 23 HDD
24 Audio system 25 Communication device 31 EC
32 Input device 33 Sensor device 34 Power supply circuit 110 Communication unit 120 Input unit 130 Control unit 131 Line-of-sight information acquisition unit 132 Operation position information acquisition unit 133 Judgment unit 134 Mode switching unit 135 Signal processing unit 136 Display processing unit 140 Storage unit 150 Display unit R1 direct operation area R2 indirect operation area

In order to solve the above-mentioned problems, in the information processing apparatus according to one aspect of the present invention, whether the user is looking inside or outside the first area of the touch screen based on the user's line of sight. The operation mode of the first area is operated by the operation of inputting the operation target displayed in the first area to the first area based on the determination unit for determining the above and the result of the determination by the determination unit. A first mode that allows the first area to function, or an operation displayed in the second area of the touch screen or a display device that is different from the first area and different from the touch screen. The determination unit includes a mode switching unit that switches the target to a second mode that makes the first area function operably by an operation input to the first area, and the determination unit acquires based on the line of sight of the user. Based on the viewpoint position of the user and the operation position in which the operation by the user is input in the first area, the user is in either the first area or the second area. Determine if you are looking inside .

In the information processing method according to one aspect of the present invention, the determination unit determines whether the user is looking inside or outside the first region of the touch screen based on the user's line of sight. , The mode switching unit can operate the operation mode of the first area based on the result of the determination by the determination unit by the operation of inputting the operation target displayed in the first area to the first area. The operation target displayed in the first mode in which the first area is made to function, or in the second area of the touch screen or a display device different from the first area but different from the first area. Is switched to a second mode in which the first area is operably functioned by an operation input to the first area, and the determination unit is acquired based on the line of sight of the user. Based on the viewpoint position of the user and the operation position in which the operation by the user is input in the first area, the user moves within either the first area or the second area. Determine if you are watching .

Claims (9)

A determination unit that determines whether the user is looking inside or outside the first area of the touch screen based on the user's line of sight.
Based on the result of the determination by the determination unit, the operation mode of the first region can be operated by the operation of inputting the operation target displayed in the first region to the first region. The operation target displayed in the first mode for functioning, or the second area in the touch screen or a display device different from the first area but different from the first area is the first area. A mode switching unit that switches to a second mode that makes the first area function so that it can be operated by the operation input to
An information processing device equipped with. When the determination unit determines that the user is looking outside the first region when the operation mode is the first mode, the mode switching unit sets the operation mode to the second mode. When the mode is switched and the determination unit determines that the user is looking inside the first region when the operation mode is the second mode, the operation mode is changed to the first mode. Switch,
The information processing apparatus according to claim 1. The determination unit is based on the viewpoint position of the user acquired based on the line of sight of the user and the operation position in which the operation by the user is input in the first area, and the user is in the first area. To determine which area within or within the second area is being viewed.
The information processing apparatus according to claim 1 or 2. When the viewpoint position and the operation position substantially match, the determination unit determines that the user is looking inside the first region, and when the viewpoint position and the operation position do not substantially match, the determination unit determines that the user is looking inside the first region. Determining that the user is looking within the second area,
The information processing apparatus according to claim 3. A storage unit that stores definitional information including information about hardware including the touch screen and a touch pad different from the touch screen.
A firmware that generates a signal including the definition information according to the operation mode of the first region and outputs the signal to the operating system.
The information processing apparatus according to any one of claims 1 to 4, further comprising. The firmware is
When the operation mode is the first mode, a first signal including the definition information of the touch screen is generated and output to the operating system.
When the operation mode is the second mode, a second signal including the definition information of the touch pad is generated and output to the operating system.
The information processing apparatus according to claim 5. The determination unit determines whether the user is looking inside or outside the first area of the touch screen based on the user's line of sight.
Based on the result of the determination by the determination unit, the mode switching unit can operate the operation mode of the first area by the operation of inputting the operation target displayed in the first area to the first area. The operation target displayed in the first mode in which the first area functions, or in the second area of the touch screen or a display device different from the first area but different from the first area. Switching to the second mode in which the first area is operably operated by the operation input to the first area, and
Information processing methods, including. An information processing device provided with a mode switching unit for switching the operation mode of the first region of the touch screen.
The mode switching unit is
When the operation target is displayed in the first area, the operation target displayed in the first area can be operated by the operation input to the first area, and the first area is made to function. Switch to the first mode,
When the operation target is displayed in at least one second area of the touch screen or a display device different from the touch screen, which is a region different from the first region, the operation target is displayed in the first region. By the input operation, the operation target displayed in the second area is switched to the second mode in which the first area functions so that the operation target can be operated.
Information processing equipment. The mode switching unit is an information processing method including switching the operation mode of the first region on the touch screen.
The mode switching unit
When the operation target is displayed in the first area, the operation target displayed in the first area can be operated by the operation input to the first area, and the first area is made to function. Switch to the first mode,
When the operation target is displayed in at least one second area of the touch screen or a display device different from the touch screen, which is a region different from the first region, the operation target is displayed in the first region. By the input operation, the operation target displayed in the second area is switched to the second mode in which the first area functions so that the operation target can be operated.
Information processing method.

Images