WO2011093033A1 - 表示制御装置、表示制御方法、プログラムおよび集積回路 - Google Patents
表示制御装置、表示制御方法、プログラムおよび集積回路 Download PDFInfo
- Publication number
- WO2011093033A1 WO2011093033A1 PCT/JP2011/000246 JP2011000246W WO2011093033A1 WO 2011093033 A1 WO2011093033 A1 WO 2011093033A1 JP 2011000246 W JP2011000246 W JP 2011000246W WO 2011093033 A1 WO2011093033 A1 WO 2011093033A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- information
- touch
- touch pad
- unit
- user
- Prior art date
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/4104—Peripherals receiving signals from specially adapted client devices
- H04N21/4112—Peripherals receiving signals from specially adapted client devices having fewer capabilities than the client, e.g. thin client having less processing power or no tuning capabilities
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
- G06F3/0354—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of 2D relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
- G06F3/03547—Touch pads, in which fingers can move on a surface
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06F3/04883—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/4104—Peripherals receiving signals from specially adapted client devices
- H04N21/4126—The peripheral being portable, e.g. PDAs or mobile phones
- H04N21/41265—The peripheral being portable, e.g. PDAs or mobile phones having a remote control device for bidirectional communication between the remote control device and client device
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
- H04N21/42204—User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor
Definitions
- the present invention relates to a display control apparatus that receives a signal from an operating device having a touch pad operated by a user using a finger and generates screen data to be displayed on the screen.
- CE Consumer Electronics
- BD Blu-ray Disc
- an application used by the screen display device there is a web browser for browsing content on the Internet.
- a photo viewer for viewing image data recorded on an SD (Secure Digital) card or the like with a digital TV or BD recorder is also one such application.
- SD Secure Digital
- a keyboard application for inputting a search character string when performing a search with a web browser or the like.
- screen display device there is a screen display device equipped with an operating device having a touch pad, instead of a remote controller having a conventional fixed hardware key.
- a screen display device free pointing is possible on a GUI (Graphical User Interface) displayed on a screen of a digital TV or the like. Then, a cursor (a pointer indicating an operation position on the GUI) displayed on the GUI is moved in response to an operation of the user tracing the touch pad with a finger.
- GUI Graphic User Interface
- an operating device constituting the screen display device has a symmetrical shape in the vertical and horizontal directions.
- the user takes care of the direction of holding the control device (which side of the control device the user recognizes as the front direction) and the handle of the control device (whether the right hand or the left hand holds the control device) You can operate the screen display device freely.
- it is essential to appropriately control and display the cursor on the GUI in response to the user's operation, regardless of the orientation of the user's operation device.
- some techniques are known as a technique of recognizing the direction of the operating device which comprises the conventional screen display apparatus.
- the direction of the controller device is the same as the direction of the screen display device main body.
- Patent Document 1 discloses a portable terminal that can be operated by a user with one hand as a screen display device.
- This portable terminal is provided with a state detection unit that recognizes which part of the main body is held (also referred to as holding), and which of the main body is in the vertical direction (vertical direction).
- the state detection means recognizes the state of the main body using a contact sensor (also called a pressure sensor, a grip sensor or a touch sensor) and a gravity sensor (also called an acceleration sensor) attached to the portable terminal main body. And this portable terminal determines the display direction of a screen, etc. using the state detection result in a state detection means.
- the portable terminal recognizes the orientation of the main body based on the detection results of the contact sensor and the gravity sensor.
- Patent Document 2 discloses, as a screen display device, a portable terminal device capable of switching the display direction on the display unit according to the button operation of the user or the holding state of the main body of the user.
- the holding state of the main body by the user is detected by a pressure sensor attached around the portable terminal device. That is, the portable terminal device recognizes the orientation of the main body based on the user's explicit instruction or the detection result of the holding state by the pressure sensor.
- Patent Document 3 as a screen display device, an electronic paper device in which a plurality of touch sensors are disposed on the outer peripheral portion of the device body and the display direction can be optimized according to how the user holds the body There is.
- Patent Document 3 discloses a technique for recognizing the orientation of the main body using a touch sensor, as in the techniques disclosed in Patent Document 1 and Patent Document 2.
- the top-bottom direction of the display may be inverted depending on how the user holds the main body.
- Patent Document 3 describes that correct display can be performed by using a gravity sensor in combination to improve the estimation accuracy of the method of holding the user.
- JP 10-301695 Japanese Patent Application Laid-Open No. 11-143604 JP 2008-52062 A
- Patent Document 1 Patent Document 2, and Patent Document 3
- the orientation of the operating device may not be recognized correctly.
- These techniques use the detection results of the sensor, such as recognizing the orientation of the main body using an acceleration sensor included in the screen display device, and recognizing the gripping direction of the remote control by a user using a pressure sensor, etc. It is a technology that recognizes the orientation of an operating device such as a remote control.
- the user's intended direction may not match. As a result, there is a possibility that the operability of the user may be reduced or the user may be confused.
- the user when the user operates using the remote control, generally, the user holds and operates the remote control in a direction looking up the screen a little.
- the screen-side surface of the remote control is positioned above the user-side surface (front side) of the remote control.
- the acceleration sensor incorporated in the remote control body detects that the surface (front side) on the user side is downward.
- the orientation of the remote control main body is recognized as the face on the user side (front side) based on a general holding method.
- the orientation of the remote control recognized by the screen display device matches the orientation of the actual remote control intended by the user. That is, in this case, the orientation of the remote control body is correctly recognized.
- the screen side surface of the remote control is positioned lower than the user side surface (front side). Therefore, the acceleration sensor detects that the surface on the screen side is downward. Then, the orientation of the remote control main body is recognized as the face on the user side (front side) based on a general holding method.
- the user side surface (front side) of the remote control is recognized as the screen side surface. That is, the orientation of the remote control body is erroneously recognized in the opposite direction to the actual orientation of the remote control intended by the user. Even when the user operates the remote control while lying down, the acceleration sensor or the like may not correctly recognize the direction intended by the user.
- the orientation of the remote control body is erroneously recognized, the movement of the cursor on the GUI is contrary to the user's intention, such as the vertical direction being reversed with respect to the movement of the user's finger on the touch pad.
- Such false recognition may cause confusion of the user, or may cause the remote control to be switched. Therefore, the comfort of the user's remote control operation is impaired.
- the screen display device disclosed in Patent Document 2 instructs the user to operate the button to instruct reverse rotation in the vertical direction of the remote control main body. That is, the screen display device causes the user to explicitly specify the orientation of the remote control main body. In such a method, the effort of the user is increased. Therefore, also in this case, it can not be said that comfortable remote control operation can be realized.
- this invention aims at providing the display control apparatus for displaying an image etc. appropriately on a screen according to the user's natural operation to the operating device operated in various directions.
- a display control device is a display control device that receives a signal from an operating device having a touch pad and generates screen data to be displayed on a screen, and the user makes a finger
- a touch information detection unit that detects touch information including position information on the touch pad touched by the finger when operating the touch pad, and a feature indicated by the touch information detected by the touch information detection unit
- a direction determining unit that determines a direction intended by the user using a feature resulting from an operation on the touch pad by the finger, and the direction determined by the direction determined by the direction determining unit.
- a screen data generation unit that generates the screen data.
- the display control apparatus determines the direction intended by the user at the time of operation from the characteristics resulting from the operation of the finger, for example, the characteristics of the operation based on the structure of the human hand such as how to put on the thumb. Therefore, the display control device can recognize the direction intended by the user by natural operation of the user. Then, the display control device can appropriately display an image or the like on the screen based on the recognized direction.
- the direction determining unit may determine, using the feature, the direction which is an up, down, left, or right direction on the touch pad when the user looks at the touch pad.
- the display control device can appropriately display an image or the like on the screen based on the top, bottom, left, and right directions on the touch pad viewed from the user.
- the direction determining unit may determine the direction which is the direction of the controller device using the feature.
- the display control device can appropriately display an image or the like on the screen based on the direction of the operation device.
- the display control device further includes a touch information storage unit that stores a plurality of touch information by storing the touch information detected by the touch information detection unit, and the direction determination unit is configured to The direction may be determined using the features indicated by the plurality of touch information stored by the information storage unit.
- the display control device can determine the direction intended by the user using more detailed features based on the plurality of touch information. Therefore, the display control apparatus can display an image etc. more appropriately on the screen.
- the direction determining unit may determine the direction using the feature that is a shape of a locus formed by transition of position information indicated by the plurality of touch information.
- the display control device can determine the direction intended by the user based on the trajectory formed in accordance with the natural operation of the finger.
- the direction determining unit determines the direction by assuming that the convex side is the upper side when the locus is convex, or the concave side is the lower side when the locus is concave. It is also good.
- the display control device can directly determine the direction intended by the user from the shape of the trajectory.
- the direction determining unit may determine the direction using the feature that is a distribution of a plurality of pieces of position information indicated by the plurality of touch information.
- the display control device can determine the direction intended by the user based on the distribution of positions touched according to the natural operation of the finger.
- the display control apparatus further determines a display position of a cursor indicating a pointing position on the screen from the position information included in the touch information detected by the touch information detection unit.
- the cursor position determination unit determines the display position depending on the direction determined by the direction determination unit, and the screen data generation unit determines the display determined by the cursor position determination unit.
- the screen data for displaying the cursor at a position may be generated.
- the display control device can display the cursor at an appropriate position based on the direction intended by the user.
- the cursor position determination unit determines the position information and the predetermined direction.
- the display position may be determined by correcting the display position to be determined.
- the cursor is displayed at an appropriate position.
- the display control device further includes a warning data generation unit that generates warning data for notifying the user of a warning, and the warning data generation unit further includes the direction determined by the direction determination unit.
- the warning data may be generated when the direction determined in advance in the operation device is different.
- the display control device is a sensor further disposed at an outer peripheral portion of the operation device, and the information of a portion where the user holds the operation device is a sensor that detects a touch of the user.
- a sensor information detection unit for detecting certain held portion information is provided, and the direction determination unit may determine the direction depending on the held portion information detected by the sensor information detection unit and the feature. Good.
- the direction is more appropriately recognized by the information of the part where the user holds the operation device and the information obtained from the natural operation by the user.
- the display control device further includes a sensor information detection unit for detecting gravity direction information which is information on a gravity direction applied to the operation device from a sensor that detects a gravity direction applied to the operation device, and the direction determination The unit may determine the direction depending on the gravity direction information detected by the sensor information detection unit and the feature.
- the direction is more appropriately recognized by the information on the direction of gravity applied to the operating device and the information obtained from the natural operation by the user.
- the display control device is a warning data generation unit that further generates warning data for notifying the user of a warning, and a sensor disposed on an outer peripheral portion of the operation device, the contact of the user being And a sensor information detection unit for detecting holding portion information which is information of a portion where the user holds the operation device from a sensor to be detected, and the warning data generation unit is detected by the sensor information detection unit.
- the warning data may be generated when the direction estimated as the direction intended by the user from the held part information is different from the direction determined by the direction determining unit.
- the display control apparatus can verify the validity of the direction determined according to the user's natural operation using the information of the part where the user holds the operation device. Therefore, a more reliable direction is obtained.
- the display control device further generates a warning data generation unit that generates warning data for notifying the user of a warning, and a sensor that detects the direction of gravity applied to the operating device, the direction of gravity applied to the operating device
- a sensor information detection unit for detecting gravity direction information which is information of the information
- the warning data generation unit is a direction estimated as a direction intended by the user from the gravity direction information detected by the sensor information detection unit.
- the warning data may be generated when the direction determined by the direction determining unit is different.
- the display control apparatus can verify the validity of the direction determined according to the natural operation of the user, using the information on the direction of gravity applied to the operation device. Therefore, a more reliable direction is obtained.
- the display control device is a warning data generation unit that generates warning data for notifying the user of a warning, and a first sensor disposed on an outer peripheral portion of the operation device, the user A second sensor which detects holding part information which is information of a part where the user holds the operating device from a first sensor which detects a contact of the second user and detects a direction of gravity applied to the operating device; And a sensor information detection unit for detecting gravity direction information which is information on a gravity direction applied to the operation device, wherein the cursor position determination unit is configured to detect the holding portion information detected by the sensor information detection unit and the gravity direction information When the direction estimated from the user as the intended direction matches the direction determined by the direction determining unit, the direction determining unit The display position is determined depending on the determined direction, and the warning data generation unit determines the user's intention from the holding part information detected by the sensor information detection unit and the gravity direction information.
- the warning data may be generated when the estimated direction does not match the direction determined by the direction determining unit.
- the cursor is displayed at an appropriate position, and when the direction is not properly recognized, the user can recognize that.
- the touch information detection unit acquires a signal from the operation device having two touch pads, and when the user operates the two touch pads with a plurality of fingers, one of the plurality of fingers is Two touch information pieces each including position information on each touched touch pad are detected, and the direction determination unit detects each of the features indicated by each of the two touch information pieces detected by the touch information detection unit. Either of the two features may be used to determine the direction if the two features being configured match each other.
- the display control device can use the information on the plurality of touch pads to more appropriately determine the direction intended by the user.
- a display control method is a display control method for receiving screen signals from an operating device having a touch pad and for displaying on a screen, the user operating the touch pad with a finger.
- the display control method may include the step of generating screen data.
- the display control device is realized as a display control method.
- the program according to the present invention may be a program for causing a computer to execute the steps included in the display control method.
- the display control method is realized as a program.
- the integrated circuit according to the present invention is an integrated circuit that receives a signal from an operating device having a touch pad and generates screen data to be displayed on a screen, and when a user operates the touch pad with a finger
- a touch information detection unit for detecting touch information including position information on the touch pad touched by the finger, and a feature indicated by the touch information detected by the touch information detection unit
- a screen configured to generate the screen data depending on a direction determining unit that determines a direction intended by the user using a feature resulting from an operation on a touch pad, and the direction determined by the direction determining unit It may be an integrated circuit including a data generation unit.
- the display control device is realized as an integrated circuit.
- an image or the like is appropriately displayed on the screen in accordance with the user's natural operation on the operating device operated in various directions. Therefore, comfortable operability of the user is ensured.
- the original function of the operation device such as the user's input to the touch pad is utilized, the additional cost can be suppressed. That is, it is possible to achieve both improvement in user operability and suppression of additional cost.
- FIG. 1 is a schematic view of a screen display device according to the first embodiment.
- FIG. 2 is a block diagram of the screen display device according to the first embodiment.
- FIG. 3 is a diagram showing a configuration example of touch information in the first embodiment.
- FIG. 4 is a diagram showing a configuration example of the touch information table in the first embodiment.
- FIG. 5 is a diagram showing an entire processing flow of the screen display device in the first embodiment.
- FIG. 6 is a diagram showing a direction determining process flow in the direction determining unit of the first embodiment.
- FIG. 7A is an explanatory drawing showing a first example of trajectory shape determination processing in the direction determination unit of the first embodiment.
- FIG. 7B is an explanatory drawing showing a second example of trajectory shape determination processing in the direction determination unit of the first embodiment.
- FIG. 7A is an explanatory drawing showing a first example of trajectory shape determination processing in the direction determination unit of the first embodiment.
- FIG. 7B is an explanatory drawing showing a second example of trajectory
- FIG. 7C is an explanatory drawing showing a third example of trajectory shape determination processing in the direction determination unit of the first embodiment.
- FIG. 7D is an explanatory drawing showing a fourth example of trajectory shape determination processing in the direction determination unit of the first embodiment.
- FIG. 8 is a schematic view of the screen display device in the second embodiment.
- FIG. 9 is a block diagram of the screen display device in the second embodiment.
- FIG. 10 is a diagram showing a configuration example of touch information in the second embodiment.
- FIG. 11 is a diagram showing a configuration example of a touch information table in the second embodiment.
- FIG. 12 is a diagram showing an entire processing flow of the screen display device in the second embodiment.
- FIG. 13 is a diagram showing a direction determining process flow in the direction determining unit of the second embodiment.
- FIG. 14 is a diagram showing a flow of trajectory shape determination processing in the direction determination unit of the second embodiment.
- FIG. 15A is an explanatory drawing showing a first example of trajectory shape determination processing in the direction determination unit of the second embodiment.
- FIG. 15B is an explanatory drawing showing a second example of trajectory shape determination processing in the direction determination unit of the second embodiment.
- FIG. 16 is a schematic view of the screen display device in the third embodiment.
- FIG. 17 is a block diagram of the screen display device in the third embodiment.
- FIG. 18 is a diagram showing an entire processing flow of the screen display device in the third embodiment.
- FIG. 19 is a diagram showing a process flow of determining a horizontal holding direction in the direction determining unit of the third embodiment.
- FIG. 15A is an explanatory drawing showing a first example of trajectory shape determination processing in the direction determination unit of the second embodiment.
- FIG. 15B is an explanatory drawing showing a second example of trajectory shape determination processing in the direction determination unit of the second embodiment.
- FIG. 20 is a diagram showing a processing flow of determining the direction at the time of longitudinal holding in the direction determining unit according to the third embodiment.
- FIG. 21 is a diagram showing a flow of trajectory shape determination processing in the direction determination unit of the third embodiment.
- FIG. 22 is a block diagram of the screen display device in the fourth embodiment.
- FIG. 23 is a diagram showing an entire processing flow of the screen display device in the fourth embodiment.
- FIG. 24 is a block diagram of the screen display device in the fifth embodiment.
- FIG. 25 is a diagram showing an entire processing flow of the screen display device in the fifth embodiment.
- FIG. 26 is a diagram showing a process flow of determining a horizontal holding direction in the direction determining unit according to the fifth embodiment.
- FIG. 27 is a diagram showing a flow of processing for determining a vertically-held direction in the direction determining unit according to the fifth embodiment.
- FIG. 28 is a diagram showing a flow of position information distribution determination processing in the direction determining unit of the fifth embodiment.
- FIG. 29A is an explanatory drawing showing a first example of position information distribution determination processing in the direction determining unit of the fifth embodiment.
- FIG. 29B is an explanatory drawing showing a second example of position information distribution determination processing in the direction determining unit of the fifth embodiment.
- FIG. 30 is a block diagram of the screen display device in the modification of the first embodiment.
- FIG. 31 is a diagram showing a state of use of the screen display device in the modification of the first embodiment.
- Embodiment 1 The screen display apparatus according to the first embodiment recognizes the direction (also referred to as the direction) of the operation device including the touch pad by using the shape of the locus of position information (coordinates) indicating the user's input on the touch pad. . Then, the screen display device corrects the display position of the cursor (a pointer indicating the operation position on the GUI) on the screen based on the direction of the operation device.
- the direction also referred to as the direction
- the screen display device corrects the display position of the cursor (a pointer indicating the operation position on the GUI) on the screen based on the direction of the operation device.
- FIG. 1 is a schematic view of a screen display device according to the first embodiment.
- the screen display device 1 shown in FIG. 1 receives an input from the user, and displays a cursor 105 and the like on the screen 104.
- the remote control 101 is an example of an operating device with which the user operates the GUI on the screen 104.
- the remote control 101 includes a touch pad 102 that receives an input from the user's finger 103.
- the reception process of the input by the touch pad 102 is realized by a method of detecting a change in capacitance or the like, but since it is a known technique, the description is omitted here. Further, although the remote control 101 transmits a signal to the screen 104 by wireless such as Bluetooth and ZigBee / IEEE 802.15.4, the transmission technology of the signal is a publicly known technology as well, and therefore the description thereof is omitted here. .
- the contact position on the touch pad 102 is detected as position information represented by the x coordinate and the y coordinate.
- the coordinate system on the touch pad 102 is in one-to-one correspondence with the coordinate system on the screen 104. Therefore, the screen display device 1 can calculate the value of the coordinates of the cursor 105 on the screen 104 based on the detected position information.
- the contact of the finger 103 also includes an indirect contact through a glove or the like.
- FIG. 1 shows the directions of upper, lower, left, and right on the touch pad 102 when the touch pad 102 is viewed from the user's eye 106. It is assumed that the intended direction when the user operates the touch pad 102 is the vertical and horizontal directions shown in FIG. That is, the direction intended by the user is assumed to be the upper, lower, left, or right direction on the touch pad 102 when the user views the touch pad 102.
- the screen display device 1 recognizes such a direction, and displays the cursor 105 at an appropriate position based on the recognized direction.
- the direction intended by the user may also be referred to as the direction intended by the user with respect to the operating device, that is, the direction of the operating device.
- the direction shown in FIG. 1 is also the direction of the operating device such as the remote control 101 or the like.
- the controller device may have predetermined upper, lower, left, and right directions.
- the predetermined direction in the operating device typically corresponds to the display direction of the logo attached to the operating device.
- the direction of the operating device may be quantitatively represented by a rotation angle from a predetermined direction.
- FIG. 2 is a block diagram of the screen display device 1 in the first embodiment shown in FIG. In FIG. 2, the same components as in FIG.
- the screen display device 1 shown in FIG. 2 includes a touch pad 102, a touch information detection unit 201, a touch information storage unit 202, a direction determination unit 203, a cursor position determination unit 204, and a screen data generation unit 205. , And a display unit 206.
- the operating device 11 illustrated in FIG. 2 is realized by the remote control 101 or the like, and includes a touch pad 102.
- the display control device 10 includes a touch information detection unit 201, a touch information storage unit 202, a direction determination unit 203, a cursor position determination unit 204, and a screen data generation unit 205.
- the display control device 10 receives a signal from the controller device 11 and generates screen data for displaying an image or the like on the screen 104.
- the display control device 10 may be incorporated in the controller device 11 or the display unit 206.
- the touch information detection unit 201 includes touch information including position information which is at least an x coordinate and ay coordinate indicating a touch position on the touch pad 102 in response to a touch operation or a press operation on the touch pad 102 by the user. Is a means of detecting
- the touch information detection unit 201 is included in the display control device 10, but may be included in the operation device 11.
- the touch information detection unit 201 detects touch information at predetermined time intervals, and passes the detected touch information to the touch information storage unit 202.
- the configuration example of the touch information will be described in detail later with reference to FIG.
- the touch information storage unit 202 is configured by a storage unit or the like, and is a unit that stores touch information transferred from the touch information detection unit 201.
- the touch information storage unit 202 stores touch information such as a predetermined time or a predetermined amount in order of time series or the like so that the direction determination unit 203 can determine the direction of the controller device 11 using touch information.
- the configuration example of the touch information table for accumulating the touch information will be described in detail later with reference to FIG.
- the direction determining unit 203 is a unit that determines the direction of the controller device 11 based on the touch information stored in the touch information storage unit 202. Specifically, the direction determining unit 203 analyzes which touch information is based on a predetermined algorithm to determine which side of the controller device 11 faces the screen 104. This algorithm will be described in detail later using FIG.
- the direction determining unit 203 determines the direction of the controller device 11 using touch information obtained by an operation by which the user naturally touches the touch pad. can do.
- touch information is obtained when the user starts holding the operating device 11, when switching an application, when the holding direction of the operating device 11 changes in the same application, or the like.
- the cursor position determination unit 204 is a unit that calculates the position of the cursor 105 to be displayed on the screen 104 from the direction of the operation device 11 determined by the direction determination unit 203 and the touch information detected by the touch information detection unit 201. . That is, the cursor position determination unit 204 determines the display position of the cursor 105 indicating the pointing position on the screen 104 from the position information included in the touch information.
- the cursor position determination unit 204 is included in the touch information when the direction determination unit 203 determines that the user does not hold the operating device 11 in the “right direction”, that is, holds the operation device 11 in the “reverse direction”. Convert the position information value (coordinate conversion). Thereby, the cursor position determination unit 204 corrects the position information so that the cursor 105 moves in the same manner as when the user holds the controller device 11 in the “right direction”.
- the cursor position determining unit 204 does not correct the position information.
- correct direction indicates that the direction is the same as the direction of the coordinate system on the touch pad 102.
- the correct direction means a state in which the direction intended by the user matches the direction determined in advance by the operation device 11.
- the “reverse direction” means a state in which the direction intended by the user and the direction determined in advance in the operation device 11 are exactly opposite. Further, “rightward” means a state in which the upward direction intended by the user is the right direction predetermined in the operation device 11. Further, “leftward” means a state in which the upward direction intended by the user is the left direction predetermined in the controller device 11.
- the cursor position determination unit 204 uses the position information received from the touch information detection unit 201 or the position information after correction by the means to display the display position (cursor position) of the cursor 105 on the screen 104. decide.
- the screen data generation unit 205 is means for generating screen data based on the display position of the cursor 105 determined by the cursor position determination unit 204 and the like.
- the screen data generation unit 205 generates screen data to be displayed on the display unit 206 by superimposing an image of an application such as a moving image viewer or a web browser on a cursor image.
- the display unit 206 is means for displaying the screen data passed from the screen data generation unit 205 as an image or the like.
- the display unit 206 is, for example, a screen of a digital TV.
- the display unit 206 may be located outside the screen display device 1.
- touch information detected by the touch information detection unit 201 will be described with reference to FIG.
- FIG. 3 is a diagram illustrating an exemplary configuration of touch information.
- the touch information 300 shown in FIG. 3 includes an operation ID 301 and position information 302.
- the operation ID 301 is an ID indicating the type of an operation performed by the user on the touch pad 102.
- the operation ID 301 includes “TOUCH” indicating a touch operation on the touch pad 102 and “CLICK” indicating a pressing operation on the touch pad 102.
- the position information 302 is composed of the x-coordinate value and the y-coordinate value on the touch pad 102 on which the operation indicated by the operation ID 301 is performed, and the format of (x-coordinate value, y-coordinate value) Is represented by
- touch information 300 indicating “the touch operation on the touch pad 102 has occurred at (100, 100)” is illustrated.
- FIG. 4 is a diagram showing a configuration example of a touch information table held by the touch information storage unit 202 shown in FIG.
- the touch information table 400 illustrated in FIG. 4 is configured by a random access memory (RAM), a flash memory, or the like.
- RAM random access memory
- flash memory or the like.
- the touch information 300 delivered from the touch information detection unit 201 is accumulated in chronological order.
- the touch information storage unit 202 adds the received touch information 300 to the touch information table 400 each time the touch information detection unit 201 receives the touch information 300.
- the example shown in FIG. 4 is (1) “A touch operation to touch pad 102 occurred at (100, 100)", (2) “A touch operation to touch pad 102 occurred at (150, 200)” , (3) “The touch operation on the touch pad 102 occurred at (200, 300)”, and (4) “The touch operation on the touch pad 102 occurred at (200, 300)” .
- the touch information detection unit 201 detects a touch operation and a press operation on the touch pad 102 at predetermined time intervals. Therefore, according to the touch information table 400 shown in FIG. 4, the user's finger movement is “through the touch pad 102 at (100, 100) and in contact with the touch pad 102 via (150, 200) It is interpreted as “moving to (200, 300) and pressing the touch pad 102 there”.
- the touch information storage unit 202 may periodically delete the old touch information 300 included in the touch information table 400 when a predetermined time has elapsed.
- the touch information storage unit 202 receives the touch information 300 from the touch information detection unit 201 and the storage amount exceeds the predetermined amount, the touch information storage unit 202 overwrites and updates the old touch information 300 with the new touch information 300. Good.
- FIG. 5 is an entire processing flow of the screen display device 1 in the first embodiment shown in FIG.
- the touch information detection unit 201 of the screen display device 1 checks whether to continue detection of an input to the touch pad 102 based on an instruction from an instruction unit (not shown in FIG. 2) (S501).
- the screen display device 1 ends the process.
- the touch information detection unit 201 when the touch information detection unit 201 is instructed to continue the detection of the input to the touch pad 102 (YES in S501), the touch information detection unit 201 confirms the presence or absence of the input to the touch pad 102. Then, the touch information detection unit 201 detects the presence or absence of the touch information 300 shown in FIG. 3 (S502).
- the touch information detection unit 201 detects “no touch information” (NO in S502), the touch information detection unit 201 checks again whether or not to continue the detection of the input (S501).
- the touch information storage unit 202 stores the touch information 300 (S503). More specifically, the touch information detection unit 201 adds the touch information 300 to the touch information table 400 shown in FIG. Further, the touch information detection unit 201 also passes the detected touch information 300 to the cursor position determination unit 204 for cursor display on the screen.
- the cursor position determination unit 204 determines whether the position information 302 included in the touch information 300 needs to be corrected. Therefore, the cursor position determination unit 204 inquires the direction determination unit 203 about the direction of the controller device 11 (S504).
- the direction determination unit 203 accesses the touch information table 400 of the touch information storage unit 202 and reads the touch information 300 (S505). At this time, the direction determining unit 203 reads out the touch information 300 for a predetermined time set in the direction determining unit 203 in advance or the touch information 300 of a predetermined amount.
- the touch information storage unit 202 sequentially stores the touch information 300 received from the touch information detection unit 201 at predetermined time intervals. Therefore, the amount of touch information for a predetermined time can be obtained by calculation.
- the touch information storage unit 202 may hold the touch information 300 that has been read once inside the direction determining unit 203 or the like. Then, the touch information storage unit 202 may read out only the touch information 300 of the difference which is newly required. This reduces the amount of touch information 300 read out later.
- the direction determining unit 203 determines the direction of the controller device 11 using the touch information 300 read out in the touch information reading process (S505) (S506: direction determining process). Thereafter, the direction determining unit 203 passes the determined direction of the operating device 11 to the cursor position determining unit 204.
- the direction determination process will be described in detail later with reference to FIG.
- the cursor position determination unit 204 receives the direction of the controller device 11 from the direction determination unit 203, and determines whether the direction of the controller device 11 is the "right direction" (S507).
- the cursor position determination unit 204 executes a process (S509) of determining the cursor position on the screen.
- the cursor position determination unit 204 selects one of the touch information 300 received from the touch information detector 201 based on the orientation of the controller device 11.
- the position information 302 is corrected (S508: position information correction processing).
- the cursor position determination unit 204 corrects the position information 302 so that the cursor is displayed at the same position as the case where the direction of the controller device 11 is the “right direction”. The position information correction process will be described in detail later.
- the cursor position determination unit 204 determines the cursor position on the screen using the position information 302 of the touch information 300 received from the touch information detection unit 201 or the corrected position information 302 (S509). . Specifically, the cursor position determination unit 204 determines the coordinate system of the touch pad 102 and the coordinate system on the screen from the minimum value and the maximum value of the coordinate system of the touch pad 102 and the coordinate system on the screen display. Determine the cursor position by obtaining the magnification ratio of.
- Maximum value) ⁇ (minimum value of coordinate system on screen display) ⁇ ⁇ ⁇ (maximum value of coordinate system on touch pad 102) ⁇ (minimum value of coordinate system on touch pad 102) ⁇ on the screen Determine the cursor position.
- the cursor position determination unit 204 passes the x coordinate and the y coordinate of the determined cursor position to the screen data generation unit 205.
- the screen data generation unit 205 generates a screen data by superimposing a cursor image on a video and an animation to be displayed on the display unit 206 based on the cursor position received from the cursor position determination unit 204 (S510). Thereafter, the screen data generation unit 205 passes the generated screen data to the display unit 206.
- the display unit 206 displays the screen data received from the screen data generation unit 205 on a screen or the like (S511). After the end of the process, the touch information detection unit 201 again executes a process (S501) of confirming whether or not to continue detection of an input to the touch pad 102.
- position information 302 included in touch information 300 read out in the touch information reading process (S505) shown in FIG. 5 is arranged in time series, and position information of adjacent two points is connected by a line segment. It is described as "trajectory”.
- FIG. 6 is a diagram showing the flow of the direction determining process in the direction determining unit 203 shown in FIG.
- the direction determining unit 203 checks the presence or absence of the touch information 300 read from the touch information table 400 (S601).
- the direction determining unit 203 determines that the direction of the controller device 11 is "indefinite" (S613).
- the direction determining unit 203 is a line segment connecting the position information 302 of the two points on the locus for the touch information 300 read in the processing of the previous stage (S601).
- the slope .DELTA. Of is calculated (S602).
- the direction determining unit 203 calculates, for the slope ⁇ calculated in the process of the previous stage (S602), the degree of change ⁇ between two adjacent slopes ⁇ in time series (S603).
- the direction determining unit 203 calculates the degree of change ⁇ for all of the slopes ⁇ calculated in the previous process (S602).
- the direction determining unit 203 determines the shape of the trajectory from the position information 302, the inclination ⁇ , and the degree of change ⁇ (S604: trajectory shape determination processing).
- the trajectory shape determination process will be described in detail later with reference to FIGS. 7A to 7D.
- the direction determination unit 203 confirms whether the shape of the trajectory determined in the trajectory shape determination process (S604) is “convex upward” (S605).
- the direction determining unit 203 determines that the direction of the user's finger 103 is the touch pad 102 in view of the structural features of the human finger. (In the same direction as the coordinate system of the touch pad 102).
- the direction determining unit 203 determines that the direction of the controller device 11 is the "right direction" (S606). Then, the direction determining unit 203 passes the determined direction of the controller device 11 to the cursor position determining unit 204, and ends this processing.
- the direction determining unit 203 determines whether the shape of the trajectory determined in the trajectory shape determination process (S604) is “convex downward”. It confirms (S607).
- the direction determining unit 203 determines that the direction of the user's finger 103 is the touch pad 102 in view of the structural features of the human finger. It is determined that the downward direction (upwardly opposite to the coordinate system of the touch pad 102, or the direction rotated 180 degrees to the right or
- the direction determining unit 203 determines that the direction of the operating device 11 is "reverse", as opposed to the case where the direction of the operating device 11 is determined to be "correct” (S606) (S608) . Then, the direction determining unit 203 passes the determined direction of the controller device 11 to the cursor position determining unit 204, and ends this processing.
- the direction determining unit 203 determines whether the shape of the locus determined in the locus shape determination process (S604) is "convex right”. It confirms (S609).
- the direction determining unit 203 determines that the direction of the user's finger 103 is the touch pad 102 in view of the structural features of the human finger. It is determined that the direction is the right direction (the direction in which the coordinate system of the touch pad 102 is rotated 90.degree. To the right).
- the direction determining unit 203 determines that the direction of the controller device 11 is "rightward" (S610). The direction determining unit 203 passes the determined direction of the operating device 11 to the cursor position determining unit 204, and ends this processing.
- the direction determining unit 203 determines whether the shape of the locus determined in the locus shape determination process (S604) is "convex left”. It confirms (S611).
- the direction determining unit 203 determines that the direction of the finger 103 of the user is the touch pad 102 in view of the structural features of the human finger. It is determined that the direction is the left direction (the direction in which the coordinate system of the touch pad 102 is rotated 90.degree. To the left).
- the direction determining unit 203 determines that the direction of the controller device 11 is "leftward" (S612). The direction determining unit 203 passes the determined direction of the operating device 11 to the cursor position determining unit 204, and ends this processing.
- the direction determining unit 203 can not determine the shape of the locus in the process of the previous stage (“shape indefinite”). In this case, the direction determining unit 203 determines that the direction of the controller device 11 is “indeterminate” (S613).
- the direction determining unit 203 may determine that the direction of the controller device 11 is the “right direction” as the default direction.
- the default direction may be a direction set in advance as a default value on the screen display device 1 at the time of shipping or the like, or a direction set in advance as a default value by the user.
- trajectory shape determination processing in the direction determination unit 203 will be described using FIGS. 7A to 7D.
- 7A to 7D determine the shape of the trajectory from the inclination ⁇ and the degree of change ⁇ , and further determine the direction of the operation device 11 by estimating the direction of the user's finger from the shape of the trajectory. It is the figure which showed the method.
- the operation device 11 has four patterns of “right direction”, “reverse direction”, “right direction”, and “left direction”. 7A shows “right”, FIG. 7B shows “reverse”, FIG. 7C shows “right”, and FIG. 7D shows “left”.
- the direction determining unit 203 determines whether the value of the slope ⁇ calculated in the calculation process (S602) of the slope ⁇ of the direction determining process flow shown in FIG. 6 is positive or negative. Further, the direction determining unit 203 determines whether the difference “ ⁇ x” of the x coordinate at the time of calculating the inclination ⁇ is positive or negative and the difference “ ⁇ y” of the y coordinate. In addition, the direction determining unit 203 determines whether the value of the degree of change ⁇ calculated in the process of calculating the degree of change ⁇ (S603) is positive or negative.
- FIGS. 7A to 7D the positive / negative discrimination results are shown in the table, and positive is indicated by “+” and negative is indicated by “ ⁇ ”.
- positive is indicated by “+”
- negative is indicated by “ ⁇ ”.
- the slope ⁇ or the change degree ⁇ is 0, it may be considered as a special state represented by “ ⁇ ”.
- the slope ⁇ can not be calculated because the slope ⁇ is infinite or when the ⁇ x becomes zero. These cases may also be considered as features of the shape of the trajectory.
- the slope ⁇ changes from “+” to “ ⁇ ”, and the degrees of change ⁇ are all “ ⁇ ”.
- the direction determining unit 203 determines that the trajectory has a “convex upward” shape in consideration of the values of ⁇ x and ⁇ y.
- the direction determining unit 203 determines that the trajectory has a “convex downward” shape in consideration of the values of ⁇ x and ⁇ y.
- the direction determining unit 203 determines that the trajectory has a “convex right” shape in consideration of the values of ⁇ x and ⁇ y.
- the direction determining unit 203 determines that the trajectory has a “convex to the left” shape in consideration of the values of ⁇ x and ⁇ y.
- the direction determining unit 203 determines that the “shape is indefinite”. For example, as a case where the direction determination unit 203 can not determine the shape of the trajectory, the case where the number of pieces of position information 302 included in the trajectory is small can be considered.
- the direction determining unit 203 is the most suitable for any pattern of FIGS. 7A to 7D even if the combination of the change of the slope ⁇ and the change of the change degree ⁇ does not correspond to any of FIGS. 7A to 7D. It may be determined whether it is close. Thereby, the direction determining unit 203 may determine the shape of the trajectory as any of FIGS. 7A to 7D.
- the cursor position determination unit 204 obtains corrected position information by calculating coordinates rotated about the center coordinate C on the touch pad 102.
- the minimum value and the maximum value of the possible values of the x coordinate of the coordinate system of the touch pad 102 are x_min and x_max, respectively, and the minimum and the maximum values of the value of the y coordinate are y_min and y_max, respectively.
- the central coordinate C is expressed by C ((x_max-x_min) ⁇ 2, (y_max-y_min) ⁇ 2).
- the cursor position determination unit 204 obtains corrected position information by calculating coordinates rotated about the center coordinate C.
- the cursor position determination unit 204 turns the position information received from the touch information detection unit 201 to the right (or left) centering on the center coordinate C. Rotate 180 degrees.
- the cursor position determination unit 204 rotates the position information received from the touch information detection unit 201 by 90 ° to the left around the center coordinate C.
- the cursor position determination unit 204 rotates the position information received from the touch information detection unit 201 by 90 ° to the right around the center coordinate C.
- the cursor position determination unit 204 corrects the cursor position so that the cursor position is equivalent to the case where the user holds the operating device 11 in any direction, in the “right direction”. can do.
- the touch information storage unit 202 of the screen display device 1 stores the touch information 300 detected by the touch information detection unit 201. Then, the direction determining unit 203 determines the direction of the controller device 11 using the shape of the trajectory of the position information 302 included in the touch information 300. The cursor position determination unit 204 appropriately corrects the position information 302 included in the touch information 300 received from the touch information detection unit 201 according to the direction, and then determines the display position of the cursor 105.
- the screen display device 1 properly and automatically orients the operation device 11 based only on the input to the touch pad 102 by the user, regardless of which side of the operation device 11 the user holds the display device 104 side. It can be determined in Thus, the screen display device 1 can display the cursor 105 at the correct position. Therefore, the screen display device 1 can realize comfortable operability of the user.
- the screen display apparatus includes two touch pads, and recognizes an orientation of the operation device using user's input on the two touch pads. Then, the screen display device corrects the display position of the cursor on the screen based on the direction of the operation device.
- FIG. 8 is a schematic view of the screen display device in the second embodiment.
- the screen display device 2 shown in FIG. 8 receives an input from the user, and displays a cursor 806 and the like on the screen 805.
- the remote control 801 is an example of an operating device with which the user operates the GUI on the screen 805.
- the remote control 801 includes a right touch pad 802 and a left touch pad 803 that receive an input from the user's finger 804.
- Acceptance processing of the input by the right touch pad 802 and the left touch pad 803 is realized by a method of detecting a change in capacitance, etc., but since it is a known technique, the description is omitted here. Further, the remote control 801 transmits a signal to the screen 805 by wireless such as Bluetooth and ZigBee / IEEE 802.15.4. However, since the signal transmission technology is a publicly known technology as well, the description will be omitted here. .
- the user holds the remote control 801 with both hands and uses the two touch pads aligned in the left and right direction (hereinafter, the holding direction is also described as “horizontal holding”). It is. Therefore, the orientation of the operation device held by the user is either “right” or “reverse” with respect to the coordinate system of position information on the two-sided touch pad.
- designated of an application etc. is shown.
- the screen display device 2 rotates the coordinate system on the two-sided touch pad by 90 degrees rightward or leftward. May be Thereby, the same processing as the example shown below is realizable.
- the contact position on the right touch pad 802 is detected as position information represented by the x coordinate and the y coordinate.
- the coordinate system on the right touch pad 802 is in one-to-one correspondence with the coordinate system on the screen 805. Therefore, the screen display device 2 can calculate the value of the coordinates on the screen 805 of the cursor 806 based on the detected position information.
- the left touch pad 803 also has a coordinate system independent of the coordinate system of the right touch pad 802. Then, the contact position on the left touch pad 803 is detected as position information represented by the x coordinate and the y coordinate.
- the coordinate system on the left touch pad 803 is in one-to-one correspondence with the coordinate system on the screen 805. Therefore, the screen display device 2 can calculate the value of the coordinates on the screen 805 of the cursor 806 based on the detected position information.
- FIG. 9 is a block diagram of the screen display device 2 in the second embodiment shown in FIG.
- the same components as in FIG. 8 will be assigned the same reference numerals and descriptions thereof will be omitted.
- the screen display device 2 shown in FIG. 9 includes a right touch pad 802, a left touch pad 803, a touch information detection unit 901, a touch information storage unit 902, a direction determination unit 903, and a cursor position determination unit 904. , A screen data generation unit 905, and a display unit 906.
- the operating device 21 illustrated in FIG. 9 is realized by the remote control 801 or the like, and includes a right touch pad 802 and a left touch pad 803.
- the display control device 20 further includes a touch information detection unit 901, a touch information storage unit 902, a direction determination unit 903, a cursor position determination unit 904, and a screen data generation unit 905.
- the display control device 20 receives a signal from the operation device 21 and generates screen data to be displayed on the screen 805.
- the display control device 20 may be incorporated in the operation device 21 or the display unit 906.
- the touch information detection unit 901 is means for detecting touch information in response to a touch operation or a press operation on the right touch pad 802 or the left touch pad 803 by the user.
- the touch information detection unit 901 is included in the display control device 20, but may be included in the operation device 21.
- the touch information includes position information that is at least an x-coordinate and a y-coordinate indicating a touch position on the right touch pad 802 or the left touch pad 803.
- the touch information detection unit 901 detects touch information at predetermined time intervals, and passes the detected touch information to the touch information storage unit 902. The configuration example of the touch information will be described in detail later with reference to FIG.
- the touch information storage unit 902 is means for storing touch information passed from the touch information detection unit 901.
- the touch information storage unit 902 stores touch information such as a predetermined time or a predetermined amount in order of time series or the like so that the direction determination unit 903 can determine the direction of the operation device 21 using touch information.
- touch information table An exemplary configuration of the touch information table will be described in detail later with reference to FIG.
- the direction determining unit 903 is a unit that determines the direction of the controller device 21 based on the touch information stored in the touch information storage unit 902. Specifically, the direction determining unit 903 analyzes the touch information based on a predetermined algorithm to determine which direction the operation device 21 faces. This algorithm will be described in detail later using FIG. 13 and FIG.
- the direction determining unit 903 determines the direction of the operating device 21 using touch information obtained by an operation by which the user naturally touches the touch pad. can do.
- touch information is obtained when the user starts holding the operating device 21, when switching the application, when the holding direction of the operating device 21 changes in the same application, or the like.
- the cursor position determination unit 904 is a unit that calculates the position of the cursor 806 to be displayed on the screen 805 from the direction of the operation device 21 determined by the direction determination unit 903 and the touch information detected by the touch information detection unit 901. .
- the cursor position determination unit 904 converts (coordinates) the value of the position information included in the touch information. Thereby, the cursor position determination unit 904 corrects the position information so that the movement of the cursor 806 is equivalent to that in the case where the user holds the controller 21 in the “right direction”.
- the cursor position determining unit 904 does not correct the position information.
- the cursor position determination unit 904 uses the position information received from the touch information detection unit 901 or the position information after correction by the means to display the display position (cursor position) of the cursor 806 on the screen 805. decide.
- the screen data generation unit 905 is a unit that generates screen data based on the display position of the cursor 806 determined by the cursor position determination unit 904 and the like.
- the screen data generation unit 905 generates screen data to be displayed on the display unit 906 by superimposing an image of an application such as a moving image viewer or a web browser on a cursor image.
- the display unit 906 is a means for displaying the screen data passed from the screen data generation unit 905.
- the display unit 906 is, for example, a screen of a digital TV.
- the display unit 906 may be located outside the screen display device 2.
- touch information detected by the touch information detection unit 901 will be described with reference to FIG.
- FIG. 10 is a diagram illustrating an exemplary configuration of touch information.
- the touch information 1000 illustrated in FIG. 10 includes a touch pad ID 1001, an operation ID 1002, and position information 1003.
- the touch pad ID 1001 is an ID indicating which of the right touch pad 802 and the left touch pad 803 is touch information 1000 by a touch operation or a press operation on the touch pad.
- the touch pad ID 1001 includes a “right touch pad” indicating the case of the right touch pad 802 and a “left touch pad” indicating the case of the left touch pad 803.
- the operation ID 1002 is an ID indicating the type of operation performed by the user on the right touch pad 802 or the left touch pad 803.
- the operation ID 1002 includes “TOUCH” indicating a touch operation on the right touch pad 802 or the left touch pad 803, and “CLICK” indicating a pressing operation on the right touch pad 802 or the left touch pad 803.
- the position information 1003 includes the x coordinate value and the y coordinate value on the right touch pad 802 or the left touch pad 803 on which the operation indicated by the operation ID 1002 is performed. It is expressed in the form of y-coordinate value).
- touch information 1000 indicating “the touch operation on the right touch pad 802 has occurred at (100, 100)” is illustrated.
- FIG. 11 is a diagram showing a configuration example of the touch information table 1100 stored by the touch information storage unit 902.
- the touch information storage unit 902 is a table in which the touch information storage unit 902 stores the touch information 1000 delivered from the touch information detection unit 901 in a RAM, a flash memory, or the like in chronological order.
- the touch information storage unit 902 adds the received touch information 1000 to the touch information table 1100 each time the touch information 1000 is passed from the touch information detection unit 901.
- the touch information detection unit 901 detects a touch operation and a press operation on the right touch pad 802 and the left touch pad 803 at predetermined time intervals.
- the movement of the user's finger is interpreted as follows. That is, first, the user's finger touches the right touch pad 802 at (100, 100). Next, the user's finger moved to (200, 300) via (150, 200) in contact with the right touch pad 802. Then, the user's finger pressed the right touch pad 802. Also, the user's finger touched the left touch pad 803 at (30, 300). Then, the finger of the user pressed the left touch pad 803. The above operation is interpreted from the touch information table 1100.
- the touch information storage unit 902 may periodically delete the old touch information 1000 included in the touch information table 1100 when the predetermined time has elapsed.
- the touch information storage unit 902 receives the touch information 1000 from the touch information detection unit 901 and the storage amount exceeds the predetermined amount, the touch information storage unit 902 overwrites and updates the old touch information 1000 with the new touch information 1000. Good.
- FIG. 12 is an entire processing flow of the screen display device 2 in the second embodiment shown in FIG.
- the touch information detection unit 901 of the screen display device 2 confirms whether to continue detection of an input to the right touch pad 802 or the left touch pad 803 based on an instruction from an instruction unit (not shown in FIG. 9). (S1201).
- the screen display apparatus 2 ends the process.
- the touch information detection unit 901 when the touch information detection unit 901 is instructed to continue the detection of the input to the right touch pad 802 or the left touch pad 803 (YES in S1201), the touch information detection unit 901 performs the touch shown in FIG. The presence or absence of the information 1000 is detected (S1202). In this case, the touch information detection unit 901 detects the presence or absence of the touch information 1000 by confirming the presence or absence of an input to the right touch pad 802 or the left touch pad 803.
- the touch information detection unit 901 detects “no touch information” (NO in S1202)
- the touch information detection unit 901 checks again whether or not to continue the detection of the input (S1201).
- the touch information detection unit 901 detects that “the touch information is present” (YES in S1202)
- the touch information detection unit 901 stores the touch information 1000 in the touch information storage unit 902 (S1203). More specifically, the touch information detection unit 901 adds the touch information 1000 to the touch information table 1100 shown in FIG. Further, the touch information detection unit 901 also passes the detected touch information 1000 to the cursor position determination unit 904 for displaying a cursor on the screen.
- the cursor position determination unit 904 determines whether it is necessary to correct the position information 1003 included in the touch information 1000. Therefore, the direction determination unit 903 is inquired of the direction of the controller device 21 (S1204).
- the direction determination unit 903 accesses the touch information table 1100 of the touch information storage unit 902, and reads the touch information 1000 (S1205). At this time, the direction determining unit 903 reads the touch information 1000 for a predetermined time set in advance in the direction determining unit 903 or the touch information 1000 of a predetermined amount.
- the touch information storage unit 902 sequentially stores touch information 1000 received from the touch information detection unit 901 at predetermined time intervals. Therefore, the amount of touch information for a predetermined time can be obtained by calculation.
- the touch information storage unit 902 may hold the touch information 1000 once read in the direction determination unit 903 or the like. Then, the touch information storage unit 902 may read out only the touch information 1000 of the difference which is newly required. This reduces the amount of touch information 1000 read out later.
- the direction determining unit 903 determines the direction of the controller device 21 using the touch information 1000 read in the process of the previous stage (S1205) (S1206: direction determining process). Thereafter, the direction determining unit 903 passes the determined direction of the operating device 21 to the cursor position determining unit 904. The direction determination process will be described in detail later with reference to FIG.
- the cursor position determination unit 904 receives the direction of the operation device 21 from the direction determination unit 903 and determines whether the direction of the operation device 21 is the “right direction” (S1207).
- the cursor position determination unit 904 executes a process (S1209) of determining the cursor position on the screen.
- the cursor position determination unit 904 corrects the position information 1003 of the touch information 1000 received from the touch information detection unit 901 (S1208: Position information correction process). In this case, the cursor position determination unit 904 corrects the position information 1003 based on the direction of the operation device 21 so that the cursor is displayed at the same position as when the direction of the operation device 21 is “right”. Do.
- the position information correction process is the same as that of the first embodiment.
- the cursor position determination unit 904 determines the cursor position on the screen using the position information 1003 of the touch information 1000 received from the touch information detection unit 901 or the corrected position information 1003 (S1209). .
- the cursor position determination unit 904 determines the coordinate system of the right touch pad 802 from the minimum value and the maximum value of the coordinate system of the right touch pad 802 and the minimum value and the maximum value in the coordinate system on the screen display. Acquire the enlargement ratio with the coordinate system on the screen display. In addition, the cursor position determination unit 904 determines the coordinate system of the left touch pad 803 and the screen display from the minimum value and the maximum value of the coordinate system of the left touch pad 803 and the minimum value and the maximum value in the coordinate system on the screen display. Get the magnification ratio with the coordinate system of. Thereby, the cursor position determination unit 904 determines the cursor position on the screen.
- the cursor position corresponding to the right touch pad 802 is determined by.
- the cursor corresponding to the left touch pad 803 by the following equation: minimum value of coordinate system on display) ⁇ ⁇ (maximum value of coordinate system on left touch pad 803) ⁇ (minimum value of coordinate system on left touch pad 803) ⁇ Determine the position.
- the cursor position determination unit 904 passes the x and y coordinates of the determined cursor position to the screen data generation unit 905.
- the coordinate system on the screen display may be assigned to be different between the right touch pad 802 and the left touch pad 803.
- “the maximum value of the coordinate system on the screen display” and “the minimum value of the coordinate system on the screen display” are provided for the right touch pad 802 and the left touch pad 803, respectively. Thereby, the respective cursor positions are determined.
- the screen data generation unit 905 generates a screen data by superimposing the cursor image on the video and animation to be displayed on the display unit 906 based on the cursor position received from the cursor position determination unit 904 (S1210). Thereafter, the screen data generation unit 905 passes the generated screen data to the display unit 906.
- the display unit 906 displays the screen data received from the screen data generation unit 905 on a screen or the like (S1211). After the end of this process, the touch information detection unit 901 checks again whether or not to continue the detection of the input (S1201).
- position information 1003 included in touch information 1000 read out in the touch information reading process (S1205) shown in FIG. 12 is arranged in time series, and position information of two adjacent points is connected by a line segment. It is described as "trajectory”.
- the direction determining unit 903 executes a process of determining a locus shape of the touch information 1000 read from the touch information table 1100 with respect to the right touch pad 802 (S1301).
- the trajectory shape determination process will be described in detail later with reference to FIG.
- the direction determining unit 903 executes a trajectory shape determination process of the touch information 1000 read from the touch information table 1100 with respect to the left touch pad 803 (S1302).
- This trajectory shape determination process is the same as the trajectory shape determination process (S1301) of the previous stage.
- the direction determining unit 903 confirms that there is touch information of both touch pads from the processing results of the trajectory shape determination processing of the right touch pad (S1301) and the trajectory shape determination processing of the left touch pad (S1302). (S1303). That is, the direction determining unit 903 confirms that both the touch information 1000 on the right touch pad 802 and the touch information 1000 on the left touch pad 803 are present.
- the direction determining unit 903 determines whether the shapes of both trajectories match. It confirms (S1304). That is, in this case, the direction determining unit 903 determines the shape of the locus of the right touch pad 802 and the left touch based on the processing results of the locus shape determination process of the right touch pad (S1301) and the locus shape determination process of the right touch pad (S1302). It is confirmed whether or not the shape of the path of the pad 803 matches.
- the shape of the locus of the right touch pad 802 and the shape of the locus of the left touch pad 803 are both “convex upward” or both “convex downward”. Check if it is.
- the processing result of the trajectory shape determination processing (S1301) of the right touch pad or the processing result of the trajectory shape determination processing (S1302) of the left touch pad may be “indeterminate in shape”. In this case, it is determined that the shape of the locus of the right touch pad 802 and the shape of the locus of the left touch pad 803 do not match.
- the direction determining unit 903 confirms that either one is present ( S1310). That is, in this case, the direction determining unit 903 determines the touch information 1000 and the left touch related to the right touch pad 802 from the processing result of the right touch pad locus shape determination processing (S1301) and the right touch pad locus shape determination processing (S1302). It is confirmed that either one of the touch information 1000 related to the pad 803 is present.
- the direction determination unit 903 executes the direction determination processing using the shape of the trajectory when the touch information 1000 regarding either the right touch pad 802 or the left touch pad 803 is detected.
- the direction determining unit 903 may execute the direction determining process only when the touch information 1000 related to both the right touch pad 802 and the left touch pad 803 is detected. In this case, the process (S1310) of confirming that there is either touch information 1000 of both touch pads is omitted.
- the direction determining unit 903 selects the operation device 21.
- the direction is determined to be "direction undefined" (S1309).
- the direction determination unit 903 may determine that the direction of the controller device 21 is the “right direction” as the default direction.
- the default direction may be a direction set in advance as a default value on the screen display device 2 at the time of shipping or the like, or may be a direction set in advance as a default value by the user.
- the direction determining unit 903 confirms whether the shape of the trajectory confirmed in the processing of the previous stage is “convex upward” (S1305).
- the direction determining unit 903 determines that the direction of the user's finger 804 is the right touch pad in view of the structural features of the human finger. It is determined that the touch panel 802 and the left touch pad 803 are in the upper direction. That is, the direction determining unit 903 determines that the direction of the user's finger 804 is the same as the coordinate system of the right touch pad 802 and the coordinate system of the left touch pad 803.
- the direction determining unit 903 determines that the direction of the controller device 21 is the “right direction” (S1306). Then, the direction determining unit 903 passes the determined direction of the operating device 21 to the cursor position determining unit 904, and ends this processing.
- the direction determining unit 903 confirms whether the shape of the locus checked in the previous process is "convex downward” (S1307) ).
- the direction determining unit 903 determines that the direction of the user's finger 804 is the right touch pad in view of the structural feature of the human finger. It is determined that the lower side 802 and the left touch pad 803 are in the downward direction. That is, the direction determining unit 903 rotates the user's finger 804 in the direction opposite to the coordinate system of the right touch pad 802 and the coordinate system of the left touch pad 803 by 180 ° to the right or to the left. I will judge.
- the direction determining unit 903 determines that the direction of the operating device 21 is "reversed” (S1308), contrary to the case where the direction of the operating device 21 is determined to be “correct” (S1306). . Then, the direction determining unit 903 passes the determined direction of the operating device 21 to the cursor position determining unit 904, and ends this processing.
- the direction determining unit 903 is similar to that in the case where the shapes of the loci of both touch pads do not match (NO in S1304). It is determined that the direction is "direction undefined" (S1309).
- the direction determining unit 903 confirms the presence or absence of the touch information 1000 read from the touch information table 1100 (S1401).
- the direction determining unit 903 ends this processing.
- the direction determining unit 903 is a line segment connecting the position information 1003 of the two points on the locus for the touch information 1000 read in the previous process (S1401).
- the direction determining unit 903 calculates, for the slope ⁇ calculated in the process of the preceding stage (S1402), the degree of change ⁇ between two adjacent slopes ⁇ in time series (S1403).
- the direction determining unit 903 calculates the degree of change ⁇ for all of the slopes ⁇ calculated in the previous process (S1402).
- the direction determining unit 903 determines the shape of the trajectory from the position information 1003, the inclination ⁇ , and the degree of change ⁇ (S1404). This process will be described in detail with reference to FIGS. 15A and 15B.
- FIGS. 15A and 15B determine the shape of the trajectory from the inclination ⁇ and the degree of change ⁇ , and further estimate the direction of the manipulation device 21 by estimating the direction of the user's finger from the shape of the trajectory It is the figure which showed the method.
- the direction of the operation device 21 has two patterns of “right direction” and “reverse direction”.
- FIG. 15A shows the “right direction”
- FIG. 15B shows the “reverse direction”.
- the direction determining unit 903 determines whether the value of the slope ⁇ calculated in the process (S1402) of calculating the slope ⁇ of the flow of the direction determining process shown in FIG. 14 is positive or negative. In addition, the direction determining unit 903 determines whether the difference “ ⁇ x” in the x coordinate and the difference “ ⁇ y” in the y coordinate in calculating the inclination ⁇ are positive or negative. In addition, the direction determining unit 903 determines whether the value of the degree of change ⁇ calculated in the process of calculating the degree of change ⁇ (S1403) is positive or negative.
- the positive / negative discrimination results are shown in the form of a table, with positive being indicated by “+” and negative being indicated by “ ⁇ ”.
- the slope ⁇ or the change degree ⁇ when the slope ⁇ or the change degree ⁇ is 0, it may be considered as a special state represented by “ ⁇ ”.
- the slope ⁇ can not be calculated because the slope ⁇ is infinite or when the ⁇ x becomes zero. These cases may also be considered as features of the shape of the trajectory.
- the slope ⁇ changes from “+” to “ ⁇ ”, and the degrees of change ⁇ are all “ ⁇ ”.
- the direction determining unit 903 determines that the trajectory has a “convex upward” shape in consideration of the values of ⁇ x and ⁇ y.
- the slope ⁇ changes from “ ⁇ ” to “+”, and all the degrees of change ⁇ are “+”.
- the direction determining unit 903 determines that the trajectory has a “convex downward” shape in consideration of the values of ⁇ x and ⁇ y.
- the direction determining unit 903 can not determine the shape of the trajectory. Therefore, the direction determining unit 903 determines that the shape of the trajectory is “undefined”. For example, as a case in which the direction determination unit 903 can not determine the shape of the trajectory, the number of pieces of positional information 1003 included in the trajectory may be small.
- direction determining unit 903 is closest to either of the patterns of FIGS. 15A and 15B. It may be determined. Thereby, the direction determining unit 903 may determine the shape of the trajectory as shown in FIG. 15A or 15B.
- the touch information storage unit 902 of the screen display device 2 stores touch information 1000 on the two touch pads of the right touch pad 802 and the left touch pad 803 detected by the touch information detection unit 901. Do. Then, from the position information 1003 included in the touch information 1000, the direction determining unit 903 determines the direction of the controller device 21 using the shapes of the loci of both the right touch pad 802 and the left touch pad 803.
- the cursor position determination unit 904 determines the display position of the cursor 806 after appropriately correcting the position information 1003 included in the touch information 1000 received from the touch information detection unit 901 according to the direction of the operation device 21.
- the screen display device 2 appropriately and automatically orients the operating device 21 based on only the user's input to the right touch pad 802 and the left touch pad 803 regardless of which direction the user holds the operating device 21. And the cursor 806 can be displayed at the correct position. Therefore, the screen display device 2 can realize the comfortable operability of the user.
- the screen display device including two touch pads further includes a sensor for detecting which part of the operation device the user is gripping.
- the screen display device of the third embodiment shows the gripping direction of the remote control shown in FIG. Also described as “longitudinal”. Then, the screen display apparatus according to the third embodiment recognizes the orientation of the operation apparatus in the holding manner (gripping direction) in both the horizontal holding and the vertical holding by using the user's input on the touch pad of the screen display. By doing this, the display position of the cursor on the screen is corrected based on the gripping direction and the direction of the operating device.
- FIG. 16 is a schematic view of the screen display device in the third embodiment.
- the screen display device 3 shown in FIG. 16 receives an input from the user, and displays the cursor 806 on the screen 805.
- one or more sensors for detecting the contact position of the user's finger are arranged at the outer peripheral portion of the remote control 1601 as compared with the remote control 801 shown in FIG. 8. Thereby, whether the screen display device 3 holds the remote control 1601 in the horizontal direction (direction of the remote control 801 shown in FIG. 8) or in the vertical direction (direction of the remote control 1601 shown in FIG. 16) Can be detected automatically.
- a sensor for detecting such a user's finger contact is a well-known technique, detailed description is abbreviate
- FIG. 17 is a block diagram of the screen display device 3 in the third embodiment shown in FIG.
- the same components as in FIG. 9 will be assigned the same reference numerals and descriptions thereof will be omitted.
- the screen display device 3 shown in FIG. 17 includes a sensor 1701, a sensor information detection unit 1702, a direction determination unit 1703, a cursor position determination unit 1704, a right touch pad 802, a left touch pad 803, and touch information.
- a detection unit 901, a touch information storage unit 902, a screen data generation unit 905, and a display unit 906 are provided.
- the operating device 31 illustrated in FIG. 17 is realized by the remote control 1601 or the like, and includes a sensor 1701, a right touch pad 802, and a left touch pad 803.
- the display control device 30 includes a sensor information detection unit 1702, a direction determination unit 1703, a cursor position determination unit 1704, a touch information detection unit 901, a touch information storage unit 902, and a screen data generation unit 905. .
- the display control device 30 receives a signal from the operation device 31 and generates screen data to be displayed on the screen 805.
- the display control device 30 may be incorporated in the operation device 31 or the display unit 906.
- the touch information detection unit 901 is included in the display control device 30, but may be included in the operation device 31.
- the sensor 1701 is a touch sensor for detecting a touch of a user's finger.
- One or more touch sensors are disposed on the outer periphery of the controller 31.
- the sensor information detection unit 1702 detects a signal from the sensor 1701 indicating detection of contact or the like of the user's finger. Then, in view of the position of the sensor 1701 disposed on the outer peripheral portion of the operation device 31, etc., the user holds the operation device 31 in the horizontal direction, that is, holds it “horizontally” and holds it in the vertical direction. It is detected as the gripping direction of the operation device 31 which one of the "longitudinal holding" is made.
- the sensor information detection unit 1702 notifies the direction determination unit 1703 of the detected gripping direction of the controller device 31 according to the request from the direction determination unit 1703 or according to the detection condition of the sensor information of the sensor information detection unit 1702. Do.
- the direction determining unit 1703 determines the direction of the operating device 31 based on the user's grip position of the operating device 31 detected by the sensor information detection unit 1702 and the touch information stored in the touch information storage unit 902. It is a means. Specifically, the direction determining unit 1703 determines from the gripping position detected by the sensor information detecting unit 1702 which gripping direction the horizontally and vertically the operating device 31 is in, and then determines the predetermined algorithm. Analyze touch information based on it. Thus, the direction determining unit 1703 determines which direction the operating device 31 is facing.
- the direction determining unit 1703 determines the direction of the operation device 31 using touch information obtained by an operation by which the user naturally touches the touch pad. can do.
- touch information is obtained when the user starts holding the operating device 31, when switching the application, when the holding direction of the operating device 31 changes in the same application, or the like.
- the cursor position determination unit 1704 displays a screen 805 from the gripping direction of the operation apparatus 31 received from the direction determination unit 1703, the direction of the operation apparatus 31 determined by the direction determination unit 1703, and the touch information detected by the touch information detection unit 901. It is a means for calculating the position of the cursor 806 to be displayed above.
- the cursor position determination unit 1704 converts (coordinates conversion) the value of the position information included in the touch information based on the holding direction (horizontal holding or vertical holding) of the operating device 31.
- the cursor position determining unit 1704 converts the value of the position information included in the touch information (coordinate conversion). Do. Thereby, the cursor position determination unit 1704 corrects the position information so that the movement of the cursor 806 is equivalent to the case where the user holds the controller 31 in the “right direction”. On the other hand, when it is determined by the direction determining unit 1703 that the user holds the operating device 31 in the “right direction”, the cursor position determining unit 1704 does not correct the position information.
- the conversion of the position information based on the holding direction (horizontal holding or vertical holding) of the operation device 31 is a process of converting the position information into the coordinate system of the holding direction set as a reference.
- the coordinate system of the right touch pad 802 and the left touch pad 803 is set to be fixed based on the fact that the operation device 31 is held vertically (the gripping direction of the remote control 1601 shown in FIG. 16), When the operating device 31 is held horizontally, the position information is converted.
- the cursor position determination unit 1704 rotates the coordinate system of the right touch pad 802 and the left touch pad 803 by 90 degrees to the left in order to correspond to the coordinate system of the gripping direction set as the reference. That is, the cursor position determination unit 1704 rotates the position information 90 ° to the left about the center coordinates of the coordinate systems of the right touch pad 802 and the left touch pad 803 as the center of rotation.
- the cursor position determination unit 1704 uses the position information received from the touch information detection unit 901 or the position information after correction by the means to display the display position (cursor position) of the cursor 806 on the screen 805. decide.
- FIG. 17 the process flow of the screen display apparatus 3 in the third embodiment shown in FIG. 17 will be described using FIGS. 18 to 20.
- FIG. 18 the process flow of the screen display apparatus 3 in the third embodiment shown in FIG. 17 will be described using FIGS. 18 to 20.
- FIG. 18 is an entire processing flow of the screen display device 3 in the third embodiment shown in FIG.
- the touch information detection unit 901 of the screen display device 3 checks whether to continue detection of an input to the right touch pad 802 or the left touch pad 803 based on an instruction from an instruction unit (not shown in FIG. 17). (S1801).
- the screen display apparatus 3 ends the process.
- touch information detection unit 901 when touch information detection unit 901 is instructed to continue detection of an input to right touch pad 802 or left touch pad 803 (YES in S1801), touch information detection unit 901 detects right touch pad 802 or the left touch pad. The presence or absence of an input to the touch pad 803 is confirmed. Thus, the touch information detection unit 901 detects the presence or absence of the touch information 1000 illustrated in FIG. 10 (S1802).
- the touch information detection unit 901 detects “no touch information” (NO in S1802)
- the touch information detection unit 901 checks again whether or not to continue the detection of the input (S1801).
- the touch information storage unit 902 causes the touch information 1000 to be stored (S1803). More specifically, the touch information detection unit 901 adds the touch information 1000 to the touch information table 1100 shown in FIG. The touch information detection unit 901 also passes the detected touch information 1000 to the cursor position determination unit 1704 for displaying a cursor on the screen.
- the cursor position determination unit 1704 receives the touch information 1000 from the touch information detection unit 901. Then, the cursor position determination unit 1704 determines whether the position information 1003 included in the touch information 1000 needs to be corrected. Therefore, the cursor position determination unit 1704 inquires the direction determination unit 1703 about the direction of the controller device 31 (S1804).
- the direction determining unit 1703 receives an inquiry from the cursor position determining unit 1704. Then, the direction determining unit 1703 accesses the touch information table 1100 of the touch information storage unit 902, and reads the touch information 1000 (S1805). At this time, the direction determining unit 1703 reads the touch information 1000 or the touch information 1000 of a predetermined amount for a predetermined time set in the direction determining unit 1703 in advance.
- the touch information storage unit 902 sequentially stores touch information 1000 received from the touch information detection unit 901 at predetermined time intervals. Therefore, the amount of touch information for a predetermined time can be obtained by calculation.
- touch information storage section 902 may hold touch information 1000 that has been read once inside direction determination section 1703 or the like. Then, the touch information storage unit 902 may read out only the touch information 1000 of the difference which is newly required. This reduces the amount of touch information 1000 read out later.
- the direction determining unit 1703 queries the sensor information detection unit 1702 about the presence or absence of sensor information. Then, the sensor information detection unit 1702 confirms the presence or absence of the signal detected by the sensor 1701 (S1806).
- the sensor information detection unit 1702 detects sensor information (YES in S1806)
- the sensor information detection unit 1702 acquires sensor information (S1807), and detects which part of the operation device 31 the user is gripping.
- the sensors 1701 which are touch sensors may be disposed on four sides, that is, two side surfaces on the long side and two side surfaces on the short side. Then, when the sensor information detection unit 1702 detects the contact by the sensors 1701 on the side surfaces on the two short sides, it is recognized that the controller device 31 is “held horizontally”. Then, when the sensor information detection unit 1702 detects a contact with one of the sensors 1701 on the side surface on the long side, it is recognized that the controller device 31 is "longitudinal”.
- the sensor information detection unit 1702 passes “horizontally held” or “longitudinally held” to the direction determination unit 1703 as a gripping direction.
- the direction determining unit 1703 checks whether the controller device 31 is “horizontally held” from the gripping direction received from the sensor information detection unit 1702 (S1808).
- the direction determining unit 1703 determines the direction of the controller device 31 using the touch information 1000 read out in the touch information reading process (S1805). Also, when the sensor information detection unit 1702 does not detect sensor information (NO in S1806), the direction determination unit 1703 determines the direction of the controller device 31 at the time of “horizontal holding”.
- the direction determining unit 1703 passes the determined direction of the operating device 31 to the cursor position determining unit 1704.
- the horizontal holding direction determination process will be described in detail later with reference to FIG.
- direction determining unit 1703 determines the orientation of operation device 31 at the time of "longitudinal” (S1810: process of determining the direction when vertically held) . In this case, the direction determining unit 1703 determines the direction of the controller device 31 using the touch information 1000 read out in the touch information reading process (S1805).
- the direction determining unit 1703 passes the determined direction of the operating device 31 to the cursor position determining unit 1704.
- the vertical holding direction determination process will be described in detail later with reference to FIG.
- the cursor position determination unit 1704 receives the direction of the controller device 31 from the direction determination unit 1703. Then, the cursor position determination unit 1704 determines whether or not the direction of the controller device 31 is the “right direction” (S1811).
- the cursor position determination unit 1704 executes a process (S 1813) of determining the position of the cursor on the screen.
- the cursor position determination unit 1704 corrects the position information 1003 of the touch information 1000 received from the touch information detection unit 901 (S 1812: Position information correction process). In this case, the cursor position determination unit 1704 corrects the position information 1003 based on the direction of the operation device 31 so that the cursor is displayed at the same position as the case where the direction of the operation device 31 is “right”. Do.
- the position information correction process is the same as that of the first embodiment.
- the cursor position determination unit 1704 uses the position information 1003 or the corrected position information 1003 of the touch information 1000 received from the touch information detection unit 901 to determine the cursor position on the screen (S 1813).
- the cursor position determination unit 1704 determines the coordinate system of the right touch pad 802 from the minimum value and the maximum value of the coordinate system of the right touch pad 802 and the minimum value and the maximum value in the coordinate system on the screen display. Acquire the enlargement ratio with the coordinate system on the screen display. In addition, the cursor position determination unit 1704 determines the coordinate system of the left touch pad 803 and the screen display from the minimum value and the maximum value of the coordinate system of the left touch pad 803 and the minimum value and the maximum value in the coordinate system on the screen display. Get the magnification ratio with the coordinate system of. Thereby, the cursor position determination unit 1704 determines the cursor position on the screen.
- the cursor position corresponding to the right touch pad 802 is determined by.
- the cursor position determination unit 1704 passes the x coordinate and the y coordinate of the determined cursor position to the screen data generation unit 905.
- the coordinate system on the screen display may be assigned to be different between the right touch pad 802 and the left touch pad 803.
- “the maximum value of the coordinate system on the screen display” and “the minimum value of the coordinate system on the screen display” are provided for the right touch pad 802 and the left touch pad 803, respectively. Thereby, the respective cursor positions are determined.
- the screen data generation unit 905 generates a screen data by superimposing a cursor image on the video and animation to be displayed on the display unit 906 based on the cursor position received from the cursor position determination unit 1704 (S1814). Thereafter, the screen data generation unit 905 passes the generated screen data to the display unit 906.
- the display unit 906 displays the screen data received from the screen data generation unit 905 on a screen or the like (S1815). After the processing is completed, the touch information detection unit 901 checks again whether or not to continue the detection of the input (S1801).
- position information 1003 included in the touch information 1000 read out in the touch information reading process (S1805) shown in FIG. 18 is arranged in time series, and position information of two adjacent points is connected by a line segment. It is described as "trajectory”.
- the direction determining unit 1703 executes a trajectory shape determination process of the touch information 1000 read from the touch information table 1100 with respect to the right touch pad 802 (S1901).
- the trajectory shape determination process will be described in detail later with reference to FIG.
- the direction determining unit 1703 executes trajectory shape determination processing of the touch information 1000 read from the touch information table 1100 with respect to the left touch pad 803 (S1902).
- the locus shape determination process is the same as the locus shape determination process (S1901) of the right touch pad.
- the direction determining unit 1703 determines the touch information 1000 and the left touch pad 803 related to the right touch pad 802 based on the processing result of the right touch pad locus shape determination processing (S1901) and the left touch pad locus shape determination processing (S1902). It is confirmed that there is both of the touch information 1000 and the related information (S1903).
- the direction determining unit 1703 determines whether the shapes of both trajectories match. It confirms (S1904). That is, the direction determining unit 1703 determines the shape of the locus of the right touch pad 802 and the shape of the left touch pad 803 from the processing results of the locus shape determination process of the right touch pad (S1901) and the locus shape determination process of the left touch pad (S1902). Check if the shape of the trajectory matches.
- the direction determining unit 1703 determines that both the shape of the locus of the right touch pad 802 and the shape of the locus of the left touch pad 803 are “convex upward” or both “convex downward”. Check if it is.
- the direction determining unit 1703 determines the trajectory shape determination process for the right touch pad (S1901) And from the processing result of the left touch pad locus shape determination processing (S1902), it is confirmed that there is either one of the touch information 1000 regarding the right touch pad 802 and the touch information 1000 regarding the left touch pad 803 (S1910).
- the direction determining unit 1703 executes the direction determining process using the shape of the trajectory when the touch information 1000 regarding either the right touch pad 802 or the left touch pad 803 is detected.
- the direction determining unit 1703 may execute the direction determining process only when the touch information 1000 related to both the right touch pad 802 and the left touch pad 803 is detected. In this case, this process (S1910) can be omitted.
- direction determining unit 1703 determines that the direction of operating device 31 is “indeterminate”. (S1909). Further, even when there is neither touch information 1000 related to the right touch pad 802 nor touch information 1000 related to the left touch pad 803 (NO in S1910), the direction determining unit 1703 determines that the direction of the operation device 31 is “undefined”. Decide.
- the direction determining unit 1703 may determine that the direction of the controller device 31 is the “right direction” as the default direction.
- the default direction may be a direction set in advance as a default value on the screen display device 3 at the time of shipping or the like, or a direction set in advance as a default value by the user.
- the direction determining unit 1703 determines that It is confirmed whether it is "" (S1905). Also, when there is either one of touch information 1000 regarding the right touch pad 802 and touch information 1000 regarding the left touch pad 803 (YES in S1910), the direction determining unit 1703 has the shape of the locus “convex upward”. Check if it is.
- the direction determining unit 1703 determines that the direction of the user's finger 804 is the right touch pad in view of the structural features of the human finger. It is determined that the touch panel 802 and the left touch pad 803 are in the upper direction. That is, the direction determining unit 1703 determines that the direction of the user's finger 804 is the same as the coordinate system of the right touch pad 802 and the coordinate system of the left touch pad 803.
- the direction determining unit 1703 determines that the direction of the controller device 31 is the “right direction” (S1906).
- the direction determining unit 1703 passes the determined direction of the operating device 31 to the cursor position determining unit 1704, and ends this processing.
- the direction determining unit 1703 confirms whether the shape of the locus confirmed in the processing of the previous stage is "convex downward" (S1907). ).
- the direction determining unit 1703 determines that the direction of the user's finger 804 is the right touch in view of the structural features of the human finger. It is determined that the pad 802 and the left touch pad 803 are in the downward direction. That is, the direction determining unit 1703 rotates the direction of the user's finger 804 by 180 ° opposite to the coordinate system of the right touch pad 802 and the coordinate system of the left touch pad 803, or up or down. I judge that there is.
- the direction determining unit 1703 determines that the direction of the operating device 31 is "reverse", as opposed to the case where the direction of the operating device 31 is determined to be "correct” (S1906) (S1908) .
- the direction determining unit 1703 passes the determined direction of the operating device 31 to the cursor position determining unit 1704, and ends this processing.
- the direction determining unit 1703 determines whether the operation device 31 has the same shape as in the case where the shapes of the loci of both touch pads do not match (NO in S1904). It is determined that the direction is "direction undefined" (S1909).
- the direction determining unit 1703 executes a trajectory shape determination process of the touch information 1000 read from the touch information table 1100 with respect to the right touch pad 802 (S2001).
- the trajectory shape determination process will be described in detail later with reference to FIG.
- the direction determining unit 1703 executes trajectory shape determination processing of the touch information 1000 read from the touch information table 1100 with respect to the left touch pad 803 (S2002).
- the locus shape determination process is the same as the locus shape determination process (S2001) of the right touch pad.
- the direction determining unit 1703 determines the touch information 1000 related to the right touch pad 802 and the left touch pad 803 based on the processing results of the right touch pad locus shape determination processing (S2001) and the left touch pad locus shape determination processing (S2002). It is confirmed that there is at least one of the touch information 1000 related to (S2003).
- the direction determining unit 1703 determines that the direction of the operating device 31 is “undefined”. And (S2008).
- the direction determining unit 1703 may determine that the direction of the controller device 31 is the “right direction” as the default direction.
- the default direction may be a direction set in advance as a default value on the screen display device 3 at the time of shipping or the like, or a direction set in advance as a default value by the user.
- the direction determining unit 1703 determines the shape of the locus checked in the preceding process. It is checked whether it is "convex upward" (S2004).
- the direction determining unit 1703 determines that the number of data of the touch information 1000 acquired from the touch information storage unit 902 is The touch information 1000 on the more touch pads may be used.
- direction determining unit 1703 determines that the user's It is determined that the direction of the finger 804 is the upward direction when the screen display device 3 is held vertically.
- the coordinate system of the right touch pad 802 and the coordinate system of the left touch pad 803 are defined in the direction in which the right touch pad 802 side is held in the forward direction.
- the orientation of the operating device 31 intended by the user is considered to coincide with the orientation of the finger 804 of the user. Therefore, the direction determining unit 1703 determines that the direction of the controller device 31 is the “right direction” (S2005). The direction determining unit 1703 passes the determined direction of the operating device 31 to the cursor position determining unit 1704, and ends this processing.
- the direction determining unit 1703 confirms whether the shape of the locus confirmed in the processing of the previous stage is “convex downward” (S2006).
- the direction determining unit 1703 displays the screen of the user's finger 804 in view of the structural features of the human finger, etc. It is determined that the device 3 is in the downward direction when held vertically.
- the direction determining unit 1703 determines that the direction of the operating device 31 is “reverse”, as opposed to the case where the direction of the operating device 31 is determined to be “right” (S2005) (S2007). .
- the direction determining unit 1703 passes the determined direction of the operating device 31 to the cursor position determining unit 1704, and ends this processing.
- the direction determining unit 1703 determines that the direction of the operation device 31 is "Similar to the case where there is no touch information for both touch pads (NO in S2003). It is determined that the direction is “indeterminate” (S2008).
- FIG. 21 the trajectory shape determination process for the right touch pad (S1901) and the trajectory shape determination process for the left touch pad (S1902) shown in FIG.
- the process flow of the process of determining the locus shape of the right touch pad (S2001) and the process of determining the locus shape of the left touch pad (S2002) will be described.
- the direction determining unit 1703 confirms the presence or absence of the touch information 1000 read from the touch information table 1100 (S2101).
- the direction determining unit 1703 ends this processing.
- the direction determining unit 1703 is a line segment connecting the position information 1003 of two points on the locus for the touch information 1000 read in the previous process (S2101).
- the slope .DELTA. Of is calculated (S2102).
- the direction determining unit 1703 calculates the degree of change ⁇ between two adjacent gradients ⁇ in time series (S2103), for the gradient ⁇ calculated in the previous process (S2102).
- the direction determining unit 1703 calculates the degree of change ⁇ for all of the slopes ⁇ calculated in the previous process (S2102).
- the direction determining unit 1703 determines the shape of the trajectory from the position information 1003, the inclination ⁇ , and the degree of change ⁇ (S2104). This process will be described again using FIGS. 15A and 15B used in the description in the second embodiment.
- FIGS. 15A and 15B determine the shape of the trajectory from the inclination ⁇ and the degree of change ⁇ , and further estimate the direction of the operation device 31 by estimating the direction of the user's finger from the shape of the trajectory It is the figure which showed the method.
- the direction of the operation device 31 has two patterns of “right direction” and “reverse direction”, FIG. 15A shows “right direction” and FIG. 15B shows “reverse direction”.
- the direction determining unit 1703 determines whether the value of the slope ⁇ calculated in the process (S2102) of calculating the slope ⁇ of the flow of the direction determining process shown in FIG. The difference “ ⁇ y” of the y-coordinates and the sign of the value of the degree of change ⁇ calculated in the process of calculating the degree of change ⁇ (S2103) are determined.
- the positive / negative discrimination results are shown in the form of a table, with positive being indicated by “+” and negative being indicated by “ ⁇ ”.
- the slope ⁇ or the change degree ⁇ when the slope ⁇ or the change degree ⁇ is 0, it may be considered as a special state represented by “ ⁇ ”.
- the slope ⁇ can not be calculated because the slope ⁇ is infinite or when the ⁇ x becomes zero. These cases may also be considered as features of the shape of the trajectory.
- the slope ⁇ changes from “+” to “ ⁇ ”, and the degrees of change ⁇ are all “ ⁇ ”.
- the direction determining unit 1703 determines that the trajectory has a “convex upward” shape, also in consideration of the values of ⁇ x and ⁇ y.
- the slope ⁇ changes from “ ⁇ ” to “+”, and all the degrees of change ⁇ are “+”.
- the direction determining unit 1703 determines that the trajectory has a “convex downward” shape in consideration of the values of ⁇ x and ⁇ y.
- the direction determining unit 1703 determines that the shape of the trajectory is “undefined”. For example, as a case where the direction determination unit 1703 can not determine the shape of the trajectory, the number of pieces of position information 1003 included in the trajectory may be small.
- direction determining unit 1703 is closest to either of FIGS. 15A and 15B. It may be determined. Accordingly, the direction determining unit 1703 may determine the shape of the trajectory as illustrated in FIG. 15A or 15B.
- the screen display device 3 detects the sensor information from the sensor 1701 by the sensor information detection unit 1702, and thereby recognizes the gripping direction of the operation device 31. Then, the touch information storage unit 902 stores touch information 1000 on the two touch pads of the right touch pad 802 and the left touch pad 803 detected by the touch information detection unit 901.
- the direction determination unit 1703 determines the shape of the locus of both or one of the right touch pad 802 and the left touch pad 803. The orientation of the operating device 31 is determined using this.
- the cursor position determination unit 1704 appropriately corrects the position information 1003 included in the touch information 1000 received from the touch information detection unit 901 according to the gripping direction and the direction of the operation device 31, and then the display position of the cursor 806 is displayed. It is decided.
- the screen display device 3 is only for the user to input the right touch pad 802 and the left touch pad 803 regardless of the orientation in which the user holds the operation device 31 in both horizontal holding and vertical holding. Based on this, the orientation of the operating device 31 can be determined appropriately and automatically to display the cursor 806 at the correct position. Thereby, the comfortable operability of the user can be realized.
- the screen display apparatus includes two touch pads, a sensor for detecting which part of the operating device is gripped by the user, and a sensor for detecting the gravity direction of the operating device. Prepare. Then, the screen display device recognizes the direction of the operation device using the user's input on the touch pad of the screen display device and the detection result of the gripping direction by the sensor, and recognizes the direction of the operation result by the recognition result and the sensor. An alert is displayed on the screen when the detection result of the direction does not match.
- FIG. 22 is a block diagram of the screen display device in the fourth embodiment.
- the same components as in FIG. 17 will be assigned the same reference numerals and descriptions thereof will be omitted.
- the screen display device 4 shown in FIG. 22 includes a warning data generation unit 2201, a screen data generation unit 2202, a cursor position determination unit 2203, a first sensor 2204, a second sensor 2205, and a right touch pad.
- a left touch pad 803, a touch information detection unit 901, a touch information storage unit 902, a display unit 906, a sensor information detection unit 1702, and a direction determination unit 1703 are provided.
- the operating device 41 shown in FIG. 22 includes a first sensor 2204, a second sensor 2205, a right touch pad 802, and a left touch pad 803.
- the display control device 40 further includes a warning data generation unit 2201, a screen data generation unit 2202, a touch information detection unit 901, a touch information storage unit 902, a cursor position determination unit 2203, and a sensor information detection unit 1702. And a direction determination unit 1703.
- the display control device 40 receives a signal from the operation device 41, and generates screen data to be displayed on the screen.
- the display control device 40 may be incorporated in the operation device 41 or the display unit 906.
- the touch information detection unit 901 is included in the display control device 40, but may be included in the operation device 41.
- the alert data generation unit 2201 determines that the user holds the operating device 41 in the “reverse direction” based on the direction of the operating device 41 determined by the direction determining unit 1703, and displays an alert message for presenting the user ( Generate warning data).
- the alert data generation unit 2201 may generate an alert message when the directions of the coordinate system of the right touch pad 802 and the coordinate system of the left touch pad 803 do not match.
- various feedback methods such as changing the color or shape of an object on the screen as well as the alert message can be applied. Furthermore, not only the screen display but also the method by sound is applicable.
- the screen data generation unit 2202 is means for generating screen data based on the message generated by the warning data generation unit 2201 and the display position of the cursor determined by the cursor position determination unit 2203.
- the screen data generation unit 2202 superimposes an image of an application such as a moving image viewer or a web browser with a cursor image, and further, when an alert message is received from the warning data generation unit 2201, the alert message is displayed as screen data. Superimpose. Thus, the screen data generation unit 2202 generates screen data to be displayed on the display unit 906.
- the cursor position determination unit 2203 receives the gripping direction of the operation device 41 received from the direction determination unit 1703, the orientation of the operation device 41 determined by the direction determination unit 1703, and the touch information detected by the touch information detection unit 901 on the screen. Means for calculating the position of the cursor displayed on the screen.
- the cursor position determination unit 2203 converts (coordinates) the value of the position information included in the touch information based on the gripping direction (horizontally or vertically held) of the operation device 41. Furthermore, when it is determined by the direction determination unit 1703 that the user does not hold the operating device 41 in the “right direction”, the warning data generation unit 2201 is warned of the user, or the display device 4 is replaced Request to generate an alert message to prompt you.
- the conversion of the position information based on the holding direction (horizontal holding or vertical holding) of the operation device 41 is processing for converting the position information into the coordinate system of the holding direction set as a reference.
- the coordinate system of the right touch pad 802 and the left touch pad 803 is set based on the vertical holding of the operation device 41 (the holding direction of the remote control 1601 shown in FIG. 16), and the operation device When 41 is held horizontally, the position information is converted.
- the cursor position determination unit 2203 rotates the coordinate system of the right touch pad 802 and the left touch pad 803 by 90 degrees to the left in order to correspond to the coordinate system of the gripping direction set as the reference. That is, the cursor position determination unit 2203 rotates the position information by 90 ° to the left about the center coordinates of the coordinate systems of the right touch pad 802 and the left touch pad 803 as the center of rotation.
- the first sensor 2204 is a touch sensor for detecting a touch of a user's finger.
- One or more touch sensors are disposed on the outer peripheral portion of the operation device 41.
- toe is a well-known technique, detailed description is abbreviate
- the second sensor 2205 is a gravity sensor for detecting the direction of gravity of the operating device 41, and is incorporated in the operating device 41.
- the gravity sensor which recognizes direction of the top-bottom direction of the operating device 41 by detecting the direction of gravity concerning such an operating device 41 is a well-known technique, detailed description is abbreviate
- the sensor information detection unit 1702 detects, from the first sensor 2204, holding portion information which is information of a portion where the user holds the operating device 41, and the second sensor 2205 detects the direction of gravity applied to the operating device 41.
- the gravity direction information which is information of is detected.
- the gravity direction information may include information on the inclination of the operating device 41 with respect to the gravity direction. Further, the gravity direction information may include information on the vertical direction of the operation device 41, that is, information indicating which surface of the operation device 41 is the upper side or the lower side with respect to the gravity direction.
- the cursor position determination unit 2203 determines the display position (cursor position) of the cursor on the screen using the position information received from the touch information detection unit 901 or the position information after conversion by the means. .
- FIG. 22 The contents of the "horizontal holding direction determination process (S2308)" and the “longitudinal holding direction determination process (S2309)" illustrated in FIG. 23 are the contents described using FIG. 19 to FIG. 21 in the third embodiment. Is the same as Therefore, the description is omitted here as referring to the description of FIGS. 19 to 21 in the third embodiment.
- the “screen display device 3” is the “screen display device 4”
- the components of the third embodiment such as the “cursor position determination unit 1704” are , “Cursor position determination unit 2203", and the like.
- the touch information detection unit 901 of the screen display device 4 confirms whether or not to continue detection of an input to the right touch pad 802 or the left touch pad 803 based on an instruction from an instruction unit (not shown in FIG. 22). (S2301).
- the screen display device 4 ends the process.
- touch information detection unit 901 when touch information detection unit 901 is instructed to continue detection of an input to right touch pad 802 or left touch pad 803 (YES in S2301), touch information detection unit 901 detects right touch pad 802 or the left touch pad. The presence or absence of an input to the touch pad 803 is confirmed. Thereby, the touch information detection unit 901 detects the presence or absence of the touch information 1000 shown in FIG. 10 (S2302).
- the touch information detection unit 901 detects “no touch information” (NO in S2302)
- the touch information detection unit 901 checks again whether or not to continue the detection of the input (S2301).
- the touch information storage unit 902 stores the touch information 1000 (S2303). More specifically, the touch information detection unit 901 adds the touch information 1000 to the touch information table 1100 shown in FIG. In addition, the touch information detection unit 901 also passes the detected touch information 1000 to the cursor position determination unit 2203 in order to display the cursor on the screen.
- the cursor position determination unit 2203 receives the touch information 1000 from the touch information detection unit 901. Then, in order to determine whether or not it is necessary to correct the position information 1003 included in the touch information 1000, the direction determination unit 1703 is inquired of the direction of the operation device 41 (S2304).
- the direction determining unit 1703 receives an inquiry from the cursor position determining unit 2203. Then, the direction determining unit 1703 accesses the touch information table 1100 of the touch information storage unit 902, and reads the touch information 1000 (S2305). At this time, the direction determining unit 1703 reads the touch information 1000 or the touch information 1000 of a predetermined amount for a predetermined time set in the direction determining unit 1703 in advance.
- the touch information storage unit 902 sequentially stores touch information 1000 received from the touch information detection unit 901 at predetermined time intervals. Therefore, the amount of touch information for a predetermined time can be obtained by calculation.
- touch information storage section 902 may hold touch information 1000 that has been read once inside direction determination section 1703 or the like. Then, the touch information storage unit 902 may read out only the touch information 1000 of the difference which is newly required. This reduces the amount of touch information 1000 read out later.
- the direction determination unit 1703 queries the sensor information detection unit 1702 about the presence or absence of sensor information from the first sensor 2204.
- the sensor information detection unit 1702 confirms the presence or absence of the signal detected by the first sensor 2204 (S2306).
- the sensor information detection unit 1702 detects sensor information from the first sensor 2204 (YES in S2306)
- the sensor information detection unit 1702 acquires sensor information from the first sensor 2204. Then, the sensor information detection unit 1702 detects which part of the operating device 41 the user is gripping, and confirms whether the operating device 41 is “horizontally held” (S2307).
- the sensors 1701 which are touch sensors may be disposed on four sides, that is, two side surfaces on the long side and two side surfaces on the short side. Then, when the sensor information detection unit 1702 detects the contact by the sensors 1701 on the side surfaces on the two short sides, it is recognized that the controller device 41 is “horizontally held”. Then, when the sensor information detection unit 1702 detects a contact with either of the sensors 1701 on the side surface on the long side, it is recognized that the controller device 41 is “longitudinally held”.
- the sensor information detection unit 1702 passes “horizontally held” or “longitudinally held” to the direction determination unit 1703 as a gripping direction.
- the direction The determination unit 1703 uses the touch information 1000 read in the touch information reading process (S2305) to determine the orientation of the operation device 41 at the time of “horizontal holding” (S2308: horizontal holding direction determination process).
- the direction determining unit 1703 passes the determined direction of the operating device 41 to the cursor position determining unit 2203.
- This horizontal holding direction determination process is equivalent to the process described with reference to FIG. 19 in the third embodiment.
- the direction determining unit 1703 uses the touch information 1000 read in the touch information reading process (S2305) to "longitudinally hold".
- the direction of the operation device 41 is determined (S2309: processing for determining the vertical holding direction).
- the direction determining unit 1703 passes the determined direction of the operating device 41 to the cursor position determining unit 2203.
- the vertical holding direction determination process is the same as the process described with reference to FIG. 20 in the third embodiment.
- the cursor position determination unit 2203 inquires the direction determination unit 1703 whether or not acquisition of sensor information from the second sensor 2205 is performed. Then, the direction determining unit 1703 checks the sensor information detection unit 1702 for the presence or absence of sensor information from the second sensor 2205 (S2310).
- the cursor position determination unit 2203 receives two directions. That is, the cursor position determination unit 2203 causes the direction determination unit 1703 to set the orientation of the controller 41, which is the result of the horizontal holding direction determination process (S2308) or the vertical holding direction determination process (S2309), and the second sensor 2205. And the direction of the operating device 41 to be detected. The cursor position determination unit 2203 determines whether the directions of the two coincide with each other (S2311).
- the cursor position determination unit 2203 requests the alert data generation unit 2201 to generate an alert message.
- the alert data generation unit 2201 generates an alert message for notifying the user that the orientation of the controller device 41 is reverse or for causing the controller device 41 to recognize the orientation (S2316).
- the alert data generation unit 2201 generates an alert message such as "Please check whether the remote control is held in the correct direction.” "Reverse the direction of the remote control.” Pass the generated alert message to the part 2202.
- the screen data generation unit 2202 When the screen data generation unit 2202 receives the alert message from the warning data generation unit 2201, the screen data generation unit 2202 superimposes the alert message on the screen data (S2317).
- the cursor position determination unit 2203 holds the operation device 41 in the gripping direction (horizontally or vertically held).
- the position information 1003 is converted on the basis of.
- the cursor position determining unit 2203 determines the position information 1003 based on the gripping direction of the operating device 41. Convert.
- the cursor position determination unit 2203 converts the position information 1003 of the touch information 1000 received from the touch information detection unit 901. Specifically, when the holding direction of the operation device 41 is “longitudinally held”, the cursor position determination unit 2203 performs position information with the center coordinates of the coordinate systems of the right touch pad 802 and the left touch pad 803 as the center of rotation. Rotate 90 degrees to the left. This provides an appropriate cursor position.
- the cursor position determination unit 2203 corrects the position information 1003 of the touch information 1000 received from the touch information detection unit 901 (S2312: position information correction process). At this time, the cursor position determination unit 2203 determines the position based on the direction of the operating device 41 which is the result of the horizontal holding direction determination processing (S2308) or the vertical holding direction determination processing (S2309) received from the direction determination unit 1703. The information 1003 is corrected.
- the cursor position determination unit 2203 corrects the position information 1003 so that the cursor is displayed at the same position as the case where the direction of the controller device 41 is the “right direction”.
- the position information correction process is the same as that of the first embodiment.
- the cursor position determination unit 2203 determines the cursor position on the screen (S2313). Specifically, the cursor position determination unit 2203 determines the coordinate system of the right touch pad 802 from the minimum value and the maximum value of the coordinate system of the right touch pad 802 and the minimum value and the maximum value in the coordinate system on the screen display. Acquire the enlargement ratio with the coordinate system on the screen display. The cursor position determination unit 2203 also uses the coordinate system of the left touch pad 803 and the screen display from the minimum value and the maximum value of the coordinate system of the left touch pad 803 and the minimum value and the maximum value in the coordinate system on the screen display. Get the magnification ratio with the coordinate system of. Thereby, the cursor position determination unit 2203 determines the cursor position on the screen.
- the cursor position corresponding to the right touch pad 802 is determined by.
- the cursor position determination unit 2203 passes the x coordinate and the y coordinate of the determined cursor position to the screen data generation unit 2202.
- the coordinate system on the screen display may be assigned to be different between the right touch pad 802 and the left touch pad 803.
- “the maximum value of the coordinate system on the screen display” and “the minimum value of the coordinate system on the screen display” are provided for the right touch pad 802 and the left touch pad 803, respectively. Thereby, the respective cursor positions are determined.
- the screen data generation unit 2202 generates a screen data by superimposing a cursor image on video and animation to be displayed on the display unit 906 (S2314) ).
- the display unit 906 displays the screen data received from the screen data generation unit 2202 on a screen or the like (S2315).
- the touch information detection unit 901 checks again whether or not to continue the detection of the input (S2301).
- the cursor position determination unit 2203 does not perform correction of the position information (S2312), and an alert is generated.
- the generation of a message (S2316) may be requested.
- the cursor position determination unit 2203 may determine the cursor position on the screen (S2313) without performing the correction.
- the touch information storage unit 902 of the screen display device 4 stores touch information 1000 on the two touch pads of the right touch pad 802 and the left touch pad 803 detected by the touch information detection unit 901. Do. Then, from the position information 1003 included in the touch information 1000, the direction determining unit 1703 determines the direction of the controller device 41 using the shapes of the loci of the right touch pad 802 and the left touch pad 803.
- the direction of the operating device 41 determined by the direction determining unit 1703 and the direction of the operating device 41 detected by the second sensor 2205 are collated. Then, when the directions of the two do not match, an alert message is displayed on the screen. Therefore, when it is determined that the user has held the operating device 41 by mistake, the screen display device 4 utilizes the input on the right touch pad 802 and the left touch pad 803 by the user, and is appropriate and quick for the user. Can be urged to change the holding of the operation device 41 or the like.
- the screen display device having two touch pads recognizes the orientation of the controller device using the distribution of positional information (coordinates) indicating the user's input on the touch pad of the screen display device. Then, the screen display device corrects the display position of the cursor on the screen based on the direction of the operation device.
- FIG. 24 is a block diagram of the screen display device in the fifth embodiment.
- the same components as in FIG. 17 will be assigned the same reference numerals and descriptions thereof will be omitted.
- the screen display device 5 shown in FIG. 24 includes a direction determining unit 2401, a right touch pad 802, a left touch pad 803, a touch information detection unit 901, a touch information storage unit 902, and a screen data generation unit 905. , A display unit 906, a sensor 1701, a sensor information detection unit 1702, and a cursor position determination unit 1704.
- the operating device 31 shown in FIG. 24 includes a right touch pad 802, a left touch pad 803, and a sensor 1701. Further, the display control device 50 includes a direction determination unit 2401, a touch information detection unit 901, a touch information storage unit 902, a screen data generation unit 905, a sensor information detection unit 1702, and a cursor position determination unit 1704. .
- the display control device 50 receives a signal from the operation device 31 and generates screen data to be displayed on the screen.
- the display control device 50 may be incorporated in the operation device 31 or the display unit 906.
- the touch information detection unit 901 is included in the display control device 50, the touch information detection unit 901 may be included in the operation device 31.
- Direction determining unit 2401 determines the direction of controller 31 based on the user's grip position of controller 31 detected by sensor information detector 1702 and touch information stored in touch information storage unit 902. It is a means. Specifically, the direction determining unit 2401 determines which of the horizontally held and vertically held gripping directions the operating device 31 has from the gripping position detected by the sensor information detection unit 1702. Then, based on the distribution of positional information included in touch information such as a predetermined time or a predetermined amount, the direction determining unit 2401 determines which direction the operating device 31 is facing.
- the direction determining unit 2401 determines the direction of the operation device 31 using touch information obtained by an operation by which the user naturally touches the touch pad. can do.
- touch information is obtained when the user starts holding the operating device 31, when switching the application, when the holding direction of the operating device 31 changes in the same application, or the like.
- FIG. 25 the processing flow of the screen display apparatus 5 in the fifth embodiment shown in FIG. 24 will be described using FIGS. 25 to 28.
- FIG. 25 the processing flow of the screen display apparatus 5 in the fifth embodiment shown in FIG. 24 will be described using FIGS. 25 to 28.
- FIG. 25 is an entire processing flow of the screen display device 5 in the fifth embodiment shown in FIG.
- the touch information detection unit 901 of the screen display device 5 confirms whether to continue detection of an input to the right touch pad 802 or the left touch pad 803 based on an instruction from an instruction unit (not shown in FIG. 24). (S2501).
- the screen display apparatus 5 ends the process.
- touch information detection unit 901 when touch information detection unit 901 is instructed to continue detection of an input to right touch pad 802 or left touch pad 803 (YES in S2501), touch information detection unit 901 detects right touch pad 802 or the left touch pad. The presence or absence of an input to the touch pad 803 is confirmed. Thereby, the touch information detection unit 901 detects the presence or absence of the touch information 1000 shown in FIG. 10 (S2502).
- the touch information detection unit 901 detects “no touch information” (NO in S2502)
- the touch information detection unit 901 checks again whether or not to continue the detection of the input (S2501).
- the touch information storage unit 902 causes the touch information 1000 to be stored (S2503). More specifically, the touch information detection unit 901 adds the touch information 1000 to the touch information table 1100 shown in FIG. The touch information detection unit 901 also passes the detected touch information 1000 to the cursor position determination unit 1704 for displaying a cursor on the screen.
- the cursor position determination unit 1704 receives the touch information 1000 from the touch information detection unit 901. Then, the cursor position determination unit 1704 inquires the direction determination unit 2401 about the direction of the operation device 31 to determine whether it is necessary to correct the position information 1003 included in the touch information 1000 (S 2504). .
- the direction determining unit 2401 receives an inquiry from the cursor position determining unit 1704. Then, the direction determining unit 2401 accesses the touch information table 1100 of the touch information storage unit 902, and reads the touch information 1000 (S2505). At this time, the direction determining unit 2401 reads the touch information 1000 for a predetermined time set in the direction determining unit 2401 in advance or the touch information 1000 of a predetermined amount.
- the touch information storage unit 902 sequentially stores touch information 1000 received from the touch information detection unit 901 at predetermined time intervals. Therefore, the amount of touch information for a predetermined time can be obtained by calculation.
- the touch information storage unit 902 may hold the touch information 1000 once read in the direction determination unit 2401 or the like. Then, the touch information storage unit 902 may read out only the touch information 1000 of the difference which is newly required. This reduces the amount of touch information 1000 read out later.
- the direction determining unit 2401 inquires of the sensor information detection unit 1702 whether or not there is sensor information. Then, the sensor information detection unit 1702 confirms the presence or absence of the signal detected by the sensor 1701 (S2506).
- the sensor information detection unit 1702 detects sensor information (YES in S2506)
- the sensor information detection unit 1702 acquires the sensor information and detects which part of the operation device 31 the user is gripping (S2507).
- the sensors 1701 which are touch sensors may be disposed on four sides, that is, two side surfaces on the long side and two side surfaces on the short side. Then, when the sensor information detection unit 1702 detects the contact by the sensors 1701 on the side surfaces on the two short sides, it is recognized that the controller device 31 is “held horizontally”. Then, when the sensor information detection unit 1702 detects a contact with one of the sensors 1701 on the side surface on the long side, it is recognized that the controller device 31 is "longitudinal”.
- the sensor information detection unit 1702 passes “horizontally held” or “longitudinally held” as a gripping direction to the direction determining unit 2401.
- the direction determining unit 2401 checks whether or not the controller device 31 is “horizontally held” from the holding direction received from the sensor information detection unit 1702 (S2508).
- the direction determining unit 2401 determines the direction of the controller device 31 at the time of "horizontally held” (S2509: processing for determining a direction when holding horizontal). In this case, the direction determining unit 2401 uses the touch information 1000 read in the touch information reading process (S2505) to determine the direction of the operation device 31 at the time of “horizontal holding”.
- the direction determination unit 2401 determines the direction of the operation device 31 at the time of “horizontal holding” using the touch information 1000. . Thereafter, the direction determining unit 2401 passes the determined direction of the operating device 31 to the cursor position determining unit 1704. This horizontal holding direction determination process will be described in detail later with reference to FIG.
- direction determination unit 2401 uses the touch information 1000 read in the touch information reading process (S2505) to operate the operation device in “longitudinal hold”
- the direction of 31 is determined (S2510: processing for determining the vertical holding direction).
- the direction determining unit 2401 passes the determined direction of the operating device 31 to the cursor position determining unit 1704.
- the vertical holding direction determination process will be described in detail later with reference to FIG.
- the cursor position determination unit 1704 receives the direction of the controller device 31 from the direction determination unit 2401. Then, the cursor position determination unit 1704 determines whether or not the direction of the controller device 31 is the “right direction” (S2511).
- the cursor position determination unit 1704 executes a process (S2513) of determining the cursor position on the screen.
- the cursor position determination unit 1704 corrects the position information 1003 of the touch information 1000 received from the touch information detection unit 901 (S2512: Position information correction process). In this case, the cursor position determination unit 1704 corrects the position information 1003 based on the direction of the operation device 31 so that the cursor is displayed at the same position as the case where the direction of the operation device 31 is “right”. Do.
- the position information correction process is the same as that of the first embodiment.
- the cursor position determination unit 1704 determines the cursor position on the screen using the position information 1003 of the touch information 1000 received from the touch information detection unit 901 or the corrected position information 1003 (S2513). .
- the cursor position determination unit 1704 determines the coordinate system of the right touch pad 802 from the minimum value and the maximum value of the coordinate system of the right touch pad 802 and the minimum value and the maximum value in the coordinate system on the screen display. Acquire the enlargement ratio with the coordinate system on the screen display. In addition, the cursor position determination unit 1704 determines the coordinate system of the left touch pad 803 and the screen display from the minimum value and the maximum value of the coordinate system of the left touch pad 803 and the minimum value and the maximum value in the coordinate system on the screen display. Get the magnification ratio with the coordinate system of. Thereby, the cursor position determination unit 1704 determines the cursor position on the screen.
- the cursor position corresponding to the right touch pad 802 is determined by.
- the cursor position determination unit 1704 passes the x coordinate and the y coordinate of the determined cursor position to the screen data generation unit 905.
- the coordinate system on the screen display may be assigned to be different between the right touch pad 802 and the left touch pad 803.
- “the maximum value of the coordinate system on the screen display” and “the minimum value of the coordinate system on the screen display” are provided for the right touch pad 802 and the left touch pad 803, respectively. Thereby, the respective cursor positions are determined.
- the screen data generation unit 905 generates a screen data by superimposing a cursor image on the video and animation to be displayed on the display unit 906 based on the cursor position received from the cursor position determination unit 1704 (S 2514). Thereafter, the screen data generation unit 905 passes the generated screen data to the display unit 906.
- the display unit 906 displays the screen data received from the screen data generation unit 905 on a screen or the like (S 2515). After the processing ends, the touch information detection unit 901 checks again whether or not to continue the detection of the input (S2501).
- the direction determining unit 2401 executes position information distribution determination processing (S2601) of the touch information 1000 read from the touch information table 1100 with respect to the right touch pad 802.
- the position information distribution determination process will be described in detail later with reference to FIG.
- the direction determining unit 2401 executes position information distribution determination processing (S2602) of the touch information 1000 read from the touch information table 1100 for the left touch pad 803.
- This position information distribution determination process is the same as the position information distribution determination process (S2601) of the right touch pad, but when determining the distribution of position information, the distribution for the left finger described later with reference to FIG. Use
- the direction determining unit 2401 determines from the processing result of the position information distribution determination process of the right touch pad (S2601) and the position information distribution determination process of the left touch pad (S2602) It is confirmed that both the touch information 1000 related to the pad 803 is present (S2603).
- the direction determining unit 2401 confirms whether the two determination results match ((2) S2604). That is, the direction determining unit 2401 determines the distribution result of the position information of the right touch pad 802 from the processing result of the position information distribution determination processing of the right touch pad (S2601) and the position information distribution determination processing of the left touch pad (S2602). It is checked whether the determination result of the distribution of the position information of the left touch pad 803 matches. Specifically, the direction determining unit 2401 checks whether the determination results of the right touch pad 802 and the left touch pad 803 are both “up” or both “down”.
- the direction determining unit 2401 checks whether any of them is present or not. (S2610). That is, in this case, the direction determining unit 2401 determines the touch information 1000 related to the right touch pad 802 from the processing results of the position information distribution determination processing of the right touch pad (S2601) and the position information distribution determination processing of the left touch pad (S2602). It is confirmed that there is either one of the touch information 1000 on the left touch pad 803.
- the direction determining unit 2401 executes the direction determining process using the distribution when the touch information 1000 related to either the right touch pad 802 or the left touch pad 803 is detected.
- the direction determining unit 2401 may execute the direction determining process only when the touch information 1000 related to both the right touch pad 802 and the left touch pad 803 is detected. In this case, this process (S2610) can be omitted.
- the direction determining unit 2401 determines that the direction of the operation device 31 is “direction It is determined that "undefined” (S2609). In addition, also when there is neither touch information 1000 related to the right touch pad 802 nor touch information 1000 related to the left touch pad 803 (NO in S2610), the direction determining unit 2401 determines that the direction of the operation device 31 is "undefined”. Decide.
- the direction determining unit 2401 may determine that the direction of the controller device 31 is the “right direction” as the default direction.
- the default direction may be a direction set in advance as a default value on the screen display device 5 at the time of shipping or the like, or may be a direction set in advance as a default value by the user.
- the direction determining unit 2401 detects the position information confirmed in the previous processing. It is checked whether or not the determination result of the distribution is “up” (S2605). Also, when there is either one of the touch information 1000 on the right touch pad 802 and the touch information 1000 on the left touch pad 803 (YES in S2610), the direction determining unit 2401 determines that the determination result of the distribution of position information is "upward”. To see if it is.
- the direction determining unit 2401 determines that the direction of the user's finger 804 is right in view of the structural feature of the human finger. It is determined that the touch pad 802 and the left touch pad 803 are in the upper direction. That is, the direction determining unit 2401 determines that the direction of the user's finger 804 is the same direction as the coordinate system of the right touch pad 802 and the coordinate system of the left touch pad 803.
- the direction determining unit 2401 determines that the direction of the controller device 31 is the “right direction” (S2606).
- the direction determining unit 2401 passes the determined direction of the operating device 31 to the cursor position determining unit 1704 and ends this processing.
- the direction determining unit 2401 determines whether the determination result of the distribution of the position information confirmed in the previous process is "downward”. It confirms (S2607).
- the direction determining unit 2401 determines the direction of the user's finger 804 in view of the structural feature of the human finger, etc. It is determined that the right touch pad 802 and the left touch pad 803 are in the downward direction. That is, the direction determining unit 2401 rotates the user's finger 804 in the direction up and down opposite to the coordinate system of the right touch pad 802 and the coordinate system of the left touch pad 803 or to the right or left by 180 °. I will judge.
- the direction determining unit 2401 determines that the direction of the operating device 31 is "reversed” (S2608), contrary to the case where the direction of the operating device 31 is determined to be “correct” (S2606). .
- the direction determining unit 2401 passes the determined direction of the operating device 31 to the cursor position determining unit 1704 and ends this processing.
- the direction determining unit 2401 operates the operating device, as in the case where the position information distributions of both touch pads do not match (NO in S2604) It is determined that the direction of 31 is "direction undefined" (S2609).
- the direction determining unit 2401 executes position information distribution determination processing (S 2701) of the touch information 1000 read from the touch information table 1100 with respect to the right touch pad 802.
- the position information distribution determination process will be described in detail later with reference to FIG.
- the direction determining unit 2401 executes position information distribution determination processing (S2702) of the touch information 1000 read from the touch information table 1100 regarding the left touch pad 803.
- This position information distribution determination processing is the same as the position information distribution determination processing (S 2701) of the right touch pad, but when determining the distribution of position information, the distribution for the left finger described later with reference to FIG. Use
- the direction determining unit 2401 determines from the processing results of position information distribution determination processing of the right touch pad (S2701) and position information distribution determination processing of the left touch pad (S2702) It is confirmed that there is at least one of the touch information 1000 related to the pad 803 (S2703).
- the direction of the operation device 31 is determined to be "direction undefined" (S2708) .
- the direction determining unit 2401 may determine that the direction of the controller device 31 is the “right direction” as the default direction.
- the default direction may be a direction set in advance as a default value on the screen display device 5 at the time of shipping or the like, or may be a direction set in advance as a default value by the user.
- the direction determining unit 2401 distributes the position information checked in the preceding process. It is confirmed whether the determination result of is "up" (S2704).
- the direction determining unit 2401 determines that the number of data of the touch information 1000 acquired from the touch information storage unit 902 is The touch information 1000 on the more touch pads may be used.
- the direction determining unit 2401 determines the direction of the user's finger 804 in view of the structural feature of the human finger, etc. It is determined that it is upward when the controller 31 is held vertically.
- the coordinate system of the right touch pad 802 and the coordinate system of the left touch pad 803 are defined in the direction in which the right touch pad 802 side is held in the forward direction.
- the orientation of the operating device 31 intended by the user is considered to coincide with the orientation of the finger 804 of the user. Therefore, the direction determining unit 2401 determines that the direction of the controller device 31 is the “right direction” (S2705). The direction determining unit 2401 passes the determined direction of the operating device 31 to the cursor position determining unit 1704 and ends this processing.
- the direction determining unit 2401 determines whether the determination result of the distribution of the position information confirmed in the preceding process is "down". It confirms (S2706).
- the direction determining unit 2401 determines the direction of the user's finger 804 in view of the structural feature of the human finger, etc. It is determined that it is downward when the controller 31 is held vertically.
- the direction determining unit 2401 determines that the direction of the operating device 31 is "reverse", as opposed to the case where the direction of the operating device 31 is determined to be "correct” (S2705) (S2707) .
- the direction determining unit 2401 passes the determined direction of the operating device 31 to the cursor position determining unit 1704 and ends this processing.
- the direction determining unit 2401 determines the direction of the operating device 31 as in the case where the touch information of both touch pads is absent (NO in S2703). Is determined to be "direction undefined” (S2708).
- FIG. 28 the position information distribution determination process (S2601) of the right touch pad and the position information distribution determination process (S2602) of the left touch pad shown in FIG.
- a processing flow of position information distribution determination processing (S2701) of the right touch pad and position information distribution determination processing (S2702) of the left touch pad shown in FIG. 27 will be described.
- the direction determining unit 2401 confirms the presence or absence of the touch information 1000 read from the touch information table 1100 (S2801).
- the direction determining unit 2401 determines that the direction of the operation device 31 is “undefined direction” (S2808).
- the direction determining unit 2401 reads the distribution of position information held internally (S2802).
- the direction determining unit 2401 determines the distribution of the position information of the right touch pad 802
- the direction determining unit 2401 reverses the distribution (for the right finger) of the position information in the correct direction shown in FIGS. 29A and 29B. Read out the distribution of position information (for the right finger).
- the direction determining unit 2401 determines the distribution of position information of the left touch pad 803, the distribution of position information (for the left finger) in the correct direction shown in FIGS. 29A and 29B and the position information in the opposite direction Read out the distribution (for the left finger).
- the direction determining unit 2401 calculates the degree of correlation between the touch information 1000 read in the touch information confirmation process (S2801) and the position information distribution read in the position information distribution read process (S2802) ( S2803).
- the calculation of the degree of correlation includes the number of position information 1003 included in the distribution of position information in the correct orientation read out in the distribution information reading process (S2802) and the position information included in the distribution of position information in the reverse direction. The number of 1003 is calculated respectively.
- FIGS. 29A and 29B ten pieces of position information 1003 from point A to point J (five pieces of position information 1003 from the right touch pad 802, and position information 1003 from the left touch pad 803). Is read out as the touch information 1000).
- FIG. 29A ten of the points A to J are included in the “distribution of positional information in the correct direction”, so the degree of correlation can be calculated as “10”.
- FIG. 29B since four points of point C, point D, point G and point I are included in the “distribution of positional information in reverse direction”, the degree of correlation can be calculated as “4”.
- the direction determining unit 2401 confirms whether the degree of correlation is “0” with respect to the degree of correlation calculated in the calculation process of the degree of correlation (S2803) (S2804).
- the direction determining unit 2401 can not determine the direction of the controller 31 from the correlation, and the direction of the controller 31 is “indeterminate”. It is determined that there is (S2808).
- the direction determining unit 2401 determines the position of the correct direction with respect to the correlation calculated in the correlation calculation process (S2803). It is checked whether or not the degree of correlation with the information) ⁇ (the degree of correlation with the position information in the reverse direction) (S2805).
- the direction determining unit 2401 determines the position included in the touch information 1000. Since the information 1003 has a high degree of correlation with the position information at the time of correct orientation, it is determined that the orientation of the controller device 31 is "upward" (S2806).
- the direction determining unit 2401 detects position information included in the touch information 1000. Since 1003 has a high degree of correlation with the position information in the reverse direction, it is determined that the direction of the operating device 31 is "downward" (S2807).
- the touch information storage unit 902 of the screen display device 5 stores the touch information 1000 detected by the touch information detection unit 901. Then, using the distribution of the position information 1003 included in the touch information 1000, the direction determining unit 2401 determines the direction of the operating device 31. In addition, according to the direction, the cursor position determination unit 1704 appropriately corrects the position information 1003 included in the touch information 1000 received from the touch information detection unit 901, and then determines the display position of the cursor.
- the screen display device 5 properly and automatically orients the operation device 31 based on only the input of the right touch pad 802 and the left touch pad 803 by the user, regardless of the direction in which the user holds the operation device 31. And the cursor can be displayed at the correct position. Thereby, the comfortable operability of the user can be realized.
- each direction determination process in the first to fourth embodiments may be replaced with the direction determination process described in the fifth embodiment.
- the present invention is not limited to these embodiments.
- the present invention also includes the embodiments obtained by applying variations that will occur to those skilled in the art to these embodiments, and other embodiments realized by arbitrarily combining the components in these embodiments.
- the present invention is not limited to the screen display device, and can be used as a display control device that receives data from an operating device having a touch pad and generates data to be displayed on the display unit.
- the display control device may be incorporated in the operation device or may be incorporated in the display unit.
- the display control device may be an independent device.
- a display control apparatus may be provided with an operating device and a display part.
- controller device shown in the embodiment may be a wireless remote controller or a wired remote controller.
- operation device may be other than the remote control, and may be physically integrated with the display unit, the display control device, or the screen display device, for example.
- the cursor position determination unit may be omitted.
- the touch information storage unit may be omitted.
- FIG. 30 is a block diagram showing a modification of the screen display device in the first embodiment.
- the controller device 11, the display control device 60 and the display unit 206 are physically integrated. Thereby, the orientations of the screen display device 6, the operation device 11, the display control device 60, and the display unit 206 coincide with each other.
- the direction determining unit 203 determines the direction of the controller device 11 based on the feature resulting from the operation of the finger. For example, the range in which the user can touch the touch pad 102 with a finger while gripping the operating device 11 with a hand is limited because of the feature of the finger structure. Therefore, the direction determining unit 203 determines the direction of the controller device 11 using one detected touch information without using a plurality of stored touch information according to such a feature. Good.
- the screen data generation unit 205 generates screen data in accordance with the direction of the controller device 11 determined by the direction determination unit 203, and causes the display unit 206 to display an image or the like.
- FIG. 31 is a conceptual diagram in the case of using the screen display device 6 shown in FIG. An image or the like is displayed in an appropriate direction on the screen display device 6 facing in various directions. That is, the display control device 60 can detect the direction appropriate for display based on the operation by the finger, and can display an image or the like.
- the direction determining unit always determines the direction of the operation device, corrects the cursor position on the screen, or generates an alert message. It was done.
- the screen display device may include the application monitoring unit, and the direction determining unit may determine the direction of the operating device when the user changes the holding of the operating device, such as when activating the application and switching the application. .
- the screen display device is configured to recognize the orientation of the operating device only when necessary, so the processing load on the screen display device is reduced.
- the screen display apparatus determines the orientation of the controller device by using the feature resulting from the operation of the finger regarding the position information on the touch pad input by the user.
- the features resulting from the operation of the finger are the features resulting from the structure, shape and movement of the finger.
- the screen display device according to the first to fifth embodiments particularly uses the inclination (the inclination approximated by a straight line) and the degree of change of the locus of the position information or the distribution of the position information as a feature to operate the operation device. Determine the orientation of
- features resulting from finger manipulation are not limited to such trajectories or distributions.
- the screen display device may approximate the locus of the position information with a curve, and use the curvature as a feature.
- the screen display device may approximate the locus of the position information with a curve, and may use an angle sandwiching the end point of the approximated curve.
- the screen display device may use whether the circular or elliptical start position is in a predetermined area on the touch pad that is easy to operate with the user's finger. Good.
- the direction determined by the direction determining unit is expressed as the direction of the controller device.
- this direction may be treated as the direction intended by the user.
- the direction determining unit specifies the shape of the locus of the position information on the touch pad input by the user by calculating the inclination and the degree of change of the straight line.
- the direction determining unit holds the trajectory predicted in advance, and calculates the degree of correlation or deviation between the trajectory on the touch pad input by the user and the trajectory held in advance.
- the shape may be specified.
- the touch pads according to the first to fifth embodiments acquire two-dimensional coordinate values of x coordinate and y coordinate as position information on the touch pad input by the user. Furthermore, a touch pad capable of acquiring three-dimensional coordinate values including the z-coordinate (z-axis direction) may be used.
- the direction determining unit may determine the direction of the controller by recognizing the shape of the user's finger from the shape of the input area.
- the locus of the position information on the touch pad may be recorded (learned) when the direction determining unit determines that the “right direction” is set. Then, the recorded trajectory may be used in the direction determination process after the next time. Also in the fifth embodiment, similarly, the distribution of position information on the touch pad may be recorded (learned).
- the effect of adapting to the user's habit or the like can be calculated by calculating the degree of correlation or deviation between the locus of positional information on the touch pad input by the user and the learned locus. I can expect it.
- the direction determining unit determines the touch pad with a large amount (number) of touch information. It may be determined that the side direction is the upward direction intended by the user.
- the display control device 40 compares the orientation of the operating device 41 determined by the orientation determination unit 1703 with the orientation of the coordinate system on the right touch pad 802 and the left touch pad 803, which are set in advance as being fixed. Good. Then, the display control device 40 may generate an alert message or the like and cause the display unit 206 to display an alert message or the like when the directions of the two do not match.
- the alert message is displayed, it is detected that the user has changed the operating device 41, or when the direction determining unit 1703 recognizes the next direction of the operating device 41.
- the alert message previously displayed by the screen data generation unit 2202 may be deleted.
- the method of warning is not limited to the alert data generation unit 2201 shown in the fourth embodiment creating alert messages.
- the screen display device may include an alert unit outside the display control device 40.
- the alert unit does not match the direction of the operating device determined by the direction determining unit 1703 with the direction of the coordinate system on the touch pad, which is set in advance as fixed, or the direction of the operating device detected by the sensor.
- alerts may be indicated to the user by various feedback methods. For example, various methods such as changing the color or shape of an object on the screen, notifying by sound, etc. can be applied.
- the screen display device provided with both the first sensor 2204 and the second sensor 2205 is shown.
- the screen display device may have only one of the sensors. The same processing can also be realized by this.
- the screen display device having only the first sensor 2204 has the direction of the coordinate system on the touch pad, which is set in advance based on the holding direction of the operation device detected by the first sensor 2204;
- the orientation determination unit 1703 compares the orientation with the determined operation device. Conversely, in the screen display device having only the second sensor 2205, the direction of the operating device based on the vertical direction detected by the second sensor 2205 and the direction of the operating device determined by the direction determining unit 1703. Make a comparison.
- the direction determining unit 1703 uses information obtained from the first sensor 2204, the second sensor 2205, the left touch pad 803, the right touch pad 802, or any combination thereof. , Direction can be determined. For example, the direction determining unit 1703 can use the information obtained from the second sensor 2205 and the left touch pad 803 to determine the direction.
- the direction determining unit 1703 may use the other information obtained from the first sensor 2204, the second sensor 2205, the left touch pad 803, the right touch pad 802, or any combination thereof to validate the direction. You may verify sex. That is, the direction determining unit 1703 determines whether or not the direction estimated as the user's intended direction from other information matches the direction determined based on the feature resulting from the operation of the finger. The validity of the direction may be verified.
- the direction determining unit 1703 may determine the direction estimated using the information obtained from the first sensor 2204 and the second sensor 2205, and the direction determined using the information obtained from the left touch pad 803 and the right touch pad 802. It may be verified whether and match. Then, if they match, the cursor position determination unit 2203 determines the cursor position, and if they do not match, the warning data generation unit 2201 may generate warning data.
- the cursor position determining unit 2203 corrects the cursor position determined in accordance with the predetermined direction. By doing this, the cursor position may be determined.
- the warning data generation unit 2201 may generate warning data.
- the display control device when the user starts holding the operating device, switching the application, changing the direction of holding the operating device in the application, etc., the display control device performs a touch operation.
- the natural operation of the finger on the pad recognizes the orientation of the operation device.
- it may be assumed that the touch information necessary for recognition can not be sufficiently acquired only by such initial movement operation.
- the display control device may automatically perform processing such as extending recognition time, attempting re-recognition, etc., or may notify the user of recognition failure.
- the user may perform an unnatural operation on the touch pad even though the screen display device displays the cursor based on the determined direction.
- the user may perform an operation such as frequently performing an operation in an area where there is no object to be selected on the screen, or repeatedly rotating a finger on the touch pad.
- the screen display device may determine that the determined direction is an erroneous recognition, and may perform processing for reversing the determined direction of the operating device in the vertical direction automatically or after confirming with the user .
- the screen display device displays the cursor based on the determined direction
- the user reverses the operation device in the vertical direction and performs holding There is.
- the screen display device does not have to execute the recognition of the direction of the operation device immediately after the user changes the holding.
- the screen display device may re-acknowledge the orientation of the operating device when the application is switched next, or when the user changes the holding of the operating device after a predetermined time has elapsed or the like.
- the screen display device may present an application screen that allows the user to naturally operate the touch pad.
- an application screen that allows the user to naturally operate the touch pad.
- a method of presenting a GUI as a part of an application in which the door is opened with both hands when the application starts and when the horizontal holding and the vertical holding of the operating device are switched in the same application may be considered.
- the screen display apparatus improves the recognition accuracy of the direction of the operating device as a result without making the user as much conscious of the special operation for recognizing the direction of the operating device as possible. be able to.
- the screen display device may present a GUI indicating whether the orientation of the controller device has been determined or not.
- GUI various methods can be considered such as changing color and intensity, changing an object on the screen, changing the position and shape of a cursor, displaying a message, and notifying by sound.
- the screen display device may perform transmissive display ( ⁇ display) such as thinning the cursor image or may not display the cursor. Then, after the direction of the operation device is determined, the screen display device may display a cursor image as usual.
- ⁇ display transmissive display
- the present invention is also applicable to an operating device provided with a plurality of touch pads on three or more surfaces on the upper surface of the operating device, or an operating device provided with a plurality of touch pads combined on both the upper surface and the lower surface (rear surface).
- the configurations shown in the second to fifth embodiments may be applied.
- the screen display device may detect a plurality of touches from one touch pad corresponding to multi-touch. For example, even if the screen display apparatus shown in the second to fifth embodiments has only one touch pad, it is possible to detect the touch of a plurality of fingers from one touch pad, Similar processing can be realized.
- the cursor display is free pointing (free cursor).
- display by the touch pad may be display by focus of a specific object or the like.
- the components of the display control apparatus shown in FIGS. 2, 9, 17, 22, 24, and 30 may be realized as an LSI (Large Scale Integration) which is an integrated circuit. These components may be individually made into one chip, or may be made into one chip so as to include some or all. Although an LSI is used here, it may be called an IC (Integrated Circuit), a system LSI, a super LSI, or an ultra LSI depending on the degree of integration.
- LSI Large Scale Integration
- the method of circuit integration is not limited to LSI's, and implementation using dedicated circuitry or general purpose processors is also possible. It is also possible to use a programmable field programmable gate array (FPGA) or a reconfigurable processor that can reconfigure connection and setting of circuit cells in the LSI.
- FPGA field programmable gate array
- reconfigurable processor that can reconfigure connection and setting of circuit cells in the LSI.
- only the means for storing data may be configured separately without being integrated into one chip.
- the present invention can be realized not only as a display control device, but also as a method in which processing means constituting the display control device are steps. The steps included in the method are typically performed by a computer. Then, the present invention can be realized as a program that causes a computer to execute the steps included in the method. Furthermore, the present invention can be realized as a computer readable storage medium such as a CD-ROM storing the program.
- the display control device can appropriately display an image etc. on the screen according to the natural operation of the user on the operating device operated in various directions, and can be applied to a screen display device such as a digital television, for example.
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Position Input By Displaying (AREA)
Abstract
Description
実施の形態1における画面表示装置は、タッチパッド上でのユーザの入力を示す位置情報(座標)の軌跡の形状を利用して、タッチパッドを含む操作装置の向き(方向ともいう)を認識する。そして、画面表示装置は、操作装置の向きに基づいて画面上でのカーソル(GUI上での操作位置を示すポインタ)の表示位置を補正する。
実施の形態2に係る画面表示装置は、2つのタッチパッドを備え、2つのタッチパッド上でのユーザの入力を利用して、操作装置の向きを認識する。そして、画面表示装置は、操作装置の向きに基づき、画面上でのカーソルの表示位置を補正する。
実施の形態3では、2つのタッチパッドを備える画面表示装置が、さらに、ユーザが操作装置のどの部分を把持しているかを検出するためのセンサーを備える。
実施の形態4に係る画面表示装置は、2つのタッチパッドと、ユーザに操作装置のどの部分が把持されているかを検出するためのセンサーと、操作装置の重力方向を検出するためのセンサーとを備える。そして、画面表示装置は、画面表示装置のタッチパッド上でのユーザの入力とセンサーでの把持方向の検知結果を利用して操作装置の向きを認識し、当該認識結果とセンサーでの操作装置の方向の検知結果とが一致しない場合に画面上にアラート表示を行う。
実施の形態5では、2つのタッチパッドを備える画面表示装置が、画面表示装置のタッチパッド上でのユーザの入力を示す位置情報(座標)の分布を利用して操作装置の向きを認識する。そして、画面表示装置は、操作装置の向きに基づいて画面上でのカーソルの表示位置を補正する。
10、20、30、40、50、60 表示制御装置
11、21、31、41 操作装置
101、801、1601 リモコン
102 タッチパッド
103、804 指
104、805 画面
105、806 カーソル
106 目
201、901 タッチ情報検出部
202、902 タッチ情報蓄積部
203、903、1703、2401 方向決定部
204、904、1704、2203 カーソル位置決定部
205、905、2202 画面データ生成部
206、906 表示部
300、1000 タッチ情報
301、1002 操作ID
302、1003 位置情報
400、1100 タッチ情報テーブル
802 右タッチパッド
803 左タッチパッド
1001 タッチパッドID
1701 センサー
1702 センサー情報検出部
2201 警告データ生成部
2204 第1のセンサー
2205 第2のセンサー
Claims (19)
- タッチパッドを有する操作装置からの信号を受け、画面に表示するための画面データを生成する表示制御装置であって、
ユーザが指で前記タッチパッドを操作するときに前記指が接触した前記タッチパッド上の位置情報を含むタッチ情報を検出するタッチ情報検出部と、
前記タッチ情報検出部により検出された前記タッチ情報によって示される特徴であって、前記指による前記タッチパッド上での操作に起因する特徴を用いて、前記ユーザが意図する方向を決定する方向決定部と、
前記方向決定部により決定された前記方向に依存させて、前記画面データを生成する画面データ生成部とを備える
表示制御装置。 - 前記方向決定部は、前記特徴を用いて、前記ユーザが前記タッチパッドを見たときの前記タッチパッド上の上下左右の方向である前記方向を決定する
請求項1に記載の表示制御装置。 - 前記方向決定部は、前記特徴を用いて、前記操作装置の方向である前記方向を決定する
請求項1または2に記載の表示制御装置。 - 前記表示制御装置は、さらに、前記タッチ情報検出部により検出された前記タッチ情報を蓄積することにより、複数のタッチ情報を蓄積するタッチ情報蓄積部を備え、
前記方向決定部は、前記タッチ情報蓄積部により蓄積された前記複数のタッチ情報によって示される前記特徴を用いて、前記方向を決定する
請求項1~3のいずれか1項に記載の表示制御装置。 - 前記方向決定部は、前記複数のタッチ情報によって示される位置情報の遷移によって形成される軌跡の形状である前記特徴を用いて、前記方向を決定する
請求項4に記載の表示制御装置。 - 前記方向決定部は、前記軌跡が凸状である場合に、凸側が上側であるとして、または、前記軌跡が凹状である場合に、凹側が下側であるとして、前記方向を決定する
請求項5に記載の表示制御装置。 - 前記方向決定部は、前記複数のタッチ情報によって示される複数の位置情報の分布である前記特徴を用いて、前記方向を決定する
請求項4に記載の表示制御装置。 - 前記表示制御装置は、さらに、前記タッチ情報検出部により検出された前記タッチ情報に含まれる前記位置情報から、前記画面上でのポインティング位置を示すカーソルの表示位置を決定するカーソル位置決定部を備え、
前記カーソル位置決定部は、前記方向決定部により決定された前記方向に依存させて、前記表示位置を決定し、
前記画面データ生成部は、前記カーソル位置決定部により決定された前記表示位置に前記カーソルを表示するための前記画面データを生成する
請求項1~7のいずれか1項に記載の表示制御装置。 - 前記カーソル位置決定部は、前記方向決定部により決定された前記方向と、前記操作装置において予め定められている方向とが異なる場合に、前記位置情報と前記予め定められている方向とによって決定される前記表示位置を補正することにより、前記表示位置を決定する
請求項8に記載の表示制御装置。 - 前記表示制御装置は、さらに、前記ユーザに警告を通知するための警告データを生成する警告データ生成部を備え、
前記警告データ生成部は、前記方向決定部により決定された前記方向と、前記操作装置において予め定められている方向とが異なる場合に、前記警告データを生成する
請求項1~7のいずれか1項に記載の表示制御装置。 - 前記表示制御装置は、さらに、前記操作装置の外周部に配置されたセンサーであって、前記ユーザの接触を検出するセンサーから、前記ユーザが前記操作装置を保持している部分の情報である保持部分情報を検出するセンサー情報検出部を備え、
前記方向決定部は、前記センサー情報検出部により検出された前記保持部分情報と、前記特徴とに依存させて、前記方向を決定する
請求項1~10のいずれか1項に記載の表示制御装置。 - 前記表示制御装置は、さらに、前記操作装置にかかる重力方向を検出するセンサーから、前記操作装置にかかる重力方向の情報である重力方向情報を検出するセンサー情報検出部を備え、
前記方向決定部は、前記センサー情報検出部により検出された前記重力方向情報と、前記特徴とに依存させて、前記方向を決定する
請求項1~10のいずれか1項に記載の表示制御装置。 - 前記表示制御装置は、さらに、
前記ユーザに警告を通知するための警告データを生成する警告データ生成部と、
前記操作装置の外周部に配置されたセンサーであって、前記ユーザの接触を検出するセンサーから、前記ユーザが前記操作装置を保持している部分の情報である保持部分情報を検出するセンサー情報検出部とを備え、
前記警告データ生成部は、前記センサー情報検出部により検出された前記保持部分情報から前記ユーザが意図する方向として推定される方向と、前記方向決定部により決定された前記方向とが異なる場合に、前記警告データを生成する
請求項1~8のいずれか1項に記載の表示制御装置。 - 前記表示制御装置は、さらに、
前記ユーザに警告を通知するための警告データを生成する警告データ生成部と、
前記操作装置にかかる重力方向を検出するセンサーから、前記操作装置にかかる重力方向の情報である重力方向情報を検出するセンサー情報検出部とを備え、
前記警告データ生成部は、前記センサー情報検出部により検出された前記重力方向情報から前記ユーザが意図する方向として推定される方向と、前記方向決定部により決定された前記方向とが異なる場合に、前記警告データを生成する
請求項1~8のいずれか1項に記載の表示制御装置。 - 前記表示制御装置は、さらに、
前記ユーザに警告を通知するための警告データを生成する警告データ生成部と、
前記操作装置の外周部に配置された第1のセンサーであって、前記ユーザの接触を検出する第1のセンサーから、前記ユーザが前記操作装置を保持している部分の情報である保持部分情報を検出し、前記操作装置にかかる重力方向を検出する第2のセンサーから、前記操作装置にかかる重力方向の情報である重力方向情報を検出するセンサー情報検出部とを備え、
前記カーソル位置決定部は、前記センサー情報検出部により検出された前記保持部分情報および前記重力方向情報から前記ユーザが意図する方向として推定される方向と、前記方向決定部により決定された前記方向とが一致する場合に、前記方向決定部により決定された前記方向に依存させて、前記表示位置を決定し、
前記警告データ生成部は、前記センサー情報検出部により検出された前記保持部分情報および前記重力方向情報から前記ユーザが意図する方向として推定される方向と、前記方向決定部により決定された前記方向とが一致しない場合に、前記警告データを生成する
請求項8に記載の表示制御装置。 - 前記タッチ情報検出部は、2つのタッチパッドを有する前記操作装置からの信号を取得し、前記ユーザが複数の指で前記2つのタッチパッドを操作するときに前記複数の指のいずれかが接触した各タッチパッド上の位置情報をそれぞれ含む2つのタッチ情報を検出し、
前記方向決定部は、前記タッチ情報検出部により検出された前記2つのタッチ情報のそれぞれによって示される前記特徴でそれぞれが構成される2つの特徴が互いに一致する場合に、前記2つの特徴のいずれかを用いて、前記方向を決定する
請求項1~15のいずれか1項に記載の表示制御装置。 - タッチパッドを有する操作装置からの信号を受け、画面に表示するための画面データを生成する表示制御方法であって、
ユーザが指で前記タッチパッドを操作するときに前記指が接触した前記タッチパッド上の位置情報を含むタッチ情報を検出するタッチ情報検出ステップと、
前記タッチ情報検出ステップにより検出された前記タッチ情報によって示される特徴であって、前記指による前記タッチパッド上での操作に起因する特徴を用いて、前記ユーザが意図する方向を決定する方向決定ステップと、
前記方向決定ステップにより決定された前記方向に依存させて、前記画面データを生成する画面データ生成ステップとを含む
表示制御方法。 - 請求項17に記載の表示制御方法に含まれるステップをコンピュータに実行させるための
プログラム。 - タッチパッドを有する操作装置からの信号を受け、画面に表示するための画面データを生成する集積回路であって、
ユーザが指で前記タッチパッドを操作するときに前記指が接触した前記タッチパッド上の位置情報を含むタッチ情報を検出するタッチ情報検出部と、
前記タッチ情報検出部により検出された前記タッチ情報によって示される特徴であって、前記指による前記タッチパッド上での操作に起因する特徴を用いて、前記ユーザが意図する方向を決定する方向決定部と、
前記方向決定部により決定された前記方向に依存させて、前記画面データを生成する画面データ生成部とを備える
集積回路。
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2011533901A JP5667984B2 (ja) | 2010-01-26 | 2011-01-19 | 表示制御装置、表示制御方法、プログラムおよび集積回路 |
US13/257,453 US8860676B2 (en) | 2010-01-26 | 2011-01-19 | Display control device, method, program, and integrated circuit |
US14/482,014 US20140375589A1 (en) | 2010-01-26 | 2014-09-10 | Display control device, method, program, and integrated circuit |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2010014651 | 2010-01-26 | ||
JP2010-014651 | 2010-01-26 |
Related Child Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/257,453 A-371-Of-International US8860676B2 (en) | 2010-01-26 | 2011-01-19 | Display control device, method, program, and integrated circuit |
US14/482,014 Division US20140375589A1 (en) | 2010-01-26 | 2014-09-10 | Display control device, method, program, and integrated circuit |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2011093033A1 true WO2011093033A1 (ja) | 2011-08-04 |
Family
ID=44319032
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2011/000246 WO2011093033A1 (ja) | 2010-01-26 | 2011-01-19 | 表示制御装置、表示制御方法、プログラムおよび集積回路 |
Country Status (3)
Country | Link |
---|---|
US (2) | US8860676B2 (ja) |
JP (1) | JP5667984B2 (ja) |
WO (1) | WO2011093033A1 (ja) |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103425445A (zh) * | 2012-05-23 | 2013-12-04 | 鸿富锦精密工业(深圳)有限公司 | 电子橱窗系统 |
CN103529962A (zh) * | 2013-05-28 | 2014-01-22 | Tcl集团股份有限公司 | 触摸板遥控器的鼠标功能实现方法及系统 |
JP2014513854A (ja) * | 2011-05-18 | 2014-06-05 | マイクロソフト コーポレーション | マルチタッチポインティングデバイスにおける意図的及び偶発的接触の曖昧性除去 |
JP2014182435A (ja) * | 2013-03-18 | 2014-09-29 | Fujitsu Ltd | 入力装置および入力制御方法並びに入力制御プログラム |
JP2017097411A (ja) * | 2015-11-18 | 2017-06-01 | 京セラ株式会社 | 携帯電子機器、制御方法および制御プログラム |
Families Citing this family (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102365611B (zh) * | 2010-02-03 | 2014-12-03 | 松下电器(美国)知识产权公司 | 显示控制装置、显示控制方法以及触摸板输入系统 |
US9507454B1 (en) * | 2011-09-19 | 2016-11-29 | Parade Technologies, Ltd. | Enhanced linearity of gestures on a touch-sensitive surface |
CN102799358B (zh) * | 2012-06-20 | 2017-08-08 | 南京中兴软件有限责任公司 | 光标显示位置的确定方法及装置 |
MX343170B (es) * | 2012-09-28 | 2016-10-26 | Afl Telecommunications Llc | Modulo optico instalable en el campo con anexion de cable. |
TW201416915A (zh) * | 2012-10-19 | 2014-05-01 | Primax Electronics Ltd | 游標控制裝置以及游標控制系統 |
CN103777780A (zh) * | 2012-10-24 | 2014-05-07 | 致伸科技股份有限公司 | 光标控制装置以及光标控制系统 |
CN109240633B (zh) * | 2013-03-13 | 2021-10-22 | 歌乐株式会社 | 显示装置以及信息终端操作方法 |
KR20150008963A (ko) * | 2013-06-27 | 2015-01-26 | 삼성전자주식회사 | 스크린을 제어하는 휴대 단말 및 방법 |
US10289260B2 (en) | 2014-08-27 | 2019-05-14 | Honda Motor Co., Ltd. | Systems and techniques for application multi-tasking |
CN106020661B (zh) | 2014-12-31 | 2020-11-24 | 本田技研工业株式会社 | 具有内部边框的跟踪板 |
JP2017134638A (ja) * | 2016-01-28 | 2017-08-03 | 株式会社東海理化電機製作所 | 操作検出装置 |
US10402161B2 (en) | 2016-11-13 | 2019-09-03 | Honda Motor Co., Ltd. | Human-vehicle interaction |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH10301695A (ja) * | 1997-04-25 | 1998-11-13 | Hitachi Ltd | 状態検出方法及び携帯端末装置 |
JP2007122287A (ja) * | 2005-10-26 | 2007-05-17 | Sharp Corp | 情報処理装置、階層情報出力方法及びプログラム |
JP2009140210A (ja) * | 2007-12-06 | 2009-06-25 | Panasonic Corp | 情報入力装置、情報入力方法、および情報入力プログラム |
Family Cites Families (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP3161794B2 (ja) * | 1992-03-05 | 2001-04-25 | 松下電器産業株式会社 | ペン入力装置及びペン入力方法 |
JPH11143604A (ja) | 1997-11-05 | 1999-05-28 | Nec Corp | 携帯端末装置 |
GB2364226A (en) * | 2000-06-30 | 2002-01-16 | Nokia Corp | Method of selecting an object by controlling an on screen pointer |
JP2003233454A (ja) * | 2002-02-13 | 2003-08-22 | Toshiba Corp | 情報処理装置 |
CN101390034B (zh) * | 2004-01-29 | 2012-03-14 | 辛纳普蒂克斯有限公司 | 采用二维定位设备产生一维信号的方法及装置 |
US7653883B2 (en) * | 2004-07-30 | 2010-01-26 | Apple Inc. | Proximity detector in handheld device |
JP2006072872A (ja) * | 2004-09-06 | 2006-03-16 | Matsushita Electric Ind Co Ltd | 携帯型情報処理装置、情報処理装置の画面回転方法および合成データ回転方法 |
US20060119585A1 (en) * | 2004-12-07 | 2006-06-08 | Skinner David N | Remote control with touchpad and method |
WO2006126310A1 (ja) * | 2005-05-27 | 2006-11-30 | Sharp Kabushiki Kaisha | 表示装置 |
JP2008052062A (ja) | 2006-08-24 | 2008-03-06 | Ricoh Co Ltd | 表示装置、表示装置の表示方法、プログラム及び記録媒体 |
US8665225B2 (en) * | 2007-01-07 | 2014-03-04 | Apple Inc. | Portable multifunction device, method, and graphical user interface for interpreting a finger gesture |
US8681104B2 (en) * | 2007-06-13 | 2014-03-25 | Apple Inc. | Pinch-throw and translation gestures |
JP4743267B2 (ja) * | 2008-12-12 | 2011-08-10 | ソニー株式会社 | 情報処理装置、情報処理方法およびプログラム |
US20110205169A1 (en) * | 2010-02-24 | 2011-08-25 | Primax Electronics Ltd. | Multi-touch input apparatus and its interface method using hybrid resolution based touch data |
-
2011
- 2011-01-19 WO PCT/JP2011/000246 patent/WO2011093033A1/ja active Application Filing
- 2011-01-19 US US13/257,453 patent/US8860676B2/en active Active
- 2011-01-19 JP JP2011533901A patent/JP5667984B2/ja active Active
-
2014
- 2014-09-10 US US14/482,014 patent/US20140375589A1/en not_active Abandoned
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH10301695A (ja) * | 1997-04-25 | 1998-11-13 | Hitachi Ltd | 状態検出方法及び携帯端末装置 |
JP2007122287A (ja) * | 2005-10-26 | 2007-05-17 | Sharp Corp | 情報処理装置、階層情報出力方法及びプログラム |
JP2009140210A (ja) * | 2007-12-06 | 2009-06-25 | Panasonic Corp | 情報入力装置、情報入力方法、および情報入力プログラム |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2014513854A (ja) * | 2011-05-18 | 2014-06-05 | マイクロソフト コーポレーション | マルチタッチポインティングデバイスにおける意図的及び偶発的接触の曖昧性除去 |
CN103425445A (zh) * | 2012-05-23 | 2013-12-04 | 鸿富锦精密工业(深圳)有限公司 | 电子橱窗系统 |
JP2014182435A (ja) * | 2013-03-18 | 2014-09-29 | Fujitsu Ltd | 入力装置および入力制御方法並びに入力制御プログラム |
CN103529962A (zh) * | 2013-05-28 | 2014-01-22 | Tcl集团股份有限公司 | 触摸板遥控器的鼠标功能实现方法及系统 |
JP2017097411A (ja) * | 2015-11-18 | 2017-06-01 | 京セラ株式会社 | 携帯電子機器、制御方法および制御プログラム |
Also Published As
Publication number | Publication date |
---|---|
US8860676B2 (en) | 2014-10-14 |
US20120013463A1 (en) | 2012-01-19 |
JP5667984B2 (ja) | 2015-02-12 |
US20140375589A1 (en) | 2014-12-25 |
JPWO2011093033A1 (ja) | 2013-05-30 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2011093033A1 (ja) | 表示制御装置、表示制御方法、プログラムおよび集積回路 | |
JP5802667B2 (ja) | ジェスチャ入力装置およびジェスチャ入力方法 | |
US10120454B2 (en) | Gesture recognition control device | |
RU2598598C2 (ru) | Устройство обработки информации, система обработки информации и способ обработки информации | |
US9778748B2 (en) | Position-of-interest detection device, position-of-interest detection method, and position-of-interest detection program | |
US20180292907A1 (en) | Gesture control system and method for smart home | |
JP5658500B2 (ja) | 情報処理装置及びその制御方法 | |
US20180150186A1 (en) | Interface control system, interface control apparatus, interface control method, and program | |
WO2016206279A1 (zh) | 触控显示装置及其触控方法 | |
CN108027656B (zh) | 输入设备、输入方法和程序 | |
JP2004280834A (ja) | 仮想筆記平面を用いたモーション認識システム及びその認識方法 | |
JPWO2007088939A1 (ja) | 情報処理装置 | |
CN109643206B (zh) | 控制装置、显示装置、程序及检测方法 | |
JP2014515147A (ja) | 拡張現実のためのジェスチャベースユーザインターフェース | |
US20200142495A1 (en) | Gesture recognition control device | |
WO2018083910A1 (ja) | 情報処理装置、情報処理方法、及び記録媒体 | |
JP4345755B2 (ja) | 入力位置設定方法、入力位置設定装置、入力位置設定プログラムおよび情報入力システム | |
JP6272069B2 (ja) | 情報処理装置、情報処理方法、コンピュータプログラム、及び記録媒体 | |
KR101281461B1 (ko) | 영상분석을 이용한 멀티 터치 입력 방법 및 시스템 | |
JP5342806B2 (ja) | 表示方法および表示装置 | |
JP4871226B2 (ja) | 認識装置および認識方法 | |
JP2016110383A (ja) | 触覚伝達装置 | |
KR100917918B1 (ko) | 디지털 페이퍼 및 디지털 펜을 이용하여 디지타이저 기능을제공하는 방법 및 장치 | |
JP2010009311A (ja) | ユーザインタフェイス装置 | |
JP2016139396A (ja) | ユーザーインターフェイス装置、方法およびプログラム |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
WWE | Wipo information: entry into national phase |
Ref document number: 2011533901 Country of ref document: JP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 13257453 Country of ref document: US |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 11736750 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 11736750 Country of ref document: EP Kind code of ref document: A1 |