JP2012252663A - Operation panel device and electronic information equipment - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an operation panel device allowing a user to perform an operation as intended by the user on a stereoscopic image in a displayed state of the stereoscopic image regardless of whether a dominant eye of the user is a left eye or a right eye to avoid feeling of discomfort with a display image by the user when the user performs a touch operation or avoid unintended image processing by the touch operation due to a difference of the dominant eye of the user.SOLUTION: An operation panel device 100 includes a three-dimensional image space processing section 104 for generating three-dimensional image data defining an object in a virtual three-dimensional space to stereoscopically display the object. The operation panel device 100 further includes a user view point determination section 105 for determining a user view point to the stereoscopically displayed object. An operation position to an operation surface of the touch panel is determined by specifying an operation position to a two-dimensional image when viewing the stereoscopically displayed object from a left eye or a right eye to be the determined user view point as a reference.

Description

  The present invention relates to an operation panel device and an electronic information device, and in particular, includes a display device provided with a touch panel, and stereoscopically displays an image (hereinafter also referred to as an object) on the display surface of the display device. In the operation panel device configured to be able to operate the obtained image (that is, the stereoscopic image) by a touch operation on the touch panel, a two-dimensional coordinate system for specifying the position of the stereoscopic image on the display surface is determined according to the user's dominant eye The present invention relates to a device to be changed and an electronic information device equipped with such an operation panel device.

  In recent years, in portable electronic devices such as portable information devices such as mobile phones and PDAs (Personal Digital Assistants) and digital video camera recorders, a touch panel is used as an input device for a user to select an object such as an icon included in a menu screen. Many are equipped with.

  This touch panel is placed on the display surface of the display device so that the screen displayed on the display surface can be seen. By touching any position directly with a finger or pen, it is equivalent to a mouse click or double click. There are various types of input devices capable of event generation operations, such as an analog capacitive coupling method and an analog resistance film method.

  For example, an operation panel device having a display device that displays an image and a touch panel attached to the display surface of the display device is referred to as a position touched by the user (hereinafter also referred to as a touch position). ) Indicating coordinates (hereinafter also referred to as touch coordinates), and an icon or button displayed at the touch position can be operated based on the calculated touch coordinates.

  By the way, in Patent Document 1, in an operation panel device combining such a touch panel and a display device, a technique for performing various processes by touch operations on an object (object) in a virtual three-dimensional space displayed on the display device. Is disclosed.

  Further, along with recent advances in liquid crystal display (LCD) technology, an operation panel device has been developed in which a display device capable of displaying an image in three dimensions is combined with a touch panel.

  In such an operation panel device capable of stereoscopic display of an image, a pair of parallax images on the display surface of the display device, that is, an image when the subject is viewed with the left eye (for the left eye) so that the user can recognize the image stereoscopically. A parallax image) and an image when the subject is viewed with the right eye (right-eye parallax image) are displayed while being shifted by an average distance between the left and right eyes of a person (hereinafter referred to as binocular parallax).

  In addition, information transmitted from the human eye to the brain was transmitted from the dominant eye of the left and right eyes to the brain faster than information from the non-dominant eye, and from the dominant eye to the brain faster. Information is the basis, and information from the non-dominant eye plays an auxiliary role, and the image is recognized three-dimensionally by the user.

  For this reason, when the user touches the touch panel for the operation of a stereoscopically displayed image (stereoscopic image), for example, a person with a dominant left eye selects a target position to be touched (for example, the center position of the object), A touch operation is performed based on the left-eye parallax image on the display surface, and a person with a right-handed eye selects the target position to be touched (for example, the center position of the object) and the right-eye parallax on the display surface. It is determined on the image and a touch operation is performed.

  Therefore, even when an individual user performs a touch operation with the intention of touching the center of the same stereoscopic image, the location on the touch panel that the user touches differs depending on whether the user's dominant eye is the right eye or the left eye. The conventional operation panel device has a problem that the operability with respect to the stereoscopic image is poor.

  By the way, in patent document 2, in order to improve the operativity with respect to the three-dimensional image in such an operation panel apparatus, the three-dimensional image displayed on the display apparatus is switched to a plane image immediately before operation is performed on a touch panel. The thing which took measures is disclosed.

  FIG. 11 is a diagram for explaining the operation panel device disclosed in Patent Document 2. FIG. 11 (a) shows the overall configuration, and FIG. 11 (b) shows the structure of the operation panel constituting this operation panel device. Show.

  An operation panel device 10 illustrated in FIG. 11A includes an operation panel 1 for performing a touch operation on an operation screen displayed on a display surface, and a control unit 9 that controls the operation panel 1. Yes. Here, the operation panel 1 has a display surface, and a display unit 2 that displays various images on the display surface, and is arranged so as to overlap the display surface of the display unit 2, and the touch position touched by the user is displayed. A touch panel 3 that detects and outputs coordinate data indicating a touch position, and is disposed at a predetermined interval A from the operation surface of the touch panel 3, and a pressing member such as a user's finger or a touch pen is operated on the operation surface of the touch panel 3. And a proximity sensor 4 for detecting the approach of a finger or the like to the operation surface before touching.

  In the operation panel 1, the control unit 9 uses the detection output from the proximity sensor 4 to display an image to be displayed on the display unit 2 as a pair of parallax images and one plane for allowing the user to recognize a stereoscopic image. It is configured to switch between images.

  Therefore, in the operation panel device 10 disclosed in Patent Document 2, a pair of parallax images for forming a stereoscopic image displayed on the display unit 2, that is, a right-eye parallax image and a left-eye parallax image are displayed on the touch panel 3. By switching to a single planar image immediately before the operation is performed, the problem that the location where the stereoscopic image is touched differs on the operation surface of the touch panel due to the difference in the dominant hand of the user as described above can be avoided.

JP 2004-259065 A Japanese Patent Application Laid-Open No. 2004-280496

  However, in Patent Document 2, in order to improve operability, the proximity sensor detects the approach of a pressing member such as a finger or a touch pen, and switches between a pair of parallax images and a single planar image based on the detection information. However, the installation of the proximity sensor deteriorates the design and also increases the cost. In addition, due to switching between a pair of parallax images that allow the user to recognize a stereoscopic image and a simple planar image, the user feels uncomfortable with the display image during a touch operation, and the reality of the display image is also impaired. End up.

  On the other hand, when a touch operation is performed on the touch panel while a pair of parallax images is displayed on the display surface, even if the user intends to touch the center of the stereoscopic image, as described above, the difference between the user's dominant eyes Thus, the user touches a different position on the operation surface of the touch panel, and the operation panel device 1 determines that an operation different from the operation intended by the user is performed, and unintended image processing is performed. There's a problem.

  The present invention has been made in order to solve the above-described problems. Regardless of whether the user's dominant eye is the left eye or the right eye, the stereoscopic image is displayed while the stereoscopic image is displayed. The touch operation as intended by the user can be performed more accurately, which makes the display image feel uncomfortable when the user performs the touch operation, or the user's dominant eye It is an object of the present invention to obtain an operation panel device capable of avoiding a touch operation from being impossible and an electronic information device having such an operation panel device.

  An operation panel device according to the present invention has a display device having a display surface for displaying an image, and an operation surface provided to be superimposed on the display surface of the display device, and inputs information by a touch operation on the operation surface. An operation panel device having a touch panel to perform, a three-dimensional image space processing unit that generates three-dimensional image data defining an object located in a virtual three-dimensional space, and the object based on the three-dimensional image data A display image generation unit that generates image data of the left-eye parallax image and right-eye parallax image of the display device and supplies the image to the display device so that the object is stereoscopically displayed on the display surface of the display device; A two-dimensional image when the stereoscopically displayed object is viewed from the determined user viewpoint. It is intended to be operated position, the objects can be achieved.

  According to the present invention, in the operation panel device, the user viewpoint determination unit calculates a user viewpoint in the virtual three-dimensional space, and the calculated user viewpoint indicates the position of the left eye and the right eye of the user in the virtual three-dimensional space. It is determined which position of the position is close, and the left eye parallax image and the right eye parallax image corresponding to the eye with the closer calculated user viewpoint is the target of the touch operation on the operation surface of the touch panel It is preferable to use as a two-dimensional image.

  According to the present invention, in the operation panel device, the user viewpoint determination unit calculates a user viewpoint in the virtual three-dimensional space, and displays a two-dimensional image when the stereoscopically displayed object is viewed from the calculated user viewpoint. It is preferable to use as a two-dimensional image to be touched on the operation surface of the touch panel.

  According to the present invention, in the operation panel device, the display image generation unit is arranged on a first two-dimensional plane perpendicular to a straight line connecting a position of the user's left eye and the center of the object in the virtual three-dimensional space. Projecting the image of the object to generate the left-eye parallax image, on a second two-dimensional plane perpendicular to a straight line connecting the position of the user's right eye in the virtual three-dimensional space and the center of the object, It is preferable that the right-eye parallax image is generated by projecting an object image.

  According to the present invention, in the operation panel device, the user viewpoint determination unit determines whether or not a user viewpoint for a stereoscopically displayed object has been determined following an initial setting after the power is turned on. As a result of the determination, when the user viewpoint is determined, either the left-eye parallax image or the right-eye parallax image is displayed on the operation surface of the touch panel using information about the user viewpoint already stored. If the user viewpoint is not determined as a result of the determination as a two-dimensional image to be touched, determine the user viewpoint for the stereoscopically displayed object, and then based on the determined user viewpoint, Either the left-eye parallax image or the right-eye parallax image is preferably a two-dimensional image that is a target of a touch operation on the operation surface of the touch panel. Arbitrariness.

  The present invention includes a storage unit that stores various information in the operation panel device, wherein the user viewpoint determination unit is a three-dimensionally displayed object when the user viewpoint is not determined as a result of the determination When the user viewpoint for is determined, information regarding the determined user viewpoint is preferably stored in the storage unit.

  According to the present invention, in the operation panel device, in response to a touch operation on the touch panel, the virtual three-dimensional space processing unit includes at least generation, rotation, movement, enlargement, and reduction of an object in the three-dimensional image space. It is preferable that a display control unit for controlling the virtual three-dimensional space processing unit is provided so as to perform specific image processing of the image processing.

  According to the present invention, in the operation panel device, a contact coordinate detection unit that is connected to the touch panel and outputs contact coordinate data indicating a touch position on an operation surface touched by a user based on a touch detection signal from the touch panel. The user viewpoint determination unit detects a touch position in the left-eye parallax image or the right-eye parallax image touched by the user based on the contact coordinate data from the contact coordinate detection unit, and indicates the touch position Touch position coordinate data is output, and the display control unit indicates the touch operation of the user based on the contact coordinate data from the contact coordinate detection unit and the touch position coordinate data output from the user viewpoint determination unit. The type of image processing is determined, and the virtual three-dimensional space processing unit is controlled so that the determined type of image processing is performed. It is preferable.

  An electronic information device according to the present invention has a display device having a display surface for displaying an image, and an operation surface provided to be superimposed on the display surface of the display device, and inputs information by a touch operation on the operation surface. An electronic information device equipped with an operation panel device having a touch panel to perform, the operation panel device being the above-described operation panel device according to the present invention, thereby achieving the above object.

  Next, the operation will be described.

  In the present invention, in the operation panel device, a three-dimensional image space processing unit that generates three-dimensional image data defining an object located in the virtual three-dimensional space, and the left-eye parallax of the object based on the three-dimensional image data The image data of the image and the right-eye parallax image is generated, a display image generation unit that supplies the display device with the display device so that the object is stereoscopically displayed on the display surface of the display device, and a user viewpoint for the stereoscopically displayed object is determined A user viewpoint determination unit, and an operation position on the operation surface of the touch panel is set as an operation position for a two-dimensional image when the stereoscopically displayed object is viewed from the determined user viewpoint. The position is determined based on the parallax image that the user's dominant eye sees, so that the user's dominant eye is the left eye. Regardless of whether the user has a right eye or a right eye, the user can operate the stereoscopic image as intended by the user while displaying the stereoscopic image. This makes the display image uncomfortable when the user performs a touch operation. It is possible to avoid unintentional image processing due to a touch operation due to a feeling or a difference in user's dominant eye.

  In the present invention, the user viewpoint determination unit calculates a user viewpoint in a virtual three-dimensional space, and displays a two-dimensional image when the stereoscopically displayed object is viewed from the calculated user viewpoint. Since it is configured to be used as a two-dimensional image to be touched on the surface, the position determination of the user operation with respect to the three-dimensionally displayed object is that the user substantially sees the three-dimensionally displayed object. This is done based on the viewpoint, and this ensures that the user feels uncomfortable when the touch operation is performed, or that unintended image processing is performed by the touch operation due to the difference in the user's dominant eye. It is possible to avoid it.

  As described above, according to the present invention, regardless of whether the user's dominant eye is the left eye or the right eye, the stereoscopic image is displayed and the operation as intended by the user is performed on the stereoscopic image. Operation that can prevent the user from feeling uncomfortable with the displayed image when performing a touch operation, or unintentional image processing due to the touch operation due to the difference in user's dominant eye A panel device and an electronic information device having such an operation panel device can be obtained.

FIG. 1 is a view for explaining an operation panel device according to Embodiment 1 of the present invention, FIG. 1 (a) is a block diagram showing the configuration of the operation panel, and FIG. 1 (b) is the operation panel. It is a perspective view which isolate | separates and shows the touch panel and display apparatus which comprise an apparatus. FIG. 2 is a diagram for explaining the operation panel device according to the first embodiment of the present invention. The display coordinate system defined on the display surface of the display device constituting the operation panel device and the touch panel constituting the operation panel device. The operation coordinate system defined on the operation surface is shown. FIG. 3 is a diagram illustrating the operation panel device according to the first embodiment of the present invention. FIG. 3A illustrates an object in the virtual three-dimensional space created by the virtual three-dimensional space processing unit of the operation panel device. FIG. 3B shows a method for projecting an object onto a two-dimensional plane in the virtual three-dimensional space. FIG. 4 is a diagram illustrating a two-dimensional image created by the display image generation unit of the operation panel device according to the first embodiment of the present invention, and FIG. 4A illustrates the object in the virtual three-dimensional space. FIG. 4B shows an image viewed from the left eye (parallax image for left eye) when viewed from a position separated by a predetermined distance, and FIG. 4B illustrates the object viewed from a position separated by a predetermined distance in the virtual three-dimensional space. At this time, an image viewed by the right eye (a parallax image for the right eye) is shown. FIG. 5 is a diagram for explaining the operation panel device according to the first embodiment of the present invention. The left-eye parallax image is displayed on the display surface so that an object in the virtual three-dimensional space is stereoscopically displayed on the display device of the operation panel device. And the right-eye parallax image are displayed in a shifted manner. FIG. 6 is a diagram for explaining the operation of the operation panel device according to the first embodiment of the present invention with a flowchart. FIG. 7 is a diagram for explaining an operation panel device according to the second embodiment of the present invention, and shows a configuration of the operation panel. FIG. 8 is a diagram for explaining an operation panel device according to the second embodiment of the present invention. A two-dimensional plane onto which an object in a virtual three-dimensional space created by a virtual three-dimensional space processing unit of the operation panel device is projected. Is shown. FIG. 9 is a diagram for explaining an operation panel device according to Embodiment 2 of the present invention, in which an object is projected on a two-dimensional plane perpendicular to a straight line connecting a user viewpoint and the center of the object in a virtual three-dimensional space. The obtained two-dimensional image is shown. FIG. 10 is a diagram for explaining a cellular phone as a portable information terminal according to Embodiment 3 of the present invention. 11A and 11B are diagrams for explaining the operation panel device disclosed in Patent Document 2. FIG. 11A is a flock diagram showing the overall configuration, and FIG. 11B is an operation panel that constitutes the operation panel device. FIG.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(Embodiment 1)
FIG. 1 is a view for explaining an operation panel device according to Embodiment 1 of the present invention, FIG. 1 (a) is a block diagram showing the configuration of the operation panel, and FIG. 1 (b) is the operation panel. It is a perspective view which isolate | separates and shows the touch panel and display apparatus which comprise an apparatus.

  The operation panel device 100 according to the first embodiment is configured to be able to operate a three-dimensionally displayed object by a touch operation on the three-dimensionally displayed object, and determines a user's dominant eye as an operation mode. A user viewpoint determination mode and a normal display mode for operating a three-dimensionally displayed object are provided. In the first embodiment, the expression “three-dimensionally displayed object” has the same meaning as the expression “three-dimensionally displayed object” used in the description of the prior art. It is also called a stereoscopic image.

  That is, the operation panel device 100 has a display surface 101a for displaying an image, the display device 101 for displaying an image based on the image data Id on the display surface 101a, and an operation surface 102a. When the operation surface 102a is touched with a member such as a finger or a pen, the contact corresponding to the position (touch position) touched by the user is detected by detecting that the user's finger or pen or the like is in contact with the operation surface 102a. A touch panel 102 that outputs a detection signal Dt.

  The display device 101 and the touch panel 102 constitute an operation panel 100a for performing an operation on an object displayed on the display surface 101a by a touch operation on the touch panel 102, and the display surface of the display device 101a. The touch panel 102 is arranged on the display device 101 so that the operation surface 102a of the touch panel 102 overlaps the 101a. The operation surface 102a of the touch panel 102 is made of a transparent member so that the display surface 101a of the display device 101 can be seen through the operation surface 102a.

  Here, as shown in FIG. 2, a display coordinate system Dcs for specifying each point of the image displayed on the display surface 101a is set on the display surface 101a of the display device 101. Further, as shown in FIG. 2, an operation coordinate system Tcs for specifying a position touched on the operation surface 102a is set on the operation surface 102a of the touch panel 102. Each of the operation coordinate system Tcs and the display coordinate system Dcs is a two-dimensional coordinate system having an X axis and a Y axis as shown in FIG. Further, the display coordinate system Dcs and the operation coordinate system Tcs are displayed with the point Tp of the same coordinate of the operation coordinate system Tcs set on the operation surface 102 a of the touch panel 102 being set on the display surface 101 a of the display device 101. It is defined to coincide with the point Dp of the same coordinate in the coordinate system Dcs.

  In the operation panel 100a having such a configuration, the user's touch operation on the touch panel 2 is performed when the user touches the operation surface 102a of the touch panel with a finger or a pen.

  The operation panel device 100 indicates touch coordinates (contact coordinates) corresponding to the touch position in the operation coordinate system Tcs defined on the operation surface 102a of the touch panel, based on the contact detection signal Dt from the touch panel 102. A contact coordinate detection unit 103 that outputs contact coordinate data Cd, and a display control unit 107 that determines a user operation based on the contact coordinate data Cd and outputs command signals Cra and Crb corresponding to the determined user operation. is doing.

  Here, the touch coordinate detection unit 103 must have one touch coordinate calculated as the touch position in the user viewpoint calculation mode, but there are a plurality of touch coordinates calculated as the touch position in the normal operation. May be.

  Based on the command signal Crb from the display control unit 107, the operation panel device 100 performs virtual three-dimensional drawing such as drawing, rotation, movement, enlargement, and reduction of the object TB in the virtual three-dimensional space Ts shown in FIG. A virtual three-dimensional space processing unit 104 that performs image processing in the space Ts and outputs three-dimensional image data Pda and Pdb, and a 3D from the virtual three-dimensional space processing unit 104 as shown in FIG. Based on the two-dimensional image data Pda, the point of the object TB on the virtual three-dimensional space Ts is projected on the two two-dimensional planes Lp and Rp with the left eye Le and the right eye Re as viewpoints, respectively, and the parallax image LB for the left eye And the right-eye parallax image RB (see FIGS. 4A and 4B) are generated, and the image data of the left-eye parallax image LB and the right-eye parallax image RB are converted into objects in a virtual three-dimensional space. And a display image generation unit 106 to output as image data Id for stereoscopic display.

  That is, the virtual three-dimensional space processing unit 104 defines the position of each point of the object TB in the virtual three-dimensional space Ts based on the command signal Crb, thereby drawing, rotating, moving, enlarging the object, The display image generating unit 106 is configured to perform image processing such as reduction, and the display device 101 displays image data Id for stereoscopic display of the object subjected to image processing by the virtual three-dimensional space processing unit 104. Is configured to output.

  FIG. 3 is a diagram for explaining the operation panel device according to the first embodiment of the present invention. FIG. 3A shows a virtual three-dimensional space created by the virtual three-dimensional space processing unit of the operation panel device. FIG. 3B shows the two-dimensional plane corresponding to the left-eye and right-eye viewpoints, the object, and the left-eye and right-eye viewpoints when the object is projected onto the two-dimensional plane in the virtual three-dimensional space. The positional relationship is shown.

  FIG. 4 is a diagram showing a left-eye parallax image LB and a right-eye parallax image RB obtained by projecting the object TB in the virtual three-dimensional space Ts onto the two-dimensional planes Lp and Rp, and FIG. FIG. 4B shows an image (left-eye parallax image) LB seen by the left eye when the object is viewed from a position separated by a predetermined distance in the virtual three-dimensional space, and FIG. 4B shows the object in the virtual three-dimensional space. When viewed from a position separated by a predetermined distance, an image (right-eye parallax image) RB viewed by the right eye is shown. Here, an image of a baseball ball is shown as the object.

  Specifically, the two-dimensional plane Lp with the left eye Le as the viewpoint shown in FIG. 4A is a plane perpendicular to the straight line LLn connecting the position of the left eye Le and the center TBc of the object TB in the virtual three-dimensional space Ts. The left-eye parallax image LB is obtained by projecting the object TB onto the two-dimensional plane Lp, and the two-dimensional plane Rp with the right eye Re shown in FIG. It is defined as a plane perpendicular to the straight line RLn connecting the position of the right eye Re and the center TBc of the object TB in the three-dimensional space Ts, and the right-eye parallax image RB is obtained by projecting the object TB onto the two-dimensional plane Lc. It is a thing.

  Here, in the virtual three-dimensional image space Ts, as shown in FIGS. 3A and 3B, three coordinate axes TX for specifying a position in the virtual three-dimensional image space Ts, A three-dimensional coordinate system Tc having TY and TZ is set, and the position P of each point in the virtual three-dimensional space Ts is represented by three-dimensional coordinates (Px, Py, Pz).

  Further, as shown in FIG. 4A, a two-dimensional coordinate system Lc having two coordinate axes LX and LY for specifying a position in the two-dimensional plane Lp is set in the two-dimensional plane Lp. The position of each point L in the two-dimensional plane Lp is represented by two-dimensional coordinates (Lx, Ly).

  In the two-dimensional plane Rp, as shown in FIG. 4B, a two-dimensional coordinate system Rc having two coordinate axes RX and RY for specifying a position in the two-dimensional plane Rp is set. The position of each point R in the two-dimensional plane Rp is represented by two-dimensional coordinates (Rx, Ry).

  In addition, the image data Id output from the display image generation unit 106 is such that the left-eye parallax image LB on the two-dimensional plane Lp and the right-eye parallax image RB on the two-dimensional plane Rp are displayed interlaced on the display surface 101a. It is the image data created as follows. For example, in the display device 101, the left eye parallax image LB is generated by the pixels in the odd rows of the plurality of pixels arranged in a matrix that configures the display surface 101a based on the image data Id. The data structure is such that the right-eye parallax image RB is displayed with pixels in even rows among the plurality of pixels.

  Further, in the display device 101, on the display surface 101a based on the image data Id, as shown in FIG. 5, the left-eye parallax image LB and the right-eye parallax image RB are left and right by a distance corresponding to the binocular parallax Ed. Displayed out of position.

  FIG. 5 shows a parallax image for left eye LB and a parallax image for right eye RB on the display surface 101a so that an object in the virtual three-dimensional space is stereoscopically displayed on the display device of the operation panel device according to the first embodiment of the present invention. The state is shown in which the is shifted.

  That is, when the left-eye parallax image LB and the right-eye parallax image RB are displayed on the display surface 101a of the display device 101, the two-dimensional coordinate system Lc that defines the left-eye parallax image LB and the two-dimensional that defines the right-eye parallax image RB. The coordinate system Rc is shifted by a distance (two-eye parallax) Ed corresponding to the distance between both eyes in the X-axis direction (that is, the horizontal direction on the display surface 102a) of the display coordinate system Dcs set on the display surface 101a. , Associated with the display coordinate system Dcs.

  As a result, the user who is located away from the display surface 101a by a distance corresponding to the distance between the object and the observation position in the virtual three-dimensional space Ts is determined by the left-eye parallax image LB and the right-eye parallax image RB displayed on the display surface 101a. The object in the virtual three-dimensional space is recognized as a stereoscopic image.

  Further, in the user viewpoint calculation mode, the operation panel device 100 applies the three-dimensionally displayed object based on the 3D image data Pdb from the virtual 3D space processing unit 104 and the contact coordinate data Cd from the contact coordinate detection unit 103. It is determined which of the left-eye parallax image LB and the right-eye parallax image RB displayed on the display surface 101a is used to determine the position of the user's touch operation. In the normal operation mode, the left eye determined in the user viewpoint calculation mode A user viewpoint determination unit (hereinafter, referred to as a user viewpoint calculation unit) 105 that determines a position touched by the user using one of the parallax image LB and the right-eye parallax image RB is provided.

  Note that the three-dimensional image data Pdb from the virtual three-dimensional space processing unit 104 is data having only position information that defines each position of the object in the virtual three-dimensional space, and the display image from the virtual three-dimensional space processing unit 104 is displayed. The data is different from the three-dimensional image data Pda having color information in addition to the position information output to the generation unit 106.

  That is, in the user viewpoint calculation mode, the user viewpoint calculation unit 105 uses the two-dimensional plane Lp with the left eye Le and the right eye Re as viewpoints using the three-dimensional image data Pdb from the virtual three-dimensional space processing unit 104. A point of the object TB on the virtual three-dimensional space Ts is projected onto Rp to generate a left-eye parallax image LB and a right-eye parallax image RB (see FIGS. 4A and 4B), and the left eye The two-dimensional coordinate system Lc that defines the parallax image for LB and the two-dimensional coordinate system Rc that defines the parallax image for the right eye RB are set in the X-axis direction (that is, the display surface 102a) of the display coordinate system Dcs set on the display surface 101a. In the horizontal direction) is shifted by a distance (binocular parallax) Ed corresponding to the distance between both eyes of the human and is associated with the display coordinate system Dcs. The image data of the right-eye parallax image RB in data and two-dimensional coordinate system Rc, and stores in the internal memory 105a is replaced with the image data Id in the display coordinate system Dcs.

  In addition, as illustrated in FIG. 5, the user viewpoint calculation unit 105 displays a specific position of a three-dimensionally displayed object (for example, the object's position) with the left-eye parallax image LB and the right-eye parallax image RB displayed on the display surface 101a. The contact coordinate data Cd of the position U touched by the user who is instructed to touch the center is acquired from the contact coordinate detection unit 103 and stored in the internal memory 105a.

  Further, the user viewpoint calculation unit 105 determines the distance DL between the position U touched by the user and the position of the center Lc of the left-eye parallax image LB, and the distance DL between the position U touched by the user and the position of the center Rc of the right-eye parallax image RB. Compared with DR, a two-dimensional coordinate system that defines a parallax image whose center is closer to the touch position U of the user is determined as the user viewpoint coordinate system.

  Here, the distance DL is obtained by converting the coordinates (Lcx, Lcy) indicating the center of the left-eye parallax image LB in the two-dimensional coordinate system Lc into the coordinates (Ldx, Ldy) of the display coordinate system. It is expressed by a formula.

DL = ((Ldx−Ux) ² + (Ldy−Uy) ² ) ^1/2 (1)
The distance DR is obtained by converting the coordinates (Rcx, Rcy) indicating the center of the right-eye parallax image RB in the two-dimensional coordinate system Lc into the coordinates (Rdx, Rdy) of the display coordinate system, and using the following equation (2): expressed.

DR = ((Rdx−Ux) ² + (Rdy−Uy) ² ) ^1/2 (2)
For example, as shown in FIG. 5, when the distance DR is smaller than the distance DL, the user viewpoint calculation unit 105 uses the two-dimensional coordinate system Rc that defines the two-dimensional plane Rp with the right eye Re as the viewpoint as the user viewpoint coordinate system. Determine as.

  On the other hand, in the normal operation mode, the user viewpoint calculation unit 105 projects an object in the virtual three-dimensional space onto the two-dimensional plane (that is, the two-dimensional plane Rp) defined by the determined user viewpoint coordinate system. The position coordinate of each object on the dimension plane is acquired, this user viewpoint coordinate system is associated with the operation coordinate system Tcs of the touch panel 102, and the touch position touched by the user operation is detected as the position of this user viewpoint coordinate system, Touch position coordinate data Vp indicating the touch position of the user is output to the display control unit 107.

  The user viewpoint calculation unit 105 updates the user viewpoint coordinate system when the distance between the left and right viewpoints is changed to adjust the binocular parallax Ed in order to correct the parallax variation by the user. It is configured.

  Therefore, the display control unit 107 determines the touch position with respect to the object in the virtual three-dimensional space based on the touch position coordinate data Vp calculated by the user viewpoint calculation unit 105, and further, from the contact coordinate detection unit 103. Based on the contact coordinate data Cd, the type of touch operation on the display surface 102a is determined, the operation performed by the user on the touch panel is determined, and an object in the virtual three-dimensional space is determined according to the operation. Is determined, and a command signal Crb is output to the virtual three-dimensional space processing unit 104.

  The operation panel device 100 also includes a storage unit 120 that stores various data, and a control unit 110 that monitors and controls the operation of each unit.

  The control unit 110 detects the power-on of the operation panel device 100 and executes an initialization sequence. In the initialization sequence, a user name as user information is read, a startup screen is displayed on the display device, and the like. It performs initial setting.

  Next, the operation of the operation panel device according to the first embodiment will be described.

  FIG. 6 is a diagram for explaining the operation of the operation panel device according to the first embodiment of the present invention with a flowchart.

  First, when the operation panel device 100 is powered on (step S1), the control unit 110 executes an initialization sequence (step S2). In this initialization sequence, the control unit 110 reads the user information stored in the storage unit 120 and performs various initial settings such as the brightness and contrast of the screen displayed on the display surface 102a in the display control unit 107. Command. Here, the user information is, for example, a registered user name.

  Further, after executing this initialization sequence, the control unit 110 outputs command signals Cr7 and Cr5 to the display control unit 107 and the user viewpoint calculation unit 105 to instruct the user viewpoint determination processing. This user viewpoint determination process is performed when the user recognizes a stereoscopic image from the left-eye parallax image LB and the right-eye parallax image RB displayed on the display surface 101a of the display device 101 of the operation panel device 100. Is a process for detecting where the user's dominant eye is, which is the dominant eye determination process for determining whether the user's dominant eye is the left or right eye.

  When the command signal Cr5 is input, the user viewpoint calculation unit 105 determines whether or not the user viewpoint determination process has been completed (step S3). This determination is performed by the user viewpoint calculation unit 105 based on whether or not the user viewpoint information indicating the user viewpoint is stored in the storage unit 120.

  If the user viewpoint determination processing is completed as a result of the determination, the user viewpoint calculation unit 105 reads out the user viewpoint information R20 stored in the storage unit 120 and stores it in the internal memory 105a, and a notification signal Nt5 is sent to the control unit 110 to notify the control unit 110 that the user viewpoint determination processing has been completed (step S4).

  When notified by the notification signal Nt5 from the user viewpoint calculation unit 105 that the user viewpoint determination process is completed, the control unit 110 outputs a command signal Cr7 to the display control unit 107, and the operation panel The display control unit 107 is instructed to execute a display process for displaying the left-eye parallax image LB and the right-eye parallax image RB on the display device so that a user operation can be performed on an object (stereoscopic image) stereoscopically displayed on 100a.

  When the display control unit 107 receives an execution instruction for display processing from the control unit 110, the display control unit 107 causes the virtual three-dimensional space processing unit 104 to draw, rotate, move, enlarge, reduce, etc. the object in the virtual three-dimensional space Tc. The left eye parallax image LB for instructing the display image generation unit 106 to perform image processing and for forming a stereoscopic image on the display image generation unit 106 based on the three-dimensional image data Pda from the virtual three-dimensional space processing unit 104 The command signal Cra is used to create image data Id for displaying the right-eye parallax image RB.

  On the other hand, if the user viewpoint calculation unit 105 determines in step S3 that the user viewpoint determination process has not been completed, the notification unit Nt5 notifies the control unit 110 that the user viewpoint determination process has not been completed. (Step S5).

  When notified from the user viewpoint calculation unit 105 that the user viewpoint determination process is not completed, the control unit 110 outputs a command signal Cr7 to the display control unit 107 to display the user viewpoint determination process. Command the control unit 107.

  When the display control unit 107 receives the execution instruction of the user viewpoint determination process from the control unit 110, the display control unit 107 sends the three-dimensional image data Pda and Pdb defining the object TB in the virtual three-dimensional space Ts to the virtual three-dimensional space processing unit 104. The left-eye parallax for instructing the display image generation unit 106 to further stereoscopically display the object in the virtual three-dimensional space based on the three-dimensional image data Pda generated by the virtual three-dimensional space processing unit 104 Command to generate image data Id of image LB and parallax image RB for right eye.

  FIG. 3 is a diagram showing the object TB in the virtual three-dimensional space Ts. Here, as an example of the object TB, a baseball ball located in the virtual three-dimensional space Tc is shown. An arbitrary position P in the virtual three-dimensional space is defined by coordinates (Px, Py, Pz) on the three-dimensional coordinate Tc having three coordinate axes TX, TY, and TZ.

  Specifically, the virtual 3D space processing unit 104 3D image data (that is, each point of the object in the virtual 3D space) that defines the object TB (see FIG. 3A) in the virtual 3D space Ts. Data indicating the position) is created by computer graphic processing. As shown in FIG. 3B, the display image generation unit 106 uses the 3D image data Pda created by the virtual 3D space processing unit 104 to place the object TB in a predetermined position in the virtual 3D space Ts. The object TB is projected onto the two-dimensional space Lp with the left eye Le as the viewpoint when viewed from the left side to create a left-eye parallax image LB (see FIG. 4A), and the display image generation unit 106 further When the object TB is viewed from a predetermined position in the virtual three-dimensional space Ts, the object TB is projected onto the two-dimensional space Rp with the right eye Re as the viewpoint, and the right-eye parallax image RB (FIG. 4B). ).

  Further, the display image generation unit 106 superimposes the left-eye parallax image LB and the right-eye parallax image RB so that the left-eye parallax image RB and the left-eye parallax image RB are displayed on the display coordinate system Dcs on the display surface with a shift corresponding to the parallax Ed. A combined image is created, and image data Id indicating this image is output.

  Accordingly, as shown in FIG. 5, the left-eye parallax image LB and the right-eye parallax image RB for stereoscopic display of the object TB in the virtual three-dimensional space Ts are displayed on the display surface 101 a of the display device 101. In the system Dcs, the image is displayed in a state shifted by a distance corresponding to the parallax between the left and right eyes.

  In FIG. 5, the coordinate axes X, Y, LX, RX, LY, RY are shown for convenience of explanation, but these coordinate axes are not displayed on the actual display surface 101a.

  At this time, the user viewpoint calculation unit 105 also uses the three-dimensional image data Pdb created by the virtual three-dimensional space processing unit 104 to perform two-dimensional viewing with the left eye and the right eye as viewpoints, similarly to the display image generation unit 106. The object TB in the virtual three-dimensional space Ts is projected on the planes Lp and Rp to create a left-eye parallax image LB and a right-eye parallax image RB, and these parallax images are displayed on display coordinates set on the display surface 101a. The position coordinate (Dx, Dy) of the display coordinate system D indicating the position Dp of each point of the left-eye parallax image LB and the right-eye parallax image RB shown in FIG. Store in the internal memory 105a.

  As described above, in the state where the position coordinates of each point of the pair of parallax images for stereoscopic display of the object TB in the virtual three-dimensional space Ts are acquired by the user viewpoint calculation unit 105, a command from the display control unit 107 is obtained. In response to the signal Cra, the display image generation unit 106 touches the display device 101, for example, “the center of the object (baseball ball) TB”. Is displayed. The method of prompting the user to touch the three-dimensionally displayed object is not limited to the method of displaying a message, but may be a method of performing by voice. Further, the touch on the touch panel by the user may be performed by the user with a finger or with a pen.

  When the user who sees this message touches the position recognized as the center position of the three-dimensionally displayed object TB on the operation surface 102a overlapping the display surface 101a, the touch position output from the touch panel 102 is displayed. The contact detection signal Dt is input to the contact coordinate detection unit 103, and the contact coordinate detection unit 103 calculates the position coordinates (Tx, Ty) of the operation coordinate system Tcs indicating the touch position based on the contact detection signal Dt. Contact coordinate data Cd indicating the position coordinates is output to the user viewpoint calculation unit 105.

  The user viewpoint calculation unit 105 converts the position coordinate (Tx, Ty) of the operation coordinate system Tcs indicated by the coordinate information data Cd from the contact coordinate detection unit 103 into the position coordinate (Dx, Dy) of the display coordinate system Dcs. However, in this embodiment, since the operation surface 102a of the touch panel 102 is superimposed on the display surface 101a of the display device 101 so that the operation coordinate system Tcs and the display coordinate system Dcs coincide with each other, the contact coordinate detection unit 103 The position coordinate (Tx, Ty) of the operation coordinate system Tcs indicated by the contact coordinate data Cd is used as the position coordinate (Dx, Dy) of the display coordinate system Dcs as it is.

  For example, when the touch position of the user is a position indicated by the position coordinate (Ux, Uy) of the display coordinate system Dcs, the position coordinate (Ux, Uy) is the center of the left-eye parallax image LB in the display coordinate system Dcs. Using the position coordinates (Lcx, Lcy) shown and the position coordinates (Rcx, Rcy) showing the center of the right-eye parallax image RB in the display coordinate system Dcs, the touch position of the user is the left-eye parallax image LB and the right-eye parallax image RB. It is determined which of the centers is close.

  Specifically, as described above, using the expressions (1) and (2), the distance DL between the center position of the left-eye parallax image LB and the touch position of the user, the center position of the right-eye parallax image RB, and the user Whether the user's viewpoint position is the left eye or the right eye is determined from the magnitude relationship with the distance DR to the touch position.

  For example, as shown in FIG. 5, when the distance DR is smaller than the distance DL, the user viewpoint calculation unit 105 uses the two-dimensional coordinate system Rc that defines the two-dimensional plane Rp with the right eye Re as the viewpoint as the user viewpoint coordinate system. Determine as.

  When determining whether the user viewpoint is the right eye or the left eye, the user viewpoint calculation unit 105 stores the determined information as user viewpoint information R20 so that the user viewpoint determination process is not performed from the next time. 120 (step S6), and notifies the control unit 110 that the user viewpoint determination processing is completed by a notification signal Nt5.

  Thereafter, when the display control unit 107 instructs the display device 101 to stereoscopically display the object TB in the virtual three-dimensional space Ts in the normal operation mode by the command signal Cr5 from the control unit 110, the display control unit 107 The virtual three-dimensional space processing unit 104 is commanded by a command signal Crb to perform image processing such as drawing, rotation, movement, enlargement, and reduction of the object TB in the virtual three-dimensional space Tc, and the display image generation unit 106 In addition, based on the three-dimensional image data Pda created by the virtual three-dimensional space processing unit 104, a command signal for creating the left-eye parallax image LB and the right-eye parallax image RB for stereoscopic display of the object Command by Cra.

  The virtual three-dimensional space processing unit 104 defines three-dimensional image data that defines the object TB (see FIG. 3A) in the virtual three-dimensional space Ts (that is, data indicating the position of each point of the object in the virtual three-dimensional space). Pda and Pdb are created by computer graphic processing. As shown in FIG. 3B, the display image generation unit 106 uses the 3D image data Pda created by the virtual 3D space processing unit 104 to place the object TB in a predetermined position in the virtual 3D space Ts. A left-eye parallax image LB (see FIG. 4A) is created by projecting the object TB onto a two-dimensional space Lp with the left eye Le as a viewpoint when viewed from the viewpoint, and the display image generation unit 106 further includes: When the object TB is viewed from a predetermined position in the virtual three-dimensional space Ts, the object TB is projected onto the two-dimensional space Rp with the right eye Re as a viewpoint, and the parallax image RB for right eye (FIG. 4B) is displayed. create.

  Further, the display image generation unit 106 superimposes the left-eye parallax image LB and the right-eye parallax image RB so that the left-eye parallax image RB and the left-eye parallax image RB are displayed on the display coordinate system Dcs on the display surface with a shift corresponding to the parallax Ed. A combined image is created, and image data Id indicating this image is output.

  Accordingly, as shown in FIG. 5, the left-eye parallax image LB and the right-eye parallax image RB for stereoscopic display of the object TB in the virtual three-dimensional space Ts are displayed on the display surface 101 a of the display device 101. The system Dcs is displayed in a state where a distance corresponding to the parallax Ed between the left and right eyes is shifted. The user recognizes a stereoscopic image of the object TB in the virtual three-dimensional space Ts (Step S7).

  Thereafter, when a touch operation is performed on the touch panel 102, the touch panel 102 detects that a member such as a user's finger or a pen has come into contact with the operation surface 102a, and a position touched by the user (touch position) is detected. A corresponding contact detection signal Dt is output (step S8).

  The contact coordinate detection unit 103 indicates touch coordinates (contact coordinates) corresponding to the touch position in the operation coordinate system Tcs defined on the operation surface 102a of the touch panel, based on the contact detection signal Dt from the touch panel 102. The contact coordinate data Cd is detected and output to the user viewpoint calculation unit 105 and the display control unit 107 as data indicating the position coordinates of the display surface 101a in the display coordinate system Dcs (step S9).

  The user viewpoint calculation unit 105 creates a two-dimensional plane whose viewpoint is the direction determined by the dominant eye determination of the left eye and the right eye (here, the two-dimensional plane Rp whose right eye is the viewpoint) in the virtual three-dimensional space Ts. The right object parallax image RB is created by projecting the object TB (step S10).

  That is, since the user viewpoint calculation unit 105 has already completed the user viewpoint determination process, the user viewpoint calculation unit 105 selects a two-dimensional plane with the user's dominant eye as a viewpoint out of the two-dimensional planes Lp and Rp with the left eye and the right eye as the viewpoint. Only, for example, when the right eye is dominant, the object TB in the virtual three-dimensional space Ts is projected only on the two-dimensional plane Rp with the right eye as the viewpoint, and a right-eye parallax image RB is created. The position coordinate (Dx, Dy) of the display coordinate system D indicating the position Dp of each point of the image RB is acquired, and information on this position coordinate is stored in the internal memory 105a as image data for determining the touch position of the user. .

  Note that the processes in steps S9 and S10 are not necessarily performed in the order described above, and may be performed in parallel.

  Furthermore, the user viewpoint calculation unit 105 determines a position where the user performs a touch operation using one of the left eye parallax image LB and the right eye parallax image RB determined in the user viewpoint calculation mode (here, the right eye parallax image RB). That is, the user viewpoint calculation unit 105 projects the right-eye parallax image in which the position indicated by the contact coordinate data Cd output from the contact coordinate detection unit 103 is projected on the two-dimensional plane Rp with the right eye as the viewpoint in the display coordinate system Dcs. The position corresponding to the RB is determined, and touch position coordinate data Vp indicating the position of the right-eye parallax image RB corresponding to the touch position is output to the display control unit 107.

  The display control unit 107 determines which position on the three-dimensionally displayed object the user has touched based on the touch position coordinate data Vp, and further, based on the contact coordinate data Cd output from the contact coordinate detection unit 103. Determine what operation was performed. For example, it is determined whether the operation is an operation for rotating an object with a vertical axis passing through the center thereof as a rotation axis, an operation for moving the object, or an operation for enlarging or reducing (step S11).

  Thereafter, when the display control unit 107 instructs the virtual three-dimensional space processing unit 104 by the command signal Crb to perform image processing according to the determined operation, the virtual three-dimensional space processing unit 104 performs user graphic processing by computer graphic processing. Image processing according to the operation is performed on the object, and the display surface 101a of the display device 101 displays a three-dimensional display of a rotating object, a moving object, or an enlarged or reduced object according to a user operation. (Step S12).

  As a result, the problem that the parallax image looks different for each user depending on the dominant eye can be solved, and the user can perform a comfortable operation.

  As described above, in the first embodiment, the user's viewpoint when viewing the stereoscopically displayed object in the virtual three-dimensional space is detected based on the positions of the right eye and the left eye in the virtual three-dimensional space, and the user viewpoint is As a two-dimensional plane for projecting an object and creating a left-eye parallax image or a right-eye parallax image for determining a user's touch position depending on which one is close to the left eye, a two-dimensional plane and a left eye that are viewed from the right eye Is set to any one of the two-dimensional planes with the viewpoint as the viewpoint, regardless of whether the user's dominant eye is the left eye or the right eye, the user can display the stereoscopic image and The intended operation can be performed with high accuracy, which makes the display image feel uncomfortable when the user performs the touch operation, or the touch operation due to the difference in the user's dominant eye It is possible to avoid or more unintended image processing is performed.

  For this reason, in order to improve operability, the proximity sensor detects the approach of a pressing member such as a finger or a touch pen, and switches between a pair of parallax images and a single planar image based on the detection information. A sensor for detecting the approach of the pressing means as in the prior art becomes unnecessary, and the design can be improved and the cost can be reduced.

  In the first embodiment, the user viewpoint is determined after the operation panel device is turned on and the initialization sequence is executed. However, the determination of the user is performed in the process in which the user operates. Based on the difference between the position where the user should touch the displayed object and the position where it is determined that the user actually touched, the viewpoint when the user is viewing the three-dimensional object is obtained. May be.

  In the first embodiment, the user viewpoint coordinate system is set to a two-dimensional coordinate system with the right eye or the left eye as a viewpoint. However, the user viewpoint coordinate system is a two-dimensional coordinate with the right eye or the left eye as a viewpoint. The system is not limited to the system, and a two-dimensional coordinate system having a viewpoint between the right eye and the left eye may be used.

(Embodiment 2)
7 to 9 are diagrams for explaining an operation panel device according to Embodiment 2 of the present invention, and FIG. 7 shows a configuration of the operation panel. FIG. 8 shows a two-dimensional plane onto which an object in the virtual three-dimensional space created by the virtual three-dimensional space processing unit of this operation panel device is projected, and FIG. 9 shows the object in the virtual three-dimensional space as the left eye. A two-dimensional image obtained by projecting onto the two-dimensional plane with the position between the right eyes as the viewpoint is shown.

  When the operation panel device 200 according to the second embodiment calculates a user viewpoint in the virtual three-dimensional space instead of the user viewpoint calculation unit 105 according to the first embodiment, and sees a stereoscopically displayed object from the calculated user viewpoint. The user viewpoint determination unit 205 that determines the two-dimensional image as a two-dimensional image to be touched on the operation surface of the touch panel, and the other configuration is the same as that of the operation panel device 100 of the first embodiment. Are the same.

  That is, the user viewpoint calculation unit 205 in the operation panel device 200 according to the second embodiment displays a stereoscopic image in a state where the left-eye parallax image LB and the right-eye parallax image RB are displayed on the display surface 101a as illustrated in FIG. The contact coordinate data Cd of the position U touched by the user who is instructed to touch a specific position of the object (for example, the center of the object) is acquired from the contact coordinate detection unit 103 and stored in the internal memory 205a.

  Further, the user viewpoint calculation unit 205 determines the distance DL between the position U touched by the user and the position of the center Lc of the left-eye parallax image LB, and the distance between the position U touched by the user and the position of the center Rc of the right-eye parallax image RB. From the DR, the user viewpoint position in the three-dimensional image space Ts corresponding to the viewpoint position when the user is viewing the stereoscopically displayed object is calculated.

  Then, as shown in FIG. 8, the user viewpoint calculation unit 205 defines a two-dimensional plane Mp as a plane perpendicular to the straight line MLn connecting the user viewpoint Me determined in this way and the center TBc of the object TB, Image data of the user viewpoint image MB (FIG. 9) obtained by projecting the object TB onto the dimension plane Mp is created.

  Then, the user viewpoint calculation unit 205 uses the user viewpoint image MB as a two-dimensional image that is a target of the touch operation on the operation surface 102a of the touch panel 102, and determines the position where the user performs the touch operation. That is, the user viewpoint calculation unit 205 determines in the display coordinate system Dcs which position of the user viewpoint image MB the position indicated by the contact coordinate data Cd output from the contact coordinate detection unit 103 corresponds to, and touches Touch position coordinate data Vp indicating the position of the user viewpoint image MB corresponding to the position is output to the display control unit 107.

  Other operations are the same as those of the operation panel device of the first embodiment.

  In the operation panel device 200 according to the second embodiment, when the right eye viewpoint, the left eye viewpoint, and the user viewpoint for generating the parallax image are rotated or moved with respect to the object in the virtual three-dimensional space, the right eye viewpoint, the left eye viewpoint In addition, the rotation is performed using the same rotation axis set in the virtual three-dimensional space, and the movement is performed by moving the three viewpoints in the same direction and the same distance in the virtual three-dimensional space so that the relative positional relationship between the user viewpoint and the user viewpoint is not lost. It is done by the process of moving only.

  In the operation panel device 200 according to the second embodiment having such a configuration, a user viewpoint in a virtual three-dimensional space is calculated, and a two-dimensional image obtained by viewing a stereoscopically displayed object from the calculated user viewpoint is displayed on the touch panel. Since it is determined as a two-dimensional image to be touched on the screen, it is possible to perform a user operation on the stereoscopically displayed object from the viewpoint that the user is substantially looking at the stereoscopically displayed object. Thus, it is possible to reliably prevent the display image from feeling uncomfortable when the user performs a touch operation, or unintentional image processing due to the touch operation due to a difference in the dominant eye of the user.

Furthermore, although not specifically described in the first and second embodiments, for example, an electronic information device using at least one of the operation panel devices of the first and second embodiments as an information input device will be briefly described below.
(Embodiment 3)
FIG. 10 is an external view showing an example of a portable information terminal according to Embodiment 3 of the present invention.

  The mobile information terminal 20 is a mobile phone including an operation panel unit that combines a display device and a touch panel.

  That is, the mobile phone 20 includes a housing 21, a speaker 11 and a transmission microphone 14 incorporated in the housing 21, an operation key 13 provided on the surface of the housing 21, and the housing 21. It has the operation panel part 12 which combined the display apparatus and touch panel which were attached.

  In the cellular phone 20, the operation panel device 12 is one of the operation panel device 100 of the first embodiment and the operation panel device 200 of the second embodiment.

  Even in such a cellular phone 20, when an object in the virtual three-dimensional space is stereoscopically displayed, the stereoscopic image of the object remains displayed regardless of whether the user's dominant eye is the left eye or the right eye. Therefore, the user's intended operation can be accurately performed on the stereoscopic image, and when the user performs a touch operation, the display image feels uncomfortable, or the user feels uncomfortable due to the difference in user's dominant eye. It is possible to avoid image processing that is not performed.

  As mentioned above, although this invention has been illustrated using preferable embodiment of this invention, this invention should not be limited and limited to this embodiment. It is understood that the scope of the present invention should be construed only by the claims. It is understood that those skilled in the art can implement an equivalent range based on the description of the present invention and the common general technical knowledge from the description of specific preferred embodiments of the present invention. It is understood that the patent documents cited in the present specification should be incorporated by reference into the present specification in the same manner as the content itself is specifically described in the present specification.

  In the field of the operation panel device and the electronic information device including such an operation panel device, the present invention displays a stereoscopic image regardless of whether the user's dominant eye is the left eye or the right eye, The user's intended operation on the stereoscopic image is performed more accurately, which may cause the display image to feel uncomfortable when the user performs a touch operation, or may not be intended by the touch operation due to a difference in the user's dominant eye An operation panel device that can avoid performing image processing and an electronic information device having such an operation panel device can be obtained.

DESCRIPTION OF SYMBOLS 11 Speaker 12 Operation panel apparatus 13 Operation key 14 Transmission microphone 20 Mobile phone 21 Case 100 Operation panel apparatus 100a Operation panel 101 Display apparatus 101a Display surface 102 Touch panel 102a Operation surface 103 Contact coordinate detection part 104 Virtual three-dimensional space processing part 105 User viewpoint calculation unit 105a Internal memory 106 Display image generation unit 107 Display control unit 110 Control unit 120 Storage unit Cd Contact coordinate data Cr5, Cr7, Cra, Crb Command signal Dcs Display coordinate system Dp, L, R, Tp point Dt Contact detection Signal Ed Parallax Id Image data LB Left-eye parallax image Lc, Rc 2D coordinate system Lp, Rp 2D plane LX, LY, RX, RY, TX, TY, TZ coordinate axis Nt5 Notification signal Pda, Pdb 3D image data R20 User viewpoint information RB Right-eye parallax image TB Object Tcs Operation coordinate system Tc 3D coordinate system Ts Virtual 3D space Vp Touch position coordinate data

Claims (9)

An operation panel device comprising: a display device having a display surface for displaying an image; and a touch panel having an operation surface provided to be superimposed on the display surface of the display device and inputting information by a touch operation on the operation surface Because
A 3D image space processing unit for generating 3D image data defining an object located in the virtual 3D space;
A display image generation unit that generates image data of a left-eye parallax image and a right-eye parallax image of the object based on the three-dimensional image data, and supplies the image to the display device so that the object is stereoscopically displayed on the display surface of the display device When,
A user viewpoint determination unit that determines a user viewpoint for a stereoscopically displayed object;
An operation panel device, wherein an operation position with respect to an operation surface of the touch panel is an operation position with respect to a two-dimensional image when the stereoscopically displayed object is viewed from a determined user viewpoint. The operation panel device according to claim 1,
The user viewpoint determination unit calculates a user viewpoint in the virtual three-dimensional space, and determines whether the calculated user viewpoint is closer to the position of the user's left eye or right eye in the virtual three-dimensional space. An operation panel that uses the left-eye parallax image and the right-eye parallax image corresponding to the eye with the closer calculated user viewpoint as a two-dimensional image to be touched on the operation surface of the touch panel. apparatus. The operation panel device according to claim 1,
The user viewpoint determination unit calculates a user viewpoint in the virtual three-dimensional space, and performs a touch operation on the operation surface of the touch panel when a two-dimensional image is viewed from the calculated user viewpoint. An operation panel device used as a two-dimensional image to be subjected to. The operation panel device according to claim 1,
The display image generation unit projects the image of the object onto a first two-dimensional plane perpendicular to a straight line connecting the position of the user's left eye and the center of the object in the virtual three-dimensional space, and the left eye The right-eye parallax is generated by generating a parallax image and projecting the image of the object onto a second two-dimensional plane perpendicular to a straight line connecting the position of the right eye of the user and the center of the object in the virtual three-dimensional space. An operation panel device for generating an image. The operation panel device according to claim 1,
The user viewpoint determination unit determines whether a user viewpoint for a three-dimensionally displayed object has been determined following the initial setting after the power is turned on,
As a result of the determination, when the user viewpoint is determined, either the left-eye parallax image or the right-eye parallax image is displayed on the operation surface of the touch panel using information about the user viewpoint already stored. A two-dimensional image to be touched,
If the user viewpoint is not determined as a result of the determination, a user viewpoint for the stereoscopically displayed object is determined, and then either the left-eye parallax image or the right-eye parallax image is determined based on the determined user viewpoint. An operation panel device, wherein the two-dimensional image is a target of a touch operation on the operation surface of the touch panel. The operation panel device according to claim 5,
A storage unit for storing various information;
When the user viewpoint is determined as a result of the determination and the user viewpoint is determined for the stereoscopically displayed object, the user viewpoint determination unit stores information on the determined user viewpoint in the storage unit Operation panel device. The operation panel device according to claim 1,
In response to a touch operation on the touch panel, the virtual three-dimensional space processing unit performs specific image processing among various image processing including at least generation, rotation, movement, enlargement, and reduction of an object in the three-dimensional image space. An operation panel device comprising a display control unit for controlling the virtual three-dimensional space processing unit so as to perform The operation panel device according to claim 7,
A touch coordinate detection unit connected to the touch panel and outputting touch coordinate data indicating a touch position on an operation surface touched by a user based on a touch detection signal from the touch panel,
The user viewpoint determination unit detects a touch position in the left-eye parallax image or the right-eye parallax image touched by the user based on the contact coordinate data from the contact coordinate detection unit, and touch position coordinates indicating the touch position Output data,
The display control unit determines the type of image processing indicated by the user's touch operation based on the contact coordinate data from the contact coordinate detection unit and the touch position coordinate data output from the user viewpoint determination unit, An operation panel device that controls the virtual three-dimensional space processing unit so that the determined type of image processing is performed. An operation panel device comprising: a display device having a display surface for displaying an image; and a touch panel having an operation surface provided to be superimposed on the display surface of the display device and inputting information by a touch operation on the operation surface An electronic information device equipped with
The operation panel device is an electronic information device which is the operation panel device according to any one of claims 1 to 8.