JP2006146440A - Electronic equipment and information display selection method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To display a content to be presented or a function provided by equipment so that a user can visually easily see and intuitively operate it. <P>SOLUTION: In the electronic equipment, an information selection processing block 53 determines a selection range in the depth direction of Lm-Ln seen from the origin of a virtual camera coordinate system when Ln and Lm are a minimum value and a minimum value on an extension linear axis L in the imaging direction of the virtual camera. It also determines a cylindrical space with a radius tr and a height Lm-Ln as a selection space area when the maximum value of the distance from the axis L is tr. With respect to the thus-regulated selection space area, an object is selected on conditions that a distance from the virtual camera origin s is minimum when (1) the center of an object or the outer edge of the object is within the selection space area of the virtual camera, (2) when the object center or the outer edge of the object is within the maximum value tr of the B-C distance, and (3) when a plurality of objects satisfying (1) and (2) are present, and a distance t from the imaging central axis of the virtual camera is minimum when a plurality of objects satisfying (1) to (3) are present. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an electronic device that allows a user to play back or browse content such as music data, video data, and a website. The present invention also relates to an information display selection method for selecting a playback content presentation method and various functions of a device in a device having a function of playing back or browsing content such as music data, video data, and a website. .

  With the spread of digitization technology, so-called content data including music data, video data and the like has been digitally recorded or reproduced on a recording medium. If an information processing apparatus such as a personal computer is used, music content and video content can be recorded on a built-in recording medium such as a hard disk and managed.

  The development of portable information processing terminals such as PDAs (Personal Digital Assistants) and portable telephones is also remarkable. For example, although it is a portable phone, music data, video data, etc. can be played back, or it is possible to connect to the Internet and browse websites. With the progress of multimedia, the functional electronic devices are no longer distinguished. In addition, along with the development of networks such as the Internet, these portable information processing terminals can also download so-called content data such as music data and video data via the network. In addition, with the increase in the density of storage media, various types of content can be sufficiently stored even in portable information processing terminals.

  Since these portable information processing terminals are small in size and have a small display unit, an image with a large amount of information may not be displayed at a time. For this reason, the display screen is reduced and displayed at one time, or an image within a certain range is displayed on the display unit, for example, and an image to be displayed on the display unit is moved in response to a button operation or the like (so-called scrolling). To be able to do that.

  However, a portable information processing terminal cannot be equipped with an input device such as a keyboard or a mouse like a stationary or notebook personal computer, and the number of buttons to be installed for operation input is limited, The user can input input keys other than numbers to a limited input part such as a numeric keypad, or input directly to the display screen with an input pen, etc. It was difficult to perform operation input. In addition, the worse the operability of the terminal, the worse the operability becomes. Especially in the case of a terminal adopting an operation method using an input button, an input pen, etc., in order for the user to operate the terminal intuitively, After all, a certain amount of “familiarity” was necessary. Furthermore, if the information processing terminal is a type that can store a large amount of content, the process of selecting the desired content for the user is more complicated and the operation is more complicated.

  The operation of home appliances is a daily thing for the user, and it is a problem that there are difficulties in the operation method of these devices. It is not necessary for the user to perform complicated operations when selecting information stored in the home appliance, changing the function of the home appliance, or the like.

  Therefore, for example, in Japanese Patent Application Laid-Open No. 2002-7027 (Patent Document 1), a display unit such as a mobile phone or a mobile computer is displayed on the display unit by moving a housing (main body) in a small device. A technique for scrolling a screen is disclosed. For example, when the user moves the device in the right direction with this device in hand, the displayed image is similarly scrolled in the right direction.

  Further, by providing a sensor for detecting not only the x and y directions but also the movement in the z direction, the screen displayed on the display unit can be enlarged or reduced, and the screen angle can be changed by moving the apparatus. The invented electronic device has been proposed by the inventor of the present invention (see Patent Document 2).

JP 2002-7027 A Japanese Patent Application Laid-Open No. 2004-102736

  In the technique disclosed in Patent Document 2, when a user selects a selection item on a screen (two-dimensional) called an initial screen, the user moves the cursor to move the cursor to the desired selection item. It was. At this time, the device is made to indicate to the user that the user has selected by simply highlighting the selection item at the screen center or the position closest to the screen center in the screen.

  However, this method can be used without any problem when the selection area selected by the user is a simple screen (two-dimensional). For example, the virtual space is expanded to three dimensions and the selection items are associated on the three-dimensional space. When such a developmental case is assumed, it is expected that it will be difficult to select selection items accurately.

  For example, when the selection area and the selection item are expanded to three dimensions, “depth” is added to the positional relationship of the selection items in the three-dimensional virtual space. Therefore, even in the same direction in the three-dimensional virtual space in which the three-dimensional object as the selection item is displayed, it may be overlapped with a portion having a different depth. As described above, it is expected that it is difficult to distinguish a three-dimensional object as a selection item in a portion having the same direction and different depth.

  Therefore, the present invention has been proposed in view of the above-described conventional situation, and in an electronic device capable of executing display in which the display method of the target object displayed on the display unit and the movement of the device are interlocked, Even when the space is expanded to three dimensions, the displayed contents such as the contents to be presented and the functions of the device are displayed in a visually easy-to-understand manner for the user and the selection operation can be performed accurately, so that the user can intuitively It is an object to provide an information display selection method that allows an electronic device that can be operated, contents displayed by the device, and functions provided by the device to be visually and easily displayed for the user and that the user can select and operate intuitively. .

  In order to achieve the above-described object, the electronic device according to the present invention is a three-dimensional virtual space data in an electronic device that can display the moving direction of the device itself and an image of an object displayed on the device in conjunction with each other. And storage means for storing the three-dimensional virtual object data, a content related to the three-dimensional virtual object constructed from the three-dimensional virtual object data by constructing a three-dimensional virtual space from the three-dimensional virtual space data, or a storage location of the content Virtual space information processing means for linking information to be shown to a three-dimensional virtual object, display means for displaying a part of the constructed three-dimensional virtual space and contents, and plane movement detection for detecting movement of the device in the plane direction Means, a vertical movement detecting means for detecting movement of the device in the vertical direction, and a three-dimensional virtual space displayed on the display means. Display processing means for scrolling according to the amount of movement in the direction and the amount of movement in the vertical direction detected by the vertical movement detection means, and a coordinate system centered on the device in the three-dimensional virtual space displayed on the display means A predetermined effective space area that is fixed and displayed on the display means and is narrower than the three-dimensional virtual space is selected as a selected space area, and a three-dimensional virtual object or at least a part that touches the selected space area is included in the effective space area. The above-described object is achieved by providing information selection processing means for selecting content linked to a three-dimensional virtual object or information indicating the storage location of the content.

  In particular, the information selection processing unit preferentially selects a three-dimensional virtual object that is closer to the central axis of the selected space region and a three-dimensional virtual object that is closer to the origin of the coordinate system centered on the device. The selected space area here may be a cylinder, a polygonal column, a cube, a sphere, or a polygonal pyramid.

  In addition, in order to achieve the above-described object, the information display selection method according to the present invention displays the information displayed and displayed in conjunction with the moving direction of the electronic device itself and the image of the object displayed on the electronic device. In the information display selection method to be selected, a three-dimensional virtual space is constructed from the three-dimensional virtual space data, and the content related to the three-dimensional virtual object constructed from the three-dimensional virtual object data or the information indicating the storage location of the three-dimensional information is displayed. Virtual space information processing step linking to a virtual object, plane movement detection step for detecting movement of the electronic device in the plane direction, vertical movement detection step for detecting movement of the electronic device in the vertical direction, and virtual space information processing Depending on the amount of movement in the plane direction detected in the plane movement detection step and the amount of movement in the vertical direction detected in the vertical movement detection step for a part of the three-dimensional virtual space constructed in the process A display processing step for scroll display and a predetermined effective space region narrower than the three-dimensional virtual space that is fixed with respect to the coordinate system centered on the electronic device in the three-dimensional virtual space and is displayed in the display processing step. Information selection for selecting a three-dimensional virtual object that has contacted the selected space area or a content linked to a three-dimensional virtual object at least part of which is included in the effective space area or information indicating the storage location of the content The object mentioned above is achieved by having a processing step.

  In particular, the information selection processing step preferentially selects a three-dimensional virtual object that is closer to the central axis of the selected space region and a three-dimensional virtual object that is closer to the origin of the coordinate system centered on the device. The selected space area here may be a cylinder, a polygonal column, a cube, a sphere, or a polygonal pyramid.

  In the display processing step, the state of the three-dimensional virtual object selected by the selected space area is changed. Specifically, the three-dimensional virtual object selected by the selected space region may be scaled up or down periodically or irregularly, the color tone of the three-dimensional virtual object may be changed periodically or irregularly, The shape of the three-dimensional virtual object may be changed periodically or irregularly. Further, the state of the three-dimensional virtual object may be changed periodically or irregularly and a sound may be generated.

  According to the present invention, in an electronic device capable of performing display in which the display method of the target object displayed on the display unit and the movement of the device are interlocked, when the virtual space is expanded in three dimensions, the presented content or device is The display contents such as the provided functions can be displayed visually and easily for the user, and the user can operate intuitively.

  Further, according to the present invention, even if there is a difference in “depth” in the positional relationship of the selection items in the three-dimensional virtual space, it is possible to distinguish and select.

  Hereinafter, electronic devices shown as specific examples of the present invention will be described in detail with reference to the drawings. An example of the appearance of an electronic device 1 shown as a specific example of the present invention is shown in FIG. 1, and the hardware configuration is shown in FIG.

  The electronic device 1 includes a main body 2, an imaging unit 3, a speaker 4, a microphone 5, a display unit 6, and an operation unit 7, and a portable information processing terminal having a function of reproducing content such as music content and video content It is.

  The electronic device 1 has an external shape as shown in FIG. The electronic device 1 has a speaker 4, a microphone 5, and an operation unit 7 on the front surface 2 a of the main body 2 that is opposed to the front of the user during normal use. The operation unit 7 is provided with buttons for inputting instructions for determining / cancelling functions or selection items of the electronic device 1 such as a determination button 7a, a cancel button 7b, and a camera selection button 7c. In the electronic device 1, the imaging unit 3 is provided on the back surface 2 b of the main body 2. The imaging unit 3 includes an optical system including a CCD (Charge Coupled Device), a focus, an iris, a shutter, a lens, and the like, and can capture an external video when the camera selection button 7c is selected. In the electronic device 1, a display unit 6 is provided on the side surface 2 c of the main body 2. For example, a color liquid crystal display (LCD) can be used as the display unit 6. On the side surface 2d opposite to the side surface including the display unit 6, a handle unit 9 for the user to hold the electronic device 1 is provided. In addition, the electronic device 1 includes an external memory mounting portion 8 that can detachably mount a semiconductor memory 10 as an external storage device, for example.

  Although not shown, the electronic device 1 may include a disk-shaped magnetic recording medium such as an HDD (Hard Disc Drive) that stores content data as an internal configuration. In addition, the recording medium may be an optical disk, a removable optical disk that can be recorded and reproduced, and the like. Further, the electronic device 1 may include a mobile phone transmission / reception circuit, a TV tuner, and the like for connecting to a mobile phone communication network. Note that the appearance of the electronic apparatus 1 described here is an example, and is not limited to the form of FIG.

  A hardware configuration of the electronic apparatus 1 will be described with reference to FIG. As shown in FIG. 2, in the electronic device 1, a CPU 12, a ROM 13, a flash memory 14, a RAM 15, an external memory interface 16, and a sensor interface 17 are connected to a main bus 11. As various interfaces, for example, TTY (Tele Typewriter) 18, Ethernet (registered trademark) board (Ethernet (registered trademark) Board) 19, image processing unit (Image Processing Unit) 20, Visca interface (Visca Interface) 21 A VGA board 22 and an audio video interface 23 are connected to each other, and each of these components is centrally controlled by the CPU 12.

  The ROM 13 stores a program and the like necessary for processing in which the electronic device 1 displays the target object displayed on the display unit 6 in conjunction with the movement of the electronic device 1 itself, and the flash memory 14 It is provided for storing necessary data such as 3D virtual space data and 3D virtual object data. The RAM 15 functions as a work area for the CPU 12. The process in which the CPU 12 displays the three-dimensional virtual space will be described in detail later.

  In the electronic device 1, the external memory interface 16 is connected to the mounting unit 8, and data is transmitted to and received from the semiconductor memory 10 inserted in the mounting unit 8. The sensor interface 17 is connected in parallel to the angular velocity sensor 26 and the acceleration sensor 27 via the A / D converter 24 and the low-pass filter 25. The TTY 18 is connected to various operation buttons provided on the operation unit 7 and accepts input from the operation unit 7. The Ethernet board 19 is an interface for connecting to Ethernet. The electronic device 1 can be connected to a host device such as a server via Ethernet, for example, and can be connected to the Internet via a host device such as a server. Moreover, the electronic device 1 includes a communication modem and may be connected to the Internet independently. The image processing unit 20 is connected to the camera interface 28, and the camera interface 28 is connected to the imaging unit 3. The electronic device 1 can also capture image data into the electronic device 1 by the CCD of the imaging unit 3 and display it on the display unit 6. The Visca interface 21 is an interface for connecting to an external information processing device such as a personal computer. Through this interface, various controls of the electronic device 1 can be executed from an external information processing device.

  The VGA board 22 is connected to the LCD which is the display unit 6. The audio / video interface 23 is an interface with an audio / video system device. A signal from the audio / video device is taken into the electronic device 1 through the audio / video interface 23, or an audio signal or an image signal is audio. Can be output to video equipment.

  Next, an angular velocity sensor and an acceleration sensor provided in the electronic apparatus 1 will be described with reference to the drawings. FIG. 3 shows the movement of the electronic device 1 detected by the angular velocity sensor 26. The movement of the human arm can be approximated to the movement on a spherical surface centered on the elbow and shoulder. As shown in FIG. 3, when the user moves the electronic device 1, for example, the movement is in the α direction along the spherical surface with the elbow 31 as a fulcrum.

  The angular velocity sensor 26 is an element that detects the angular velocity in the rotation direction of the electronic device 1 and outputs it as a voltage. Since the angular velocity sensor 26 detects only the angular velocity of rotation in one direction, the angular velocity sensor 26 has two or more rotation directions, for example, the α direction. In the case of detecting movements in the two directions β and β, as shown in FIG. 4, the angular velocity sensor 26 is installed according to each rotation axis. At this time, the sensor detection result in the angular velocity sensor 26 is supplied to the CPU 12 through the low-pass filter 25. The CPU 12 performs time integration by a numerical integration method and calculates a displacement component in the direction of the rotation axis of each electronic device 1.

  In the electronic device 1, the angular velocity sensor 26 includes an angular velocity sensor 26 a that detects movement in the X direction, an angular velocity sensor 26 b that detects movement in the Y direction, and an angular velocity sensor 26 c that detects movement in the Z direction. . The X direction and the Y direction indicate a planar direction with respect to the display unit 6, and the Z direction indicates a vertical direction with respect to the display unit 6. The mounting position of each sensor is not particularly limited as long as it is a position that does not impair the function of the electronic device 1 or a position where the detection operation is not hindered by an external factor.

  On the other hand, the acceleration sensor 27 is an element that detects a biaxial or triaxial acceleration and outputs it as a voltage when the user moves the electronic device 1 while holding it, that is, moves the space. It is. One acceleration sensor 27 is a type of sensor that can detect movement and a stationary state in the X-axis direction, the Y-axis direction, and the Z-axis direction. Since the angular velocity sensor 26 is for detecting movement due to rotation, it cannot detect movement in the straight line direction accurately. Therefore, the acceleration sensor 27 may be used supplementarily to detect movement in the linear direction. The acceleration sensor 27 is preferably provided close to the angular velocity sensor 26a and the angular velocity sensor 26b, but may be provided at a distance from these. The angular velocity sensor 26 and the acceleration sensor 27 are connected to the CPU 12 via the low-pass filter 25 and the A / D converter 24.

  As described above, when the acceleration sensor 27 is used as a sensor that detects acceleration in three axial directions, the vertical component, the horizontal component, and the front-rear direction component of the acceleration applied to the electronic device 1 can be detected. Then, a time integration operation is executed for each component to calculate a speed component and a displacement component. When the electronic device 1 is in a stationary state, no acceleration is applied to the electronic device 1, so the output voltage from the acceleration sensor 27 is a voltage when no acceleration is applied. Accordingly, the displacement and the stationary state of the electronic device 1 can be detected.

  In general, the relationship between acceleration and speed is expressed by the following equation.

  If there is an acceleration sensor that detects acceleration based on the equation (1), the moving speed can be obtained by integrating the data obtained in a certain period width T. If the moving speed of the electronic device 1 for a certain period is obtained, the moving distance can be obtained by integrating the speed value, and the scroll amount of the screen may be determined based on the moving amount.

  However, in order to obtain the moving distance of the electronic device 1 from the acceleration, it is necessary to perform double integration by time, so that responsiveness is delayed or an accumulated error tends to occur. In this specific example, the angular velocity sensor 26 is used as a means for detecting the movement amount of the electronic device 1 because the angular velocity may be used to detect the movement of the arm described above. When the scroll amount of the screen is obtained from the angular velocity, the equation (2) is obtained.

  In this specific example, the electronic device 1 scrolls the screen, enlarges / reduces, or changes the angle according to the result. The calculation results of the angular velocity sensor 26 and the acceleration sensor 27 are used for image scrolling to be described later.

  Next, processing for selecting a three-dimensional virtual object in the three-dimensional virtual space displayed on the display unit 6 based on the sensor data detected by the angular velocity sensor 26 and the acceleration sensor 27 described above will be described.

  FIG. 5 illustrates a process in which the electronic device 1 displays the target object displayed on the display unit 6 in conjunction with the movement of the electronic device 1 itself, and the three-dimensional virtual object included in the displayed screen. The functional block of CPU12 in the electronic device 1 for the process to select is represented. The electronic device 1 includes a virtual space information processing block 51, a display processing block 52, and an information selection processing block 53 as functional blocks of the CPU 12.

  The virtual space information processing block 51 constructs a three-dimensional virtual space from the three-dimensional virtual space data stored in the flash memory 14 and further constructs a three-dimensional virtual object (hereinafter referred to as an object) constructed from the three-dimensional virtual object data. ) To the content related to the object or information indicating the storage location of the content is linked to form a display screen.

  The display processing block 52 performs processing for changing the three-dimensional virtual space displayed on the display unit 6 in accordance with sensor data from the angular velocity sensor 26 and the acceleration sensor 27. The display process includes a process of changing an angle such as horizontal or vertical scrolling, enlargement / reduction, roll (rolling direction), pan (horizontal swing), tilt (vertical swing).

  The information selection processing block 53 determines whether or not the object displayed on the display unit 6 is in a predetermined space area fixed with respect to the coordinate system centered on the electronic device 1 itself in the three-dimensional virtual space. An object is selected. The predetermined space area here is defined as a space area narrower than the portion of the three-dimensional virtual space currently displayed on the display unit 6. Hereinafter, this space area is referred to as a selected space area. In the three-dimensional virtual space, the information selection processing block 53 is linked to a three-dimensional virtual object that is in contact with the selected space area, or a three-dimensional virtual object that is at least partially included in the selected space area. The content or information indicating the storage location of the content is selected data.

  The electronic device 1 can display the three-dimensional virtual space on the display unit 6 by executing the display process using the functional blocks described above. If the user moves the device itself as if looking in the virtual space while looking at the virtual space displayed on the display unit 6, the information displayed on the screen changes in conjunction with this movement, so it is more intuitive. Input operation can be realized.

  In this way, a user interface that allows the user to select an image or information displayed on the screen of the apparatus by moving the apparatus itself may be represented as HAB (Human-centric Augmented Browser).

  The display selection method in this specific example will be specifically described below. First, the selected space area defined in the three-dimensional virtual space will be described.

  The electronic device 1 shown as a specific example of the present invention assumes, for example, an imaging range (effective space region) of a virtual camera that exists in a three-dimensional virtual space as a selected space region.

FIG. 6 illustrates an object, a selected space area, and a virtual camera coordinate system in the three-dimensional virtual space. FIG. 6 shows a state where the objects O _{1 to} O ₈ are arranged in the three-dimensional virtual space S ₀ constructed based on the absolute coordinate system (X, Y, Z). The position of the object can be expressed in absolute coordinates. In the three-dimensional virtual space shown in FIG. 6, a virtual camera coordinate system (x, y, z) of a virtual camera assumed for the electronic device 1 existing in the virtual space is shown. The coordinates (x0, y0, z0) of this virtual camera coordinate system represent the position of the virtual camera in the three-dimensional virtual space. FIG. 7 illustrates a case in which the selected space area is the imaging area of the virtual camera in the electronic device 1 and illustrates a state viewed from the direction perpendicular to the imaging direction of the virtual camera assumed in the three-dimensional virtual space. .

  Let A be the position of the virtual camera in the three-dimensional virtual space, and B be the center position of the object. Also, let L be a straight line that passes through A and is parallel to the direction vector d in the imaging direction of the virtual camera. At this time, let C be the leg of the perpendicular line from the object center B to the straight line L, s be the distance between AC, and t be the distance between BC. When the origin is 0, it can be expressed as OC = OA + s * d. Here, a sign is defined for the inter-AC distance s.

  For the three-dimensional virtual space represented in this way, the selected space area, that is, the imaging area of the virtual camera is determined as follows. First, a selection range in the depth direction of the virtual camera is provided. For example, as shown in FIG. 8, a minimum value Ln and a maximum value Lm are determined in advance on the L axis, and Lm-Ln is set as a selection range in the depth direction when viewed from the origin of the virtual camera coordinate system. Further, the maximum value tr of the distance between BCs is determined. At this time, the minimum value Ln may be zero. That is, the selected space area may start from the origin of the virtual camera in the three-dimensional virtual space. As shown in FIG. 6, the maximum value tr is a display method in which the central axis in the imaging direction of the virtual camera is aligned with the perpendicular of the display screen, and the origin of the virtual camera coordinate system is on the approximate display screen. , Represented as area A on the display screen.

Therefore, in the electronic apparatus 1 shown as a specific example of the invention, as shown in FIG. 6, the area A _min at a distance of the minimum value Ln on the extension of the virtual camera imaging direction from the virtual camera origin is the top surface, and the maximum value Lm A cylindrical space whose bottom surface is the region _Amax at a distance is a selected space region. However, the selected space area is invisible to the user on the display screen.

  An object satisfying the following condition is selected for the selected space area defined in this way. (1) When the object center or the outer edge of the object is within the selected space area of the virtual camera. (2) When the object center or the outer edge of the object is within the maximum value tr of the distance between BCs. When there are a plurality of objects satisfying the above (1) and (2), (3) among these objects, an object having the smallest distance s from the virtual camera origin is preferentially selected. Further, when there are a plurality of objects satisfying (1) to (3), (4) among these objects, an object having the smallest distance t from the virtual camera imaging central axis is selected. Further, (5) the priority order of the objects is determined in advance so that the object can be selected even when there are a plurality of objects satisfying (1) to (4). For example, an object created first among the objects may be selected.

At this time, a rectangular parallelepiped s ₀ that is the smallest rectangular parallelepiped including the selected space area of the virtual camera and whose surfaces are parallel or perpendicular to the absolute coordinate system is used as a temporary selected space area, and objects that are not included in the temporarily selected space area are displayed. If the information is excluded from the selection candidates, the amount of calculation in the information selection processing block 53 can be reduced.

  As described above, the information selection processing block 53 selects an object that satisfies all the conditions (1) to (5), and extracts information linked to the object as selection data. If no object satisfies the condition, no object is selected. Further, the selection is limited only to the object to which the content and related information related to the content are linked. Examples of the related information include a URL as address information of a storage location where content is stored on a network.

At this time, the virtual space information processing block 51 and the display processing block 52 perform a process of changing the state of the selected object, and presents to the user that the object has been selected in an easy-to-understand manner. As an object display pattern, for example, a method of enlarging or reducing a selected object can be cited. In this case the virtual space information processor block 51, the display processing block 52, as shown in FIG. 9, to enlarge the object O ₅ which is selected. Further, as shown in FIG. 10, it is also possible to change the color tone of the object O ₅ which is selected. Further, as shown in FIG. 11, may be clearly to the user that the object by changing the shape of the object O ₅ which is selected is selected. Such object changes may be repeated periodically or irregularly. Further, in addition to the change of the state of the selected object, a sound indicating that the object has been selected may be generated. Also, it is possible to use a combination of sound generation and state change.

  In FIGS. 6 to 8, the description is simplified by using symbols such as A and C for convenience of explanation, but more precisely, it can be clarified by space vector calculation based on a three-dimensional virtual space. it can. Moreover, since the virtual camera assumed here is for clarifying the concept of the selected space region, the virtual camera may be displayed visually to the user in the virtual space displayed on the display unit 6 or displayed. It may not be done.

  Further, the example shown in FIG. 7 is a display method in which the central axis in the imaging direction of the virtual camera matches the perpendicular line of the display screen. On the other hand, there is a display method in which the vertical line of the display screen and the center axis of the imaging direction of the virtual camera do not coincide with each other, but considering that the user can operate more intuitively, as shown in FIG. A display method is preferred in which the central axis of the imaging direction coincides with the perpendicular of the display screen and the origin of the virtual camera coordinate system is assumed on the approximate display screen. This also simplifies 3D vector operations.

Further, the selection space area is not limited to a cylinder. It may be a rectangular parallelepiped including a cube, a sphere, or a polygonal pyramid. Therefore, the shapes of the region A _min and the region A _max that define the selected space region are not limited to a circle. In particular, using a polygon has the advantage that the amount of computation can be further reduced.

  As described above, according to the electronic device 1, when an object is selected in the three-dimensional virtual space, even if the object exists in the same direction in the three-dimensional space, even if the object overlaps at different depths. , It is possible to distinguish and select objects located at different depths. As a result, the electronic device 1 displays the display contents such as the content to be presented and the functions provided in the device so that the user can easily understand it visually, and the user can operate intuitively.

  Hereinafter, a state in which the three-dimensional virtual space displayed on the display unit 6 of the electronic device 1 and the display screen are changed in conjunction with the operation of the electronic device 1 will be described with reference to FIGS. 11 to 15. FIG. 12 shows the direction in which the electronic device 1 is actually moved. 13 to 15 show examples of a three-dimensional virtual space and objects displayed on the display unit 6 of the electronic device 1.

  Assume that the electronic device 1 is activated and displays a three-dimensional virtual space. At this time, the electronic device 1 is at a position O in FIG. The displayed three-dimensional virtual space is prepared as a selection screen for selecting content stored in the electronic device 1 or content acquired from an external storage medium or network connected to the electronic device 1. This selection screen is displayed on a shelf behind a table 63 on which a portable music player 61 and a portable telephone 62 are placed as shown in FIG. 13, for example, so that the user can easily recall the classification of contents to be selected. The three-dimensional virtual space configuration is a room 60 in which a television 65 is arranged at 64. The portable music player 61 is an object linked to a selection screen for selecting music content, and the television 65 is an object linked to a selection screen for selecting video content. The mobile phone 62 is an object for selecting the mobile phone function of the electronic device 1. The virtual space information processing block 51 described above links to these objects content related to the object, information (URL) indicating the storage location of the content, and the like.

  In human sense, when looking at a distant building, the whole can be seen with a wide viewing angle, but it cannot be discerned to details. Further, when a nearby building is viewed, it can be partially determined in detail, but it is difficult to grasp the entire building. Therefore, when a three-dimensional virtual space is displayed on the display unit 6, when the entire image is desired to be viewed with a wide field of view, the electronic device 1 is moved backward, for example, away from the building. Then, the image displayed on the display unit 6 is zoomed out so that a human moves backward and looks at the building. Further, when the user wants to see a detailed portion of the image, the electronic device 1 is moved forward so as to approach the building, for example. Then, the image displayed on the display unit 6 is zoomed in (enlarged) so that the person moves forward and looks at the building.

  When the display device 1 is gradually moved from the position O to the position A in FIG. 12, the three-dimensional virtual space displayed on the display unit 6 changes every moment and zooms in. Assuming that the electronic device 1 reaches the position A and is stationary, as shown in FIG. 14, the three-dimensional virtual space is enlarged and displayed as compared with the image shown in FIG.

  Further, when the electronic device 1 is gradually moved from this state toward the original position O, the three-dimensional virtual space shown on the display unit 6 is gradually zoomed out. When the electronic apparatus 1 is stopped at the position O, the three-dimensional virtual space is displayed in a state of being reduced and returned to the original state as shown in FIG.

  The angle change mode is a function for changing the angle of the display image based on the result detected by the angular velocity sensor 26 when the electronic device 1 is rotated. In human sense, for example, when looking at the sky, the viewpoint is turned up. When looking at the ground, look down at the viewpoint. Therefore, when an image of a room in the display unit 6 is displayed, when the ceiling of the room is to be displayed, the electronic device 1 is turned up so that the viewpoint is directed to the sky. At this time, the angle of the image displayed on the display unit 6 is changed and the ceiling is displayed so that a human looks at the sky with the viewpoint facing upward. For example, when it is desired to display the floor of this room, the electronic device 1 is turned downward so that the viewpoint is directed to the ground. At this time, the angle of the image displayed on the display unit 6 is changed and the floor is displayed so that a human looks at the ground with the viewpoint facing down. By combining these two modes, an operation close to human intuition is possible.

  When the electronic device 1 is gradually moved from the state fixed at the position O toward the position B shown in FIG. 12, the angle of the three-dimensional virtual space displayed on the display unit 6 changes to the movement of the electronic device 1. In conjunction, it gradually changes as it wraps around the displayed object. When the electronic device 1 is fixed at the position B, the displayed three-dimensional virtual space is displayed at the image angle shown in FIG. Further, when the electronic device 1 is gradually moved from this state toward the original position O, the angle of the three-dimensional virtual space shown on the display unit 6 is gradually changed. When the electronic device 1 is fixed at the position O, the three-dimensional virtual space is displayed at the original angle shown in FIG.

  Although not shown, the three-dimensional virtual space also supports angle changes such as roll (rolling direction), pan (horizontal swing), tilt (vertical swing) with the electronic device 1 stationary. Changed.

  In the configuration of the display space shown in FIG. 13, the user moves the electronic device 1 with the intention of selecting the portable music player 61 to change the angle and display state of the three-dimensional virtual space of the display screen, and thereby the portable music player. When guiding 61 near the center of the screen, the television 65 may be located behind the portable music player 61 depending on the relative positional relationship between the user and the object in the three-dimensional virtual space.

  Even in such a case, the electronic apparatus 1 can select the portable music player 61 based on the display selection process using the selected space area described with reference to FIG. In the display space shown in FIG. 13, the portable music player 61 is “in front” of the object represented as the portable music player 61 and the object represented as the television 65.

  At this time, the information selection processing block 53 in the electronic apparatus 1 is the object selection condition described above, that is, (1) When the object center or the outer edge of the object is within the selection space area of the virtual camera. (2) When the object center or the outer edge of the object is within the maximum value tr of the distance between BCs. When there are a plurality of objects satisfying the above (1) and (2), (3) based on the selection condition that the object having the smallest distance s from the virtual camera origin is selected preferentially among these objects. To select an object. The configuration of the three-dimensional virtual space shown in FIG. 13 corresponds to the case where there are a plurality of objects satisfying (1) and (2). (3) Among these objects, the distance s from the virtual camera origin is the smallest. Select objects with priority. Further, when the portable music player 61 and the portable telephone 62 are located at substantially the same distance from the virtual camera origin due to the configuration of the displayed three-dimensional virtual space, (4) The object having the smallest distance t from the virtual camera imaging center axis is selected. In the three-dimensional virtual space shown in FIG. 13, since information is not linked to the table 63, the shelf 64, and the like, even if they are included in the selected space area, they are not selected.

  The priority order in the selection conditions (1) to (4) can be freely set by the manufacturer or the user. For example, in the case of the display screen configuration shown in FIG. 13, the portable music player 61 is closer to the user's viewpoint of the virtual camera, but priority is given to the shorter distance between the straight line L extended in the imaging direction and the object center. If the setting is selected, that is, if the order of the condition (3) is raised over the condition (4), the television 65 at the “back” at the same angle is preferentially selected.

  Here, as shown in FIG. 16, the selected portable music player 61 is displayed larger than the original display size and highlighted. The display method of the selected portable music player 61 is not limited to enlargement / reduction, and may be a display method that enhances the user's visibility, such as blinking display, repeated display of enlargement / reduction, or changing the color tone. Can be applied.

  As described above, according to the electronic apparatus 1 shown as a specific example of the present invention, the television object displayed behind the portable music player 61 that the user wants to select or displayed as if it was placed nearby. The mobile phone objects that are present are not selected at the same time. In this way, the electronic device 1 can distinguish and select the difference in “depth” in the three-dimensional virtual space and the objects displayed in a spatially overlapping manner.

It is an external view explaining an example of the external appearance of the electronic device shown as a specific example of this invention. It is a block diagram explaining the hardware constitutions of the said electronic device. It is a schematic diagram explaining the movement of the electronic device which arises by the movement of a human arm. It is a schematic diagram explaining the angular velocity sensor which detects the motion of 2 directions, (alpha) direction and (beta) direction. It is a functional block diagram explaining the function of CPU for displaying how the electronic device displays the target object displayed on the display unit and the movement of the electronic device itself in conjunction with each other. It is a figure explaining the object in the three-dimensional virtual space, the selection space area | region, and a virtual camera coordinate system. It is a figure explaining the case where the selection space area | region is assumed as an imaging area of a virtual camera in the said electronic device. It is a figure explaining the said selection space area | region prescribed | regulated in the said three-dimensional virtual space. It is a figure explaining the case where the selected object is enlarged / reduced. It is a figure explaining the case where the color tone of the selected object is changed. It is a figure explaining the case where the shape of the selected object is changed. It is a figure explaining a mode that an electronic device is moved. It is a figure which shows an example of the three-dimensional virtual space displayed on the display part of the said electronic device. It is a figure which shows an example of the three-dimensional virtual space displayed on the display part of the said electronic device. It is a figure which shows an example of the three-dimensional virtual space displayed on the display part of the said electronic device. It is a figure showing that the object was selected in the three-dimensional virtual space displayed on the display part of the said electronic device.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 Electronic device, 2 Main body, 3 Imaging part, 4 Speaker, 5 Microphone, 6 Display part, 7 Operation part, 8 External memory mounting part, 9 Handle part, 10 Semiconductor memory, 26 Angular velocity sensor, 27 Acceleration sensor, 51 Virtual space Information processing section lock, 52 display processing block, 53 information selection processing block,

Claims (21)

In an electronic device that can display the moving direction of the device itself and the image of the object displayed on the device in conjunction with each other,
Storage means for storing 3D virtual space data and 3D virtual object data;
A virtual that builds a three-dimensional virtual space from the three-dimensional virtual space data and links the content related to the three-dimensional virtual object constructed from the three-dimensional virtual object data or information indicating the storage location of the content to the three-dimensional virtual object. Spatial information processing means;
Display means for displaying a part of the constructed three-dimensional virtual space and contents;
Plane movement detecting means for detecting movement of the device in the plane direction;
Vertical movement detecting means for detecting movement of the device in the vertical direction;
Display processing means for scrolling the three-dimensional virtual space displayed on the display means in accordance with the movement amount in the plane direction detected by the plane movement detection means and the movement amount in the vertical direction detected by the vertical movement detection means; ,
A predetermined effective space region that is fixed with respect to a coordinate system centered on the device in the three-dimensional virtual space displayed on the display unit and is narrower than the three-dimensional virtual space displayed on the display unit is selected as a selection space region. Information selection processing means for selecting a content linked to a 3D virtual object in contact with the selected space area or a 3D virtual object at least part of which is included in the effective space area or information indicating a storage location of the content An electronic device comprising:   2. The electronic apparatus according to claim 1, wherein the information selection processing unit preferentially selects a three-dimensional virtual object located closer to the central axis of the selected space area.   2. The electronic apparatus according to claim 1, wherein the information selection processing means preferentially selects a three-dimensional virtual object located closer to a coordinate system origin centered on the apparatus in the selected space area.   The electronic device according to claim 1, wherein the selected space region is represented by a cylinder.   The electronic device according to claim 1, wherein the selected space area is represented by a polygonal column.   The electronic device according to claim 1, wherein the selected space area is represented by a cube.   The electronic device according to claim 1, wherein the selected space area is represented by a sphere.   The electronic device according to claim 1, wherein the selected space area is represented by a polygonal pyramid. In the information display selection method for selecting the displayed information displayed in conjunction with the moving direction of the electronic device itself and the image of the object displayed on the electronic device,
Virtual space information that builds a three-dimensional virtual space from the three-dimensional virtual space data and links the content related to the three-dimensional virtual object constructed from the three-dimensional virtual object data or information indicating the storage location of the content to the three-dimensional virtual object Processing steps;
A plane movement detection step of detecting movement of the electronic device in the plane direction;
A vertical movement detecting step of detecting movement of the electronic device in the vertical direction;
Depending on the amount of movement in the plane direction detected in the plane movement detection step and the amount of movement in the vertical direction detected in the vertical movement detection step for a part of the three-dimensional virtual space constructed in the virtual space information processing step A display processing step for scroll display;
A predetermined effective space area narrower than the three-dimensional virtual space that is fixed with respect to the coordinate system centered on the electronic device in the three-dimensional virtual space and is displayed in the display processing step is selected as the selected space area. An information selection processing step of selecting content linked to a three-dimensional virtual object that has touched or a three-dimensional virtual object at least part of which is included in the effective space region or information indicating a storage location of the content. A method for selecting and displaying information.   10. The information display selection method according to claim 9, wherein the information selection processing step preferentially selects a three-dimensional virtual object located closer to the central axis of the selected space area.   10. The information display selection method according to claim 9, wherein the information selection processing step preferentially selects a three-dimensional virtual object located closer to the origin of the coordinate system centered on the device in the selected space area. .   10. The information display selection method according to claim 9, wherein the selection space area is represented by a cylinder.   10. The information display selection method according to claim 9, wherein the selection space area is represented by a polygonal column.   10. The information display selection method according to claim 9, wherein the selected space area is represented by a cube.   10. The information display selection method according to claim 9, wherein the selected space area is represented by a sphere.   10. The information display selection method according to claim 9, wherein the selection space area is represented by a polygonal pyramid.   10. The information display selection method according to claim 9, wherein the display processing step changes a state of the three-dimensional virtual object selected by the selected space area.   18. The information display selection method according to claim 17, wherein the display processing step enlarges / reduces the three-dimensional virtual object selected by the selected space region periodically or irregularly.   18. The information display selection method according to claim 17, wherein the display processing step changes the color tone of the three-dimensional virtual object selected by the selected space region periodically or irregularly.   18. The information display selection method according to claim 17, wherein the display processing step changes the shape of the three-dimensional virtual object selected by the selected space region periodically or irregularly.   18. The information display selection method according to claim 17, wherein in the display processing step, the state of the three-dimensional virtual object selected by the selected space region is changed periodically or irregularly and a sound is generated.

Images