JP4799269B2 - Image processing apparatus and program - Google Patents

Description

The present invention relates to an image processing instrumentation 置及 beauty programs, in particular, to three-dimensional computer graphics.

  An object is arranged in a virtual three-dimensional space, and a spatial image showing a state in which a line-of-sight direction toward the object is viewed from a viewpoint arranged in the virtual three-dimensional space is generated and displayed on a monitor, and an operation device A technique for changing the position and orientation of an object in a virtual three-dimensional space in accordance with an operation signal input from is known. As operation devices, there are devices that detect whether or not a button is pressed, and devices that detect the posture of an operation member such as a stick, and conventionally, the position and posture of an object are detected using operation signals indicating these detection results. It was changed.

  Furthermore, it has been studied to attach a gyroscope or an acceleration sensor to the operation device, detect the attitude of the operation device, and change the position and attitude of the object in the virtual three-dimensional space according to the detection result. In this way, the user can feel as if he / she is directly manipulating the virtual object displayed on the monitor via the operation device, which is a real object, and intuitively understand it. A user interface that is easy to do can be realized.

  However, when the posture of the operation device is used, if the line-of-sight direction used when generating the spatial image changes, it becomes difficult to operate the virtual object represented in the spatial image without a sense of incongruity. For example, the object shown in the space image is facing diagonally forward, but what kind of posture the operating device is facing to the monitor will move the object forward in the virtual three-dimensional space It is difficult for the user to intuitively understand whether or not the

The present invention was made in view of the above problems, an image processing instrumentation 置及 beauty program can intuitively manipulate objects arranged in a virtual three-dimensional space by the posture of the operation device Is to provide.

  In order to solve the above-described problem, the image processing apparatus according to the present invention looks at the viewing direction toward the object from the viewpoint arranged in the virtual three-dimensional space in the virtual three-dimensional space in which the object is arranged. A spatial image display means for displaying a spatial image showing, and a gaze direction changing means for changing the gaze direction by moving a relative position of at least one of the viewpoint or the object in the virtual three-dimensional space; A direction for calculating direction difference information indicating a difference between the posture direction acquired by the object posture direction acquiring unit and an object posture direction acquiring unit that acquires a posture direction of the object in the virtual three-dimensional space; The difference information calculation means and posture information indicating the posture of the operation device for operating the object are obtained. Operating device attitude information acquisition means for performing object movement means for moving the object in the virtual three-dimensional space in a direction according to the direction difference information and the attitude information acquired by the operation device attitude information acquisition means , Including.

  In the aspect of the invention, the posture information indicates a tilt direction of the operation device.

  Further, the image processing method according to the present invention displays a spatial image showing a state in which a gaze direction toward the object is viewed from a viewpoint arranged in the virtual three-dimensional space in the virtual three-dimensional space in which the object is arranged. A spatial image display step, a visual line direction changing step of changing the visual line direction by moving a relative position of the viewpoint or at least one of the objects in the virtual three-dimensional space, and the virtual 3 of the object An object posture direction acquisition step for acquiring a posture direction in a three-dimensional space; a direction difference information calculation step for calculating direction difference information indicating a difference between the posture direction and the line-of-sight direction acquired by the object posture direction acquisition step; Attitude information indicating the attitude of the operating device for operating the object is acquired. An operation device posture information acquisition step to perform, and an object movement step to move the object in the virtual three-dimensional space in a direction according to the direction difference information and the posture information acquired by the operation device posture information acquisition step. , Including.

  Further, the program according to the present invention is a space for displaying a spatial image showing a state in which a gaze direction toward the object is viewed from the viewpoint arranged in the virtual three-dimensional space in the virtual three-dimensional space in which the object is arranged. Image display means, visual axis direction changing means for changing the visual line direction by moving a relative position of the viewpoint or at least one of the objects in the virtual three-dimensional space, and the posture of the object in the virtual three-dimensional space An object posture direction acquiring unit for acquiring a direction, a direction difference information calculating unit for calculating direction difference information indicating a difference between the posture direction and the line-of-sight direction acquired by the object posture direction acquiring unit, and for operating the object Acquire operation device attitude information to acquire attitude information indicating the attitude of the operating device For causing the computer to function as an object moving unit that moves the object in the virtual three-dimensional space in a direction according to the stage and the direction difference information and the posture information acquired by the operation device posture information acquisition unit. It is a program. This program may be stored in a computer-readable information storage medium such as a CD-ROM or DVD-ROM.

  Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a diagram illustrating a hardware configuration of an entertainment system (image processing apparatus) according to the present embodiment. As shown in the figure, the entertainment system 10 includes an MPU (Micro Processing Unit) 11, a main memory 20, an image processing unit 24, a monitor 26, an input / output processing unit 28, an audio processing unit 30, and a speaker. 32, an optical disk reading unit 34, an optical disk 36, a hard disk 38, interfaces (I / F) 40 and 44, an operation device 42, a camera unit 46, and a network interface 48. System.

  FIG. 2 is a diagram showing the configuration of the MPU 11 (program execution means). As shown in the figure, the MPU 11 includes a main processor 12, sub processors 14a, 14b, 14c, 14d, 14e, 14f, 14g, and 14h, a bus 16, a memory controller 18, and an interface (I / F) 22. And comprising.

  The main processor 12 is an operating system stored in a ROM (Read Only Memory) (not shown), for example, a program and data read from an optical disc 36 such as a DVD (Digital Versatile Disk) -ROM, and a program supplied via a communication network. Based on the data and the like, various information processing is performed, and the sub-processors 14a to 14h are controlled.

  The sub-processors 14a to 14h perform various types of information processing in accordance with instructions from the main processor 12, and each part of the entertainment system 10 is read from the optical disk 36 such as a DVD-ROM or the like via a communication network. Control based on supplied programs and data.

  The bus 16 is used for exchanging addresses and data among the various parts of the entertainment system 10. The main processor 12, the sub processors 14a to 14h, the memory controller 18, and the interface 22 are connected to each other via the bus 16 so as to be able to exchange data.

  The memory controller 18 accesses the main memory 20 in accordance with instructions from the main processor 12 and the sub processors 14a to 14h. Programs and data read from the optical disk 36 and the hard disk 38 and programs and data supplied via the communication network are written into the main memory 20 as necessary. The main memory 20 is also used for work of the main processor 12 and the sub processors 14a to 14h.

  An image processing unit 24 and an input / output processing unit 28 are connected to the interface 22. Data exchange between the main processor 12 and the sub processors 14 a to 14 h and the image processing unit 24 or the input / output processing unit 28 is performed via the interface 22.

  The image processing unit 24 includes a GPU (Graphical Processing Unit) and a frame buffer. The GPU renders various screens in the frame buffer based on the image data supplied from the main processor 12 and the sub processors 14a to 14h. A screen formed in the frame buffer, that is, a screen showing the execution result of the MPU 11 is converted into a video signal at a predetermined timing and output to the monitor 26. As the monitor 26, for example, a home television receiver is used.

  An audio processing unit 30, an optical disk reading unit 34, a hard disk 38, and interfaces 40 and 44 are connected to the input / output processing unit 28. The input / output processing unit 28 controls data exchange between the main processor 12 and the sub-processors 14a to 14h, the audio processing unit 30, the optical disc reading unit 34, the hard disk 38, the interfaces 40 and 44, and the network interface 48.

  The sound processing unit 30 includes an SPU (Sound Processing Unit) and a sound buffer. The sound buffer stores various audio data such as game music, game sound effects and messages read from the optical disk 36 and the hard disk 38. The SPU reproduces these various audio data and outputs them from the speaker 32. As the speaker 32, for example, a built-in speaker of a home television receiver is used.

  The optical disk reading unit 34 reads programs and data stored on the optical disk 36 in accordance with instructions from the main processor 12 and the sub processors 14a to 14h. Note that the entertainment system 10 may be configured to be able to read a program or data stored in a computer-readable information storage medium other than the optical disk 36.

  The optical disk 36 is a general optical disk (computer-readable information storage medium) such as a DVD-ROM. The hard disk 38 is a general hard disk device. Various programs and data are stored in the optical disk 36 and the hard disk 38 so as to be readable by a computer.

  The interfaces (I / F) 40 and 44 are interfaces for connecting various peripheral devices such as the operation device 42 and the camera unit 46. For example, a USB (Universal Serial Bus) interface is used as such an interface.

  The operation device 42 is general-purpose operation input means, and is used by the user to input various operations (for example, game operations). The input / output processing unit 28 acquires the state of each unit from the operation device 42 every predetermined time (for example, 1/60 seconds) by wireless or wired communication, and sends an operation signal representing the result to the main processor 12 or the sub processor 14a. To 14h. The main processor 12 and the sub processors 14a to 14h determine the content of the operation performed by the user based on the operation signal. The entertainment system 10 is configured to be capable of communication connection of a plurality of operation devices 42, and the main processor 12 and the sub processors 14a to 14h execute various processes based on operation signals input from the operation devices 42. It is like that.

  The camera unit 46 includes, for example, a known digital camera, and inputs black and white, grayscale, or color photographed images every predetermined time (for example, 1/60 seconds). The camera unit 46 in the present embodiment is configured to input a captured image as image data in JPEG (Joint Photographic Experts Group) format. The camera unit 46 is installed on the monitor 26 with the lens facing the player, for example, and is connected to the interface 44 via a cable. The network interface 48 is connected to the input / output processing unit 28 and a communication network such as the Internet, and relays data communication between the entertainment system 10 and other computer systems such as the entertainment system 10 via the communication network. It has become.

  The operation device 42 is a portable small computer, and is formed in a rectangular thin box, for example, as shown in FIG. Further, a direction key 50 is provided on the left side of the main body surface, and a button group 52 including buttons 52A to 52D is provided on the right side. Auxiliary buttons 54L and 54R are provided on one side of the operation device 42. The operation device 42 has a gyroscope 58 built in the center in the longitudinal direction, and an acceleration sensor 56 built in the side thereof. The acceleration sensor 56 is a three-axis acceleration sensor that detects accelerations in three directions orthogonal to each other. As shown in FIG. 4, gravity or the operation device 42 in the X direction that is the left-right direction (longitudinal direction) of the operation device 42. Acceleration in the Y direction, which is the depth direction (short direction) of the operation device 42, acceleration due to gravity or movement of the operation device 42, and Z direction, which is the thickness direction of the operation device, It detects gravity and acceleration caused by movement of the operation device 42. The operation device 42 and the entertainment system 10 can detect the attitude of the operation device 42 according to the acceleration in these three directions. That is, it is possible to detect how much the operation device 42 has rotated with respect to the Y axis (reference characters XU and XD) and how much the operation device 42 has rotated with respect to the X axis (reference symbols YU and YD). Moreover, the acceleration at the time of the operation device 42 moving to a perpendicular direction is also detectable (code | symbol ZU and ZD). Accordingly, the operation device 42 and the entertainment system 10 can detect that the operation device 42 is lifted in the vertical direction or the operation device 42 is placed in the vertical direction. Specifically, as shown in FIG. 5, the acceleration sensor 56 outputs a voltage proportional to the acceleration in each of the above three directions. The entertainment system 10 can grasp whether it is tilted only.

  Further, as described above, the operation device 42 has the built-in gyroscope 58, whereby the angular velocity ω of rotation around the Z direction that is the thickness direction of the operation device 42 can be detected.

  The entertainment system 10 and the operation device 42 include wired communication means such as USB, and wireless communication means such as BlueTooth (trademark) and wireless LAN. The operation device 42 includes data detected by the acceleration sensor 56 and the gyro 58. The input states of the button group 32, the auxiliary button 34, and the direction key 30, and the processing results thereof can be transmitted to the entertainment system 10 by wireless or wired communication. When the entertainment system 10 receives data transmitted from the operation device 42, the entertainment system 10 executes various types of information processing according to the data. An execution result of the information processing is displayed on a monitor 26 such as a television receiver, or is output as sound through a speaker 32 such as a built-in speaker.

  Hereinafter, a technique for realizing a new user interface using the operation device 42 in the entertainment system 10 having the above hardware configuration will be described.

  In the present embodiment, a virtual three-dimensional space is constructed on the main memory 20 of the entertainment system 10. FIG. 6 shows an example of this virtual three-dimensional space. As shown in the figure, a world coordinate system 61 is set in the virtual three-dimensional space 60, and an object 64 that is a user's operation target is arranged at one point. A viewpoint 66 is also arranged in the virtual three-dimensional space 60, and a line-of-sight direction 68 is set in a direction from the viewpoint 66 toward a representative point in the object 64 (for example, an origin of an object coordinate system 65 described later). Then, a spatial image showing a state in which the viewing direction 68 is viewed from the viewpoint 66 is generated and displayed on the monitor 26. At this time, since the line-of-sight direction 68 is set so as to be directed from the viewpoint 66 to the object 64, the object 64 appears in the spatial image.

  The viewpoint 66 moves on the viewpoint trajectory 62 set above the object 64 (for example, the Z direction in the world coordinate system 61) according to an operation by the operation device 42. That is, when the auxiliary button 54L of the operation device 42 is pressed, the viewpoint 66 moves on the viewpoint trajectory 62 counterclockwise. In addition, when the auxiliary button 54R is pressed, the viewpoint 66 moves clockwise on the viewpoint trajectory 62.

  An object coordinate system 65 is set for the object 64. The Ya axis of the object coordinate system 65 matches the front-rear direction of the object 64, the Xa axis matches the left-right direction, and the Za axis matches the vertical direction. In the present embodiment, the position and orientation of the object 64 in the virtual three-dimensional space 60 can be changed by tilting the operation device 42. At this time, if the operation device 42 is tilted in any direction by the angle θ formed by the line-of-sight direction 68 and the posture direction (Ya axis direction) of the object 64 defined by the object coordinate system 65, the object 64 moves in which direction. Or change posture.

  That is, as shown in FIG. 7A, when a spatial image showing the virtual three-dimensional space 60 viewed from the viewpoint 66 located behind the object 64 is displayed on the monitor 26, the front part of the object 64 is On the upper side of the screen, the rear part is displayed on the lower side of the screen, the right part is displayed on the right side of the screen, and the left part is displayed on the left side of the screen. In this case, as shown in FIG. 5B, the object 64 is tilted to the right (R) in the virtual three-dimensional space 60 by tilting the operation device 42 to the right. Further, when the operation device 42 is tilted to the left, the object 64 tilts to the left (L) in the virtual three-dimensional space 60. Further, by tilting the operation device 42 to the back side, the object 64 moves forward (F) in the virtual three-dimensional space 60, and by tilting the operation device 42 to the front, the object 64 moves backward in the virtual three-dimensional space 60. Move to (B).

  Further, as shown in FIG. 8A, when a spatial image showing the virtual three-dimensional space 60 viewed from the viewpoint 66 located diagonally to the left of the object 64 is displayed on the monitor 26, FIG. ), The object 64 is tilted to the right (R) in the virtual three-dimensional space 60 by tilting the operation device 42 to the left back. Further, when the operation device 42 is tilted to the right front, the object 64 tilts to the left (L) in the virtual three-dimensional space 60. Further, by tilting the operation device 42 to the left front, the object 64 moves forward (F) in the virtual three-dimensional space 60, and by tilting the operation device 42 to the right back, the object 64 is moved in the virtual three-dimensional space 60. Move backward (B).

  The above user interface is specifically realized by the following processing. The following processing is realized by executing a user interface program in the entertainment system 10. This program may be downloaded to the entertainment system 10 via a communication network such as the Internet, or may be stored in a computer-readable information storage medium such as a CD-ROM or DVD-ROM, and read from there by the entertainment system 10. Good.

  In the processing according to the present embodiment, first, an angle θ formed by the line-of-sight direction 68 and the posture direction (Ya axis direction) of the object 64 is calculated. Alternatively, the vector of the line-of-sight direction 68 may be projected onto the XY plane in the object coordinate system 65, and the angle θ formed by the direction indicated by the projected vector and the posture direction of the object 64 may be used.

  Next, the output value Xc of the acceleration sensor 56 indicating the acceleration in the X direction and the output value Yc of the acceleration sensor 56 indicating the acceleration in the Y direction are acquired. Xc is generated based on the raw output value (the above voltage value) of the acceleration sensor 56, and is positive when the acceleration in the X direction is positive (that is, when the operation device 42 is tilted to the right). It takes a value and takes a zero value when it is zero (that is, when the operating device 42 is not tilted in the left-right direction), and takes a negative value when it is negative (that is, when the operating device 42 tilts to the left). Similarly, Yc is also generated based on the raw output value (the above voltage value) of the acceleration sensor 56, and when the acceleration in the Y direction is positive (that is, when the operation device 42 is tilted to the back). Takes a positive value, takes a zero value when it is zero (that is, when the operating device 42 is not tilted in the front-rear direction), and a negative value when it is negative (that is, when the operating device 42 tilts forward). Take.

  Then, the output value pair (Xc, Yc) (attitude information of the operation device 42) is subjected to a process of rotating in the XY plane by the angle θ formed as described above, and the result is corrected. It is assumed that the output value pair (Xc ′, Yc ′). Then, the position and orientation of the object 64 are changed in the virtual three-dimensional space 60 according to the corrected output value pair. That is, if Yc ′ is positive, the position of the object 64 is updated so that the object 64 is moved forward (F) in the virtual three-dimensional space 60. If Yc ′ is negative, the position of the object 64 is updated so that the object 64 is moved backward (B) in the virtual three-dimensional space 60. If Xc ′ is positive, the posture is updated so that the object 64 is tilted to the right (R) in the virtual three-dimensional space 60. If Xc ′ is negative, the posture is updated so that the object 64 is tilted to the left (L) in the virtual three-dimensional space 60. Note that one direction is identified based on each component of the corrected output value pair (Xc ′, Yc ′), that is, both Xc ′ and Yc ′, and the object 64 is caused to perform an action in the direction. It may be. For example, one direction may be calculated based on the corrected output value pair (Xc ′, Yc ′), and the object 64 may be turned so that the posture direction of the object 64 is directed in this direction.

  FIG. 11 is a flowchart showing processing for updating the position and orientation of an object. The processing shown in the figure is executed at predetermined time intervals according to the user interface program. First, the MPU 11 is instructed to press the auxiliary button 54L or 54R of the operation device 42 to move the viewpoint 66, and accordingly It is then determined whether the line-of-sight direction 68 changes (S101). When the line-of-sight direction 68 changes, the angle θ formed by the line-of-sight direction 68 and the posture direction of the object 64 is calculated and stored in the main memory 20 (S102). When the line-of-sight direction 68 does not change, the process of S102 is skipped.

  Next, a pair (Xc, Yc) of output values of the acceleration sensor 58 is acquired (S103). On the other hand, a rotation process using the angle θ formed in S102 and already stored in the main memory 20 is performed. The corrected output value pair (Xc ′, Yc ′) is calculated (S104). Based on the corrected output value pair (Xc ′, Yc ′), the position and orientation of the object 64 are changed in the virtual three-dimensional space 60.

  According to the present embodiment described above, the angle between the posture direction (Ya axis direction) of the object 64 in the virtual three-dimensional space 60 and the line-of-sight direction 68, that is, the direction difference information indicating the difference between both directions is calculated. Since the object 64 is moved in the virtual three-dimensional space 60 in the direction corresponding to the difference information and the output value of the acceleration sensor 58 of the operation device 42, that is, the posture information of the operation device 42, the object 64 displayed on the monitor 26 is displayed. Depending on the posture, the direction in which the object 64 moves when the operation device 42 is tilted can be controlled, and an extremely intuitive user interface can be realized.

  The present invention is not limited to the above embodiment. For example, in the above description, the object 64 is tilted left and right according to the attitude of the operation device 42, but may be moved left and right. Similarly, when the position and orientation of the object 64 are changed in accordance with the output of the gyroscope 58, the orientation of the object 64 depends on the angle θ formed by the line-of-sight direction 68 and the orientation direction of the object 64. When the posture of the operation device 42 is changed, the direction in which the object 64 moves may be controlled.

It is a figure which shows the hardware constitutions of the entertainment system which concerns on one Embodiment of this invention. It is a figure which shows the structure of MPU. It is a perspective view which shows the external appearance of an operation device. It is a figure explaining the detection content of the acceleration sensor provided in an operation device. It is a figure explaining the output of an acceleration sensor. It is a figure which shows an example of virtual three-dimensional space. It is a figure which shows the operation method of the space image displayed on a monitor, and an operation device. It is a figure which shows the operation method of the space image displayed on a monitor, and an operation device. It is a figure which shows angle (theta) which the attitude | position direction of an object and the gaze direction make. It is a figure explaining the process which converts the output value of an acceleration sensor. It is a flowchart which shows the update process of the position and attitude | position of an object.

Explanation of symbols

  10 Entertainment System, 11 MPU, 12 Main Processor, 14a-14h Sub-Processor, 16 Bus, 18 Memory Controller, 20 Main Memory, 22, 40, 44 Interface, 24 Image Processing Unit, 26 Monitor, 28 Input / Output Processing Unit, 30 Audio processing unit, 32 speakers, 34 optical disk reading unit, 36 optical disk, 38 hard disk, 42 operation device, 46 camera unit, 50 direction keys, 52 buttons, 54 auxiliary buttons, 56 acceleration sensor, 58 gyroscope, 60 virtual three-dimensional space 61 World coordinate system, 62 viewpoint trajectory, 64 objects, 65 object coordinate system.

Claims (3)

In a virtual three-dimensional space in which an object is arranged, a spatial image display means for displaying a spatial image showing a state of viewing a line-of-sight direction toward the object from a viewpoint arranged in the virtual three-dimensional space;
Gaze direction changing means for changing the gaze direction by moving a relative position of at least one of the viewpoint or the object in the virtual three-dimensional space;
Object posture direction acquisition means for acquiring a posture direction of the object in the virtual three-dimensional space;
Every predetermined time, and direction difference information calculating means for calculating the direction difference information indicating the difference between the current the posture direction of current the sight line direction acquired by the object orientation direction obtaining means,
Operation device posture information acquisition means for acquiring posture information indicating the posture of the operation device for operating the object;
Object moving means for moving the object in the virtual three-dimensional space in a direction according to the direction difference information and the current posture information acquired by the operating device posture information acquiring means at predetermined time intervals;
An image processing apparatus comprising: The image processing apparatus according to claim 1.
The posture information indicates a tilt direction of the operation device.
An image processing apparatus. In a virtual three-dimensional space in which an object is arranged, a spatial image display means for displaying a spatial image showing a state of viewing a line-of-sight direction toward the object from a viewpoint arranged in the virtual three-dimensional space;
Gaze direction changing means for changing the gaze direction by moving a relative position of at least one of the viewpoint or the object in the virtual three-dimensional space;
Object posture direction acquisition means for acquiring a posture direction of the object in the virtual three-dimensional space;
Every predetermined time, the direction difference information calculating means for calculating the direction difference information indicating the difference between the current of the orientation direction of the current the sight line direction acquired by the object orientation direction obtaining means,
Operation device attitude information acquisition means for acquiring attitude information indicating the attitude of the operation device for operating the object; and
A computer as an object moving unit that moves the object in the virtual three-dimensional space in a direction according to the direction difference information and the current posture information acquired by the operation device posture information acquisition unit at predetermined time intervals. A program to make it work.

Images