JP2007310424A - 3d coordinate input system, apparatus, method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To realize intuitive 3D coordinate input without adding a new special sensor. <P>SOLUTION: A 3D movement calculation means, from a moving picture obtained by photographing a photographic subject with a camera, calculates 3D movement of the camera and a terminal mounted therewith generated as a user has moved the terminal. Subsequently, an operation information generation means converts the 3D movement computed with the 3D movement calculation means into operation information for operation object software, and if an operation information validation switch is turned on, inputs the operation information to the operation object software. Then a display means presents a result obtained by inputting the operation information to the operation object software to the user. Accordingly, the user can intuitively input 3D coordinates by operating the terminal. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a coordinate input system and the like, and more particularly, to a three-dimensional coordinate input system, a three-dimensional coordinate input device, a three-dimensional coordinate input method, and a three-dimensional coordinate input program using a camera capable of moving image shooting.

  Conventionally, as disclosed in Patent Documents 1 to 3, a 3D coordinate input is performed by installing a mechanism for inputting a z coordinate in a mouse for inputting a normal xy coordinate. There was an original coordinate input device.

  Also, as disclosed in Patent Documents 4 and 5, a trackball is mounted on a normal mouse, the inclination of the plane in the three-dimensional space is specified by the trackball, and the two-dimensional coordinates on the same plane are the mouse. There is also a three-dimensional coordinate input device in which three-dimensional coordinate input is performed by designating by.

  On the other hand, as disclosed in Patent Documents 6 and 7, an inertial sensor corresponding to each dimension of xyz is arranged on each terminal, and the movement of the terminal is read from the detection result of each inertial sensor, so that three-dimensional Some of them also input coordinates.

JP 2001-360252 A JP-A-11-134111 Japanese Patent Laid-Open No. 8-17983 JP-A-9-26851 Japanese Patent Laid-Open No. 1-100621 Japanese Patent Laid-Open No. 7-98630 JP-A-1-96720

  However, the conventional techniques described above have the following problems.

  The first problem is that the apparatus itself is large in the apparatus in which the z coordinate input mechanism is mounted on the mouse.

  A second problem is that it is difficult to intuitively input coordinates in an apparatus in which a z coordinate input mechanism is mounted on a mouse. The reason is that the user needs to operate the z coordinate input mechanism and the mouse at the same time.

  A third problem is that in a device in which the three-dimensional coordinates of the terminal are acquired based on the sensing result of the inertial sensor, the terminal is large and expensive.

  Furthermore, the fourth problem is that in a device in which the three-dimensional coordinates of the terminal are acquired based on the detection result of the inertial sensor, the user is on a moving body such as a car or a train even when the user stops the terminal. In this case, an operation is input to the software against the user's intention. The reason is that the inertial sensor detects the absolute movement of the terminal, not the relative movement between the user and the terminal.

  An object of the present invention is to solve the above-described problems and to realize intuitive three-dimensional coordinate input without newly adding a special sensor.

  The three-dimensional coordinate input system of the present invention includes a moving image photographing unit (for example, camera 110) for photographing a moving image, a display unit (for example, display unit 130) for displaying an image, and predetermined software (for example, operation target software 103). Display control means (for example, the computer 100) for controlling the display means based on the above, and a casing (for example, the terminal device 300) in which the moving image shooting means is mounted based on the moving image shot by the moving image shooting means. Three-dimensional motion calculation means (for example, three-dimensional motion calculation means 101) that calculates a motion in the three-dimensional space as a change in three-dimensional coordinates, and operation information that generates operation information based on the motion calculated by the three-dimensional motion calculation means Generating means (for example, the operation information generating means 102), and the display control means is the operation information generated by the operation information generating means. Therefore, changing the image displayed on the display unit (step A4) can be characterized.

  By configuring as described above, it is possible to perform 3D coordinate input based on the captured image of the moving image capturing means without adding a new special sensor. Will be able to enter.

  An operation information enabling switch (for example, an operation information enabling switch 120) for enabling or disabling the operation information generated by the operation information generating means in accordance with a user operation may be provided.

  The display unit and the display control unit may be configured to be controlled by a computer different from the computer that controls the apparatus including the moving image shooting unit including the casing on which the moving image shooting unit is mounted. Good.

  The display unit may be installed in the casing, and the moving image photographing unit may be installed on the back side opposite to the front side of the casing where the display unit is installed (see, for example, FIG. 2).

  The display unit may be installed in the casing, and the moving image capturing unit may be installed on the front side of the casing where the display unit is installed (see, for example, FIG. 4).

  The moving image photographing means includes a first moving image photographing means and a second moving image photographing means, the display means is installed in the housing, and the first moving image photographing means is the display means in the housing. It is good also as a structure installed in the installed front side and the 2nd moving image imaging | photography means being installed in the back side opposite to the front side in a housing | casing (for example, refer FIG. 5).

  The display means includes a backlight for brightly displaying the display screen, and the moving image photographing means installed on the front side of the casing together with the display means is used as a light source for illuminating the subject when photographing the moving image. A light may be used.

  The moving image shooting means may include a light source for illuminating the subject when shooting a moving image.

  The moving image photographing means is a subject on which a texture (for example, a lattice, a character, etc.) prepared in advance is printed in order to obtain a moving image for causing the three-dimensional movement calculating means to calculate the movement in the three-dimensional space of the housing. You may comprise so that it may image | photograph.

  The three-dimensional coordinate input device of the present invention is equipped with a moving image photographing means (for example, the camera 110) for photographing a moving image and the moving image photographing means based on the moving image photographed by the moving image photographing means. 3D motion calculation means (for example, 3D motion calculation means 101) for calculating the motion of the housing in the 3D space as a change in 3D coordinates, and display means based on the motion calculated by the 3D motion calculation means An operation information generation unit (for example, the operation information generation unit 102) that generates operation information for changing an image displayed on the display unit (for example, the display unit 130), and a display unit that displays the operation information generated by the operation information generation unit are predetermined. Operation information output means (for example, operation information generation means 102) for output to the display control means for controlling based on the software.

  By configuring as described above, it is possible to perform 3D coordinate input based on the captured image of the moving image capturing means without adding a new special sensor. Will be able to enter.

  The 3D coordinate input method of the present invention includes a step of capturing a moving image, and a step of calculating an operation in the 3D space of the housing as a change of the 3D coordinate based on the captured moving image (for example, Step A1), a step of generating operation information based on the calculated motion in the three-dimensional space (eg, step A2), and a display means for displaying an image is controlled based on predetermined software, and the generated operation information And changing the image displayed on the display means (for example, step A4).

  By configuring as described above, it is possible to perform 3D coordinate input based on the captured moving image without adding a new special sensor. Therefore, intuitive 3D coordinates can be input at low cost. Will be able to.

  Furthermore, the three-dimensional coordinate input program of the present invention captures a moving image on a computer (for example, the computer 100), and the operation in the three-dimensional space of the casing based on the captured moving image. A step (eg, step A1) that is calculated as a change in the step, a step (eg, step A2) that generates operation information based on the calculated motion in the three-dimensional space, and a display means that displays an image based on predetermined software. And a step of changing the image displayed on the display means (for example, step A4) in accordance with the operation information generated by the control.

  By configuring as described above, it is possible to cause a computer to execute 3D coordinate input based on a captured moving image without adding a new special sensor. You will be able to enter.

  According to the present invention, since it is possible to perform 3D coordinate input based on the captured image of the moving image capturing means without adding a special sensor, it is possible to perform 3D coordinate input at a compact and low cost. There is an effect that can be.

  In addition, according to the present invention, when inputting three-dimensional coordinates, it is not necessary to operate a plurality of mechanisms at the same time, and it is only necessary to move the casing in the three-dimensional space. There is an effect that it can be performed.

  Furthermore, according to the present invention, when the user who is operating the casing or the moving body on which the user is riding and the casing can be detected, the user is on the moving body. However, it is possible to prevent an operation instruction based on software from being input against the user's intention.

Embodiments of the present invention will be described below with reference to the drawings.
Embodiment 1 FIG.
FIG. 1 is a block diagram showing a configuration example of a three-dimensional coordinate input system according to the first embodiment of the present invention.

  As shown in FIG. 1, the three-dimensional coordinate input system of this example includes a computer (central processing unit; processor; data processing unit) 100 that operates under program control, a camera 110, an operation information enabling switch 120, and a display. Means 130. FIG. 1 also shows a subject 140 photographed by the camera 110.

  The computer 100 includes a three-dimensional motion calculation unit 101, an operation information generation unit 102, and operation target software 103.

  As the camera 110, for example, a digital camera having a function of shooting a moving image is used. Note that the camera 110 may be provided with a light source such as a flash device or a light for clearly photographing the subject 140.

  As the display means 130, for example, a display device that displays various information such as an LCD (Liquid Crystal Display) is used. The display unit 130 presents the result obtained by inputting the operation information to the operation target software 103 to the user.

  FIG. 2 is an explanatory diagram showing an example of a terminal device 300 to which the three-dimensional coordinate input system of this example is applied. That is, the terminal device 300 includes a computer 100, and the three-dimensional motion calculation unit 101, the operation information generation unit 102, and the operation target software 103 operate on the built-in computer 100.

  In this example, the camera 110 is installed in the terminal device 300 so as to photograph the subject 140 located on the opposite side of the user 302 with the terminal device 300 interposed therebetween. The display unit 130 is attached on the terminal device 300 toward the user 302.

  That is, the terminal device 300 has a camera on the side opposite to the side where the display unit 130 is installed, in other words, on the side opposite to the side where the head of the user 302 viewing the display screen of the display unit 130 is located. 110 is installed.

  Here, the subject 140 may be a dedicated subject used when the apparatus 300 is used, such as a landscape around the terminal device 300 or an object on which a texture such as a grid or a character is printed.

  The three-dimensional motion calculation means 101 calculates the three-dimensional motion of the camera 110 and the terminal device 300 equipped with the camera 110 that are generated when the user 302 moves the terminal device 300 from the moving image obtained by photographing the subject 140 with the camera 110. Execute the process.

  The operation information generation unit 102 executes a process of converting the three-dimensional motion calculated by the three-dimensional motion calculation unit 101 into operation information of the operation target software 103.

  The operation information enabling switch 120 outputs a control signal for enabling or disabling the operation information generated by the operation information generating unit 102 to the computer 100 according to the operation of the user 302.

Next, the operation of the three-dimensional coordinate input system of this example will be described.
FIG. 3 is a flowchart showing an example of a three-dimensional coordinate input process by the three-dimensional coordinate input system of this example.

  In the three-dimensional coordinate input processing, first, the three-dimensional motion calculation unit 101 uses the third order of the camera 110 and the terminal 300 on which the camera 110 is mounted by moving the terminal 300 from the moving image obtained by shooting the subject 140 with the camera 110. The original motion is calculated (step A1).

  Next, the operation information generation unit 102 converts the three-dimensional motion calculated by the three-dimensional motion calculation unit 101 into operation information of the operation target software 103 (Step A2).

  Next, the operation information enabling switch 120 is referred to. When the switch 120 is ON, the operation information is input to the operation target software 103. When the switch 120 is OFF, no operation information is input (step A3).

  Then, the display unit 130 presents the result obtained by inputting the operation information to the operation target software 103 to the user (step A4).

  As described above, in the first embodiment described above, since the three-dimensional coordinates are calculated from the moving image captured by the camera 110 mounted on the terminal device 300, a new mechanism is used as the terminal device. It is not necessary to add to 300, and a compact and inexpensive three-dimensional coordinate input device can be realized.

  In the first embodiment described above, since it is not necessary to operate a plurality of mechanisms at the same time, and the 3D motion generated by the user moving the terminal device 300 in the space is used as the 3D coordinate input, Coordinate input is possible.

  Further, in the first embodiment described above, the relative movement between the user himself / herself or a moving body (such as an automobile) on which the user is riding and the terminal device 300 can be detected. Even when riding, it is possible to prevent unintentional operation information from being generated due to the movement of the moving body, and to prevent unintended operation information from being input to the software.

Embodiment 2. FIG.
Next, a second embodiment of the present invention will be described with reference to the drawings.
Since the configuration and operation of the three-dimensional coordinate input system in the second embodiment are the same as those of the three-dimensional coordinate input system (see FIG. 1) in the first embodiment described above, detailed description thereof is omitted. .

  FIG. 4 is an explanatory diagram showing an example of a terminal device 400 to which the three-dimensional coordinate input system of this example is applied. As illustrated in FIG. 4, the terminal device 400 is different from the terminal device 300 described above in that the camera 111 is installed on the terminal 400 toward the user 302 in order to use the face of the user 302 as a subject.

  That is, the three-dimensional coordinate input system of this example is different from the three-dimensional coordinate input system (see FIG. 1) in the first embodiment described above in that a camera 111 is provided instead of the camera 110.

  The camera 111 provided in the three-dimensional coordinate input system of this example is configured in the same manner as the camera 110 in the first embodiment described above.

  Also in this example, the camera 111 may be configured to include a light source such as a flash device or a light for clearly capturing the user 302 as a subject. In this example, the backlight of the display unit 130 attached to the user 302 may be used as the light source.

  As described above, in the above-described second embodiment, since the camera 111 is installed so as to capture the face of the user 302, the subject 302 can always be captured within a certain distance from the camera 111. Even when an appropriate subject does not exist around the user, the three-dimensional motion can be calculated stably.

  In the second embodiment described above, since the display unit 130 is installed toward the user 302, when the display unit 130 has a backlight, the face of the user 302 that is the subject is stable. It is possible to illuminate and to calculate 3D motion stably even in the dark.

Embodiment 3 FIG.
Next, a third embodiment of the present invention will be described with reference to the drawings.
The configuration and operation of the three-dimensional coordinate input system in the third embodiment are the same as those of the three-dimensional coordinate input system (see FIG. 1) in the first embodiment described above, and thus detailed description thereof is omitted. .

  FIG. 5 is an explanatory diagram showing an example of a terminal device 500 to which the three-dimensional coordinate input system of this example is applied.

  As illustrated in FIG. 5, the terminal device 500 includes a camera 110 installed in the terminal device 500 so as to photograph a subject 140 located on the opposite side of the user 302 across the terminal device 500, and the face of the user 302. Is different from the above-described terminal devices 300 and 400 in that a camera 111 installed on the terminal 400 is provided toward the user 302 so as to be a subject.

  That is, the terminal device 500 has both the functions of the terminal device 300 and the terminal device 400 described above.

  As described above, in the above-described third embodiment, the camera 111 that captures the face of the user 302 and the camera 110 that captures the subject 140 that is located on the opposite side of the user 302 as viewed from the terminal device 500. Therefore, even when only one of the user 302 and the subject 140 can be photographed, the three-dimensional motion can be calculated stably.

  In addition, when both the user 302 and the subject 140 can be photographed, the accuracy of the three-dimensional motion can be improved.

Embodiment 4 FIG.
Next, a fourth embodiment of the present invention will be described with reference to the drawings.
FIG. 6 is a block diagram illustrating a configuration example of a three-dimensional coordinate input system according to the fourth embodiment of the present invention.

  As shown in FIG. 6, the three-dimensional coordinate input system of this example includes two computers (central processing unit; processor; data processing unit) 600 and 650 that operate by program control, a camera 110, and an operation information enabling switch. 120. FIG. 6 also shows a subject 140 photographed by the camera 110. Display means 130 and

  The computer 600 includes a three-dimensional motion calculation unit 101 and an operation information generation unit 102. The computer 650 includes operation target software 103 and display means 130.

  It is assumed that the computer 600 and the computer 650 are connected by a serial or parallel interface such as USB (Universal Serial Bus). The computer 600 and the computer 650 may be connected via a communication network such as Ethernet (registered trademark).

Next, the operation of the three-dimensional coordinate input system of this example will be described.
In the three-dimensional coordinate input process, first, the three-dimensional motion calculation means 101 moves a terminal (not shown) equipped with the three-dimensional coordinate input system of this example from a moving image obtained by photographing the subject 140 with the camera 110. The three-dimensional movements of the camera 110 and the terminal caused by the above are calculated.

  Next, the operation information generation unit 102 converts the three-dimensional motion calculated by the three-dimensional motion calculation unit 101 into operation information of the operation target software 103.

  Next, the operation information generation unit 102 refers to the operation information enable switch 120, and inputs the operation information to the operation target software 103 when the switch 120 is ON. That is, operation information is output. When the switch 120 is OFF, no operation information is input.

  Then, the display unit 130 presents the result obtained by inputting the operation information to the operation target software 103 to the user.

  It should be noted that the attachment position of the camera of this example is the same as any one of the attachment positions in the first to third embodiments described above. That is, the camera 111 may be used instead of the camera 110, or two cameras 110 and 111 may be used.

  As described above, even in the case of the configuration including the two computers 600 and 650 as in the above-described fourth embodiment, the description is made in the above-described first to third embodiments. Various effects can be obtained.

  Although not specifically mentioned in the above-described embodiment, the terminal devices 300, 400, and 500 are, for example, PDAs (Personal Digital Assistants) having a camera function, mobile phone terminals, personal computers, portable game machines, and the like. Applied. The terminal devices 300, 400, and 500 may be devices that can be connected to a communication network, or devices that cannot be connected to a communication network.

  In addition, each part of the three-dimensional coordinate input system in each embodiment described above executes various processes described above according to a control program (including operation target software: three-dimensional coordinate input program) installed in the system. The three-dimensional coordinate input program includes, for example, a step of photographing a moving image on the computer 100, and a step of calculating an operation in the three-dimensional space of the housing as a change in three-dimensional coordinates based on the photographed moving image. The step of generating operation information based on the calculated motion in the three-dimensional space and the display means for displaying an image are controlled based on predetermined software, and are displayed on the display means according to the generated operation information. A control program for executing a step of changing an image.

Next, specific examples of the present invention will be described.
The example described below corresponds to the second embodiment described above.

  Here, a case will be described in which the above-described three-dimensional coordinate input system is applied to a three-dimensional coordinate input device that operates an image viewer operating on a digital camera.

  FIG. 7 is an explanatory diagram showing a three-dimensional coordinate system in the present embodiment. As shown in FIG. 7, in this embodiment, the origin 700 is set at the intersection of the optical axis of the digital camera and the image plane, and the x-axis 701 and the y-axis 702 are made to coincide with the x-axis and y-axis of the image plane of the digital camera, respectively. . The z-axis 703 is an axis that passes through the origin 700 and is orthogonal to the x-axis 701 and the y-axis 702, and the direction in which the subject exists is positive.

  The three-dimensional motion calculation means 101 in the computer 100 calculates the three-dimensional motion of the digital camera based on each frame of the moving image photographed by the digital camera.

  Here, the three-dimensional motion is composed of a parallel movement amount of each digital camera in the x, y, and z axis directions and a rotation amount around each axis. In the present embodiment, the three-dimensional motion is calculated by the method described in Non-Patent Document 1.

  Non-Patent Document 1: Ikeya, Nakajima, Sato, Ikeda, Kambara, Yokoya, Yamada, “Video mosaicing and super-resolution by camera path estimation for paper”, IEICE Technical Report TL2003-37 PRMU2003-233, 2004 February 19, pp. 49-54

  In the three-dimensional motion calculation method described in Non-Patent Document 1, first, a point where edges in a plurality of directions called feature points intersect is obtained on the image of the previous frame.

  Next, for each feature point, corresponding coordinates in the current frame are obtained by tracking processing by template matching.

  Then, the three-dimensional coordinates of each feature point obtained in the previous frame are projected on all frame images from the initial frame to the current frame, and the deviation between the projected coordinates and the coordinates obtained by the tracking process The three-dimensional motion of the camera and the three-dimensional coordinates of each feature point are calculated by nonlinear minimization.

  By repeating the above processing, the three-dimensional motion of the digital camera in each frame is calculated as a change in three-dimensional coordinates in the three-dimensional space.

  FIG. 8 is an explanatory diagram illustrating a specific example of the calculation state of the three-dimensional motion. For example, as shown in FIG. 8B, the image taken by the digital camera is as shown in FIG. 8B from the state where the face of the user 302 as the subject 704 exists in the left area of the taken image as shown in FIG. Assume that the face of the user 302 has changed to a state in the right area of the captured image.

  That is, as shown in FIG. 8, as a three-dimensional motion, it is assumed that a rightward movement (positive direction of the x axis) is observed on the image. According to this observation result, the three-dimensional motion calculation means 101 calculates the parallel movement in the negative direction of the x axis and the amount of movement.

  FIG. 9 is an explanatory diagram illustrating another specific example of the calculation state of the three-dimensional motion. For example, as shown in FIG. 9B, the image taken by the digital camera is as shown in FIG. 9B from the state where the face of the user 302 as the subject 704 exists near the center of the taken image as shown in FIG. It is assumed that the face of the user 302 has changed to a state where the face of the photographed image is enlarged in the vertical and horizontal directions.

  That is, as a three-dimensional movement, as shown in FIG. 9, it is assumed that a radial movement due to movement in the direction toward the subject 704 (the negative direction of the z axis) is observed on the image. According to this observation result, the three-dimensional motion calculation unit 101 calculates the parallel movement in the positive direction of the z axis and the amount of movement.

  Next, the operation information generation unit 102 displays the three-dimensional movement amount (movement direction and movement amount) calculated by the three-dimensional movement calculation unit 101, and the image viewer unit (computer 100 that performs control according to the operation target software 103) displays the display unit 130. It is converted into operation information necessary for control, and if it is in a state where the operation information enabling switch 120 is pressed (ON state), it is input to the image viewer unit.

  Here, the movement amount that is the same as the movement amount of the digital camera in the x- and y-axis directions is the scroll amount in the image viewer unit, and the movement amount that is the same as the movement amount in the z-axis direction is the scale for enlargement / reduction in the image viewer unit. It shall be mapped.

  In addition, when the user wants to scroll the image greatly on the viewer, the user needs to greatly translate the terminal even in the real world. This makes scrolling operations difficult when a large space cannot be secured in the environment where the user is present.

  In such a case, the digital camera is slightly translated while pressing the operation information enable switch 120, and then the operation information enable switch 120 is released (turned off) to return the digital camera to the original position. A desired scroll amount can be obtained by repeating the above operation a plurality of times.

  Further, a movement amount proportional to the movement amount of the digital camera, not the same movement amount as the movement amount of the digital camera, may be used as the scroll amount or enlargement / reduction scale in the image viewer unit.

  Then, the image viewer unit displays the display image generated based on the input operation information on the display unit 130 and presents it to the user.

  FIG. 10 is an explanatory diagram illustrating a specific example of an image displayed on the display unit 130 by display control of the image viewer unit. Here, a case where the image captured by the digital camera changes from FIG. 8A to FIG. 8B will be described as an example.

  For example, it is assumed that an image as shown in FIG. 10C is displayed on the display means 130 when the image of FIG. An image shown in FIG. 10C is a partial image 1002 that is a part of the entire image shown in FIG.

  FIG. 10A shows an entire image that can be displayed on the display unit 130 by the image viewer unit in this example.

  Next, when the digital camera is moved in the negative x-axis direction while the operation information validation switch 120 is pressed by the user, the subject 704 appears to move in the positive x-axis direction on the captured image of the digital camera. The image shown in FIG. 8B is captured.

  The operation information generation unit 102 displays the three-dimensional momentum calculated by the three-dimensional motion calculation unit 101 (the momentum of the subject from the state of FIG. 8A to the state of FIG. 8B) by the image viewer unit. Is converted into operation information necessary for controlling the operation information, and since the operation information enabling switch 120 is pressed, the information is input to the image viewer unit.

  When the operation information is input, the image viewer unit moves the frame in which the partial image 1002 is located by the movement amount indicated by the operation information in the movement direction (the negative x-axis direction) indicated by the operation information. A partial image 1001 that is an image surrounded by a frame is cut out from the entire image.

  Then, the image viewer unit changes the display image of the display unit 130 from the partial image 1002 shown in FIG. 10C to the partial image 1001 shown in FIG.

  FIG. 11 is an explanatory diagram illustrating another specific example of an image displayed on the display unit 130 by display control of the image viewer unit. Here, a case where the image captured by the digital camera changes from FIG. 9A to FIG. 9B will be described as an example.

  For example, it is assumed that an image as shown in FIG. 11C is displayed on the display unit 130 when the image of FIG. An image illustrated in FIG. 11C is a partial image 1102 that is a part of the entire image illustrated in FIG.

  Note that FIG. 11A shows an entire image that can be displayed on the display means 130 by the image viewer unit in this example.

  Next, when the digital camera is moved in the positive z-axis direction with the operation information enabling switch 120 pressed by the user, the subject 704 appears to move in the negative z-axis direction on the captured image of the digital camera. The image shown in FIG. 9B is captured.

  The operation information generating unit 102 displays the three-dimensional momentum (the momentum of the subject from the state of FIG. 9A to the state of FIG. 9B) calculated by the three-dimensional motion calculating unit 101 by the image viewer unit. Is converted into operation information necessary for controlling the operation information, and since the operation information enabling switch 120 is pressed, the information is input to the image viewer unit.

  When the operation information is input, the image viewer unit moves the frame in which the partial image 1102 is located by the movement amount indicated by the operation information in the movement direction (z-axis positive direction) indicated by the operation information. A partial image 1101 that is an image surrounded by a frame is cut out from the entire image.

  Then, the image viewer unit changes the display image of the display unit 130 from the partial image 1102 shown in FIG. 11C to the partial image 1101 shown in FIG.

  As described above, if the operation information enabling switch 120 is pressed, the display screen of the display unit 130 moves as the digital camera moves.

  The above example is a specific example of the second embodiment, but the same processing is performed for other embodiments. However, when there is a camera subject on the user side as in the second embodiment, and when there is a camera subject on the opposite side of the user as in the first embodiment, the image is captured by the camera. The moving direction of the subject to be displayed is different from the moving direction of the display image on the display unit 130. That is, when there is a camera subject on the user side, the moving direction of the subject imaged by the camera and the moving direction of the display image on the display unit 130 are opposite in the z-axis direction and the same in the xy-axis direction. Become a direction. When there is a camera subject on the opposite side of the user, the moving direction of the subject imaged by the camera and the moving direction of the display image on the display unit 130 are opposite to each other in the xy axis direction, and the z axis direction. Then the same direction.

  In the above example, when there is a camera subject on the user side, the moving direction of the subject imaged by the camera and the moving direction of the display image of the display unit 130 are opposite in the z-axis direction, and xy The axial direction is the same direction, but the xy axis direction may be the opposite direction, and the z axis direction may be the same direction. In this case, the display image of the display unit 130 moves (scrolls, enlarges / reduces) in the direction opposite to the direction in which the user moves the digital camera. Similarly, when there is a camera subject on the opposite side of the user, the moving direction of the subject imaged by the camera and the moving direction of the display image of the display unit 130 are opposite in the z-axis direction, The direction may be the same in the xy axis direction.

  INDUSTRIAL APPLICABILITY According to the present invention, it is useful for application to applications such as viewers for images, documents, web pages, etc. on mobile terminals, and user interfaces for browser operations.

  Further, according to the present invention, the present invention can be applied to an application such as a user interface for operating characters around the virtual world in a game.

It is a block diagram which shows the structural example of the three-dimensional coordinate input system in the 1st Embodiment of this invention. It is explanatory drawing which shows the example of the terminal device to which the three-dimensional coordinate input system in the 1st Embodiment of this invention is applied. It is a flowchart which shows the example of the operation | movement in the 1st Embodiment of this invention. It is explanatory drawing which shows the example of the terminal device to which the three-dimensional coordinate input system in the 2nd Embodiment of this invention is applied. It is explanatory drawing which shows the example of the terminal device to which the three-dimensional coordinate input system in the 3rd Embodiment of this invention is applied. It is a block diagram which shows the structural example of the three-dimensional coordinate input system in the 4th Embodiment of this invention. It is explanatory drawing which shows the three-dimensional coordinate system in a specific Example. It is explanatory drawing which shows the mode of the three-dimensional motion calculation in a specific Example. It is explanatory drawing which shows the mode of the three-dimensional motion calculation in a specific Example. It is explanatory drawing which shows the mode of a change of the display image based on the operation information in a specific Example. It is explanatory drawing which shows the mode of a change of the display image based on the operation information in a specific Example.

Explanation of symbols

100, 600, 650 Computer (central processing unit; processor; data processing unit)
DESCRIPTION OF SYMBOLS 101 Three-dimensional motion calculation means 102 Operation information generation means 103 Operation target software 110,111 Camera 120 Operation information validation switch 130 Display means 140,705 Subject 300,400,500 Terminal device 302 User 700 Origin 701 x-axis 702 y-axis 703 z-axis 704 Digital camera image plane

Claims (12)

A moving image shooting means for shooting a moving image;
Display means for displaying an image;
Display control means for controlling the display means based on predetermined software;
Three-dimensional motion calculation means for calculating, as a change in three-dimensional coordinates, a motion in a three-dimensional space of a casing on which the moving image photographing means is mounted, based on a moving image photographed by the moving image photographing means;
Operation information generating means for generating operation information based on the motion calculated by the three-dimensional motion calculating means,
The three-dimensional coordinate input system, wherein the display control means changes an image displayed on the display means in accordance with the operation information generated by the operation information generation means. The three-dimensional coordinate input system according to claim 1, further comprising an operation information validation switch for validating or invalidating the operation information generated by the operation information generating unit according to a user operation. The display unit and the display control unit are controlled by a computer different from a computer that controls the apparatus including the moving image shooting unit including a casing on which the moving image shooting unit is mounted. The described three-dimensional coordinate input system. The display means is installed in the housing,
The three-dimensional coordinate input system according to any one of claims 1 to 3, wherein the moving image photographing unit is installed on a back side of the housing opposite to the front side on which the display unit is installed. The display means is installed in the housing,
The three-dimensional coordinate input system according to any one of claims 1 to 3, wherein the moving image photographing unit is installed on a front side of the housing where the display unit is installed. The moving image photographing means includes a first moving image photographing means and a second moving image photographing means,
The display means is installed in the housing,
The first moving image photographing means is installed on the front side of the housing where the display means is installed,
The three-dimensional coordinate input system according to any one of claims 1 to 3, wherein the second moving image photographing unit is installed on a back side of the housing opposite to the front side. The display means includes a backlight for brightly displaying the display screen,
The three-dimensional coordinate according to claim 5 or 6, wherein the moving image photographing means installed on the front side of the casing together with the display means uses the backlight as a light source for illuminating the subject when photographing a moving image. Input system. The three-dimensional coordinate input system according to any one of claims 1 to 7, further comprising a light source for illuminating a subject when the moving image capturing means captures a moving image. The moving image photographing means photographs a subject on which a texture prepared in advance is printed in order to obtain a moving image for causing the three-dimensional movement calculating means to calculate the movement in the three-dimensional space of the casing. The three-dimensional coordinate input system according to claim 8. A moving image shooting means for shooting a moving image;
Three-dimensional motion calculation means for calculating, as a change in three-dimensional coordinates, a motion in a three-dimensional space of a casing on which the moving image photographing means is mounted, based on a moving image photographed by the moving image photographing means;
Operation information generating means for generating operation information for changing the image displayed on the display means based on the motion calculated by the three-dimensional motion calculating means;
A three-dimensional coordinate input device comprising: an operation information output unit that outputs the operation information generated by the operation information generation unit to a display control unit that controls the display unit based on predetermined software. . A step of shooting a moving image;
Calculating the movement of the housing in the three-dimensional space as a change in three-dimensional coordinates based on the captured moving image;
Generating operation information based on the calculated motion in the three-dimensional space;
Controlling the display means for displaying an image based on predetermined software, and changing the image displayed on the display means in accordance with the generated operation information. Method. On the computer,
A step of shooting a moving image;
Calculating the movement of the housing in the three-dimensional space as a change in three-dimensional coordinates based on the captured moving image;
Generating operation information based on the calculated motion in the three-dimensional space;
A three-dimensional coordinate input program for controlling display means for displaying an image based on predetermined software, and executing the step of changing the image displayed on the display means in accordance with the generated operation information.

Images