JP2005242854A - Method and device for displaying three-dimensional data, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To permit a portable communication terminal with a camera, or the like to transmit a command without making contact while seeing the same marker. <P>SOLUTION: A display position calculation part 11 calculates a display position, on a screen, of 3D data displayed on a viewer. In a specific area information DB12, positional information on a specific area on the screen is registered. A specific area contact determination part 13 compares the present position of the 3D data with the positional information registered in the specific area information DB12, and determines whether the 3D data has overlapped the specific area. A command transmission part 14 transmits a specific command, if determined that the 3D data has overlapped the specific area. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a three-dimensional data display method for capturing an image including a marker and displaying three-dimensional data (3D data) corresponding to the marker at a position corresponding to the marker.

  Here, as 3D data, CG data (for example, data described in VRML (Virtual Reality Modeling Language)) that can be expressed in a three-dimensional space having three-dimensional information of x, y, and z, which are widely used. In addition, as shown in Non-Patent Document 1, there are data (a CG and a live-action image) created by combining a plurality of two-dimensional images.

  A marker, for example, a marker in the shape of a black rectangular frame, is photographed by a camera, 3D position information of the marker is calculated, and 3D data of an object placed on the marker is displayed on a viewer (such as a mobile phone). The system is known (Non-Patent Document 1).

Further, a pointing pointer for designating a video object in the screen by pointing from a place away from the screen has been developed (Non-Patent Document 2).
Hirokazu Kato, Mark Billinghurst, Koichi Asano, Keihachiro Tachibana “Augmented Reality System Based on Marker Tracking and Its Calibration” Transactions of the Virtual Reality Society of Japan Vol. 4, no. 4,1999 Yasuaki Kaneji, Seiko Nagashima, Mitsuho Yamada, Toshihiro Shimizu “A Method for Specifying the Cursor Position of a Pointer Pointer” IEICE Technical Committee, 2002.2.1.29

  In the above-described system of Non-Patent Document 1 that displays 3D data using markers, different markers are required to display different 3D data.

  In addition, as in Non-Patent Document 2, it has been developed to specify a video object in the screen by pointing from a place away from the screen. However, in the above system, when 3D data is displayed, a key or the like is displayed. The command could not be sent without contact without operating.

  An object of the present invention is to switch to another 3D data while viewing the same marker without changing the marker and without operating a key or the like when 3D data is displayed, 3D data display method, apparatus, and program can be provided.

According to the first aspect of the present invention, a method for displaying three-dimensional data includes:
Obtaining a display position on the screen of the display device of the three-dimensional data displayed on the screen;
Determining whether the three-dimensional data overlaps with a specific area previously provided on the screen;
A step of transmitting a specific command corresponding to the specific area when the three-dimensional data overlaps the specific area.

According to the second aspect of the present invention, a method for displaying three-dimensional data includes:
Obtaining the display position on the screen of the display position of the three-dimensional data displayed on the screen;
Obtaining a moving position and a moving direction of the display position at a certain time;
A step of transmitting a specific command based on the obtained moving distance and moving direction.

  As a result, the 3D data corresponding to the marker displayed on the screen can be replaced with another 3D data, or the size can be changed (zoom-in / In the case of three-dimensional data of zooming out and moving images, it is possible to perform control such as reproduction, stop, and fast forward.

  According to the present invention, a specific command is generated by a simple operation of moving (tilting) the viewer (mobile communication terminal with camera) while viewing the same marker without changing the marker, and switching to another 3D data, The function of 3D data can be switched.

  Next, embodiments of the present invention will be described with reference to the drawings.

[First Embodiment]
FIG. 1 is a configuration diagram of a three-dimensional data display apparatus according to a first embodiment of the present invention, and FIG. 2 is a flowchart showing a flow of the processing.

The display position calculation unit 11 calculates the display position on the screen of the 3D data displayed on the viewer (step 101). In the specific area information DB 12, position information of a specific area on the screen is registered. The specific area contact determination unit 13 compares the current position of the 3D data with the position information registered in the specific area information DB 12, and determines whether the 3D data overlaps with the specific area (steps 103 and 104). When it is determined that the 3D data overlaps the specific area, the command transmission unit 14 transmits a specific command (step 105).

For example, when one of the specific areas is a rectangle represented by a range of (x0 ≦ x ≦ x1, y0 ≦ y ≦ y1) and the current position of the 3D data is (xdata, ydata), the former is the specific area Information, the latter corresponds to display position information. At this time, in the specific area contact determination process of FIG. 3, when both the information are always checked and (x0 ≦ xdata ≦ x1) and (y0 ≦ ydata ≦ y1) are satisfied, the 3D data is in contact with the specific area. Can be determined.

  FIG. 3 is a diagram showing a specific example of this embodiment. It is assumed that the specific area is set to the left and right at the edge of the screen, and the display position of 3D data is at the center of the screen (FIG. 3 (1)). When the viewer is moved (or tilted) in this state, the display position of the 3D data on the screen is shifted and overlaps with a specific area set near the edge (FIG. 3 (2)). Different 3D data from the server (also stored in the viewer) is displayed on the screen via the network (FIG. 3 (3)). Alternatively, display and non-display of 3D data are switched.

  Note that when the 3D data overlaps the specific area, the display of the 3D data may be turned off. Nothing may be displayed in the specific area, but an icon indicating the specific area may be displayed. If 3D data overlaps the same specific area twice or more within a certain time, another command can be transmitted. Furthermore, if 3D data overlaps with a specific combination in different areas within a certain time, another command can be transmitted. The shape (rectangle, circle, etc.), size, and number of specific areas are arbitrary. 4 (1), (2), and (3) show other examples (shaded portions) of the specific region. The viewer may be a mobile phone, a PDA (Personal Digital Assistant), a notebook PC (Personal Computer), or an HMD (Head Mounted Display). The same applies to the following embodiments.

[Second Embodiment]
FIG. 5 is a block diagram of a three-dimensional data display apparatus according to the second embodiment of the present invention, and FIG. 6 is a flowchart showing the processing flow.

  The display position calculation unit 21 calculates the display position of the 3D data on the screen (step 201). The movement information calculation unit 22 calculates the movement distance and movement direction of the 3D data (step 202). Here, it is assumed that the display position of the 3D data on the screen is always detected. If the X and Y coordinates at a certain time t0 are (Xt0, Yt0) and the X and Y coordinates at the time t1 are (Xt1, Yt1), the distance D01 moved between t0 and t1 is D01 = {(Xt1 -Xt0) ^ 2 + (Yt1-Yt0) ^ 2} ^ (1/2). The moving direction is obtained from the slope of the straight line connecting the two points. For example, the inclination is calculated as A01 = (Yt1-Yt0) / (Xt1-Xt0). Alternatively, the angle θ01 is obtained as sin θ01 = (Yt1−Yt0) / D01. The movement information determination unit 23 determines whether or not the movement distance thus obtained exceeds a predetermined threshold Dth and the angle in the movement direction is within a predetermined range (steps 203 and 204). When it is determined that the 3D data has moved in a certain direction by the threshold value Dth or more, the command transmission unit 24 transmits a specific command.

  FIG. 7 shows a specific example of this embodiment. When the 3D data is displayed (FIG. 7 (1)), when the screen is moved in the upper right direction, the display position of the 3D data moves in the lower left direction as shown by an arrow (FIG. 7 (2)). When the movement distance D01 exceeds the threshold, in this example, in the range of −45 to 45 degrees, 45 to 135 degrees, 135 to 225 degrees, and 225 to 315 degrees (= −45 degrees) in this example. A command is transmitted according to whether it is entered (FIG. 7 (4)), and the display is switched to different 3D data or the display of 3D data is turned off (FIG. 7 (3)). Thus, only by moving the display device, any one of a plurality of types of commands can be selected and transmitted by recognizing the movement.

  8 (1), (2), and (3) show other examples of the moving direction.

[Third Embodiment]
FIG. 9 is a flowchart showing processing of the command transmission unit of the three-dimensional data display apparatus according to the third embodiment of the present invention. Other configurations are the same as those of the first embodiment. FIG. 10 is an explanatory diagram of a specific example of this embodiment.

This embodiment
For example, a specific area is provided in which the data number is +1 at the right end of the display screen and the data number is -1 at the left end (FIG. 10 (1)). When moved left and right (FIGS. 10 (2) and (3)), data can be switched one after another (FIGS. 10 (4) and (5)). For example, while looking at a marker corresponding to an object such as a product, it is possible to switch products one after another simply by moving the display device while viewing the same marker. As a result, the products in the product catalog can be switched and displayed one after another, or for example, a product suitable for one's room can be searched.
For example, if a plurality of data has a hierarchical structure, a specific area having a function of switching 3D data by raising the hierarchy at the upper end of the display screen and lowering the hierarchy at the lower end can be provided.
-For example, switch to 3D data with different time on the left and right. If the image is continuously superimposed on the right specific area, the 3D data of the object changes to a new one in time, and on the left, it changes to an old one. It is necessary to create a plurality of 3D data with different times for the same object in advance.
When switching 3D data at an arbitrary viewpoint, the image is switched to an image of the same viewpoint at the time of switching.
This method can be realized not only by a method using a specific area, but also by the method using motion recognition according to the second embodiment.

  The command transmission unit first determines whether there is a command transmission request (step 301). If not, the process returns to the processing of the display position calculation unit. When there is a command transmission request, if the specific area is the left specific area, it is a data advance command, so it switches to the next 3D data (steps 302 and 303). Switch to the previous 3D data (steps 304 and 305).

[Fourth Embodiment]
FIG. 11 is a flowchart showing processing of the command transmission unit of the three-dimensional data display apparatus according to the fourth embodiment of the present invention. Other configurations are the same as those of the three-dimensional data display apparatus according to the first embodiment. FIG. 12 is an explanatory diagram of a specific example.

  This embodiment is a case where 3D data is a moving image, and specific regions are provided at the left end and the right end of the screen, respectively, as in the first to third embodiments (FIG. 12 (1)). If the command transmission unit determines that there is a command transmission request in step 401, if the 3D data overlaps with the leftmost specific area (FIG. 12 (2)), the command transmission unit outputs a command for reproducing the 3D data of the moving image (step 402, 403), the 3D data of the moving image is reproduced (FIG. 12 (4)), and when the 3D data overlaps with the specific area on the right end (FIG. 12 (3)), a command for stopping the 3D data of the moving image is output. (Steps 404 and 405), and the 3D data of the moving image stops (FIG. 12 (5)).

  A specific area may be further provided to output commands such as fast forward and rewind. Further, the present embodiment can be similarly realized by the motion recognition method of the second embodiment.

[Fifth Embodiment]
FIG. 13 is a flowchart showing processing of the command transmission unit of the three-dimensional data display apparatus according to the fifth embodiment of the present invention. Other configurations are the same as those of the three-dimensional data display apparatus according to the first embodiment. FIG. 14 is an explanatory diagram of a specific example thereof.

  In the present embodiment, specific areas are provided at the upper and lower ends of the screen, respectively (FIG. 14 (1)). If the command transmission unit determines that there is a command transmission request in step 501, if the 3D data overlaps the specific area at the top (FIG. 14 (2)), the command transmission unit outputs a command to zoom in on the 3D data (steps 502 and 503). The 3D data is zoomed in (FIG. 14 (4). When the 3D data overlaps the specific area at the bottom (FIG. 14 (3)), a command for zooming out the 3D data is output (steps 504 and 505). The 3D data is zoomed out (FIG. 14 (5)).

  This embodiment has an advantage that the display size of the target object can be changed without the user moving.

  Note that this embodiment can be similarly realized by the motion recognition method of the second embodiment.

  In the above embodiment, 2D data downloaded from a server via the Internet can be used as 3D data to be displayed.

  In addition to what is implemented by dedicated hardware, the present invention records a program for realizing the function on a computer-readable recording medium, and the program recorded on the recording medium is stored in a computer system. It may be read and executed. The computer-readable recording medium refers to a recording medium such as a floppy disk, a magneto-optical disk, a CD-ROM, or a storage device such as a hard disk device built in the computer system. Furthermore, a computer-readable recording medium is a server that dynamically holds a program (transmission medium or transmission wave) for a short period of time, as in the case of transmitting a program via the Internet, and a server in that case. Some of them hold programs for a certain period of time, such as volatile memory inside computer systems.

1 is a block diagram of a three-dimensional data display device according to a first embodiment of the present invention. It is a flowchart which shows the process of the three-dimensional data display apparatus of 1st Embodiment. It is explanatory drawing of the specific example of 1st Embodiment. It is a figure which shows the other example of a specific area | region. It is a block diagram of the three-dimensional data display apparatus of the 2nd Embodiment of this invention. It is a flowchart which shows the process of the three-dimensional data display apparatus of 2nd Embodiment. It is explanatory drawing of the specific example of 2nd Embodiment. It is a figure which shows the other example of a moving direction. It is a flowchart which shows the process of the command transmission part of the three-dimensional data display apparatus of the 3rd Embodiment of this invention. It is explanatory drawing of 3rd Embodiment. It is a flowchart which shows the process of the command transmission part of the three-dimensional data display apparatus of the 4th Embodiment of this invention. It is explanatory drawing of 4th Embodiment. It is a flowchart which shows the process of the command transmission part of the three-dimensional data display apparatus of the 5th Embodiment of this invention. It is explanatory drawing of 5th Embodiment.

Explanation of symbols

11 Display position calculation unit 12 Specific area information DB
DESCRIPTION OF SYMBOLS 13 Specific area | region contact determination part 14 Command transmission part 21 Display position calculation part 22 Movement information calculation part 23 Movement determination part 24 Command transmission part 101-105, 201-205, 301-305, 401-405, 501
505 steps

Claims (7)

In the method of displaying three-dimensional data, an image including a marker is taken and three-dimensional data corresponding to the marker is displayed at a position corresponding to the marker.
Obtaining a display position on the screen of the display device of the three-dimensional data displayed on the screen;
Determining whether the three-dimensional data overlaps with a specific area previously provided on the screen;
And a step of transmitting a specific command corresponding to the specific region when the three-dimensional data overlaps the specific region. In the method of displaying three-dimensional data, an image including a marker is taken and three-dimensional data corresponding to the marker is displayed at a position corresponding to the marker.
Obtaining the display position on the screen of the display position of the three-dimensional data displayed on the screen;
Obtaining a moving position and a moving direction of the display position at a certain time;
And a step of transmitting a specific command based on the obtained moving distance and moving direction.   The method of displaying three-dimensional data according to claim 1 or 2, wherein the command is a command for replacing three-dimensional data corresponding to a marker displayed on the screen with another three-dimensional data.   The three-dimensional data display method according to claim 1, wherein the command changes an image of three-dimensional data corresponding to a marker displayed on a screen. In a three-dimensional data display device that captures an image including a marker and displays three-dimensional data corresponding to the marker at a position corresponding to the marker,
Means for obtaining a display position on the screen of the display device of the three-dimensional data displayed on the screen;
Means for determining whether or not the three-dimensional data and a specific area previously provided on the screen overlap;
And a means for transmitting a specific command corresponding to the specific area when the three-dimensional data overlaps the specific area. In a three-dimensional data display device that captures an image including a marker and displays three-dimensional data corresponding to the marker at a position corresponding to the marker,
Means for obtaining a display position of the display position of the three-dimensional data displayed on the screen on the screen;
Means for determining a moving position and a moving direction of the display position at a certain time;
A display device for three-dimensional data, comprising means for transmitting a specific command based on the obtained moving distance and moving direction.   The program for making a computer perform the display method of three-dimensional data of any one of Claim 1 to 4.

Images