JP7475625B2 - Method and program for receiving and displaying input in three-dimensional space, and device for receiving and displaying input in three-dimensional space - Google Patents

Description

The present invention relates to a method, program, and device for accepting and displaying input in a three-dimensional space.

A method for creating 3D data in which text data is embedded in a 3D model is known in which the text data is linked to position data of a specified portion of the 3D model designated by the user (Patent Document 1).

JP 2019-087069 A

However, as shown in FIG. 11, in Patent Document 1, the text data (the portion shown as "text" in the figure) is two-dimensionally input and displayed on an image in three-dimensional space (the portion shown as "screen" in the figure) (Patent Document 1, Figure 4). For this reason, in Patent Document 1, although the text data is linked to the three-dimensional model, it is difficult to say that it is input in three-dimensional space.

In view of the above, the present invention provides a system that enables input reception and input display in three-dimensional space.

In order to achieve the above object, a method for receiving and displaying an input in a three-dimensional space according to the present invention includes a three-dimensional space output step, a two-dimensional structure generation step, and a user input receiving step,
The three-dimensional space output step outputs an image of the three-dimensional space to be displayed on an image display unit,
The two-dimensional structure generating step generates, in the three-dimensional space, a two-dimensional structure for receiving an input to the three-dimensional space;
The user input receiving step receives an input by a user into the three-dimensional space,
when receiving an input to the three-dimensional space, the input content to the three-dimensional space is linked to the three-dimensional space position of the two-dimensional structure based on a display position of the two-dimensional structure in an image of the three-dimensional space and a three-dimensional space position of the two-dimensional structure;
the three-dimensional space output step outputs an image of the three-dimensional space such that the input content into the three-dimensional space is displayed at the associated three-dimensional space position;
Each of the above steps is performed by a computer.

The program of the present invention is capable of executing the method of the present invention on a computer.

The input reception and input display device in a three-dimensional space of the present invention includes a three-dimensional space output means, a two-dimensional structure generation means, and a user input reception means,
the three-dimensional space output means outputs an image of the three-dimensional space to be displayed on the image display unit;
the two-dimensional structure generating means generates, in the three-dimensional space, a two-dimensional structure for receiving an input to the three-dimensional space;
the user input receiving means receives an input into the three-dimensional space by a user;
when receiving an input to the three-dimensional space, the input content to the three-dimensional space is linked to the three-dimensional space position of the two-dimensional structure based on a display position of the two-dimensional structure in an image of the three-dimensional space and a three-dimensional space position of the two-dimensional structure;
The three-dimensional space output means outputs an image of the three-dimensional space such that the input content into the three-dimensional space is displayed at the linked three-dimensional space position.

The present invention provides a system that allows input reception and display in three-dimensional space.

FIG. 1 is a block diagram showing an example of a configuration of an input receiving and displaying device according to a first embodiment of the present invention. FIG. 2 is a block diagram showing an example of a hardware configuration of the input receiving and displaying device according to the first embodiment of the present invention. FIG. 3A is a diagram showing an example of an image in a three-dimensional space according to the first embodiment of the present invention. FIG. 3B is a diagram showing an example of an image in a three-dimensional space in the first embodiment of the present invention. FIG. 4 is a diagram showing an example of an image in a three-dimensional space in the first embodiment of the present invention. FIG. 5 is a diagram showing an example of an image in a three-dimensional space in the first embodiment of the present invention. FIG. 6 is a diagram for explaining a display position in an image in a three-dimensional space in the first embodiment of the present invention. FIG. 7 is a schematic diagram showing the correspondence between the display position of a two-dimensional structure in an image in three-dimensional space and the three-dimensional spatial position of the two-dimensional structure in the first embodiment of the present invention. FIG. 8 is a diagram for explaining the center of gravity of a virtual quadrangle including a series of input contents according to the first embodiment of the present invention. FIG. 9 is a schematic diagram showing an example of a case where input content into a three-dimensional space is displayed with a thickness from a two-dimensional structure surface in the first embodiment of the present invention. FIG. 10 is a flowchart showing an example of an input reception and display method according to the first embodiment of the present invention. FIG. 11 is a diagram showing an example of a case in which text data is two-dimensionally input and displayed on an image in a three-dimensional space in the prior art.

According to the present invention, as described above, based on the display position of a two-dimensional structure in an image in three-dimensional space and the three-dimensional spatial position of the two-dimensional structure, the input contents to the three-dimensional space are linked to the three-dimensional spatial position of the two-dimensional structure. This makes it possible to accept and display input in areas in the three-dimensional space other than the area where an object (e.g., the "three-dimensional model" in Patent Document 1) exists.

In the present invention, a "user" is a user of the input reception and input display device of the present invention (also called an "input reception display device"). The user may be one person or multiple people.

The embodiments of the present invention will be described with reference to the drawings. The present invention is not limited or restricted in any way by the following embodiments. In each of the drawings, the same parts are given the same reference numerals. The explanations of each embodiment can be used interchangeably unless otherwise specified.

[Embodiment 1]
1 is a block diagram showing an example of a configuration of an input reception/display device 10 according to the present embodiment. The input reception/display device 10 includes a three-dimensional space output unit 11, a two-dimensional structure generation unit 12, and a user input reception unit 13.

The input reception display device 10 may be, for example, a single device including each of the above-mentioned parts, or may be a device (system) in which each of the above-mentioned parts can be connected via a wired or wireless communication network. The communication network is not particularly limited and any known network can be used, and may be, for example, wired or wireless. Examples of the communication network include the Internet line, telephone line, LAN (Local Area Network), and WiFi (Wireless Fidelity). The input reception display device 10 may be, for example, incorporated into a server as a system. Furthermore, the input reception display device 10 may be, for example, a computer in which the program of the present invention is installed.

In this embodiment, the input receiving and displaying device 10 is a personal computer (PC). However, this is not limited to this, and the input receiving and displaying device 10 may be, for example, a mobile terminal such as a smartphone, a tablet terminal, or a wearable terminal, or a game console, etc.

Furthermore, FIG. 2 illustrates a block diagram of the hardware configuration of the input reception display device 10. The input reception display device 10 has, for example, a CPU (central processing unit) 101, a memory 102, a bus 103, a communication device 104, a storage device 105, and the like. Each part of the input reception display device 10 is connected, for example, by each interface (I/F) via the bus 103.

The CPU 101 is responsible for the overall control of the input reception display device 10. In the input reception display device 10, the CPU 101 executes, for example, the program of the present invention and other programs, and also reads and writes various information. Specifically, in the input reception display device 10, the CPU 101 functions, for example, as a three-dimensional space output means 11, a two-dimensional structure generation means 12, and a user input reception means 13.

The bus 103 connects between each of the functional units, such as the CPU 101, memory 102, etc. The bus 103 can also connect to, for example, an external device. The external device can be, for example, an image display device, an input device, and an audio output device such as a speaker. The image display device and the input device will be described later. The input receiving and display device 10 can be connected to a communication network by a communication device 104 connected to the bus 103, and can also be connected to the external device via the communication network.

The memory 102 includes, for example, a main memory, which is also called a primary storage device. When the CPU 101 performs processing, the memory 102 reads various operating programs 106, such as the program of the present invention, which are stored, for example, in an auxiliary storage device described below, and the CPU 101 receives data from the memory 102 and executes the programs 106. The main memory is, for example, a RAM (random access memory). The memory 102 further includes, for example, a ROM (read only memory).

The storage device 105 is also referred to as an auxiliary storage device, as opposed to the main memory. The storage device 105 includes, for example, a storage medium and a drive for reading and writing data from and to the storage medium. The storage medium is not particularly limited and may be either built-in or external, and examples of the storage medium include HD (hard disk), CD-ROM, CD-R, CD-RW, MO, DVD, flash memory, and memory card, while the drive is not particularly limited. The storage device 105 may be, for example, a hard disk drive (HDD) in which a storage medium and a drive are integrated. The storage device 105 stores, for example, the operating program 106, as described above. The storage device 105 may also store, for example, information on three-dimensional space, as described below.

In the input reception and display device 10, the memory 102 and storage device 105 can also store access information and log information from users, as well as information obtained from an external database (not shown).

The three-dimensional space output means 11 outputs an image of the three-dimensional space to be displayed on the image display unit. Examples of the image display unit include a display (liquid crystal display, organic EL display, etc.), a touch panel, a screen, etc. The image display unit may be built into the input receiving and display device 10 (not shown), or may be an externally connected image display device. The three-dimensional space output means 11 outputs, for example, information on the image of the three-dimensional space to the image display unit via the bus 103.

The three-dimensional space is, for example, a space defined by three values, an X coordinate, a Y coordinate, and a Z coordinate (e.g., width, depth, and height, respectively). The three-dimensional space is not particularly limited, and specific examples include a virtual reality space, a game space, a simulation (performance test, etc.), a training (driving training, surgery training, etc.), a design/drawing space, and an image space captured by a three-dimensional camera.

The three-dimensional space may be fixed (stationary or moving) or may change based on user input, etc. In the latter case, the three-dimensional space may be controlled by, for example, a three-dimensional space control unit (e.g., a CPU).

The image of the three-dimensional space is a two-dimensional image. The two-dimensional image includes, for example, an image for pseudo three-dimensional display (stereoscopic television, VR (Virtual Reality) goggles, etc.). The image of the three-dimensional space includes information on the display position, as described below.

The three-dimensional space output means 11, for example, calculates an image of the three-dimensional space to be displayed on the image display unit from the information of the three-dimensional space in the output, and outputs this. The information of the three-dimensional space is not particularly limited, and examples thereof include information of objects included in the three-dimensional space (information on color, shape, etc., position information, etc.), as well as information on two-dimensional structures described below, and input contents by the user into the three-dimensional space. The object is not particularly limited, and can be called, for example, an "object." The "object" may specifically be, for example, a structure in the virtual reality space, a designed model in the design space, a character in the game space, etc. Note that the input contents by the user into the three-dimensional space may be the object, or may not be the object.

The image of the three-dimensional space can be calculated, for example, based on setting information (position, angle, zoom, perspective, viewing angle, etc.) of a virtual camera that virtually captures the three-dimensional space. The setting information of the virtual camera can be determined, for example, according to an input from the user.

Figure 3 (A) is a diagram showing an example of an image of the three-dimensional space displayed on the image display unit. From the three-dimensional space image 100 shown in Figure 3 (A), it can be seen that the object is included in the three-dimensional space (left and right sides in the figure). Also, in the figure, the object of "ground" exists in the white area.

The three-dimensional space output means 11 can, for example, output a plurality of different images of one three-dimensional space. That is, for example, different images of the three-dimensional space may be output to one user according to the setting information of the virtual camera, or different images of the three-dimensional space may be output to multiple users who share one three-dimensional space (for example, simultaneously).

The two-dimensional structure generating means 12 generates a two-dimensional structure in the three-dimensional space for receiving input to the three-dimensional space. "For receiving input to the three-dimensional space" means, for example, that the main purpose is to receive input to the three-dimensional space. The two-dimensional structure is, for example, different from the object contained in the three-dimensional space. In addition, the two-dimensional structure is, for example, temporarily generated while receiving input to the three-dimensional space.

Figure 3(B) is a diagram showing an example of the two-dimensional structure. In Figure 3(B), it can be seen that a two-dimensional structure P for receiving input to the three-dimensional space has been generated in the three-dimensional space.

The shape of the two-dimensional structure is not particularly limited, and may be, for example, a flat surface or a curved surface as shown in FIG. 3(B). The curved surface is not particularly limited, and may be, for example, a sphere, a cylinder, a whole or a part of a quadratic curved surface, a wavy structure, a random curved surface, etc. The shape and size of the two-dimensional structure can be set and changed, for example, according to the user's input. The two-dimensional structure may have a three-dimensional structure (for example, a three-dimensional decoration, etc.) to the extent that it does not interfere with the user's input into the three-dimensional space, as described below.

The two-dimensional structure may have transparency as shown in FIG. 3(B). The term "having transparency" may mean transparent or semi-transparent. By having the transparency, for example, in an image 100 in three-dimensional space, it is possible to prevent the object, etc. in the back from being obscured by the two-dimensional structure in the foreground. The two-dimensional structure may also include grid lines drawn at a predetermined interval as shown in FIG. 3(B). By including the grid lines, it is possible to make it easier to understand, for example, the position and shape, etc. of the two-dimensional structure.

In the example of FIG. 3B, the two-dimensional structure P is hidden (not generated) at the intersection between the two-dimensional structure P and the object. This makes it possible, for example, to prevent the user from making an input at the position in the three-dimensional space where the object exists, as described below. This makes it possible to prevent problems such as an awkward feeling caused by the input being made inside the object, or the input content not being displayed because it is blocked by the object despite the input being made. However, this is not limited to this, and the two-dimensional structure P may be displayed (generated) at the intersection between the two-dimensional structure P and the object.

Also, as shown in FIG. 4(A), when the two-dimensional structure P intersects with the object, the two-dimensional structure P may be displayed (generated) so as to bypass the surface of the object. In FIG. 4(A), the two-dimensional structure P intersects with a triangular pyramid-shaped object (center in the figure), and the intersection line between them (dotted line in the figure) is closed, and the height h of the object from the surface of the two-dimensional structure P is equal to or less than a predetermined value, so the two-dimensional structure P is displayed so as to bypass the surface of the object. FIG. 4(B) is a cross-sectional view (schematic diagram) of the triangular pyramid-shaped object when viewed from a direction horizontal to the ground in the three-dimensional space shown in FIG. 4(A), and shows the height h of the object. On the other hand, in FIG. 4(A), the two-dimensional structure P intersects with the two objects on the left and right sides of the figure, but the intersection lines are not closed, so the two-dimensional structure P does not bypass the surface of the object.

The two-dimensional structure generating means 12 may, for example, further move and rotate the two-dimensional structure in response to the user's input. The two-dimensional structure generating means 12 may, for example, rotate the two-dimensional structure after determining a rough three-dimensional spatial position of the two-dimensional structure. The three-dimensional spatial position will be described later. The rotation axis and the rotation center are not particularly limited and may be on the two-dimensional structure, for example. Specifically, the two-dimensional structure generating means 12, for example, first generates a provisional two-dimensional structure at an arbitrary three-dimensional spatial position in the three-dimensional space. Next, the provisional two-dimensional structure is moved in the depth direction in the image of the three-dimensional space in response to the user's input, and a rough three-dimensional spatial position of the provisional two-dimensional structure is determined. The provisional two-dimensional structure is then rotated in response to the user's input (e.g., tilting the provisional two-dimensional structure perpendicular to the ground, rotating it around an axis perpendicular to the ground, etc.), and the three-dimensional spatial position of the provisional two-dimensional structure after the rotation is determined as the three-dimensional spatial position of the two-dimensional structure.

The user input receiving means 13 receives input into the three-dimensional space by a user. The input by the user can be performed by an input unit such as a touch panel, a mouse, a keyboard, or a controller. The input unit may be built into the input receiving and display device 10 (not shown) or may be an externally connected input device.

When inputting into the three-dimensional space, for example, the user can input into the three-dimensional space at the display position of any of the two-dimensional structures in the image of the three-dimensional space while viewing the image of the three-dimensional space.

Specifically, for example, when the input device is a mouse, the user can input a line on the image of the three-dimensional space by operating the mouse to move a cursor displayed on the image of the three-dimensional space. FIG. 3C is a diagram showing an example of input to the three-dimensional space. In FIG. 3C, a line showing a circle or the like is input as input content in an area in the image 100 of the three-dimensional space where a two-dimensional structure P is displayed. Also, when the input device is a keyboard, the input position on the image of the three-dimensional space can be determined and characters can be input by operating the keyboard. FIG. 5A is a diagram showing another example of input to the three-dimensional space. In FIG. 5A, the characters "NG!" and "OK" are input as input content in an area in the image 100 of the three-dimensional space where a two-dimensional structure P is displayed.

When receiving an input to the three-dimensional space, the user input receiving means 13 links the input content to the three-dimensional space to the three-dimensional space position of the two-dimensional structure based on the display position of the two-dimensional structure in the image of the three-dimensional space and the three-dimensional space position of the two-dimensional structure.

The "display position in the three-dimensional space image" is a position displayed in the three-dimensional space image, and for example, as shown in FIG. 6, is a vertical and horizontal position (also called coordinates) in the three-dimensional space image 100. In FIG. 6, the left-right direction is the x direction, the up-down direction is the y direction, and the coordinates of the four corners in the three-dimensional space image 100 are (0,0), (xn,0), (0,yn), and (xn,yn), respectively. The "three-dimensional space position" is a position in the three-dimensional space, and for example, is a position (also called coordinates) in the width direction, depth direction, and height direction in the three-dimensional space. In FIG. 6, for example, the three-dimensional space position (X1, Y1, Z1) corresponds to the display position (x1, y1) in the three-dimensional space image.

7 is a schematic diagram showing the correspondence between the display position of the two-dimensional structure P in the image 100 in the three-dimensional space and the three-dimensional spatial position of the two-dimensional structure P. FIG. 7 is a schematic cross-sectional view of the three-dimensional space cut by a plane that passes through the position of the virtual camera in the three-dimensional space (shown by a picture of a human eye in the figure) and intersects with the image 100 in the three-dimensional space and the two-dimensional structure P. In FIG. 7(A), the display position of the two-dimensional structure P in the image 100 in the three-dimensional space and the three-dimensional spatial position of the two-dimensional structure P have a one-to-one correspondence (for example, the correspondence between "A(x1, y1)" and "A'(X1, Y1, Z1)" in the figure, and the correspondence between "B(x2, y2)" and "B'(X2, Y2, Z2)"). Note that the correspondence is changed as appropriate, for example, in response to the change in the setting information of the virtual camera (for example, the change in position) as described above.

The correspondence may be, for example, a one-to-many correspondence as shown in FIG. 6(B). In this case, one of the multiple three-dimensional spatial positions of the two-dimensional structure P can be made to correspond by the user's input and default settings. In FIG. 6(B), the three-dimensional spatial position of the two-dimensional structure P on the front side (indicated by "A" in the figure) corresponds to the display position of the two-dimensional structure P in the image 100 in the three-dimensional space (indicated by "A" in the figure). That is, when accepting an input to the three-dimensional space, for example, if the three-dimensional spatial positions of the multiple two-dimensional structures correspond to the display positions of the two-dimensional structures in the image in the three-dimensional space, the user input accepting means 13 can select the three-dimensional spatial position of one of the two-dimensional structures and link the input content to the three-dimensional space to the three-dimensional spatial position of the selected two-dimensional structure.

Specifically, the user input acceptance means 13 may be, for example, any one of a first user input acceptance means 131, a second user input acceptance means 132, and a third user input acceptance means 132. Any one of the first user input acceptance means 131, the second user input acceptance means 132, and the third user input acceptance means 132 may be executed, for example, in response to the input of the user.

For example, when receiving an input to the three-dimensional space, the first user input receiving means 131 links the input content to the three-dimensional space to the three-dimensional space position of the two-dimensional structure based on the display position of the two-dimensional structure in the image of the three-dimensional space and the three-dimensional space position of the two-dimensional structure, so as to map the input content to the three-dimensional space.

"Linking the input content into the three-dimensional space to the three-dimensional spatial positions of the two-dimensional structure so as to form a mapping" means, for example, as shown in FIG. 3(C), obtaining the display positions in the three-dimensional image 100 for each of the points that make up the input content (lines showing circles, etc. in the figure), and linking each of the points that make up the input content to the three-dimensional spatial positions of the two-dimensional structure P that correspond to the display positions.

This allows the user to, for example, freely input data into areas of the three-dimensional space other than the area in which the object exists, by using the two-dimensional structure P as a canvas.

The second user input receiving means 132, for example, in receiving an input to the three-dimensional space, further sets a center point for the input content to the three-dimensional space, and links the center point to the three-dimensional space position of the two-dimensional structure based on the display position of the two-dimensional structure in the image of the three-dimensional space and the three-dimensional space position of the two-dimensional structure, and sets the input content to the three-dimensional space so that it is displayed in accordance with any display direction of the image of the three-dimensional space by rotating around the center point.

The center point is not particularly limited and can be any point in the image of the three-dimensional space.

The center point can be, for example, the center of gravity Q of a virtual rectangle that includes the series of input contents (the characters "NG!" in the figure), as shown in FIG. 8. However, without being limited to this, the center point can be, for example, any point (such as one of the four corners of a rectangle) in a virtual figure that includes the series of input contents, or the start point of writing the input contents, or it can be determined by user input. The center point can also be set before the input contents are entered. The series of input contents can be set appropriately by user input, etc.

"Linking the center point to the three-dimensional spatial position of the two-dimensional structure" means, for example, obtaining the display position of the center point in the three-dimensional image, and linking it to the three-dimensional spatial position of the two-dimensional structure that corresponds to the display position.

"Rotating around the center point" is not particularly limited as long as the front orientation of the input content changes around the center point. As mentioned above, the image in the three-dimensional space is a two-dimensional image. The input content can also be considered as a two-dimensional image that exists on the image in the three-dimensional space. For this reason, the "front orientation of the input content" can also be referred to as, for example, the front orientation of a two-dimensional image of the input content.

"The input content into the three-dimensional space is displayed so as to follow any display direction of the image of the three-dimensional space" means, for example, that when the image of the three-dimensional space containing the input content is displayed in a number of arbitrary display directions, the input content is displayed facing forward in each of the display directions.

Specifically, for example, as shown in FIG. 5(A), when the characters "NG!" and "OK" are input as the input content, even if the image of the three-dimensional space including the input content is displayed with the upward direction in FIG. 5(A) as the new display direction as shown in FIG. 5(C), the input content will be displayed facing forward with respect to the new display direction.

As a result, for example, even if the display direction of the image of the three-dimensional space including the input content is changed, the input content is displayed facing forward with respect to the display direction. Therefore, even if the input content is text, the input content can be understood from any of a number of display directions.

The third user input receiving means 132, for example, in receiving an input to the three-dimensional space, further sets a center point for the received input content to the three-dimensional space, links the center point to the three-dimensional space position of the two-dimensional structure based on the three-dimensional space position of the two-dimensional structure, and sets the input content to the three-dimensional space so that it is rotated around the center point and displayed on the surface of the two-dimensional structure.

The central point is, for example, the same as described above. Also, "linking the central point to the three-dimensional spatial position of the two-dimensional structure" and "rotating around the central point" are, for example, the same as described above.

"On the surface of the two-dimensional structure" means, for example, if the surface of the two-dimensional structure is curved, it may be on a plane tangent to the two-dimensional structure, or on the curved surface.

Conventionally, for example, it has been difficult to input data in the three-dimensional space by setting a plane that is not parallel to the virtual plane that contains the image of the three-dimensional space as the front direction. Also, according to the first user input receiving means 131, for example, as described above, the two-dimensional structure can be treated as a canvas and input data can be freely entered. However, if the two-dimensional structure has a large angle with respect to the virtual plane that contains the image of the three-dimensional space, there is a possibility that a series of input data will be elongated and linked to the two-dimensional structure.

According to the third user input receiving means 132, as described above, the center point is linked to the three-dimensional spatial position of the two-dimensional structure, and the input content into the three-dimensional space is set to be displayed on the surface of the two-dimensional structure by rotating around the center point. Therefore, even if the two-dimensional structure extends in the depth direction of the image in the three-dimensional space, for example, the input content can be prevented from being stretched out in the depth direction. Therefore, even if the input content is text, it can be made easy to read.

Then, the three-dimensional space output means 11 outputs an image of the three-dimensional space so that the input content into the three-dimensional space is displayed at the linked three-dimensional space position.

The three-dimensional space output means 11 may or may not output the two-dimensional structure. Figure 5(A) shows an example of the output image of the three-dimensional space in the former case, and Figure 5(B) shows an example of the output image of the three-dimensional space in the latter case.

The three-dimensional space output means 11 may output an image of the three-dimensional space so that the input contents to the three-dimensional space are displayed with a thickness from the two-dimensional structure surface, as shown in Fig. 9, for example. This makes it possible to prevent the display of the input contents from disappearing or becoming less noticeable, even if, for example, the display direction of the image of the three-dimensional space (the direction in which the virtual camera is located) is close to parallel to the two-dimensional structure surface (for example, the direction indicated by the arrow in the figure) when the input contents to the three-dimensional space are received by the first user input receiving means 131 or the third user input receiving means 132.

The input receiving and display device 10 may further include a storage means, which stores the input contents to the three-dimensional space and the associated three-dimensional space position. The storage means may be, for example, a storage device 105. This allows the input contents to the three-dimensional space and the associated three-dimensional space position to be stored.

Next, an example of processing in the input acceptance display method of this embodiment will be described with reference to the block diagram of FIG. 1 and the flowchart of FIG. 10. The input acceptance display method of this embodiment includes a three-dimensional space output process (A1), a two-dimensional structure generation process (A2), a user input acceptance process (A3), and a three-dimensional space output process (A1'), and can be implemented, for example, as follows using the input acceptance display device 10 of FIG. 1. Note that the input acceptance display method of this embodiment is not limited to use of the input acceptance display device 10 of FIG. 1.

First, the three-dimensional space output means 11 outputs an image of the three-dimensional space to be displayed on the image display unit (step (A1)). Figure 3 (A) shows an example of the state in which the three-dimensional space image 100 is output in step (A1) and the three-dimensional space image 100 is displayed.

Next, the two-dimensional structure generating means 12 generates in the three-dimensional space a two-dimensional structure P for receiving input to the three-dimensional space (step (A2)). FIG. 3(B) is a diagram showing an example of a state in which a two-dimensional structure P for receiving input to the three-dimensional space is generated in the three-dimensional space in step (A2), and an image 100 of the three-dimensional space including this two-dimensional structure P is displayed.

Next, the user input receiving means 13 receives input into the three-dimensional space by the user (step (A3)). In the step (A3), based on the display position of the two-dimensional structure P in the image 100 in the three-dimensional space and the three-dimensional spatial position of the two-dimensional structure P, the input content into the three-dimensional space is linked to the three-dimensional spatial position of the two-dimensional structure P.

Next, the three-dimensional space output means 11 outputs an image 100 of the three-dimensional space so that the input content into the three-dimensional space is displayed at the linked three-dimensional space position (step (A1')).

The steps (A3) and (A1') may be carried out, for example, in this order or in parallel.

Figures 3(C) and (D) and Figures 5(A) and (B) are diagrams showing an example of a state in which input to the three-dimensional space by a user is received in steps (A3) and (A1') and an image 100 of the three-dimensional space including the input content is displayed.

As shown in Figures 3(D) and 5(B), the two-dimensional structure P may be erased after receiving the input to the three-dimensional space in step (A3), or after outputting the image 100 of the three-dimensional space in step (A1').

Then, the process ends (END), for example, in response to user input.

The steps (A2) to (A1') can be repeated, for example. This allows multiple inputs to be made by changing the position of the two-dimensional structure P, for example.

That is, after steps (A2) to (A1'), in the second step (A2), the two-dimensional structure generating means 12 generates a new two-dimensional structure P' in the three-dimensional space for receiving input to the three-dimensional space. FIG. 3(E) is a diagram showing an example of a state in which a new two-dimensional structure P' for receiving input to the three-dimensional space is generated in the three-dimensional space in the second step (A2), and an image 100 of the three-dimensional space including this new two-dimensional structure P' is displayed.

Next, after the second step (A2), in the second step (A3), the user input receiving means 13 receives new input from the user into the three-dimensional space. Figure 3 (F) is a diagram showing an example of a state in which new input from the user into the three-dimensional space (indicated by a thick arrow in the figure) is received in the second step (A3) and an image 100 of the three-dimensional space including the input content is displayed.

Then, after the second step (A3), in the second step (A1'), the three-dimensional space output means 11 outputs an image 100 of the three-dimensional space so that the new input content to the three-dimensional space is displayed at the linked three-dimensional space position.

[Embodiment 2]
The program of this embodiment is a program capable of executing the above-mentioned input reception and display method on a computer. The program of this embodiment may be recorded, for example, in a computer-readable recording medium. The recording medium is not particularly limited, and examples thereof include a random access memory (RAM), a read-only memory (ROM), a hard disk (HD), an optical disk, and the like.

The present invention has been described above with reference to the embodiments, but the present invention is not limited to the above embodiments. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

Some or all of the above embodiments may be described as follows, but are not limited to the following:

(Appendix 1)
The method includes a three-dimensional space output step, a two-dimensional structure generation step, and a user input reception step,
The three-dimensional space output step outputs an image of the three-dimensional space to be displayed on an image display unit,
the two-dimensional structure generating step generates, in the three-dimensional space, a two-dimensional structure for receiving an input to the three-dimensional space;
The user input receiving step receives an input into the three-dimensional space by a user,
when receiving an input to the three-dimensional space, the input content to the three-dimensional space is linked to the three-dimensional space position of the two-dimensional structure based on a display position of the two-dimensional structure in an image of the three-dimensional space and a three-dimensional space position of the two-dimensional structure;
the three-dimensional space output step outputs an image of the three-dimensional space such that the input content into the three-dimensional space is displayed at the associated three-dimensional space position;
Each of the steps is performed by a computer.
A method for accepting and displaying input in three-dimensional space.
(Appendix 2)
the user input receiving step includes at least one selected from the group consisting of a first user input receiving step, a second user input receiving step, and a third user input receiving step;
the first user input receiving step includes, in receiving an input to the three-dimensional space, linking the input content to the three-dimensional space to the three-dimensional space position of the two-dimensional structure so as to form a mapping based on a display position of the two-dimensional structure in an image of the three-dimensional space and a three-dimensional space position of the two-dimensional structure;
The second user input receiving step further includes, in receiving an input to the three-dimensional space, setting a center point for the input content to the three-dimensional space, and linking the center point to the three-dimensional space position of the two-dimensional structure based on a display position of the two-dimensional structure in the image in the three-dimensional space and a three-dimensional space position of the two-dimensional structure, and setting the input content to the three-dimensional space to be displayed so as to follow an arbitrary display direction of the image in the three-dimensional space by rotating around the center point;
The third user input receiving step further includes, in receiving an input to the three-dimensional space, setting a center point for the received input content to the three-dimensional space, linking the center point to the three-dimensional space position of the two-dimensional structure based on the three-dimensional space position of the two-dimensional structure, and setting the input content to the three-dimensional space so as to be rotated around the center point and displayed on a surface of the two-dimensional structure.
The method described in Appendix 1.
(Appendix 3)
The method according to claim 2, wherein the user input receiving step executes any one of the first user input receiving step, the second user input receiving step, and the third user input receiving step in response to the input of the user.
(Appendix 4)
4. The method according to claim 1, wherein the two-dimensional structure generating step further comprises at least one of moving and rotating the two-dimensional structure in response to an input from the user.
(Appendix 5)
5. The method of any one of claims 1 to 4, wherein the two-dimensional structure is a plane.
(Appendix 6)
6. The method of any one of claims 1 to 5, wherein the two-dimensional structure is transparent.
(Appendix 7)
A method according to any one of appendices 1 to 6, wherein the user input receiving step, when receiving input into the three-dimensional space, selects the three-dimensional space position of one of the two-dimensional structures when the three-dimensional space positions of a plurality of the two-dimensional structures correspond to the display position of the two-dimensional structure in the image in the three-dimensional space, and links the input content into the three-dimensional space to the three-dimensional space position of the selected two-dimensional structure.
(Appendix 8)
8. The method according to claim 1, wherein the two-dimensional structure generating step generates the two-dimensional structure in a manner that detouring a surface of an object contained in the three-dimensional space when the two-dimensional structure intersects with the object.
(Appendix 9)
The method of claim 8, wherein the two-dimensional structure generation process generates the two-dimensional structure by bypassing the surface of the object when the two-dimensional structure intersects with an object contained in the three-dimensional space, the intersection line between the two-dimensional structure and the object is closed, and the height of the object from the surface of the two-dimensional structure is equal to or less than a predetermined value.
(Appendix 10)
A method according to any one of appendix 1 to 9, wherein the three-dimensional space output step outputs an image of the three-dimensional space so that input content into the three-dimensional space is displayed having a thickness from the two-dimensional structure surface.
(Appendix 11)
Further comprising a storing step,
11. The method according to any one of claims 1 to 10, wherein the storing step stores the input content into the three-dimensional space and the associated three-dimensional spatial position.
(Appendix 12)
A program capable of executing the method according to any one of claims 1 to 11 on a computer.
(Appendix 13)
The apparatus includes a three-dimensional space output means, a two-dimensional structure generating means, and a user input receiving means,
the three-dimensional space output means outputs an image of the three-dimensional space to be displayed on the image display unit;
the two-dimensional structure generating means generates, in the three-dimensional space, a two-dimensional structure for receiving an input to the three-dimensional space;
the user input receiving means receives an input into the three-dimensional space by a user;
when receiving an input to the three-dimensional space, the input content to the three-dimensional space is linked to the three-dimensional space position of the two-dimensional structure based on a display position of the two-dimensional structure in an image of the three-dimensional space and a three-dimensional space position of the two-dimensional structure;
the three-dimensional space output means outputs an image of the three-dimensional space such that the input content into the three-dimensional space is displayed at the associated three-dimensional space position.
An input receiving and display device in three-dimensional space.
(Appendix 14)
the user input receiving means includes at least one selected from the group consisting of a first user input receiving means, a second user input receiving means, and a third user input receiving means;
the first user input receiving means, in receiving an input into the three-dimensional space, links the input content into the three-dimensional space to the three-dimensional space position of the two-dimensional structure so as to form a mapping based on a display position of the two-dimensional structure in an image of the three-dimensional space and a three-dimensional space position of the two-dimensional structure;
the second user input accepting means, in accepting an input into the three-dimensional space, further sets a center point for the input content into the three-dimensional space, and links the center point to the three-dimensional space position of the two-dimensional structure based on a display position of the two-dimensional structure in the image in the three-dimensional space and a three-dimensional space position of the two-dimensional structure, and sets the input content into the three-dimensional space to be displayed so as to follow an arbitrary display direction of the image in the three-dimensional space by rotating around the center point;
the third user input receiving means, in receiving an input to the three-dimensional space, further sets a center point for the received input content to the three-dimensional space, links the center point to the three-dimensional space position of the two-dimensional structure based on the three-dimensional space position of the two-dimensional structure, and sets the input content to the three-dimensional space so as to be rotated around the center point and displayed on a surface of the two-dimensional structure.
14. The apparatus of claim 13.
(Appendix 15)
15. The device according to claim 14, wherein the user input accepting means executes any one of the first user input accepting means, the second user input accepting means, and the third user input accepting means in response to an input from the user.
(Appendix 16)
16. The apparatus of claim 13, wherein the two-dimensional structure generating means further moves and/or rotates the two-dimensional structure in response to an input from the user.
(Appendix 17)
17. The apparatus of any one of claims 13 to 16, wherein the two-dimensional structure is planar.
(Appendix 18)
18. The device of any one of claims 13 to 17, wherein the two-dimensional structure is transparent.
(Appendix 19)
The device described in any of Appendix 13 to 18, wherein the user input accepting means, when accepting input into the three-dimensional space, selects the three-dimensional space position of one of the two-dimensional structures when the three-dimensional space positions of a plurality of the two-dimensional structures correspond to the display position of the two-dimensional structure in the image in the three-dimensional space, and links the input content into the three-dimensional space to the three-dimensional space position of the selected two-dimensional structure.
(Appendix 20)
20. The device according to any one of appendices 13 to 19, wherein the two-dimensional structure generation means generates the two-dimensional structure in a manner that detouring a surface of an object when the two-dimensional structure intersects with an object included in the three-dimensional space.
(Appendix 21)
The device described in Appendix 20, wherein the two-dimensional structure generation means generates the two-dimensional structure by bypassing the surface of the object when the two-dimensional structure intersects with an object contained in the three-dimensional space, the intersection line between the two-dimensional structure and the object is closed, and the height of the object from the two-dimensional structure surface is equal to or less than a predetermined value.
(Appendix 22)
22. The device described in any one of appendixes 13 to 21, wherein the three-dimensional space output means outputs an image of the three-dimensional space so that input content into the three-dimensional space is displayed with a thickness from the two-dimensional structure surface.
(Appendix 23)
Further comprising a storage means,
23. The device according to any one of appendices 13 to 22, wherein the storage means stores the input content into the three-dimensional space and the associated three-dimensional spatial position.

The present invention provides a system that allows input reception and display in three-dimensional space.

10 Input reception display device 11 Three-dimensional space output means 12 Two-dimensional structure generation means 13 User input reception means

Claims (9)

The method includes a three-dimensional space output step, a two-dimensional structure generation step, and a user input reception step,
The three-dimensional space output step outputs an image of the three-dimensional space to be displayed on an image display unit,
the two-dimensional structure generating step generates, in the three-dimensional space, a two-dimensional structure for receiving an input to the three-dimensional space;
The user input receiving step receives an input into the three-dimensional space by a user,
when receiving an input into the three-dimensional space, the input content into the three-dimensional space is linked to the three-dimensional spatial position of the two-dimensional structure based on the display position of the two-dimensional structure in the image of the three-dimensional space and the three-dimensional spatial position of the two-dimensional structure, and further, a rotation center point is set for the input content into the three-dimensional space, and the rotation center point is linked to the three-dimensional spatial position of the two-dimensional structure based on the display position of the two-dimensional structure in the image of the three-dimensional space and the three-dimensional spatial position of the two-dimensional structure, and the input content into the three-dimensional space is set so as to be displayed in accordance with an arbitrary display direction of the image of the three-dimensional space by rotating around the rotation center point;
the three-dimensional space output step outputs an image of the three-dimensional space such that the input content into the three-dimensional space is displayed at the associated three-dimensional space position;
Each of the steps is performed by a computer.
A method for accepting and displaying input in three-dimensional space. The method includes a three-dimensional space output step, a two-dimensional structure generation step, and a user input reception step,
The three-dimensional space output step outputs an image of the three-dimensional space to be displayed on an image display unit,
the two-dimensional structure generating step generates, in the three-dimensional space, a two-dimensional structure for receiving an input to the three-dimensional space;
The user input receiving step receives an input into the three-dimensional space by a user,
when receiving an input into the three-dimensional space, the input content into the three-dimensional space is linked to the three-dimensional spatial position of the two-dimensional structure based on a display position of the two-dimensional structure in an image of the three-dimensional space and the three-dimensional spatial position of the two-dimensional structure, further, a rotation center point is set for the received input content into the three-dimensional space, and the rotation center point is linked to the three-dimensional spatial position of the two-dimensional structure based on the three-dimensional spatial position of the two-dimensional structure, and the input content into the three-dimensional space is set so as to be rotated around the rotation center point and displayed on the surface of the two-dimensional structure;
the three-dimensional space output step outputs an image of the three-dimensional space such that the input content into the three-dimensional space is displayed at the associated three-dimensional space position;
Each of the steps is performed by a computer.
A method for accepting and displaying input in three-dimensional space. The method includes a three-dimensional space output step, a two-dimensional structure generation step, and a user input reception step,
The three-dimensional space output step outputs an image of the three-dimensional space to be displayed on an image display unit,
the two-dimensional structure generating step generates, in the three-dimensional space, a two-dimensional structure for receiving an input to the three-dimensional space;
The user input receiving step receives an input into the three-dimensional space by a user,
when receiving an input into the three-dimensional space, the input content into the three-dimensional space is linked to the three-dimensional spatial position of the two-dimensional structure based on the display position of the two-dimensional structure in the image of the three-dimensional space and the three-dimensional spatial position of the two-dimensional structure, and further, when the three-dimensional spatial positions of a plurality of the two-dimensional structures correspond to the display positions of the two-dimensional structures in the image of the three-dimensional space, the three-dimensional spatial position of any one of the two-dimensional structures is selected in response to an input by the user, and the input content into the three-dimensional space is linked to the three-dimensional spatial position of the selected two-dimensional structure;
the three-dimensional space output step outputs an image of the three-dimensional space such that the input content into the three-dimensional space is displayed at the associated three-dimensional space position;
Each of the steps is performed by a computer.
A method for accepting and displaying input in three-dimensional space. The method includes a three-dimensional space output step, a two-dimensional structure generation step, and a user input reception step,
The three-dimensional space output step outputs an image of the three-dimensional space to be displayed on an image display unit,
The two-dimensional structure generating step generates, in the three-dimensional space, a two-dimensional structure for receiving an input to the three-dimensional space;
generating the two-dimensional structure such that, when the two-dimensional structure intersects with an object included in the three-dimensional space, the two-dimensional structure is generated in a manner that bypasses a surface of the object or in a manner that the two-dimensional structure is hidden at a portion where the two-dimensional structure intersects with the object;
The user input receiving step receives an input by a user into the three-dimensional space,
when receiving an input to the three-dimensional space, the input content to the three-dimensional space is linked to the three-dimensional space position of the two-dimensional structure based on a display position of the two-dimensional structure in an image of the three-dimensional space and a three-dimensional space position of the two-dimensional structure;
the three-dimensional space output step outputs an image of the three-dimensional space such that the input content into the three-dimensional space is displayed at the associated three-dimensional space position;
Each of the steps is performed by a computer.
A method for accepting and displaying input in three-dimensional space. A program capable of executing the method according to any one of claims 1 to 4 on a computer. The apparatus includes a three-dimensional space output means, a two-dimensional structure generating means, and a user input receiving means,
the three-dimensional space output means outputs an image of the three-dimensional space to be displayed on the image display unit;
the two-dimensional structure generating means generates, in the three-dimensional space, a two-dimensional structure for receiving an input to the three-dimensional space;
the user input receiving means receives an input into the three-dimensional space by a user;
when receiving an input into the three-dimensional space, the input content into the three-dimensional space is linked to the three-dimensional spatial position of the two-dimensional structure based on the display position of the two-dimensional structure in the image of the three-dimensional space and the three-dimensional spatial position of the two-dimensional structure, and further, a rotation center point is set for the input content into the three-dimensional space, and the rotation center point is linked to the three-dimensional spatial position of the two-dimensional structure based on the display position of the two-dimensional structure in the image of the three-dimensional space and the three-dimensional spatial position of the two-dimensional structure, and the input content into the three-dimensional space is set so as to be displayed in accordance with an arbitrary display direction of the image of the three-dimensional space by rotating around the rotation center point;
the three-dimensional space output means outputs an image of the three-dimensional space such that the input content into the three-dimensional space is displayed at the associated three-dimensional space position.
An input receiving and display device in three-dimensional space. The apparatus includes a three-dimensional space output means, a two-dimensional structure generating means, and a user input receiving means,
the three-dimensional space output means outputs an image of the three-dimensional space to be displayed on the image display unit;
the two-dimensional structure generating means generates, in the three-dimensional space, a two-dimensional structure for receiving an input to the three-dimensional space;
the user input receiving means receives an input into the three-dimensional space by a user;
when receiving an input into the three-dimensional space, the input content into the three-dimensional space is linked to the three-dimensional spatial position of the two-dimensional structure based on a display position of the two-dimensional structure in an image of the three-dimensional space and the three-dimensional spatial position of the two-dimensional structure, further, a rotation center point is set for the received input content into the three-dimensional space, and the rotation center point is linked to the three-dimensional spatial position of the two-dimensional structure based on the three-dimensional spatial position of the two-dimensional structure, and the input content into the three-dimensional space is set so as to be rotated around the rotation center point and displayed on the surface of the two-dimensional structure;
the three-dimensional space output means outputs an image of the three-dimensional space such that the input content into the three-dimensional space is displayed at the associated three-dimensional space position.
An input receiving and display device in three-dimensional space. The apparatus includes a three-dimensional space output means, a two-dimensional structure generating means, and a user input receiving means,
the three-dimensional space output means outputs an image of the three-dimensional space to be displayed on the image display unit;
the two-dimensional structure generating means generates, in the three-dimensional space, a two-dimensional structure for receiving an input to the three-dimensional space;
the user input receiving means receives an input into the three-dimensional space by a user;
when receiving an input into the three-dimensional space, the input content into the three-dimensional space is linked to the three-dimensional spatial position of the two-dimensional structure based on the display position of the two-dimensional structure in the image of the three-dimensional space and the three-dimensional spatial position of the two-dimensional structure, and further, when the three-dimensional spatial positions of a plurality of the two-dimensional structures correspond to the display positions of the two-dimensional structures in the image of the three-dimensional space, the three-dimensional spatial position of any one of the two-dimensional structures is selected in response to an input by the user, and the input content into the three-dimensional space is linked to the three-dimensional spatial position of the selected two-dimensional structure;
the three-dimensional space output means outputs an image of the three-dimensional space such that the input content into the three-dimensional space is displayed at the associated three-dimensional space position.
An input receiving and display device in three-dimensional space. The apparatus includes a three-dimensional space output means, a two-dimensional structure generating means, and a user input receiving means,
the three-dimensional space output means outputs an image of the three-dimensional space to be displayed on the image display unit;
the two-dimensional structure generating means generates, in the three-dimensional space, a two-dimensional structure for receiving an input to the three-dimensional space;
generating the two-dimensional structure such that, when the two-dimensional structure intersects with an object included in the three-dimensional space, the two-dimensional structure is generated in a manner that bypasses a surface of the object or in a manner that the two-dimensional structure is hidden at a portion where the two-dimensional structure intersects with the object;
the user input receiving means receives an input into the three-dimensional space by a user;
in receiving an input to the three-dimensional space, linking the input content to the three-dimensional space to the three-dimensional space position of the two-dimensional structure based on a display position of the two-dimensional structure in an image of the three-dimensional space and a three-dimensional space position of the two-dimensional structure;
the three-dimensional space output means outputs an image of the three-dimensional space such that the input content into the three-dimensional space is displayed at the associated three-dimensional space position.
An input receiving and display device in three-dimensional space.

Images