WO2017195299A1

WO2017195299A1 - Simulation system

Info

Publication number: WO2017195299A1
Application number: PCT/JP2016/064021
Authority: WO
Inventors: 幸宏陽奥; 鈴木　達也; 遠藤　康浩
Original assignee: 富士通株式会社
Priority date: 2016-05-11
Filing date: 2016-05-11
Publication date: 2017-11-16

Abstract

Provided is a simulation system that is easy to use. The simulation system comprises: a display unit for displaying a pointer having a position that can be manipulated by a user and an image of an article based on article data representing the shape and the position of the article; a data storage unit for storing the article data; a first detection unit that detects an indication operation in which the user indicates the position of the pointer; a second detection unit that detects the position of the pointer in the coordinate system of the display unit on the basis of the indication operation detected by the first detection unit; an area generation unit for generating a detection area that includes the pointer and that is larger than the pointer; a size setting unit that sets the size of the pointer in accordance with the type of an article at least partially positioned within the detection area; and an output unit that causes the display unit to display the pointer at the size set by the size setting unit.

Description

Simulation system

The present invention relates to a simulation system.

Conventionally, there is an information input device for a user to control a graphic cursor displayed on a display. A first sensor that generates first sensor data in response to a first type of user motion that is a motion of a part of the user's body, and a motion of a part of the user's body that is finer than the first type of user motion And a second sensor that generates second sensor data in response to a second type of user action.

The graphic having a wide-range movement component corresponding to the first type of user action and a high-precision range movement component corresponding to the second type of user action and representing a higher-precision movement than the wide-range movement component. The apparatus further includes at least one processor that calculates a hybrid cursor movement signal that is a signal for moving the cursor.

The at least one processor is based on a first sensitivity parameter representing a sensitivity of the first sensor with respect to the first type of user action, which is determined by adding the second sensor data to the first sensor data. A wide range moving component is calculated.

The at least one processor is based on a second sensitivity parameter representing a sensitivity of the second sensor with respect to the second type of user action, which is determined by adding the first sensor data to the second sensor data. A high-accuracy range moving component is calculated (see, for example, Patent Document 1).

The at least one processor sets the first sensitivity parameter to be smaller as the second type user action is more intense when both the first type user action and the second type user action are executed. The wide-range moving component is suppressed, and the second sensitivity parameter is set to be smaller as the first type user operation is more intense, thereby suppressing the high-accuracy range moving component.

The graphic cursor includes at least a first cursor and a second cursor located in the first cursor and smaller than the first cursor, and the at least one processor further has a large first sensitivity parameter. The smaller the second sensitivity parameter is, the smaller the size of the first cursor is set.

JP 2014-528625 A

By the way, the conventional information input device operates the two cursors of the first cursor and the second cursor, and thus there is a problem that the usability is not good.

Therefore, an object is to provide a simulation system that is easy to use.

A simulation system according to an embodiment of the present invention stores a display unit that displays an image of the article based on article data representing the shape and position of the article, a pointer whose position is manipulated by a user, and the article data. A data storage unit, a first detection unit that detects an instruction operation in which the user indicates the position of the pointer, and a coordinate system of the display unit based on the instruction operation detected by the first detection unit. According to a second detection unit that detects the position of the pointer, an area generation unit that includes the pointer and generates a detection area that is larger than the pointer, and the type of the article that is at least partially located within the detection area A size setting unit for setting the size of the pointer, and an output for causing the display unit to display the pointer having a size set by the size setting unit. And a part.

A simulation system with good usability can be provided.

It is a figure which shows the simulation system of embodiment. It is a figure which shows the structure of the processing apparatus of a simulation system. 1 is a perspective view of a computer system to which a processing apparatus according to an embodiment is applied. It is a block diagram explaining the structure of the principal part in the main-body part of a computer system. It is a figure which shows shape data. It is a figure which shows an example of the image of articles | goods. It is a figure which shows size data. It is a figure which shows the method of determining the size of a pointer with the size of articles | goods. It is a figure which shows the method of determining the size of a pointer with the size of articles | goods. It is a figure which shows the method of determining the size of a pointer with the size of articles | goods. It is a figure which shows the method of determining the size of a pointer with the size of articles | goods. It is a figure which shows the method of determining the size of a pointer with the size of articles | goods. It is a figure which shows the method of determining the size of a pointer with the size of articles | goods. It is a figure which shows the method of determining the size of a pointer with the size of articles | goods. It is a figure which shows the method of determining the size of a pointer with the size of articles | goods. It is a figure which shows the method of determining the size of a pointer with the size of articles | goods. It is a figure which shows the size data which linked | related article ID and pointer size. It is a figure which shows the relationship between an article | item and the size of a pointer. It is a figure which shows the relationship between an article | item and the size of a pointer. It is a flowchart which shows the process which the processing apparatus 120 of embodiment performs. It is a figure which shows the relationship between an article | item and the size of a pointer. It is a figure which shows the size data which linked | related article ID and pointer size. It is a flowchart which shows the process which the processing apparatus of embodiment performs. It is a figure which shows the display of a screen at the time of actually performing instruction | indication operation | movement with a simulation system. FIG. 25 is a diagram illustrating a locus of a pointer displayed by the simulation system when the instruction operation described in FIG. 24 is performed. It is a figure which shows the result of having performed the instruction | indication operation | movement which moves a pointer to a marker.

Hereinafter, embodiments to which the simulation system of the present invention is applied will be described.

<Embodiment>
FIG. 1 is a diagram illustrating a simulation system 100 according to an embodiment. FIG. 2 is a diagram illustrating a configuration of the processing device 120 of the simulation system 100.

The simulation system 100 includes a screen 110A, a projection device 110B, 3D (3-dimensional) glasses 110C, a processing device 120, and a position measurement device 140.

The simulation system 100 according to the embodiment can be applied to an assembly support system in order to grasp assembly workability in a virtual space, for example. In the assembly support system, for example, an operation of assembling an electronic component such as a CPU (Central Processing Unit) module, a memory module, a communication module, or a connector on a mother board or the like can be performed in a virtual space.

However, the simulation system 100 according to the embodiment can be applied not only to the assembly support system but also to various systems for confirming workability in a three-dimensional space.

As the screen 110A, for example, a projector screen can be used. The size of the screen 110A may be set as appropriate according to the application. An image projected by the projection device 110B is displayed on the screen 110A. Here, it is assumed that images of the article 111, the

buttons

111A and 111B, and the pointer 130A are displayed on the screen 110A.

In the simulation system 100, as an example, the pointer 130A is displayed in the direction in which the user 1 points the hand toward the screen 110A. The user 1 may be in a state where the fingertip of the right hand is opened or held. The pointer 130A is displayed on the screen 110A in the direction in which the user 1 moves the right arm and points with the hand. An operation in which the user 1 instructs with the right arm to move the pointer 130A is referred to as an instruction operation.

The projection device 110B may be any device that can project an image on the screen 110A. For example, a projector can be used. The projection device 110B is connected to the processing device 120 via a cable 110B1, and projects an image input from the processing device 120 onto the screen 110A. Here, the projection device 110B is of a type that can project a 3D image (stereoscopic image) onto the screen 110A.

The screen 110A and the projection device 110B are examples of a display unit.

The user 1 who uses the simulation system 100 wears the 3D glasses 110C. The 3D glasses 110C may be any glasses that can convert an image projected on the screen 110A by the projection device 110B into a 3D image. For example, polarized glasses for polarizing incident light or liquid crystal shutter glasses having a liquid crystal shutter are used. Can do.

For example, a liquid crystal display panel may be used instead of the screen 110A and the projection device 110B. Further, when the 3D glasses 110C are unnecessary, the 3D glasses 110C may not be used. Further, instead of the screen 110A and the projection device 110B, a head mounted display capable of viewing a 3D image may be used.

The processing device 120 includes a human body detection unit 121, a position detection unit 122, a detection region generation unit 123, an operation detection unit 124, an object determination unit 125, a pointer generation unit 126, a data holding unit 127, and a video output unit 128. The processing device 120 is realized by a computer having a memory, for example.

The human body detection unit 121 determines whether or not the body of the user 1 exists based on data that three-dimensionally represents the position and shape of the body of the user 1 input from the position measurement device 140. When doing so, the coordinates indicating the position of each part of the body of the user 1 are obtained. As an example, the position of each part of the body of the user 1 is represented by the position of the skeleton of the user 1. The position of the skeleton includes, for example, the position of the head, shoulder, elbow, wrist, and hand. The human body detection unit 121 is an example of a second detection unit together with the position detection unit 122.

The position detection unit 122 obtains coordinates P (P _x , P _y , P _z ) based on the coordinates representing the position of each part of the body of the user 1 input from the human body detection unit 121. The position detection unit 122 is an example of a second detection unit together with the human body detection unit 121.

The position detection unit 122 obtains a straight line connecting the right shoulder and the right wrist of the user 1 based on the coordinates representing the position of each part of the body of the user 1 input from the human body detection unit 121, and the straight line and the screen 110A. Find the coordinates of the intersection with.

The position detection unit 122 converts the coordinate value of the intersection of the straight line and the screen 110A into the coordinate in the image projected on the screen 110A, and outputs it as coordinates P (P _x , P _y , P _z ). Note that the position measurement device 140 may detect the coordinates P (P _x , P _y , P _z ).

Here, an X axis is defined in the horizontal direction parallel to the screen 110A, a Y axis is defined in the vertical direction, and a Z axis is defined in the horizontal direction perpendicular to the screen 110A. Here, if the coordinates of the right shoulder of the user 1 are S (S _x , S _y , S _z ) and the coordinates of the right wrist are H (H _x , H _y , H _z ), the right shoulder and the right wrist are A vector SH in the connecting direction is expressed by the following equation (1). Vector SH represents a direction in which user 1 indicates the position of pointer 130A with the right hand.

The magnitude of the vector SH is given by the following equation (2) using the right shoulder coordinates S (S _x , S _y , S _z ) and the right wrist coordinates H (H _x , H _y , H _z ). expressed.

A coordinate P1 (P1 _x , P1 _y , P1 _z ) represented by the following expression (3) is obtained by setting ΔL as an amount (offset amount) offset by the user 1 from the screen 10A in the Z-axis direction.

The coordinates P1 (P1 _x , P1 _y , P1 _z ) are coordinates obtained by the equations (1), (2), and (3), and the straight line connecting the right shoulder and the right wrist of the user 1 and the screen 110A. The coordinates of the intersection. The coordinates of this intersection are the coordinates in the real space.

The position detection unit 122 converts the coordinates P1 (P1 _x , P1 _y , P1 _z ) of the intersection into coordinates in the image projected on the screen 110A, and outputs the coordinates P (P _x , P _y , P _z ). To do. Coordinates _{_{P (P x, P y,}} P z) are coordinates obtained by converting the coordinates in the image to be projected intersection coordinates _P1 and (P1 _x, P1 y, P1 _z) of the screen 110A.

As described above, the position detector 122 converts the coordinates P (P _x , P _y , P) obtained by converting the coordinates P1 (P1 _x , P1 _y , P1 _z ) of the intersection in the real space into values in the coordinate system in the virtual space. _z ).

Detection area generator 123 generates a detection region 130B to the center coordinates _{_{P (P x, P y,}} P z) a. The detection area generation unit 123 is an example of an area generation unit. The pointer 130A is a sphere centered on the coordinates P (P _x , P _y , P _z ), and the radius of the detection area 130B is larger than the radius of the pointer 130A. For this reason, the detection region 130B is arranged concentrically with the pointer 130A.

The detection area 130B is an area outside the surface of the pointer 130A and included in a sphere defined by a predetermined radius with the coordinates P (P _x , P _y , P _z ) as the center. The detection region 130B includes a spherical surface defined by a predetermined radius.

The pointer 130A and the detection area 130B are the same in that they move according to the user's instruction operation. The pointer 130A is displayed as an image on the screen 110A, but the detection area 130B is not displayed on the screen 110A.

The pointer 130A is used for determining contact with the article 111 or the

button

111A or 111B displayed on the screen 110A, whereas the detection area 130B is the article 111 or the

button

111A or 111B existing around the pointer 130A. Used to determine the presence of.

The motion detection unit 124 detects the motion of the user 1 based on the coordinates representing the position of each part of the body of the user 1 input from the human body detection unit 121. The operation of the user 1 is an operation such as a gesture performed by the user 1, and here, as an example, an operation for resetting the size of the pointer 130A, an operation for determining an input to the

button

111A or 111B, and There is an operation of canceling the input to the

button

111A or 111B.

The operation for determining input to the

buttons

111A and 111B is an operation of tapping the

buttons

111A and 111B, respectively. Further, the operation of canceling the input to the button 111A is an operation of paying while tracing the button 111A. Here, since the user 1 operates using the right hand 2, more specifically, the operation of canceling the input to the button 111A is performed by moving the pointer 130A leftward while the pointer 130A touches the button 111A. It is an operation to move to.

Further, the operation of canceling the input to the button 111B is an operation of paying while tracing the button 111B. Here, since the user 1 operates using the right hand 2, more specifically, the operation of canceling the input to the button 111B is performed by moving the pointer 130A in the right direction while the pointer 130A touches the button 111B. It is an operation to move to.

Further, the operation for resetting the size of the pointer 130A is an operation of crossing both arms. The motion detection unit 124 is an example of a first determination unit and a second determination unit.

The object determination unit 125 determines the type of article at least a part of which is located inside the detection area 130B. The type of article is, for example, the kind of article 111 or

button

111A or 111B.

Whether or not at least a part is located inside the detection area 130B may be determined by whether or not the detection area 130B and the display area of the article 111 or the

button

111A or 111B have an intersection. When at least a part of the article is located inside the detection area 130B, the case where the article is located on the outer peripheral surface (boundary) of the detection area 130B is also included.

The pointer generator 126 generates the pointer 130A as a spherical image centered on the coordinates P (P _x , P _y , P _z ). The radius of the pointer 130A is smaller than the radius of the detection area 130B. Since the pointer 130A and the detection area 130B are arranged concentrically, the detection area 130B exists around the pointer 130A. The pointer generation unit 126 is an example of a size setting unit.

The pointer 130A moves according to the user's instruction operation, and is used to determine contact with the article 111 or the

button

111A or 111B displayed on the screen 110A.

Whether the pointer 130A and the article 111 or the

button

111A or 111B are in contact with each other is determined based on the coordinates P (P _x , P _y , P _z ) and within the pointer 130A having a predetermined radius. The determination may be made based on whether at least a part of the display area of the

button

111A or 111B is included. The interior of the pointer 130A includes the surface of the pointer 130A.

Whether at least a part of the display area of the article 111 or the

button

111A or 111B is included in the pointer 130A depends on whether or not the pointer 130A and the display area of the article 111 or the

button

111A or 111B have an intersection. What is necessary is just to judge.

When at least a part of the display area of the article 111 or the

button

111A or 111B is included in the pointer 130A, the display area of the article 111 or the

button

111A or 111B is located on the surface (boundary) of the pointer 130A Is also included.

In this embodiment, it is not described in detail whether the pointer 130A is in contact with the article 111 or the

button

111A or 111B. For example, the pointer 130A is in contact with the article 111 or the

button

111A or 111B. In this case, it is only necessary to transmit the contact to the user 1 by changing the color of the pointer 130A.

The size of the pointer 130A generated by the pointer generator 126 is set based on data representing the size of the pointer 130A held in the data holding unit 127. In addition, the pointer generation unit 126 sets the size of the pointer 130A according to the type of article existing inside the detection area 130B. In addition, the pointer generation unit 126 sets the size of the pointer 130 </ b> A according to the operation of the user 1. A specific method for setting the size of the pointer 130A will be described later.

The data holding unit 127 holds data such as article data representing the coordinates and shape of the article 111 or the

button

111A or 111B, image data of the pointer 130A, and data representing the size of the pointer 130A. The data holding unit 127 is realized by a memory and is an example of a data storage unit.

The output terminal of the video output unit 128 is connected to the projection device 110B by a cable 110B1. The video output unit 128 outputs an image specified by the article data of the article 111 held in the data holding unit 127 to the projection device 110B and displays it on the screen 110A.

Also, the video output unit 128 displays the pointer 130A on the projection device 110B. The image of the pointer 130A is generated by the pointer generator 126.

The position measuring device 140 is a device that acquires data that three-dimensionally represents the position and shape of the body of the user 1. The position measuring device 140 is installed above the screen 110A and has a detection range 142 in front of the screen 110A. The detection range 142 extends from the camera 140A of the position measurement device 140 to the front of the screen 110A. The position measuring device 140 is connected to the processing device 120 by a cable 141.

More specifically, the position measuring device 140 calculates a distance (depth) to a point included in the image based on, for example, the time from irradiating the subject with an infrared laser and receiving the reflected light. Device. The position measurement device 140 acquires an image of the user 1 who performs an instruction operation toward the screen 110A, and acquires three-dimensional distance image data representing the posture of the user 1, a gesture, and the like. The position measurement device 140 transmits the acquired three-dimensional data to the processing device 120 via the cable 141.

FIG. 3 is a perspective view of a computer system to which the processing device 120 of the embodiment is applied. A computer system 10 shown in FIG. 3 includes a main body 11, a display 12, a keyboard 13, a mouse 14, and a modem 15.

The main unit 11 includes a CPU (Central Processing Unit), an HDD (Hard Disk Drive), a disk drive, and the like. The display 12 displays an analysis result or the like on the screen 12A according to an instruction from the main body 11. The display 12 may be a liquid crystal monitor, for example. The keyboard 13 is an input unit for inputting various information to the computer system 10. The mouse 14 is an input unit that designates an arbitrary position on the screen 12 </ b> A of the display 12. The modem 15 accesses an external database or the like and downloads a program or the like stored in another computer system.

A program for causing the computer system 10 to function as the processing device 120 is stored in a portable recording medium such as the disk 17 or downloaded from the recording medium 16 of another computer system using a communication device such as the modem 15. Are input to the computer system 10 and compiled.

A program that causes the computer system 10 to have a function as the processing device 120 causes the computer system 10 to operate as the processing device 120. This program may be stored in a computer-readable recording medium such as the disk 17. The computer-readable recording medium is limited to a portable recording medium such as a disk 17, an IC card memory, a magnetic disk such as a floppy (registered trademark) disk, a magneto-optical disk, a CD-ROM, or a USB (Universal Serial Bus) memory. It is not something. The computer-readable recording medium includes various recording media accessible by a computer system connected via a communication device such as a modem 15 or a LAN.

FIG. 4 is a block diagram illustrating a configuration of a main part in the main body 11 of the computer system 10. The main body 11 includes a CPU 21 connected by a bus 20, a memory unit 22 including a RAM or a ROM, a disk drive 23 for the disk 17, and a hard disk drive (HDD) 24. In the embodiment, the display 12, the keyboard 13, and the mouse 14 are connected to the CPU 21 via the bus 20, but these may be directly connected to the CPU 21. The display 12 may be connected to the CPU 21 via a known graphic interface (not shown) that processes input / output image data.

In the computer system 10, the keyboard 13 and the mouse 14 are input units of the processing device 120. The display 12 is a display unit that displays input contents and the like for the processing device 120 on the screen 12A.

The computer system 10 is not limited to the configuration shown in FIGS. 3 and 4, and various known elements may be added or alternatively used.

FIG. 5 is a diagram showing shape data.

The article data is data representing the coordinates and shape of the article displayed on the screen 110A. The article data has an article ID, a shape type, reference coordinates, a size, and a rotation angle.

The shape type represents the outer shape of the article. In FIG. 5, as an example, the shape types indicate Cuboid (cuboid) and Cylinder (cylindrical body).

The reference coordinate indicates the coordinate value of a point that serves as a reference for coordinates representing the entire article. The unit of the coordinate value is meter (m). An XYZ coordinate system is used as the coordinate system.

The size represents the length of the article in the X-axis direction, the length in the Y-axis direction, and the length in the Z-axis direction. The unit is meters (m). As an example, the length in the X-axis direction represents the vertical length, the length in the Y-axis direction represents the height, and the length in the Z-axis direction represents the depth (the length in the horizontal direction).

The rotation angle is represented by rotation angles θx, θy, and θz with respect to the X-axis direction, the Y-axis direction, and the Z-axis direction. The unit is degree (deg.). The rotation angle θx is an angle for rotating the article about the X axis as a rotation axis. Similarly, the rotation angles θy and θz are angles at which the article is rotated about the Y axis and the Z axis as rotation axes, respectively. The positive directions of the rotation angles θx, θy, and θz may be determined in advance.

If such article data is used, an image specified by the article data can be represented in the same manner as the article image displayed by the CAD data.

The article data is stored in the data holding unit 127 of the processing device 120.

FIG. 6 is a diagram illustrating an example of an image of an article.

FIG. 6 shows three articles represented by the article data in FIG.

An article with an article ID of 001 has a shape type of Cuboid (cuboid), reference coordinates (X, Y, Z) of (0.0, 0.0, 0.0), and a size of (0.8, 0.2, 0.4), and the rotation angles θx, θy, θz are (0.0, 0.0, 0.0).

Since the reference coordinates (X, Y, Z) are (0.0, 0.0, 0.0), one vertex of the article whose article ID is 001 coincides with the origin O of the XYZ coordinate system. .

An article with an article ID of 002 has a shape type of Cuboid (cuboid), reference coordinates (X, Y, Z) of (0.6, 0.2, 0.0), and a size of (0.2, 0.2, 0.1), and the rotation angles θx, θy, θz are (0.0, 0.0, 0.0).

For this reason, the article with the article ID 002 is arranged on the article with the article ID 001.

The article with the article ID 003 has a shape type of Cylinder, a reference coordinate (X, Y, Z) of (0.8, 0.3, 0.1), and a size of (0.2 , 1.0, 0.3), and the rotation angles θx, θy, θz are (0.0, 0.0, 90.0).

For this reason, the article with the article ID 003 is connected to the X axis positive direction side of the article with the article ID 002 in a state where the article ID is rotated 90 degrees about the Z axis.

As described above, in the embodiment, the article data in the image projected on the screen 110A using the article data having the article ID, shape type, reference coordinates, size, and rotation angle shown in FIG. Define coordinates and shape.

For example, when the shape type is Cuboid, the coordinates of the eight vertices are the length in the X-axis direction, the length in the Y-axis direction, the length in the Y-axis direction, and the Z-axis direction with respect to the reference coordinates. Can be obtained by adding or subtracting the length. The coordinates of the eight vertices represent the coordinates of the corner of the article whose shape type is Cuboid.

If the coordinates of eight vertices are obtained, an expression representing 12 sides can be obtained. The expression representing the 12 sides is an expression representing the coordinates of the Edge of the article whose shape type is Cuboid.

Further, if the coordinates representing the eight vertices and / or the expressions representing the 12 sides are obtained, the expressions representing the six surfaces of the article whose shape type is Cuboid are obtained, and the coordinates of the surface are obtained. be able to.

In addition, when the shape type is Cylinder (cylindrical body), based on the length in the X-axis direction, the length in the Y-axis direction, and the length in the Z-axis direction of the article represented by the size, An expression representing a certain circle (or ellipse) can be obtained. Further, if an equation representing a circle (or ellipse) at both ends and a reference coordinate are used, an equation representing the coordinates of the circle (or ellipse) at both ends can be obtained. The coordinates of the side surface of the cylinder can be obtained by using an expression representing the coordinates of the circles (or ellipses) at both ends.

Here, the articles whose shape types are Cuboid (cuboid) and cylinder (cylindrical body) have been described, but articles of various shapes such as a sphere, a triangular pyramid, and a rectangular parallelepiped having a concave portion are similarly projected onto the screen 110A. The coordinates and shape in the image can be obtained.

FIG. 7 is a diagram showing size data.

The size data is data in a table format in which the article ID of the article displayed on the screen 110A is associated with the pointer size of the pointer 130A. Pointer size _(X _p, Y p, _{Z p)} is the X-axis direction width _X p, Y-axis direction of the height _{Y p,} and represents the depth _{Z p} in the Z-axis direction.

Pointer 130A has a width _{X p,} is displayed as an ellipsoid having a height _{Y p,} and the depth _{Z p.} As an example, the pointer size (X _p , Y _p , Z _p ) associated with the article having the article ID 001 is (0.05, 0.02, 0.05). Pointer size article ID is associated with the article _{_{002 (X p, Y p,}} Z p) is a (0.01,0.01,0.01). The pointer size (X _p , Y _p , Z _p ) associated with the article with the article ID 002 is (0.015, 0.05, 0.015).

The reason why the pointer size is set in accordance with the article will be described with reference to FIGS.

8 to 16 are diagrams showing a method for determining the size of the pointer 130A according to the size of the article.

The surface (ellipsoidal surface) of the pointer 130A is represented by Expression (4).

Half the values of the width X _p , the height Y _p , and the depth Z _p are the parameters a, b, and c in the equation (4), respectively. That _is, _{a = X p / 2, b} = Y p / 2, c = Z p / 2.

As shown in FIG. 8, when an article 111-1 having a width X1, a height Y1, and a depth Z1 is displayed on the screen 110A, the parameters a, b, and c of the ellipsoid are, for example, a = kX1, b = KY1, c = kZ1, and the parameter k is set to 0.05.

As shown in FIG. 9, when an article 111-2 having a width X2, a height Y2, and a depth Z2 is displayed on the screen 110A, the parameters a, b, and c of the ellipsoid are, for example, a = kX2, b = KY2, c = kZ2. The parameter k is 0.05.

Article 111-2 has a shape in which three L-shaped blocks are connected. The size of the pointer 130A is a size that can enter the gap between the three blocks of the article 111-2.

As shown in FIG. 10, when an article 111-3 having a width X3, a height Y3, and a depth Z3 is displayed on the screen 110A, the parameters a, b, and c of the ellipsoid are, for example, a = kX3, b = KY3, c = kZ3. The parameter k is 0.05.

The article 111-3 is provided with a rectangular parallelepiped hole in the rectangular parallelepiped 111-31, and a prismatic portion 111-32 is provided in the hole. The size of the pointer 130A is a size that can enter the gap between the hole of the rectangular parallelepiped 111-31 and the prism portion 111-32.

In addition, as shown in FIG. 11, when the article 111-1 having the width X1, the height Y1, and the depth Z1 is displayed on the screen 110A, for example, the parameters a, b, and c of the ellipsoid are expressed by the following equation (5). It may be set to l1 obtained in step (1). In this case, the parameter k is 0.05 and a = b = c = l1.

Also, as shown in FIG. 12, when displaying an article 111-2 having a width X2, a height Y2, and a depth Z2 on the screen 110A, for example, the parameters a, b, and c of an ellipsoid are expressed by the following equation (6). It may be set to l2 obtained by the above. In this case, the parameter k is 0.05, and a = b = c = 12.

The size of the pointer 130A obtained by Expression (6) is a size that can enter the gap between the three blocks of the article 111-2.

Further, as shown in FIG. 13, when displaying an article 111-3 having a width X3, a height Y3, and a depth Z3 on the screen 110A, for example, parameters a, b, and c of an ellipsoid are expressed by the following equation (7). It may be set to l3 obtained in step (1). In this case, the parameter k is 0.05 and a = b = c = 13.

The size of the pointer 130A obtained by Expression (7) is a size that can enter the gap between the hole of the rectangular parallelepiped 111-31 and the prism portion 111-32.

Further, as shown in FIG. 14, when the article 111-1 is displayed on the screen 110A, the parameters a, b, and c of the ellipsoid are set to a = b = c = kX1 as an example, and the parameter k is set to You may set to 0.5. That is, the parameters a, b, and c may be set to half the minimum value of the width X1, the height Y1, and the depth Z1.

Further, as shown in FIG. 15, when the article 111-2 is displayed on the screen 110A, the ellipsoid parameters a, b, and c may be set to a = b = c = kX2A as an example. The parameter k is 0.5.

X2A is the size of the gap between the three L-shaped blocks of the article 111-2 and is the minimum value of the outer dimensions of the article 111-2. In this way, the parameters a, b, and c may be set to a value that is half the minimum value of the outer dimensions of the article 111-2.

The size of the pointer 130A obtained in this way is a size that can enter the gap between the three blocks of the article 111-2.

Further, as shown in FIG. 16, when the article 111-3 is displayed on the screen 110A, the parameters a, b, and c of the ellipsoid may be set to a = b = c = kY3A as an example. The parameter k is 0.5.

Y3A is the height in the Y-axis direction of the rectangular column part 111-32 of the rectangular parallelepiped 111-31, and is the minimum value of the outer dimensions of the article 111-3. In this way, the parameters a, b, and c may be set to a value that is half the minimum value of the outer dimensions of the article 111-3.

The size of the pointer 130A thus obtained is a size that can enter the gap between the hole of the rectangular parallelepiped 111-31 and the prism portion 111-32.

FIG. 17 is a diagram showing size data in which an article ID is associated with a pointer size. Here, an example of the

buttons

111A and 111B and the article ID of the article 111 illustrated in FIG. 1 and a pointer size associated with the article ID will be described.

Suppose that the item ID of the button 111A is 011, the item ID of the button 111B is 012, and the item ID of the item 111 is 013.

The pointer size (X _p , Y _p , Z _p ) for the button 111A with the article ID 011 is (0.01, 0.01, 0.01). The pointer size (X _p , Y _p , Z _p ) of the button 111B with the article ID 012 is (0.01, 0.01, 0.01). The article 111 of the article ID is 013, the size of a pointer _{_{_{(X p, Y p, Z}}} p) is a (0.04,0.04,0.04).

18 and 19 are diagrams showing the relationship between the article and the size of the pointer 130A. Here, as an example, a case where the pointer 130A is displayed based on the size data shown in FIG. 17 when the image of the article 111 and the

buttons

111A and 111B is displayed on the screen 110A as shown in FIG. 1 will be described.

A pointer 130A is displayed on the screen 110A, and a detection area 130B is set around the pointer 130A. The detection area 130B is not displayed on the screen 110A.

As shown in FIG. 18, when the pointer 130A is at the point A1, the size of the pointer 130A is set to an initial value. The initial value is, for _example, a _{_{(X p, Y p, Z}} p) = (0.03,0.03,0.03). The initial value may be set according to, for example, the size of the screen 110A and the appropriate position of the user 1 with respect to the screen 110A.

When the pointer 130A moves from the point A1 to the point B1 and the

buttons

111A and 111B enter the detection area 130B, the pointer size (X _p , Y _p , Z _p ) becomes (0.01, 0.01, 0.01).

Thus, when a plurality of articles (

buttons

111A and 111B) enter the detection area 130B, the pointer 130A is displayed using the minimum value among the plurality of pointer sizes associated with the plurality of articles. . Since the pointer sizes associated with the

buttons

111A and 111B are equal to each other, the pointer size (X _p , Y _p , Z _p ) is set to (0.01, 0.01, 0.01) here. .

Because the

buttons

111A and 111B are relatively small articles, the user 1 can easily select the

buttons

111A and 111B by making the pointer 130A smaller than the initial value.

As shown in FIG. 19, when the pointer 130A moves from the point A1 to the point B2 and the article 111 enters the detection area 130B, the pointer size (X _p , Y _p , Z _p ) becomes (0 .04, 0.04, 0.04).

This is because the article 111 is larger than the

buttons

111A and 111B and is a relatively large article, so that the user 1 can easily select the article 111 by making the pointer 130A larger than the initial value.

Thus, the simulation system 100 changes the size of the pointer 130A according to the size of the article existing in the detection area 130B. This is to improve the usability of the simulation system 100 by making it easy for the user 1 to see the pointer 130A.

FIG. 20 is a flowchart illustrating processing executed by the processing device 120 according to the embodiment. The flowchart shown in FIG. 20 shows processing for setting the pointer size of the pointer 130A and displaying the pointer 130A on the screen 110A.

Here, as an example, a case will be described in which images of the article 111 and the

buttons

111A and 111B are displayed on the screen 110A as shown in FIG.

Processing device 120 starts processing after power is turned on (start).

The processing apparatus 120 acquires article data from the data holding unit 127 (step S1).

The processing device 120 generates a video signal using the article data, and causes the projection device 110B to project an image (step S2). As a result, an image of the stereoscopic model of the article 111 and the

buttons

111A and 111B is displayed on the screen 110A. The image of the article 111 and the

buttons

111A and 111B displayed on the screen 110A represents a virtual object existing in the virtual space.

Note that the processing of steps S1 and S2 is performed by the video output unit 128.

The processing device 120 acquires data representing the position and shape of the body of the user 1 three-dimensionally from the position measurement device 140 (step S3). The process of step S3 is performed by the human body detection unit 121.

The processing device 120 determines whether the body of the user 1 exists based on the data acquired in step S3 (step S4). The process of step S4 is performed by the human body detection unit 121.

If the processing device 120 determines that the body of the user 1 exists (S4: YES), the processing device 120 obtains coordinates representing the position of each part of the body of the user 1 (step S5). The process of step S5 is performed by the human body detection unit 121.

The processing device 120 detects the coordinates P (P _x , P _y , P _z ) (step S6). The coordinates P (P _x , P _y , P _z ) are obtained by converting the coordinates of the intersection point of the straight line connecting the right shoulder and the right wrist of the user 1 and the screen 110A to the coordinates in the image projected on the screen 110A. The coordinates are obtained by the position detector 122. The process of step S6 is performed by the position detection unit 122.

The processing device 120 generates the detection area 130B (step S7). The processing in step S7 is performed by the detection area generation unit 123. The detection area generator 123 generates a detection area 130B having a predetermined radius centered on the coordinates P (P _x , P _y , P _z ).

The detection area 130B is an area outside the surface of the pointer 130A and included in a sphere defined by a predetermined radius with the coordinates P (P _x , P _y , P _z ) as the center.

The processing device 120 determines whether or not an article exists in the detection area 130B (step S8). The process of step S8 is performed by the object determination unit 125. The object determination unit 125 determines whether or not the detection area 130B and the article 111 or the display area of the

button

111A or 111B have an intersection, and whether or not there is an article that is at least partially located inside the detection area 130B. judge. When an article exists inside the detection area 130B, the object determination unit 125 determines the type of the article.

If the processing device 120 determines that there is no article in the detection area 130B (S8: NO), the processing device 120 sets the pointer size to an initial value (step S9). The processing in step S9 is performed by the pointer generator 126. The initial value of the pointer size is held in the data holding unit 127.

The processing device 120 displays the pointer 130A on the screen 110A (step S10). The processing in step S10 is performed by the video output unit 128. The video output unit 128 causes the projection device 110B to display the image of the pointer 130A generated by the pointer generation unit 126 on the screen 110A.

When the pointer 130A is displayed on the screen 110A, a series of processing ends (end).

If the processing device 120 determines that an article exists in the detection area 130B (S8: YES), the processing apparatus 120 acquires the article ID of the article existing in the detection area 130B (step S11). The process of step S11 is performed by the object determination unit 125. In step S11, when there are a plurality of articles in the detection area 130B, a plurality of article IDs are acquired.

The object determination unit 125 acquires the article ID of the article 111 or the

button

111A or 111B having an intersection with the detection area 130B.

The processing device 120 reads the pointer size corresponding to the article ID acquired in step S11 from the size data (see FIGS. 7 and 18) (step S12). The processing in step S12 is performed by the pointer generation unit 126. When there are a plurality of article IDs acquired in step S11, the pointer generation unit 126 reads a plurality of pointer sizes.

This corresponds to, for example, reading out two pointer sizes associated with the

article IDs

011 and 012 shown in FIG. 17 when the

buttons

111A and 111B exist in the detection area 130B as shown in FIG. .

When the processing device 120 reads out a plurality of pointer sizes in step S11, the processing device 120 selects the smallest pointer size among the plurality of pointer sizes (step S13). The processing in step S13 is performed by the pointer generation unit 126. Note that, when the number of article IDs acquired in step S11 is one, the pointer generation unit 126 does not perform any particular process in step S13.

The processing device 120 determines whether the pointer size is smaller than a predetermined lower limit (step S14). The processing in step S14 is performed by the pointer generator 126.

The pointer generation unit 126 reads out a predetermined lower limit value of the pointer size from the data holding unit 127, one pointer size read in step S12, or the pointer size selected in S13 and the pointer size read from the data holding unit 127 Is compared with a predetermined lower limit value.

The reason why the pointer size is compared with the predetermined lower limit in this way is to prevent the pointer 130A from becoming too small by reducing the pointer size in the process of step S109 described later. The predetermined lower limit value may be set according to the size of the screen 110A, the appropriate position of the user 1 with respect to the screen 110A, and the like.

If the processing device 120 determines that the pointer size is smaller than the predetermined lower limit value (S14: YES), the processing device 120 corrects the pointer size to the predetermined lower limit value (step S15). The processing in step S15 is performed by the pointer generator 126. If the pointer size is smaller than the predetermined lower limit value, it is difficult for the user 1 to see, and therefore the pointer size is corrected to be increased to the lower limit value before being displayed on the screen 110A.

The processing device 120 displays the pointer 130A having the pointer size corrected to the lower limit value on the screen 110A (step S10). The processing in step S10 is performed by the video output unit 128.

On the other hand, when determining that the pointer size is not smaller than the predetermined lower limit (S14: NO), the processing device 120 displays the pointer 130A having the pointer size set in step S12 or S13 on the screen 110A (step S10). .

As described above, the pointer 130A having the pointer size set by the processing device 120 is displayed on the screen 110A. The flow shown in FIG. 20 is repeatedly executed.

FIG. 21 is a diagram showing the relationship between the article and the size of the pointer 130A. In FIG. 21, as in FIGS. 18 and 19, images of the article 111 and the

buttons

111A and 111B are displayed on the screen 110A.

Here, as an example, when the user 1 wants to determine the input to the button 111A, the determination operation to the button 111B is erroneously performed, and the operation to cancel the input to the button 111B is performed. A case where the size of 130A is changed will be described.

As described above, the operation for determining the input to the

buttons

111A and 111B and the operation for canceling the input to the

buttons

111A and 111B are determined as operations that can be detected by the operation detection unit 124.

First, when the pointer 130A is at the point A2, the size of the pointer 130A is set to an initial value. The initial value is, for _example, a _{_{(X p, Y p, Z}} p) = (0.03,0.03,0.03).

When the pointer 130A moves from the point A2 to the point B3 and the

buttons

The user 1 wants to tap the button 111A in this state, but the right hand 2 swings, the pointer 130A moves from the point B3 to the point B4, and the pointer 130A does not touch the button 111A but touches the button 111B. It becomes a state.

When the user 1 taps in this state, the input to the button 111B is determined. Since the user 1 wants to determine the input to the button 111A, the user 1 performs a cancel operation. Specifically, the user 1 performs an operation of paying the right hand 2 to the right side with the pointer 130A touching the button 111B. Thereby, the canceling operation is detected by the operation detecting unit 124.

When the cancel operation is performed, the pointer size (X _p , Y _p , Z _p ) associated with the button 111B for which the cancel operation has been performed is set to a size of 90%. That is, the pointer 130A is 10% smaller. Such a function of reducing the pointer size in accordance with the cancel operation is referred to as a learning function.

User 1 moves the pointer 130A from point B4 to point B5, and the pointer 130A is in a state of touching the button 111A. The user 1 may tap the button 111A in this state. Since the pointer 130A is 10% smaller, it is easier to select the button 111A, and erroneous input can be suppressed.

FIG. 22 is a diagram showing size data in which an article ID is associated with a pointer size. Here, a change in size data before and after the cancel operation is performed will be described.

The size data shown on the left side of FIG. 22 is the size data before the cancel operation is performed, and is the same as the size data shown in FIG.

As described with reference to FIG. 21, when the input to the button 111B is canceled, the pointer size associated with the item ID 012 corresponding to the button 111B is 10% smaller as shown on the right side of FIG. Is done.

In this way, when a cancel operation is performed, the pointer size is reduced to prevent subsequent erroneous input.

Note that the pointer size is reduced when the cancel operation is performed when the cancel operation is performed within a predetermined time for the same button as the button for which the cancel operation is performed. This is because when the user 1 notices an erroneous input, it is considered that the cancel operation is performed within a long time after the determination operation is performed. In addition, if the cancel operation is performed after a certain amount of time has elapsed since the determination operation was performed, it is considered that the erroneous input is not canceled but the intention to cancel is performed.

For this reason, the predetermined time for determining whether or not the canceling operation is performed may be set to an appropriate value through experiments or simulations. For example, the predetermined time is 5 seconds. The determination operation is an operation (selection operation) for determining selection of the

button

111A or 111B.

FIG. 23 is a flowchart illustrating processing executed by the processing device 120 according to the embodiment. Here, as an example, a case will be described in which images of the article 111 and the

buttons

111A and 111B are displayed on the screen 110A as shown in FIG.

Processing device 120 starts processing after power is turned on (start).

The processing device 120 displays the pointer 130A on the screen 110A (step S101). The process of step S101 is the process shown in FIG.

The processing device 120 detects the operation of the user 1 based on the coordinates representing the position of each part of the body of the user 1 input from the human body detection unit 121 (step S102). The process in step S102 is performed by the operation detection unit 124.

The operation of the user 1 is an operation such as a gesture performed by the user 1, and here, as an example, an operation for resetting the size of the pointer 130A, an operation for determining an input to the

button

111A or 111B, and The operation for canceling the input to the

button

111A or 111B is determined in advance as an operation that can be detected by the operation detection unit 124.

The processing device 120 determines whether or not the operation of the user 1 detected in step S102 is a reset operation (step S103). The process in step S103 is performed by the operation detection unit 124.

If the processing device 120 determines that the operation is not a reset operation (S103: NO), it determines whether or not the operation of the user 1 detected in step S102 is a determination operation (step S104). The process in step S104 is performed by the operation detection unit 124.

If the processing device 120 determines that the operation is not a determination operation (S104: NO), the processing device 120 determines whether the operation of the user 1 detected in step S102 is a cancel operation (step S105). The process in step S105 is performed by the operation detection unit 124.

If the processing device 120 determines that the operation is not a cancel operation (S105: NO), the series of processing ends (end). If it is determined that none of the reset operation, the determination operation, and the cancel operation, the process is terminated. If the power is turned on, the process is repeated from the start.

If it is determined that the processing device 120 is the determining operation (S104: YES), the processing device 120 stores the item ID of the item for which the determining operation has been performed (step S106). The processing in step S106 is performed by the pointer generator 126. For example, when the article for which the determination operation has been performed is the button 111A, the article ID of the button 111A is stored.

When the processing device 120 stores the article ID, the processing unit 120 ends the series of processing (end). If the power is on, the process is repeated from the start.

If it is determined that the processing device 120 is a cancel operation (S105: YES), the processing device 120 determines whether or not the cancel operation has been performed for the same article as the already stored article ID (step S107). The processing in step S107 is performed by the pointer generator 126.

If the processing device 120 determines that the article ID is the same as the already-stored article ID (S107: YES), the elapsed time from the storage of the article ID of the article for which the determination operation has been performed until the cancel operation is performed Is determined within a predetermined time (step S108).

The processing in step S108 is performed by the pointer generator 126. Whether or not the elapsed time is within the predetermined time may be determined by whether or not the elapsed time is equal to or shorter than the predetermined time.

When determining that the elapsed time is within the predetermined time (S108: YES), the processing device 120 reduces the pointer size associated with the canceled article to 90% of the pointer size included in the size data. (Step S109). The processing in step S109 is performed by the pointer generator 126.

This is because it is considered that the user 1 has performed a cancel operation in order to cancel the erroneous operation. For example, as described with reference to FIG. 21, when the user 1 erroneously determines the input to the button 111B and cancels the input to the button 111B, the pointer size (X _p associated with the button 111B is canceled. , Y _p , Z _p ) is set to a size of 90%.

When the pointer size is reduced to 90%, the processing device 120 ends the series of processing (end). If the power is on, the process is repeated from the start.

In addition, when it is determined that the processing device 120 is not the same product as the previously stored product ID (S107: NO), and when it is determined that the elapsed time is not within the predetermined time (S108: NO), a series of End processing (END). If the power is on, the process is repeated from the start.

If it is determined that the article is not the same as the already-stored article ID (S107: NO), it is considered different from the case where the user 1 performs a cancel operation in order to cancel the erroneous operation. Exit.

If it is determined that the elapsed time is not within the predetermined time (S108: NO), it is considered that the user 1 performs a cancel operation in order to cancel the erroneous operation, and thus the series of processing ends. .

If the processing device 120 determines that the reset operation is being performed (S103: YES), the processing device 120 sets the pointer size to an initial value (step S110). The processing in step S110 is performed by the pointer generation unit 126. The initial value of the pointer size is held in the data holding unit 127 similarly to the size data shown in FIG.

FIG. 24 is a diagram showing a display on the screen 110A when an instruction operation is actually performed in the simulation system 100. FIG. On the screen 110A, nine markers are arranged in three rows and three columns. The marker is displayed as a sphere like the pointer 130A, and the center marker is a marker C1.

When the user 1 performs an instruction operation to move the right hand 2 from the upper right to the lower left as indicated by an arrow and tries to move the pointer 130A from the start point A6 to the marker C1, the positional deviation between the marker C1 and the pointer 130A is detected. It was measured.

The interval between the nine markers is 100 pixels. For 100 pixels, when the width of the screen 110A in the X-axis direction is 3 meters, the interval between the markers is 150 mm. The radius of the marker is 30 mm, and the radius of the detection region 130B is 100 mm. The initial value of the radius of the pointer 130A is 40 mm, and the radius of the pointer 130A when the marker exists in the detection area 130B is 20 mm. The radius of the pointer 130A shown in FIG. 24 is 20 mm.

FIG. 25 is a diagram illustrating a locus of the pointer 130A displayed by the simulation system 100 when the instruction operation described with reference to FIG. 24 is performed. That is, FIG. 25 shows the locus of the coordinates P (P _x , P _y , P _z ) of the pointer 130A obtained by the simulation system 100.

In FIG. 25, the points surrounded by a broken-line circle indicate the coordinates P (P _x , P _y when an instruction operation is performed to keep the pointer 130A aligned with the marker C1 after the pointer 130A reaches the marker C1. , P _z ).

For this reason, in FIG. 25, the points scattered from the upper right to the lower left toward the broken circle indicate the locus when the pointer 130A is moved from the start point A6 to the marker C1 shown in FIG. In addition, in FIG. 25, the numerical value (pixel value) of the XY coordinate used by calculation is shown.

As shown in FIG. 25, the position of the point surrounded by the broken-line circle fluctuates because the right hand 2 shakes in an attempt to keep the pointer 130A aligned with the marker C1 after the pointer 130A reaches the marker C1. is there.

FIG. 26 shows the result of the instruction operation for moving the pointer 130A to the marker C1 in this way.

FIG. 26 is a diagram showing a result of an instruction operation for moving the pointer 130A to the marker C1. Here, when there is a marker in the detection area 130B, a case where the process of changing the radius of the pointer 130A from the initial value of 40 mm to 20 mm is described as “with process”.

For comparison, a case where the radius of the pointer 130A is fixed to the initial value of 40 mm even when a marker is present in the detection area 130B is referred to as “no processing”.

In addition, when the cancel operation is performed in a state where the radius of the pointer 130A is set to 20 mm, the result when the pointer 130A is set to 90% size by the learning function is also referred to as “processing & learning function”. Show. The radius of the pointer 130A in the case of “with processing & learning function” is 18 mm.

Note that the learning function is not used in the case of “with processing” and “without processing”.

In these three cases, when the pointer 130A is moved to the marker C1 and the pointer 130A is continuously aligned with the marker C1, the frame in which the pointer 130A contacts only the marker C1 and the pointer 130A other than the marker C1 And a frame in contact with the marker (any one of the eight markers around the marker C1).

As a result, in the case of “no processing”, the number of frames in which the pointer 130A touches only the marker C1 is 633, and the number of frames in which the pointer 130A touches a marker other than the marker C1 is 24%. For this reason, the probability of contacting a marker other than the marker C1 was 196.

In the case of “with processing”, the number of frames in which the pointer 130A touches only the marker C1 is 582, and the number of frames in which the pointer 130A touches a marker other than the marker C1 is 59. For this reason, the probability of contacting a marker other than the marker C1 was 9%.

Thus, in the case of “with processing”, the probability that the pointer 130A touches only the number of frames that touch only the marker C1 is significantly reduced as compared with the case of “without processing”.

In the case of “with processing & learning function”, the number of frames in which the pointer 130A contacts only the marker C1 is 540, and the number of frames in which the pointer 130A contacts a marker other than the marker C1 is 35. For this reason, the probability of contacting a marker other than the marker C1 was 6%.

Thus, in the case of “with processing & learning function”, the probability that the pointer 130A touches only the number of frames that touch only the marker C1 is significantly reduced as compared with the case of “without processing”. In addition, it was further improved over the case of “with treatment”.

As described above, according to the embodiment, when at least a part of an article is present in the detection area 130B, the size of the pointer 130A is changed according to the type of the article. When the article is relatively small, the pointer size is made smaller than the initial value, and when the article is relatively large, the pointer size is made larger than the initial value.

Thus, since the pointer size is set according to the size of the article existing around the pointer 130A, the usability of the simulation system 100 is greatly improved.

In addition, when the cancel operation is performed within a predetermined time after the determination operation is performed on the article such as the

buttons

111A and 111B, the pointer size is reduced to 90% by the learning function. When 1 tries to move the pointer 130A to the same article again, it is easy to move the pointer 130A to the desired article.

Also, if the cancel operation is performed again after the learning function is performed once, the pointer size is reduced to 81%. As described above, when the cancel operation is performed a plurality of times on the same article, the size decreases by 10%.

Since the pointer size is reduced by the learning function according to the movement characteristics of the user 1, the operability when the user 1 operates the pointer 130A after the learning function is performed is as follows. It will be improved.

Therefore, it is possible to provide the simulation system 100 with greatly improved usability.

In the above description, the article 111 or the

button

111A or 111B is used. In particular, the determining operation and the canceling operation have been described as a mode of inputting or canceling the

button

111A or 111B.

However, for example, when the article 111 is selected in order to move the article 111 using the pointer 130A, a determination operation may be performed. After performing the determination operation, the article 111 may be moved together with the pointer 130A. And after moving the article | item 111 in this way, in order to cancel selection of the article | item 111, you may make it perform a cancellation operation | movement. In this way, it is possible to realize an easy-to-use assembly support system with the simulation system 100.

In the above description, when the input to the button 111B is canceled, as shown on the right side of FIG. 22, the pointer size associated with the item ID 012 corresponding to the button 111B is reduced by 10%. explained.

However, when the input to the button 111B is canceled, the pointer size associated with the article ID 011 corresponding to the button 111A in the detection area 130B may be reduced by 10%. In this case, the pointer size associated with the

article IDs

011 and 012 corresponding to the

buttons

111A and 111B in the detection area 130B is reduced by 10%.

In the above description, a mode has been described in which whether or not at least a part of the detection area 130B is located is determined by whether or not the detection area 130B and the display area of the article 111 or the

button

111A or 111B have an intersection. .

However, whether at least partially located within the detection area 130B, the coordinates _{_{P (P x, P y,}} P z) in the detection area 130B centered on, the display area of the article 111 or

buttons

111A or 111B You may determine by whether it is contained.

In the above description, the pointer 130A is displayed as an ellipsoid. However, the pointer 130A may be displayed as an image having a shape other than the ellipsoid.

In the above description, the detection area 130B is an ellipsoidal area, but the detection area 130B may be an area having a shape other than an ellipsoid. Further, although the detection area 130B has been described as an area that exists outside the pointer 130A, it may be an area that also includes the inside of the pointer 130A.

Although the simulation system of the exemplary embodiment of the present invention has been described above, the present invention is not limited to the specifically disclosed embodiment, and does not depart from the scope of the claims. Various modifications and changes are possible.

DESCRIPTION OF SYMBOLS 10 Computer system 11 Main body part 12 Display 13 Keyboard 14 Mouse 15 Modem 100 Simulation system 110A Screen 110B Projection apparatus 110C 3D glasses 120 Processing apparatus 121 Human body detection part 122 Position detection part 123 Detection area generation part 124 Motion detection part 125 Object determination part 126 Pointer generation unit 127 Data holding unit 128 Video output unit 130A Pointer 130B Detection area 140 Position measurement device

Claims

A display unit for displaying an image of the article based on article data representing the shape and position of the article, and a pointer whose position is operated by a user;
A data storage unit for storing the article data;
A first detection unit that detects an instruction operation in which the user indicates the position of the pointer;
A second detection unit that detects a position of a pointer in the coordinate system of the display unit based on the instruction operation detected by the first detection unit;
An area generation unit that includes the pointer and generates a detection area larger than the pointer;
A size setting unit that sets the size of the pointer according to the type of the article at least a part of which is located inside the detection area;
An output unit that causes the display unit to display the pointer having a size set by the size setting unit.
The data storage unit further stores size data in which the type of the article is associated with the size of the pointer,
The simulation system according to claim 1, wherein the size setting unit sets the size of the pointer to a size corresponding to a type of the article at least a part of which is located inside the detection area in the size data.
The pointer is displayed so as to have the size around the position detected by the second detection unit,
The simulation system according to claim 1, wherein the detection area is an area that is centered on a position detected by the second detection unit, includes the pointer, and is larger than the pointer.
The simulation system according to claim 3, wherein the pointer is displayed as an ellipsoid centered on a position detected by the second detection unit.
The simulation according to any one of claims 1 to 4, wherein the size setting unit sets the size of the pointer to a predetermined size when there is no article that is at least partially located inside the detection area. system.
A first determination unit that determines whether the selection operation of the article by the pointer has been performed;
A second determination unit that determines whether the selection operation has been canceled when the first determination unit determines that the selection operation has been performed;
The simulation system according to any one of claims 1 to 5, wherein the size setting unit sets the size of the pointer to be smaller by a predetermined ratio when it is determined by the second determination unit that the selection operation has been canceled. .
The size setting unit, when it is determined that the selection operation is canceled by the second determination unit within a predetermined time after it is determined that the selection operation is performed by the first determination unit. The simulation system according to claim 6, wherein the size is set smaller by a predetermined ratio.
The size setting unit sets the size of the pointer to the lower limit value when the size of the pointer is set smaller by a predetermined ratio and the reduced size is smaller than a predetermined lower limit value. The simulation system described.