JP2016110249A - Spatial handwriting input system, spatial handwriting input method, and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To expand the expression breadth of a handwriting input system by making an operation locus on a virtual plane in the depth direction reflected in output, and enable a user to grasp the position of the virtual plane visually.SOLUTION: Provided is a spatial handwriting input system characterized by comprising: a coordinate detection unit for detecting the three-dimensional coordinates of the movement locus of a pointer; a virtual plane setting unit for setting a virtual plane in a three-dimensional space; a coordinate conversion unit for converting the three-dimensional coordinates of the movement locus of the pointer; an input locus acquisition unit for acquiring a locus on an XY plane among the movement loci of the pointer as an input locus; a contact degree acquisition unit for calculating and acquiring the degree of contact with the virtual plane from the position in the direction of an axis Z among the movement loci of the pointer; and a display unit for displaying the input locus on a GUI screen when the degree of contact exceeds a threshold, and displaying an indicator for indicating the position of the pointer and the distance to the virtual plane on the GUI screen, the display unit changing the expression of a displayed locus in accordance with the degree of contact.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a spatial handwriting input system, and more specifically, spatial handwriting input that enables an output reflecting an operation trajectory in the depth direction with respect to a virtual plane with respect to a user's handwritten input via a virtual plane in space. About the system.

  In recent years, a technology called a natural user interface (NUI) has been developed, which enables information to be exchanged with an information processing apparatus such as a computer through a more natural and intuitive operation such as a touch panel or a gesture. Yes.

  For example, in Patent Document 1, a space handwriting system in which an electronic pen and a display device are communicably connected, and the electronic pen includes a coordinate detection unit that detects coordinates of a three-dimensional space of a pen tip, A virtual plane creating unit that creates a virtual plane based on the set point, and a pen tip movement in the normal direction of the virtual plane based on a change in coordinates of the three-dimensional space, and the method A stroke detection unit that recognizes a stroke break when the movement amount or movement speed or movement acceleration of the pen tip in the linear direction exceeds a predetermined threshold, and a continuous stroke from the stroke break to the next break A coordinate conversion unit that converts coordinates in the three-dimensional space indicating the trajectory of the pen tip into plane coordinates having a specific point on the virtual plane as an origin, and information on the converted plane coordinates in the display device A spatial handwriting system, wherein the display device receives the information of the plane coordinates and displays the locus of the pen tip on the virtual plane on the screen. Has been. Patent Document 2 also discloses a similar spatial handwriting system.

JP2013-3961A JP 2013-125487 A

  As in Patent Document 1 or 2, a system that sets a virtual plane in a predetermined space and allows a user to exchange information with a computer or the like by handwriting input via the virtual plane is a keyboard or mouse operation. Compared to the above, it is possible to provide the user with a more natural and intuitive input operation, and at the same time, to reduce physical restrictions (input device, installation location, etc.) at the time of input. The man-machine interface using the virtual plane can be applied to a scene where high hygiene management is required and avoiding unnecessary physical contact, such as a medical site or a food manufacturing site.

  Until now, in a system for inputting characters and figures using a virtual plane, a two-dimensional trajectory based on the virtual plane is extracted from a user's operation in a three-dimensional space, and the characters and figures appearing there are more accurately represented. While techniques for recognizing are disclosed, much attention has not been paid to the method of expressing the operation trajectory itself. For example, Patent Document 1 discloses a technique for detecting a break between characters input by a user from an operation trajectory in a three-dimensional space, while an output representation of a trajectory detected as a stroke constituting a character is two-dimensional. The trajectory in coordinates was simply represented by a uniform line.

  In addition, as a basic problem in the input using the virtual plane, it is not easy for the user to accurately grasp the position of the virtual plane because the virtual plane is not visible to the user. It was. For this reason, when the user inputs characters or figures on the virtual plane, it is difficult for the user to understand the contact point between the virtual plane and the pointer (such as the user's finger or pen-type device), which has been a factor that hinders natural input.

  The present invention has been made to solve the above-described problems, and in handwriting input using a virtual plane, the operation trajectory in the depth direction with respect to the virtual plane is reflected in the output, thereby expressing the expression of the handwriting input system. The purpose is to expand the width and allow the user to visually grasp the position of the virtual plane.

  In order to solve the above-described problems, a spatial handwriting input system according to the present invention includes a coordinate detection unit that detects a three-dimensional coordinate of a movement locus of a pointer in a three-dimensional space, and a virtual plane corresponding to a GUI (Graphical User Interface) screen. A virtual plane setting unit that sets the three-dimensional space in the three-dimensional space, and the three-dimensional coordinates of the movement trajectory of the pointer, taking the X axis and the Y axis on the virtual plane, and the Z axis in a direction orthogonal to the XY plane A coordinate conversion unit that converts the coordinate into a coordinate, an input trajectory acquisition unit that acquires a trajectory on the XY plane among the movement trajectories of the pointer as an input trajectory, and the Z axis among the movement trajectories of the pointer A contact degree acquisition unit that calculates a contact degree with respect to the virtual plane from a position in a direction and acquires the contact degree in association with the input locus, and the contact degree exceeds a predetermined threshold value The input locus is displayed on the GUI screen as a display locus, and an indicator for indicating the position of the pointer on the XY plane and the distance in the Z-axis direction to the virtual plane is displayed on the GUI screen. A display unit for displaying, wherein the display unit changes the expression of the display locus according to the degree of contact.

  According to an embodiment of the present invention, in the spatial handwriting input system according to the present invention, the threshold is set as the value of the contact degree at the intersection with the virtual plane, and the contact degree is from the near side of the virtual plane to the back side. It is calculated so that it may become high toward the side.

  According to an embodiment of the present invention, the spatial handwriting input system according to the present invention is characterized in that the display unit increases the thickness of the line of the displayed trajectory as the contact degree is higher.

  According to an embodiment of the present invention, the spatial handwriting input system according to the present invention is characterized in that the display unit increases the color density of the line of the locus to be displayed as the contact degree is higher. .

  According to an embodiment of the present invention, in the spatial handwriting input system according to the present invention, the display unit displays an indicator indicating the current position of the pointer on the XY plane on the GUI screen, and the indicator is the contact. The color, shape, size, or expression method is changed according to the degree.

  According to one aspect of the present invention, a spatial handwriting input method according to the present invention includes a coordinate detection step for detecting a three-dimensional coordinate of a moving locus of a pointer in a three-dimensional space, and a virtual corresponding to a GUI (Graphical User Interface) screen. A virtual plane setting step for setting a plane in the three-dimensional space, and the three-dimensional coordinates of the movement locus of the pointer are set in a direction orthogonal to the XY plane by taking the X axis and the Y axis on the virtual plane. A coordinate conversion step of converting the coordinates into axes, an input trajectory acquisition step of acquiring a trajectory on the XY plane among the movement trajectories of the pointer as an input trajectory, and the Z of the movement trajectories of the pointer A contact degree acquisition step of calculating a contact degree with respect to the virtual plane from an axial position and acquiring the contact degree in association with the input locus, and the contact degree exceeds a predetermined threshold value. A display step for displaying the input locus as a display locus on the GUI screen, and an indicator for indicating the position of the pointer on the XY plane and the distance in the Z-axis direction to the virtual plane. Displaying on the screen, wherein the display step changes the expression of the display trajectory in accordance with the degree of contact.

  According to one aspect of the present invention, a computer program according to the present invention causes a computer to execute the information processing method according to the present invention.

  According to the present invention, the contact degree is calculated from position information of the pointer in the depth direction (Z-axis direction) with respect to the virtual plane, and the expression of the display trajectory such as the thickness of the line and the color density is changed according to the contact degree. Therefore, the width of the output expression in the handwriting input system using the virtual plane can be expanded, and the amount of information in the output can be increased.

  In addition, since it is possible to visually feedback the distance between the pointer and the virtual plane to the user, the user can input characters and figures on the virtual plane more naturally by referring to the distance. It becomes. Thereby, more accurate input to the virtual plane is achieved, which contributes to improvement in accuracy such as character recognition that can be performed following the input.

  Furthermore, as a result of the above-mentioned advantages, it becomes possible to output precise handwritten input as intended by the user with a wide range of expressions, so that the amount of information in handwritten input using a virtual plane can be as large as that of a paper medium such as a pen or brush. Compared with the amount of information of handwritten input to a plane, it is inferior, and it can be possible to use handwritten input on a virtual plane for artistic purposes such as calligraphy and painting.

It is a figure which shows typically an example of a structure of the spatial handwriting input system which concerns on this invention. It is a block diagram which shows an example of the main structures of the spatial handwriting input system which concerns on this invention. It is a figure which shows the relationship between the initial coordinate axis and conversion coordinate axis in one Embodiment of this invention. FIG. 4A is a perspective view showing the relationship between the virtual plane, the pointer, and the contact degree. FIG. 4B is a side view showing the relationship between the virtual plane, the pointer, and the contact degree. FIG. 4C is a diagram illustrating the relationship between the depth of the pointer and the contact degree with respect to the virtual plane. It is a figure which shows the relationship between an input locus | trajectory, a contact degree, and a display locus | trajectory in one Embodiment of this invention. It is a figure which shows an example of the indicator display according to a contact degree. It is a flowchart which shows the flow of a process of the spatial handwriting input system which concerns on one Embodiment of this invention.

(Example)
Hereinafter, embodiments of the spatial handwriting input system of the present invention will be described with reference to the drawings. The embodiments shown below are preferred specific examples of the spatial handwriting input system of the present invention, and various limitations are attached according to the hardware configuration and structure of general information processing devices, sensors, display devices, and the like. However, the technical scope of the present invention is not limited to these embodiments unless specifically described to limit the present invention. In addition, the constituent elements in the embodiments described below can be appropriately replaced with existing constituent elements, and various variations including combinations with other existing constituent elements are possible. Therefore, the description of the embodiment described below does not limit the contents of the invention described in the claims.

(Configuration overview)
FIG. 1 is a diagram schematically showing an example of the configuration of a spatial handwriting input system according to the present invention. The spatial handwriting input system 1 in this embodiment includes a sensor 101, an information processing device 102, and a display device 103. When a user moves a pointer on a virtual plane 104 set in a three-dimensional space and handwrites a character or a figure, the information processing apparatus 102 uses a method described later based on an operation trajectory 106 of the pointer 105 acquired via the sensor 101. Is used to determine the display trajectory 108 and display it on a GUI (Graphical User Interface) screen 107 of the display device 103.

  The sensor 101 only needs to be capable of detecting the three-dimensional coordinates of the pointer 105 that is the detection target within a predetermined range of space. For example, a sensor including an infrared camera and an infrared emitting LED can be used. Preferably, the sensor 101 can detect the shape of a human finger used as a pointer, can recognize the tip of the finger as a feature point, and detect the coordinates thereof. The sensor 101 is connected to the information processing apparatus 102 by wire or wireless.

  The information processing apparatus 102 only needs to be capable of information processing described later, and includes a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), an external memory, an input I / F, and an output. A personal computer (PC) equipped with an I / F or the like, a portable terminal, or the like can be used.

  The CPU generally controls each device connected to the system bus according to a program stored in a ROM or the like that is a storage unit. The RAM functions as a main memory and work area of the CPU, and is also used as an input information expansion area and an environment data storage area. The ROM stores various programs and data. The input I / F controls input from the sensor 101. The output I / F controls screen display on the display device 103. Depending on the configuration, audio output to the speaker may be controlled. The external memory functions as a storage medium that can be stored or read, and stores an operating system (OS), a Web browser, and an application. An application and a program of each module (software) are stored in an external memory, read into a RAM as necessary, and executed by a CPU. Thereby, the function of the application or each module (software) is realized. The processing described in the present embodiment is realized by loading a program recorded in the external memory into the RAM and executing it by the CPU. Note that the program may be stored in RAM or ROM in addition to the external memory.

  In the present embodiment, the sensor 101, the information processing apparatus 102, and the display apparatus 103 are described as separate components. For example, the sensor 101 may be built in the information processing apparatus 102, or the information processing apparatus You may comprise as HMD (Head Mounted Display) with which 102 and the display apparatus 103 were integrated.

(Software configuration)
FIG. 2 is a functional block diagram illustrating processing performed by the spatial handwriting input system according to the present embodiment. Each component shown in FIG. 2 is realized by the CPU of the information processing apparatus 102 shown in FIG. 1 loading the program stored in the ROM or the external memory into the RAM, and the CPU executing the loaded program. The

  The coordinate detection unit 202 detects the three-dimensional coordinates of the movement locus of the pointer in a predetermined space based on the measurement data sent from the sensor 101 after the sensor 201 is activated. Preferably, at least one point of the pointer is recognized as a feature point, and the coordinates of the feature point can be detected. More preferably, the elongated portion of the pointer to be detected or the tip of the rod-like portion is recognized as a feature point, and its coordinates can be detected. For example, when the pointer is a human hand, the tip of each finger can be recognized as a feature point when the palm of the hand is widened, but when only one finger is extended by holding a fist, the tip of the finger Only the part can be recognized as the feature point. Note that the coordinate detection unit 202 may be provided in the sensor 201.

  The virtual plane setting unit 203 sets a virtual plane at an arbitrary position, size, and angle in the space of the detection range of the sensor 201 and stores the coordinates as virtual plane information 204. The setting of the virtual plane may be automatically performed when the sensor is activated, or may be set according to a specific operation of the user after the sensor is activated.

  The coordinate conversion unit 205 converts the three-dimensional coordinates (initial coordinates) representing the movement locus of the pointer detected by the coordinate detection unit 202 into coordinates (conversion coordinates) based on the set virtual plane. FIG. 3 shows an example of the relationship between the initial coordinate axis and the converted coordinate axis in the present embodiment. The coordinate detection unit 202 acquires the three-dimensional coordinates of the movement locus of the pointer on the preset coordinate axis X′Y′Z ′ (initial coordinates). For processing such as acquisition of an input trajectory and contact degree, which will be described later, the coordinate conversion unit 205 refers to the virtual plane information 204 and uses the three-dimensional coordinates (X′Y′Z ′ coordinates) detected on the initial coordinate axis. Then, the XY axis is taken on the virtual plane, and converted to the coordinate (XYZ coordinate) on the conversion coordinate axis taking the Z axis in the direction orthogonal to the virtual plane.

  The input trajectory acquisition unit 206 refers to the X-axis and Y-axis values of the coordinates after conversion of the movement trajectory of the pointer, and acquires two-dimensional coordinates on the XY plane as an input trajectory. On the other hand, the contact degree acquisition unit 207 refers to the Z-axis value of the coordinates after conversion of the pointer movement locus, calculates the contact degree with respect to the virtual plane using the value, and obtains it in association with the input locus. In the present embodiment, the degree of contact is calculated so as to increase from the near side of the virtual plane toward the far side.

  The display unit 208 determines a display locus that is actually output to the GUI 211 based on the input locus and the degree of contact. Specifically, when the contact degree exceeds a preset threshold value, the input locus associated with the contact degree exceeding the threshold value is set as the display locus, and the display locus expression is changed according to the height of the contact degree. . The display trajectory information 210 determined in this way is used for output to the GUI 211.

  Further, the display unit 208 acquires the position of the current pointer on the XY plane and the distance to the virtual plane based on the input trajectory and the contact degree, and stores them as indicator display information 209. Based on the indicator display information 209, the display unit 208 displays an indicator on the GUI 211 as a visual feedback for the user to grasp the position of the virtual plane.

  FIG. 4 is a diagram illustrating the relationship between the virtual plane, the pointer, and the contact degree. FIGS. 4A and 4B are a perspective view and a side view, respectively, showing a state where the pointer is in contact with the virtual plane. The pointer 402 approaches the set virtual plane 401, and the feature point 403 is located on the back side by the depth 404 from the virtual plane 401. In FIG. 4, the threshold value of the contact degree for determining the display locus is set as a value at the intersection with the virtual plane, and the feature point 403 has a contact degree exceeding this threshold value. FIG. 4C is a diagram illustrating the relationship between the depth of the pointer and the contact degree with respect to the virtual plane. In FIG. 4C, the depth b in the Z-axis direction indicates the above-described threshold value, and in this example indicates the intersection with the virtual plane. When the pointer (feature point) is located behind the depth b, the contact degree is higher than the threshold value, and the input locus is determined as the display locus. Further, the closer the depth in the Z-axis direction is to b to c, the higher the contact degree becomes, and the upper limit is reached at the depth c. When the depth c is exceeded, the degree of contact thereafter becomes constant. On the other hand, when the pointer (feature point) is on the near side of the depth b, the contact degree falls below the threshold value, and the input locus at this time is not determined as the display locus. As the depth in the Z-axis direction approaches b to a, the degree of contact decreases, and when the depth is a, the lower limit is reached. In this embodiment, the threshold value of the contact degree is set to the value of the intersection with the virtual plane. However, in another embodiment, the threshold value of the contact degree can be set before or behind the virtual plane. . In this example, the depths a and c are set as arbitrary values for determining the upper limit and the lower limit of the contact degree. However, the depths a and c may not be set in another embodiment. Furthermore, although the contact degree is described as linearly changing according to the depth in the Z-axis direction in FIG. 4C, the contact degree can be changed in a curve. For example, it is possible to set so that the change in the contact degree is small within a certain range from the threshold, and the contact degree is greatly changed when the certain range is exceeded. It is also possible to prepare a plurality of types of changes in the degree of contact so that the user can select them by an operation on the GUI screen.

  FIG. 5 is a diagram illustrating the relationship between the input trajectory, the contact degree, and the display trajectory. When the input trajectory acquisition unit 206 acquires an input trajectory composed of two-dimensional coordinates, the contact degree acquisition unit 207 also acquires the degree of contact associated with the two-dimensional coordinates of the input trajectory. The display unit 208 determines the display trajectory and its expression based on the input trajectory and the contact degree. In FIG. 5, the degree of contact between the start and end portions of the input locus is below the threshold value, and therefore the start and end portions of the input locus do not appear in the display locus. Further, the contact degree of the input trajectory increases from the start end portion to the intermediate portion and decreases again from the intermediate portion to the end portion. Reflecting such a change in the degree of contact, the line of the display trajectory becomes thicker from the start end to the middle portion, and the line becomes thinner again from the middle portion to the end portion. Thus, in the embodiment in which the thickness of the line of the display trajectory is changed in accordance with the change in the contact degree, the character can be touched lightly as in writing a character or a picture with a brush on a real plane such as a paper medium. It is possible to allow the user to experience the feeling of “writing pressure” in which a thin line is drawn and a thick line is drawn if pressed firmly. Note that the change in the expression of the display locus according to the contact degree is not limited to the change in the thickness of the line. In another embodiment, the color intensity of the line may be changed according to the contact degree, or the line thickness and the color intensity may be changed according to the contact degree.

  FIG. 6 is a diagram illustrating an example of a change in the indicator expression method according to the contact degree. In this example, when the pointer gets close to the virtual plane to some extent and the contact level reaches the arbitrarily set lower limit value, a circle with only a contour line is displayed on the GUI screen as an indicator, and the position of the pointer on the XY plane is displayed by the user. Notify When the pointer gets closer to the virtual plane and the contact degree exceeds the lower limit, a new circle colored within the circumference is generated at the center of the contour circle. As the pointer approaches the virtual plane and the contact degree approaches the threshold value, the colored circle is displayed gradually larger so that the outline circle and the colored circle overlap when the contact degree reaches the threshold value. In this way, the position on the XY plane is indicated by the position of the indicator, and the distance to the virtual plane is indicated by a change in the expression of the indicator, so that the user can determine the three-dimensional positional relationship between the pointer and the invisible virtual plane. It becomes possible to grasp visually. Note that the color, size, shape, transparency, etc. of the indicator may be changed instead of or in addition to the change in the method of expressing the indicator for visual feedback to the user.

(flowchart)
FIG. 7 is a flowchart showing a process flow of the spatial handwriting input system according to the present embodiment. After the sensor 201 is activated, a virtual plane is set in a space within the detection range of the sensor 201 in step 700. The virtual plane may be set automatically in association with the activation of the sensor 201 or may be set according to a specific operation of the user. When the user moves the pointer in the detection space, in step 702, the three-dimensional coordinates of the movement locus of the pointer are acquired. In step 704, the acquired three-dimensional coordinates are converted into the coordinates of the coordinate axes with reference to the virtual plane. Next, in step 706, the input trajectory and the contact degree are acquired from the converted coordinates.

  Next, in step 708, it is determined whether or not the contact degree exceeds a predetermined lower limit value. If no contact degree exceeding the lower limit value is detected, the process is repeated from step 702 again. When a contact degree exceeding the lower limit is detected, in step 710, an indicator for visually feeding back the positional relationship between the pointer and the virtual plane to the user is displayed on the GUI screen. Thereafter, in step 712, it is determined whether or not the pointer further approaches the virtual plane and the contact degree exceeds the threshold value. If the contact degree exceeding the threshold value is not detected, the display of the indicator in step 710 is continued. When the contact degree exceeding the threshold is detected, in step 714, the input trajectory associated with the contact degree exceeding the threshold is displayed on the GUI screen as a display trajectory expressed in accordance with the contact degree.

  After that, a continuous display trajectory is displayed until it is detected in step 716 that the contact degree is lower than the threshold value. When it is detected that the contact degree is lower than the threshold value, in step 718, the display trajectory is displayed. Display ends. This operation appears as a line break displayed on the GUI screen. Thereafter, in step 720, it is determined whether or not the pointer has moved away from the virtual plane and the degree of contact has fallen below a lower limit value. As long as the contact degree does not fall below the lower limit value, the indicator display in step 710 is continued. When it is detected that the contact degree falls below the lower limit value, the indicator display ends in step 722. The above processing is repeatedly performed during activation of the present system.

  As described above, according to the present embodiment, the degree of contact with the virtual plane is calculated from the position information in the depth direction (Z-axis direction) of the pointer with respect to the virtual plane to determine the display trajectory, and the line thickness is determined according to the degree of contact. Since it is possible to change the expression of the display trajectory such as the color and the color density, the width of the output expression in the handwriting input system using the virtual plane is expanded, and the amount of information in the output can be increased. In addition, it is possible to visually feedback the distance between the pointer and the virtual plane to the user, making it easier for the user to grasp the distance to the virtual plane, enabling more natural input and accurate input. It is expected to contribute to the improvement of performance.

(Other embodiments)
The preferred embodiments of the present invention have been described above, but the present invention is not limited to these embodiments. An object of the present invention is to supply a storage medium storing a program code (computer program) for realizing the functions of the above-described embodiments to a system or apparatus, and a program in which the computer of the supplied system or apparatus is stored in the storage medium. It is also achieved by reading and executing the code. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention. In the above-described embodiment, the information processing apparatus functions as each processing unit by executing a program. However, part or all of the processing is configured by a dedicated electronic circuit (hardware). It doesn't matter. The present invention is not limited to the specific embodiments described, but can be variously modified or changed within the scope of the gist of the present invention described in the claims.

101 Sensor 102 Information Processing Device 103 Display Device 104 Virtual Plane 105 Pointer 106 Operation Trajectory 107 GUI Screen 108 Display Trajectory

Claims (7)

A coordinate detection unit for detecting the three-dimensional coordinates of the movement trajectory of the pointer in the three-dimensional space, a virtual plane setting unit for setting a virtual plane corresponding to a GUI (Graphical User Interface) screen in the three-dimensional space,
A coordinate conversion unit that converts the three-dimensional coordinates of the movement trajectory of the pointer into coordinates that take an X axis and a Y axis on the virtual plane and take a Z axis in a direction orthogonal to the XY plane;
An input trajectory acquisition unit that acquires a trajectory on the XY plane as an input trajectory among the movement trajectories of the pointer;
A contact degree acquisition unit that calculates a contact degree with respect to the virtual plane from a position in the Z-axis direction out of the movement locus of the pointer and acquires the contact degree in association with the input locus;
When the degree of contact exceeds a predetermined threshold, the input locus is displayed on the GUI screen as a display locus, and the pointer position on the XY plane and the distance in the Z-axis direction to the virtual plane are indicated. A display unit for displaying an indicator for displaying on the GUI screen,
The display unit changes the expression of the display locus according to the contact degree.
Spatial handwriting input system. The threshold is set as a value of the contact degree at the intersection with the virtual plane,
The spatial handwriting system according to claim 1, wherein the contact degree is calculated so as to increase from the near side to the far side of the virtual plane.   The spatial handwriting input system according to claim 2, wherein the display unit increases a thickness of a line of the locus to be displayed as the contact degree is higher.   The spatial handwriting system according to claim 2, wherein the display unit increases a color density of a line of the locus to be displayed as the contact degree is higher. The display unit displays an indicator indicating the current position of the pointer on the XY plane on the GUI screen,
The indicator changes in color, shape, size, or expression method according to the degree of contact.
The spatial handwriting system of any one of Claims 1-4. A coordinate detection step of detecting the three-dimensional coordinates of the movement locus of the pointer in the three-dimensional space;
A virtual plane setting step of setting a virtual plane corresponding to a GUI (Graphical User Interface) screen in the three-dimensional space;
A coordinate conversion step of converting the three-dimensional coordinates of the movement trajectory of the pointer into coordinates having an X axis and a Y axis on the virtual plane and a Z axis in a direction perpendicular to the XY plane;
An input trajectory acquisition step of acquiring, as an input trajectory, a trajectory on the XY plane of the movement trajectory of the pointer;
A contact degree acquisition step of calculating a contact degree with respect to the virtual plane from a position in the Z-axis direction out of the movement locus of the pointer and acquiring the contact degree in association with the input locus;
A display step of displaying the input locus as a display locus on the GUI screen when the contact degree exceeds a predetermined threshold;
Displaying on the GUI screen an indicator for indicating a position of the pointer on the XY plane and a distance in the Z-axis direction to the virtual plane,
The display step changes the expression of the display trajectory according to the contact degree.
Spatial handwriting input method.   A computer program for causing a computer to execute the method according to claim 6.

Images