JP2013003961A - Spatial handwriting system and electronic pen - Google Patents

Abstract

PROBLEM TO BE SOLVED: To represent a break of a stroke through natural operation, and to display/save a handwriting image on which the break of the stroke is accurately reflected.SOLUTION: There is provided an electronic pen capable of communicating with external equipment, the electronic pen including a coordinate detection part which detects coordinates of a pen tip in a three-dimensional space; a virtual plane generation part which generates a virtual plane based upon a point set in the space; a stroke detection part which detects a movement of the pen tip in a normal direction of the virtual plane based upon change of coordinates in the three-dimensional space, and recognizes that a stroke is broken when a movement quantity, a moving speed or moving acceleration of the pen tip in the normal direction exceeds a predetermined threshold; a coordinate conversion part which converts coordinates in the three-dimensional space which show a track of the pen tip corresponding to a continuous stroke from a break of the stroke to a next break into plane coordinates based upon the specific point on the virtual plane as the origin; and a communication part which outputs information on the converted plane coordinates to the external equipment.

Description

  The present invention relates to a spatial handwriting system and an electronic pen, and more particularly to a spatial handwriting system that enables display / saving of handwritten characters and figures on a virtual plane and an electronic pen that handwrites characters and figures on a virtual plane.

  As a method for writing down ideas, a method using a pen and a notebook is generally used. However, this method is inconvenient because you always have to carry a pen and notebook to write down ideas that you do not know where to come up. From such a background, a technique for displaying or storing an image handwritten in the air using an electronic pen has been proposed.

  For example, in Patent Document 1 below, an image input unit that inputs a written image obtained by handwriting on a virtual surface set in a desired space, and a handwritten locus is extracted from the written image by image processing. Handwriting extracting means for obtaining an image; relative coordinate detecting means for detecting relative coordinates relating to a writing position of the handwritten image; image recording means for recording the handwritten image together with the relative coordinates; and image display means for displaying the handwritten image And a handwritten image display device having at least a transflective optical element that reflects light from the image display means and transmits light from a real field of view of the user's line of sight, and a user of the handwritten image display device The virtual image of the handwritten image displayed by the image display means is superimposed and displayed in a real field in front of the transflective optical element as viewed from the top. Handwritten image display system is disclosed that. Patent Document 2 below also discloses a similar handwritten image display system.

  Further, in Patent Document 3 below, a sensor mounted on a fingertip, a detection unit that sequentially detects a spatial coordinate corresponding to a spatial position accompanying the movement of the sensor, and a spatial coordinate sequentially detected by the detection unit, Measurement means for measuring the movement speed and acceleration of the sensor, extraction means for extracting a trace image drawn by the sensor based on the movement speed and acceleration measured by the measurement means, and extraction by the extraction means A spatial handwritten character graphic input device including a storage means for storing a trace image is disclosed.

  Patent Document 4 listed below is a system that performs control according to a gesture for a virtual object, and stores virtual object information that includes information representing the position of the virtual object, and a photographing unit that photographs the gesture for the virtual object. A storage means, a crossing degree specifying means for specifying a crossing degree for the virtual object of the operation unit by a gesture based on the captured image acquired by the photographing means and the virtual target information, and a specified crossing A control system is disclosed that includes output control means for performing an output corresponding to the degree to an operator.

JP 2006-154900 A JP 2006-154902 A JP-A-6-28096 JP 2010-205223 A

  As described above, Patent Documents 1 and 2 disclose techniques for acquiring a handwritten image by extracting a handwritten locus with a camera and displaying the handwritten image on a transparent type HMD (Head Mounted Display). When handwritten in space, the line breaks between characters and figures are not clear, so how to show the handwritten characters and figures to the user and how to make the user feel it is an important issue.

  In response to this problem, Patent Document 3 measures the movement speed and acceleration of a sensor that moves integrally with the fingertip, and if the speed falls below a certain speed, it is a break in the stroke (operation for drawing characters and figures). However, with this method, the speed of movement of the sensor changes greatly at the corners and the “splash” part of a character with a square or “splash” such as “mouth”. There is a problem that stroke breaks cannot be accurately determined depending on the shape of the character.

  In Patent Document 4, the display output is changed according to the degree of intersection between the finger and the virtual plane. However, in this method, the user cannot feed back the feeling of touching the virtual plane to the user. There is a problem that it is not possible to recognize the hand break and handwritten.

  The present invention has been made in view of the above-mentioned problems, and its main purpose is to represent stroke breaks with natural motion, and to display / write a handwritten image that accurately reflects stroke breaks. It is to provide a spatial handwriting system and an electronic pen that can be stored.

  In order to achieve the above object, the present invention provides a spatial handwriting system in which an electronic pen and a display device are communicably connected, and the electronic pen detects a coordinate in a three-dimensional space of a pen tip. And a virtual plane creation unit that creates a virtual plane based on a point set in the space, and detects a pen tip movement in the normal direction of the virtual plane based on a change in coordinates of the three-dimensional space. A stroke detection unit that recognizes a stroke break when the movement amount or movement speed or movement acceleration of the pen tip in the normal direction exceeds a predetermined threshold, and from the stroke break to the next break A coordinate conversion unit that converts coordinates in a three-dimensional space indicating the trajectory of the pen tip of a continuous stroke into plane coordinates having a specific point on the virtual plane as an origin, and information on the converted plane coordinates is displayed. And a communication unit for outputting the location, the display device receives the information of the plane coordinates, and displays the pen tip trajectory on the virtual plane on the screen.

  In addition, the present invention is an electronic pen that can communicate with an external device, and creates a virtual plane based on a coordinate detection unit that detects coordinates in a three-dimensional space of a pen tip and points set in the space. Based on a change in the coordinates of the virtual plane and the coordinates in the three-dimensional space, the movement of the pen point in the normal direction of the virtual plane is detected, and the movement amount, movement speed, or movement acceleration of the pen point in the normal direction When the stroke exceeds a predetermined threshold, a stroke detection unit that recognizes as a stroke break, and coordinates in a three-dimensional space indicating the locus of the pen tip of a continuous stroke from the stroke break to the next break, A coordinate conversion unit that converts the coordinate point of the virtual plane into a plane coordinate having an origin, and a communication unit that outputs information of the converted plane coordinate to the external device.

  According to the spatial handwriting system and the electronic pen of the present invention, stroke breaks can be expressed by natural operations, and a handwritten image in which the stroke breaks are accurately reflected can be displayed / saved.

  The reason is that when the electronic pen transmits coordinate information indicating the movement locus of the pen tip on the virtual plane created in the three-dimensional space, the amount of movement of the pen tip in the direction orthogonal to the virtual plane (normal direction) This is because stroke breaks are detected on the basis of the moving speed and acceleration, and continuous strokes (lines drawn continuously) can be distinguished.

It is a figure which shows typically the structure of the spatial handwriting system which concerns on one Example of this invention. It is a block diagram which shows the main structures of the electronic pen which concerns on one Example of this invention. It is a figure explaining the relationship between an absolute coordinate axis and a virtual plane coordinate axis. It is a figure explaining operation | movement of the electronic pen on a virtual plane. It is a figure which shows the example which produces a virtual plane from three points. It is a figure which shows the example which produces a virtual plane from two points. It is a figure which shows the other example which produces a virtual plane from two points. It is a figure which shows the example which produces a virtual plane from a character stroke. It is a flowchart figure which shows operation | movement of the spatial handwriting system (electronic pen) which concerns on one Example of this invention.

  As shown in the background art, a technique for displaying the movement locus of the pen tip when handwritten with an electronic pen in a three-dimensional space on a display device such as a head-mounted display has been proposed. Depending on the shape, there is a problem in that it is impossible to accurately determine stroke breaks, and it is impossible to feed back to the user the feeling of touching the virtual plane.

  Therefore, in one embodiment of the present invention, first, the coordinates of the pen tip of the electronic pen are detected in the three-dimensional space to identify one or a plurality of points, and two points are placed in the three-dimensional space based on the identified points. Create a dimensional virtual plane. For example, two points in a three-dimensional space are specified, a surface is created vertically or horizontally from a straight line connecting the two points, and the surface is set as a virtual plane. Alternatively, a rectangular area whose diagonals are the two points is set as a virtual plane, a plane specified by one point in the three-dimensional space and the angle of the central axis of the electronic pen is set as a virtual plane, A surface specified by three points is set as a virtual plane, or a surface specified by the locus of the first stroke is set as a virtual plane.

  Then, the character or figure is handwritten by moving the electronic pen on the virtual plane created by the above method. However, if the stroke break is unclear, the handwritten character or figure cannot be displayed accurately. . On the other hand, as a method of expressing stroke breaks, when using a method that is greatly different from handwriting with a pen on paper (or handwriting with a touch pen on a touch panel), handwriting in space like handwriting on a real plane Can not be used, and the usability deteriorates.

  Therefore, when transmitting coordinate information indicating the movement trajectory of the pen tip on the virtual plane, the pen tip is moved in the z-axis direction (normal direction of the virtual plane) with respect to the virtual plane (preferably on the real plane). If it is detected that the stroke is a stroke break when it is detected (hand-drawn as in handwriting), the stroke of the pen tip can be distinguished as a continuous stroke (line drawn continuously) until the stroke break. To. For example, send coordinate information in units of consecutive strokes, send coordinate information with information indicating stroke breaks, send handwritten image data created by combining multiple consecutive strokes, etc. To do.

  In addition, according to the operation in the z-axis direction (normal direction) with respect to the virtual plane of the pen tip, a feedback function such as vibrating the electronic pen or generating sound or light is provided so that the pen tip can be moved from the virtual plane. The user is explicitly notified that he / she has left or that the pen tip is in contact with the virtual plane.

  As a result, the user can express stroke breaks with natural motion, and can reliably recognize stroke breaks. A handwritten image in which stroke breaks are accurately reflected can be displayed on a display device such as an HMD that can communicate with the electronic pen, or handwritten image data in which stroke breaks are accurately reflected can be communicated with the electronic pen. It can be saved on a server.

  In order to describe the above-described embodiment of the present invention in more detail, a spatial handwriting system and an electronic pen according to an embodiment of the present invention will be described with reference to FIGS. FIG. 1 is a diagram schematically showing the configuration of the spatial handwriting system of the present embodiment, and FIG. 2 is a block diagram showing the main configuration of the electronic pen of the present embodiment. FIG. 3 is a diagram for explaining the relationship between the absolute coordinate axis and the virtual plane coordinate axis, and FIG. 4 is a diagram for explaining the operation of the electronic pen on the virtual plane. 5 to 8 are diagrams illustrating examples of creating a virtual plane, and FIG. 9 is a flowchart illustrating the operation of the spatial handwriting system (electronic pen) according to the present embodiment.

  In the present embodiment, a stroke refers to an operation for drawing a character or a figure, and a stroke break refers to a place where an arbitrary line constituting the character or figure and the next line are not drawn. These two lines may be lines constituting one character or figure, or may be lines constituting different letters or figures. A continuous stroke refers to a continuous line between any cut (including the drawing start position) and the next cut (including the drawing end position) of the stroke, and a plurality of continuous strokes are combined. Things are called handwritten images.

  As shown in FIG. 1A, the spatial handwriting system 10 of the present embodiment includes an electronic pen 20 and a display device 30, which are short-range wireless communication represented by Bluetooth (registered trademark) and infrared rays. Connected by communication, wired communication, etc.

  The schematic operation of the space handwriting system 10 will be described with reference to FIG. 1B. When the user moves the electronic pen 20 on the virtual plane created in the three-dimensional space and handwrites characters, figures, and the like, the electronic pen 20 Outputs coordinate information indicating the movement locus of the pen tip. At that time, since the movement trajectory of the pen tip is connected like a single stroke, the handwritten characters and figures cannot be accurately represented on the virtual plane. Is used to express stroke breaks so that the movement trajectory of the pen tip up to the stroke break can be recognized as a continuous stroke.

  On the other hand, when receiving the coordinate information from the electronic pen 20, the display device 30 (here, the sunglasses-type HMD) sequentially displays successive strokes based on the coordinate information, and browses handwritten characters and figures on the virtual plane. It can be so. Further, as necessary, the server 40 receives and stores coordinate information from the electronic pen 20, and uses a computer device or the like connected to be able to communicate with the server 40 to write handwritten characters and figures on the virtual plane. Make it available.

  The electronic pen 20 only needs to have a function of transmitting coordinate information indicating the movement trajectory of the pen tip, and may have a function as a writing tool such as a ballpoint pen, or a function as a writing tool. It does not have to be provided. Moreover, the electronic pen 20 is not limited to a shape like a ball-point pen, and may be any shape as long as the tip can be recognized. For example, the electronic pen 20 may have a shape like a cap fitted to a fingertip.

  1 shows a sunglasses-type HMD as the display device 30, the display device 30 may be any portable device that can display a handwritten image based on coordinate information received from the electronic pen 20. For example, a mobile phone or a tablet terminal may be used. In addition, the display device 30 may have a function of storing handwritten image data as well as displaying a handwritten image. In this case, the server 40 may be omitted.

  In FIG. 1A, the display device 30 is included in the spatial handwriting system 10, but the display device 30 may be omitted when it is not necessary to view the handwritten image on the spot. In that case, the coordinate information transmitted from the electronic pen 20 may be stored in the server 40, or the coordinate information may be stored in the storage unit of the electronic pen 20.

  FIG. 2 is a block diagram showing the main configuration of the electronic pen 20. The electronic pen 20 of this embodiment includes an operation unit 21, a sensor 22, a three-dimensional coordinate detection unit 23a, a three-axis acceleration detection unit 23b, a movement amount detection unit 23c, a virtual plane and data creation unit 24, The stroke detection unit 25, the plane coordinate conversion unit 26, the virtual surface ground feedback unit 27, the communication unit 28, and a power source (storage battery) (not shown) are included.

  The operation unit 21 includes switches, buttons, and the like. The electronic pen 20 is turned on / off, a handwriting operation start instruction, a plane creation mode (a mode for creating a virtual plane based on a specified point), and a drawing mode. It is possible to perform operations such as a switching instruction with (a mode in which a pen tip is moved on a virtual plane and a character or a figure is handwritten).

  The sensor 22 is a sensor that detects the coordinates and acceleration of the pen tip in a three-dimensional space, the inclination of the central axis of the electronic pen 20, and the like, and is configured by, for example, a three-axis gyro sensor and a three-axis acceleration sensor.

  The three-dimensional coordinate detection unit 23a detects the coordinates of the pen tip in a three-dimensional space based on the output of the sensor 22 (three-axis gyro sensor).

  The triaxial acceleration detector 23b detects the triaxial acceleration of the pen tip based on the output of the sensor 22 (triaxial acceleration sensor).

  The movement amount detection unit 23c detects the movement amount of the pen tip per unit time (that is, the movement speed of the pen tip) based on the coordinates of the three-dimensional space detected by the three-dimensional coordinate detection unit 23a.

  The virtual plane and data creation unit 24 creates a two-dimensional virtual plane in the three-dimensional space based on a point, a stroke, a tilt of the central axis of the electronic pen 20 specified by the electronic pen 20. . In addition, handwritten image data is created by combining the trajectories of successive strokes divided at stroke breaks.

  The stroke detection unit 25 compares the movement amount, movement speed, and acceleration of the nib in the z-axis direction (normal direction of the virtual plane) with respect to the two-dimensional virtual plane by comparing a predetermined threshold with a stroke break. To detect.

  The plane coordinate conversion unit 26 converts the three-dimensional coordinates of the pen tip up to the break of each stroke detected by the stroke detection unit 25 into two-dimensional coordinates based on a specific point (for example, upper left) on the virtual plane, and converts The obtained two-dimensional coordinates are notified to the virtual plane and data creation unit 24.

  The virtual surface contact feedback unit 27 vibrates a vibrator provided in advance in the electronic pen 20 when the stroke detection unit 25 detects a stroke break (that is, when the pen tip moves away from the virtual plane) or from a speaker. A sound is generated or a lamp is lit / flashed to notify the user of the fact and feed back the state where the pen tip is away from the virtual surface. In this embodiment, the state in which the pen tip is separated from the virtual plane is fed back. However, when the stroke detection unit 25 does not detect a stroke break (that is, the pen tip is in contact with the virtual plane). (When it is determined that characters or figures are being drawn), the user may be notified by vibration, sound, light, or the like, and the state where the pen tip is in contact with the virtual surface may be fed back. .

  The communication unit 28 establishes communication with the display device 30 and the server 40, transmits the two-dimensional coordinates converted by the plane coordinate conversion unit 26 to the display device 30, or creates a handwritten image created by the virtual plane and data creation unit 24. The data is transmitted to the server 40.

  FIG. 2 shows an example of the electronic pen 20 according to the present embodiment. The configuration of the electronic pen 20 is appropriately set as long as the stroke break can be detected based on the movement of the pen tip in the z-axis direction (normal direction of the virtual plane). It can be changed. For example, in FIG. 2, a three-dimensional coordinate detection unit 23a, a three-axis acceleration detection unit 23b, and a movement amount detection unit 23c are provided. However, the three-dimensional coordinate detection unit 23a determines a movement amount or movement from coordinate information associated with time. The configuration may be such that the velocity and the movement acceleration are calculated. In this case, the triaxial acceleration detection unit 23b and the movement amount detection unit 23c can be omitted. In FIG. 2, the virtual plane and data creation unit 24 is configured to create a virtual plane and handwritten image data. However, the virtual plane and data creation unit 24 is a virtual plane that creates a virtual plane. You may divide | segment into a creation part and the data creation part which produces the data of a handwritten image. Further, a storage unit is provided in the electronic pen 20 to store handwritten image data, and when the device capable of communicating with the electronic pen 20 is recognized or the storage battery is charged, the handwritten image data is transmitted to the device. It is good also as a structure.

  Also, a three-dimensional coordinate detection unit 23a, a three-axis acceleration detection unit 23b, a movement amount detection unit 23c, a virtual plane and data creation unit 24, a stroke detection unit 25, a plane coordinate conversion unit 26, a virtual plane grounding feedback unit 27, a communication unit 28 or the like may be configured as hardware or software.

  Hereinafter, a specific method of handwriting on a virtual plane using the electronic pen 20 having the above configuration will be described.

  First, the relationship between the three-dimensional space and the virtual plane will be described. FIG. 3 is a diagram for explaining the relationship between the absolute coordinate axes and the virtual plane coordinate axes. The three-dimensional coordinates (XYZ coordinates) detected by the sensor 22 are converted into planes of the virtual plane by the plane coordinate conversion unit 26. It is converted into coordinates (X′Y′Z ′ coordinates) with the vertical direction (normal direction) to the virtual plane as the z axis.

  FIG. 4 is a diagram for explaining the operation of the electronic pen 20 on the virtual plane. As shown in FIG. 4A, the stroke detection unit 25 determines the distance of the electronic pen 20 from the virtual plane based on the movement amount, movement speed, acceleration, and the like of the electronic pen 20 in the Z′-axis direction. The stroke break is detected from the determination result. At this time, since the user cannot accurately recognize the virtual plane, a threshold is set for the movement amount, the movement speed, the acceleration, and the like, and a stroke break is determined based on the threshold.

  For example, as shown in FIG. 4B, when the moving speed (the time during which the pen tip moves from the first boundary to the second boundary) or the moving acceleration is greater than or equal to a predetermined value, it is determined that there is a stroke break. Up to the X′Y ′ coordinate at the boundary time is recognized as a continuous stroke. When moving the pen tip quickly, a method of determining stroke breaks based on the moving speed or the moving acceleration is effective. However, when the pen tip is moved largely, the coordinates of the pen tip are set to threshold values (for example, the first It can also be determined that the stroke is broken when the boundary is exceeded.

  Next, a method for creating a two-dimensional virtual plane in a three-dimensional space will be described with reference to FIGS.

  FIG. 5 shows an example of creating a virtual plane using three points. In this example, when the user operates the operation unit 21 of the electronic pen 20 (for example, presses a predetermined button) to set three points, the virtual plane and data creation unit 24 connects the point 1 and the point 2. Specify a two-dimensional plane circumscribing points 1, 2, and 3 with the straight line as the X ′ axis, the straight line in the direction of the point 3 with the point 1 as the origin, and the straight line in the direction of the point 3 as the Y ′ axis. The dimension plane is a virtual plane.

  FIG. 6 shows an example in which a virtual plane is created using two points. In this example, similarly, when the user operates the operation unit 21 of the electronic pen 20 to set two points, the virtual plane and data creation unit 24 uses the straight line connecting the points 1 and 2 as the X ′ axis. A two-dimensional plane having a length in the X′-axis direction from point 1 to point 2 and a predetermined length in the Y-axis direction is specified, and the two-dimensional plane is set as a virtual plane.

  FIG. 7 shows another example in which a virtual plane is created using two points. In this example, similarly, when the user operates the operation unit 21 of the electronic pen 20 to set two points, the virtual plane and data creation unit 24 uses the straight line connecting the points 1 and 2 as the X ′ axis. A two-dimensional plane having a length in the X′-axis direction from point 1 to point 2 and a predetermined length in the Z-axis direction is specified, and the two-dimensional plane is set as a virtual plane.

  FIG. 8 shows an example of creating a virtual plane using a stroke. In this example, the first coordinate (point 1) at the time of activation of the electronic pen 20 is set as the origin, and a straight line connecting the coordinates of the pen tip (point 2) detected after a predetermined time has elapsed is defined as the X ′ axis. Further, the center line of the electronic pen 20 is taken as the Z ′ axis, and straight lines each having an angle of 90 degrees with respect to the X ′ axis and the Z ′ axis are taken as the Y ′ axis. Then, a two-dimensional plane having a predetermined length in the X′-axis direction and the Y′-axis direction is specified with the origin as a base point, and the two-dimensional plane is set as a virtual plane.

  Note that the method of creating the virtual plane is not limited to FIGS. For example, in the above description, a virtual plane is created using two or three points, but it is also possible to create a virtual plane using one point. In this case, a surface (or a surface inclined at a predetermined angle) that passes through the first coordinates (point 1) at the time of activation of the electronic pen 20 and is orthogonal to the center line of the electronic pen 20 is specified, and the surface is defined as a virtual plane. You can also

  Hereinafter, the operation of the spatial handwriting system 10 (electronic pen 20) of the present embodiment will be described with reference to the flowchart of FIG. In the following description, it is assumed that the system shown in FIG. 1B is used, and data is output every time a stroke break is detected.

  First, the virtual plane and data creation unit 24 determines whether a predetermined switch of the operation unit 21 (an activation switch that instructs the start of a handwriting operation) is ON (S101). When the activation switch is ON, the virtual plane and data creation unit 24 determines whether the mode set by the user operating the operation unit 21 is a plane creation mode for creating a virtual plane, or characters and figures on the virtual plane. It is determined whether or not the drawing mode is for handwriting (S105).

  In the plane creation mode, the virtual plane and data creation unit 24 determines whether a point is designated (S106). If the point is not specified, the process waits for the point to be specified. If the point is specified, the virtual plane and data creation unit 24 uses the method shown in FIGS. Is created (S107). Then, the generated coordinate axis information (origin position, coordinate axis direction, virtual plane size, etc.) of the virtual plane is stored and notified to the stroke detection unit 25 and the plane coordinate conversion unit 26 (S108), and the process returns to S101.

  On the other hand, in the drawing mode, the virtual plane and data creation unit 24 determines whether the virtual plane has been set (S109). If the virtual plane has not been set, the process returns to S105, and the plane creation mode is set. Wait for it. When the virtual plane has been set, the three-dimensional coordinate detector 23a and the three-axis acceleration detector 23b detect the three-dimensional coordinates and the three-axis acceleration based on the output of the sensor 22, respectively, and the movement amount detector 23c. Detects the movement amount (movement speed) of the pen tip per unit time based on the three-dimensional coordinates detected by the three-dimensional coordinate detection unit 23a (S110).

  Next, the plane coordinate conversion unit 26 sequentially converts the three-dimensional coordinates detected by the three-dimensional coordinate detection unit 23a into coordinates on a virtual plane (S111). Then, the stroke detection unit 25 compares the movement amount, movement speed, and movement acceleration of the virtual plane coordinates in the z-axis direction with predetermined threshold values to determine whether a stroke break has occurred (S112). If no stroke break occurs, the process returns to S110 and the same processing is repeated.

  On the other hand, when a stroke break occurs, the plane coordinate conversion unit 26 stores the X′Y ′ coordinate of the pen tip converted into the coordinate of the virtual plane as a trajectory of a continuous stroke (S113). Then, if necessary, the virtual surface ground feedback unit 27 vibrates the electronic pen 20 according to the degree of intersection with the virtual plane (the movement amount, movement speed, and movement acceleration in the z-axis direction), and generates sound and light. Or causing the user to be aware of stroke breaks (S114).

  Thereafter, the virtual plane and data creation unit 24 plots the X′Y ′ coordinates of the pen tip of the continuous stroke with a predetermined position (for example, upper left) on the virtual plane as the origin, and indicates the locus of the continuous stroke. Is created (S115), and the created data is transmitted to the display device 30 (S116). The display device 30 displays a locus of continuous strokes on the screen based on the received data.

  Until the start switch is turned off, the data divided at the stroke break is transmitted to the display device 30. When the start switch is turned off, the virtual plane and data creation unit 24 combines the trajectories of a plurality of continuous strokes. Then, handwritten image data is created, and attributes such as virtual plane creation date and time information, identification information of the electronic pen 20, position information and date and time information at the time of drawing are associated with the handwritten image data (S102). Then, it is determined whether there is untransmitted handwritten image data (S103). If there is untransmitted handwritten image data, the handwritten image data is transmitted to the server 40 (S104). The image data is stored in association with the attribute.

  In the above flow, the electronic pen 20 generates data indicating the locus of the continuous stroke and transmits it to the display device 30. However, the coordinate information is sequentially transmitted until the stroke break is detected, and the stroke of the stroke is detected. While the break is detected, the transmission of the coordinate information may be interrupted, and the coordinate information may be transmitted again when drawing on the virtual plane is started. Further, the time-series pen tip X′Y ′ coordinate and information indicating stroke breaks are transmitted to the display device 30, and a plurality of continuous stroke trajectories are identified based on the information received by the display device 30. It is good also as a structure to display. The electronic pen 20 may be configured to transmit handwritten image data to the display device 30 after combining a plurality of continuous stroke trajectories to create handwritten image data.

  In this way, when transmitting coordinate information indicating the movement trajectory of the pen tip on the two-dimensional virtual plane, based on the movement in the z-axis direction with respect to the virtual plane (preferably, the movement toward the front side). By determining the stroke break, the handwritten image can be divided into continuous strokes by a natural operation. Further, by vibrating the electronic pen 20 or generating sound or light at the stroke break, the user can recognize the stroke break and can draw an accurate handwritten image. Also, by displaying the trajectory of the electronic pen 20 on the virtual plane on the display device or storing the data of the handwritten image in a server or the like, it is possible to write down necessary information when necessary regardless of the location. And user convenience can be improved.

  In addition, this invention is not limited to the said implementation, The structure and control can be changed suitably, unless it deviates from the meaning of this invention.

  For example, in the above embodiment, it is determined that there is a stroke break when the electronic pen 20 is pulled toward the near side from a two-dimensional virtual plane. It may be determined that there is a stroke break when

  In the above embodiment, the stroke break is detected based on the z-axis movement of the pen tip. For example, instead of the z-axis movement or in addition to the z-axis movement. Thus, a stroke break may be detected based on a change in the tilt of the electronic pen 20.

  The present invention can be used for a system including an electronic pen capable of transmitting coordinate information indicating a movement locus of a pen tip.

DESCRIPTION OF SYMBOLS 10 Spatial handwriting system 20 Electronic pen 21 Operation part 22 Sensor 23a Three-dimensional coordinate detection part 23b Three-axis acceleration detection part 23c Movement amount detection part 24 Virtual plane document creation part 25 Character stroke detection part 26 Plane coordinate conversion part 27 Virtual surface grounding feedback Unit 28 communication unit 30 display device 40 server

Claims (10)

A spatial handwriting system in which an electronic pen and a display device are communicably connected,
The electronic pen is
A coordinate detector for detecting the coordinates of the nib in a three-dimensional space;
A virtual plane creation unit that creates a virtual plane based on points set in space;
Based on the change in the coordinates of the three-dimensional space, the movement of the pen tip in the normal direction of the virtual plane is detected, and the movement amount or movement speed or movement acceleration of the pen tip in the normal direction is set to a predetermined threshold value. A stroke detector that recognizes a stroke break when exceeded,
A coordinate conversion unit that converts coordinates in a three-dimensional space indicating the trajectory of a pen tip of a continuous stroke from the stroke break to the next break into plane coordinates having a specific point on the virtual plane as an origin;
A communication unit that outputs the information of the converted plane coordinates to the display device,
The spatial handwriting system, wherein the display device receives the information of the plane coordinates and displays a locus of a pen tip on the virtual plane on a screen. The communication unit of the electronic pen outputs information of the plane coordinates in units of continuous strokes,
2. The spatial handwriting system according to claim 1, wherein the display device displays a locus of a pen tip on the virtual plane on a screen in units of continuous strokes. The electronic pen further includes
3. The feedback unit according to claim 1, further comprising a feedback unit that notifies the user that the stroke break has been recognized by vibration, sound, or light when the stroke detection unit recognizes the stroke break. 4. Space handwriting system.   The virtual plane creation unit is configured with three set points, or two set points, or two set points and one axis of the three-dimensional space, or one set point and The spatial handwriting system according to any one of claims 1 to 3, wherein the virtual plane is created based on any one of angles of a central axis of the electronic pen. A server for storing data is communicatively connected to the electronic pen;
The electronic pen further includes
Combining the traces of the pen tip of a plurality of continuous strokes to create a handwritten image drawn on the virtual plane, the date and time information of the virtual plane, the drawing position information of the handwritten image in the data of the handwritten image A data creation unit for adding one or more attribute information selected from the drawing date and time information of the handwritten image and the identification information of the electronic pen;
The communication unit outputs data of the handwritten image to which the attribute information is added to the server,
The spatial handwriting system according to any one of claims 1 to 4, wherein the server stores the acquired handwritten image data in association with the one or more attribute information. An electronic pen that can communicate with an external device,
A coordinate detector for detecting the coordinates of the nib in a three-dimensional space;
A virtual plane creation unit that creates a virtual plane based on points set in space;
Based on the change in the coordinates of the three-dimensional space, the movement of the pen tip in the normal direction of the virtual plane is detected, and the movement amount or movement speed or movement acceleration of the pen tip in the normal direction is set to a predetermined threshold value. A stroke detector that recognizes a stroke break when exceeded,
A coordinate conversion unit that converts coordinates in a three-dimensional space indicating the trajectory of a pen tip of a continuous stroke from the stroke break to the next break into plane coordinates having a specific point on the virtual plane as an origin;
A communication unit that outputs information of the converted plane coordinates to the external device;
An electronic pen characterized by comprising:   The electronic pen according to claim 6, wherein the communication unit outputs information on the planar coordinates in units of continuous strokes.   Furthermore, when the said stroke detection part recognizes the stroke break, it is provided with the feedback part which notifies a user that the stroke break was recognized by vibration, sound, or light. Electronic pen described in 1.   The virtual plane creation unit is configured with three set points, or two set points, or two set points and one axis of the three-dimensional space, or one set point and The electronic pen according to any one of claims 6 to 8, wherein the virtual plane is created based on any one of angles of a central axis of the electronic pen. Further, the pen tip trajectories of a plurality of continuous strokes are combined to create a handwritten image drawn on the virtual plane, and the creation date and time information of the virtual plane and the handwritten image are drawn on the data of the handwritten image. A data creation unit for adding one or more attribute information selected from position information, drawing date / time information of a handwritten image, and identification information of the electronic pen;
The electronic pen according to claim 6, wherein the communication unit outputs the handwritten image data to which the attribute information is added to the external device.

Images