WO2023157556A1 - ペン及びペン位置検出装置を含むシステム、ペン位置検出装置、並びに、ペンに内蔵されるハプティクス素子を動作させる方法 - Google Patents

ペン及びペン位置検出装置を含むシステム、ペン位置検出装置、並びに、ペンに内蔵されるハプティクス素子を動作させる方法 Download PDF

Info

Publication number
WO2023157556A1
WO2023157556A1 PCT/JP2023/001554 JP2023001554W WO2023157556A1 WO 2023157556 A1 WO2023157556 A1 WO 2023157556A1 JP 2023001554 W JP2023001554 W JP 2023001554W WO 2023157556 A1 WO2023157556 A1 WO 2023157556A1
Authority
WO
WIPO (PCT)
Prior art keywords
pen
vibration control
signal
control signal
position detection
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2023/001554
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
淳 門脇
築 石丸
慕慈 陳
登 山口
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wacom Co Ltd
Original Assignee
Wacom Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wacom Co Ltd filed Critical Wacom Co Ltd
Priority to JP2024501039A priority Critical patent/JPWO2023157556A1/ja
Priority to CN202380014624.6A priority patent/CN118284871A/zh
Publication of WO2023157556A1 publication Critical patent/WO2023157556A1/ja
Priority to US18/799,321 priority patent/US12619312B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/038—Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
    • G06F3/0383—Signal control means within the pointing device
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/016—Input arrangements with force or tactile feedback as computer generated output to the user
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0354—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of two-dimensional [2D] relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0354—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of two-dimensional [2D] relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
    • G06F3/03545—Pens or stylus
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/038—Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/0416—Control or interface arrangements specially adapted for digitisers
    • G06F3/04162—Control or interface arrangements specially adapted for digitisers for exchanging data with external devices, e.g. smart pens, via the digitiser sensing hardware
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/044—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by capacitive means
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04883—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text

Definitions

  • the present invention relates to a system including a pen and a pen position detection device, a pen position detection device, and a method of operating a haptic element built into the pen.
  • Patent Documents 1 to 5 disclose examples of systems using this type of pen.
  • the systems described in Patent Documents 1 and 2 are configured to vibrate the pen when the pen is positioned over an item displayed on the screen.
  • the systems described in Patent Documents 3 to 5 are configured to vibrate the tip of the pen during writing in order to reproduce the feeling of writing on paper.
  • one object of the present invention is a system including a pen and a pen position detection device, a pen position detection device, and a pen that can generate timely tactile feedback when the pen position crosses the boundary of any region.
  • another object of the present invention is to provide a system including a pen and a pen position detection device, a pen position detection device, and a method of operating a haptic element built in a pen, which can reproduce a realistic writing feel. to provide.
  • a system is a system including a pen and a pen position detection device, wherein the pen includes a signal transmitter for transmitting a pen signal, a receiver for receiving a vibration control signal, and the vibration control signal. a haptic element that operates in response to a control signal, wherein the pen position detection device derives a pen position by detecting the pen signal; detecting the occurrence of an event that causes at least one of crossing or contact with a line segment of and operating the haptic element by transmitting the vibration control signal in response to detecting the occurrence of the event; System.
  • a pen position detection device is a pen position detection device for use with a pen having a haptic element that operates in response to a vibration control signal, the pen position detection device detecting a pen signal transmitted by the pen.
  • a position is derived, an occurrence of an event in which at least one of intersection or contact occurs between a drawn line indicated by the derived series of pen positions and a predetermined line segment is detected, and the occurrence of the event is detected.
  • a pen position sensing device responsively activating the haptic element by sending the vibration control signal.
  • a method is a method of operating a haptic element built into a pen, comprising the steps of: sending a pen signal from the pen; and an event in which at least one of intersection or contact occurs between a drawn line indicated by the series of pen positions derived by the pen position detection device and a predetermined line segment. detecting an occurrence; the pen position detection device sending a vibration control signal in response to detecting the occurrence of the event; the pen receiving the vibration control signal; a pen operating the haptic element in response to the received vibration control signal.
  • a system is a system including a pen and a pen position detection device, wherein the pen includes a signal transmitter for transmitting a pen signal, a receiver for receiving a vibration control signal, and the vibration control signal. a haptic element that operates in response to a control signal, wherein the pen position detection device derives a pen position by detecting the pen signal and derives a writing direction of the pen based on the history of the pen position. and operating the haptic element by transmitting a vibration control signal indicating the derived vibration pattern corresponding to the writing direction of the pen.
  • a pen position detection device is a pen position detection device for use with a pen having a haptic element that operates in response to a vibration control signal, the pen position detection device detecting a pen signal transmitted by the pen.
  • the haptic element is operated by deriving a position, deriving a writing direction of the pen based on the history of the pen position, and transmitting a vibration control signal indicating a vibration pattern corresponding to the derived writing direction of the pen.
  • a method is a method of operating a haptic element built into a pen, comprising the steps of: sending a pen signal from the pen; a step of deriving a pen position by the pen position detection device deriving a writing direction of the pen based on the history of the pen position; and a step of the pen position detection device deriving the derived writing of the pen transmitting a vibration control signal indicative of a direction-dependent vibration pattern; receiving the vibration control signal by the pen; and operating the haptic element in response to the vibration control signal received by the pen.
  • a method comprising the steps of: sending a pen signal from the pen; a step of deriving a pen position by the pen position detection device deriving a writing direction of the pen based on the history of the pen position; and a step of the pen position detection device deriving the derived writing of the pen transmitting a vibration control signal indicative of a direction-dependent vibration pattern; receiving the vibration control signal by the pen; and operating the haptic element in response to the vibration control signal received
  • the first aspect of the present invention it is possible to generate tactile feedback in a timely manner when the pen position crosses the boundary of any region.
  • FIG. 4 is a processing flow diagram showing processing executed by the sensor controller 32 according to the first embodiment of the present invention
  • FIG. 3 is a diagram showing an example of a screen displayed on panel surface 3a by a drawing application according to the first embodiment of the present invention
  • 4 is a processing flow diagram showing processing executed by a host processor 33 according to the first embodiment of the present invention
  • FIG. FIG. 8 is a diagram showing another example of the during-touch vibration control line according to the first embodiment of the present invention
  • FIG. 11 is a processing flow diagram showing processing executed by a host processor 33 according to the second embodiment of the present invention
  • FIG. 9 is a processing flow diagram showing processing executed by a sensor controller 32 according to a second embodiment of the present invention
  • FIG. 4 is a diagram showing an example of setting a vibration control area during hover 50 on the screen shown in FIG. 3
  • FIG. 11 is a processing flow diagram showing processing executed by a sensor controller 32 according to a third embodiment of the present invention
  • FIG. 11 is a processing flow diagram showing processing executed by a host processor 33 according to a third embodiment of the present invention
  • FIG. 11 is a processing flow diagram showing processing executed by a sensor controller 32 according to a fourth embodiment of the present invention
  • FIG. 14 is a diagram showing an example of a paint screen displayed on panel surface 3a by a paint application according to a fifth embodiment of the present invention
  • 6 is a diagram showing a brush selection menu 61a displayed when a user taps a brush selection menu activation icon 61 using the pen 2.
  • FIG. 8 is a diagram showing a line width selection menu 62a displayed when a user taps a line width selection menu activation icon 62 using the pen 2.
  • FIG. 6 is a diagram showing an area selection menu 63a displayed when a user taps an area selection menu activation icon 63 with the pen 2.
  • FIG. 4 is a processing flow diagram showing processing executed by a host processor 33 according to instructions from a paint application;
  • FIG. 4 is a processing flow diagram showing processing executed by a host processor 33 according to instructions from a paint application;
  • FIG. 4 is a processing flow diagram showing processing executed by a host processor 33 according to instructions from a paint application;
  • FIG. 4 is a processing flow diagram showing processing executed by
  • FIG. 1 is a diagram showing the configuration of a position detection system 1 according to the first embodiment of the present invention. As shown in the figure, the position detection system 1 comprises a pen 2 and a pen position detection device 3 .
  • the pen position detection device 3 is a computer having a function of detecting the pen 2, and as shown in FIG. 32 , a host processor 33 , and a wireless communication unit 34 .
  • the pen position detection device 3 is a personal computer such as a tablet terminal or a notebook computer.
  • the display 31 is a display device having a display surface such as a liquid crystal display or an organic EL display.
  • the panel surface 3a constitutes the display surface of the display 31.
  • the host processor 33 is a central processing unit of the pen position detection device 3 having a function of controlling each part of the pen position detection device 3 including the display 31 .
  • the host processor 33 is configured to be able to execute the operating system of the pen position detection device 3 and various applications including drawing applications by executing programs stored in an internal memory.
  • the display 31 plays a role of displaying various screens (images or videos) on the display surface based on a video signal generated by an operating system or application.
  • the wireless communication unit 34 is a device for communicating with other devices including the pen 2 by short-range wireless communication such as Bluetooth (registered trademark).
  • the host processor 33 is configured to be able to communicate with the pen 2 via the wireless communication unit 34 by short-range wireless communication.
  • the sensor 30 is a device used by the sensor controller 32 to communicate with the pen 2, and includes a plurality of X electrodes each extending in the y direction within the panel surface 3a and juxtaposed at equal intervals in the x direction. , and a plurality of Y electrodes each extending in the x direction in the panel surface 3a and arranged in parallel in the y direction at equal intervals.
  • a plurality of X electrodes and a plurality of Y electrodes are each independently connected to the sensor controller 32 .
  • One of the plurality of X electrodes and the plurality of Y electrodes may also be used as a common electrode within the display 31, in which case the pen position detection device 3 is called an "in-cell type".
  • the plurality of X electrodes and the plurality of Y electrodes may not be used as common electrodes in the display 31, and the pen position detection device 3 in that case is called "on-cell type" or "out-cell type”.
  • the sensor controller 32 By communicating with the pen 2 via the sensor 30, the sensor controller 32 derives the position of the pen 2 within the panel surface 3a, acquires data from the pen 2, and acquires the derived position and the acquired data. It is an integrated circuit having the function of sequentially supplying reports containing data to the host processor 33 .
  • the sensor controller 32 implements these functions by executing a program implemented as hardware or a program stored in a built-in memory, and is configured to be able to execute various processes described later.
  • Communication between the sensor controller 32 and the pen 2 via the sensor 30 is preferably performed by, for example, an active capacitance method.
  • an active capacitance method Although the following description assumes the use of the active capacitance method, it is of course possible to use the electromagnetic induction method or other methods.
  • uplink signal US the signal that the sensor controller 32 transmits to the pen 2 via the sensor 30
  • pen signal PS the signal that the pen 2 will transmit to the sensor controller 32 via the sensor 30
  • the sensor controller 32 is configured to communicate with the pen 2 in units of frames having a predetermined length of time.
  • an uplink signal US is transmitted using a plurality of X electrodes or a plurality of Y electrodes. I do.
  • the uplink signal US thus transmitted has the role of notifying the pen 2 of the temporal position of the frame and the timing (time slot) within which the pen 2 should transmit the pen signal PS.
  • the uplink signal US includes a command indicating an instruction from the sensor controller 32 to the pen 2 .
  • the pen signal PS can include a position signal for causing the sensor controller 32 to detect the position of the pen 2 and a data signal modulated by data transmitted from the pen 2 to the sensor controller 32 .
  • the sensor controller 32 receives position signals from each of the plurality of X electrodes and the plurality of Y electrodes that constitute the sensor 30, and approximates the distribution of the reception intensity of the position signals in each of the x direction and the y direction using a normal distribution curve. , to derive the position of the pen 2 within the panel surface 3a (hereinafter referred to as "pen position") by deriving the respective peak positions.
  • the sensor controller 32 receives the data signal by one of the X electrodes or the Y electrodes closest to the pen position derived immediately before, and demodulates it to obtain the data transmitted by the pen 2 (hereinafter referred to as "pen data"). is configured to obtain
  • the pen data includes the response to the command in the uplink signal US, the writing pressure value indicating the pressure applied to the pen tip of the pen 2, and the ON/OFF states of the push button switches 27a and 27b (described later) provided on the pen 2. and switch information indicating the
  • FIG. 2 is a processing flow diagram showing processing executed by the sensor controller 32.
  • FIG. Referring to FIG. 2, the processing executed by the sensor controller 32 will be described in detail.
  • the sensor controller 32 transmits an uplink signal US at the beginning of a frame (step S1).
  • the sensor controller 32 executes the processing of steps S3 to S7 each time the pen 2 transmits the pen signal PS (timing notified by the uplink signal US) in the corresponding frame (step S2).
  • the sensor controller 32 first attempts to detect the pen signal PS using the sensor 30 (step S3). As a result, it is determined whether or not the pen signal PS has been detected (step S4), and if it is determined that it has not been detected, the process proceeds to the next timing. On the other hand, the sensor controller 32 determined to be "detected" in step S4 derives the pen position (step S5) and acquires pen data (step S6) based on the detected pen signal PS. Then, a report including the derived pen position and the acquired pen data is output to the host processor 33 (step S7), and the process returns to step S3.
  • a series of reports supplied by the sensor controller 32 to the host processor 33 are used in the host processor 33 for processing performed by the drawing application.
  • the processing of the drawing application here includes generation and display of digital ink, movement of the cursor, detection of various gestures such as tapping and dragging, control of the haptics element 28 (described later) built in the pen 2, and the like.
  • the drawing application first creates one stroke based on a series of pen positions and pen data that are continuously acquired when the pen pressure value is greater than 0. It is configured to generate data and generate digital ink from the series of stroke data thus generated. Each time the drawing application derives a new pen position, it also uses one or more past pen positions included in the same stroke data to generate a spline curve such as a Catmull-Rom curve for rendering. At this time, the drawing application also performs processing for controlling the appearance of the generated spline curve based on the pen data. This processing includes processing for controlling line width or transparency based on pen pressure values. The drawing application generates a video signal based on the rendered spline curve and supplies it to the display 31 . Thereby, the spline curve rendered by the drawing application is displayed on the display 31 .
  • a spline curve such as a Catmull-Rom curve for rendering.
  • the pen 2 includes a core body 20, a pen tip electrode 21, a pressure sensor 22, a battery 23, an integrated circuit 24, a wireless communication circuit 25, a seesaw switch 26, and a push button switch. 27 a, 27 b and a haptic element 28 .
  • the core 20 is a columnar member that constitutes the pen shaft of the pen 2 .
  • the tip of the core body 20 constitutes the pen tip of the pen 2 and the distal end is in contact with the pressure sensor 22 .
  • the pen tip electrode 21 is a conductor arranged on the pen tip of the pen 2 and electrically connected to the integrated circuit 24 .
  • the pressure sensor 22 is a sensor that detects pressure applied to the tip of the core 20 .
  • the pressure detected by the pressure sensor 22 is supplied to the integrated circuit 24 and placed in the pen signal PS by the integrated circuit 24 as the pen pressure value described above.
  • Battery 23 serves to provide the power necessary for integrated circuit 24, wireless communication circuit 25, and haptic element 28 to operate.
  • the integrated circuit 24 is an integrated circuit composed of various circuits including a booster circuit, a transmission circuit, a reception circuit, and a processing circuit.
  • the receiving circuit is connected to the pen tip electrode 21, and by using the pen tip electrode 21 to detect the uplink signal US, it serves as a receiving section for receiving the uplink signal US.
  • the transmitting circuit is also connected to the pen tip electrode 21, and functions as a signal transmitting section that transmits the pen signal PS by changing the pen tip electrode 21 using the booster circuit at the timing indicated by the uplink signal US.
  • the processing circuit is a circuit that functions as a control unit that controls each part of the pen 2, generates a pen signal PS based on the uplink signal US received by the receiving circuit, and transmits the generated pen signal PS to the transmitting circuit. process to cause
  • the wireless communication circuit 25 is a device for communicating with other devices including the pen position detection device 3 by short-range wireless communication such as Bluetooth (registered trademark). Since short-range wireless communication is two-way communication, wireless communication circuit 25 functions as a transmitter for transmitting signals and also as a receiver for receiving signals.
  • the integrated circuit 24 is configured to be able to communicate with the pen position detection device 3 via this wireless communication circuit 25 by short-range wireless communication.
  • the host processor 33 is configured to transmit vibration control signals to the pen 2 via this short-range wireless communication.
  • a vibration control signal is a signal for instructing the operation of the haptic element 28 . Specifically, it may be a signal for simply instructing to operate the haptic element 28, or a signal for instructing to operate the haptic element 28 at a predetermined timing.
  • Integrated circuit 24 is configured to, upon receiving a vibration control signal via wireless communication circuitry 25, control operation of haptic element 28 in response to the received vibration control signal.
  • the seesaw switch 26 is a switch-pressing member having a plate-like portion 26p arranged along the surface of the housing and two legs 26a and 26b projecting from the plate-like portion 26p toward the inside of the housing. is.
  • the leg portions 26a and 26b are provided in the vicinity of one end and the other end of the plate-like portion 26p in the pen axis direction, respectively, and their tips are in contact with the push button switches 27a and 27b.
  • the leg portion 26a moves inward and presses the push button switch 27a.
  • the leg portion 26b moves inward and presses the push button switch 27b.
  • the push button switches 27a and 27b are configured so that when one of them is turned on, the other is turned off, and when one of them is turned off, the other is turned on.
  • the ON/OFF states of the pushbutton switches 27a and 27b are supplied to the integrated circuit 24 and placed in the pen signal PS by the integrated circuit 24 as switch information described above.
  • the haptic element 28 is an element that operates under the control of the integrated circuit 24 and is arranged in the region between the two legs 26a and 26b of the seesaw switch 26.
  • the haptic elements 28 are constituted by vibrators or actuators.
  • the haptic element 28 may be configured by a piezo vibrator that incorporates a weight and a piezoceramic in a case, moves the weight by expanding and contracting the piezoceramic under control from the integrated circuit 24, and generates vibration as a result. is preferred.
  • the reason why the haptic element 28 is arranged in the region between the legs 26a and 26b is to particularly vibrate the portion of the surface of the pen 2 that comes into contact with the finger of the person holding the pen 2 (the portion close to the finger). is. However, it is possible to position the haptic element 28 at a position other than between the legs 26a, 26b.
  • FIG. 3 is a diagram showing an example of a screen displayed on the panel surface 3a by the drawing application (host processor 33) according to this embodiment.
  • FIG. 4 is a processing flow diagram showing processing executed by the host processor 33 according to this embodiment. Processing for generating tactile feedback in a timely manner when the pen position straddles the boundary of arbitrary regions will be described in detail below with reference to these figures.
  • the drawing application has a drawing area 40 including a signature field 41 on the panel surface 3a, and a save button 42 and an erase button 43 arranged outside the drawing area 40. showing the screen.
  • the pen position and pen data are sequentially supplied from the sensor controller 32 to the host processor 33 .
  • the drawing application uses a series of pen positions and pen data thus supplied to generate stroke data in real time, stores the generated stroke data in memory, renders the data, and displays the data on the panel surface 3a. I do.
  • FIG. 3 shows a state in which the user has entered the character string "John.T” using the pen 2, and as a result, the drawing application has displayed the character string "John.T” on the panel surface 3a. ing.
  • the save button 42 is a button for causing the drawing application to save the series of generated stroke data as digital ink.
  • the drawing application detects this tap and stores the series of stroke data stored in the temporary memory so far as digital ink representing the user's signature.
  • a process of saving in the memory in the device 3 is performed.
  • the erase button 43 is a button for causing the drawing application to erase the entered character string.
  • the drawing application detects this tap, erases the series of stroke data corresponding to the written character string from the memory, and deletes the series of stroke data being displayed.
  • a process of generating a video signal for erasing and supplying it to the display 31 is performed.
  • FIG. 3 it is of course possible to provide various other user interfaces in the screen.
  • Various user interfaces may include an interface for changing the screen displayed on panel surface 3a to another screen by user operation.
  • the drawing application has a function of detecting the occurrence of such overrunning and notifying the user of the overrunning by operating the haptic element 28 of the pen 2 in real time.
  • the user notices that the signature has protruded from the signature column 41, and can press the erase button 43 to rewrite it.
  • a line to be notified is referred to as a "vibration control line during touch”.
  • the outline of the signature column 41 is the vibration control line during touch.
  • the during-touch vibration control line is closed, but the during-touch vibration control line can also be configured by an open line. Specific processing for realizing timely haptic feedback when the pen position straddles the during-touch vibration control line will be described below with reference to FIG. 4 .
  • FIG. 4 shows processing executed by the host processor 33 according to instructions from the drawing application.
  • the host processor 33 first displays an initial screen on the panel surface 3a (step S10). This initial screen may or may not include the vibration during touch control line. Subsequently, the host processor 33 receives a report including the pen position and pen data from the sensor controller (step S11), and determines whether the pen pressure value included in the received pen data is greater than 0 (that is, if the pen 2 is on the panel surface). 3a) or 0 (that is, whether the pen 2 is hovering over the panel surface 3a) (step S12). The host processor 33, which determines that the writing pressure value is 0 in this determination, moves the cursor displayed on the panel surface 3a based on the pen position (step S13), and returns the process to step S11.
  • the host processor 33 determines whether or not the current pen position is within the drawing area (step S14). In this determination, if it is determined that it is "not within the drawing area", the host processor 33 attempts to detect a gesture with the pen 2 (step S21). Then, as a result of the trial, it is determined whether or not a gesture (such as a tap of a button displayed on the screen) has been detected (step S22). On the other hand, if "detected" is determined, the process corresponding to the gesture is executed (step S23), and then the process returns to step S11.
  • the processing executed in step S23 includes processing for displaying a screen including the vibration control line during touch on the panel surface 3a.
  • the host processor 33 which has determined "within the drawing area" in the determination of step S14, generates or updates stroke data based on the pen position and pen data, renders the stroke data obtained as a result, and renders the panel.
  • Draw on surface 3a step S15).
  • the host processor 33 determines whether or not there is a touch vibration control line on the screen being displayed (step S16).
  • the host processor 33 that has determined “no” in this determination returns the process to step S11.
  • the host processor 33 that has determined "yes” predicts the pen position that will be included in the next report received from the sensor controller 32 (step S17).
  • the next pen position can be predicted by calculating the moving speed of the pen position from two or more latest pen positions, and deriving the next pen position assuming that the moving speed will continue.
  • character recognition technology may be used to determine the character the user is currently writing, and the next pen position may be predicted based on the shape of the determined character.
  • the stroke data of the figure is learned in advance by artificial intelligence, and the stroke data currently being generated is input to this artificial intelligence to determine the character or figure that the user is writing, and based on the determined shape of the character or figure It is also possible to predict the next pen position.
  • the prediction of the next pen position based on the moving speed calculates the next pen position using a simple calculation of two coordinates as shown in the following equations (1) and (2). It may be done by In other words, the moving speed vector is obtained by the equation (1), and the next pen position is calculated by using the obtained moving speed vector by the equation (2). However, in equations (1) and (2), the next pen position is (x p , y p ), the newest pen position is (x 1 , y 1 ), the second newest pen position is (x 2 , y 2 ). This makes it possible to reduce the processing load on the host processor 33 for prediction.
  • acceleration may be used in addition to moving speed to predict the next pen position.
  • the next pen position can be predicted by calculating the next pen position using a simple calculation of the three coordinates, as shown in the following equations (3)-(6). That is, two moving speed vectors are obtained from the equations (3) and (4), the acceleration vector is obtained from the equation (5) using the two moving speed vectors obtained, and the obtained moving speed vector and the acceleration vector are used in the equation
  • the next pen position can be calculated by (6).
  • the next pen position is (x p , y p )
  • the newest pen position is (x 1 , y 1 )
  • the second newest pen position is (x 2 , y 2 )
  • the third newest pen position is (x 3 , y 3 ).
  • the host processor 33 After the pen position prediction is completed, the host processor 33 detects events in which at least one of crossing and contact occurs between the drawing line indicated by the series of pen positions including the predicted pen position and the vibration-on-touch control line. An attempt is made to detect the occurrence (step S18). Specifically, the predicted pen position is temporarily added to the currently generated stroke data, a spline curve is generated based on the added stroke data, and the generated spline curve is used as the drawing line to try the above detection. do it. Further, the host processor 33 generates a line segment connecting any of the one or more pen positions forming the stroke data generated in step S15 and the predicted pen position, and the generated line segment is used as the drawing line. may be tried.
  • the host processor 33 determines whether or not the occurrence of the event has been detected in step S18 (step S19), and if it determines "not detected", returns to step S11 and repeats the process.
  • the host processor 33 determines "detected"
  • the host processor 33 transmits a predetermined vibration control signal to the pen 2 by short-range wireless communication via the wireless communication unit 34 shown in FIG. 1 (step S20).
  • the vibration control signal transmitted in this way may be a signal for simply instructing to operate the haptic element 28, or a signal for instructing to operate the haptic element 28 after a predetermined time. may Alternatively, if the host processor 33 can predict the timing at which crossing or contact occurs from the moving speed of the pen 2, etc., the vibration control signal is a signal containing information for operating the haptic element 28 at the predicted timing. It can be a certain thing.
  • the integrated circuit 24 of the pen 2 Upon receiving the vibration control signal, the integrated circuit 24 of the pen 2 operates the haptic element 28 at the timing indicated by the vibration control signal. This provides a vibrating notification to the user so that the user can know that the line being written crosses or touches (or has crossed or touched) the vibrating control line during touch.
  • the host processor 33 controls the distance between the during-touch vibration control line set in the screen and the drawing line indicated by the stroke data being input. detecting the occurrence of an event that causes at least one of crossing and/or contact with the pen 2, and in response to detecting the occurrence of this event, operating the haptic element 28 of the pen 2 by sending a vibration control signal via short-range wireless communication.
  • This makes it possible to generate tactile feedback in a timely manner when the pen position crosses arbitrary region boundaries. Therefore, the user can timely know when the line he is writing crosses or touches the vibrating control line during touch.
  • the host processor 33 performs the process of predicting the next pen position in step S17 of FIG. It is due to If the delay is negligible, the host processor 33 does not execute the process of step S17, and in step S18, the drawing line indicated by the acquired series of pen positions intersects the vibration-on-touch control line. Alternatively, it may attempt to detect the occurrence of an event in which at least one of the contacts occurs.
  • the contour line of the signature field 41 is exemplified as the during-touch vibration control line, but it goes without saying that other lines may be used as the during-touch vibration control line.
  • FIG. 5 is a diagram showing another example of the during-touch vibration control line according to the present embodiment.
  • the figure shows an example in which an electronic ruler 45 is displayed within the drawing area 40 .
  • the electronic ruler 45 is a virtual ruler that is displayed within the drawing area 40 by the drawing application, and is configured to be able to be displayed at any position at any angle and size according to the user's operation.
  • the position detection system 1 provides timely tactile feedback when the pen position crosses the contour line of the electronic ruler 45. Since it can be generated, it is possible to reproduce the thumping sensation when the pen 2 rides on the ruler and the jerkiness when the pen 2 falls off the ruler, even though there are no unevenness in reality. become.
  • the haptic element 28 is not controlled when the pen 2 is hovering, the pen 2 moves over the contour line of the electronic ruler 45. It is possible to prevent the above-mentioned thumping or snapping sensation from occurring just by doing so.
  • the position detection system 1 according to the present embodiment differs from the position detection system 1 according to the first embodiment in that the sensor controller 32, not the host processor 33, transmits the vibration control signal for operating the haptic element 28. ing.
  • Other points are the same as the position detection system 1 according to the first embodiment, including the configuration of the position detection system 1 shown in FIG. Description will be made by focusing on the differences from.
  • FIG. 6 is a processing flow diagram showing processing executed by the host processor 33 according to this embodiment.
  • the processing shown in FIG. 4 is also processing executed by the host processor 33 in accordance with instructions from the drawing application, as in FIG.
  • FIG. 7 is a processing flow diagram showing processing executed by the sensor controller 32 according to this embodiment.
  • the processing performed by the host processor 33 according to the present embodiment is that steps S24 and S25 are executed after step S10, and steps S26 and S27 are executed after step S23. and steps S16 to S20 are not executed.
  • Step S24 is a step for determining whether or not the initial screen includes the touch vibration control line. If it is determined to be "included” here, the host processor 33 transmits the vibration control position information to the sensor controller 32 (step S25).
  • the vibration control position information is information indicating the position of the during-touch vibration control line (the position on the sensor 30) within the panel surface 3a, and is expressed using coordinates within the panel surface 3a. Specifically, the vibration control position information may be a series of coordinates or a function representing a line segment.
  • the host processor 33 causes at least crossing or contact between the drawn line indicated by the series of pen positions and the during-touch vibration control line. It causes the sensor controller 32 to perform on its own behalf the process of detecting the occurrence of an event that one occurs and, in response to detecting the occurrence of this event, operating the haptic element 28 by transmitting a vibration control signal.
  • Step S26 is a process of determining whether or not the position of the touch vibration control line has been changed by the process of step S23 (process corresponding to the gesture). If it is determined that the touch vibration control line has been changed, the host processor 33 again transmits vibration control position information indicating the changed position of the touch vibration control line to the sensor controller 32 (step S27). Although details will be described later, the sensor controller 32 that has thus received the new vibration control position information will thereafter execute the above processing based on the newly received vibration control position information.
  • the processing performed by the sensor controller 32 according to the present embodiment executes steps S30 to S38. It is different from the processing to be performed.
  • the sensor controller 32 first attempts to receive vibration control position information from the host processor 33 (step S30). Then, as a result of this trial, it is determined whether or not the vibration control position information has been received (step S31), and when it is determined that it has been received, the received vibration control position information is stored (step S32). If other vibration control position information is already stored when the vibration control position information is received, the sensor controller 32 updates the stored vibration control position information with the received vibration control position information. good.
  • step S33 determines whether or not the vibration control position information is stored. Then, when it is determined that "not stored”, the process moves to step S1. After that, the same processing as the processing of steps S1 to S7 described with reference to FIG. 2 is performed, but after step S7, the processing of temporarily storing the derived pen position and the obtained pen data in the memory is performed (step S38). The pen data stored here may be only the writing pressure value. After the repeated processing of step S2 ends, the sensor controller 32 returns the processing to step S30.
  • the sensor controller 32 determines whether or not the drawing application is currently drawing based on the latest stored pen pressure value (step S34). Specifically, if the latest stored pen pressure value is 0, it is determined that "drawing is not in progress", and if it is greater than 0, it is determined that "drawing is in progress". When the sensor controller 32 determines that "drawing is not in progress" in step S34, the process proceeds to step S1.
  • the sensor controller 32 which has determined that drawing is being performed in step S34, draws a line between the drawn line indicated by the series of stored pen positions and the during-touch vibration control line indicated by the stored vibration control position information. An attempt is made to detect the occurrence of an event in which at least one of crossing and contact occurs (step S35). Specifically, among a series of stored pen positions, a spline curve is generated based on one or more pen positions for which the corresponding pen pressure value is continuously greater than 0 from the latest pen position, The above detection can be tried by using the generated spline curve as the drawing line. Further, the sensor controller 32 determines whether or not the pen pressure value corresponding to the second newest pen position among the series of stored pen positions is greater than 0.
  • the second a line segment connecting the new pen position and the latest pen position may be generated, and the detection may be attempted using the generated line segment as the drawing line. Furthermore, the sensor controller 32 detects the occurrence of the event when the latest one of the stored series of pen positions is on the vibration control line during touch indicated by the vibration control position information. good.
  • step S36 determines whether or not the occurrence of the above event has been detected in step S35 (step S36), and if it determines that "not detected", the process proceeds to step S1.
  • step S37 the uplink signal US including the vibration control signal is transmitted at the beginning of the frame (step S37), and the process proceeds to step S2.
  • the vibration control signal is supplied to the integrated circuit 24 of the pen 2 through the uplink signal US, and the integrated circuit 24 operates the haptic element 28 at the timing indicated by the vibration control signal.
  • a vibrating notification is provided to the user so that the user can know that the line being written has crossed or touched the vibrating control line during touch.
  • the sensor controller 32 controls the distance between the during-touch vibration control line set in the screen and the drawing line indicated by the stroke data being input. detecting the occurrence of an event in which at least one of crossing and/or contacting the pen 2 occurs, and in response to detecting the occurrence of this event, operating the haptic element 28 of the pen 2 by sending a vibration control signal using the uplink signal US This makes it possible to generate tactile feedback in a timely manner when the pen position crosses arbitrary region boundaries. Therefore, the user can timely know when the line he is writing crosses or touches the vibrating control line during touch.
  • the reason why the prediction of the pen position (step S17 in FIG. 4) performed by the host processor 33 according to the first embodiment is not performed in this embodiment is that the haptic element 28 is controlled by the uplink signal US. is due to the fact that almost no delay can be expected. If the delay cannot be ignored, the sensor controller 32 may predict the pen position in the same way as the host processor 33 according to the first embodiment, and use the result to perform the trial of step S35. .
  • the position detection system 1 according to the present embodiment operates the haptic element 28 even when the pen 2 is hovering over the panel surface 3a. is different from Other points are the same as the position detection system 1 according to the first embodiment, including the configuration of the position detection system 1 shown in FIG. Description will be made by focusing on the differences from.
  • FIG. 8 is a diagram showing an example of setting a space area for vibrating the pen 2 (hereinafter referred to as a "vibration control area during hovering") on the screen shown in FIG.
  • the vibration control area during hover 50 is defined as a spatial area from H L to H H (>H L ) above the signature field 41 (the distance in the z direction shown). Specific processing for realizing timely haptic feedback when the position of the tip of the pen 2 is within the vibration control area during hover will be described below.
  • FIG. 9 is a processing flow diagram showing processing executed by the sensor controller 32 according to this embodiment.
  • FIG. 10 is a processing flow diagram showing processing executed by the host processor 33 according to this embodiment. The processing shown in FIG. 4 is also processing executed by the host processor 33 in accordance with instructions from the drawing application, as in FIG.
  • the processing performed by the sensor controller 32 according to the present embodiment includes executing step S40 after step S5 and executing step S41 instead of step S7. This is different from the process performed by the sensor controller 32 according to the first embodiment in this point.
  • the sensor controller 32 acquires the reception strength of the pen signal PS received from the pen 2 (hereinafter referred to as "pen signal reception strength").
  • the sensor controller 32 may acquire, as the pen signal reception strength, the maximum reception strength of the pen signal PS at each of the plurality of X electrodes and the plurality of Y electrodes that constitute the sensor 30.
  • the peak value of the above-described normal distribution curve may be obtained for each of the y directions, and the larger of the obtained two peak values may be obtained as the received pen signal intensity.
  • step S41 the sensor controller 32 outputs to the host processor 33 a report containing the pen position derived in step S5, the pen signal reception strength obtained in step S40, and the pen data obtained in step S6.
  • This process differs from step S7 shown in FIG. 2 in that the pen signal reception strength is added to the report.
  • the host processor 33 that receives the pen signal reception strength uses the pen signal reception strength to derive the height of the pen 2 (the distance in the z direction from the panel surface 3a to the pen tip).
  • step S51 is executed instead of step S11, and steps S52 and S53 are executed after step S13. is different from the processing performed by the host processor 33 according to the first embodiment.
  • Step S51 differs from step S11 shown in FIG. 4 in that the report received from the sensor controller 32 includes the pen signal reception strength.
  • the details of the received pen signal strength are as described above.
  • the host processor 33 determines whether or not the tip of the pen 2 is positioned within the vibration control area during hover (step S52). Specifically, the host processor 33 first derives the pen height based on the received pen signal strength. Specifically, a table or function that associates pen heights with pen signal reception intensities is stored, and the pen signal reception intensities are converted into pen heights using this table or function. Then, the host processor 33 makes the determination in step S52 by determining whether the spatial coordinates indicated by the pen position and the pen height are located within the vibration control area during hover. Taking the vibration control area 50 during hovering in FIG. It will be determined that the tip is positioned within the vibration control area during hover. If it is determined in step S52 that the tip of the pen 2 is not located within the vibration control area during hovering, the host processor 33 returns to step S51 to continue processing.
  • the host processor 33 which has determined in step S52 that the tip of the pen 2 is positioned within the vibration control area during hover, performs the same processing as step S20 in FIG. Then, a predetermined vibration control signal is transmitted (step S53). This causes the haptic element 28 of the pen 2 to operate when the tip of the pen 2 is within the vibration control region during hover.
  • the host processor 33 determines whether or not the tip of the pen 2 is within the vibration control area during hovering, and determines that it is. Accordingly, since the haptic element 28 of the pen 2 is operated by transmitting a vibration control signal via short-range wireless communication, when the tip of the pen 2 is within the vibration control region during hover, the time It will be possible to generate tactile feedback to Lee. Therefore, the user can timely know that the tip of the pen 2 is within the vibration control area during hover.
  • the haptic element 28 of the pen 2 is activated by transmitting a vibration control signal.
  • the host processor 33 executes a series of processes to operate
  • the sensor controller 32 may execute these processes.
  • information indicating the vibration control area during hover is added to the vibration control position information shown in FIG. It may be determined whether or not the object is positioned, and if it is determined that the object is positioned, the vibration control signal may be transmitted by the uplink signal US as in the example of FIG.
  • the position detection system 1 according to the present embodiment differs from the second embodiment in terms of the timing at which the sensor controller 32 transmits the vibration control signal and in that the sensor controller 32 generates vibration control signals indicating various vibration patterns. is different from the position detection system 1 in the form of Other points are the same as the position detection system 1 according to the second embodiment, including the configuration of the position detection system 1 shown in FIG. Description will be made by focusing on the differences from.
  • Step S60 is a process of deriving the writing direction of the pen 2 based on a series of stored pen positions (history of pen positions). For example, the direction of the moving velocity vector derived from the above equation (1) may be used as the writing direction derived in step S60.
  • the sensor controller 32 that derived the writing direction determines the vibration pattern of the haptic element 28 based on the derived writing direction (step S61). Then, an uplink signal US including a vibration control signal indicating the determined vibration pattern is transmitted at the beginning of the frame (step S62). This makes it possible to vibrate the haptic element 28 with different vibration patterns for each writing direction.
  • the sensor controller 32 derives the writing direction of the pen 2 based on the history of the pen position, and determines the derived writing direction of the pen 2. Since the vibration control signal indicating the vibration pattern is sent, the friction between the pen tip and the paper fibers, which changes with the change in the moving direction of the pen 2, can be reproduced by the vibration of the pen 2. Therefore, it becomes possible to reproduce a realistic writing feeling.
  • the vibration pattern of the haptic element 28 is determined based on the writing direction, but the vibration pattern of the haptic element 28 may be determined based on other information. For example, based on the direction of change in writing direction (in one example, the orientation of the acceleration vector derived by equation (5) above), the writing pressure value, tilt value, azimuth angle value, etc. received from the pen 2, the haptic element 28 may be used to determine the vibration pattern of Alternatively, the vibration pattern of haptic element 28 may be determined based on a combination of two or more of these pieces of information.
  • the sensor controller 32 has been described as deriving the writing direction, determining the vibration pattern, and transmitting the vibration control signal. I don't mind. In this case, transmission of the vibration control signal may be performed by short-range wireless communication, as in the first embodiment.
  • the position detection system 1 according to the present embodiment is different from the position detection system according to the first embodiment in that the haptic element 28 is operated in accordance with various user operations related to the paint application, which is a type of drawing application described above. is different from 1.
  • Other points are the same as the position detection system 1 according to the first embodiment, including the configuration of the position detection system 1 shown in FIG. Description will be made by focusing on the differences from.
  • FIG. 12 is a diagram showing an example of a screen (hereinafter referred to as "paint screen") displayed on the panel surface 3a by the paint application.
  • 13 to 15 shown below are screenshots of the software "Paint" installed in Windows 11 (registered trademark), an operating system sold by Microsoft (registered trademark). is.
  • the paint screen includes a menu screen 46 in addition to the drawing area 40 described above.
  • the menu screen 46 includes various icons including a color selection palette 60 including a plurality of color icons, a brush selection menu activation icon 61, a line width selection menu activation icon 62, and an area selection menu activation icon 63.
  • Each icon corresponds to different processes, and when the user taps each icon on the menu screen 46 with the pen 2 , the corresponding process is executed by the host processor 33 .
  • Each icon will be described in detail below.
  • the color selection palette 60 is an icon for the user to specify the color of lines to be drawn.
  • the host processor 33 associates different colors with the respective color icons forming the color selection palette 60 and stores them. is stored as the color of the line to be drawn in (hereinafter referred to as "effective color").
  • the host processor 33 updates the stored effective color with the color stored in association with the tapped color icon.
  • the user can specify the color of the line to be drawn by tapping the color icon.
  • the host processor 33 associates and stores different vibration patterns with the respective color icons forming the color selection palette 60, and also stores the region above each color icon as the vibration control region during hovering described above. .
  • the host processor 33 determines whether or not the position of the tip of the pen 2 is within the hovering vibration control region by the same processing as in the third embodiment. Then, the haptic element 28 is vibrated using the vibration pattern stored in association with the corresponding color icon. This allows the user to know that the pen tip is positioned above the color icon, and also the color icon that is directly below the pen tip (i.e., the color icon that will be tapped if the pen is down at that position). ) is changed, it is possible to know that by vibration.
  • the vibration control signal according to the present embodiment includes a vibration pattern selection signal that indicates the vibration pattern (vibration waveform) of the haptic element 28 and a reproduction method instruction signal that indicates the reproduction method of the vibration pattern.
  • the reproduction method includes a method of reproducing the vibration pattern indicated by the vibration pattern selection signal only once, a method in which the user moves the pen 2 (while the pen tip is in contact with the panel surface 3a, touches the pen tip on the panel surface 3a). and a method of repeatedly reproducing a vibration pattern indicated by a vibration pattern selection signal during the movement operation).
  • the reproduction method when the tip of the pen 2 is positioned above the color icon is the former method of reproducing only once.
  • the host processor 33 selects one vibration pattern and one reproduction method each time it transmits a vibration control signal, and generates a vibration including a vibration pattern selection signal indicating the selected vibration pattern and a reproduction method instruction signal indicating the selected reproduction method. configured to transmit control signals;
  • the integrated circuit 24 of the pen 2 that has received this vibration control signal uses the vibration pattern indicated by the vibration pattern selection signal to vibrate the haptic element 28 in the reproduction method indicated by the reproduction method instruction signal.
  • the brush selection menu activation icon 61 is an icon for user designation of the type of line to be drawn (hereinafter referred to as "brush").
  • FIG. 13 is a diagram showing a brush selection menu 61a displayed when the user taps the brush selection menu activation icon 61 with the pen 2.
  • the brush selection menu 61a includes a plurality of brush icons.
  • the host processor 33 associates different brushes and different vibration patterns with each of the plurality of brush icons and stores them.
  • the host processor 33 selects the brush corresponding to any one of the plurality of brush icons as the type of line to be drawn (hereinafter referred to as "effective brush") in step S15 shown in FIG. 4 (and FIG. 17 described later). ) is remembered as When the user taps any brush icon with the pen 2, the host processor 33 updates the stored effective brush with the brush stored in association with the tapped brush icon.
  • the host processor 33 performing this processing, the user can specify the type of line to be drawn by tapping the brush icon. Also, when the user taps any brush icon using the pen 2, the host processor 33 vibrates the haptic element 28 using the vibration pattern stored in association with the tapped brush icon. process. This allows the user to understand by vibration whether he has chosen the correct brush or not.
  • the host processor 33 may also change the effective brush by pressing the push button switches 27a and 27b of the pen 2.
  • the host processor 33 associates different brushes with each of the push button switches 27a and 27b and stores them in advance. If the pen data indicates that the push button switch 27b is ON, the brush stored in association with the push button switch 27b is used as the effective brush. It can be regarded as an effective brush. This allows the user to switch between effective brushes with a switch at hand. Further, when the effective brush is changed by pressing the push button switches 27a and 27b, the host processor 33 uses the vibration pattern stored in association with the brush icon corresponding to the changed brush to move the haptic element 28. may be vibrated. In this way, even when the user changes the effective brush by depressing the push button switches 27a and 27b, it is possible for the user to understand from the vibration whether or not he/she has selected the correct brush.
  • the line width selection menu activation icon 62 is an icon for user designation of the width of the line to be drawn.
  • FIG. 14 is a diagram showing a line width selection menu 62a displayed when the user taps the line width selection menu activation icon 62 with the pen 2.
  • the line width selection menu 62a includes a plurality of line width icons.
  • the host processor 33 associates different line widths and different vibration patterns with each of the plurality of line width icons and stores them.
  • the host processor 33 also stores the line width corresponding to any one of the plurality of line width icons as the width of the line drawn in step S15 shown in FIG. 4 (hereinafter referred to as "effective line width"). ing.
  • the host processor 33 updates the stored effective line width with the line width stored in association with the tapped line width icon. process.
  • the host processor 33 performs this process.
  • the user can specify the width of the line to be drawn by tapping the line width icon.
  • the host processor 33 uses the vibration pattern stored in association with the tapped line width icon to move the haptic element 28. Perform vibration processing. This allows the user to know by vibration whether he has chosen the correct line width.
  • each line width icon is preferably associated with vibration patterns having different vibration frequencies. More specifically, it is preferable to associate a thicker line width with a lower frequency vibration pattern. By doing so, the user can naturally understand the width of the line tapped by the user.
  • the area selection menu activation icon 63 is an icon for specifying an area within the drawing area 40 by the user.
  • FIG. 15 is a diagram showing an area selection menu 63a displayed when the user taps the area selection menu activation icon 63 with the pen 2.
  • the area selection menu 63a includes a plurality of selection method icons.
  • the host processor 33 associates and stores different selection methods with each of the plurality of selection method icons.
  • a "square" is a line segment that connects the position of the pen tip when the pen 2 is moved and the current position of the pen tip by moving the pen 2. This is a method of selecting a rectangular area with a diagonal line.
  • "Free form” is a method of drawing a closed curve with the pen 2 and selecting an area surrounded by the closed curve.
  • the host processor 33 temporarily stores the selection method corresponding to the one tapped with the pen 2 among the plurality of selection method icons, and executes the selection operation by the stored selection method at the next move. .
  • the host processor 33 performs processing for vibrating the haptic element 28 using a predetermined vibration pattern while the user is performing a move for selection operation. This allows the user to understand from the vibration that the move he or she is performing is for the selection operation.
  • FIG. 15 also shows an example of processing executed by the host processor 33 after the "square" icon among the plurality of selection method icons included in the area selection menu 63a is tapped.
  • a trajectory 63b shown in the figure is the trajectory of the movement of the pen 2 performed by the user immediately after the "square" icon is tapped.
  • the host processor 33 acquires a rectangular area 63c whose diagonal is a line segment connecting the start position SP of the trajectory 63b and the current position CP of the trajectory 63b as a selected area. According to this processing, while the movement of the pen 2 continues, the size and shape of the area 63c change every moment.
  • the host processor 33 determines the area 63c at that time as the selected area and performs processing to display it.
  • the user can specify the selection area by tapping the selection method icon and then moving the icon.
  • 16 to 18 are processing flow diagrams showing processing executed by the host processor 33 according to instructions from the paint application. Processing executed by the host processor 33 according to the present embodiment to operate the haptic element 28 in accordance with various user operations related to the paint application will be described below in detail with reference to these drawings.
  • the host processor 33 performs steps S10, S51, and S12 in the same manner as in FIG.
  • the initial screen displayed in step S10 in this case is the paint screen shown in FIG.
  • the host processor 33 which has determined that the writing pressure value is 0 in step S12, performs the processes of steps S13 and S52, as in the case of FIG.
  • the vibration control area during hover to be determined in step S2 includes the area set above each of the color icons forming the color selection palette 60 shown in FIG.
  • the host processor 33 shifts the process to step S71.
  • step S52 if it is determined in step S52 that the tip of the pen 2 is positioned within the vibration control region during hovering, the host processor 33 controls the vibration control region during hovering in which the tip of the pen 2 is positioned. , to select a vibration pattern (step S70). More specifically, if the vibration control area during hover where the tip of the pen 2 is positioned is set above any color icon, the host processor 33 associates and stores the area with that color icon. Select the vibration pattern that is If the hovering vibration control region where the tip of the pen 2 is positioned is the hovering vibration control region 50 shown in FIG. It is sufficient to select the vibration pattern that
  • the host processor 33 uses near field communication to transmit a vibration control signal to the pen 2 (step S71).
  • the vibration control signal transmitted here includes a vibration pattern selection signal indicating the vibration pattern selected in step S70, and a reproduction method instruction signal indicating a reproduction method for reproducing the vibration pattern indicated by the vibration pattern selection signal only once. is included. This provides a vibrating notification to the user, allowing the user to know that the pen tip is positioned above the color icon, as well as the color icon directly below the pen tip (i.e., When the color icon (that is to be tapped when the pen is down at that position) changes, it is possible to know that by vibration.
  • the host processor 33 attempts to detect a predetermined user operation (step S72), and determines whether or not it has been detected (step S73).
  • User operations to be detected here are, for example, pressing operations of push button switches 27 a and 27 b provided on the pen 2 .
  • the host processor 33 may detect the user's operation based on the pen data received in step S51.
  • the host processor 33 that has determined "detected" in step S73 subsequently executes processing according to the detected user operation (step S74).
  • the detected user operation is an operation of pressing the push button switches 27a and 27b and brushes are stored in association with the respective push button switches 27a and 27b
  • the process executed by the host processor 33 is performed to detect the effective brushes. is switched to the brush stored in association with the pressed push button switch.
  • the host processor 33 which has executed processing according to the detected user operation, then determines whether or not the detected user operation is an operation to be vibrated (step S75). As a result of the determination, when it is determined that "the operation was a vibration target", the host processor 33 first selects a vibration pattern (step S76). For example, if the effective brush is changed by pressing the push button switches 27a and 27b, the host processor 33 selects the vibration pattern stored in association with the brush icon corresponding to the changed brush. do it.
  • the host processor 33 uses near field communication to transmit a vibration control signal to the pen 2 (step S77).
  • the vibration control signal transmitted here includes a vibration pattern selection signal indicating the vibration pattern selected in step S76, and a reproduction method instruction signal indicating a reproduction method for reproducing the vibration pattern indicated by the vibration pattern selection signal only once. is included.
  • a vibration notification is given to the user, and the user can know that the operation performed by the user has been accepted. Also, when the brush is changed, it becomes possible to know that by vibration.
  • step S73 If it is determined that "not detected” in step S73, if it is determined that "the operation was not subject to vibration" in step S75, and if the process of step S77 is completed, the host processor 33 performs step Return to S51 to continue processing.
  • the host processor 33 which has determined in step S12 that the pen pressure value is greater than 0, determines whether the current pen position is within the drawing area as shown in FIG. 17 (step S14). Then, if it is determined that it is "within the drawing area" in this determination, the host processor 33 determines the current mode of the paint application (step S80).
  • the mode here includes a drawing mode in which lines are drawn with the pen 2 and an operation mode in which various operations are performed with the pen 2 .
  • the host processor 33 Having determined that the current paint application mode is the drawing mode, the host processor 33 generates or updates stroke data based on the pen position and pen data acquired in step S51, as in the case of FIG.
  • the obtained stroke data is rendered and drawn on the panel surface 3a (step S15).
  • the host processor 33 tries to detect a figure to be vibrated (step S81), and determines whether or not it has been detected (step S82).
  • the figure to be detected here is, for example, an arrow-shaped figure or an arbitrary polygon.
  • the host processor 33 associates and stores in advance the features of the figure to be vibrated and the vibration pattern. Attempt discovery. If the figure to be detected is an arbitrary polygon, the host processor 33 may store the vibration pattern in association with the number of corners of the figure.
  • the host processor 33 that has determined "detected" in step S82 selects the vibration pattern stored in association with the detected figure (step S83). Then, a vibration control signal is transmitted to the pen 2 using short-range wireless communication (step S84).
  • the vibration control signal transmitted here includes a vibration pattern selection signal indicating the vibration pattern selected in step S83, and a reproduction method instruction signal indicating a reproduction method for reproducing the vibration pattern indicated by the vibration pattern selection signal only once. is included.
  • the notification by vibration is given to the user, so that it is possible to give the user an effect corresponding to the predetermined figure. For example, it is possible to give the user who has drawn the arrow the feeling of shooting a bow.
  • step S82 If it is determined "not detected” in step S82, and if step S84 is completed, the host processor 33 shifts the process to step S51 of FIG.
  • the host processor 33 having determined in step S80 that the current paint application mode is the operation mode, determines whether or not the pen 2 is being moved (step S85). As a result of this determination, the host processor 33 that has determined "not in the process of moving" shifts the processing to step S51 in FIG. On the other hand, the host processor 33, having determined that it is in the process of moving, starts processing according to the move (step S86). For example, immediately after tapping any of the selection method icons in the area selection menu 63a described above, this process displays a selection area that changes moment by moment (for example, the area 63c shown in FIG. 15(b)). It becomes the processing to do. In addition, for example, a process of enlarging a graphic being displayed, a process of rotating a graphic being displayed, and the like may be performed.
  • the host processor 33 which has started the process corresponding to the move, determines whether or not the started process is subject to vibration (step S87). As a result of the determination, when it is determined that "the process is subject to vibration", the host processor 33 first selects a vibration pattern (step S88). For example, if the started process is the region selection operation described above, the host processor 33 may select a predetermined vibration pattern stored in association with the selection operation. The host processor 33 can also start a process of enlarging the figure being displayed and a process of rotating the figure being displayed. It suffices to select a predetermined vibration pattern.
  • the host processor 33 uses near field communication to transmit a vibration control signal to the pen 2 (step S89).
  • the vibration control signal transmitted here includes a vibration pattern selection signal indicating the vibration pattern selected in step S88, and the vibration pattern indicated by the vibration pattern selection signal is repeatedly reproduced while the movement of the pen 2 continues. and a reproduction method instruction signal indicating a reproduction method.
  • the vibration is continuously given to the user while the move continues, and the user can know that the processing corresponding to the move continues.
  • step S90 determines whether or not the move has ended.
  • the host processor 33 repeatedly makes this determination while the pen 2 continues to move.
  • step S91 ends the process started in step S86 (step S91), and shifts the process to step S51 in FIG.
  • the host processor 33 that has determined "not within the drawing area” in step S14 attempts to detect a gesture (step S21), and determines whether or not it has been detected (step S22).
  • Gestures to be detected here are, for example, one of the color icons in the color selection palette 60 shown in FIG. 12, the brush selection menu activation icon 61, the line width selection menu activation icon 62, or the The icon 63 is tapped, any brush icon within the brush selection menu 61a shown in FIG. 13 is tapped, and any line width icon is tapped within the line width selection menu 62a shown in FIG.
  • the host processor 33 that has determined "detected" in step S22 executes processing corresponding to the gesture (step S23), and then determines whether or not the detected gesture is an operation to be vibrated (step S100). ). For example, if the gesture is a tap of a color icon, the process executed in step S23 is a process of setting the color corresponding to the color icon as an effective color. This is the process of setting the brush corresponding to the icon as the effective brush. For example, if the gesture is a tap of the line width icon, the process of setting the line width corresponding to the line width icon as the effective line width.
  • step S100 if the host processor 33 stores a vibration pattern associated with the tapped icon, the host processor 33 determines that "the operation is a vibration target", and if not stores a vibration pattern. It should be determined that the operation is not a target operation.
  • the host processor 33 having determined that "the operation is the target of vibration" in step S100, selects a vibration pattern corresponding to the detected gesture (step S101).
  • the host processor 33 may select a vibration pattern stored in association with the tapped icon.
  • the host processor 33 uses near field communication to transmit a vibration control signal to the pen 2 (step S102).
  • the vibration control signal transmitted here includes a vibration pattern selection signal indicating the vibration pattern selected in step S101, and a reproduction method instruction signal indicating a reproduction method for reproducing the vibration pattern indicated by the vibration pattern selection signal only once. is included. This provides a vibration notification to the user, allowing the user to know that the gesture he or she has made has been accepted by the paint app.
  • the position detection system 1 it is possible to operate the haptics element 28 in the pen 2 according to various user operations related to the paint application.
  • a haptic element in the position detection device 3 may be operated. In this way, the user can be notified by vibration of the panel surface 3a. Therefore, even when the pen 2 having no haptic element 28 is operated by the pen 2 or the finger, the user can be notified by vibration. becomes possible.
  • the host processor 33 executes processing according to the drawing application, but the host processor 33 may execute part or all of the processing according to the operating system.
  • the pen 2 acquires the writing pressure value based on the pressure detected by the pressure sensor 22 that detects the pressure applied to the pen tip.
  • the writing pressure value may be acquired based on the pressure detected by a pressure sensor that detects the pressure applied to the button provided on the surface.
  • the present invention can be applied to pen pressure values that are simulated in extended reality (XR) spaces such as virtual reality (VR), augmented reality (AR), and mixed reality (MR). can be applied.
  • XR extended reality
  • VR virtual reality
  • AR augmented reality
  • MR mixed reality

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Position Input By Displaying (AREA)
PCT/JP2023/001554 2022-02-21 2023-01-19 ペン及びペン位置検出装置を含むシステム、ペン位置検出装置、並びに、ペンに内蔵されるハプティクス素子を動作させる方法 Ceased WO2023157556A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2024501039A JPWO2023157556A1 (https=) 2022-02-21 2023-01-19
CN202380014624.6A CN118284871A (zh) 2022-02-21 2023-01-19 包含笔及笔位置检测装置的系统、笔位置检测装置以及使内置于笔的触觉元件进行动作的方法
US18/799,321 US12619312B2 (en) 2022-02-21 2024-08-09 System including pen and pen position detection apparatus, pen position detection apparatus, and method of activating haptic element built in pen

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2022024461 2022-02-21
JP2022-024461 2022-02-21

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/799,321 Continuation US12619312B2 (en) 2022-02-21 2024-08-09 System including pen and pen position detection apparatus, pen position detection apparatus, and method of activating haptic element built in pen

Publications (1)

Publication Number Publication Date
WO2023157556A1 true WO2023157556A1 (ja) 2023-08-24

Family

ID=87578365

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2023/001554 Ceased WO2023157556A1 (ja) 2022-02-21 2023-01-19 ペン及びペン位置検出装置を含むシステム、ペン位置検出装置、並びに、ペンに内蔵されるハプティクス素子を動作させる方法

Country Status (3)

Country Link
JP (1) JPWO2023157556A1 (https=)
CN (1) CN118284871A (https=)
WO (1) WO2023157556A1 (https=)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7393568B1 (ja) 2023-01-04 2023-12-06 レノボ・シンガポール・プライベート・リミテッド 情報処理システム、及び制御方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140043242A1 (en) * 2012-08-08 2014-02-13 Microsoft Corporation Physically modulating friction in a stylus
JP2017529623A (ja) * 2014-09-29 2017-10-05 マイクロソフト テクノロジー ライセンシング，エルエルシー ウェットインク予測器
JP2021114202A (ja) * 2020-01-20 2021-08-05 株式会社ワコム センサシステム
JP2021192213A (ja) * 2020-04-01 2021-12-16 株式会社ワコム 手書きデータ生成装置、手書きデータ再生装置、及びデジタルインクのデータ構造

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140043242A1 (en) * 2012-08-08 2014-02-13 Microsoft Corporation Physically modulating friction in a stylus
JP2017529623A (ja) * 2014-09-29 2017-10-05 マイクロソフト テクノロジー ライセンシング，エルエルシー ウェットインク予測器
JP2021114202A (ja) * 2020-01-20 2021-08-05 株式会社ワコム センサシステム
JP2021192213A (ja) * 2020-04-01 2021-12-16 株式会社ワコム 手書きデータ生成装置、手書きデータ再生装置、及びデジタルインクのデータ構造

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7393568B1 (ja) 2023-01-04 2023-12-06 レノボ・シンガポール・プライベート・リミテッド 情報処理システム、及び制御方法
JP2024096593A (ja) * 2023-01-04 2024-07-17 レノボ・シンガポール・プライベート・リミテッド 情報処理システム、及び制御方法
US12242680B2 (en) 2023-01-04 2025-03-04 Lenovo (Singapore) Pte. Ltd. Information processing system and control method

Also Published As

Publication number Publication date
JPWO2023157556A1 (https=) 2023-08-24
CN118284871A (zh) 2024-07-02
US20240402817A1 (en) 2024-12-05

Similar Documents

Publication Publication Date Title
JP6553136B2 (ja) タッチ感応表面上でのマルチ圧力相互作用のためのシステムと方法
US9170649B2 (en) Audio and tactile feedback based on visual environment
US8451245B2 (en) Multi-touch device having dynamic haptic effects
KR200258353Y1 (ko) 터치패드용 햅틱 피드백 및 기타 터치 제어장치
JP3996852B2 (ja) 表示されたスライドの予め選択された部分を強調表示するためのタッチパッドを備えたリモートコントロール
JP5635274B2 (ja) 触感呈示装置および触感呈示方法
EP2610738A2 (en) Method and device for displaying image
JP2009193423A (ja) 電子機器の入力装置
WO2013018480A1 (ja) タッチ入力が可能な画面内に原画像を縮小表示するタッチパッドを有するユーザインタフェース装置、入力処理方法及びプログラム
EP2075671A1 (en) User interface of portable device and operating method thereof
KR20110041915A (ko) 데이터 표시 방법 및 그를 수행하는 단말기
CN108579077A (zh) 信息处理方法、装置、存储介质及电子设备
JP2014063387A (ja) 電子機器及び電子機器の制御方法
KR20130030727A (ko) 힘을 이용한 인터페이스 제어 장치 및 방법
JP2006092321A (ja) 電子機器およびタッチパッド装置
KR20130091140A (ko) 터치스크린 기기와 보조 기기 간 햅틱 피드백 장치 및 방법
WO2023157556A1 (ja) ペン及びペン位置検出装置を含むシステム、ペン位置検出装置、並びに、ペンに内蔵されるハプティクス素子を動作させる方法
JP2010165291A (ja) 表示装置および拡大表示方法
US12619312B2 (en) System including pen and pen position detection apparatus, pen position detection apparatus, and method of activating haptic element built in pen
CN107016983A (zh) 一种钢琴操控方法、系统以及一种电子设备
KR20110011845A (ko) 터치 스크린이 구비된 이동통신 단말기 및 그 제어 방법
JP2011070600A (ja) 入力装置
KR20090122571A (ko) 터치스크린에서의 입력 피드백 방법 및 이를 이용한 장치
KR20080017194A (ko) 무선 마우스 및 그의 구동 방법
JP2024137189A (ja) 情報処理システム、情報処理装置、情報処理方法及びコンピュータプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23756084

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2024501039

Country of ref document: JP

Kind code of ref document: A

WWE Wipo information: entry into national phase

Ref document number: 202380014624.6

Country of ref document: CN

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 23756084

Country of ref document: EP

Kind code of ref document: A1