WO2012008469A1 - Feuille auxiliaire d'entrée d'informations, système de traitement d'informations, dispositif d'entrée d'informations, système d'entrée d'informations, feuille d'entrée manuscrite, et système d'entrée/sortie manuscrites utilisant des points de flux - Google Patents

Feuille auxiliaire d'entrée d'informations, système de traitement d'informations, dispositif d'entrée d'informations, système d'entrée d'informations, feuille d'entrée manuscrite, et système d'entrée/sortie manuscrites utilisant des points de flux Download PDF

Info

Publication number
WO2012008469A1
WO2012008469A1 PCT/JP2011/065914 JP2011065914W WO2012008469A1 WO 2012008469 A1 WO2012008469 A1 WO 2012008469A1 JP 2011065914 W JP2011065914 W JP 2011065914W WO 2012008469 A1 WO2012008469 A1 WO 2012008469A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
dot pattern
input
coordinate
display
Prior art date
Application number
PCT/JP2011/065914
Other languages
English (en)
Japanese (ja)
Inventor
吉田　健治
Original Assignee
Yoshida Kenji
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from JP2010158344A external-priority patent/JP2012022418A/ja
Priority claimed from JP2010158120A external-priority patent/JP2012022400A/ja
Priority claimed from JP2010158321A external-priority patent/JP2012022415A/ja
Application filed by Yoshida Kenji filed Critical Yoshida Kenji
Publication of WO2012008469A1 publication Critical patent/WO2012008469A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0354—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of 2D relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
    • G06F3/03545—Pens or stylus
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/0304—Detection arrangements using opto-electronic means
    • G06F3/0317—Detection arrangements using opto-electronic means in co-operation with a patterned surface, e.g. absolute position or relative movement detection for an optical mouse or pen positioned with respect to a coded surface
    • G06F3/0321—Detection arrangements using opto-electronic means in co-operation with a patterned surface, e.g. absolute position or relative movement detection for an optical mouse or pen positioned with respect to a coded surface by optically sensing the absolute position with respect to a regularly patterned surface forming a passive digitiser, e.g. pen optically detecting position indicative tags printed on a paper sheet

Definitions

  • the present invention relates to a sheet on which a stream dot pattern is printed, and an information processing system using the sheet.
  • the present invention also provides a technique for inputting information by using a medium on which a dot pattern is printed, a display on which the dot pattern is displayed, or a transparent sheet on which the dot pattern is printed, and syntactic analysis and image analysis of electronic data. About.
  • the present invention also relates to a handwriting input / output system and a handwriting input sheet for inputting handwritten information such as characters and figures into a computer system.
  • the present invention relates to an information input system and an information input auxiliary sheet for assisting input to a computer system.
  • Patent Document 1 Conventionally, in the operation of a personal computer, a PDA, a handy terminal, etc., a technique for inputting directly on the screen with a fingertip or a touch pen is known (for example, Patent Document 1).
  • the coordinate operated by the touch pen is recognized by a pressure-sensitive or optical touch panel, and the control is notified to the control means.
  • Patent Document 2 a part of a touch panel for detachably mounting on a display has been developed (for example, Patent Document 2).
  • a system for inputting handwritten information to a computer has been proposed as a solution to such a problem.
  • Such a system includes a base and a drawing device.
  • graphical information handwritten information
  • the base is provided with a position encoding pattern which is a fine dot, and position information on the base is registered in this position encoding pattern.
  • the drawing device reads the position coding pattern.
  • the computer system records position information on the base from the drawing device as a graphical input.
  • the drawn content can be input to the computer system and output to the display or stored in the memory (for example, Patent Document 4).
  • the conventional detachable touch panel has only been proposed to be mounted on a display, and no other touch panel has been proposed for use on printed materials or media.
  • This information input auxiliary sheet is arranged on an existing display screen, for example.
  • the user touches an icon or the like on the display screen with a scanner pen through the information input auxiliary sheet information corresponding to the icon or the like is output.
  • this information input auxiliary sheet is used, for example, on an existing printed material.
  • information corresponding to the character or design is output from a display, a speaker, or the like.
  • the icon is touched with a finger, the icon is also provided with a large tolerance for error during input, and the input accuracy of the touch panel need not be so high.
  • the display of the existing icon display or printed matter does not consider touch instructions, and the display is fine.
  • high input accuracy may be required.
  • the present invention has been made in view of these points, and provides an information input auxiliary sheet and an information processing system that are simple and low-cost, have extremely high input accuracy and input efficiency, and do not select a mounting target. And a calibration method for using the information input auxiliary sheet that can accurately associate the aspect ratio of the information input auxiliary sheet with the screen or the printed matter and realize high input accuracy by a simple method. Providing is a technical issue.
  • a dot code / address correspondence table indicating addresses where related information corresponding to dot codes exists is created in advance instead of the dot code / specific information correspondence table. It took time and effort.
  • the present invention has been made in view of such a point, and a mask area is defined in advance for a specific information that is printed or displayed on a screen and for which a hyperlink or the like is not set.
  • Information input that allows the user to input specific information without embedding a dot code corresponding to the specific information or creating a dot code / specific information correspondence table that predefines specific information corresponding to the dot code It is a technical problem to provide a device.
  • the position coding pattern used in the system of Patent Document 4 has only position information registered. Therefore, the computer system can input only the graphical information drawn by the drawing device. For example, in order to input and record various information related to the graphical information such as the page number of the booklet, An area different from the above area must be provided separately. Therefore, the handwriting input / output system becomes complicated, and there is a problem that the user needs to operate in a plurality of areas.
  • a reference table for inputting and recording various information related to the graphical information must be provided for all position information corresponding to the graphical information drawn by the drawing apparatus. In particular, when the drawing area is not a rectangle but an area surrounded by a polygon or a curve, a reference table for all position information in the area is required.
  • the present invention has been made in view of such a problem, and an object of the present invention is to provide a handwritten input / output system capable of inputting characters, figures and the like by handwriting in a simple and inexpensive manner. It is another object of the present invention to provide a handwriting input / output system that can input not only information on characters and figures but also information related to them in a lump.
  • An information input auxiliary sheet is disposed on a printed surface of a printed material or a display screen of a display, and is read by an optical reading device in order to output information related to the image of the printed material or the display on the display.
  • An information input auxiliary sheet provided with a dot pattern to be arranged, and for arranging various reference dots continuously in a line according to a predetermined rule for various multimedia information output and / or operation instructions; Providing a first virtual reference line composed of a straight line, a broken line and / or a curve connecting the plurality of reference dots; and defined at a predetermined position from the reference dot and / or the first virtual reference line.
  • the bright and clear dot pattern reflects the infrared rays irradiated from the infrared irradiation means without affecting the characteristics such as absorption and transmission of infrared rays on the medium surface on which the information input auxiliary sheet is arranged.
  • the dot code can be analyzed accurately.
  • a simple and low-cost information input auxiliary sheet can be provided.
  • the XY coordinate values are defined in the information input auxiliary sheet without any gaps.
  • characters, staffs, maps, figures, etc. are printed on the information input auxiliary sheet, and when operating on the line segment by tracing or touching with a scanner pen, a stream dot pattern is formed only along that line segment.
  • the dot pattern can be arranged rationally.
  • the shape of the rectangular area where the dot pattern is formed as a two-dimensional code is not constrained by the definition of the index based on the XY coordinate values, and a free shape that matches the information area visually formed on the medium surface
  • the stream dot pattern is used to define the second virtual reference line and / or the direction of the dot pattern and one XY coordinate value and / or
  • a reference dot serving as a reference is provided at a predetermined position. According to this, by setting a new reference point, it is possible to easily define the direction of the stream dot pattern and a set of constant information without using information dots, thereby suppressing unnecessary information reduction. It is done. Furthermore, the position of the virtual reference point that is the starting point of the information dot can be accurately indicated by the arrangement of the new reference point.
  • the dot pattern layer reflects an infrared ray from one side or both sides and transmits infrared light, or one side or both sides. It is characterized in that it is provided on one side or both sides of an infrared diffuse reflection layer having the characteristics of diffusely reflecting infrared rays from the side and transmitting visible light.
  • the infrared irradiation means irradiates without affecting the characteristics such as infrared absorption / transmission on the medium surface on which the information input auxiliary sheet is disposed. Infrared light can be reflected, and only bright and clear dot patterns can be photographed, and dot codes can be analyzed accurately.
  • the infrared diffuse reflection layer is provided, the infrared ray can be reflected and diffused in the direction in which the infrared ray is irradiated by one layer, the information input auxiliary sheet is made thinner, and the information input auxiliary sheet is manufactured. The number of processes can be reduced and the cost can be reduced.
  • the information input auxiliary sheet is provided with the infrared reflection layer, and transmits visible light between the infrared reflection layer and the dot pattern layer.
  • An infrared diffusion layer for diffusing the infrared rays is provided.
  • the infrared ray can be uniformly incident on the lens by diffusely reflecting the irradiated infrared ray.
  • the information input auxiliary sheet is provided with the infrared reflecting layer, and is provided on one side or both sides of the infrared reflecting layer and is provided on the outer surface side of the dot pattern layer. Is characterized by being provided with an infrared diffusion layer that transmits visible light and diffuses infrared light.
  • the infrared diffusion layer after providing the dot pattern layer on the upper surface of the infrared reflection layer, the infrared diffusion layer can be provided, or after providing the dot pattern layer on the back surface of the infrared diffusion layer, the infrared reflection layer can be provided.
  • the infrared diffusion layer also serves as a simple protective layer for the dot pattern layer.
  • the information input auxiliary sheet according to the present invention is an information input auxiliary sheet in which the dot pattern is provided on one surface side, and an adhesive layer is provided on the outer surface side where the dot pattern is read by an optical reading device. It is provided.
  • the information input auxiliary sheet can be easily pasted on a display or the like.
  • a protective layer having a property of transmitting infrared light and visible light is provided at least on the outer surface side where the dot pattern is read by the optical reading device.
  • a dot pattern on the medium is read by a dot pattern reading unit, and various multimedia information output and / or operation instructions are performed. It is processed into a bookmark or sticky note.
  • the information input auxiliary sheet can be processed into various forms to provide an information input auxiliary sheet that is more convenient and flexible.
  • the information input auxiliary sheet according to any one of claims 1 to 8 is disposed on a display screen of a display of the information processing apparatus, and has at least an XY coordinate value.
  • a function as a touch panel can be provided simply by pasting an information input auxiliary sheet on which a dot pattern is formed on the display screen of the display and touching it with a dot pattern reading means, which is simple and low cost.
  • An information processing system can be realized.
  • the information input assisting sheet is a display screen of a display for displaying contents expressed by text, illustrations, photographs, etc. in the information processing apparatus.
  • the XY coordinate value and / or code value of the dot pattern on the information input auxiliary sheet is read by the dot pattern reading means and converted into xy coordinates on the format information of the content displayed on the display. By doing so, it is possible to input in a touch panel format, to recognize content specifying information at a position touched by the dot pattern reading means, and to search and output content related information related to the content specifying information. .
  • an information input auxiliary sheet is arranged on the display, and a text, an illustration, a photograph, etc. on the display can be specified and a detailed description thereof can be displayed on the screen simply by touching with a dot pattern reading means. Can do. For this reason, even text, illustrations, photos, addresses, etc. that are not linked on the homepage can be accessed by touching them, making it more convenient for the user. .
  • the content specifying information may instruct an operation.
  • the information processing device is a personal computer, PDA, television receiver, front and rear projector, game device, karaoke device, mobile phone terminal device, POS terminal device, ATM, KIOSK.
  • an information processing system using an information input auxiliary sheet can be used for any device having a display and a screen.
  • the information input auxiliary sheet according to any one of claims 1 to 8 is used on a printed surface of a printed matter, and at least an XY coordinate value is defined.
  • the dot pattern reading means By reading the dot pattern on the information input auxiliary sheet by the dot pattern reading means and converting the recognized XY coordinates on the printed surface and the printed matter specifying information for specifying the printed matter into xy coordinates on the printed information Recognizing print information, information related to the print information is retrieved and output.
  • an information input auxiliary sheet is arranged on the display, and a text, an illustration, a photograph, etc. on the display can be specified and a detailed description thereof can be displayed on the screen simply by touching with a dot pattern reading means. Can do. For this reason, even text, illustrations, photos, addresses, etc. that are not linked on the homepage can be accessed by touching them, making it more convenient for the user. .
  • the content specifying information may instruct an operation.
  • an icon means an area in which a specific meaning is recognized by a text, an illustration, or a symbol in a rectangle, circle, or other figure.
  • the information input auxiliary sheet is arranged on a display screen of a plurality of displays connected to one or two or more information processing apparatuses, A code value for identifying a display is printed as a dot pattern, the code value and XY coordinate values read by the dot pattern reading means are transmitted to the information processing apparatus, and the display is recognized by the code value and the touch panel.
  • the information processing apparatus recognizes various multimedia information output and / or operation instructions corresponding to each display, and displays the output information on the display.
  • the thin client system can be provided by using the information input auxiliary sheet.
  • Multiple displays to which video and images are distributed from the server are inexpensive, enabling information input auxiliary sheets to be attached to general TV monitors and screens that do not have a built-in CPU, enabling touch-panel input using dot pattern reading means Since a simple input / output terminal can be realized, a thin client information processing system can be constructed at low cost.
  • the information input auxiliary sheet is arranged on a display screen of a plurality of displays connected to one or more information processing apparatuses.
  • the XY coordinates that uniquely identify the display are printed as a dot pattern, and the XY coordinate values read by the dot pattern reading means are transmitted to the information processing device, and the touch panel format can be input by specifying the display
  • the information processing apparatus recognizes various multimedia information output and / or operation instructions corresponding to each display, and displays the output information on the display.
  • the display can be identified only by the XY coordinate values.
  • a seal on which a code value for identifying each display is printed as a dot pattern is attached to the casing of the display, and the code read by the dot pattern reading unit A value is transmitted to the information processing apparatus, which display is recognized to enable touch-panel input, and the information processing apparatus recognizes various multimedia information output and / or operation instructions corresponding to each display.
  • the output information is displayed on the display.
  • the display since the display is identified by the sticker, the display can be identified even if the same information input auxiliary sheet is used.
  • the sticker on which the dot pattern is printed may be affixed on any place such as an information input auxiliary sheet or a display housing.
  • the information input device is DTP data, HTML, flash, CAD data, map data, etc., and is intended for two-dimensional format information for printing on a medium.
  • information visualized as text, diagrams, illustrations, photographs, etc. information corresponding to a predetermined standard is formatted in advance as specific information, and the medium is used for various multimedia information output and / or operation instructions
  • identification information for identifying the format information
  • the identification information is stored in one or more of the two-dimensional format information stored in advance.
  • the coordinate conversion means for converting the value into (xt, yt), and the xy coordinate system of the two-dimensional format information, in order to specify the position of the specific information, determined when the specific information is laid out and printed,
  • the coordinate value (xi, yi) in the xy coordinate system of the two-dimensional format information is used as selection position information, and the xy of the two-dimensional format information converted from the coordinate value (Xt, Yt) of the dot pattern at the touch position.
  • the distance from the coordinate value (xt, yt) in the coordinate system to one or more selection position information within a predetermined distance is calculated, and the closest coordinates of the selection position information are obtained.
  • the specific information is characterized in that and a specific information selecting means for selecting as input to the own device.
  • 2D format information refers to DTP (Desk Top Publishing) data, etc., which becomes the base electronic data when creating printed matter.
  • DTP Desk Top Publishing
  • content data created by HTML, flash, or the like, CAD data, or map data may be used.
  • Specified information is information such as text, charts, illustrations, photographs, etc. included in 2D format information, and is a unit identified by syntax analysis and image analysis.
  • the electronic data created by the word processor is two-dimensional format information.
  • the photo and text included in the electronic data are specific information.
  • the image is decomposed into Tokyo Tower, Building, and Sky by image analysis, and the text is decomposed into “Hana no Miyako”, “Tokyo”, “Hana”, “Miyako”, “East”, “Kyo”. , Each becomes specific information.
  • the position information for selection is information used to select specific information on the printed matter according to the position touched by the user. For example, the central coordinate value of the area where the phrase “Tokyo” is laid out on the DTP data. It is.
  • 2D format information including specific information analyzed and set with position information for selection is stored in the 2D format information storage selection means.
  • the identification information is information for uniquely specifying the two-dimensional format information.
  • the file name of DTP data may be used, or a predetermined document ID may be used. It may also be a URL.
  • a photograph and text are printed on the medium, and a dot pattern obtained by patterning XY coordinate values and / or code values is superimposed and printed thereon.
  • the coordinate conversion means calculates the xy coordinate value of the touch position on the two-dimensional format information from the extracted two-dimensional format information and the XY coordinate value on the medium of the touch position.
  • the specific information selection means uses the coordinates of the touch position represented by the xy coordinate system on the two-dimensional format information and the selection position information included in each specific information in the two-dimensional format information.
  • the specific information of the touch position is selected by a predetermined algorithm and used as an input to the information input device. Therefore, a mask area is defined in advance for specific information that is not printed with hyperlinks, etc., or a dot code corresponding to the specific information is embedded in the mask area, or specific information corresponding to the dot code.
  • a mask area is defined in advance for specific information that is not printed with hyperlinks, etc.
  • a dot code corresponding to the specific information is embedded in the mask area, or specific information corresponding to the dot code.
  • the XY coordinate values are defined in the medium without any gaps.
  • characters, staffs, maps, figures, etc. are printed on the medium and the operation is performed by tracing or touching the line segment with a scanner pen, by forming a stream dot pattern only along the line segment, A dot pattern can be arranged rationally.
  • the shape of the rectangular area where the dot pattern is formed as a two-dimensional code is not constrained by the definition of the index based on the XY coordinate values, and a free shape that matches the information area visually formed on the medium surface
  • the information input device is a DTP data, HTML, flash, CAD data, map data, etc. for solving the above problems, and is intended for two-dimensional format information to be displayed on a display.
  • information corresponding to a predetermined standard is preliminarily formatted as specific information, and the display is used for various multimedia information output and / or operation instructions
  • the step of providing and at least a straight line and / or a curved line defined at a predetermined position from the reference dot and / or the first virtual reference line Both of the step of providing one or more second virtual reference lines, the step of providing a plurality of virtual reference points at predetermined positions on the second virtual reference line, and the end point represented by a vector using the virtual reference points as starting points.
  • a dot pattern reading for reading a dot pattern at a touch position, which is used for touching the display when the user designates the specific information visualized on the display by a predetermined operation. 1 and stored in advance using the identification information for identifying the two-dimensional format information that has been input.
  • two-dimensional format information storage / selection means for selecting the two-dimensional format information corresponding to the identification information from two or more of the two-dimensional format information, and coordinate values in the XY coordinate system of the dot pattern at the touch position ( Xt, Yt) is converted to coordinate values (xt, yt) in the xy coordinate system of the two-dimensional format information, and the position of the specific information is specified in the xy coordinate system of the two-dimensional format information Therefore, the coordinate value (Xt, Yt) of the touch position is used by using (xi, yi) in the xy coordinate system of the two-dimensional format information, which is determined when the specific information is displayed on the display, as the position information for selection.
  • 2D format information refers to content data that is electronic data of content displayed on a display.
  • the selection position information is information used to select specific information on the display according to the position touched by the user.
  • the content is displayed on the display, and a dot pattern obtained by patterning the XY coordinate value and / or the code value is displayed on the display.
  • the coordinate conversion means calculates the xy coordinate value of the touch position on the two-dimensional format information from the extracted two-dimensional format information and the XY coordinate value on the display of the touch position.
  • the specific information selection means uses the coordinates of the touch position represented by the xy coordinate system on the two-dimensional format information and the selection position information included in each specific information in the two-dimensional format information.
  • the specific information of the touch position is selected by a predetermined algorithm and used as an input to the information input device. Therefore, for specific information that is displayed on the display and for which hyperlinks are not set, a mask area is defined in advance, a dot code corresponding to the specific information is embedded in the mask area, or dot code is supported. There is an effect that the user can input the specific information without creating a dot code / specific information correspondence table in which the specific information is defined in advance.
  • XY coordinate values are defined on the display without any gaps.
  • a dot pattern can be arranged rationally.
  • the shape of the rectangular area where the dot pattern is formed as a two-dimensional code is not constrained by the definition of the index based on the XY coordinate values, and a free shape that matches the information area visually formed on the medium surface
  • the identification information is the code value of the dot pattern printed on the medium or displayed on the display, and the XY coordinate value of the dot pattern is printed or It is preferable to print or display in at least a part of the displayed area together with the XY coordinate value of the dot pattern, or to print or display in a different area from the area.
  • the identification information is the code value
  • the identification information is read by the user's touch operation, so that it is possible to save the user from inputting the identification information separately.
  • the identification information is an XY coordinate value of the dot pattern that uniquely identifies the two-dimensional format information
  • the XY coordinate value is an XY coordinate of the dot pattern.
  • the value is printed over the entire area where it is printed or displayed.
  • the DTP depends on which XY coordinate value is used. Data can be identified.
  • the identification information is an XY coordinate value of the dot pattern that uniquely identifies the two-dimensional format information, and thus there is an additional effect that a process of including the identification information as a code value in the dot pattern becomes unnecessary.
  • the display has a structure in which a dot portion of the dot pattern absorbs or transmits infrared rays irradiated from an infrared irradiation unit included in the dot pattern reading unit.
  • the region excluding the portion is preferably a structure that diffusely reflects the infrared rays.
  • the region other than the dot portion of the display has an additional effect that reflection unevenness that occurs during specular reflection can be prevented by diffusing and reflecting infrared rays.
  • the dot pattern reading means can increase the contrast ratio between the dots and the other portions so that the dot pattern can be read reliably. It has the additional effect of being able to.
  • the information input device has a linear shape on the surface of the information input auxiliary sheet according to a predetermined rule for various multimedia information output and / or operation instructions.
  • Arranging a plurality of reference dots in succession providing a first virtual reference line composed of a straight line, a broken line and / or a curve connecting the plurality of reference dots, and the reference dots and / or the first Providing at least one or more second virtual reference lines made of straight lines and / or curves defined at predetermined positions from one virtual reference line, and a plurality of positions at predetermined positions on the second virtual reference line
  • An XY coordinate value and / or code value is defined by a step of providing a virtual reference point and a distance and direction from the virtual reference point at an end point represented by a vector with the virtual reference point as a starting point.
  • information that meets a predetermined standard is pre-defined as specific information and formatted, and the two-dimensional format information is printed
  • the user designates the information input auxiliary sheet placed or pasted on the displayed medium or the display to be displayed and the specific information visualized on the medium or the display by a predetermined operation Used to touch the medium or the display through the information input auxiliary sheet.
  • 2D format information storage / selection means for selecting the 2D format information corresponding to the identification information from among the coordinates, and the coordinate values (Xt, Yt) in the XY coordinate system of the dot pattern at the touch position
  • the specific information is laid out in a coordinate conversion means for converting the coordinate value (xt, yt) in the xy coordinate system of the information (Xi, yi) in the xy coordinate system of the two-dimensional format information determined when printing or when the specific information is displayed on the display 1 or within a predetermined distance from the coordinate value (xt, yt) in the xy coordinate system of the two-dimensional format information converted from the coordinate value (Xt, Yt) of the dot pattern at the touch position.
  • the information input auxiliary sheet is a transparent sheet, and a dot pattern is provided by printing or the like. It is used by being superimposed on a medium on which content is printed or a display on which content is displayed. In order to input the specific information on the medium or the display, the user touches the desired specific information on the information input auxiliary sheet by the dot pattern reading unit.
  • the user inputs information for specifying the printed or displayed content as necessary.
  • Two-dimensional format information corresponding to the content is extracted from the two-dimensional format information storage / selection means.
  • the coordinate conversion means calculates the xy coordinate value of the touch position on the two-dimensional format information from the extracted two-dimensional format information and the XY coordinate values on the touch position information input auxiliary sheet.
  • the specific information selection means uses the coordinates of the touch position represented by the xy coordinate system on the two-dimensional format information and the selection position information included in each specific information in the two-dimensional format information.
  • the specific information of the touch position is selected by a predetermined algorithm and used as an input to the information input device. Therefore, even if the dot pattern is not printed directly on a printed matter such as a book, magazine, or newspaper, a mask area can be defined or masked for specific information that is printed or displayed without hyperlinks set.
  • the user can input specific information without embedding a dot code corresponding to the specific information in the area or creating a dot code / specific information correspondence table in which the specific information corresponding to the dot code is defined in advance. There is an effect.
  • the XY coordinate values are defined in the information input auxiliary sheet without any gaps.
  • characters, staffs, maps, figures, etc. are printed on the information input auxiliary sheet, and when operating on the line segment by tracing or touching with a scanner pen, a stream dot pattern is formed only along that line segment.
  • the dot pattern can be arranged rationally.
  • the shape of the rectangular area where the dot pattern is formed as a two-dimensional code is not constrained by the definition of the index based on the XY coordinate values, and a free shape that matches the information area visually formed on the medium surface
  • an index for identifying the two-dimensional format information printed on the medium or displayed on the display is provided on the information input auxiliary sheet by the user.
  • a plurality of icons for input are printed together with the dot pattern, and the two-dimensional format information storage and selection unit uses the dot pattern reading unit to read the index input by touching the icon. It is preferable to convert the identification information and select the two-dimensional format information using the converted identification information.
  • the predetermined operation is performed by the user touching or touching and tracing the specific information visualized by using the dot pattern reading unit. Preferably it is an operation.
  • the predetermined operation is performed by a user touching the visualized specific information using the dot pattern reading unit and tracing an area where the specific information is displayed. It is preferable that the operation be surrounded.
  • the specific information selection unit only needs to select specific information having the enclosed selection position information. Therefore, when the specific information is selected based on the coordinate value of only one point obtained by the touch operation. In comparison, there is an additional effect that the specific information desired by the user can be selected more reliably.
  • the position for selection is set for each character.
  • the specific information is one chart, illustration, photograph, etc. and there are a plurality of objects constituting these, it is preferable that the position information for selection is included for each object.
  • the specific information since the specific information has position information for selection for each character at a distant position, and for each chart, illustration, and photo, the additional effect that the specific information can be reliably selected is provided. Play.
  • the selection position information is a phrase that is specific information or a central coordinate value of each character constituting the phrase, or a chart, illustration, or photo that is specific information Or the centroid coordinate values of the objects constituting them.
  • the selection position information is a phrase that is specific information, or the phrase or the character at any position in the height direction of each character constituting the phrase. It is a combination of a start point coordinate value (xi1, yi1) and an end point coordinate value (xi2, yi2) of the line segment indicating the width, and the specific information selecting means calculates the line from the coordinate value (xt, yt) of the touch position It is preferable to select the specific information whose distance to the minute is within an arbitrary distance and has the closest line segment.
  • the specific information selecting means is a text within an arbitrary distance from the coordinate value (xt, yt) of the touch position in the two-dimensional format information by a predetermined algorithm. 1 and / or two or more of the specific information is recognized, and the position information for selection of the specific information is predetermined.
  • the specific information is selected from the coordinate values (xt, yt), the recognized specific information, and the calculated position information for selection.
  • the specific information selection unit acquires the two-dimensional format information and then performs syntax analysis / image analysis on a limited range close to the touch position in the content, the syntax analysis is performed in advance. -It has the further effect of saving the trouble of preparing two-dimensional format information in which image analysis is performed and selection position information is set.
  • the specific information includes text analysis and / or file search and / or image analysis such as a figure, an illustration, and a photograph included in the two-dimensional format information.
  • the position information for selection of the specific information is calculated in advance by a predetermined algorithm, and the specific information and the position information for selection are recorded in the two-dimensional format information. Preferably it is.
  • the two-dimensional format information is analyzed in advance, the position information for selection is set, the specific information is specified, and the specific information selection unit acquires the two-dimensional format information and then parses it. Since there is no need to perform image analysis, there is a further effect that the processing time from acquiring the two-dimensional format information to selecting specific information can be shortened.
  • the two-dimensional format information includes text and / or a figure having a point, a line segment, a plane, and a three-dimensional object as the specific information drawn by CAD. It is preferable that
  • the two-dimensional format information is map data
  • the map data has specific information classified into categories such as text, road, feature, intersection, and the like.
  • the specific information preferably includes a region surrounded by a road, a feature region, a road center line, and a feature centroid as selection position information.
  • An information input system comprises the information input device according to claim 1, a two-dimensional format information server, and a network connecting them in order to solve the problem.
  • the information input device transmits the input identification information for identifying the two-dimensional format information to the two-dimensional format information server, instead of the two-dimensional format information storage and selection unit.
  • Communication means for receiving two-dimensional format information corresponding to the identification information from the two-dimensional format information server, the two-dimensional format information server using the identification information received from the information input device, Two-dimensional format information storage selection means for selecting two-dimensional format information corresponding to the identification information from one or more of the two-dimensional format information stored in advance, and the identification information Received from the serial data input device, and the selected the two-dimensional format information characterized by comprising: a communication unit for transmitting to the information input device.
  • the 2D format information server may be a server that provides 2D format information including specific information that is analyzed in advance and set with position information for selection, or may be a normal Web server.
  • the specific information selecting unit acquires the two-dimensional format information, Analysis is performed so as to set position information for selection and specific information.
  • the information input device has a client / server configuration in which the two-dimensional format information corresponding to the identification information is acquired from the two-dimensional format information server on the network. There is an effect that the two-dimensional format information server can be shared.
  • An information input system comprises an information input device according to claim 2, a two-dimensional format information server, and a network connecting them in order to solve the problem.
  • the information input device transmits the input identification information for identifying the two-dimensional format information to the two-dimensional format information server, instead of the two-dimensional format information storage and selection unit.
  • Communication means for receiving two-dimensional format information corresponding to the identification information from the two-dimensional format information server, the two-dimensional format information server using the identification information received from the information input device, Two-dimensional format information storage selection means for selecting two-dimensional format information corresponding to the identification information from one or more of the two-dimensional format information stored in advance, and the identification information Received from the serial data input device, and the two-dimensional format information selected by comprising the communicating means for transmitting to said information input device.
  • the information input device has a client / server configuration in which the two-dimensional format information corresponding to the identification information is acquired from the two-dimensional format information server on the network. There is an effect that the two-dimensional format information server can be shared.
  • An information input system includes an information input device according to claim 1, a two-dimensional format information server, and a network connecting them in order to solve the problem.
  • the information input device transmits the input identification information for identifying the two-dimensional format information to the two-dimensional format information server, instead of the two-dimensional format information storage and selection unit.
  • Communication means for receiving two-dimensional format information corresponding to the identification information from the two-dimensional format information server, the two-dimensional format information server using the identification information received from the information input device, Two-dimensional format information storage selection means for selecting two-dimensional format information corresponding to the identification information from one or more of the two-dimensional format information stored in advance, and the identification information Received from the serial data input device, and the two-dimensional format information selected by comprising the communicating means for transmitting to said information input device.
  • the information input device has a client / server configuration in which the two-dimensional format information corresponding to the identification information is acquired from the two-dimensional format information server on the network. There is an effect that the two-dimensional format information server can be shared.
  • the stream dot defines a second virtual reference line and / or defines the direction of the dot pattern and one XY coordinate value and / or code value. Therefore, it is preferable to provide a reference dot serving as a reference at a predetermined position.
  • a part of the code value includes at least a code value for specifying the medium, the display, or the information input auxiliary sheet.
  • a handwriting input sheet includes a step of arranging a plurality of reference dots continuously in a line according to a predetermined rule in at least a part or a plurality of arbitrarily shaped regions; Providing a first virtual reference line composed of a straight line, a polygonal line and / or a curve connecting the reference dots, and a straight line and / or defined at a predetermined position from the reference dot and / or the first virtual reference line Alternatively, a step of providing at least one or more second virtual reference lines made of a curve, a step of providing a plurality of virtual reference points at predetermined positions on the second virtual reference line, and a vector starting from the virtual reference points Stream dots arranged in accordance with the process of arranging XY coordinate values or information dots in which XY coordinate values and code values are defined by the distance and direction from the virtual reference point at the represented end points. Characterized by comprising the medium having a plurality side by side writing area on
  • the XY coordinate values are defined without any gaps in the writing area, and trajectory information can be generated, enabling handwritten input.
  • a stream dot pattern is formed only along that line segment.
  • a dot pattern can be arranged rationally.
  • a dot pattern in which XY coordinates are defined is formed as a two-dimensional code (used as an index), it is not restricted by the shape of the rectangular area, and is matched to the information area visually formed on the medium surface.
  • it is possible to form a dot pattern by repeating a set of constant information in a free shape.
  • the stream dot is a reference that is further used as a reference in order to define a second virtual reference line and / or to define the direction of the dot pattern and one XY coordinate value and / or code value. You may provide a dot in a predetermined position.
  • the direction of the stream dot pattern and the set of constant information can be easily defined without using information dots, and the reduction of unnecessary information can be suppressed. Furthermore, the position of the virtual reference point that is the starting point of the information dot can be accurately indicated by the arrangement of the new reference point.
  • a handwriting input / output system is a handwriting input / output system using the handwriting input sheet according to claim 1 or 2, wherein the stream dot pattern formed in the writing area is imaged, From the coordinate information, the coordinate information, and the code information by storing the image data of the stream dot pattern captured and transmitted by the imaging unit in the storage element and analyzing the code of the image data Dot pattern analysis means for obtaining the trajectory information, and processing command means for transmitting the trajectory information to the information processing means.
  • the processing instruction means sends the information to the information processing unit.
  • the trajectory information and processing instructions based on the trajectory information can be transmitted in a batch.
  • the handwriting input / output system further includes a trajectory recognizing unit for recognizing the trajectory information of the characters and / or figures traced on the writing area surface based on the change of the analyzed coordinate information. It is preferable that the processing command means for transmitting a processing command based on the recognized information to the information processing means.
  • the handwriting input / output system can leave a locus in the vicinity of the imaging means on the writing area surface.
  • information specifying the medium is defined in at least a part of the code value, and / or specifying information specifying the writing area is included in at least a part of the code value.
  • Operation instructions program start, command input, data input / output / search, search target specification, etc.
  • write processing are defined in at least a part of the code value.
  • the stream dot pattern formed in and / or around the print area is overlaid with the associated text, graphics, or photo, and / or braille in and / or around the writing area And / or a narrow convex guide along the outer periphery of the writing area so that the imaging means for imaging the dot pattern does not protrude when writing to the writing area.
  • the above-mentioned stream dot pattern which is a sticker, sticky note, or other medium that can be applied and / or used on a monitor surface or a printed medium on which a writing area is displayed. It may be a transparent information input auxiliary sheet on which is formed.
  • the handwriting input / output system can easily know what the user should write in the position of the writing area and the writing area.
  • the user can easily know what meaning the code information has (what processing instruction is transmitted) by looking at the icon.
  • a guide is provided so that the writing area can be accurately traced and drawn by the imaging means, which contributes to maintaining the same input efficiency as that of a healthy person when a visually handicapped person performs handwriting input.
  • the flexibility of the layout of the writing area is further increased, and the search and output of information input by handwriting in the writing area can be freely executed according to the convenience and preference of the user.
  • a touch panel with good handling can be provided.
  • handwriting input is possible by placing the grid sheet on the medium surface without overlapping the grid sheet on the monitor surface.
  • the medium forms a dot pattern in which code information for specifying the first medium surface and / or the writing area surface is repeatedly defined with the medium provided with the writing area as the first medium.
  • the code information formed on the second medium is read by the imaging unit, and is printed on the print information on the first medium surface and / or the writing area surface.
  • the trajectory information and / or information related to the trajectory information may be output.
  • the handwriting input / output system prints code information for specifying a medium surface on another medium (such as a notebook) and touches it with an imaging unit, thereby handwriting input on the medium surface. Can be searched and output.
  • a wide space in the writing area of the medium it is possible to take a wide space in the writing area of the medium.
  • the imaging means further includes an instruction protrusion and / or at least one pen tip, and characters and / or traced on the writing area surface with the instruction protrusion and / or at least one pen tip.
  • the pointing projection further includes a storage space for controlling and a button for controlling storage / removal of the pen tip.
  • the handwriting input / output system makes it easy to visually recognize the position of the imaging means on the writing area surface.
  • a locus near the imaging means can be left on the writing area surface.
  • the imaging means further includes an erasing means for erasing characters and / or figures drawn on the writing area surface on the instruction protrusion, and an area traced by the erasing means is formed on the writing area surface.
  • the processing instruction means outputs a processing instruction for erasing the locus information of the character and / or figure and the locus information. May be sent to.
  • the handwriting input / output system can cope with not only writing information input on the writing area surface and the information processing means but also erasing the writing information.
  • the trajectory information includes coordinate information of the imaging center of the imaging unit, or coordinate information of a position where the pointing protrusion or the pen tip provided near the imaging unit touches the writing area surface and the code
  • the information is obtained by interpolation calculation based on coordinate information defined in the dot pattern by the dot pattern analysis means.
  • the handwriting input / output system matches the actual trajectory by correcting the trajectory information even when the trajectory information recognized by the trajectory recognition unit is different from the trajectory on the medium. Or it can be approximated to such an extent that there is no trouble in practical use.
  • the trajectory recognition unit further includes a timing unit, and the timing unit measures a time when the imaging unit touches the writing area surface and a time away from the writing area surface, and the imaging unit touches the writing area surface. It is also possible to record the order of the traces drawn and the touching time, and transmit them to the information processing means.
  • the handwriting input / output system grasps the time required for the handwriting input by the information processing means and compares it with the normal time, so that the character's personality, age, gender, etc. Various research is possible.
  • the processing command means may specify, based on the code information and / or the recognition information, specific information and / or an operation instruction (program activation, command input, data input / output / search, It may be possible to uniquely input a search target designation or the like.
  • the processing command means can give a unique instruction to the information processing means based on the code information and / or the recognition information. This is the most important function in the present invention, and it is possible to easily specify which writing area is always written or what operation instruction is given to which writing area.
  • the trajectory recognition unit may further recognize the trajectory information with high accuracy by referring to the character and / or graphic information patterns classified in advance regarding the contents to be written in the writing area based on the code information. preferable.
  • the handwriting input / output system refers to the character and / or graphic information pattern table previously classified based on the code information when recognizing the trajectory information. It can be recognized with accuracy.
  • the trajectory recognizing means After recognizing the trajectory information, the trajectory recognizing means further determines the character and / or the figure, or a phrase composed of one or more of the characters, based on the change in the analyzed coordinate information.
  • the semantic information is converted into a semantic information code by referring to a predefined semantic information code table (dictionary) corresponding to the code information, and the processing instruction means is based on the semantic information code and the semantic information.
  • the processing instruction is transmitted to the information processing means.
  • the handwriting input / output system recognizes the trajectory information with high accuracy by referring to the semantic information code table at the time of trajectory recognition, and further converts it into a semantic information code.
  • a processing command can be transmitted as that meaning.
  • the processing instruction means may be a first operation for inputting the recognition result of the trajectory information by the trajectory recognition means as a document and / or a graphic, or a second operation for inputting as a comment, or a document and
  • the information processing unit may be instructed as a third operation for editing the document and / or the figure and / or the comment based on the edit symbol and / or the figure and / or the comment that edits the comment. .
  • the handwriting input / output system can write a document or draw a picture by handwriting input.
  • comment documents can be input in addition to documents and pictures input by handwriting input.
  • documents and images can be edited by handwriting input.
  • the imaging optical axis of the imaging means that is in contact with the medium surface is tilted and tilted by being tilted with respect to a vertical line of the medium surface.
  • an angle measuring means for measuring the inclination and / or an action of tilting the imaging means with reference to the inclination and the direction measured when the imaging means is in contact with the medium surface, that is, a grid tilt action.
  • an angle change recognizing means for analyzing a predetermined change in the direction, wherein the processing instruction means may transmit a predetermined processing instruction to the information processing means based on the result of the analysis. Good.
  • the handwriting input / output system can give an operation instruction by the grid tilt operation of the imaging means.
  • the dot pattern arranged at a predetermined angle with respect to the direction of the writing area is formed in the writing area, and the angle measuring unit and the angle change recognizing unit are configured by the imaging unit. Based on the direction of the dot pattern read when contacting the surface of the writing area, a predetermined change in the direction of tilting and / or the inclination with respect to the direction of the writing area may be analyzed.
  • the handwriting input / output system can recognize the inclination and the change thereof with reference to the direction of the dot pattern in addition to the change of the angle.
  • the handwriting input / output system is configured such that the imaging optical axis of the imaging unit is substantially vertical with respect to the writing area surface, and the direction of the dot pattern read when the writing area surface contacts the writing area surface.
  • a rotation angle reading means for reading a rotation angle formed by the angle of the imaging means, and an operation for rotating the imaging means with reference to the rotation angle measured when the imaging means comes into contact with the medium surface.
  • rotation angle change recognizing means for analyzing a predetermined change in the rotation angle by rotating the imaging optical axis in a predetermined rotation direction by a grid turn operation, and the processing command means includes: Based on the result of the analysis, a predetermined processing command may be transmitted to the information processing means.
  • the handwriting input / output system can give an instruction by a grid turn operation.
  • the handwriting input / output system further captures the writing area by the imaging means, obtains coordinate information and code information by the dot pattern analysis means, and then performs the writing area by the imaging means by a predetermined operation.
  • a voice recording means for recording voice instead of tracing or drawing, and a predefined voice recognition information table (dictionary) corresponding to the code information, the recorded voice is recognized and converted into character information.
  • Voice recognition means for converting and transmitting to the information processing means may be further provided.
  • the handwriting input / output system can perform voice input instead of handwriting input.
  • a device equipped with any display means such as a general-purpose computer, PDA, or mobile phone, can be used as a touch panel type input means very easily.
  • the information input apparatus is capable of outputting various multimedia information and / or operation instructions by formatting the specific information visualized as text, diagrams, illustrations, photographs, and the like. Therefore, when the user designates the specific information visualized on the printed medium together with the dot pattern obtained by patterning the XY coordinate value and / or the code value by a predetermined algorithm by a predetermined operation, the medium 1 or 2 stored in advance by using dot pattern reading means for reading the dot pattern at the touch position, and input identification information for identifying the two-dimensional format information.
  • a 2D document for selecting the 2D format information corresponding to the identification information from the 2D format information.
  • Information storage selection means and coordinates for converting the coordinate value (Xt, Yt) in the XY coordinate system of the dot pattern at the touch position to the coordinate value (xt, yt) in the xy coordinate system of the selected two-dimensional format information In the xy coordinate system of the conversion means and the two-dimensional format information, in order to specify the position of the specific information, the position information for selection included in the specific information is used, and the coordinate value of the dot pattern of the touch position From the coordinate value (xt, yt) in the xy coordinate system of the two-dimensional format information converted from (Xt, Yt), the specific information having the coordinates of the position information for selection that is within an arbitrary distance and is closest And a specific information selecting means for selecting as an input to the own apparatus.
  • the information input device can output various multimedia information and two-dimensional format information obtained by formatting specific information visualized as text, diagrams, illustrations, photographs, and the like.
  • the user displays a display to be displayed together with a dot pattern obtained by patterning XY coordinate values and / or code values by a predetermined algorithm, and the specific information visualized on the display by a user.
  • the dot pattern reading means for reading the dot pattern at the touch position which is used for touching the display
  • the input identification information for identifying the two-dimensional format information From one or more of the two-dimensional format information stored in advance
  • Two-dimensional format information storage / selection means for selecting the two-dimensional format information corresponding to the identification information, and the coordinate values (Xt, Yt) in the XY coordinate system of the dot pattern at the touch position are selected as the selected two-dimensional format information.
  • the coordinate information is converted into coordinate values (xt, yt), and the specific information has the specific information in order to specify the position of the specific information in the xy coordinate system of the two-dimensional format information.
  • Specific information selection means for selecting specific information having the coordinates of the closest position information for selection as input to the own apparatus is provided.
  • the information input device prints a dot pattern in which XY coordinate values and / or code values are patterned by a predetermined algorithm for various multimedia information output and / or operation instructions.
  • An information input auxiliary sheet in which two-dimensional format information in which specific information visualized as text, diagrams, illustrations, photographs, etc. is formatted is placed or pasted on a printed medium or a display to be displayed;
  • the user designates the specific information visualized on the medium or the display by a predetermined operation, it is used to touch the medium or the display through the information input auxiliary sheet.
  • Dot pattern reading means to read the dot pattern at the touch position, and input Using the identification information for identifying the two-dimensional format information, the two-dimensional format information corresponding to the identification information is selected from one or more of the two-dimensional format information stored in advance. And a coordinate value (Xt, Yt) in the XY coordinate system of the dot pattern at the touch position, and a coordinate value (xt, yt) in the xy coordinate system of the selected two-dimensional format information.
  • the position information for selection included in the specific information is used to specify the position of the specific information, and the dot of the touch position From the coordinate value (xt, yt) in the xy coordinate system of the two-dimensional format information converted from the coordinate value (Xt, Yt) of the pattern, the closest selection position information within the arbitrary distance.
  • Specific information having mark is characterized in that a specific information selecting means for selecting as input to the own device.
  • the information input method can output various multimedia information and / or operations using two-dimensional format information in which specific information visualized as text, diagrams, illustrations, photographs, etc. is formatted.
  • a dot pattern reading unit for reading a dot pattern at a touch position, and input identification information for identifying the two-dimensional format information
  • a dot pattern reading step for reading a dot pattern at a touch position
  • input identification information for identifying the two-dimensional format information
  • the coordinate conversion step for converting to the coordinate
  • the information input method can output various multimedia information and / or operations using two-dimensional format information in which specific information visualized as text, diagrams, illustrations, photographs, etc. is formatted.
  • two-dimensional format information in which specific information visualized as text, diagrams, illustrations, photographs, etc. is formatted.
  • the user designates the specific information visualized on the displayed display together with the dot pattern obtained by patterning the XY coordinate value and / or the code value by a predetermined algorithm by a predetermined operation
  • a dot pattern reading step for reading a dot pattern at a touch position, and input identification information for identifying the two-dimensional format information
  • a two-dimensional format information storage selection step for selecting the two-dimensional format information corresponding to the identification information, and the coordinate values (Xt, Yt) in the XY coordinate system of the dot pattern at the touch position are selected as the selected two-dimensional format information.
  • the specific information includes the specific information.
  • the information input method prints a dot pattern in which XY coordinate values and / or code values are patterned by a predetermined algorithm for various multimedia information output and / or operation instructions.
  • An information input auxiliary sheet in which two-dimensional format information in which specific information visualized as text, diagrams, illustrations, photographs, etc. is formatted is placed or pasted on a printed medium or a display to be displayed;
  • the user designates the specific information visualized on the medium or the display by a predetermined operation, it is used to touch the medium or the display through the information input auxiliary sheet.
  • the dot pattern reading step for reading the dot pattern at the touch position and the input identification information for identifying the two-dimensional format information are used to store one or more of the two-dimensional format information stored in advance.
  • the two-dimensional format information storage and selection step for selecting the two-dimensional format information corresponding to the identification information from the inside, and the coordinate values (Xt, Yt) in the XY coordinate system of the dot pattern at the touch position are selected.
  • the specific information is present.
  • a mask area is defined in advance for the specific information that is printed / displayed and no hyperlinks are set, or a dot code corresponding to the specific information is embedded in the mask area.
  • the present invention provides a handwriting input / output system that has various features described above and can give a user various and new added value and convenience that are completely different from those of the prior art. Can do.
  • FIG. 1 (a) is a figure which shows the mode of operation
  • FIG.1 (b) is a figure which shows a mode that specific information is selected.
  • FIG. 8A shows an example of a dot code format.
  • FIG. 8A shows an example in which the dot code includes an XY coordinate value, a code value, and a parity.
  • FIG. 8B shows a dot code 103 depending on where the dot pattern 103 is provided.
  • FIG. 18C shows an example in which the format is changed, and FIG. 18C shows an example in which the dot code includes XY coordinate values and parity.
  • FIG. 31A is a cross-sectional view of the printed material 101a, and FIG. 31A is a diagram illustrating a state in which infrared rays are diffused and reflected on a portion of the printing medium 101c where nothing is printed, and FIG. FIG.
  • FIG. 31C is a diagram showing a state where the infrared rays hit the dot pattern 103 and is absorbed.
  • FIG. 2 is a diagram illustrating a structure of a portion of a scanner 104 that reads a dot pattern 103 and how the dot pattern 103 on a printed material 101a is read. It is a functional block diagram of the information input device 105 which concerns on 1st this embodiment. It is a figure which shows the example in the case of using the center coordinate value or centroid coordinate value of an object as position information for selection, FIG.34 (a) is a case where specific information is a text, FIG.34 (b) The specific information is a chart, and FIG. 34 (c) is a case where the specific information is an illustration / photo.
  • FIG.35 (a) is an example which uses the starting point coordinate and height of a word as position information for selection
  • FIG.35 (b) is the starting point coordinate of a word
  • FIG. 35C shows an example in which the start point coordinates, the end point coordinates, and the height of each character are selected when the characters constituting the word are separated. This is an example of information.
  • FIG. 39A is a diagram illustrating an example of a method for selecting specific information
  • FIG. 39A illustrates a case where a perpendicular foot drawn from a touch position with respect to a line segment does not intersect the line segment
  • FIG. This is a case where a perpendicular foot drawn from the touch position with respect to the line segment intersects the line segment.
  • FIG. 40A is a diagram illustrating an example of a method for selecting specific information.
  • FIG. 40A illustrates a case where the touch position is within the mask area 102b
  • FIG. 40B illustrates that the touch position is within the mask area 102b.
  • 10 is a flowchart of a procedure of information input processing performed by the information input device 105. It is a flowchart explaining in detail the selection process of specific information performed in process S40. It is a functional block diagram of the information input device 105a which concerns on 2nd Embodiment. It is a functional block diagram of information processor 112 concerning a 3rd embodiment. 10 is a flowchart for describing a procedure of information processing performed by the information processing apparatus 112. It is a figure which shows the mode of the information processing which information processing apparatus 112 performs.
  • FIG. 49A is a cross-sectional view of an organic EL display provided with a dot portion
  • FIG. 49A is a diagram illustrating a state in which infrared rays incident on the dot portion are transmitted or absorbed
  • FIG. 50A is a diagram illustrating an example in which a dot portion is provided on a screen of a front projector or a rear projector
  • FIG. 50A is a diagram illustrating a state in which infrared light incident on the dot portion is transmitted or absorbed
  • FIG. 50A is a cross-sectional view of an organic EL display provided with a dot portion
  • FIG. 49A is a diagram illustrating a state in which infrared rays incident on the dot portion are transmitted or absorbed
  • FIG. 50A is a diagram illustrating a state in which infrared light incident on the dot portion is transmitted or absorbed
  • FIG. 50A is a cross-sectional view of an organic EL display provided with a dot portion
  • FIG. (2) explaining the function of the imaging part used for the handwriting input / output system concerning this invention.
  • FIG. (3) explaining the function of the image pick-up part used for the handwriting input / output system concerning this invention.
  • FIG. (14) explaining the function of the medium in which the writing area used for the handwriting input / output system concerning this invention was formed. It is a figure explaining the discriminating method of the operation
  • FIG. (2) explaining the 3rd operation in the handwriting input / output system concerning this invention.
  • FIG. (3) explaining the 3rd operation in the handwriting input / output system concerning this invention.
  • FIG. (4) explaining the 3rd operation in the handwriting input / output system concerning this invention.
  • FIG. (5) explaining the 3rd operation in the handwriting input / output system concerning this invention.
  • FIG. (6) explaining the 3rd operation in the handwriting input / output system concerning this invention.
  • 1 and 2 sequentially show an example of a process for forming a stream dot pattern.
  • the dot pattern according to the present invention is different from the conventional dot pattern.
  • step 1 corresponding to the visible information on the surface of the medium, the reference dot 1 is continuously arranged in a line at a position where information is input / output. Place multiple.
  • the reference dots 1 are arranged in a curved line.
  • the arrangement of the reference dots 1 is not limited to this, and is composed of a straight line and a curved line or a plurality of line segments.
  • Various modifications can be made to form the dot pattern in a shape that matches the area where information is input and output, such as a polygonal line.
  • the reference dot 1 may be arranged on a real line visually formed on the medium surface, or the reference dot 1 may be arranged according to a predetermined rule along the real line.
  • the real line here is a concept with respect to a virtual line, and includes all the lines that actually exist. For example, a solid line, a broken line, a dotted line, a straight line, a curved line, and the like can be given. Absent.
  • the dot pattern may be unevenness such as printing, display display, or a hole or groove on a metal or plastic.
  • the reference dots are not limited to this, and a plurality of intervals are mixed to define a set of constant information of the dot pattern, It is also possible to define both a set of constant information of a dot pattern and a direction of the dot pattern by arrangement intervals of three different reference dots in the set of fixed information.
  • a first virtual reference line 2 that connects the reference dots 4 arranged in a line is provided.
  • the first virtual reference line 2 is provided by a curved line, but the first virtual reference line 2 is not limited to this, and the reference virtual dots 1 arranged in a curved line are not limited thereto.
  • the straight first virtual reference line 2 may be provided, or the curved first virtual reference line 2 may be provided for the reference dots 1 arranged in a straight line. That is, the second virtual reference line 3, virtual reference point 4, and information dot 5 in Step 3 to Step 5 to be described later are arranged at positions where the reference dots are connected to form a first line consisting of a straight line, a broken line and / or a curve.
  • One virtual reference line 2 can be freely defined.
  • the first virtual reference line 2 in the case of a curve is preferably a Bezier curve.
  • the reference dots 4 on the first virtual reference line 2 are P0 and P3, and P1 and P2 are given control points.
  • points P4, P5, and P6 are obtained by dividing the three line segments P0-P1, P1-P2, and P2-P3 obtained by connecting the control points in order at a one-to-one ratio.
  • points P7 and P8 are obtained by dividing the two line segments P4-P5 and P5-P6 obtained by connecting these points in order at a one-to-one ratio.
  • the first virtual reference line 2 may be provided by using various algorithms such as a spline curve obtained by using a spline function, an nth order polynomial, an elliptic arc, and the like, not limited to a Bezier curve.
  • a second virtual reference line 3 defined at a predetermined position from the reference dots 1 and / or the first virtual reference line 6 arranged in a line is provided.
  • the second virtual reference line 3 is adjacent to the reference dot 1 adjacent to the predetermined position on the vertical line with respect to the tangent line of the first virtual reference line 3 at the midpoint between the adjacent reference dots 1.
  • the second virtual reference line 3 is not limited to this, and a virtual reference point is provided in accordance with an area in which information is input / output by a dot pattern as will be described later. In addition, it can be defined by various methods.
  • the second virtual reference line 3 may be provided only on one side with respect to the first virtual reference line 2 to define the direction of the dot pattern, or may be provided on both sides to increase the amount of information. .
  • a plurality of virtual reference points 4 are provided at predetermined positions on the second virtual reference line 3.
  • the virtual reference point 4 is an isosceles triangle having an intersection of the second virtual reference lines 3, that is, a straight line connecting adjacent reference dots 1 as a base and the second virtual reference line 3 as an opposite side.
  • the position of the virtual reference point 4 is not limited to this, and is provided at the midpoint of the second virtual reference line 3 or instead of the second virtual reference line 3.
  • Various modifications such as provision on the dot 1 are possible.
  • step 5 the information dot 5 is arranged at the end point expressed by a vector with the virtual reference point 4 as the start point.
  • one information dot 5 is arranged for one virtual reference point 4 so that the vector direction from the virtual reference point 4 is eight directions and the distance from the virtual reference point 4 is equal.
  • the arrangement of the information dots 5 is not limited to this, and the information dots 5 are arranged on the virtual reference point 4, the vector direction is set to 16 directions, or two are arranged for one virtual reference point 4. It is possible to arrange a plurality of elements in any direction and in any length.
  • FIG. 4 is an enlarged view showing an example of a dot pattern information dot and a bit display of data defined therein.
  • Information dot 5 is a dot for recognizing various information.
  • the information dot 5 is arranged at the end point represented by a vector with the virtual reference point 4 as the start point. For example, as shown in FIG. 4, the information dot 5 is rotated by 45 degrees in the clockwise direction because a dot 0.1 mm away from the virtual reference point 5 has a direction and a length expressed by a vector. These are arranged in 8 directions to represent 3 bits.
  • 3 bits are expressed by arranging in 8 directions, but the present invention is not limited to this, and 4 bits can be expressed by arranging in 16 directions. Of course, it can be arranged in length.
  • the information dot 5 is arranged at the end position with the virtual reference point 4 as a starting point.
  • Information may be defined depending on whether or not is arranged. For example, information can be defined as “1” if a dot is arranged on a virtual reference point, and “0” if no dot is arranged.
  • FIG. 5 is an example of information dots and bit display of data defined therein, and shows other forms.
  • the information dot 5 if two types of short (upper part in FIG. 5) and long (lower part in FIG. 5) are used from the virtual reference point 5 derived from the reference dot 1 and the vector direction is eight directions, then 4 bits. Can be expressed. At this time, it is desirable that the longer one is about 25 to 30% of the distance between adjacent virtual reference points 5, and the shorter one is about 15 to 20%. However, it is desirable that the center interval between the long and short information dots 3 is longer than the diameter of these dots.
  • the information dot 5 is preferably one dot in consideration of its appearance. However, if the appearance is ignored and the amount of information is to be increased, a large amount of information can be obtained by assigning 1 bit to each vector and expressing the information dot 3 with a plurality of dots.
  • an information dot 3 defined from the reference dots 4 can represent information of 2 8, and 2 64 in chunks of the information dot eight constant information 1.
  • the stream dot pattern according to the present invention is continuous with a line including a curve unlike the reference dot formed in a two-dimensional lattice pattern in the conventional dot pattern proposed by the present inventor. Are formed based on the reference dots arranged in this manner.
  • FIG. 6 is a diagram illustrating an example of a state where stream dot patterns are arranged in the vertical direction.
  • key dots and side dots are arranged in addition to the reference dots and information dots.
  • Key dots are dots arranged at both ends of a set of fixed information.
  • This key dot is a representative point of the dot pattern 1 for one region representing a group of information dots.
  • the side dots are dots arranged on the positive and negative extension lines of the shift of the key dot 2.
  • (B) in FIG. 6 has reference dots and stream dot patterns arranged at equal intervals.
  • the stream dot pattern according to the present invention is not limited to this, and the interval between the dot patterns may be arbitrarily set as shown in FIG. Also, the interval between the reference dots can be set arbitrarily.
  • the stream dot pattern according to the present invention is continuous with a line including a curve unlike the reference dot formed in a two-dimensional lattice pattern in the conventional dot pattern proposed by the present inventor. Are formed based on the reference dots arranged in this manner.
  • the dot pattern in which the XY coordinates are defined is formed in the information area visually formed on the surface of the medium without being restricted by the shape of the rectangular area when used as a two-dimensional code (used as an index). It is possible to form a dot pattern by repeating a set of constant information in a combined free shape.
  • the virtual reference line and the virtual reference point according to the present invention are not actually printed and formed on the surface of the medium, but only when the dot pattern is arranged on the image memory of the computer or when the dot pattern is read. Is set virtually.
  • FIGS. 7 to 8 are explanatory views showing an embodiment in which the grid sheet 7 which is an information input auxiliary sheet according to the present invention is mounted on a display and used.
  • FIG. 7 is a diagram in which a grid sheet is used in a general-purpose computer system that is an information processing apparatus.
  • a grid sheet is pasted on a display screen such as a liquid crystal display (LCD) such as a personal computer or a CRT (CRT).
  • the grid sheet is formed of a transparent film, and is printed with a dot pattern.
  • this dot pattern is obtained by patterning XY coordinate values and / or code values with a predetermined algorithm.
  • a scanner which is a dot pattern reading unit is connected to the computer main body. The user touches the grid sheet with the scanner in accordance with the instructions on the fluoroscopic screen. The scanner reads the dot pattern and transmits it to the personal computer via the USB cable.
  • the central processing unit (CPU) of the personal computer analyzes the dot pattern to obtain the XY coordinate value on the grid sheet, and further, xy on the display. Conversion to coordinate values enables input in touch panel format from the coordinate value information.
  • a personal computer is used as the information processing device.
  • the information processing device is a personal computer, a PDA, a television receiver, a front and rear projector, a game device, a karaoke device, a portable device.
  • a telephone terminal device, a POS terminal device, an ATM, a KIOSK terminal, a car navigation system, a pachinko machine, a watch, a smartphone, or the like may be used.
  • a grid sheet is arranged on a display screen or a display screen as a touch panel type input device.
  • FIG. 8 is a cross-sectional view showing the internal structure of the grid sheet described in FIG.
  • (A) is a longitudinal sectional view showing a state where the grid sheet is touched by the scanner.
  • the grid sheet has a structure in which an infrared reflection layer, a dot pattern layer, and a protective transparent sheet (protective layer) are laminated from the display device side.
  • the infrared reflective layer has a configuration in which an infrared reflective material is vapor-deposited on a transparent sheet for vapor deposition made of a material that transmits visible light, such as vinyl, envipet, and polypropylene.
  • the infrared reflection layer reflects the infrared ray irradiated from the infrared irradiation means of the scanner and transmitted through the protective transparent sheet to the scanner and transmits visible light. At the same time, infrared rays from the display device are blocked.
  • the infrared light irradiated from the infrared irradiation means of the scanner can be used as the irradiation light, and only a bright and clear dot pattern can be photographed, and the dot code can be analyzed accurately.
  • the dot pattern layer is printed with a dot pattern in which dots made of an infrared absorbing material such as carbon ink are arranged according to a predetermined rule as will be described later.
  • Protective transparent sheet is made of a material that transmits visible light and infrared rays, such as vinyl, envipette, and polypropylene.
  • a protective transparent sheet it is possible to prevent dot wear and dirt and to use the sheet for a long period of time.
  • the scanner incorporates an IR-LED that is an infrared irradiation means, an IR filter that cuts a predetermined wavelength component of the reflected light, a C-MOS sensor that is an image sensor, and a lens.
  • the scanner captures reflected light of the irradiation light irradiated on the grid sheet.
  • the dot pattern is printed with ink having the characteristic of absorbing infrared rays, only the dot portion is captured black in the captured image by the C-MOS sensor.
  • the reflection layer that is specularly reflected by the infrared reflection layer is photographed in a state where the reflected light is not incident on the lens and the center is black as shown in FIG. End up. Therefore, it is impossible to photograph the dot pattern without omission. Therefore, it is necessary to diffusely reflect infrared light and to enter the lens. That is, as shown in FIG. 9A, an infrared diffusion layer is provided between the dot pattern printing layer and the infrared reflection layer. This infrared diffusion layer is made of a transparent or translucent material. As a result, as shown in FIG. 5B, the infrared light irradiated from the IR-LED is specularly reflected by the infrared reflecting layer and diffused by the infrared diffusing layer, and the reflected light of all the imaging regions is incident on the lens.
  • the grid sheet is provided with an infrared diffusing layer.
  • the present invention is not limited to this, and a filter made of an infrared diffusing material may be provided at the photographing port of the scanner.
  • FIG. 10 is a cross-sectional view showing various structures of the grid sheet.
  • FIG. 1 is a grid sheet including an infrared reflection layer having a characteristic of reflecting infrared light and transmitting visible light, and a dot pattern layer.
  • the infrared rays irradiated from the infrared irradiation means are first absorbed by the portion of the dot pattern layer where the dots are formed (hereinafter referred to as the dot portion) and transmitted through the other regions.
  • the transmitted infrared ray is specularly reflected by the infrared reflection layer and passes through the dot pattern layer except for the dot portion.
  • (B) of the figure is a grid sheet in which an infrared diffusion layer that transmits visible light and diffuses the infrared light is provided between the infrared reflection layer and the dot pattern layer.
  • the infrared rays irradiated from the infrared irradiation means are first absorbed by the dot portions of the dot pattern layer and transmitted in other regions.
  • the transmitted infrared rays are diffused by the infrared diffusion layer, then specularly reflected by the infrared reflection layer, diffused again by the infrared diffusion layer, and transmitted through the dot pattern layer except for the dot portion.
  • (C) in the figure is a grid sheet in which an infrared diffusion layer that transmits visible light and diffuses infrared light is provided on one side of the dot pattern layer, that is, on the opposite side of the infrared reflection layer.
  • the infrared rays irradiated from the infrared irradiation means are first diffused by the infrared diffusion layer, then absorbed by the dot portion of the dot pattern layer, and transmitted in the other regions.
  • the transmitted infrared rays are specularly reflected by the infrared reflection layer, and are again transmitted through the dot pattern layer except for the dot portion and diffused by the infrared diffusion layer.
  • FIG. 4D is a grid sheet composed of an infrared diffuse reflection layer having a characteristic of diffusing and reflecting infrared light from one side and transmitting visible light, and a dot pattern layer.
  • the infrared rays irradiated from the infrared irradiation means are first absorbed by the dot portions of the dot pattern layer and transmitted in other regions.
  • the transmitted infrared light is diffusely reflected by the infrared diffuse reflection layer, and passes through the dot pattern layer except for the dot portion.
  • (E) in the figure is a grid sheet in which a protective layer having a property of transmitting infrared light and visible light is provided on the outer surface of the grid sheet shown in the figure (a) on the dot pattern layer side.
  • the protective layer may be provided not only on the figure (a) but also on the outer surface of the grid sheet shown in (b) to (d) on the dot pattern layer side.
  • (F) in the figure is a grid sheet having a characteristic of reflecting infrared rays from the side opposite to the dot pattern layer in the grid sheet shown in (a).
  • the grid sheets shown in (b) to (e) in addition to (a) in the figure may have a characteristic of reflecting infrared rays from the side opposite to the dot pattern layer (the other side). .
  • (G) in the figure is a grid sheet in which an adhesive layer is provided on the anti-dot pattern layer side of the infrared reflecting layer in the grid sheet shown in (a).
  • the adhesive layer is made of a removable material.
  • the pressure-sensitive adhesive layer is necessary only when the grid sheet is used by being attached to a display device or a medium, and is not necessary when it is placed on a printed material or sandwiched as described later. By providing such an adhesive layer, the grid sheet can be easily attached to a display or the like.
  • the pressure-sensitive adhesive layer may be provided not only on the same figure (a) but also on the anti-dot pattern layer side of the infrared reflecting layer of the grid sheets shown in (b) to (e).
  • FIG. 11 shows another embodiment of the grid sheet.
  • a dot pattern layer is provided on both sides of the sheet.
  • FIG. 4B is a grid sheet in which an infrared diffusion layer that transmits visible light and diffuses infrared light from each direction is provided between the infrared reflective layer and the dot pattern layer on each surface. It is.
  • (C) is a grid sheet in which an infrared diffusion layer is provided on the outer surface of the infrared reflection layer and the dot pattern layer on each side.
  • FIG. 4D is a grid sheet composed of an infrared diffuse reflection layer and dot pattern layers provided on both sides of the infrared diffuse reflection layer.
  • the figure (e) is an information input auxiliary sheet in which a protective layer having a property of transmitting infrared light and visible light from each direction is provided on both outer surfaces of the grid sheet shown in the figure (a). is there.
  • protective layer may be provided on both outer surfaces of the grid sheets shown in (b) to (d) as well as in FIG.
  • the figure (f) is a grid sheet in which an adhesive layer is provided on each anti-dot pattern layer side of the infrared reflection layer, that is, both outer surfaces of the grid sheet, in the grid sheet shown in the figure (a). is there.
  • the adhesive layer may be provided on both outer surfaces of the grid sheet shown in (b) to (e) as well as in FIG.
  • each layer and the description of reflection in the grid sheets in FIGS. 9A to 9F are the same as those described above, and thus the description thereof is omitted here.
  • different coordinate values and / or code values may be patterned in the dot pattern layer on one side and the other side of the infrared reflecting layer or infrared diffuse reflecting layer.
  • FIGS. 12 to 17 are diagrams for explaining another embodiment of the present invention.
  • a grid sheet is used on a printed material.
  • FIG. 12 is a diagram showing a case where a grid sheet on which a dot pattern is printed only on one side is used.
  • FIG. 13A is a front view showing the surface of the grid sheet.
  • a dot pattern that means XY coordinates is printed on almost the entire area, and at the bottom, icons with numbers from 0 to 9 and characters “page input”, “decision”, and “stop” are written, A dot pattern that means a code value is overprinted.
  • This icon is an icon for inputting an index for specifying a printed matter using a scanner.
  • (B) is a longitudinal sectional view of the grid sheet.
  • This grid sheet is configured from the back side as a layer of a protective transparent sheet, a transparent infrared diffuse reflection layer, a dot pattern + graphic printing layer, and a protective transparent sheet.
  • the graphic print layer is a visible print layer.
  • FIG. 14 is a diagram showing a case where a grid sheet having a dot pattern printed on both sides is used.
  • the grid sheet can be used while being sandwiched. That is, when a grid sheet is used on the right page, the grid sheet is put on the right page as shown in FIG. Then, the side on which the icon is printed on the right side is up. Next, when the grid sheet is used for the left page, the grid sheet is turned to the left side with the grid sheet being sandwiched as shown in FIG. Then, the side on which the icon is printed on the left side is up. The icon is printed on both sides with ink.
  • FIG. 15A is a front view showing the front surface of the grid sheet
  • FIG. 15B is a front view showing the back surface.
  • a dot pattern that means XY coordinates is printed on almost the entire area, and numbers from 0 to 9 and characters “page input”, “decision”, and “stop” are written on the right side.
  • An icon and a dot pattern representing a code value are printed in a superimposed manner. This icon is an icon for inputting an index for specifying a printed matter using a scanner.
  • a dot pattern that means XY coordinates is printed on almost the entire area, and the same icon and dot pattern as the front surface are superimposed and printed on the left side.
  • (C) is a cross-sectional view of the grid sheet.
  • This grid sheet is constituted by a protective transparent sheet, a dot pattern + graphic print layer, an infrared diffuse reflection layer, a dot pattern + graphic print layer, and a protective transparent sheet from the back side.
  • FIG. 16 is a diagram showing a system for displaying information on the display by the user touching the grid sheet.
  • the user puts a grid sheet on the printed material and touches the grid sheet with a scanner.
  • a central processing unit (CPU) in the scanner analyzes the dot pattern, converts it into coordinate values and / or code values, and transmits them to a personal computer via a USB cable. It should be noted that the central processing unit in the personal computer may be analyzed and converted into coordinate values and / or code values.
  • the coordinate value means the value of the XY coordinate in the dot coordinate system.
  • an index (page, title, etc.) for specifying the printed paper surface is input by touching the icon printed on the grid sheet, or the index code value input with the keyboard is, for example, the number of pages on the printed paper surface. I mean. Therefore, it is assumed that n pages are designated. If you set in advance a mask that specifies print information such as text, illustrations, and photos with XY coordinate values and multimedia information that corresponds to the mask number, the XY coordinates of the dot pattern of the grid sheet read by the scanner Based on the value, the central processing unit in the personal computer uses the dot coordinate-mask No. stored in the hard disk. Refer to the correspondence table ((b) in the figure).
  • the table is provided for each page.
  • the dot coordinates of n pages-mask No. Refer to the correspondence table.
  • the touch position mask No. Is detected For example, when the crossed portion shown in FIG. Is 1.
  • the page No. shown in FIG. -Mask No. References the multimedia information file correspondence table.
  • page No. And mask No. A multimedia information file is registered correspondingly.
  • a web address is registered in the n-page mask 1
  • a local drive and an execution file are registered in the n-page mask 2.
  • page No. And mask no. Search and output multimedia information files corresponding to. That is, page No. Is n and mask no. Since 1 is 1, the Web page at the registered Web address is displayed.
  • FIG. 17 is a diagram showing another form of a system for displaying information on the display by the user touching the sheet.
  • the central processing unit (CPU) in the scanner analyzes the dot pattern, converts it into coordinate values or / and code values, and dot XY coordinates, and transmits them to a personal computer via a USB cable. It should be noted that the central processing unit in the personal computer may be analyzed and converted into coordinate values and / or code values.
  • the central processing unit (CPU) of the personal computer converts the printed paper surface ID and the dot XY coordinates obtained as described above into DTPxy coordinates (xy coordinates on the DTP surface, which is the original data of the printed material).
  • the central processing unit (CPU) of the personal computer accesses the document management server via the network (NW).
  • the document management server stores a DTP file corresponding to each printed sheet.
  • the DTP coordinates (xt, yt) are the coordinates of the center position of the text area.
  • the dictionary server is accessed and a data file indicating the specified text, illustration, photo, etc. is searched.
  • the dictionary server stores related information such as explanations, URLs, and multimedia information corresponding to data files indicating text, illustrations, photographs, and the like.
  • the central processing unit (CPU) of the document management server searches for “Tokyo” from the dictionary server based on the program, and automatically transmits the related information to the personal computer via the network (NW). As a result, the explanation regarding “Tokyo” is output and displayed on the display of the personal computer.
  • FIG. 18 is an explanatory diagram showing an embodiment in which a grid sheet is attached to a display, content specifying information touched by a scanner is recognized, and related information is automatically searched and output.
  • a grid sheet is pasted on the display screen of the personal computer.
  • the user touches the dot pattern of the portion displayed as “Tokyo” with the scanner.
  • the central processing unit (CPU) in the scanner analyzes the dot pattern, converts it into XY coordinates, and transmits it to the personal computer via the USB cable.
  • the XY coordinates are XY coordinate values of the touch position in the dot coordinate system.
  • you may analyze and convert into a coordinate value with the central processing unit in a personal computer.
  • the central processing unit (CPU) of the personal computer accesses the Web server of each content displayed on the display via the network (NW).
  • Each content file is stored in the Web server.
  • the central processing unit retrieves the displayed content file (HTML, XML, flash, etc.).
  • the XY coordinate value of the touch position in the dot coordinate system is converted into the xy coordinate value in the format information of the content such as HTML, XML, and flash.
  • the displayed content file is on a personal computer, there is no need to access the Web server and retrieve the content file.
  • the coordinate value (xt, yt) of the content specifying information such as text, illustration, photo, etc. displayed on the display closest to the touch position coordinate value (x, y) is searched.
  • the coordinate values (xt, yt) of the content specifying information are the coordinates of the center position of the content specifying information.
  • various multimedia information such as audio information and image information corresponding to the content identification information such as a data file indicating the recognized text, illustration, photo, etc. is stored in an electronic dictionary or a multimedia corresponding beforehand.
  • the dictionary server stores content-related information such as explanations, URLs, and multimedia information corresponding to data files indicating text, illustrations, photographs, and the like.
  • the central processing unit (CPU) retrieves “Tokyo” from the dictionary server, and automatically outputs the content related information to the personal computer via the network (NW). As a result, the explanation about “Tokyo” is displayed on the display of the personal computer.
  • FIG. 19 is a longitudinal sectional view of a grid sheet used in such an example.
  • the grid sheet is composed of an adhesive layer, an infrared diffuse reflection layer, a dot pattern + graphic printing layer, and a protective transparent sheet from the back side.
  • FIG. 20 is a diagram illustrating a case where the grid sheet is used in the front projector system.
  • a grid sheet is pasted on the screen.
  • a predetermined image is projected and displayed on the screen by the projector.
  • the projector is connected to a personal computer or a video distribution system (not shown).
  • a personal computer or a video distribution system not shown.
  • the dot pattern at that position is read into the scanner and the personal computer uses the scanner. Converted to coordinate values.
  • a hard disk device or video distribution server of a personal computer refers to an index table in which coordinate values are associated with instructions, addresses, etc., and outputs multimedia information and operation instructions defined in the corresponding addresses.
  • FIG. 21 shows an example using a rear projector panel, which is a system for projecting an image from the rear projector.
  • a grid sheet is affixed to the surface of the rear projector panel.
  • the structure of the grid sheet is the same as that shown in FIG.
  • a predetermined image is projected and displayed on the rear projector panel by the rear projector.
  • the rear projector is connected to a personal computer or video distribution system (not shown), and when a desired position on the rear projector panel is touched using the scanner connected to the personal computer, the dot pattern at that position is read into the scanner.
  • a personal computer or video distribution system (not shown)
  • a hard disk device or video distribution server of a personal computer refers to an index table in which coordinate values are associated with instructions, addresses, etc., and outputs multimedia information and operation instructions defined in the corresponding addresses.
  • FIG. 22A shows an example in which a grid sheet is used for a mobile terminal.
  • a grid sheet is attached to the display of the portable terminal. Then, the grid sheet is touched with a scanner connected by wireless or cable. Then, the dot pattern at that position is read into the scanner and converted into coordinate values in the scanner or portable terminal.
  • an index table in which coordinate values are associated with commands or addresses is stored.
  • the central processing unit in the portable terminal refers to the index table, and outputs multimedia information and an operation instruction specified by the corresponding address.
  • FIG. 22B shows an example in which a grid sheet is used for the car navigation device.
  • the scanner when used in a car navigation device, it is preferable that communication between the scanner and the car navigation device is performed by wireless communication using Bluetooth or the like. Further, the scanner may be connected to a mobile terminal and transmit data corresponding to the read dot pattern to the server via the mobile terminal network.
  • car navigation shown in the figure may be used in combination with a map such as a tourist guide on which a dot pattern is printed.
  • FIG. 23 to FIG. 24 are diagrams for explaining an embodiment in which the grid sheet according to the present invention is used in the thin client system.
  • Thin client is a general term for a system in which a client terminal used by a user has only a minimum function and manages resources such as application software and files on the server side.
  • a plurality of displays are connected to one server (information processing apparatus) via a network.
  • this display various types of displays other than a general television monitor and a personal computer display which do not have a built-in CPU can be used.
  • An information input auxiliary sheet is affixed on the display screen of each display.
  • the server and each display are connected by a wired system or a wireless system such as RDP, USB, BNC (Bayonet Neil Concelman).
  • the server and the scanner are connected by a wired method or a wireless method.
  • the dot pattern at that position is read into the scanner and converted into XY coordinate values within the scanner.
  • an index table in which XY coordinate values are associated with commands or addresses is stored.
  • the central processing unit of the server refers to the index table, and outputs multimedia information and an operation instruction specified by the corresponding address.
  • a code value may be registered together with the XY coordinate value.
  • the code value is for identifying each display.
  • the scanner when it reads the dot pattern, it transmits the code value and the XY coordinate value to the server (information processing apparatus).
  • the server recognizes which display from the code value. Then, various multimedia information and / or operation instructions corresponding to each display are recognized, and output information is displayed on the display.
  • an XY coordinate value for uniquely identifying the display may be printed as a dot pattern.
  • XY coordinates obtained by dividing a large area in advance are printed on the sheet, and this sheet is used separately for each display. That is, the XY coordinate values are different for each display.
  • the server specifies which display is operated from the XY coordinate value. Then, various multimedia information output and / or operation instructions corresponding to each display are recognized, and the output information is displayed on the display.
  • a sticker on which a code value for identifying each display is printed as a dot pattern may be attached to the display.
  • the user touches the grid sheet after touching the seal.
  • the server specifies which display is operated by the code value registered in the dot pattern of the seal.
  • the touch position of the user is recognized from the XY coordinate values transmitted from the dot pattern of the grid sheet, various multimedia information and / or operation instructions corresponding to each display are recognized, and output information is displayed on the display.
  • the seal may be made of the same material as that used for the adhesive sticker on paper on which a normal dot pattern is printed or the grid sheet of the present invention.
  • FIG. 25 to FIG. 28 are diagrams for explaining calibration performed when the grid sheet is used.
  • the positional relationship between the display or the printed material and the grid sheet needs to be matched in order to output information that accurately reflects the instruction displayed at the touch position. . Therefore, calibration is performed to properly associate the coordinate system of the display or printed material with the coordinate system of the grid sheet.
  • FIG. 25 to FIG. 26 are diagrams illustrating a case where calibration is performed on the display screen of the display.
  • FIG. 25 is a diagram illustrating a case where calibration marks are provided on the grid sheet.
  • calibration marks are provided in the vicinity of the four corners on one side of the grid sheet.
  • the calibration mark is not necessarily provided near the four corners, and may be provided at two or more corners or may be provided at the center. Moreover, you may provide in two or more corner parts and the center.
  • the user moves the cursor to the calibration mark and left-clicks the mouse.
  • the central processing unit of the personal computer recognizes the clicked position. Then, the display coordinate system and the grid sheet coordinate system are appropriately associated with each other. As a result, calibration is performed.
  • the calibration mark is formed on a detachable transparent seal. After the calibration is completed, the user peels off the seal from the grid sheet as shown in FIG.
  • the calibration mark may be directly printed on the grid sheet in addition to the case where it is formed on the transparent sticker. Moreover, it is provided in a removable state on the grid sheet, and may be removed from the grid sheet after calibration is completed.
  • FIG. 26 is a diagram illustrating a case where a calibration mark is displayed on the display screen of the display.
  • calibration marks are displayed near the four corners of the display. Note that the calibration mark is not necessarily displayed near the four corners, and may be displayed at two or more corners, or may be displayed at the center. Moreover, you may display on both the corner part and center of 2 or more.
  • the user touches the grid sheet at the position where the calibration mark is displayed as shown in FIG.
  • the scanner reads the dot pattern on the grid sheet and transmits it to the personal computer.
  • the central processing unit of the personal computer recognizes the dot XY coordinates (X 1 , Y 1 ) of the touch position from the transmitted dot pattern, and performs calibration to properly associate the display coordinate system and the grid sheet coordinate system. Do.
  • FIGS. 27 to 28 are diagrams illustrating a case where calibration is performed on a printed material.
  • FIG. 27 is a diagram illustrating a case where calibration marks are provided on both the printed material and the grid sheet for the printed material.
  • calibration marks are printed in the vicinity of the four corners of the grid sheet and the printed material. Note that the calibration mark is not necessarily printed in the vicinity of the four corners, and may be printed in two or more corner portions.
  • the user puts the grid sheet on the printed matter by matching both calibration marks. As a result, calibration for properly associating the coordinate system of the printed material with the coordinate system of the grid sheet is performed.
  • FIG. 28 is a diagram illustrating a case where a calibration mark is printed only on a printed material.
  • calibration marks are printed in the vicinity of the four corners of the printed material. Note that the calibration mark is not necessarily printed in the vicinity of the four corners, and may be printed in two or more corner portions.
  • the user puts a grid sheet on the printed material. Then, the scanner is matched with the mark. The scanner reads the dot pattern on the grid sheet and transmits it to the personal computer.
  • the central processing unit of the personal computer recognizes the XY coordinates (X 1 , Y 1 ) of the touch position from the transmitted dot pattern, and performs calibration for appropriately associating the coordinate system of the printed material with the coordinate system of the grid sheet. .
  • a grid sheet when used for a display screen or the like, it may be used by other methods such as using a grid sheet attached from the upper part of the display screen, in addition to being attached with an adhesive.
  • FIG. 29 shows an outline of the present embodiment.
  • FIG. 29A is a diagram showing a state of operation in the present embodiment
  • FIG. 29B is a diagram showing a state of selecting specific information.
  • the user inputs information to be input to the information input device 105 on the printed material 101a (medium) on which the dot pattern 103 (details will be described later) is printed together with the content. Touch using the dot pattern reading means.
  • the word “Tokyo” 102 a printed on the printed material 101 a is used as the scanner 104. Is touching.
  • an XY coordinate value on the printed material 101a and a document ID (identification information) for specifying the printed material 101a are coded in advance, and when the user touches the printed material 101a with the scanner 104, The coordinate value (Xt, Yt) in the XY coordinate system on the printed material 101a at the touch position and the document ID of the printed material 101a are read into the information input device 105.
  • the information input device 105 uses the read XY coordinate values, the document ID, and DTP (Desk Top Publishing) data 101b (two-dimensional format information), which is electronic data based on the prepared printed matter 101a.
  • the touch position recognizes that the word “Tokyo” 102 a is present (details will be described later), and the word “Tokyo” 102 a is input to the information input device 105.
  • the phrase “Tokyo” 102a input to the information input device 105 is entered in the location column of the input form displayed on the screen.
  • the DTP data 101b has its own xy coordinate system. Selection position information used to select these is set in text, charts, illustrations, photographs, etc. (specific information) included in the DTP data 101b.
  • a corresponding mask area 102b and / or a center coordinate value (xi, yi) is set.
  • FIG. 29B shows the coordinate value (xt, yt) of the touch position on the DTP data 101b converted from the coordinate value (Xt, Yt) of the touch position on the printed material 101a touched by the scanner 104, and a phrase.
  • region 102b corresponding to "Tokyo" 102a and center coordinate value (xi, yi) is shown.
  • the phrase “Tokyo” 102a among many specific information included in the DTP data 101b is input to the information input device 105. Selected.
  • the method for selecting the specific information is that the coordinate value (xt, yt) of the touch position is greater than the distance from the coordinate value (xt, yt) of the touch position to the center coordinate value (xi, yi) of the other specific information. ) To the center coordinate value (xi, yi) of the phrase “Tokyo” 102a is shorter, so the phrase “Tokyo” 102a may be selected as an input. Details of the method for selecting the specific information will be described later. ⁇ About dot code format>
  • a dot code is information recorded in the dot pattern 103.
  • the dot pattern is a dot pattern composed of unit areas of 2 rows ⁇ 4 columns of information dots in which a set of constant information is defined, and is partitioned into C 0 to C 31 within this unit. ing.
  • the dot code format of each area is shown in (a) to (c).
  • FIG. 30A is an example of a format in which the dot code includes an XY coordinate value, a code value, and parity
  • FIG. 30B is an example in which the format is changed depending on where the dot pattern 103 is provided.
  • 30 (c) is an example of a format in which the dot code includes an XY coordinate value and a parity.
  • the X coordinate value at the position where the dot pattern 103 is provided is expressed using 8 bits of bits C0 to C7, and the Y coordinate value is expressed using bits C8 to C15.
  • the code value is then represented using 14 bits, bits C16 to C29.
  • This code value can be used to represent arbitrary information in accordance with the purpose of use of the dot pattern 103. In the present embodiment, it can be used to express a document ID.
  • two bits C30 and C31 are used as the parity of the dot code. Note that the parity calculation method may be a generally known method, and a description thereof will be omitted.
  • the format is changed depending on where the dot pattern 103 is provided.
  • a place where the dot pattern 103 is provided is divided into an XY coordinate area and a code value area.
  • the XY coordinate area In the XY coordinate area, the XY coordinate area format is used, and in the code value area, the code value area format is used.
  • the X coordinate is expressed using 15 bits of bits C0 to C14
  • the Y coordinate is expressed using 15 bits of C15 to C29
  • the code value is expressed using 30 bits from C0 to C29.
  • the bit string representation is made so that the bit strings representing the XY coordinate value and the code value do not overlap. It is good to decide rules.
  • the same format as the XY coordinate area format in FIG. 30 (b) is used. Since there is no code value column in the format, the document ID cannot be directly recorded in the dot pattern 103, and the document ID is indirectly distinguished by dividing the XY coordinate value assignment area.
  • the document ID indicating which DTP data the printed material 101a corresponds to is included in the dot code together with the XY coordinate value in at least a part of the dot pattern 103 printed on the printed material 101a.
  • the printed matter 101a may be divided into a dot code area that includes only XY coordinate values and a dot code area that includes only the document ID.
  • the document ID may be manually input to the information input device 105 by the user using an input unit such as a keyboard. ⁇ About the structure of printed matter>
  • FIG. 31 is a cross-sectional view of the printed material 101a.
  • FIG. 31 (a) is a diagram showing a state in which infrared rays hit and diffusely reflect a portion of the print medium 101c where nothing is printed
  • FIG. 31C is a diagram showing a state in which infrared rays hit the content printing portion of the printed matter 101a and is diffusely reflected
  • FIG. 31 (a) is a diagram showing a state in which infrared rays hit and diffusely reflect a portion of the print medium 101c where nothing is printed
  • FIG. 31C is a diagram showing a state in which infrared rays hit the content printing portion of the printed matter 101a and is diffusely reflected
  • the ink is shown so as to swell on the surface of the print medium, but this arrangement is the case where the print medium 101c has a characteristic of not absorbing ink,
  • the print medium 101c has a characteristic of absorbing ink, but the viscosity of the ink is extremely high.
  • the print medium 101c is printed so as to bleed on the print medium.
  • the surface of the print medium 101c on which the content and the dot pattern 103 are printed has a characteristic of diffusing and reflecting infrared rays.
  • the reason is that when the surface of the print medium 101c has a characteristic of specularly reflecting infrared rays, the reflected light from a part of the surface read by the scanner 104 does not enter the lens (described later) properly, and one of the read images This is because the image is dark and the image analysis becomes difficult.
  • the infrared light hits the part printed using the infrared diffuse reflection ink 110, and the infrared light hits the part printed using the infrared transmission ink 111. After the transmission, the state of diffuse reflection on the surface of the print medium 101c is shown.
  • the infrared diffuse reflection ink 110 is formed in a donut shape around the previously printed dot pattern 103.
  • infrared transmissive ink 111 is disposed. Ordinary viscous inks will blend into each other.
  • the scanner 104 irradiates the reading surface on which the dot pattern 103 of the printed material 101a is printed with infrared rays and shoots the reflected light as an image (details will be described later), the dot pattern 103 printed with the infrared absorbing ink is In the image, the portion that is black and diffusely reflects infrared light is white.
  • the infrared diffuse reflection ink 110 and the infrared transmission ink 111 have a characteristic of reflecting or absorbing a specific wavelength of visible light, and therefore printed when a person sees the printed material 101a printed using these inks. Content can be read in the same way as normal printing.
  • the scanner 104 includes an IR-LED that is a means for irradiating infrared rays, a lens that collects reflected light reflected from the surface of the printed material 101a by the infrared rays irradiated from the IR-LED, and reflected light that has passed through the lens.
  • An IR filter that cuts a predetermined wavelength component and a C-MOS sensor that is an image sensor are incorporated.
  • the scanner 104 irradiates the printed matter 101 a with infrared rays, and images reflected light that is diffusely reflected from the surface of the printing medium 101 c and a portion printed by the infrared diffuse reflection ink 110.
  • the dot pattern 103 is printed with ink having a characteristic of absorbing infrared rays, only the dot portion of the dot pattern 103 is captured black in the image captured by the C-MOS sensor.
  • the scanner 104 includes a central processing unit (not shown).
  • the central processing unit analyzes the dot pattern 103 input from the C-MOS sensor with analysis software, Convert to dot code and output.
  • the scanner 104 may include a hardware circuit having similar functions instead of the central processing unit and the analysis software.
  • the process of analyzing the dot pattern 103 and converting it into a dot code may be performed on a device connected to the scanner 104, for example, a personal computer, instead of being performed by the central processing unit on the scanner 104.
  • the scanner 104 and a device to which the scanner 104 is connected may be connected by a wired method such as a USB cable, or may be connected by a wireless method.
  • the information input device 105 includes a scanner 104, a document management unit 120 (two-dimensional format information storage selection unit), a coordinate conversion unit 130 (coordinate conversion unit), and a specific information selection unit 140 (specific information selection unit). ing.
  • the scanner 104 reads the dot pattern 103 at the position touched by the user using the scanner 104 on the printed material 101a, converts the dot pattern 103 into a dot code, and outputs the document ID portion in the dot code to the document management unit 120. Further, the scanner 104 outputs the XY coordinate value (xt, yt) portion in the dot code to the coordinate conversion unit 130.
  • the dot code includes an XY coordinate value and a code value will be described.
  • the document management unit 120 uses the document ID input from the scanner 104 to search the document DTP data stored in the document management unit 120 in advance.
  • the DTP data 101b corresponding to the input document ID is output to the coordinate conversion unit 130 and the specific information selection unit 140.
  • the coordinate conversion unit 130 uses the XY coordinate values (Xt, Yt) input from the scanner 104 and the DTP data 101b input from the document management unit 120 to convert the touch position coordinates into a coordinate system on the printed material 101a.
  • XY coordinate value (Xt, Yt) at is converted into an xy coordinate value (xt, yt) in the coordinate system on the DTP data 101b, and the converted xy coordinate value (xt, yt) is output to the specific information selection unit 140. .
  • the specific information selection unit 140 uses the DTP data 101b input from the document management unit 120 and the xy coordinate values (xt, yt) input from the coordinate conversion unit 130 to display texts, charts, and diagrams included in the DTP data 101b.
  • the specific information such as illustrations and photographs
  • the specific information considered to be instructed by the touch operation by the user is selected and output.
  • the details of the selection method of specific information will be described later, for example, there is a method of selecting specific information at a position closest to the xy coordinate value (xt, yt) of the touch position.
  • the specific information output from the specific information selection unit 140 is used as an input in another device or the like.
  • the information input device 105 include personal computers, PDAs, television receivers, front projectors, rear projectors, game devices, karaoke devices, mobile phone terminal devices, POS terminal devices, ATMs, KIOSK terminals, car navigation systems, and pachinko machines. , Watches, smartphones and the like. ⁇ About DTP data>
  • DTP data is cited as an example of two-dimensional format information. More specifically, the DTP data is electronic data created by DTP software such as a word processor, authoring software that creates files in a format such as CAD software, HTML, XML, flash, PDF, etc. For example, in addition to the text body, the type and size of the font (vector font) used when displaying or printing the text, layout information, and the like are included.
  • the DTP data when the DTP data includes charts, illustrations, photographs, etc., it also includes the name of the file in which each image data is stored and the location information of the area occupied when displaying or printing. It is.
  • DTP data is managed so that it can be uniquely identified by the document ID.
  • the document ID is configured so that each page can be distinguished.
  • the DTP data may be map data. ⁇ About position information for selection>
  • the timing for setting the selection position information for each specific information may be set in advance when the DTP data is stored in the document management unit 120, or the specific information selection unit 140 may receive the DTP data from the document management unit 120. It may be set by the specific information selection unit 140 after receiving the data 101b.
  • the specific information selection unit 140 may analyze the entire DTP data 101b received from the document management unit 120 at a time, and set selection position information for all the specific information included in the DTP data 101b. In the DTP data 101b, only the specific information in the vicinity of the coordinate value (xt, yt) of the touch position may be analyzed to set the position information for selection.
  • the analysis method of the specific information when the specific information is text, a generally known ordinary syntax analysis method may be used.
  • CAD data points, line segments, planes, solid objects, etc.
  • the specific information is represented as a set of hierarchized objects
  • the hierarchical structure and the software that can handle the objects Analysis can be performed.
  • the specific information is an illustration or a photograph represented by a bitmap
  • the bitmap data is acquired from the file name of the specific information included in the DTP data, and a generally known ordinary image is obtained.
  • the bitmap may be analyzed using an analysis method.
  • FIG. 34 shows an example in which the center coordinate value or centroid coordinate value of an object is used as position information for selection of specific information.
  • 34A shows an example in which the specific information is text
  • FIG. 34B shows an example in which the specific information is a chart (pie chart)
  • FIG. 34C shows the specific information.
  • This is an example of illustrations / photos.
  • a plus mark is used to indicate the position of the center coordinate value or centroid coordinate value of the object, but this mark is shown for explanation, and is printed or displayed. Is not to be done.
  • the phrases “Hana no Miyako”, “Osaka”, and “Tokyo Tower” are shown as specific information.
  • the center of the display area is set as selection position information.
  • the specific information selection unit 140 When the user touches the vicinity of the character “flower”, the specific information selection unit 140 cannot determine whether the phrase that the user wants to specify is the phrase “flower capital” or the phrase “flower”. There is a case. In that case, the specific information selection unit 140 may present both the phrase “flower capital” and the phrase “flower” to the user as selection candidates, and the user may select a desired phrase from the selection candidates.
  • the line between the character “Dai” and the character “Osaka” is separated by a line feed. Also in this case, the user can appropriately select the word “Osaka” by setting the character center coordinate value for each character as the position information for selection.
  • the pie chart and the pie constituting the pie chart are shown as the specific information.
  • Center position coordinates of a circle are set as position information for selection of the entire pie chart, and centroid coordinates of 27%, 18%, and 55% of each pie (objects A, B, and C, respectively)
  • a value is set as position information for selection of each object.
  • contours of illustrations of Kawama (object D) and bear (object E) are shown as specific information.
  • the centroid coordinate value of each object is set as selection position information. Note that the centroid coordinate value to be set does not need to be calculated accurately, and may be an approximate value that can appropriately select an object.
  • the user can give more specific information instructions by setting the position information for selection for each small object. .
  • FIG. 35 shows another example of position information for selection when text and phrases are specified information.
  • FIG. 35A is an example in which the starting point coordinates and height of a word are used as selection position information
  • FIG. 35B is an example in which the starting point coordinates, end point coordinates, and height of a word are used as selection position information
  • FIG. 35C shows an example in which the start point coordinates, end point coordinates, and height for each character are used as position information for selection when the characters constituting the phrase are separated.
  • the height hi of the phrase “Tokyo” and the coordinate value (xi, yi) of the midpoint of the left side (length hi) of the rectangular area occupied by the phrase “Tokyo” are The position information for selection is used.
  • Xi2, yi2 is used as position information for selection.
  • the position information for selection is set by line segments indicating the height and width of the characters. That is, the character “Large” has the height hi1, the start point coordinates (xi11, yi11), and the end point coordinates (xi12, yi12) as position information for selection, and the character “Osaka” has the height hi2. It has start point coordinates (xi21, yi21) and end point coordinates (xi22, yi22) as selection position information.
  • the position, height, and width of the area occupied by the text, the phrase, and the character can be known by using various selection position information shown in FIG. 35 for the text, the phrase, and the character. Mask areas for these texts, phrases, and characters in the coordinate system can be easily calculated.
  • the selection position information may be composed of a set of all coordinate values (xia, yia) included in the mask area and a set of coordinate values (xie, yie) of the contour part of the mask area.
  • the DTP data is map data
  • the area surrounded by the road, the feature area, the road center line, and / or the feature centroid are the position information for selection.
  • the specific information is classified into categories such as text, road, feature, and intersection.
  • selecting specific information one of the categories is selected to recognize the specific information. Naturally, it may be selected from candidates for specific information, or may be touched after a category is selected in advance.
  • FIG. 36 shows an example of an operation in which the user touches one point of the specific information using the scanner 104.
  • the coordinate value (Xt, Yt) of the touch position is recognized by the specific information selection unit 140 as a point coordinate.
  • FIG. 37 shows an example of an operation in which the user touches one point of the specific information and then traces on the specific information to be specified using the scanner 104.
  • the coordinate information (Xt, Yt) of the locus traced by touching is recognized by the specific information selection unit 140 as a line segment.
  • FIG. 38 shows an example of an operation of tracing and surrounding the specific information to be designated after the user touches one point of the specific information using the scanner 104.
  • the coordinate information (Xt, Yt) of the locus traced by touching is recognized by the specific information selection unit 140 as a closed curve.
  • the specific information selection unit 140 may form a complete closed curve by correcting the start point and the end point of the trace traced by touching.
  • the enclosing operation here includes a case where the enclosing is completely included so that the specific information to be specified is included therein, and an enclosing operation where the traced trace is overlapped on the specific information.
  • the traced trace is enclosed so as to overlap the specific information
  • at least the position represented by the selection position information may be enclosed. If at least the position represented by the position information for selection is not surrounded, the specific information corresponding to the position of the position information for selection closest to the plurality of coordinate values (xt, yt) constituting the trace is selected. Is done. ⁇ How to select specific information>
  • Case 1 When the trajectory of the touch operation is a point and the position information for selection is the center point, the specific information selection unit 140 has the center coordinate (xi, yi) closest to the point of the coordinate value (xt, yt) of the touch position. Alternatively, specific information having centroid coordinates (xi, yi) is selected.
  • Case 2 When the locus of the touch operation is a point and the position information for selection is a line segment
  • the user designates specific information by the touch operation, and the phrase “Tokyo” 102a is used as the position information for selection.
  • An example of a method for selecting specific information when a line segment is present will be described.
  • FIG. 39 (a) shows an example in which the perpendicular foot drawn from the point of the coordinate value (xt, yt) of the touch position does not intersect the line segment.
  • the distance d from the point of the coordinate value (xt, yt) to the line segment is the coordinate value (xt, yt) and the start point coordinate value (xi1, yi1) or end point coordinate value (xi2, yi2) of the line segment. The shorter of the distances.
  • FIG. 39 (b) shows an example in which a perpendicular foot drawn from the point of the coordinate value (xt, yt) of the touch position intersects the line segment.
  • the distance d from the point of the coordinate value (xt, yt) to the line segment is the length of the perpendicular drawn from the point of the coordinate value (xt, yt) to the line segment.
  • the specific information selection unit 140 obtains the distance d between the line segment representing each specific information and the point of the coordinate value (xt, yt) of the touch position by the above method, and selects the specific information with the shortest distance d. .
  • Case 3 When the locus of the touch operation is a point and the position information for selection is a mask area
  • the user designates specific information by the touch operation, and the phrase “Tokyo” 102a is used as the position information for selection.
  • An example of a method for selecting specific information when the mask area 102b is provided will be described.
  • the specific information selection unit 140 specifies the specific information having the matching coordinate value. Select. If there is no coincident coordinate value, the specific information having the coordinate value (xie, yie) of the mask region outline portion closest to the coordinate value (xt, yt) of the touch position may be selected.
  • the selection method using this mask area can be used not only for text but also for objects such as charts, illustrations, and photographs that are specific information.
  • FIG. 40A shows an example in which the coordinate value (xt, yt) of the touch position is within the mask area 102b.
  • the specific information selection unit 140 selects the phrase “Tokyo” 102a, which is specific information corresponding to the mask area 102b.
  • FIG. 40B shows a case where the coordinate value (xt, yt) of the touch position does not enter the mask area 102b of the phrase “Tokyo” 102a nor the mask area 102b ′ of the phrase “Osaka” 2a ′. It is an example.
  • the specific information selection unit 140 obtains distances d1 and d2 from the coordinate values (xt, yt) of the touch position to the mask area 102b and the mask area 102b ', and compares these distances. Since the distance d1 is shorter, the specific information selection unit 140 selects the phrase “Tokyo” 102a, which is specific information corresponding to the mask area 102b.
  • Case 4 When the trajectory of the touch operation is a line segment and the position information for selection is the center point, the specific information selection unit 140, like the case 2, the center coordinates (xi, yi) or the centroid coordinates (xi, yi) ) To the specific information with the shortest distance d to the line segment of the trajectory.
  • Case 5 When the touch operation trajectory is a line segment and the selection position information is a line segment
  • the specific information selection unit 140 for example, the start point coordinate value (xt1, yt1) of the line segment that is the trajectory and the end point coordinate value Select specific information that minimizes the area of the quadrangle formed by (xt2, yt2), the start point coordinate value (xi1, yi1) of the line segment that is the position information for selection, and the end point coordinate value (xi2, yi2) To do.
  • Case 6 When the locus of the touch operation is a line segment, and the position information for selection is a mask area
  • the specific information selection unit 140 is the most of the plurality of coordinate values (xt, yt) constituting the line segment that is the locus. Specific information having a mask area including many coordinate values (xt, yt) is selected.
  • Case 7 When the locus of the touch operation is a closed curve and the position information for selection is the center point
  • An example of a method for selecting specific information when the center position value (xi, yi) is used as the position information for use will be described.
  • the specific information selection unit 140 selects the specific information having the position information for selection surrounded by the trajectory input by the tracing operation.
  • Case 8 When the trajectory of the touch operation is a closed curve and the position information for selection is a line segment
  • the specific information selection unit 140 has a plurality of coordinate values ( xi, yi), the specific information having the largest number of coordinate values included in the closed curve as the locus is selected.
  • one dot code includes both an XY coordinate value and a code value
  • the specific information in the DTP data 101b corresponding to the phrase “Tokyo” 102a on the printed material 101a includes selection position information.
  • a case where the mask region 102b is set in advance will be described.
  • the scanner 104 reads the dot pattern 103 printed at a location on the printed matter 101a touched by the user using the scanner 104. .
  • the read dot pattern 103 is converted into a dot code by analysis software in the central processing unit of the scanner 104.
  • the scanner 104 sends the document ID portion of the converted dot code to the document management unit 120 and sends the XY coordinate value portion to the coordinate conversion unit 130 (step 10, hereinafter abbreviated as S10).
  • the document management unit 120 selects DTP data 101b corresponding to the document ID from the DTP data stored in advance.
  • the selected DTP data 101b is sent to the coordinate conversion unit 130 and the specific information selection unit 140 (S20).
  • the coordinate conversion unit 130 receives the coordinate value (Xt, Yt) in the XY coordinate system received from the scanner 104, and the coordinate value (xt, yt) in the xy coordinate system in the DTP data 101b received from the document management unit 120. (S30).
  • the specific information selection unit 140 selects appropriate specific information from the specific information in the DTP data 101b received from the document management unit 120 based on the coordinate values (xt, yt) received from the coordinate conversion unit 130. (S40).
  • the specific information selection unit 140 outputs the selected specific information so as to be input to another device or to be input for the next arbitrary process performed in the information input device 105.
  • the specific information selection unit 140 determines whether or not the locus of the coordinate value (xt, yt) of the touch position obtained by the user's touch operation is a point (S41a).
  • the specific information selection unit 140 determines whether or not the coordinate value (xt, yt) of the point matches any of the coordinate values (xia, yia) in the mask area 102b (S42a).
  • the specific information selection unit 140 sets the coordinate value (xt, yt) of the point.
  • the closest specific information having the coordinate value (xie, yie) of the contour portion is selected, and the process ends (S44a).
  • the specific information selection unit 140 determines whether at least a part of the line segment is included in the mask region 102b. (S42b).
  • the specific information selection unit 140 selects specific information including the largest number of coordinate values (xt, yt) in the mask region, and performs processing. The process ends (S43b).
  • the specific information selection unit 140 selects specific information having the coordinate value (xie, yie) of the contour portion closest to the line segment, and ends the process. (S44b).
  • the specific information selection unit 140 selects the locus of the coordinate value (xt, yt) of the touch position as a closed curve. Whether or not (S41c).
  • the specific information selection unit 140 determines whether at least a part of the area surrounded by the closed curve overlaps with the mask area 102b. (S42c).
  • the specific information selection unit 140 displays the specific information corresponding to the mask region 102b having the largest overlap with the region surrounded by the closed curve. The process is terminated (S43c).
  • the specific information selection unit 140 specifies specific information having the coordinate value (xie, yie) of the contour portion closest to the region surrounded by the closed curve. Is selected and the process is terminated (S44c).
  • One embodiment of the information input device according to the present invention that uses a medium on which a dot pattern is printed together with content and acquires DTP data from a server on the network will be described with reference to FIG. is there.
  • the information input device 105a includes a scanner 104a, a coordinate conversion unit 130a, a specific information selection unit 140a, and a communication unit 150 (communication means).
  • the information input system includes an information input device 105a, a document management server 113 (two-dimensional format information server), and a network connecting them.
  • the scanner 104a reads the dot pattern 103 at the touch position, sends the document ID portion of the dot code to the communication unit 150, and sends the XY coordinate value portion of the dot code to the coordinate conversion unit 130a.
  • the communication unit 150 transmits the document ID received from the scanner 104a to the document management server 113, and receives the DTP data 101b corresponding to the document ID received from the document management server 113 as the coordinate conversion unit 130a and the specific information selection unit 140a. Send to.
  • the specific information selection unit 140a selects specific information in the DTP data 101b using the xy coordinate values received from the coordinate conversion unit 130a and the DTP data 101b received from the communication unit 150, and outputs the selected specific information To do.
  • the document management server 113 includes a document management unit 120a (two-dimensional format information storage selection unit) and a communication unit 151 (communication unit).
  • the function of the document management unit 120a is the same as that of the document management unit 120 in the first embodiment.
  • the document management unit 120a is provided only on the document management server 113. However, as in the first embodiment, the document management unit 120 may also be provided in the information input device 105a. Good.
  • the document management unit 120 may cache the DTP data 101b acquired from the document management unit 120a.
  • steps S10a to S40a in the following description correspond to S10 to S40 in the flowchart shown in FIG.
  • a case where one dot code includes both an XY coordinate value and a code value will be described.
  • the scanner 104a reads the dot pattern 103 printed at a location on the printed matter 101a touched by the user using the scanner 104a. .
  • the read dot pattern 103 is converted into a dot code by analysis software in the central processing unit of the scanner 104a.
  • the scanner 104 sends the document ID part of the converted dot code to the document management part 120a on the document management server via the communication parts 150 and 151, and sends the XY coordinate value part to the coordinate conversion part 130 (S10a). ).
  • the document management unit 120a uses the document ID received from the scanner 104a to select DTP data 101b corresponding to the document ID from the DTP data stored in advance.
  • the selected DTP data 101b is sent to the coordinate conversion unit 130a and the specific information selection unit 140a via the communication units 151 and 50 (S20a).
  • the coordinate conversion unit 130a receives the coordinate value (Xt, Yt) in the XY coordinate system received from the scanner 104a, and the coordinate value (xt, yt) in the xy coordinate system in the DTP data 101b received from the document management unit 120a. (S30a).
  • the specific information selection unit 140a selects appropriate specific information from the specific information in the DTP data 101b received from the document management unit 120a based on the coordinate values (xt, yt) received from the coordinate conversion unit 130a. (S40a).
  • the specific information selection unit 140a outputs the selected specific information so as to be input to another device or to be input for the next arbitrary process performed in the information input device 105a.
  • the information input device 105 a may further include a printing unit (not shown) (printing unit) so that the newly acquired DTP data 101 b can be printed on the medium together with the dot pattern 103.
  • the dot code of the dot pattern 103 desirably includes a document ID for identifying the newly acquired DTP data 101b in addition to the XY coordinate values.
  • the printed material 101a used for input processing can be created from the newly acquired DTP data 101b.
  • the information processing apparatus 112 includes a scanner 104a, a coordinate conversion unit 130a, a specific information selection unit 140b, a communication unit 150b, a related information search unit 160 (related information search unit), and a display unit 170.
  • a specific example of the information processing apparatus 112 is a personal computer, for example.
  • the information processing device 112 is a PDA, a television receiver, a front projector, a rear projector, a game device, a karaoke device, a mobile phone terminal device, a POS terminal device, an ATM, a KIOSK terminal, a car navigation system, a pachinko machine, a watch, a smartphone, or the like. May be.
  • the information processing system includes an information processing apparatus 112, a document management server 113, a dictionary server 114, and a network connecting them.
  • the scanner 104a reads the dot pattern 103 at the touch position, sends the document ID portion of the dot code to the communication unit 150b, and sends the XY coordinate value portion of the dot code to the coordinate conversion unit 130a.
  • the communication unit 150b transmits the document ID received from the scanner 104a to the document management server 113, and receives the DTP data 101b corresponding to the document ID received from the document management server 113 as the coordinate conversion unit 130a and the specific information selection unit 140b. Send to. Further, the communication unit 150b transmits the specific information received from the related information search unit 160 to the dictionary server 114, and the related information of the specific information searched in the dictionary server 114 based on the transmitted specific information. And pass it to the related information search unit 160.
  • the specific information selection unit 140b selects specific information in the DTP data 101b using the xy coordinate values received from the coordinate conversion unit 130a and the DTP data 101b received from the communication unit 150b, and associates the selected specific information with the related information.
  • the information is output to the information search unit 160.
  • the related information search unit 160 sends the specific information received from the specific information selection unit 140b to the dictionary server 114 via the communication unit 150b, receives the related information as a search result from the dictionary server 114 via the communication unit 150b, The received related information is output to the display unit 170.
  • the related information search unit 160 is configured to search one or more dictionary servers 114 on the network. However, as a function to replace the dictionary server 114 on the network, a preset link table is set. A database and / or an electronic dictionary may be provided in the information processing apparatus 112, and the related information search unit 160 may search related information without accessing the network.
  • examples of the dictionary server 114 include a general-purpose link table, a general-purpose database, and a general-purpose electronic dictionary.
  • the display unit 170 presents the related information received from the related information search unit 160 to the user using a display or the like.
  • the related information search unit 160 provided in the information processing apparatus 112 communicates with the dictionary server 114 to acquire the related information, but instead of this configuration, the document management server 113 The related information search unit 160 may be provided.
  • the specific information selection unit 140b transmits the selected specific information to the related information search unit 160 on the document management server 113 via the communication unit.
  • the related information search unit 160 communicates with the dictionary server 114, acquires related information of the selected specific information, and transmits it to the information processing apparatus 112.
  • the dictionary server 114 stores explanations corresponding to data files indicating text, illustrations, photographs, and the like, URLs, and related information such as multimedia information.
  • the dictionary server 114 searches for data files indicating related information such as text, illustrations, and photographs that are the selected specific information.
  • the dictionary server 114 may be a search engine such as Wikipedia, Yahoo, Google, or a specialized database for membership. ⁇ Information processing procedure>
  • one dot code includes both an XY coordinate value and a code value
  • the specific information in the DTP data 101b corresponding to the phrase “Tokyo” 102a on the printed material 101a includes selection position information.
  • a case where the center coordinate value (xi, yi) is set in advance will be described.
  • the scanner 104a reads the dot pattern 103 printed at a location on the printed matter 101a touched by the user using the scanner 104a. .
  • the read dot pattern 103 is converted into a dot code by analysis software in the central processing unit of the scanner 104a.
  • the scanner 104a sends the document ID part of the converted dot code to the document management part 120a on the document management server 113 via the communication parts 150b and 151, and sends the XY coordinate value part to the coordinate conversion part 130 ( S10a).
  • the document management unit 120a uses the document ID received from the scanner 104a to select DTP data 101b corresponding to the document ID from the DTP data stored in advance.
  • the selected DTP data 101b is sent to the coordinate conversion unit 130a and the specific information selection unit 140b via the communication units 151 and 150b (S20a).
  • the coordinate conversion unit 130a receives the coordinate value (Xt, Yt) in the XY coordinate system received from the scanner 104a, and the coordinate value (xt, yt) in the xy coordinate system in the DTP data 101b received from the document management unit 120a. To the specific information selection unit 140b (S30a).
  • the specific information selection unit 140b selects appropriate specific information from the specific information in the DTP data 101b received from the document management unit 120a based on the coordinate values (xt, yt) received from the coordinate conversion unit 130a. To do.
  • the specific information selection unit 140b passes the selected specific information to the related information search unit 160 (S40a).
  • the phrase “Tokyo” 102 a is selected as an input to the information processing apparatus 112.
  • the related information search unit 160 transmits the specific information received from the specific information selection unit 140b to the dictionary server 114 via the communication unit 150b, and the related information of the specific information searched in the dictionary server is Received from the dictionary server 114 via the communication unit 150b. Then, the related information search unit 160 presents the received related information to the user via the display unit 170 (S50a).
  • the phrase “Tokyo” 102a is transmitted to the dictionary server 114 via the network.
  • the dictionary server 114 On the dictionary server 114, an explanatory sentence “What is Tokyo?” Corresponding to the headword “Tokyo” is searched and returned to the information processing apparatus 112.
  • the related information may be a data file indicating an illustration or a photograph in addition to the text.
  • FIGS. 48 to 52 One embodiment of the information processing apparatus according to the present invention that uses a display on which a dot pattern 103 is displayed together with content will be described with reference to FIGS. 48 to 52 as follows.
  • the printed material 101a is used in the information input device 105 in the first embodiment, the information input device 105a in the second embodiment, and the information processing device 112 in the third embodiment.
  • the information processing apparatus 112b will be described as an example of a modification of the information processing apparatus 112. Since the method of transforming the information input device 105 and the information input device 105a into a configuration using a display instead of the printed material 101a is the same, the description is omitted.
  • FIG. 48 illustrates an outline of the present embodiment.
  • the user downloads appropriate content from the Web server 113b (two-dimensional format information server) and displays it on the screen 15 of the information processing apparatus 112b.
  • the Web server 113b two-dimensional format information server
  • the user uses the scanner 104b to touch information to be searched using the information processing apparatus 112b on the display 115 on which the dot pattern 103 is displayed together with the content.
  • the user wants to search related information related to the phrase “Tokyo” 102a among the contents displayed on the display 115, and therefore touches the phrase “Tokyo” 102a displayed on the display 115 with the scanner 104b. ing.
  • the XY coordinate value of the display 115 is coded in advance, and when the user touches the display 115 with the scanner 104b, the coordinate value (Xt, Yt) in the XY coordinate system on the display 115 at the touch position. ) Is read into the information processing apparatus 112b.
  • the information processing apparatus 112b recognizes the specific information at the touch position using the read XY coordinate value and the content data 1d (two-dimensional format information) of the content downloaded and displayed from the Web server 113b.
  • the content data in the content file has its own xy coordinate system.
  • selection position information used to select them is set.
  • a corresponding mask area 102b and / or a center coordinate value (xi, yi) is set.
  • the phrase “Tokyo” 102a is selected from a lot of specific information included in the content data 1d.
  • the information processing apparatus 112b searches the dictionary server 114 using the selected phrase “Tokyo” 102a.
  • the related information that is the search result is returned from the dictionary server 114 to the information processing apparatus 112 b and displayed on the display 115.
  • the dot code in this embodiment may include only XY coordinate values.
  • the content ID corresponds to the document ID in the above embodiment.
  • a dot code including a display ID can also be used as a code value in order to distinguish the displays 115.
  • the display 115 may display the dots of the dot pattern 103 by adjusting the RGB values using a general display, or display a dot portion representing the dot pattern 103 separately from the pixels of the display 115. 115 may be arranged on 115.
  • the expression “display” is used for dots and the like. However, as described above, this expression includes “a dot portion arranged on the display 115”. The case is also included.
  • FIG. 49 shows a cross-sectional view of an organic EL display in which a dot portion is provided separately from a pixel.
  • FIG. 49A is a diagram illustrating a state in which infrared rays incident on the dot portion are transmitted or absorbed
  • FIG. 49B is a diagram in which infrared rays are incident on light emitting elements other than the dot portion and diffusely reflected. It is a figure which shows a mode.
  • the dot pattern can be expressed by making the infrared reflection characteristics different between the light emitting element portion and the dot portion.
  • FIG. 50 shows an example in which a dot portion is provided on the screen of a front projector or a rear projector as a modification of the display 115.
  • FIG. 50A is a diagram illustrating a state in which infrared rays incident on the dot portion are transmitted or absorbed
  • FIG. 50B is a diagram in which infrared rays are incident on a screen portion other than the dot portion and diffusely reflected. It is a figure which shows a mode. ⁇ About content data>
  • content data examples include data created by HTML, XML, flash, or the like.
  • DTP data exemplified in the above embodiment can also be handled as content data.
  • FIG. 51 shows a functional block diagram of the information processing apparatus 112b according to the present embodiment.
  • the information processing apparatus 112b includes a scanner 104b, a coordinate conversion unit 130b, a specific information selection unit 140c, a communication unit 150c, a related information search unit 160, a display unit 170b, a display 115, and a content acquisition unit 180.
  • a specific example of the information processing apparatus 112b is a personal computer, for example.
  • the information processing system includes an information processing apparatus 112b, a Web server 113b, a dictionary server 114, and a network connecting them.
  • the content acquisition unit 180 transmits a content request to the Web server 113b via the communication unit 150c, and receives and receives the content data 1d returned from the Web server 113b via the communication unit 150c.
  • the content data 1d is presented to the user via the display unit 170b and the display 115.
  • the scanner 104b reads the dot pattern 103 at the touch position and sends the XY coordinate value of the dot code to the coordinate conversion unit 130b.
  • the coordinate conversion unit 130b uses the XY coordinate values (Xt, Yt) received from the scanner 104b and the coordinate values (Xt, Yt) of the touch position in the xy coordinate system on the content data 1d using the content data 1d received from the content acquisition unit 180. xt, yt), and the converted coordinate value (xt, yt) is sent to the specific information selection unit 140c.
  • the specific information selection unit 140c selects the specific information in the content data 1d using the xy coordinate value received from the coordinate conversion unit 130b and the content data 1d received from the content acquisition unit 180, and selects the selected specific information.
  • the information is output to the related information search unit 160.
  • the related information search unit 160 sends the specific information received from the specific information selection unit 140c to the dictionary server 114 via the communication unit 150c, receives the related information as a search result from the dictionary server 114 via the communication unit 150c, The received related information is presented to the user via the display unit 170b and the display 115.
  • the communication unit 150c transmits the content request received from the content acquisition unit 180 to the Web server 113b, and displays the content data 1d corresponding to the content request received from the Web server 113b, the display unit 170b, the coordinate conversion unit 130b, and the identification
  • the information is sent to the information selection unit 140c.
  • the communication unit 150c transmits the specific information received from the related information search unit 160 to the dictionary server 114, and the related information of the specific information searched in the dictionary server 114 based on the transmitted specific information. And pass it to the related information search unit 160.
  • the display unit 170b presents the content data 1d received from the content acquisition unit 180 and the related information received from the related information search unit 160 to the user via the display 115.
  • the center coordinate value (xi, yi) is set in advance as the position information for selection in the specific information in the content data 1d corresponding to the phrase “Tokyo” 102a displayed on the display 115. Will be described.
  • the content acquisition unit 180 transmits a content request to the Web server 113b via the communication unit 150c, and receives the content data 1d corresponding to the content request from the Web server 113b via the communication unit 150c.
  • the content acquisition unit 180 displays the received content data 1d on the display 115 via the display unit 170b (S5).
  • the content acquisition unit 180 also outputs the content data 101d of the content being displayed to the coordinate conversion unit 130b and the specific information selection unit 140c.
  • the process of S5 represents a state where the user is browsing the content data downloaded from the Web server 113b by surfing the Internet using a general Web browser.
  • the scanner 104b displays the dot pattern 103 set at a location on the display 115 touched by the user using the scanner 104b. read.
  • the read dot pattern 103 is converted into a dot code by analysis software in the central processing unit of the scanner 104b.
  • the scanner 104b sends the XY coordinate value of the converted dot code to the coordinate conversion unit 130b (S10b).
  • the coordinate conversion unit 130b receives the coordinate value (Xt, Yt) in the XY coordinate system received from the scanner 104b, and the coordinate value (xt, yt) in the xy coordinate system in the content data 1d received from the content acquisition unit 180. To the specific information selection unit 140c (S30a).
  • the specific information selection unit 140c selects appropriate specific information from the specific information in the content data 1d received from the content acquisition unit 180 based on the coordinate values (xt, yt) received from the coordinate conversion unit 130b. To do.
  • the specific information selection unit 140c passes the selected specific information to the related information search unit 160 (S40b).
  • the phrase “Tokyo” 102a is selected as an input to the information processing apparatus 112b.
  • the related information search unit 160 transmits the specific information received from the specific information selection unit 140c to the dictionary server 114 via the communication unit 150c, and the related information of the specific information searched in the dictionary server is Received from the dictionary server 114 via the communication unit 150c. Then, the related information search unit 160 presents the received related information to the user via the display unit 170b and the display 115 (S50a).
  • the phrase “Tokyo” 102a is transmitted to the dictionary server 114 via the network.
  • an explanatory sentence “What is Tokyo?” Corresponding to the headword “Tokyo” is searched and returned to the information processing apparatus 112 b.
  • an explanatory note “What is Tokyo” On the screen of the information processing apparatus 112b, an explanatory note “What is Tokyo” which is related information corresponding to the phrase “Tokyo” 102a is displayed on the screen.
  • a transparent sheet on which a dot pattern 103 is printed is placed on a printing surface of a normal printed material and used is described below with reference to FIG. is there.
  • this transparent sheet is referred to as a grid sheet 101g (information input auxiliary sheet).
  • FIG. 53 shows an outline of the present embodiment.
  • the user inputs an index for specifying a normal printed matter 101e on which the dot pattern 103 is not printed, to the information processing apparatus 112c.
  • the input method may be a method in which the user touches an icon (described later) printed on the grid sheet 101g to input an index for specifying the printed material 101e, or a method in which the user inputs an index using a keyboard or the like.
  • the index may be converted into a document ID in the document management server 113, and the DTP data on the document management server 113 may be searched using the converted document ID.
  • the user accurately overlays the grid sheet 101g on the printing surface of the printed material 101e.
  • the user touches the phrase “Tokyo” 102a, which is specific information on the print surface that the user wants to search for related information, through the grid sheet 101g.
  • the scanner 104a reads the coordinate values (Xt, Yt) in the XY coordinate system on the grid sheet 101g at the touch position.
  • the process after the index and the coordinate value (Xt, Yt) of the touch position are input to the information processing apparatus 112c is the same as the process in the third embodiment.
  • FIG. 54 shows an outline of the present embodiment.
  • the user places the grid sheet 101g on the display screen of the information processing apparatus 112d.
  • the placement may be performed by sticking the grid sheet 101g to the display. It is desirable that a calibration operation is performed after the grid sheet 101g is arranged.
  • the user downloads a desired content file from the Web server 113b to the information processing apparatus 112d using a Web browser or the like.
  • the content in the downloaded content file is displayed on the screen of the information processing apparatus 112d.
  • the user touches the phrase “Tokyo” 102a, which is specific information for searching related information on the screen, using the scanner 104a through the grid sheet 101g arranged on the display screen.
  • the scanner 104a reads the coordinate values (Xt, Yt) in the XY coordinate system on the grid sheet 101g at the touch position.
  • the information processing apparatus 112d recognizes the specific information at the touch position using the read XY coordinate value and the content data 1d of the content downloaded and displayed from the Web server 113b.
  • the processing after recognizing the specific information is the same as the processing in the fourth embodiment.
  • FIGS. 55 to 57 An embodiment using a television receiver having a display on which a dot pattern 103 is displayed together with content in the information processing apparatus according to the present invention will be described with reference to FIGS. 55 to 57 as follows. .
  • the printed material 101a is used in the information input device 105 in the first embodiment, the information input device 105a in the second embodiment, and the information processing device 112 in the third embodiment.
  • the television receiver 112e will be described as an example in which the information processing apparatus 112 is modified. Since the method of transforming the information input device 105 and the information input device 105a into a configuration using a display instead of the printed material 101a is the same, the description is omitted.
  • FIG. 55 shows an overview of the present embodiment.
  • the user first receives a broadcast radio wave in which the dot pattern 103 is superimposed on a normal image using the television receiver 112e, and displays the received image on the display 115. Then, the user uses the scanner 104c to touch information to be input to the television receiver 112e on the display 115 on which the dot pattern 103 is displayed together with the received image.
  • the user wants to search related information related to the phrase “Tokyo” 102a in the received image displayed on the display 115, and therefore touches the phrase “Tokyo” 102a displayed on the display 115 with the scanner 104c. is doing.
  • the XY coordinate value of the display 115 and the content ID for specifying the received image are coded.
  • the XY coordinate on the display 115 at the touch position is displayed.
  • the coordinate value (Xt, Yt) and content ID in the system are read into the television receiver 12e.
  • the television receiver 112e uses the read content ID, and the content corresponding to the content ID among the content data in which the specific information and the position information for selection are set in advance from the content server 113c (two-dimensional format information server). Download data 1d. Then, the specific information at the touch position is selected using the read XY coordinate values and the downloaded content data 1d.
  • the content data in the content file has its own xy coordinate system.
  • selection position information used to select them is set.
  • a corresponding mask area 102b and / or a center coordinate value (xi, yi) is set.
  • the phrase “Tokyo” 102a is selected from a lot of specific information included in the content data 1d.
  • the television receiver 12e searches the dictionary server 114 using the selected phrase “Tokyo” 102a. Related information as a search result is returned from the dictionary server 114 to the television receiver 12e and displayed on the display 115.
  • the dot code of the dot pattern 103 in the present embodiment includes the content ID for identifying the content received by the television receiver 112e as a broadcast wave and the XY coordinate value.
  • a dot code and a dot pattern 103 are generated from an XY coordinate value generated by the television receiver 112e itself and a content ID acquired from EPG (Electronic Program Guide) data obtained by the television receiver 112e, and the display 115 may be displayed on the screen 115.
  • EPG Electronic Program Guide
  • the dot code of the dot pattern 103 displayed on the display 115 may include only XY coordinate values.
  • the dot pattern 103 may have a configuration in which the dot portion of the dot pattern 103 is fixedly provided in advance alongside the pixels on the display 115 instead of the configuration represented by RGB pixels.
  • the content ID is a corresponding portion of a print medium such as a television guide in which the content ID is printed as the dot pattern 103. It may be configured to be read by touching, or may be configured to be input manually by a user using a paper keyboard and the scanner 104c.
  • the content ID displayed on the display 115 may be displayed together with the XY coordinate value of the dot pattern 103 in at least a part of the region where the XY coordinate value of the dot pattern 103 is displayed, or is different from the region. It may be displayed in the area.
  • the content ID displayed on the display 115 is the XY coordinate value of the dot pattern 103 that uniquely identifies the content data.
  • This XY coordinate value is an area where the XY coordinate value of the dot pattern 103 is displayed.
  • the configuration may be displayed entirely.
  • content data examples include moving image data such as MPEG, and data created by HTML, XML, flash, and the like.
  • moving image data such as MPEG
  • HTML HyperText Markup Language
  • XML HyperText Markup Language
  • flash Portable Network Markup Language
  • content data examples include moving image data such as MPEG, and data created by HTML, XML, flash, and the like.
  • DTP data exemplified in the above embodiment can also be handled as content data.
  • the content data basically includes two-dimensional format information, but in addition to this, MPEG data or the like may have three-dimensional format information including a time axis. Further, if it is three-dimensional stereoscopic image data, it may have three-dimensional format information including the Z axis in the front-rear direction toward the display 115. Furthermore, as long as it is moving image data of a three-dimensional stereoscopic image, it may have four-dimensional format information having an XYZ axis and a time axis.
  • FIG. 56 shows a functional block diagram of the television receiver 112e according to the present embodiment.
  • the television receiver 112e includes a scanner 104c, a coordinate conversion unit 130c, a specific information selection unit 140d, a communication unit 150d, a related information search unit 160, a display unit 170c, a display 115, and a television image reception unit 190. I have.
  • the information processing system includes a television receiver 112e, a content server 113c, a dictionary server 114, and a network connecting them.
  • the television receiver 190 receives broadcast waves.
  • a dot pattern 103 in which a content ID included in a broadcast wave or obtained from an EPG and an XY coordinate value on a display screen is combined is added to the received image, and the user is notified via the display unit 170c and the display 115.
  • the dot pattern 103 may be added to the received image only when the user uses the scanner 104c.
  • the scanner 104c reads the dot pattern 103 at the touch position, sends the XY coordinate value of the dot code to the coordinate conversion unit 130c, and sends the content ID of the dot code to the content server 113c via the communication unit 150d.
  • the content ID may be directly sent from the television receiver 190 to the content server 113c when the scanner 104c reads the XY coordinate values instead of being converted into the dot pattern 103 and read by the scanner 104c.
  • the coordinate conversion unit 130c uses the XY coordinate values (Xt, Yt) of the touch position received from the scanner 104c and the xy coordinates on the content data 1d using the content data 1d received from the content server 113c via the communication unit 150d.
  • the coordinate value (xt, yt) of the touch position in the system is converted, and the converted coordinate value (xt, yt) is sent to the specific information selection unit 140d.
  • the specific information selection unit 140d selects specific information in the content data 1d using the xy coordinate values received from the coordinate conversion unit 130c and the content data 1d received from the content server 113c via the communication unit 150d, The selected specific information is output to the related information search unit 160.
  • the related information search unit 160 sends the specific information received from the specific information selection unit 140d to the dictionary server 114 via the communication unit 150d, receives related information as a search result from the dictionary server 114 via the communication unit 150d, The received related information is presented to the user via the display unit 170c and the display 115.
  • the communication unit 150d transmits the content ID received from the scanner 104c to the content server 113c, and transmits the content data 1d corresponding to the content ID received from the content server 113c to the coordinate conversion unit 130c and the specific information selection unit 140d. . Further, the communication unit 150d transmits the specific information received from the related information search unit 160 to the dictionary server 114, and the related information of the specific information searched in the dictionary server 114 based on the transmitted specific information. And pass it to the related information search unit 160.
  • the display unit 170c presents the received image to which the dot pattern 103 is added received from the television receiver 190 and the related information received from the related information search unit 160 to the user via the display 115. ⁇ Information processing procedure>
  • the center coordinate value (xi, yi) is set in advance as the position information for selection in the specific information in the content data 1d corresponding to the phrase “Tokyo” 102a displayed on the display 115. Will be described.
  • the television receiver 190 receives a broadcast wave, and superimposes the dot pattern 103 including the content ID acquired from the broadcast wave or the Internet and the XY coordinate value on the screen of the display 115 on the received image to display the display unit.
  • the image is displayed on the display 115 via 170c (S5b).
  • the scanner 104c displays the dot pattern 103 set at a location on the display 115 touched by the user using the scanner 104c. read.
  • the read dot pattern 103 is converted into a dot code by analysis software in the central processing unit of the scanner 104c.
  • the scanner 104c transmits the XY coordinate value of the converted dot code to the coordinate conversion unit 130c, and transmits the content ID of the converted dot code to the content server 113c via the communication unit 150d (S10c).
  • the content server 113c uses the content ID received from the scanner 104c to select content data 1d corresponding to the content ID from the content data stored in advance. Then, the selected content data 1d is sent to the coordinate conversion unit 130c and the specific information selection unit 140d via the communication unit 150d (S20b).
  • the coordinate conversion unit 130c converts the coordinate value (Xt, Yt) in the XY coordinate system received from the scanner 104c to the coordinate value (xt, yt) in the xy coordinate system in the content data 1d acquired from the content server 113c.
  • the data is converted and passed to the specific information selection unit 140d (S30a).
  • the specific information selection unit 140d selects appropriate specific information from the specific information in the content data 1d received from the content server 113c based on the coordinate values (xt, yt) received from the coordinate conversion unit 130c. .
  • the specific information selection unit 140d passes the selected specific information to the related information search unit 160 (S40b).
  • the phrase “Tokyo” 102a is selected as an input to the television receiver 112e.
  • the related information search unit 160 transmits the specific information received from the specific information selection unit 140d to the dictionary server 114 via the communication unit 150d, and the related information of the specific information searched in the dictionary server 114 is obtained. And received from the dictionary server 114 via the communication unit 150d. Then, the related information search unit 160 presents the received related information to the user via the display unit 170c and the display 115 (S50a).
  • the phrase “Tokyo” 102a is transmitted to the dictionary server 114 via the network.
  • an explanatory sentence “What is Tokyo” corresponding to the headword “Tokyo” is searched and returned to the television receiver 112 e.
  • an explanatory note “What is Tokyo” which is related information corresponding to the phrase “Tokyo” 102a is displayed on the screen.
  • the content data is analyzed in advance and the specific information and the position information for selection are already set.
  • the present invention is not limited to this configuration.
  • the specific information selection unit 140d is a text, figure, illustration, photograph, etc. within a predetermined distance from the coordinate value (xt, yt) of the touch position for the first time.
  • the content data may be analyzed to recognize specific information and set position information for selection.
  • the television receiver 112e is described as an example of the information processing apparatus.
  • the television receiver is a normal one, and is combined with a set-top box to be additionally installed.
  • the function of the machine 112e may be realized.
  • the user touches the medium on which the dot pattern is printed, the display on which the dot pattern is displayed or provided, or the grid sheet on which the dot pattern is provided with the scanner.
  • the input method of the coordinate position is not limited to these, and a method of touching a normal touch panel or touch screen provided on a normal display with a finger or a method using a normal mouse operation may be used.
  • the information input device uses the coordinate value (xt, yt) of the touch position to search for the specific information, based on the selection position information setting method according to any one of the above, One or two or more information search methods may be assigned, and one of the search methods may be selected by a predetermined algorithm to search for the specific information.
  • the information input device selects one or more specific information from a plurality of the specific information searched within an arbitrary distance from the coordinate value (xt, yt), and stores related information of the specific information in advance.
  • a search may be performed from a set link table, database, or electronic dictionary, and the related information may be output.
  • the coordinate value (xt, yt) of the touch position among all the specific information searched within an arbitrary distance from the coordinate value (xt, yt) is the total coordinate value in the mask.
  • the position information for selecting the specific information is used as the centroid of the mask, and the touch position
  • the specific information having the shortest distance from the coordinate value (xt, yt) of the specific information to the position information for selection of the plurality of searched specific information is searched, or candidates for the specific information are searched in order of the shortest distance
  • the selection may be a feature.
  • each block of the information input devices 105 and 105a, the information processing devices 112 to 112d, and the television receiver 112e, particularly the coordinate conversion units 130 to 130c, the specific information selection units 140 to 140d, and the related information search unit 160 You may comprise by a hardware logic and may implement
  • the information input devices 105 and 105a, the information processing devices 112 to 112d, and the television receiver 112e each have a CPU (central processing unit) that executes instructions of a control program that realizes each function, and a ROM (read that reads the program). only (memory), RAM (random access memory) for expanding the program, and a storage device (recording medium) such as a memory for storing the program and various data.
  • An object of the present invention is to provide program codes (execution format program, intermediate code program) of control programs for the information input devices 105 and 105a, the information processing devices 112 to 112d, and the television receiver 112e, which are software for realizing the functions described above.
  • a source program) recorded on a computer readable medium is supplied to the information input devices 105 and 105a, the information processing devices 112 to 112d, and the television receiver 112e, and the computer (or CPU or MPU) records them. This can also be achieved by reading and executing the program code recorded on the medium.
  • Examples of the recording medium include a tape system such as a magnetic tape and a cassette tape, a magnetic disk such as a floppy (registered trademark) disk / hard disk, and an optical disk such as a CD-ROM / MO / MD / DVD / CD-R.
  • Card system such as IC card, IC card (including memory card) / optical card, or semiconductor memory system such as mask ROM / EPROM / EEPROM / flash ROM.
  • the information input devices 105 and 105a, the information processing devices 112 to 112d, and the television receiver 112e may be configured to be connectable to a communication network, and the program code may be supplied via the communication network.
  • the communication network is not particularly limited.
  • the transmission medium constituting the communication network is not particularly limited.
  • infrared rays such as IrDA and remote control, Bluetooth ( (Registered trademark), 802.11 wireless, HDR, mobile phone network, satellite line, terrestrial digital network, and the like can also be used.
  • the present invention can also be realized in the form of a computer data signal embedded in a carrier wave in which the program code is embodied by electronic transmission.
  • FIG. 60 is an external view showing an example of a usage state of the handwriting input / output system 201 according to the present invention.
  • the document as written is displayed on the display 206.
  • the scanner 203 taking an image of the dot pattern on the written locus and analyzing the computer 204 (accessing the server 205 as necessary).
  • the computer 204 may be built in the scanner 203.
  • FIG. 61 is a block diagram showing the configuration of the first embodiment of the handwriting input / output system 201 according to the present invention.
  • the handwriting input / output system 201 includes at least a medium 202 (handwriting input sheet), an imaging unit 207 (imaging unit), a dot pattern analysis unit 208 (dot pattern analysis unit), a locus recognition unit 209 (trajectory recognition unit), and a processing instruction unit.
  • 210 processing instruction means
  • FIG. 62 is a block diagram showing a configuration of the second embodiment of the handwriting output / input system 1 according to the present invention.
  • an imaging device 212 In addition to the configuration provided in the first embodiment, an imaging device 212, an irradiation unit 213, and a light control unit 214 are further provided.
  • the imaging device 212 and the irradiation unit 213 are integrated with the imaging unit 207, and the light control unit 214 is integrated with the dot pattern analysis unit 208.
  • FIG. 63 is a block diagram showing the configuration of the third embodiment of the handwriting input / output system 201 according to the present invention.
  • a timer 215 is further provided.
  • the time measuring unit 215 is integrated with the locus recognizing unit 209.
  • FIG. 64 is a block diagram showing a configuration of the fourth embodiment of the handwriting input / output system 201 according to the present invention.
  • an angle measuring unit 216 and an angle change recognizing unit 217 are further provided.
  • the angle measurement unit 216 and the angle change recognition unit 217 are integrated with the dot pattern analysis unit 208.
  • FIG. 65 is a block diagram showing the configuration of the fifth embodiment of the handwriting input / output system 201 according to the present invention.
  • a rotation angle reading unit 218 and a rotation angle change recognition unit 219 are further provided.
  • the rotation angle reading unit 218 and the rotation angle change recognition unit 219 are integrated with the dot pattern analysis unit 208.
  • FIG. 66 is a block diagram showing the configuration of the sixth embodiment of the handwriting input / output system 201 according to the present invention.
  • a voice recording unit 220 in addition to the configuration provided in the first embodiment, a voice recording unit 220, a voice recognition unit 221, and a voice output unit 222 are further provided.
  • FIG. 67 is a diagram for explaining the format of the dot pattern formed on the medium 202 in the handwriting input / output system 201 according to the present invention.
  • the dot pattern is a dot pattern composed of unit areas of 2 rows ⁇ 5 columns of information dots in which a set of constant information is defined, and C 0 to C 29 in this unit. It is divided into.
  • the dot code format of each area is shown in (a) to (c).
  • FIG. 67 (a) shows a format in which a page code, an X coordinate, and a Y coordinate are registered.
  • the page code means the number of pages of the medium 202 when the medium 202 is in the form of a booklet, and the X coordinate and the Y coordinate each mean a position (coordinate value) on the medium. Is.
  • the scanner 203 the number of pages of the medium and the coordinate position of the read position are simultaneously recognized by one reading operation.
  • the action code means an operation instruction. For example, it is defined only in the area where icons indicating operations such as “enlarge” and “reduce” provided on the page are drawn, and is mainly included in the dot pattern used for the icon area described later. ing.
  • the command processing may be associated with the action code and executed.
  • the command processing refers to processing for recognizing characters during handwriting input or after handwriting input, or transmitting to another information processing apparatus.
  • a plurality of specific write area codes in parentheses may be provided in one medium surface, and these write areas must be defined by unique codes. This makes it possible to determine where the data is written instantaneously. Of course, it may be defined together with the page code so that it can be determined which specific write area of which page. Furthermore, when the content input by handwriting belongs to a predetermined category, the recognition rate is remarkably improved by collation with the dictionary of the category. Needless to say, the contents input by handwriting may be not only characters but also symbols and illustrations. Of course, the action code may be defined together with the specific write area code and the instruction process may be executed.
  • this write area flag may be accompanied by the page code (specific write area code) and action code shown in FIGS.
  • the dot pattern of the present invention can register various code information such as page code and action code and XY coordinates in one format.
  • various code information such as page code and action code and XY coordinates in one format.
  • Imaging unit> 68 to 81 are diagrams for explaining the imaging unit 207 used in the handwriting input / output system 201 according to the present invention.
  • the imaging unit 207 preferably has a size and shape suitable for a person to hold, and most preferably has a pen shape as shown in FIG.
  • FIG. 9A is a diagram showing the most basic embodiment of the imaging unit 207 having a pen shape.
  • an instruction projection 228 and a transmission unit 223 are provided, and a C-MOS sensor or the like (not shown) is generally built on the axis of the pen.
  • the transmission unit 223 is for the imaging unit 207 to transmit a signal to another member using a method such as infrared communication or wireless communication, and is illustrated for convenience of explanation. It is not necessary to provide it at the position.
  • FIG. 6B is a diagram showing another embodiment of the imaging unit 207 having a pen shape.
  • two buttons for accepting an operation by the user are provided.
  • the two buttons are a send button 224 and a re-send button 225, and the user can instruct the user to send a signal from the imaging unit 207 to another member.
  • the re-transmission button 225 is used to re-transmit the same signal when the transmission of the signal by the transmission button 224 cannot be transmitted to the receiving member due to the presence of a shield or the like.
  • FIG. 10C is a diagram showing another embodiment of the imaging unit 207 having a pen shape.
  • a microphone 226 that accepts voice input by the user and an audio playback button 227 are further provided.
  • the microphone 226 is used for recording a voice used for voice recognition described later, and the voice playback button 227 is used for playing a voice guide described later.
  • FIG. 4D is a diagram showing another embodiment of the imaging unit 207 having a pen shape.
  • the pen 229 is stored in the internal space of the instruction projection 228, and the pen 229 comes out when the pen storage button 230 is pressed.
  • FIG. 69 is a cross-sectional view showing the distal end portion of a pen-type scanner 231 that is one embodiment of the imaging unit 207 constituting the handwriting input / output system 201 according to the present invention.
  • the pen type scanner 231 has a light guide 232 having a hollow portion formed therein and having an opening at the tip thereof, a lens 233 disposed at a position facing the opening of the light guide 232, and the same as the lens 233.
  • An LED 234 serving as a light source for irradiating light of a predetermined wavelength onto a medium surface on which a dot pattern is formed, and a C-MOS sensor 235 (imaging) disposed at a position retracted from the lens 233 An element 212) and a PCB 236 at a further retracted position and a CPU (not shown) installed therein are provided.
  • the hollow portion of the light guide 232 is formed in a tapered shape that gradually increases in diameter from the lens 233 toward the opening.
  • the CPU has any or all of the functions of the dot pattern analysis unit 208, the trajectory recognition unit 209, the processing command unit 210, and the information processing unit 211 described above.
  • the light guide 232 is made of transparent or milky white resin, and the inside functions as a light guide. Irradiation light from the LED 234 travels inside the light guide 232 and is irradiated to the medium 202 from the opening.
  • the light guide 232 is made of milky white resin, the light emitted from the LED 234 is appropriately diffused when traveling through the light guide 232, so that the light irradiating the medium 202 from the opening is made more uniform. I can do it.
  • the pen-type scanner 231 is equipped with a pen 229 as a writing instrument.
  • a writing instrument such as a normal pen
  • the outer periphery of the pen-type scanner 231 is formed so as to be inclined up to 45 degrees.
  • the handwriting can be analyzed more accurately by offset correction described later.
  • the pen 229 as a writing instrument can be stored by various methods so that no handwriting is left on the medium.
  • a technique used in a general pen such as a ballpoint pen can be used as it is.
  • a knock type or rotary type storage method may be employed, or a cap may be put on.
  • the pen can be changed like a general three-color ballpoint pen.
  • the pens of different colors may be replaced, but an eraser or a correction liquid may be used instead of the pen.
  • an instruction protrusion 228 (stylus pen) may be provided to image the vicinity of the handwriting of the instruction protrusion 228.
  • a storage space may be provided in the instruction protrusion 228 and the pen 229 may be stored to be used as an instruction protrusion.
  • the XY coordinate value defined by the dot pattern captured by the imaging unit 207 is different from the XY coordinate value defined by the dot pattern on the locus actually written by the pen 229. Furthermore, the XY coordinate value defined by the dot pattern on the locus needs to be converted into the xy coordinate value in the writing area on the medium surface to perform writing input and operation instruction.
  • FIG. 70 shows an example of the write area.
  • the writing area coordinate system is (0,0) in the lower left corner.
  • the xy coordinate value of the writing area is expressed in mm.
  • a coordinate position of (16, 40) indicates a position moved 16 mm to the right and 40 mm upward from the lower left corner of the writing area.
  • At least dot pattern units that define at least one set of XY coordinate values in the dot pattern coordinate system are continuously formed.
  • the XY coordinate value indicates the position of the center point of the unit. In order to distinguish the coordinate value of the dot pattern coordinate system from the coordinate value of the writing area coordinate system, it is expressed as [X, Y].
  • the unit is a stream dot in which a set of constant information is defined, and the center of the first virtual reference line connecting the reference dots arranged in a line is the center of the unit.
  • the XY in the dot pattern coordinate system As shown in FIG. 71, a dot pattern is formed such that the lower left corner is [ ⁇ 0.5, ⁇ 0.5] in any writing area as shown in FIG.
  • a unique code value is defined for the dot pattern formed in each writing area, and each writing area is specified. If there is no need to specify each other, the same code value may be defined.
  • a dot pattern in which XY coordinate values are defined is formed on the entire surface of the paper, and a unique code value is defined in the writing area in addition to the XY coordinate values.
  • the XY coordinate value of the lower left corner of the writing area is [Xn0, Yn0] (n is a number specifying the writing area).
  • the XY coordinate values are converted to the writing area coordinate system as follows.
  • the relationship between the writing area coordinate system and the dot pattern coordinate system in the example of FIG. 72 is the same as the example of the enlarged view portion of FIG.
  • ⁇ Calculation of XY coordinate values [Xc, Yc] in the dot pattern coordinate system of the imaging center C A calculation method of the XY coordinate values [Xc, Yc] of the imaging center C will be described with reference to FIG.
  • the number u of pixels constituting the interval between the unit center P [Xp, Yp] closest to the imaging center C and the imaging center C is obtained.
  • the direction of the CQ vector starting from the imaging center C and ending at the pen tip is the direction of the pen scanner 231.
  • the angle between the direction of the dot pattern and the direction of the pen-type scanner 231 is ⁇ (left rotation is the positive direction).
  • the pen body may be provided with an inclination measuring means such as an inclination sensor.
  • a table that associates ⁇ BL with ⁇ may be created in advance and ⁇ may be calculated.
  • ⁇ Media 202> 80 to 93 are diagrams for explaining the medium 202 used in the handwriting input / output system 201 according to the present invention.
  • FIG. 80 is a diagram showing the medium 202 that is printed (formed) on the entire surface to form a writing area.
  • This dot pattern defines XY coordinate values and page codes.
  • the page code means a code for specifying a medium using code information.
  • FIG. 81 is a diagram showing a medium 202 on which another dot pattern further defining an action code is printed in an area where a dot pattern defining a page code and an XY coordinate value is printed.
  • the action code means a code for instructing the information processing unit 211 to perform a predetermined operation (especially an operation of an application) with respect to a handwritten input to a specified medium by using code information.
  • a predetermined operation especially an operation of an application
  • This media consists of multiple sheets, and a different page code is assigned to each sheet.
  • the page code is “01” for the first page, “02” for the second page, and “03” for the third page.
  • an action code in an area printed with a text icon “start” superimposed thereon instructs the information processing unit 211 to start an application for handwriting input / output.
  • the action code of the area printed with the text icon “Cancel” superimposed thereon instructs the information processing unit 211 to discard (cancel) the input result by the handwriting input / output application, or the information processing unit 211
  • the information processing unit 211 is instructed to stop the execution of the process.
  • the action code in the area printed with the text icon “save” superimposed instructs the information processing unit to save the input result by the handwriting input / output application.
  • the action code in the area printed with the text icon “End” superimposed thereon instructs the information processing unit 211 to end the handwriting input / output application.
  • FIG. 82 is a diagram showing a medium 202 in which an area where a dot pattern defining XY coordinate values and a page code is printed and an area where a dot pattern defining an action code and a page code are printed are separated.
  • various commands can be executed only for the page specified by the page code without selecting which page of the medium is the target. Thereby, security can be improved and an erroneous operation can be reduced.
  • FIG. 83 is a diagram illustrating a first medium on which a dot pattern in which XY coordinate values and page codes are defined is printed, and a second medium that is a paper controller on which a dot pattern in which only action codes are defined is printed.
  • various commands for handwritten input can be executed on any page of the medium.
  • the writing area and the icon are provided separately, as shown in FIG. 85, in addition to the paper controller, the writing area and the icon may be affixed to another medium as a sticker, sticky note, or card.
  • the action code of the area printed with the text icon “text input” superimposed is used to instruct the information processing unit 211 to input characters or figures as the text in the operation by the handwriting input / output application.
  • the action code of the area printed with the text icon “comment input” superimposed thereon instructs the information processing unit 211 to input a character or a figure to be a comment in the operation by the handwriting input / output application.
  • the action code in the area printed with the text icon “edit” superimposed thereon instructs the information processing unit 211 to edit the character or figure in the operation by the handwriting input / output application (that is, handwriting input / output).
  • the action code of the area printed with the text icon “determine” superimposed is used to execute the process for the information processing unit 211 when the information processing unit 211 determines whether to execute the predetermined process. Command to decide.
  • FIG. 84 shows a first medium on which a dot pattern defining XY coordinate values and page code A is printed, and a second medium which is a paper controller on which a dot pattern defining action code and page code A is printed. It is a figure which shows what became one booklet.
  • a dot pattern in which the same XY coordinates of each page are registered may be printed in the writing area.
  • the writing area can be specified by using the icon of the controller area in which the action code and page code are registered. That is, it can be said that the position and page of the writing area can be changed without changing the XY coordinate values.
  • various commands can be executed only for the page specified by the page code without selecting which page of the medium is the target. Thereby, security can be improved and an erroneous operation can be reduced.
  • a dot pattern defining an action code for instructing the information processing unit to start an application for handwriting input / output may be printed over the entire writing area. In this case, when writing is performed on the medium surface, the handwriting input / output application is automatically activated.
  • the number of write areas on the medium is not limited to one, and a plurality of write areas can be defined. (Regarding the case where multiple writing areas are provided)
  • a dot pattern in which XY coordinate values based on different dot pattern coordinate systems and code values for specifying a writing area are formed is formed for each of a plurality of writing areas.
  • the code value can include not only the writing area but also the classification number that classifies the page number by defining multiple information.
  • the page number When the page number is included, it is possible to determine which writing area of which page, and there are selection items such as information, processing instructions, and alternatives in the writing area, and when processing as a common code, the defined page number In addition, information specifying a selection item can be combined and defined as a code value.
  • different code values can be defined by dividing the area in the write area.
  • the code values are n1, n2, n3, and n4. Thereby, a plurality of selection items can be set in one writing area.
  • This first example has the advantage that it is easy to handle because the lower left corner of each area is always 0 (origin).
  • a dot pattern in which the XY coordinate value is defined by one dot pattern coordinate system is formed on the entire surface of the predetermined area, and the code value is defined together with the XY coordinate in the dot pattern of the writing area. This is the case.
  • a dot pattern in which XY coordinate values are defined in one dot pattern coordinate system with the lower left corner of the predetermined area as the origin is formed on the entire surface of the predetermined area.
  • the two writing areas in the predetermined area are defined with XY coordinate values whose lower left corners are [Xm0, Ym0] and [Xn0, Yn0] and code values m and n for specifying the writing area, respectively. ing.
  • code values can be defined by dividing the area within the write area.
  • the third example is a case where a write area is further inserted into the write area.
  • a dot pattern defining an XY coordinate value in one dot pattern coordinate system with the origin at the lower left corner of a predetermined area and a code value L for specifying a writing area is formed on the entire predetermined area.
  • Two write areas in the predetermined write area are defined by XY coordinate values whose lower left corners are [Xm0, Ym0] and [Xn0, Yn0] and code values m and n for specifying the write area, respectively. Has been.
  • the writing area can be formed in the writing area in one coordinate system, the writing mode in the writing area can be ignored and the writing operation can be performed by changing the input mode. It becomes possible. For example, when writing to a writing area in the writing area, a setting that is effective only when the input mode is switched or a specific symbol is drawn may be provided.
  • the code value can also be defined by dividing the area within the write area.
  • one coordinate system with the lower left corner as the origin is defined in a predetermined area, and the lower left corner is [Xm0, Ym0] and [Xn0, Yn0] in the two writing areas in this area, respectively.
  • a dot pattern in which a certain XY coordinate value and code values m and n for specifying a writing area are defined is formed.
  • no dot pattern is formed in a range other than the writing area.
  • the area of the writing area is automatically laid out by arranging the mask according to the coordinate system of the block of printed material. It is possible to create a dot pattern block by performing a mask process, and a dot pattern can be easily issued.
  • the XY coordinate value is read to find where the writing area is laid out within the predetermined area. It becomes possible to use for.
  • the dot pattern coordinate system is used for all description, but it is natural that the writing area coordinate system converted from the dot pattern coordinate system is the same.
  • FIG. 90 is a diagram showing a document as a specific example in which a plurality of write areas are defined on a medium.
  • (A) of the figure shows a document in which a specific code for instructing the origin of the XY coordinate values and the writing area is defined in each writing area.
  • the “name” writing area has the “phonetic” writing area, the “address” writing area the “zip code” writing area, and the “neighboring map” writing area.
  • Write areas for “walking (time)”, “train (use time)”, and “bus (use time)” are defined.
  • the “sex” writing area is divided into “male” and “female” writing areas, and the “birth date” writing area is divided into “year”, “month”, and “day” writing areas. .
  • Such a layout of the writing area is possible by assigning different specific codes to each.
  • (B) of the drawing shows that in such a document, the origin of the XY coordinate values is not defined in each writing area, and only a specific code for instructing specification of the writing area is defined.
  • the position of the write information on the entire medium surface can be specified.
  • FIG. 91 is a diagram showing a state in which handwriting input is actually performed on the document of FIG.
  • FIG. 9 is a diagram showing the document of FIG. 90 (a) in which handwriting input is actually performed.
  • (B) in the figure is a diagram showing a state in which the result of the process performed by the information processing unit 211 is received on the display 6 or output by printing in response to the handwritten input in the figure (a).
  • the handwriting input / output system 201 performs output as shown in FIG. 5B by recognizing handwritten input characters and figures by several methods.
  • handwriting input can be performed, the information processing unit 211 can reflect the handwriting input, and handwriting input can be performed again.
  • FIG. 92A is a diagram showing the medium 202 printed by superimposing the trajectory information recognized by the trajectory recognition unit 209 and the dot pattern. Although not shown, a dot pattern different from the dot pattern printed on the medium 202 on which handwriting input was first performed may be newly printed on the medium 202 shown in FIG.
  • FIG. 92 (b) is a diagram showing a medium 202 on which a result of processing performed by the information processing unit 211 is printed with a dot pattern superimposed thereon.
  • a dot pattern different from the dot pattern printed on the medium 202 on which handwriting input is first performed may be newly printed on the medium 202 shown in FIG.
  • the XY coordinates are the same, and only the page code may be changed and printed.
  • the medium 202 may further be provided with a bank guide 38 (narrow convex portion) or Braille 39.
  • FIG. 93 shows a bank guide 38 provided on the outer periphery of the writing area, and a braille 39 protrusion along with a dot pattern in the writing area.
  • the dot pattern analysis unit 208 obtains XY coordinate information and code information defined by the dot pattern by performing code analysis on the image data in accordance with the information input / output method using the dot pattern in the stream dots described above.
  • the dot pattern analysis unit 208 sequentially performs code analysis on the trajectory written (traced) on the medium by the imaging unit 207, the XY coordinate information and the code information are the trajectory. (Trajectory information).
  • the trajectory recognition unit 209 recognizes characters and graphics written on the medium based on changes in trajectory information.
  • the trajectory recognition method is a method that considers characters and figures as graphics, and refers to the XY coordinate patterns of the characters and figures in the database, like a general OCR (optical character reader).
  • the contents to be written in the writing area can be designated in advance by code information. That is, the trajectory information can be recognized with high accuracy by referring to the character and / or graphic information pattern table classified in advance. As a specific example, if it is instructed in advance to describe only Arabic numerals, the recognition rate increases because the trajectory recognition unit 209 refers only to a table of Arabic numerals. If recognition is not possible, all characters and figures in the database can be referenced as usual.
  • the semantic information of a character and / or figure written in the writing area or a phrase composed of one or more characters is defined as predefined semantic information corresponding to the code information.
  • the code table (dictionary)
  • it can be converted into a semantic information code.
  • the trajectory recognition unit 209 first performs pattern recognition one character at a time, “East”, “Kyo”, “Miyako”, “Kyo”, “Ku”.
  • the semantic information code table (dictionary)
  • the semantic information code defined as corresponding to “Tokyo” and the semantic information code defined as corresponding to “Bunkyo-ku” are converted.
  • the information processing unit can recognize that the place names “Tokyo” and “Bunkyo-ku” have been input.
  • semantic information code table (dictionary) also assists character pattern recognition. For example, if the user has written the character “east” of “Tokyo” dirty, and it is difficult for the locus recognition unit 209 to recognize whether the user has written “east” or “car”, the semantic information If the idiom "Tokyo” is defined in the code table and the idiom "Kakkyo” is not defined (the idiom "Kakkyo" usually does not exist), the user wrote the word "east”
  • the trajectory recognition unit 209 can recognize as follows.
  • the handwriting input / output system 201 may further include a timer unit 215.
  • the timer unit 215 has a function of detecting time information.
  • the time information is (1)
  • the order of writing history drawn by the imaging unit 207 touching the writing area surface is used.
  • the handwriting input / output system 201 may further include an angle measurement unit 216 and an angle change recognition unit 217.
  • the angle measurement unit 216 and the angle change recognition unit 217 measure and recognize the inclination of the imaging unit 207 with respect to the medium surface.
  • the above-described method for calculating the pen inclination ⁇ can be used as it is. In this case, it is possible to measure and recognize the tilt even if the imaging unit 207 does not include a pen. Further, the slope calculation method disclosed in the invention of the information output apparatus according to Japanese Patent Application No. 2005-267565 (PCT / SG2006 / 000267) can be used. However, other methods may be used.
  • the angle change with respect to the medium surface of the image capturing unit 207 which is a reference for measuring and recognizing the inclination, is recognized not only by the angle when the image capturing unit 207 is set up vertically with respect to the medium surface, but also by the image capturing unit 207 The angle when touching the surface can be used as a reference.
  • the handwriting input / output system 201 may further include a rotation angle reading unit 218 and a rotation angle change recognition unit 219.
  • the rotation angle reading unit 218 and the rotation angle change recognizing unit 219 read and recognize the rotation angle when the imaging optical axis of the imaging unit 207 is set to be substantially vertical and rotated.
  • the method for obtaining the angle between the direction of the dot pattern and the direction of the imaging unit 207 has already been described in the section of offset correction, but this method can be used for a method of reading and recognizing the rotation angle regardless of the presence or absence of a pen. it can. That is, since the direction of the dot pattern is always constant, the angle ⁇ 1 formed by the direction of the dot pattern and the direction of the image capturing unit 207 when the image capturing unit 207 is set up vertically with respect to the medium surface for the first time is set to a predetermined value. It is possible to read and recognize the difference between the direction of the dot pattern after the elapse of the time and the angle ⁇ 2 (the left rotation is a positive direction) formed by the direction of the imaging unit 207 as the rotation angle.
  • the dot pattern in the writing area can be captured at the moment when the imaging unit 207 touches the writing area.
  • the imaging unit 207 is moved away from the writing area, the irradiation light from the LED 234 is not reflected on the medium surface, and the C-MOS sensor 235 does not detect the irradiation light from the LED 234 (reflection light from the medium surface).
  • the dot pattern cannot be imaged.
  • the dot pattern in the writing area can be picked up at the moment when the image pickup unit 207 touches the writing area.
  • the operations up to are the same as the operation of touching the writing area.
  • the tip of the imaging unit 207 is still directed to the writing area for a predetermined time, so that the irradiation light from the LED 234 is reflected on the medium surface.
  • the sensor 235 temporarily captures the dot pattern on the medium surface.
  • the dot pattern cannot be analyzed blurry as shown in FIG.
  • the image capturing unit 207 touches the writing area to capture a dot pattern, and analyzes the captured image data after a predetermined time (eg, 0.2 seconds later) with the analyzed moment as a reference time,
  • a predetermined time e.g. 0.2 seconds later
  • the imaging unit 207 is preliminarily defined as a touch operation
  • the processing command unit 210 is pre-defined as corresponding to the touch operation with respect to the information processing unit 211 Send processing instructions.
  • the image capturing unit 207 performs a jumping operation, and the processing command unit 210 instructs the information processing unit 211 to Then, a processing command defined in advance as corresponding to the bouncing action is transmitted.
  • a processing command defined in advance as corresponding to the bouncing action is transmitted.
  • FIG. 5E when a dot pattern can be analyzed, it is nothing but a normal handwriting input operation.
  • the processing command unit 210 transmits a processing command based on the information recognized by the trajectory recognition unit 209 to the information processing unit 211 together with the trajectory information.
  • the instruction specifying the page based on the page code, the instruction specifying a certain writing area based on the specific code, and the various instructions based on the action code defined by the dot pattern printed on the medium surface are the processing instruction unit 210. Is actually performed on the information processing unit 211.
  • the processing command unit 210 commands the information processing unit 211 to execute various operations in the handwriting input / output application.
  • predetermined processing instructions based on the measurement / recognition result by the angle measurement unit 216 / angle change recognition unit 217 and the read / recognition result by the rotation angle reading unit 218 / rotation angle change recognition unit 219 are sent to the information processing unit 211. Command against.
  • the information processing unit 211 performs various processes in response to processing instructions performed by the handwriting input / output system 201 according to the present invention.
  • the handwriting input / output system 201 may access the server 5 in which various information is recorded as necessary.
  • the server 5 can store information for locus recognition, for example.
  • information for locus recognition for example.
  • OCR optical character reader
  • XY coordinate patterns are defined and stored in a plurality of characters and figures.
  • meaning information code for outputting a phrase or figure consisting of one or a plurality of characters, or a combination thereof is defined and stored in the characters and figures by referring to the meaning information code table (dictionary). Has been.
  • the writing area can be specified using not only the XY coordinate information but also the code information, it is easily managed even if the amount (number) of the writing area is enormous. can do. This is particularly effective when writing information is stored in a shared global server that anyone can use.
  • the server 5 can store voice, document, image, video, application, and other various information in addition to the information for recognizing the trajectory.
  • a handwriting input / output application can be stored. That is, when the information processing unit 211 does not have a handwriting input / output application, it can search the server and execute the handwriting input / output application on the server. However, the handwriting input / output application on the server may be executed with priority.
  • server 5 can be implemented with various modifications other than the above embodiment.
  • Such an application is a system in which writing on a medium surface is reflected on a computer by executing various operations.
  • the writing on the medium surface may be performed while viewing the display 6 on which the processing result is displayed by instructing the information processing unit 211 in real time by the processing command unit 210 or writing without looking at the display 6. It is assumed that the processing instruction unit 210 may instruct the information processing unit 211 after writing.
  • the writing operation is performed by writing the character or figure to be input into the writing area on the medium surface using the imaging unit 207.
  • the imaging unit 207 captures a dot pattern printed in a superimposed manner at a predetermined time interval while the imaging unit 207 moves on the medium surface. The subsequent steps are as already described.
  • FIG. 95 is an image diagram for explaining a first operation for inputting text characters by handwriting.
  • the image pickup unit 207 When characters are written in the writing area using the image pickup unit 207 as shown in FIG. 11A, the image pickup unit 207 sequentially picks up the dot patterns on the locus as shown in FIG.
  • the unit 208 acquires trajectory information composed of XY coordinate information and code information.
  • the interval between the imaging positions of the dot pattern on the locus is wide in FIG. 5B, but actually it is narrower.
  • the information processing unit 211 executes the first operation and outputs the result to the display 6.
  • the XY coordinate information obtained by the dot pattern analysis unit 208 may be input / output as it is without recognizing characters and figures.
  • the handwriting input / output system 201 according to the present invention functions as a pen tablet.
  • the second operation is an operation of inputting a comment character by handwriting.
  • the comment part is surrounded by a broken line, but it is not necessary to actually write the broken line.
  • the recognition method of characters or figures is the same as the first operation method.
  • the character of the comment input in the second operation is recognized as a comment different from the text in the information processing unit 211 as well.
  • the comment entered between the lines may be reflected in the information processing unit 211 as it is as shown in FIG. 97.
  • FIG. 98 by specifying the part in the text where the comment is to be inserted with an underline, Comments may be listed in the processing unit 211 beside the text.
  • the third operation is an operation for editing a handwritten character.
  • Each editing is performed by writing, using the imaging unit 207, characters or figures that are defined in advance as editing symbols in the information processing unit 211.
  • the recognition method of characters or figures is the same as the first operation method.
  • FIG. 99 is a diagram for explaining editing symbols used in the editing operation which is the third operation of the present invention.
  • a line forming one closed area means an edit symbol for selecting a character or figure in the area.
  • the red dot means an edit symbol that specifies the cut target when cutting and pasting.
  • Double line means delete (erase) edit symbol.
  • “Toru” means an edit symbol for deletion (deletion).
  • ⁇ Zigzag line more than 3 round trips means edit symbol for canceling edit.
  • a bidirectional arrow means an edit symbol that replaces selected characters or figures.
  • Letters (A), (B), (C). . . Means an edit symbol that copies (cuts) a selected character or figure with a flag and designates a paste destination at a distant position.
  • FIG. 100 is a diagram showing the medium 202 before the third operation is executed.
  • the medium 202 is filled with a document input by handwriting by executing the first operation or a document printed as a result of handwriting input.
  • FIG. 101 is a diagram for explaining an editing state on the medium 202 in the third operation.
  • the handwritten input / output application When writing a line that closes the area as shown in FIG. 5A, the handwritten input / output application recognizes the area closed by the line as one editing target.
  • FIG. 102 is a diagram showing a sentence in which the result of each editing in the third operation performed on the medium 202 is reflected in the handwriting input / output application.
  • FIG. 103 is a diagram for explaining an operation of canceling the editing and returning to the state before the editing after the third operation shown in FIG. 101 is edited.
  • FIG. 104 is a diagram showing a sentence in which the result of performing the edit cancellation in the third operation on the medium 202 is reflected in the handwriting input / output application.
  • Each operation may be a parallel operation of the third operation with the first and second operations.
  • the third operation may be a sub-operation subordinate to the first operation and the second operation.
  • the handwriting input / output system 201 can be modified and implemented as necessary in addition to the first to third operations.
  • the processing command unit 210 rotates the image pickup unit 207 rightward by a predetermined rotation angle (for example, 10 degrees or more) while touching the medium surface, so that the processing command unit 210 receives the input character and document structure result (file The information processing unit 211 is instructed to save the content change result.
  • a predetermined rotation angle for example, 10 degrees or more
  • the imaging unit 207 main body has a function capable of recording an editing history
  • the result of the document configuration is transmitted to the information processing unit 211.
  • the processing command unit 210 discards the input character and the result of the document structure by rotating the imaging unit 207 to the left by a predetermined rotation angle (eg, 10 degrees or more) while touching the medium surface. To instruct the information processing unit 211 to cancel the change of the file contents.
  • a predetermined rotation angle eg, 10 degrees or more
  • ⁇ Starting and exiting the application> As shown in FIG. 107, while the image pickup unit 207 is touched on the medium surface, the imaging unit 207 is rotated to the right by a predetermined rotation angle (eg, 10 degrees or more), and the predetermined rotation angle (eg, within 1 second) is directly maintained. (Example: 10 degrees or more) By rotating counterclockwise, the processing command unit 210 commands the information processing unit 211 to start a handwriting input / output application.
  • a predetermined rotation angle eg, 10 degrees or more
  • the image pickup unit 207 is touched on the medium surface and is rotated to the left by a predetermined rotation angle (eg, 10 degrees or more), and is left as it is within a predetermined time (eg, within 1 second). (Example: 10 degrees or more)
  • a predetermined rotation angle eg, 10 degrees or more
  • a predetermined time eg, within 1 second.
  • This method uses the angle measurement / angle change recognition method described above.
  • the image pickup unit 207 is set up on the medium surface, and a predetermined angle (for example, 15 degrees or more) is set on the left side of the medium surface with reference to the angle formed by the medium surface and the image pickup unit 207.
  • a predetermined angle for example, 15 degrees or more
  • the processing instruction unit 210 instructs the information processing unit 211 to start the first operation for inputting the text by handwriting.
  • the image pickup unit 207 is set up on the medium surface, and a predetermined angle (for example, 15 degrees or more) is set on the right side of the medium surface on the basis of the angle formed by the medium surface and the image pickup unit 207.
  • a predetermined angle for example, 15 degrees or more
  • the processing command unit 210 commands the information processing unit 211 to start a second operation for inputting a comment by handwriting.
  • the image pickup unit 207 is set up on the medium surface, and a predetermined angle (for example, 15 degrees or more) is set above the medium surface by using the angle formed by the medium surface and the image pickup unit 207 as a reference.
  • a predetermined angle for example, 15 degrees or more
  • the processing command unit 210 commands the information processing unit 211 to start a third operation for editing the text and comments.
  • the image pickup unit 207 stands on the medium surface, and the lower side of the medium surface by a predetermined angle (for example, 15 degrees or more) on the basis of the angle formed by the medium surface and the image pickup unit 207.
  • the processing command unit 210 instructs the information processing unit 211 to end the currently executed operation and start a preset normal operation (startup) operation.
  • FIG. 110 is a plan view seen from directly above, supplementing the explanation of the first method for switching operations.
  • the operation switching may be performed by an input means such as a button provided on the imaging unit 207 main body.
  • the operation may be switched by an external input unit such as a mouse or a keyboard.
  • the operation may be switched in conjunction with the pen tip switching operation.
  • the information processing unit 211 may erase characters or figures instead of writing.
  • the imaging unit 207 is set up vertically on the medium surface, and once touched on the medium surface.
  • the processing command unit 210 instructs the information processing unit 211 to determine execution of the processing.
  • a processing command instructs the information processing unit 211 to stop the execution of the process.
  • ⁇ Voice recognition> Although the present invention is a handwriting input / output system, speech recognition and voice guidance may be performed as a method for assisting handwriting input.
  • the audio recording unit 220 is configured around a central processing unit (MPU) 240. That is, under the control of the central processing unit (MPU) 240, audio information input from the microphone 241 is processed by the A / D converter 243 and the compression circuit 244 via the amplifier 242 and recorded as digital audio information.
  • MPU central processing unit
  • the voice recognition unit 221 converts the recorded voice information into corresponding text information, and the processing command unit 210 transmits a processing command based on the text information to the information processing unit 211 together with the text information.
  • the accuracy of voice recognition is improved by referring to the text information in the speech recognition information table corresponding to the classification code without referring to all the text information when searching the database and outputting the text information.
  • ⁇ Trace specific symbols> As a first method of switching to the sound recording mode, there is a method of switching to the sound recording mode by the imaging unit 207 writing (tracing or drawing) a specific symbol that means switching to the sound recording mode.
  • the trajectory recognition unit 209 recognizes the trajectory of a specific symbol that means switching to the voice recording mode and transmits it to the voice recording unit 220 (or by transmitting it via the processing command unit 210), thereby recording the voice.
  • the unit 220 starts audio recording.
  • ⁇ Switching by voice recognition> there is a method of recording a predetermined voice in the voice recording unit 220 and switching to the voice recording mode when the voice recognition unit 221 recognizes the predetermined voice.
  • the processing command unit 210 commands the information processing unit 211 to perform voice input instead of handwriting input in the handwriting input / output application when an operation by the above-described various methods is performed.
  • the audio output unit 22 reproduces a predetermined audio file in response to an instruction from the processing command unit 210.
  • the audio file that has been digitized and stored in advance is output from the speaker 245 or the earphone 246 via the A / D converter 42 and the amplifier 242.
  • voice guidance may be given to a user who performs handwriting input by using code information.
  • a voice guide code that instructs the information processing unit 211 to reproduce voice is defined in the dot pattern printed in the writing area, so that handwriting input is started (the imaging unit 207 changes the dot pattern).
  • the sound output unit 22 reproduces the sound file stored in association with the sound guide code in advance.
  • the audio guide code and the corresponding audio file may be used for purposes other than the audio guide.
  • the medium can be pasted on the monitor surface of the display 6 to form a touch panel.
  • means other than the grid sheet may be adopted.
  • the grid sheet may be used by being superimposed on the medium 202.
  • Such a usage method is such that when a document or image printed (described) on a medium that is not for handwriting input / output is first edited using the handwriting input / output system 201 according to the present invention, the image pickup unit is used. This is particularly effective when the user feels resistance to pressing 207 on the monitor surface.
  • the writing area is rectangular, but when the writing area is specified using the code information as in the present invention, the shape of the writing area is, for example, a polygon as shown in FIG. , Circular, elliptical, heart-shaped, star-shaped, or when an illustration and a dot pattern are overprinted, the same analysis efficiency can be achieved regardless of the shape along the actual line of the illustration. be able to.
  • the information dots are placed on the + direction line (up and down Information can be defined for the combination of information dot arrangement directions (second digitized information), which is arranged on the horizontal direction line) or on the x direction line (on the diagonal direction line).
  • second digitized information which is arranged on the horizontal direction line
  • the x direction line on the diagonal direction line.
  • the distance (arrangement distance) at which the information dots are arranged from the virtual reference point is set to two or more patterns so that the information dots are arranged at a distance from the virtual reference point.
  • the second digitized information can be defined also by the combination.
  • the second digitized information may be defined by a combination of both the information dot arrangement direction and the arrangement distance.
  • the second digitized information means information defined by the information dot arrangement rule itself, such as the information dot arrangement direction and arrangement distance.
  • the inventor has so far proposed a dot pattern format that defines both code information and coordinate information.
  • the dot pattern in such a format increases the dot pattern distribution area. Accordingly, the optical reading unit of the pen scanner needs to have a large reading area in order to read the dot pattern.
  • the present invention can be used as a touch panel by sticking on a screen of any display such as a personal computer, PDA, bank ATM or the like. Moreover, by covering the printed material and outputting information related to the information described in the printed material on the screen, it can be used for a dictionary, a catalog of mail order, a learning material, and the like.
  • An information input device includes a personal computer, a PDA, a television receiver, and a front projector that can easily input text, diagrams, illustrations, photographs, etc. printed on a print medium or displayed on a screen.
  • Rear projectors game devices, karaoke devices, mobile phone terminal devices, POS terminal devices, ATMs, KIOSK terminals, car navigation systems, pachinko machines, pachislot machines, watches, smartphones, and the like.
  • the information processing apparatus is a personal computer, PDA, and television receiver that can easily retrieve related information such as text, diagrams, illustrations, and photos that are printed on a print medium or displayed on a screen. It can be used as a device, a front projector, a rear projector, a game device, a karaoke device, a mobile phone terminal device, a POS terminal device, an ATM, a KIOSK terminal, a car navigation system, a pachinko machine, a pachislot machine, a watch, a smartphone, and the like.
  • the handwriting input / output system has the same feeling as that a person who has not mastered the technique of using an existing input device such as a mouse, a keyboard, and a tablet used for computer operation writes on a paper medium.
  • the present invention has industrial applicability as an input device that can be used in a computer. Furthermore, it has industrial applicability as toys, especially educational toys, mainly for children.
  • the handwriting input / output system can specify the writing area by the code information read simultaneously with the coordinate information, if the content input by handwriting belongs to a predetermined category, the dictionary of the category The recognition rate is greatly improved by collating with.
  • the handwriting input / output system determines whether or not it is a writing area simply by reading the dot pattern without any special operation by the operator, and a writing instrument that everyone is familiar with is used. Intuitive handwriting input as you use it.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Position Input By Displaying (AREA)
  • Character Discrimination (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

La feuille auxiliaire d'entrée d'informations à faible coût et simple et le système de traitement d'informations de l'invention présentant une précision d'entrée et un rendement d'entrée extrêmement élevés peuvent être utilisés avec n'importe quel élément d'équipement, et utiliser des motifs de points de flux pour un grand nombre de commandes de sortie et/ou d'opération d'informations multimédias, lesdits motifs de points de flux étant agencés par : une étape à laquelle une pluralité de points de référence sont placés continûment et linéairement en suivant des règles prédéterminées ; une étape à laquelle une première ligne de référence virtuelle est créée comprenant une droite, une ligne polygonale et/ou une ligne incurvée reliant la pluralité de points de référence ; une étape à laquelle au moins une deuxième ligne de référence virtuelle est créée comprenant une droite et/ou une ligne incurvée, et définie à une position prédéterminée à partir des points de référence et/ou de la première ligne de référence virtuelle ; une étape à laquelle une pluralité de points de référence virtuels sont créés à des positions prédéterminées sur la deuxième ligne de référence virtuelle ; et une étape à laquelle un point d'informations ayant des valeurs de coordonnées XY et/ou une valeur de code définies par la distance et la direction du point par rapport à un point de référence virtuel est agencé sur le point d'extrémité d'un vecteur ayant un point initial au point de référence virtuel. L'invention concerne en outre un dispositif d'entrée d'informations à faible coût et simple utilisant les motifs de points de flux pour permettre à l'utilisateur d'entrer facilement des informations d'un contenu dans lequel des liens ou similaire ne sont pas fixés à l'avance, et un système d'entrée/sortie manuscrites utilisant les motifs de points de flux pour permettre l'entrée de caractères, de symboles et similaire par une écriture manuscrite.
PCT/JP2011/065914 2010-07-12 2011-07-12 Feuille auxiliaire d'entrée d'informations, système de traitement d'informations, dispositif d'entrée d'informations, système d'entrée d'informations, feuille d'entrée manuscrite, et système d'entrée/sortie manuscrites utilisant des points de flux WO2012008469A1 (fr)

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
JP2010-158321 2010-07-12
JP2010158344A JP2012022418A (ja) 2010-07-12 2010-07-12 ストリームドットを用いた手書き入力シートおよび手書き入出力システム
JP2010158120A JP2012022400A (ja) 2010-07-12 2010-07-12 ストリームドットを用いた情報入力補助シートおよび情報処理システム
JP2010-158120 2010-07-12
JP2010158321A JP2012022415A (ja) 2010-07-12 2010-07-12 ストリームドットを用いた情報入力装置および情報入力システム
JP2010-158344 2010-07-12

Publications (1)

Publication Number Publication Date
WO2012008469A1 true WO2012008469A1 (fr) 2012-01-19

Family

ID=45469466

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2011/065914 WO2012008469A1 (fr) 2010-07-12 2011-07-12 Feuille auxiliaire d'entrée d'informations, système de traitement d'informations, dispositif d'entrée d'informations, système d'entrée d'informations, feuille d'entrée manuscrite, et système d'entrée/sortie manuscrites utilisant des points de flux

Country Status (1)

Country Link
WO (1) WO2012008469A1 (fr)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2015041362A1 (fr) * 2013-09-20 2015-03-26 グリッドマーク株式会社 Feuille d'aide à la saisie d'informations, dispositif de lecture optique de code de points utilisant ladite feuille, système de traitement d'informations de code de points, et procédé d'étalonnage
CN108604234A (zh) * 2015-07-15 2018-09-28 查比公司 用于屏幕截图链接的系统及方法
CN112511883A (zh) * 2020-12-09 2021-03-16 广东长虹电子有限公司 一种具有手写输入功能的遥控器、电视系统及控制方法
CN112991447A (zh) * 2021-03-16 2021-06-18 华东理工大学 一种动态环境下视觉定位与静态地图构建方法及系统

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006190270A (ja) * 2002-09-26 2006-07-20 Kenji Yoshida 媒体上に形成されたアイコン
JP2009163354A (ja) * 2007-12-28 2009-07-23 Kenji Yoshida 情報記録のための媒体、情報読み取り装置、情報入力システム
JP3156526U (ja) * 2009-10-21 2010-01-07 大日本印刷株式会社 情報処理システム

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006190270A (ja) * 2002-09-26 2006-07-20 Kenji Yoshida 媒体上に形成されたアイコン
JP2009163354A (ja) * 2007-12-28 2009-07-23 Kenji Yoshida 情報記録のための媒体、情報読み取り装置、情報入力システム
JP3156526U (ja) * 2009-10-21 2010-01-07 大日本印刷株式会社 情報処理システム

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2015041362A1 (fr) * 2013-09-20 2015-03-26 グリッドマーク株式会社 Feuille d'aide à la saisie d'informations, dispositif de lecture optique de code de points utilisant ladite feuille, système de traitement d'informations de code de points, et procédé d'étalonnage
CN108604234A (zh) * 2015-07-15 2018-09-28 查比公司 用于屏幕截图链接的系统及方法
CN112511883A (zh) * 2020-12-09 2021-03-16 广东长虹电子有限公司 一种具有手写输入功能的遥控器、电视系统及控制方法
CN112991447A (zh) * 2021-03-16 2021-06-18 华东理工大学 一种动态环境下视觉定位与静态地图构建方法及系统
CN112991447B (zh) * 2021-03-16 2024-04-05 华东理工大学 一种动态环境下视觉定位与静态地图构建方法及系统

Similar Documents

Publication Publication Date Title
JP4385169B1 (ja) 手書き入出力システム、手書き入力シート、情報入力システム、情報入力補助シート
JP6366636B2 (ja) 情報入力システム
JP4243642B2 (ja) キャリブレーション方法および情報入力補助シート
KR20110099068A (ko) 캘리브레이션 방법 및 이에 이용되는 정보 입력 보조 시트
WO2012008469A1 (fr) Feuille auxiliaire d'entrée d'informations, système de traitement d'informations, dispositif d'entrée d'informations, système d'entrée d'informations, feuille d'entrée manuscrite, et système d'entrée/sortie manuscrites utilisant des points de flux
JP5848230B2 (ja) 手書き入出力システム、手書き入力シート、情報入力システム、情報入力補助シート
JP6056263B2 (ja) 端末装置、電子ペンシステム及びプログラム
JP5544609B2 (ja) 手書き入出力システム
JP2012022418A (ja) ストリームドットを用いた手書き入力シートおよび手書き入出力システム
JP5358759B2 (ja) 光学読み取り装置
JP6857867B2 (ja) 手書き入力シート
JP6267074B2 (ja) 手書き入出力システムおよび光学読取装置
JP2014220006A (ja) 手書き入出力システムおよび光学読取装置
JP2019036324A (ja) 手書き入力シート
JP2012022400A (ja) ストリームドットを用いた情報入力補助シートおよび情報処理システム
JP2014157621A (ja) 手書き入出力システム
JP2012022415A (ja) ストリームドットを用いた情報入力装置および情報入力システム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 11806799

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 11806799

Country of ref document: EP

Kind code of ref document: A1