Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
  • H04N1/0035—User-machine interface; Control console
  • H04N1/00352—Input means
  • H04N1/00392—Other manual input means, e.g. digitisers or writing tablets
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
  • G06Q10/00—Administration; Management
  • G06Q10/10—Office automation; Time management
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
  • H04N1/32—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
  • H04N1/32101—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
  • H04N1/32106—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title separate from the image data, e.g. in a different computer file
  • H04N1/32112—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title separate from the image data, e.g. in a different computer file in a separate computer file, document page or paper sheet, e.g. a fax cover sheet
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
  • H04N1/32—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
  • H04N1/32101—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
  • H04N1/32128—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title attached to the image data, e.g. file header, transmitted message header, information on the same page or in the same computer file as the image
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N2201/00—Indexing scheme relating to scanning, transmission or reproduction of documents or the like, and to details thereof
  • H04N2201/0077—Types of the still picture apparatus
  • H04N2201/0084—Digital still camera
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N2201/00—Indexing scheme relating to scanning, transmission or reproduction of documents or the like, and to details thereof
  • H04N2201/32—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
  • H04N2201/3201—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
  • H04N2201/3225—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title of data relating to an image, a page or a document
  • H04N2201/3249—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title of data relating to an image, a page or a document data relating to a linked page or object, e.g. hyperlink
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N2201/00—Indexing scheme relating to scanning, transmission or reproduction of documents or the like, and to details thereof
  • H04N2201/32—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
  • H04N2201/3201—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
  • H04N2201/3225—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title of data relating to an image, a page or a document
  • H04N2201/3253—Position information, e.g. geographical position at time of capture, GPS data
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N2201/00—Indexing scheme relating to scanning, transmission or reproduction of documents or the like, and to details thereof
  • H04N2201/32—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
  • H04N2201/3201—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
  • H04N2201/3261—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title of multimedia information, e.g. a sound signal
  • H04N2201/3264—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title of multimedia information, e.g. a sound signal of sound signals
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N2201/00—Indexing scheme relating to scanning, transmission or reproduction of documents or the like, and to details thereof
  • H04N2201/32—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
  • H04N2201/3201—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
  • H04N2201/3261—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title of multimedia information, e.g. a sound signal
  • H04N2201/3266—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title of multimedia information, e.g. a sound signal of text or character information, e.g. text accompanying an image
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N2201/00—Indexing scheme relating to scanning, transmission or reproduction of documents or the like, and to details thereof
  • H04N2201/32—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
  • H04N2201/3201—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
  • H04N2201/3261—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title of multimedia information, e.g. a sound signal
  • H04N2201/3267—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title of multimedia information, e.g. a sound signal of motion picture signals, e.g. video clip
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N2201/00—Indexing scheme relating to scanning, transmission or reproduction of documents or the like, and to details thereof
  • H04N2201/32—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
  • H04N2201/3201—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
  • H04N2201/3269—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title of machine readable codes or marks, e.g. bar codes or glyphs

Definitions

• a digital paper system includes an input device such as a touch board, a microphone, a keyboard, a digital ink tablet, and a projector, a writing surface and a writing utensil in which the utensil knows its location in real time when it is marking on the surface.
• the writing surface may take the form of a digital tablet or digital paper, for example digital paper made by the Anoto Group AB and having an ANOTO® pattern.
• Various types of conventional digital pens include, but are not limited to, the MAXELL® digital pen, the NOKIA® digital pen, the LEAPFROG FLYFUSION® digital pen, the ANOTO® digital pen, the LIVESCRIBE® PULSE smart pen, and the LOGITECH® digital pen.
• a human operator can issue instructions and provide information via microphone and keyboard. The operator can receive information via the audio speaker.
• a projector projects images on the map that are visible to the operator.
• a method for multimedia object association to digital paper spatial diagrams includes the steps of (1) capturing a media object to be imported into a spatial information system; (2) linking the captured media object to a digital paper document containing a spatial diagram, using a input device configured to select a linking action area located in a first area of the digital paper document and configured to select a location for the captured multimedia data object on a second area of the digital paper document; (3) transferring the media object and the data from the input device to a spatial information system; and (4) combining the media object and the data of the input device to create an updated digital paper document containing the media object.
• FIGURE 1 shows an example method for multimedia object association to digital paper spatial diagrams
• FIGURE 2 shows an example system for multimedia object association to digital paper spatial diagrams.
• the present invention is generally directed to linking data from a multimedia capture device such as a photos, videos, audio, and tags to a digital paper- enabled product for digital data collection using digital paper systems, such as maps, blueprints, diagrams, CAD drawings, and other documents having spatially relevant information printed or applied thereon.
• the digital paper-enabled product allows a user to create, import, edit, share, and act on paper-based data in and between spatial information systems.
• the plurality of media objects are associated with particular locations in a spatial information system. For example a note written with a digital pen within a digital paper enabled notebook is linked with a particular location on a digital- enabled product containing a map, sometimes called a spatial diagram.
• At least one embodiment of the present invention relates generally to linking a media object with a digital paper-enabled product having digital paper with a digital pattern that spatially cooperates with one or more features or boundaries printed on the pattern, and more specifically to spatial information systems that communicate with a writing instrument previously used to effect changes to the digital paper-enabled product and communicate with a multimedia device used to capture a media object.
• the spatial information system further combines the media object with the selected location within the digital paper-enabled document.
• the digital paper-enabled product may take the form of a sheet of digital paper bearing a digital pattern and where features (e.g., such as map features) or a spatial reference frame printed onto the digital pattern and where at least one action palette is printed onto the digital pattern, correlated therewith, and usable to effect changes to the features and to link media objects with the digital paper-enabled product.
• the at least one action palette may be printed onto a separate digital paper-enabled product having a digital pattern.
• a user may take a picture of an area shown on the digital paper-enabled product, then the user may use a writing instrument or indication device, such as a digital pen, to link the picture to the document through executable actions produced by interaction of the writing instrument with the digital paper-enabled product.
• a writing instrument or indication device such as a digital pen
• a method of processing information obtained from a digital paper-enabled product and capture device includes (1) receiving information with a input device that indicated a link of at least one media object to the digital paper-enabled product, the at least one media object correlated to a location on the digital paper; (2) placing the input device in communication with a spatial information system configured to manage the at least one media object link; (3) transferring the information from the input device to the spatial information system to update the linked media object within the spatial information system; (4) capturing the linked media object from a capture device; and (5) synchronizing the spatial information system to generate an updated version of the digital paper-enabled product.
• a method of using a digital paper-enabled product when de-coupled from a spatial information system includes the steps of (1) accessing the digital paper-enabled product having a sheet of digital paper bearing a digital pattern and having at least one identifiable region spatially correlated to the digital pattern with a coordinate system; (2) indicating on the product linking action causing a media object to be linked to a spatial information system; and (3) recording the linking action with an indicating device.
• FIGURE 1 shows an example method 100 for multimedia object association to digital paper spatial diagrams.
• a link is created in a spatial diagram to a media object.
• the link refers to an indication of the desire to connect or anchor a media object such as a note, photo, video, or (barcode) tag to a specific location of a spatial diagram, or to a specific object (for example a wall or a window on an architectural drawing) represented within the diagram.
• linking is activated by an explicit activation of an interface action, for example by ticking a checkbox in an action palette, such as "Add note", "Add photo”, “Add Media Link”, followed by a selection gesture indicating the location in the special diagram.
• the selection may consist of any number of strokes in the location onto which the media object is to be linked to, or a selection gesture (checkmark, lasso) on a specific object (such as a wall or window) to which the media object is linked to.
• linking is activated by sketching a symbol representing a link on the region of the media object is to be linked to or on a specific object to which the media object is to be linked to.
• the symbol may include identifiers (such as numbers or names) that are used during the association of media objects to links; identifiers are described in more detail in reference to block 120.
• linking is activated by leveraging locational information that may be embedded in the media object.
• a media object is associated to a link within the spatial diagram.
• Association refers to the means by which specific media objects are connected to individual links.
• One or more media objects may be associated with any link.
• only a subset of the media objects that have been collected may be associated to drawing links. For example pages of notes, some of which, are associated with locations in a spatial diagrams others are not.
• associations can be established by an explicit selection of a user interface action on the media object, for example by ticking a checkbox on a note page indicating it is to be associated with a link, or selecting an association interface option on a capture device such as a camera or similar device.
• associations can be established by using timestamps to associate media objects captured on the temporal vicinity of a linking action, for example by associating with a link those objects whose timestamps fall within plus/minus seconds of the linking action.
• associations can be established by using identifiers, such as photo or page numbers that are matched to identifiers specified in the linking symbol.
• associations can be established by location information that may be embedded in the media object, for example GPS location embedded in a photo. The association in this case preferably corresponds to the automatically created link location.
• linking may occur using a gesture, a voice command, or a multi-modal instruction.
• the media object is visualized within an electronic representation of a spatial diagram. For example after a user has retrieved the media object from the media device into a computer a user can see the media object on the spatial diagram.
• linked and/or associated media objects may be presented on an electronic representation of a spatial diagram as icons or thumbnails.
• a selection action causes the associated media object to be displayed.
• an electronic representation of a media object such as the note or photo
• a user interface action causes the spatial diagram location to which the media object is associated to be displayed.
• media object links and associations are corrected.
• media object links are corrected on an electronic representation of a spatial diagram.
• the corrections include, but are not limited to deleting a link, inserting a new link, changing the association between links and media objects. This review may take place within an electronic editing environment or in some embodiments can be performed with a digital pen on digital paper printouts of the spatial diagram and associated media object representations such as printout of note pages, photos or representative images of movies.
• media objects are managed.
• Media objects associated to links in a spatial diagram may be stored in separate memory areas, in which case the correlation between object and diagram is represented as connection meta-data, or the media objects may be inserted into the same file as the spatial diagram.
• a media object that is inserted into the same file as the spatial diagram is self contained and can be conveniently transported via email, or other electronic means as a unit.
• object management includes retrieving from the capture devices such as a digital pen or a camera only those objects that have been associated to links to one or more spatial diagrams. For example, photos that are not associated to any link remain in the capture device; similarly, only the electronic ink pertaining to note pages that have been associated to links is removed from the digital pen, and others remain. In one embodiment removal is achieved either by selective removal, when the capture device supports this function or by fully removing and then reinserting those objects that have not been associated with links.
• FIGURE 2 shows an example system for multimedia object association to digital paper spatial diagrams.
• a spatial information system 210 takes the form of computational processing device, such as a computing device, which may be, but is not limited to, a personal, mainframe, handheld, or other type of computer with sufficient memory and computing capacity to process the data.
• the data may digitally define features, icons, graphical elements, symbols, characters, indicia, mathematical or computational elements, or any other feature (hereinafter generally referred to as feature or features) that may be used to represent some portion of a map, blueprint, drawing, photo, image, diagram, or other document.
• the spatial information system is in communication with a spatial diagram 220 printed on digital paper through a digital pen (not shown).
• the spatial information system 210 is in data communication with a plurality of capture devices containing media objects, such but not limited to as a note 230 written on digital paper and transmitted to the spatial information system 210 using a digital pen and a camera 240.
• a linking action in the spatial diagram 220 correlated with an associating action in at least one of a note 230 or a photo from a camera 240 results in a media object being linked to the spatial diagram 220 in the spatial information system 210.
• a capture device such as a camera 240 takes a picture of a wall shown in the spatial diagram 220.
• a user selects an icon that associates the picture of the wall to the spatial diagram 220.
• the user selects a link icon within the spatial diagram 220 and then selects the wall in the spatial diagram 220 using a lasso motion.
• the user then goes to a spatial information system 210 where when the digital pen and camera 240 are linked to the system 210 the photo of the wall is placed as an icon at the location of the wall in the spatial diagram. Therefore allowing a user to see the wall as represented by the diagram 220 and by selecting the object sees the photograph of the wall.
• this link allows the photo of the wall to be directly linked to the location on the diagram therefore advantageously providing organization and more detail to the diagram 220.

Landscapes

• Engineering & Computer Science (AREA)
• General Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Business, Economics & Management (AREA)
• Strategic Management (AREA)
• Entrepreneurship & Innovation (AREA)
• Human Resources & Organizations (AREA)
• Quality & Reliability (AREA)
• Economics (AREA)
• Marketing (AREA)
• Operations Research (AREA)
• Data Mining & Analysis (AREA)
• Tourism & Hospitality (AREA)
• Physics & Mathematics (AREA)
• General Business, Economics & Management (AREA)
• General Physics & Mathematics (AREA)
• Theoretical Computer Science (AREA)
• User Interface Of Digital Computer (AREA)

Applications Claiming Priority (6)

Publications (2)

Family

ID=40094394

Family Applications (1)

Country Status (1)

Citations (2)

• 2008
  • 2008-06-02 EP EP08769992A patent/EP2168055A4/de not_active Withdrawn

Patent Citations (2)

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events