TW201407533A - Enhancing captured data - Google Patents

Abstract

Captured data is obtained, including various types of captured or recorded data (e.g., image data, audio data, video data, etc.) and/or metadata describing various aspects of the capture device and/or the manner in which the data is captured. One or more elements of the captured data that can be replaced by one or more substitute elements are determined, the replaceable elements are removed from the captured data, and links to the substitute elements are associated with the captured data. Links to additional elements to enhance the captured data are also associated with the captured data. Enhanced content can subsequently be constructed based on the captured data as well as the links to the substitute elements and additional elements.

Description

Enhance the information that has been captured

The present invention relates to enhancing the information that has been learned.

Digital cameras are now found in many different types of devices, including dedicated digital cameras, cellular phones, computers, game consoles, and more. This wide availability of digital cameras allows users to take large numbers of digital photos, but the problem persists. One such problem is that current digital cameras are typically simple image capture devices with basic functionality. This situation is restrictive for the user because the user can only retrieve digital photos that are snapshots in a particular scene.

This [invention] is provided to introduce some concepts in a simplified form, which are further described in the following [Embodiment]. This Summary is not intended to identify key features or basic features of the claimed subject matter, and is not intended to limit the scope of the claimed subject matter.

Obtained data on the environment based on one or more aspects. One or more additional elements are determined based, at least in part, on the retrieved material, and a link to one or more additional elements related to the retrieved material is added. Empowering the use of one or more additional elements and at least a portion of the extracted material to enhance the construction of the content.

Depending on one or more aspects, it is determined whether the photographic element of the photo is replaceable, including identifying a replaceable photographic element and one or more substitute photographic elements. The replaceable photographic element is removed from the photo and one or more links to replace the photographic element are added to produce a compressed photo. Links to additional elements associated with alternative photographic elements or alternative photographic elements are also added. The use of a compressed photo and one or more alternative photo elements and additional element link enhancements enhances the construction of the photo.

100‧‧‧Model System

102‧‧‧Selection device

104‧‧‧Playback device

106‧‧‧Crowd Outsourcing Information Service

108‧‧‧Network

200‧‧‧Exemplary Data Enhancement System

202‧‧‧Environment capture module

204‧‧‧Link Insertion Module

206‧‧‧Crowd Outsourcing Data Module

208‧‧‧Playback module

210‧‧‧Data repository

212‧‧‧Retrieved information

214‧‧‧Link information

216‧‧‧Promotional information

218‧‧‧ has been enhanced

300‧‧‧Executive program

302‧‧‧ action

304‧‧‧ action

306‧‧‧ Action

308‧‧‧ action

400‧‧‧Executive program

402‧‧‧ action

404‧‧‧ action

406‧‧‧ action

408‧‧‧ action

410‧‧‧ action

412‧‧‧ action

414‧‧‧ action

500‧‧‧Executive Computing Devices

502‧‧‧Processing system

504‧‧‧ Computer readable media

506‧‧‧I/O interface

508‧‧‧ hardware components

510‧‧‧Memory/storage

520‧‧‧ Data Acquisition System

The use of the same element symbols in all figures indicates similar features.

FIG. 1 illustrates an exemplary system for implementing enhanced data retrieval in accordance with one or more embodiments.

FIG. 2 illustrates an exemplary data enhancement system in accordance with one or more embodiments.

Figure 3 is a flow diagram illustrating an exemplary program for enhancing retrieved data in accordance with one or more embodiments.

4 is a flow chart illustrating another exemplary program for enhancing captured data in accordance with one or more embodiments.

FIG. 5 illustrates an exemplary computing device that can be configured to implement enhanced data retrieval in accordance with one or more embodiments.

This article discusses the promotion of data that has been extracted. Obtaining the retrieved information, which may include various types of recorded data (eg, image data, audio data, video material, etc.) and/or descriptions of various aspects of the capture device and/or manner of recording the data Meta data. Decide on one or more alternative elements Replace one or more elements of the recorded material. The replaceable element is removed from the logged data, and the link to the alternate element is associated with the retrieved material. Links to additional elements that have been extracted are also associated with the retrieved information. The promotional content can then be constructed based on the recorded material and the links of the alternate elements and additional elements.

FIG. 1 illustrates an exemplary system 100 for implementing enhanced data retrieval in accordance with one or more embodiments. System 100 includes one or more capture devices 102 that retrieve and provide data to one or more playback devices 104. The playback device 104 communicates with a crowd sourcing data service 106 via the network 108. Network 108 can be a variety of different networks, including the Internet, a local area network (LAN), a public telephone network, an internal network, other public and/or proprietary networks, combinations thereof, and the like. The one or more capture devices 102 may also communicate with the crowdsourcing data service 106 and/or one or more playback devices 104 via the network 108 as appropriate. Although illustrated as a separate device, it should be noted that the particular device may be both the capture device 102 and the playback device 104.

The capture device 102 retrieves information about a particular environment, which is also referred to as the environment associated with the captured material. The environment in which data is retrieved or directed to the environment means that the environment surrounding the device 102 is captured when the information is retrieved. The environment can be, for example, the interior, exterior, concert, sporting event, party, etc. of the building. Each capture device 102 can be any of a variety of different types of devices capable of capturing information about the environment, such as a camera or camcorder, a tablet or notepad computer, a cellular phone or other wireless telephone, a game console, Car computers, exclusive data capture devices (providing minimal additional functionality in addition to data). Different picking devices in the capturing device 102 may be the same or not The same type of device.

Extracting information means recording information about the environment and/or metadata. Recording information about the environment means recording or sensing the characteristics of the environment itself. Any or more of various types of information about the environment (such as still image data, video data, audio data, combinations of the above, etc.) may be recorded. For example, the capture device 102 can record still images (eg, photos) of a particular environment, audio sensed in a particular environment, and the like.

Recording metadata about the environment means recording metadata describing various aspects of the capture device 102, the manner in which the data was retrieved, and/or other aspects of the environment. This metadata may include the point of view or direction of the device when recording information about the environment (eg, determined by the compass or other directional component of the capture device). This metadata may also include the geographical location of the device (and therefore also the environment) when recording information about the environment (such as the geographic location determined by the Global Navigation Satellite System (GNSS) or other positioning component of the capture device). ). This metadata may also include the date and/or time of recording information about the environment. In the event that the retrieval device 102 records the metadata while the device 102 records the information about the environment (or within a threshold amount of time for the device 102 to record information about the environment), the metadata is also considered to be associated with the recorded data.

In yet another embodiment, this operation is performed only after receiving the user's consent to record the geographic location of the capture device 102. This user agrees to opt-in consent, where the user takes a positive action and requires a geographic location. Alternatively, the user agrees to opt-out consent, wherein the user takes a positive action and asks not to record the geographic location. If the user does not select the opt-out of this location record, the user implicitly agrees to the record Location. Private statements can also be displayed to the user to explain to the user how to maintain the confidentiality of the recorded location. In addition, it should be noted that the record of the geographic location of the device does not have to and typically does not include any personal information identifying a particular user. Thus, although the geographic location of a particular user can be recorded, no indication of the particular user is recorded.

In one or more embodiments, the capture device 102 records metadata describing various aspects of the capture device, the manner in which the data is retrieved, and/or other aspects of the environment, but does not capture or record information about the environment. other information. For example, the capture device 102 can record the viewpoint or direction of the capture device and the geographic location of the capture device (and optionally the date and/or time of recording the metadata), but does not record any still images of the environment. , video and / or audio. Thus, the capture device 102 can simply record only the location of the capture device that can be used to enhance the retrieved data technology as discussed below (and record the viewpoint or direction, date and/or time, etc. as appropriate).

The playback device 104 is a device that plays back the captured data enhanced using the techniques discussed herein. The extracted data can be enhanced by replacing the elements or parts of the data with alternative elements or partial links. The retrieved material can also be enhanced by adding a link to one or more additional elements. Adding a link with an alternate element or portion instead of an element or portion and/or adding one or more additional elements or allowing for enhanced content based on the retrieved material construct. These techniques for enhancing captured data are discussed in additional detail below.

Each playback device 104 can be a variety of different types of devices, such as a desktop computer, a server computer, a laptop computer or a laptop computer, a tablet or a notepad computer, a mobile station, an entertainment appliance, and a communication device. Coupled to A set-top box of a display device, a television or other display device, a cellular phone or other wireless telephone, a game console, a car computer, and the like. Thus, the playback device 104 can range from a full resource device (eg, a personal computer, game console) with a large amount of memory and processor resources to a low resource device with limited memory and/or processing resources (eg, a traditional set top box) ,TV). Different playback devices 104 may be the same and/or different types of devices.

The capture device 102 provides the retrieved data and the recorded metadata to one or more playback devices 104. Alternatively, the retrieved data can be played back on the capture device, in which case the capture device 102 is also the playback device 104. The capture device 102 can be removable in various ways, such as via the network 108, via another link or connection (eg, a wired connection or a wireless connection, such as a universal serial bus (USB) or wireless USB connection) The memory device (eg, a flash memory device removed from the capture device 102 and inserted into the playback device 104) or other storage device, etc.) provides the retrieved data and recorded metadata to the playback device.

The mass outsourcing information service 106 maintains information used to enhance the data that has been retrieved. In one or more embodiments, the data retrieved by the capture device 102 is provided to the crowdsourcing data service 106. Additionally or alternatively, the information used to enhance the retrieved material may be provided to the service 106 from other sources. For example, a collection of materials or a repository may be provided to the service 106 from a variety of sources. The service 106 is referred to as a crowdsourcing service because the service 106 relies on receiving data from multiple sources to enhance the retrieved material rather than receiving it from a single source.

The crowdsourcing service 106 is implemented using one or more of a variety of different types of devices. For example, as described above with respect to playback device 104 Any of the various types of devices discussed implements service 106. Service 106 may be implemented using one or more of the same and/or different types of devices.

FIG. 2 illustrates an exemplary data enhancement system 200 in accordance with one or more embodiments. The system 200 includes an environment capture module 202, a link insertion module 204, a crowdsourcing data module 206, and a playback module 208. Although a particular module is illustrated in FIG. 2, it should be noted that additional modules may be included in the data enhancement system 200. In addition, it should be noted that the functions of the plurality of modules illustrated in the data enhancement system 200 can be combined into a single module, and/or the functions of one or more of the modules illustrated in the data enhancement system 200 can be separated. To multiple modules.

Module 202 to module 208 can be implemented by one or more devices. In one or more embodiments, module 202 to module 208 are implemented by different devices. Alternatively, two or more of modules 202 through 208 may be implemented at least partially in the same device.

The environment capture module 202 is included in a capture device (such as the capture device 102 of FIG. 1) and retrieves information about the environment. The retrieved data 212 can be retrieved as discussed above and provided to the link insertion module 204, which can be various types of data and/or metadata about the environment. The link insertion module 204 can be included in the same or a different device than the environment capture module 202. The capture module 202 can provide the retrieved data 212 to the link insertion module 204 in various manners, such as including the retrieved data 212 as a parameter and accessible in the module 204 when the interface of the module 204 is invoked. The saved location 212 is saved in the location, an email is sent, or other messaging protocols are used to communicate the retrieved data 212 to the module 204 and the like.

The link insertion module 204 identifies that it can be stored from the data repository. The elements of the other material in 210 replace the elements of the data 212. For the elements of the retrieved data 212 (also referred to as replaceable elements) identified by the module 204 that can be replaced by elements from other materials, the link insertion module 204 removes the elements that can be replaced from the retrieved data 212. And adding a link to the alternate element that replaces the identifying element (associated with the retrieved material 212). Different types of elements may be identified based on the type of data that has been retrieved, such as if the captured data 212 includes photographic material or image data, and is identified as a photographic element; if the captured data 212 includes audio data, it is identified as an audio element.

The data repository 210 maintains data for enhancing the retrieved material, and the data repository 210 can be implemented by the crowdsourcing data service 106 of FIG. The data included in the data repository 210 can be obtained from a variety of sources, such as various capture devices 102 of FIG. 1, other data collections or museums, and the like.

The link insertion module 204 determines elements of the retrieved data 212 that can be replaced by elements from other materials by identifying elements (also referred to as portions) of the retrieved data that are identical or similar to the data in the data repository 210. Any of a variety of different publicly available techniques and/or proprietary techniques for performing pattern matching and/or object matching may be used to identify such elements. The use of such techniques identifies objects or patterns in the retrieved material 212 and identifies other materials in the data repository 210 that include the same or similar pieces or patterns. For example, if the data 212 has been retrieved as a photo, the module 204 identifies the building or landmark in the photo and also identifies one or more other photos in the data repository 210 including the same building or landmark. .

Pattern matching and/or object matching may take into account various factors from recorded data included in the retrieved material 212, such as line position in the image and Vertex position, color in the image, sound pattern in the audio, etc. Pattern matching and/or object matching may also take into account various factors from the metadata included in the retrieved data 212. For example, such factors from the metadata may include the geographic location of the device that has retrieved the data, the point or direction of the device when the device retrieved the data, the date and/or time at which the data was retrieved, and the like.

For elements of the retrieved data 212 identified by the link insertion module 204 that can be replaced by elements from other materials, the link insertion module 204 removes the identified elements that can be replaced from the retrieved data 212. This replaceable element can be removed from the retrieved data 212 in a different manner. In one or more embodiments, the data of the identified element is replaced with some other material (usually high compression ratio compression), such as a series of all "0" bit values or all "1" bit values. For example, if the element of the photo (the data 212 has been retrieved) is removed, the data of the element included in the element may be replaced by the value "0" or "1" in the photo data. In other embodiments, a table or other record identifying the removed element and its corresponding location in the material may be maintained and the material of the identified element simply deleted. For example, if the element of the photo (the data 212 has been retrieved) is removed, a table or other record identifying which element in the photo corresponds to the element can be maintained, and in the material of the photo, the corresponding element can be simply deleted. The material of the picture element.

For each element of the retrieved material 212 removed by the link insertion module 204, the link insertion module 204 adds a link associated with the retrieved material 212 that replaces the alternate element of the identified element. The links to the alternate elements include various information identifying where the alternate elements are stored and/or how the alternate elements can be retrieved, allowing subsequent retrieval of the alternate elements as discussed in more detail below.

In one or more embodiments, the alternate element is saved as its own individual material (eg, its own file), and the link indicates where the individual data is stored in the data repository 210. Instructions for where individual data is stored may take different forms, such as file path names, uniform resource indicators (URIs), locations in the database, and the like. The individual modules or services may be interleaved (e.g., by means of a link insertion module 204 (e.g., in response to identifying an element that may be replaced by an alternate element, the module 204 may save the alternate element as a separate file)) data.

In other embodiments, the surrogate element is saved as part of other material (eg, including a portion of the file for saving other materials), and the link indicates where other materials are stored in the data repository 210 and the alternate elements are in other materials. Where are the two. Instructions for where other data is stored may take different forms, such as file path names, URIs, locations in the database, and the like. The indication of where the alternative element is in other materials may take different forms, such as a particular data range, information associated with a particular element, and the like.

The indication also records where the replaceable element is located in the retrieved material 212. The indication of where the replaceable element is in the retrieved material 212 may take a different form, such as a particular data range, material associated with a particular primitive, and the like. As discussed in more detail below, this indication of maintaining where the replaceable element is located in the retrieved material 212 allows for the addition of a replacement element when the promotional content is generated. The indication of where the replaceable element 212 has been retrieved, such as a portion of the link saved as an alternate element, is stored in the additional material associated with the retrieved material 212, in a different manner.

It should be noted that it is used to identify the retrieved data that can be replaced by substitute elements. The pattern matching and/or object matching techniques of the elements of 212 may select alternative elements that are similar to the identified elements of the retrieved material 212. An alternate element that is similar to an identified element means an element that is different from the identified element, but has a smaller margin difference from the identified element. There may be different ways (such as having at least one threshold portion (e.g., at least one of the limited number of primitives in the photo is the same value) having at least one threshold portion within the threshold of the identified element (eg, At least one of the limited number of primitives in the photo is within a threshold of each other, having a scribe line within a threshold of each other, etc.) determining the threshold difference and the like.

In one or more embodiments, where the replaceable elements and the substitute elements are not identical, the link insertion module 204 also records a change profile indicating the difference between the replaceable element and the substitute element. This change data can be recorded in different ways. For example, if the data 212 has been retrieved as a photo, the change data may be the difference between the values of the corresponding elements of the replaceable element and the substitute element. This change data (such as a portion of the link saved as an alternate element, saved in the additional material associated with the retrieved material 212) may be saved in a different manner.

Alternatively, where the replaceable elements and the substitute elements are not identical, the link insertion module 204 records no change data regarding the difference between the replaceable elements and the substitute elements. Thus, the link insertion module 204 can replace the elements of the image with other views of the element (eg, removing the view of the graffiti on the landmark, removing the view of the person standing in front of the landmark, etc.). The user interface is optionally provided by the link insertion module 204, which allows the module 204 to receive user input identifying which of the plurality of possible views of the element will be used as an alternate element of the replaceable element.

The link insertion module 204 provides the linked data 214 to the playback module. 208. The linked material 214 is the retrieved material 212 in which the replaceable element has been removed. In one or more embodiments, the linked material 214 also includes (e.g., as associated metadata) a link to an alternate element of the replaceable element, and optionally other additional associated with the retrieved material 212. data. Thus, in such embodiments, the linked material 214 includes the retrieved material 212 (excluding the removed element) and the link to the alternate element (and other additional material that is optionally associated with the retrieved material 212).

Alternatively, in addition to the link including the alternate element and/or other additional information associated with the retrieved material 212 in the linked material 214, the linked material 214 and the link associated with the retrieved material may be maintained in other manners and/or Or an association between additional information. In one or more embodiments, the links and/or other additional materials associated with the retrieved material 212 are maintained in a form or other record accessible by the playback module 208. Different methods may be used (such as based on the identifier of the retrieved data 212 (eg, generated by the capture device, based on the retrieved data 212 itself (eg, based on the hash value generated by the retrieved data 212), etc. The link associated with the retrieved material 212 and/or other additional information is identified as being associated with the retrieved material 212. The links associated with the retrieved material 212 and/or other additional materials may be saved in the data repository 210 or saved as part of another service or module, and accessed to discuss the generation of promotional content in more detail below. .

It should be noted that the link insertion module 204 can identify a plurality of elements of the retrieved material 212 that can be replaced with different replacement elements. In such cases, for each of the plurality of identified elements, the module 204 removes the identified elements from the retrieved material 212 and will add a chain of substitute elements in place of the identified elements. Pick up.

It should also be noted that the alternate elements of the link may have a different resolution than the element being replaced. For example, if the material 212 has been retrieved as a photo with a particular resolution, the alternate element may be included in a photo with a higher resolution (eg, more cells per square inch). Thus, the surrogate element may have additional details that are not available before being replaced in the identified element. This additional detail may allow, for example, the user to zoom in on the alternate element when subsequently viewing the promotional content and viewing details that are not available in the replaced identified element (eg, text, characters, artwork, etc.).

The link insertion module 204 can also add a link to one or more additional elements to the linked material 214. In one or more embodiments, in addition to removing the replaceable elements and adding the alternate element links, the module 204 also adds links to one or more additional elements. In other embodiments, module 204 adds a link to one or more additional elements without removing the replaceable element and adding a link to the alternate element. Thus, in some embodiments, system 200 does not replace the elements of the retrieved material with substitute elements.

Links to one or more additional elements may be links to materials in the data repository 210, and such links may identify where additional elements are stored and/or how additional elements may be retrieved in various ways, such links are similar to Links to alternative elements discussed in this article. Such links, which may include one or more additional elements, are associated with the linked material 214, or with the linked material 214, similar to the links of the alternate elements as discussed above.

The link of one or more additional elements enhances the retrieved data 212 by adding various types of information to the retrieved material. One or more extra elements It may be of the same type as the retrieved material 212 and/or of a different type than the retrieved data 212.

In one or more embodiments, one or more additional elements include elements of the same type of material as the retrieved material 212. For example, if the data 212 has been captured as an image (photo), one or more additional elements include image data. The link insertion module 204 can be determined in various ways to include additional elements. In one or more embodiments, module 204 uses various pattern matching and/or object matching techniques to identify captured data 212 that is the same as or similar to the data in data repository 210 (and/or links to and An element that captures the associated element of the data 212). This identification of elements that are the same or similar to the material in the data repository 210 can be performed in the same manner as the above discussed elements that can be retrieved from elements of other materials. Since the additional elements are identified based on the elements of the retrieved material and/or the additional elements are linked to the alternate elements associated with the retrieved material 212, such additional elements are also referred to as elements of the retrieved material and/or Alternate elements are associated.

Additional elements of the same type may be elements that include additional material or detail as compared to the identified elements. For example, if the retrieved material 212 is a photo that includes a particular landmark as an identified element, the additional element may be a photo that provides additional detail about the same landmark. The link insertion module 204 can determine additional elements that include additional material or detail as compared to the identified elements in different ways. In one or more embodiments, the module 204 determines to include a higher resolution than the resolution of the identified element (eg, more elements per square inch for the image data, for the audio material) Additional elements in the data taken at higher sampling frequencies, etc.) include additional information compared to identified elements Or details. In other embodiments, module 204 determines the element resolution for both the identified element and the additional element. The resolution of an element means how much data or detail is included in the element (eg, how many primitives, the number of bytes of data, etc.). Module 204 determines that if an additional element has a higher element resolution than the additional element, the additional element includes additional material or detail as compared to the identified element.

It should be noted that the link insertion module 204 can determine that a plurality of additional elements have additional data or details as compared to the same identified elements. In such cases, module 204 can include a link to one of a plurality of additional elements in linked material 214, or a link to a plurality of additional elements (eg, each).

The link insertion module 204 can additionally or otherwise determine one or more additional elements that include the same type of material as the retrieved material 212. In one or more embodiments, the link insertion module 204 determines that one or more additional elements are included in the linked material 214 based on the metadata of the retrieved material 212. This metadata may include various information such as the geographic location of the capture device, the point of view or direction of the capture device, and the date and/or time at which the data was retrieved as discussed above. The information included in the data repository 210 also includes associated metadata that may include the same information as the metadata associated with the retrieved data 212. For example, the data included in the data repository 210 may have associated meta-data that identifies the geographic location of the device that retrieved the data when the data was retrieved, and the observation point of the device that retrieved the data when the data was retrieved. Or direction, date and/or time of drawing data, etc.

The link insertion module 204 identifies the data in the data repository 210 having associated meta-data that matches the meta-data of the retrieved data 212. Metadata if the information in the metadata is the same or within each other's limit match. For example, if the metadata of the retrieved data 212 includes the geographical location of the capture device, and the metadata associated with the data in the data repository 210 includes the geographic location, the geographic location in the two metadata is In the case of the same or within a certain distance of each other (for example, 10 meters, 50 meters, etc.), the two metadata match. By way of further example, if the metadata of the retrieved data 212 also includes the direction of the capture device, and the metadata associated with the data in the data repository 210 includes directions, the geographic location in the two metadata is The two meta-data match if they are the same or within the marginal distance of each other and the directions are the same or within a margin of each other (eg, 3 degrees, 10 degrees, etc.).

Thus, even if the additional elements are not the same or similar elements as the elements in the retrieved material 212, the link insertion module 204 can identify additional elements that enhance the retrieved material 212 in the data repository 210. Such additional elements may, for example, extend the field of view of the image or video, extend the captured audio, and the like. For example, if the retrieved data 212 is an image captured at a specific time and at a specific time of the day by a pointing device pointing in a specific direction, the additional element may be an environment not included in the retrieved data 212. An additional portion (eg, a person, building, landscape, etc. outside the field of view of the capture device when capturing the image), but the additional portions are from the same (or near) geographic location, pointing to the same (or The similar) direction of the device is taken at the same (or similar) time of the day. Thus, even if the retrieved data 212 does not include such additional portions of the environment, the additional portions of the linked material 214 may be included.

In one or more embodiments, one or more additional elements include elements of a different type of material than the retrieved material 212. For example, if The captured data 212 is an image (photo), and one or more additional elements may include audio data, video data, and the like. These different types of data may be any of the types of data that may be retrieved by the retrieval device (e.g., the retrieval device 102 of Figure 1), or may be other types of data. For example, one or more additional elements can be textual type of material (eg, encyclopedia entries or other written descriptions), schema type materials, and the like. One or more additional elements may also include data types other than those of the type supported by the capture device. For example, if the retrieved data 212 is retrieved by a device that does not record audio data, one or more additional elements may include audio material.

The link insertion module 204 can determine that additional elements are included in the linked material 214 in various ways. In one or more embodiments, the link insertion module 204 determines additional elements of different material types based on the metadata associated with the retrieved material 212 and the metadata associated with the data in the data repository 210. Linked to data 214. Similar to the discussion above regarding the determination of additional elements of the same data type as the retrieved material 212, the link insertion module 204 identifies data having associated associated metadata in the data repository 210 that matches and has been associated The metadata associated with the data 212 is taken, but the additional elements are different types of data than the retrieved data 212.

Thus, the link insertion module 204 can identify additional elements in the data repository 210 that enhance the retrieved data by providing a type of data not included in the retrieved data 212. For example, if the captured data 212 is an image captured at a particular geographic location and at a particular time on a particular day, the additional elements may be additional types of material (eg, audio material) for the environment, such additional Type information is not included in the retrieved data 212. But the same (or similar) time for the same day is taken at the same (or close) geographic location. Therefore, even if the retrieved data 212 does not include their additional types of data, the additional data of the linked data 214 may be included.

In addition to or instead of using the meta-data of the retrieved data 212, the link insertion module 204 may also determine that additional elements of different data types are included in the linked material 214 in other manners. In one or more embodiments, the link insertion module 204 determines that additional elements of different material types based on the retrieved material 212 are included in the linked material 214 (or linked to an alternate element associated with the retrieved material 212). ). For example, module 204 can use various pattern matching and/or object matching techniques to identify particular elements (or links to alternate elements) that have captured data 212, such as particular landmarks, specific individuals, and the like. These techniques can also be used to identify various general environmental types, such as concerts, sporting events, and the like. The identified elements or general environmental types of the data in the data repository 210 are also identified, such as based on the data itself or an indication based on the identified elements or the general environmental types included in the metadata associated with the data in the data repository 210. . Since the additional elements are identified based on the elements of the retrieved material and/or the additional elements are linked to the alternate elements associated with the retrieved material 212, such additional elements are also referred to as elements and/or alternatives to the retrieved data. Elements are associated.

These various pattern matching and/or object matching techniques can be used to identify specific elements of the data in the data repository 210 (based on the data itself in the data repository 210 or metadata associated with the data in the data repository 210). And the link insertion module 204 can recognize that the same feature is included in the data repository 210. The information of the element. For example, module 204 may determine the particular type of environment in which data 212 is captured (eg, image data for a sporting event) and identify other types of material having the same environmental type in data repository 210 as additional elements (eg, Sports event audio information). By way of another example, module 204 can determine a particular landmark for capturing data 212 (eg, image and/or video material) and will have other types of material (eg, textual descriptions) of the same particular landmark in data repository 210. Recognized as an extra element.

It should be noted that the additional elements determined by the link insertion module 204 may be portions of other materials or other complete materials. For example, the captured data 212 can be an image or video, and the additional elements can be portions of other images, video, and/or audio, or can be complete other images, video, and/or audio. The additional elements may be determined by the module 204 in a different manner as discussed above based on the metadata associated with the retrieved material 212, the retrieved data 212 itself, and/or the alternate elements. For example, the module 204 can identify one or more images for a limited amount of time (eg, fifteen seconds) before the time the retrieved data 212 is retrieved as additional material.

It should also be noted that the environment capture module 202 can retrieve metadata describing various aspects of the capture device (eg, geographic location of the device, orientation of the device, or observation point, etc.), but does not record any other information about the environment. (For example, do not capture images, do not capture audio, do not capture video, etc.). The capture module 202 can thus be included in a device that does not have an environmental sensor for sensing environmental characteristics (eg, a sensor for images or audio in a non-recording environment), or included in recording metadata but not An environmental sensor for sensing environmental characteristics is recorded in a device that records information about the environment. However, based on metadata, it can be inserted by the link The incoming module 204 determines one or more additional elements to add images, audio, video, etc. to the linked material 214. Therefore, the capture device may not include an image capture component, does not include an audio capture component, etc., but only records metadata describing the aspect of the capture device (eg, the geographic location and orientation or viewing point of the device). The image data and/or audio material in the data repository 210 can be identified as additional elements based on the metadata as discussed above, and links to additional elements can be added by the module 204 to the linked material 214.

It may be the case that the retrieved data 212 includes multiple types of data (eg, video material and audio material). The link insertion module 204 can optionally replace one of the types of data by determining alternative elements and/or additional elements. For example, one type of material (eg, image material or video material) can be used to identify a particular type of environment in which the data has been retrieved, and can identify additional elements (eg, audio material) having the same environmental type. The module 204 can replace the corresponding type of data in the retrieved data 212 with a link to the additional element. Thus, for example, one type of data (eg, video material or video material) in the retrieved data 212 can be retained in the linked material 214, and other types of data (eg, audio data) in the data 212 can be retrieved. Replace with a link to an extra element.

Linked material 214 is provided to playback module 208. The link insertion module 204 can provide the linked data 214 to the playback module 208 in various manners, such as including the data 214 as a parameter when the interface of the module 208 is invoked, and saving the data 214 at a location accessible by the module 208. E-mail or use other messaging protocols to communicate the data 214 to the module 208 and the like. The playback module 208 is required in the data 214 from the crowdsourcing data module 206. The element of the link. As discussed above, the links of elements identify where the alternate elements (and/or additional elements) are stored and/or how the alternate elements (and/or additional) may be retrieved. The crowdsourcing data module 206 obtains the linked elements from the data repository 210 and provides the linked elements to the playback module 208 as promotional material 216.

The playback module 208 receives the promotional material 216 and combines the promotional material 216 with the linked material 214 to generate or construct the enhanced content 218. The data 214 and the data 216 can be combined in different ways based on the type of elements included in the promotional material 216. For example, an alternate element can be added to the material 214 (where the replaceable element is located in the retrieved material 212). Additional elements may be added to the material 214 via another example. For example, the photo may be enhanced to include additional portions not included in the retrieved material 212, the audio data may be added to the image data, and the captured audio material may be replaced with other audio data to enhance the environment without other The retrieved meta-data of the data has been retrieved to include audio and/or video material. The playback module 208 can play back (e.g., display or render) the promoted content 218, or take other actions on the promotional content 218 (e.g., hold the enhanced content 218 on a particular storage device, and transfer the enhanced content 218 to another Device or module, etc.).

FIG. 3 is a flow chart illustrating an exemplary program 300 for enhancing captured data in accordance with one or more embodiments. The program 300 is a method or scheme performed by one or more modules, such as the link insertion module 204 of FIG. 2, and may be implemented in a combination of software, firmware, hardware, or the like. The program 300 is illustrated as a collection of actions and is not limited to the illustrated order of operations for performing various actions. The program 300 is an exemplary process for enhancing the retrieved data. In this article, this article includes additional discussion on the enhancements that have been made to the different figures.

In program 300, retrieved information about the environment is obtained (act 302). As discussed above, the retrieved data can be retrieved or provided to one or more modules of the implementation program 300 in a variety of manners.

One or more additional elements are determined (act 304). The one or more additional elements are determined based on the retrieved material as discussed above. User input that facilitates determining one or more additional elements (eg, user input indicating factors or techniques for identifying additional elements, user input indicating data types for additional elements, multiple additional) may optionally be received The user of one or more of the elements chooses, etc.).

One or more links to one or more additional elements associated with the retrieved material are added (act 306). As discussed above, these one or more links can be added in a variety of different ways.

The construction of the enhanced content using one or more additional elements and at least a portion of the retrieved material is enabled (act 308). The linking of one or more additional elements allows for the construction of promotional content. As discussed above, this configuration includes combining one or more additional elements with at least a portion of the retrieved material.

FIG. 4 is a flow chart illustrating another exemplary program 400 for enhancing captured data in accordance with one or more embodiments. The program 400 is a method or scheme performed by one or more modules, such as the link insertion module 204 of FIG. 2, and may be implemented in a combination of software, firmware, hardware, or the like. The program 400 is illustrated as a collection of actions and is not limited to the illustrated order of operations for performing various actions. The program 400 is an exemplary program for enhancing the retrieved material, and this article includes additional discussion of promotional data for different graphs.

In program 400, retrieved information about the environment is obtained (act 402). As discussed above, the retrieved data can be retrieved or provided to one or more modules of the implementation program 400 in a variety of manners.

A decision is made as to whether one or more elements of the retrieved material are replaceable with one or more alternate elements (act 404). As discussed above, this decision includes the steps of identifying one or more replaceable elements and one or more substitute elements.

One or more identified replaceable elements are removed from the retrieved material (act 406) to produce a compressed photo. As discussed above, such replaceable elements can be removed in different ways.

One or more links of one or more alternate elements are added (act 408). One or more of the links associated with the retrieved material are added and can be added in a variety of different ways as discussed above. User input that facilitates determining one or more alternate elements (eg, user input indicating factors or techniques for identifying alternate elements, user selection of one of a plurality of alternate elements, etc.) may be received as appropriate.

One or more additional elements are also determined (act 410). The one or more additional elements are determined based on the retrieved material (and/or alternate elements), and as discussed above, the one or more additional elements are associated with the replaceable element and/or the substitute element. User input that facilitates determining one or more additional elements (eg, user input indicating factors or techniques for identifying additional elements, user input indicating data types for additional elements, multiple additional) may optionally be received The user of one or more of the elements chooses, etc.).

Add one or more links to one or more additional elements (action 412). These one or more links are associated with the retrieved material and can be added in a variety of different ways as discussed above. The photo is compressed due to, for example, an alternate element removed from the photo, and an additional element that enhances the linked photo, rather than a link to the additional element itself included in the photo.

The construction of the enhanced content using the compressed retrieved material and the link of one or more alternate elements and additional elements is enabled (act 414). The linking of one or more alternate elements and/or one or more additional elements allows for the construction of promotional content. As discussed above, this configuration includes combining one or more additional elements, replacement elements, and compressed captured data.

The enhanced data technology discussed in this paper supports various usage scenarios. For example, users can take photos of various landmarks, artworks, landscapes, and the like. The parts of the photographs of various landmarks, artworks, landscapes, etc. can therefore be replaced with parts of other higher resolution photographs of the same landmark, artwork, landscape, etc., providing the user with a higher resolution than the photograph taken by him or her. Resolution photo (and resolution that may be higher than the resolution of photos that his or her camera can take). The user is thus able to view additional details, as appropriate, to zoom in on portions of the photo to view characters, handwriting, designs, etc. that are not visible in the photos he or she took. Audio data can also be added to the photo, allowing playback of the audio when the photo is subsequently displayed. Therefore, the user can present audio information corresponding to the landmark, artwork, landscape, etc., even if the user does not capture the audio in his or her photo.

Although various discussions are made herein for photographs with reference to the extracted materials, it should be noted that the techniques discussed herein are applicable to a variety of other types of retrieved materials (such as video material). View video resources as a sequence of frames or arrays Each of the frames is processed as a photo as discussed herein. The techniques discussed herein are applicable to multiple frames of a frame in a sequence or array (eg, each frame). For a particular frame in the video, one or more of the alternate elements may be elements from other photos or videos and/or elements from other frames of the same video. Therefore, instead of compressing the video based on the difference between the key frame and the key frame and subsequent frames in the video, the frame of the video can be replaced by a link with an alternative element in the video or other frame. The element in it.

Various actions (such as communicating, receiving, providing, recording, saving, generating, obtaining, etc.) performed by various modules are discussed herein. The particular module discussed herein for performing an action includes a particular module that performs the action itself, or a module that invokes or accesses a particular module of another component or performs an action (or performs an action in conjunction with a particular module). Thus, a particular module that performs an action includes the particular module itself that performs the action and/or another module that is invoked or accessed by a particular module that performs the action.

FIG. 5 illustrates an exemplary computing device 500 that can be configured to implement enhanced learned data in accordance with one or more embodiments. The computing device 500 can be, for example, the device 102 or 104 of FIG. 1, implementing at least a portion of the crowdsourcing data service 106 of FIG. 1, implementing one or more of the modules 202 to 208, and the like of FIG.

The computing device 500 as shown includes a processing system 502, one or more computer readable media 504, and one or more I/O interfaces 506 that are communicatively coupled to each other. Although not shown, computing device 500 can further include a system bus or other material and a command transmission system that couples the various components to each other. The system bus can include any one or group of different bus structures In conjunction, such as a memory bus or memory controller, a peripheral bus, a universal serial bus, and/or a processor or local bus using any of a variety of bus architectures. Various other examples (such as control lines and data lines) are also contemplated.

Processing system 502 represents functionality for performing one or more operations using hardware. Accordingly, processing system 502 is illustrated as including hardware elements 508 that can be configured as processors, functional blocks, and the like. This may include implementation in a hardware as a special application integrated circuit or other logic device formed using one or more semiconductors. The hardware component 508 is not limited to the materials used to form the hardware component 508 or the processing mechanisms employed in the hardware component 508. For example, a processor can include one or more semiconductors and/or transistors (eg, an electronic integrated circuit (IC)). In this case, the instructions executable by the processor may be electronically executable instructions.

Computer readable medium 504 is illustrated as including a memory/storage 510. Memory/storage 510 represents memory/storage capacity associated with one or more computer readable media. The memory/storage 510 can include volatile media (such as random access memory (RAM)) and/or non-volatile media (such as read only memory (ROM), flash memory, compact discs, and magnetic disks. Wait). The memory/storage 510 can include fixed media (eg, RAM, ROM, fixed hard drives, etc.) as well as removable media (eg, flash memory, removable hard drives, optical disks, etc.). Computer readable medium 504 can be configured in various other manners as further described below.

One or more input/output interfaces 506 represent functions that allow a user to enter commands and information into computing device 500, and also allow information to be presented to the user and/or other components or devices using various input/output devices. Input Examples include keyboards, cursor control devices (eg, mice), microphones (eg, for voice or other audio input), scanners, touch-sensitive functions (eg, capacitive sensing configured to detect physical touches) Or other sensors), cameras (eg, cameras that can detect movements that do not involve touch (eg, gestures) using visible or invisible wavelengths (such as infrared frequencies), and the like. Examples of output devices include display devices (eg, monitors or projectors), speakers, printers, network cards, tactile response devices, and the like. Thus, computing device 500 can be configured in various ways to support user interaction.

Computing device 500 also includes retrieved data enhancement system 520. The retrieved data enhancement system 520 provides various functions for enhancing the retrieved material, including capturing data, inserting links, constructing promotional content for playback, and/or providing crowdsourcing materials as discussed above. The retrieved data enhancement system 520 can be, for example, one or more of the modules 202-208 of FIG.

This article describes various techniques in the general environment of software, hardware components, or program modules. In general, such modules include routines, programs, objects, components, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The terms "module," "function," and "component" as used herein generally mean a combination of software, firmware, hardware, or a combination of the above. The features of the techniques described herein are platform independent, meaning that such techniques can be implemented on a variety of commercial computing platforms having a variety of processors.

Implementations of the described modules and techniques can be stored on or transmitted across some form of computer readable media. Computer readable media can include a variety of media that can be accessed by computing device 500. By way of example and not limitation, computer readable media may include "computer readable storage medium" and "computer readable communication" No. Media."

"Computer-readable storage medium" means a medium and/or device that enables permanent and/or non-transitory storage of information as compared to signal transmission, carrier or signal itself. Thus, a computer readable storage medium means a medium that is not signaled. Computer-readable storage media includes hardware (such as volatile and non-volatile) implemented in a method or technique suitable for storage of information (such as computer readable instructions, data structures, program modules, logic elements/circuits, or other materials). Media, removable and non-removable media and/or storage devices). Examples of computer readable storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile compact disc (DVD) or other optical storage, hard drive, tape A cartridge, tape, disk storage or other magnetic storage device, or other storage device, tangible media, or an article suitable for storing the desired information and accessible by a computer.

"Computer-readable signal media" means a signal-bearing medium that is configured to transfer instructions to the hardware of computing device 500, such as via a network. Signal media usually contains computer readable instructions, data structures, program modules, or other data (such as carrier waves), data signals, or other transport mechanisms in the modulated data signal. The signal media also includes any information delivery media. The term "modulated data signal" means a signal that has one or more of its feature sets or that changes in this manner of information in the encoded signal. By way of example and not limitation, communication media includes wired media (such as a wired network or direct wired connection), and wireless media (such as sonic, radio, infrared, and other wireless media).

As previously described, hardware component 508 and computer readable medium 504 represent instructions, modules, programmable device logic, and/or solids implemented in hardware. Device logic, which may be used in some embodiments to implement at least some aspects of the techniques described herein. The hardware component 508 can include an integrated circuit or a system on a wafer, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a composite programmable logic device (CPLD), and/or other hardware devices. Components of other implementations. In this context, a hardware component is operative to perform processing tasks defined by instructions, modules, and/or logic, the instructions, modules, and/or logic being included in hardware components and for storing execution instructions A hardware device (for example, a computer readable storage medium as described above).

Combinations of the above may also be used to implement the various techniques and modules described herein. Accordingly, software, hardware or program modules and other program modules can be implemented as one or more instructions embodied in some form of computer readable storage medium and/or one or more hardware elements 508 and/or Or logic. Computing device 500 can be configured to implement specific instructions and/or functions corresponding to the software and/or hardware modules. Thus, implementation of a module executable by the computing device 500 as a software can be implemented, at least in part, in hardware, for example, via a hardware component 508 using a computer readable storage medium and/or processing system. The instructions and/or functions may be executed/operated by one or more articles of manufacture (eg, one or more computing devices 500 and/or processing system 502) to implement the techniques, modules, and examples described herein.

Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined by the appended claims Rather, the specific features and acts described above are disclosed in the form of example embodiments.

200‧‧‧Exemplary Data Enhancement System

202‧‧‧Environment capture module

204‧‧‧Link Insertion Module

206‧‧‧Crowd Outsourcing Data Module

208‧‧‧Playback module

210‧‧‧Data repository

212‧‧‧Retrieved information

214‧‧‧Link information

216‧‧‧Promotional information

218‧‧‧ has been enhanced

Claims (20)

A method comprising the steps of: obtaining learned data about an environment; determining one or more additional elements based at least in part on the captured data; adding the one associated with the retrieved material One or more links of one or more additional elements; and enabling the use of the one or more additional elements and at least a portion of the retrieved material to construct promotional content. The method of claim 1, the method further comprising the steps of: determining one or more elements of the retrieved material that can be replaced by one or more substitute elements; removing the one from the retrieved data Or more elements; and adding a link to the one or more replacement elements associated with the retrieved material. The method of claim 1, wherein the captured data comprises an image. The method of claim 3, the one or more additional elements comprising audio material relating to the environment. The method of claim 1, wherein the captured data comprises audio data. The method of claim 5, the one or more additional elements comprising image material relating to the environment. The method of claim 1, the captured data includes metadata describing a geographic location of the device when the captured data is retrieved by a device, but the captured data is not included in the geographic location. The image or audio captured. The method of claim 7, the one or more additional elements comprising an image captured by another device at the geographic location. The method of claim 1, wherein the captured data comprises meta-data describing a geographic location of the device when the captured data is retrieved by a device, and the determining step comprises the step of: determining based on the metadata The one or more additional elements. The method of claim 9, the metadata further describing a point of view or direction of the device. The method of claim 1, the method further comprising the steps of: identifying one or more elements in the retrieved material; and the determining step comprising the step of: based on the identified one or more elements Decide on the one or more additional elements. The method of claim 1, the method further comprising the step of playing back the promotional content. A compression scheme for a promotional photo, the method comprising the steps of: determining whether a photographic element of a photo is replaceable, comprising identifying the replaceable photographic element and one or more substitute photographic elements; removing the replaceable photographic element An element to generate a compressed photo; adding one or more of the one or more alternative photographic elements; adding one of an additional element associated with the replaceable photographic element or the substitute photographic element; and energizing The promotional photo is constructed using the compressed photo and the one or more alternate photographic elements and links to the additional element. The compression scheme of claim 13, the determining step comprising the steps of: identifying the replaceable photographic element; and determining that the replaceable photographic element is identical to an alternate photographic element or has a smaller threshold difference, replaceable This replaceable photographic element. The step of adding one or more links of the one or more substitute photographic elements, as in the compression scheme of claim 13, further comprising the step of: saving, for each replaceable element, the replaceable photographic element and The information on the change in the difference between the alternative photographic elements. The compression photographic element of claim 13 includes a higher resolution than the replaceable photographic element. The additional element expands the field of view of the photo as in the compression scheme of claim 13. The compression scheme of claim 13 includes a different type of material than the image material. The compression scheme of claim 18, the different types of data comprising audio data or text data. A computer storage medium having a plurality of instructions stored thereon, the instructions being caused by one or more processors of a device to cause the one or more processors to perform the steps of: determining a photo element of a photo Whether replaceable, comprising identifying the replaceable photographic element and one or more substitute photographic elements; removing the replaceable photographic element from the photo to produce a compressed photo; adding the one or more associated with the photo Substituting one or more links of the photographic element; determining an additional element based on the metadata associated with the photo, the metadata describing a geographic location of the capture device when the photo is captured by a capture device; Adding a link to one of the additional elements associated with the photo, the additional element containing audio material; and enabling the photo to be constructed using the compressed photo and the one or more alternate photographic elements and links to the additional element.