JP6067041B2 - Automatic privacy management for image sharing networks - Google Patents

Description

  Unless otherwise indicated herein, the materials described herein are not prior art to the claims of this application and are not admitted to be prior art by inclusion in this part.

  Image sharing services can provide users with the ability to upload images, such as photos and videos, to the network cloud that hosts those images. Once on the network cloud, these images can be made publicly available to allow users of the service to share the images with their friends, peers, and others. However, image sharing services do not provide any mechanism for protecting the privacy of people in publicly shared images. Specifically, image sharing services do not provide a mechanism for protecting the privacy of those who have not approved that their images have been captured. Furthermore, image sharing services do not provide users with tools to manage their own images in order to protect the privacy of third parties that the users capture in the images. Instead, users of the image sharing service can protect third party privacy by manually reviewing each image to ensure that the third party images are not publicly shared on the image sharing service. It is possible to protect.

  The techniques described herein generally relate to managing images in an image sharing network.

  In some examples, a method for managing captured images is described. The method can include receiving image data that includes image position data regarding where the image was captured. The method can also include receiving device data from the device. This device data may include device position data. The method can also include correlating device data with image data. The method may also include modifying the image based on the correlation between the device data and the image data.

  In some examples, a method for managing captured images is described. The method can include receiving image data that includes image position data regarding where the image was captured. The method can also include receiving device data from the device. The device data can include device location data. The method can also include correlating device data with image data. The method may also include calculating a probability that a user of the device will appear in the image based on the correlation between the device data and the image data.

  In some examples, a system is described that may include a data correlator and an image modifier. The data correlator can be configured to correlate image data with device data. The image data can include image position data regarding where the image was captured, and the device data can include device position data regarding the position of the device. The image corrector can be configured to correct the image based on the correlation between the image position data and the device position data.

  In some examples, a system is described that may include a data correlator and a calculation unit. The data correlator can be configured to correlate the image data with the device data. The image data can include image position data regarding where the image was captured, and the device data can include device position data regarding the position of the device. The calculation unit may be configured to calculate a probability that a user of the device will appear in the image based on the correlation between the device data and the image data.

  In some examples, a computer readable storage that, when executed by a processor, can cause the processor to receive image data including image location data regarding where the image was captured when the content is executed by the processor. The medium is described. Execution of these contents can also cause the processor to receive device data from the device. The device data can also include device location data. Execution of these contents can also cause the processor to correlate the device data with the image data. Execution of these contents may cause the processor to modify the image based on the correlation between the device data and the image data.

  The foregoing summary is merely exemplary and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features will become apparent by reference to the drawings and the following detailed description.

1 is a block diagram of an exemplary image management system. FIG. 2 is a block diagram illustrating components of an image sharing network within the exemplary image management system shown in FIG. 3 is an exemplary flow diagram of a method that may be implemented within the image management system of FIGS. 3 is an exemplary flow diagram of a method that may be implemented within the image management system of FIGS. 3 is an exemplary flow diagram of a method that may be implemented within the image management system of FIGS. FIG. 6 is a block diagram illustrating an example computing device configured to manage captured images.

  Each figure is all configured in accordance with at least some embodiments described herein.

  In the following detailed description, reference is made to the accompanying drawings, which form a part hereof. In these drawings, similar symbols typically identify similar components, unless the content clearly dictates otherwise. The illustrative embodiments described in the detailed description, drawings, and claims are not meant to be limiting. Other embodiments may be utilized and other changes may be made without departing from the spirit or scope of the subject matter presented herein. The aspects of the present disclosure that are generally described herein and illustrated in the drawings may be configured, replaced, combined, or separated in a wide variety of different structures, all of which are explicitly contemplated herein. And it will be readily understood that it is possible to design.

  Some embodiments described herein generally maintain the privacy of a cloud-based image sharing network or image sharing service and third parties captured in images shared on the image sharing network. Regarding ability and. For example, an image sharing network can maintain the privacy of unintentionally photographed people by obscuring the person's face in the photo.

  In general, for example, an image sharing network is capable of receiving images, such as photos or videos, from users of the image sharing network. An image can include image position data, such as information regarding when and where the image was taken. The image sharing network can also receive location data from a third party registered in the image sharing service in order to protect its privacy. The location data can include when a third party was at a particular location. In some embodiments, location data from a third party can be transmitted to an image sharing network from a mobile communication device, such as a smartphone, associated with the third party.

  The image sharing network can correlate the third party location data with the image location data to determine the probability that one or more third parties have been captured in the image. If the image sharing network determines that one or more third parties have been captured in the image, the image can be modified to protect the privacy of the third party. In some embodiments, the image is obscured by a third party face in the image, by removing a third party face in the image, or the image is on an image sharing service. It can be modified by marking the image as private so that it is not publicly shared. In some embodiments, the image sharing network can determine whether a third party knows the user who uploaded the image. If the third party knows the user, the image sharing network can determine that the third party was the subject of the image and not modify the image.

  FIG. 1 is a block diagram of an exemplary image management system 100 configured in accordance with at least some embodiments described herein. In the exemplary embodiment, image management system 100 can include an image sharing network 110, a communication network 112, a mobile device 120, and an image capture device 130.

  In general, the communication network 112 includes one or more wide area networks (WANs) and / or local area networks (LANs) that allow the mobile device 120 and the image capture device 130 to communicate with the image sharing network 110. It is possible to include. In some embodiments, the communication network 112 may include the Internet, including a global internetwork formed by logical and physical connections between multiple WANs and / or LANs. Alternatively or additionally, communication network 112 may include, but is not limited to, one or more cellular RF networks and / or one or more wired networks and / or 802. It can include wireless networks such as xx networks, Bluetooth access points, wireless access points, IP-based networks, and the like. The communication network 112 may also include a server that allows one type of network to interface with another type of network.

  The image sharing network 110 can be configured to store and share images uploaded to the image sharing network 110 by users of the image sharing network 110. The image sharing network 110 can store images such as photographs. These photos are not limited to JPEG, TIFF, Exif, RAW, PNG, GIF, BMP, WEBP, PPM, PGM, PBM, and PNM, but can also be stored in various file formats. is there. The image sharing network 110 can also store images such as videos. Videos can be stored in a variety of file formats, including but not limited to DV and MP4. As described in more detail below, the image sharing network 110, among other things, modifies images to maintain the privacy of third parties captured within images shared on the image sharing network 110. Can be configured.

  The image sharing network 110 can receive images from an image capture device such as the image capture device 130. The image capture device 130 can capture an image and upload the image to the image sharing network 110 via the communication network 112. The image capture device 130 can also create data relating to images uploaded to the communication network 112. For example, the image capture device 130 can create image data for each image that can include, but is not limited to, the location at which the image was captured and the time at which the image was captured. is there. Image capture device 130 may determine the location at which the image was captured using GPS, wireless network, wired network, or by receiving location data from image capture party 119. For example, in some embodiments, the location of the image capture device 130 can be determined based on signal strength from surrounding mobile telephone base stations. In some embodiments, the image data can be embedded in the image as metadata and uploaded to the image capture device 130 along with the image. In some embodiments, the image data may be separated from the image and may be uploaded to the image sharing network 110 separately from the image.

  The image capture device 130 may be a simple image capture device such as a digital camera or a digital camcorder. In some embodiments, the image capture device 130 is limited to a mobile communication device such as a mobile phone or smartphone, a gaming system, a personal digital assistant (PDA), a laptop computer, a desktop computer, a handheld electronic device, or a tablet computer. Although not included, they can be included in another device.

  In some embodiments, the image capture device 130 may not communicate with the image sharing network 110 via the communication network 112. The image capture device 130 can download the images and image data to a secondary device (not shown), such as a personal computer, laptop computer, tablet computer, or other electronic device. The secondary device can upload images and image data to the image sharing network 110 via the communication network 112. In some embodiments, the image capture device 130 may not generate image data. The secondary device can generate image data and upload the image data to the image sharing network 110 along with the image from the image capture device 130.

  The image sharing network 110 can also receive data from a device such as the mobile device 120. The mobile device 120, among other things, executes one or more applications that interface with the image sharing network 110, generate device data, and send the device data to the image sharing network 110 via the communication network 112. A device data generator 122 may be included that may be configured as such. The device data can include, but is not limited to, the location of the mobile device 120 and the time that the mobile device 120 is in that location. For example, the device data can indicate that the mobile device 120 is at a longitude and latitude at a certain time.

  In some embodiments, the device data generator 122 can determine the location of the mobile device 120 using GPS. Alternatively or additionally, the device data generator 122 may use a mobile device using one or more wireless or wired networks, or any other means for determining its location. It is possible to determine 120 positions. For example, in some embodiments, the location of the device 120 can be determined based on signal strength from neighboring base stations of the mobile communication network. Further, in some embodiments, the device data generator 122 may add additional information for transmission to the image sharing network 110 each time the device data generator 122 determines that the mobile device 120 has moved to a new location. Device data can be generated. Alternatively or additionally, the device data generator 122 may add additional device data at one or more set time intervals, eg, every minute, every two minutes, every five minutes, or other time intervals. Can be generated periodically. Alternatively or additionally, the device data generator 122 monitors the location of the mobile device 120 and retrieves additional device data after the mobile device 120 has moved a predetermined distance from the previously reported location. It is possible to generate.

  In some embodiments, the device data generator 122 can transmit device information in real time. Alternatively or additionally, the device data generator 122 stores device information and transmits the device information less frequently, such as every hour, every two hours, every twelve hours, or once a day. Is possible. In some embodiments, the device data generator 122 can automatically send device data to the image sharing network 110. Alternatively or additionally, the device data generator 122 can send device data to the image sharing network 110 at the request of a third party 118 that is the owner of the mobile device 120.

  Mobile device 120 may be any electronic device capable of generating location data and time data. For example, in some embodiments, the mobile device 120 is a mobile communication device, such as a mobile phone or smartphone, tablet, GPS navigation system device, gaming system, media player, or other electronic device that can determine its location. However, it is not limited to this. In some embodiments, the mobile device 120 may not communicate with the image sharing network 110 via the communication network 112. For example, the device data generator 122 generates secondary device data and uploads the device data to the image sharing network 110, such as a personal computer, laptop, mobile communication device, tablet, or other electronic device. It is possible to download the device data (not shown).

  Device data received from the mobile device 120 determines the image sharing network 110 current and past locations of the third party 118 owners of the mobile device 120 and the time the third party 118 was at those locations. Make it possible to do. As described in more detail below, using the device data received from the mobile device 120 and the image data received from the image capture device 130, the image sharing network 110 can be used by a third party 118, Image capture device 130 can be used to determine the probability of being captured in an image taken by image capture party 119. Based on this determination, in some embodiments, image sharing network 110 may modify the image to prevent third party 118 images from being publicly displayed by image sharing network 110. Is possible. In some embodiments, the third party 118 that is the owner of the mobile device 120 turns off the device data generator 122 so that device data is not generated or transmitted to the image sharing network 110. Can do. If device data is not generated, the image sharing network 110 may not prevent third party 118 images from being publicly displayed by the image sharing network 110.

  FIG. 2 is a block diagram illustrating components of the image sharing network 110 in the image management system 100 shown in FIG. 1 configured in accordance with at least some embodiments described herein. As illustrated, the image sharing network 110 includes a device data analyzer 212, an analysis data storage device 214, an image storage device 220, an image analyzer 222, an analysis image data storage device 224, and a data correlator 230. And a calculation unit 232 and an image modifier 234 can be included. In some embodiments, the analytical data storage device 214, the image storage device 220, and the analytical image data storage device 224 can each include a non-transitory computer readable storage medium and / or It is possible to represent logical partitions in the same non-transitory computer-readable storage medium or in different non-transitory computer-readable storage media. Each of device data analyzer 212, image analyzer 222, data correlator 230, calculation unit 232, and image modifier 234 may be implemented in hardware, software, or a combination of both hardware and software. Is possible. Alternatively or additionally, the image sharing network 110 is illustrated as having various individual components, but the various components may be divided into additional components and combined depending on the desired implementation. Fewer components may be formed or removed.

  Device data analyzer 212 may be configured to receive device data from device data generator 122 of mobile device 120. The device data analyzer 212 may alternatively or additionally be configured to use the device data to calculate the travel path of the mobile device 120. In some embodiments, device data received by the image sharing network 110 may represent the location of the mobile device 120 at different time intervals. For example, the device data may include the location of mobile device 120 at time A and the location of mobile device 120 at time B, but includes the location of mobile device 120 during the period between times A and B. There is no possibility. For example, if the mobile device 120 is moving and the device data generator 122 is generating device data every minute, the position of the mobile device 120 between position data points in the device data is included in the device data. It may not be possible. In at least these cases, the device data analyzer 212 can interpolate the position of the mobile device 120 in one or more time frames based on the device data provided by the mobile device 120.

  In some embodiments, the device data analyzer 212 can use the road information to interpolate the position of the device in one or more time frames. The device data analyzer 212 can obtain road information from one or more sources, including road information of the geographic information system. The device data analyzer 212 can retrieve the latest and shortest combination of roads to link the two locations from the device data. Using this information, device data analyzer 212 can interpolate the position of mobile device 120 in the time frame between data points provided by mobile device 120. For example, if the mobile device 120 generates device data every T seconds and is at point D at time A and is at point F at time C, the device data analyzer 212 may be after time A and before time C. If the mobile device 120 can be located at a point E between points D and F at time B occurring at, it can be interpolated.

  In some embodiments, the device data analyzer 212 divides the earth using a grid pattern to create device position cells and places the device data points and interpolated device data points within the appropriate device cells. It is also possible to arrange. In this data structure, each device location cell may correspond to a specific area of the real world. This allows the mobile device 120 to be associated with a device location cell that includes the location that the mobile device 120 traverses. The device location cell may be determined based on one or more of longitude and latitude, geographic boundaries, streets, city boundaries, or other factors. Further, the size of the device location cell may vary and may depend on one or more factors such as, but not limited to, the desired resolution, memory constraints, processing power, and other factors. Device data received from the mobile device 120 and any interpolated device data can be stored in the analysis data store 214 for later use. Device data and interpolated device data may be stored according to its associated device position cell in some embodiments.

  The image sharing network 110 may also be configured to receive images from the image capture device 130 and store those images in the image storage device 220. The image sharing network 110 can also be configured to receive image data from the image capture device 130. As described above, in some embodiments, the image data may be embedded as part of the image. In these and other embodiments, these images can be sent to the image analyzer 222 to extract image data from the images. In some embodiments, the images are analyzed by the image analyzer 222 before the images are stored in the image store 220. In some embodiments, these images are stored in the image storage device 220 and later accessed and analyzed by the image analyzer 222 to extract image data.

  The image data can be analyzed whether extracted from the image or received independently from the image capture device 130. In some embodiments, the image analyzer 222 divides the earth using a grid pattern to create an image position cell and identifies the image based on the position data in the image data. It can be placed in an image location cell. In this data structure, each image location cell may correspond to a specific area of the real world. This allows the image to be associated with an image location cell that includes the location where the image was captured. For example, in some embodiments, the image location cell may be a four city block. If an image is captured within those four city blocks, it can be linked to its image location cell. The image location cell can be determined based on one or more of longitude and latitude, geographic boundaries, streets, city boundaries, or other factors. Further, the size of the image location cell may vary and may depend on one or more factors such as, but not limited to, the desired resolution, memory constraints, processing power, and other factors. In some embodiments, the image location cell may be similar to the location cell used by the device data analyzer 212 in terms of size to analyze the device data.

  The analysis image data storage device 224 can store the image data including the image position and the image time. In some embodiments, the image data can be stored in the analytic image data store 224 according to the image location cell associated with the image data.

  The data correlator 230 compares the device data with the image data to determine the fact that the image was captured at the same location at the approximate time that the mobile device 120 was at a location, thereby analyzing the data store 214. The device data stored therein may be configured to correlate with the image data stored in the analysis image data storage device 224. The data correlator 230 can correlate the device data with the image data using any known algorithm. In some embodiments where the image data is associated with an image location cell, the data correlator 230 determines whether the mobile device 120 has crossed the image location cell or the image location cell to correlate the device data with the image data. It is possible to determine whether and within which time the mobile device 120 was within the image location cell or the approximate time that the mobile device 120 was within the image location cell. In some embodiments in which the image data is associated with an image location cell and the device data is associated with a device location cell, the data correlator 230 is configured such that the device location cell traversed by the mobile device 120 is associated with that image. It is possible to determine whether or not the image position cell is close. If the device location cell and the image location cell match, overlap, or are in close relationship, the data correlator 230 may store the device location cell to determine the correlation between the device data and the image data. It is possible to determine whether the associated time data and the time data associated with the image location cell match, overlap or are closely related.

  In some embodiments, the data correlator 230 is known to the third party 118 where the image capture party 119 is the owner of the mobile device 120 if the time and position data of the device data and image data correspond. It is possible to determine whether or not. If data correlator 230 determines that image capture party 119 is known to third party 118, data correlator 230 approves any image that third party 118 has captured by image capture party 119. It is possible to determine that As a result, the image sharing network 110 may not modify the image.

  Data correlator 230 determines whether image capture party 119 is known to third party 118 by comparing name information associated with the image to a list of people associated with third party 118. Is possible. For example, in some embodiments, the image sharing network 110 can associate an uploaded image with the name of the image capture party 119 that uploaded the image. A third party 118 may provide a list of friends to the image sharing network 110. Data correlator 230 can use the list of associates and associated names to determine whether third party 118 knows image capture party 119. In some embodiments, the third party 118 can grant the image sharing network 110 access to the social network in which the third party 118 is participating, and the image sharing network 110 It is possible to generate a list of friends based on the social relationships of other social networks.

  If there is a correlation between the device data and the image data, the data correlator 230 can be configured to deliver the image data and the device data to the calculation unit 232. The calculation unit 232 can be configured to calculate the probability that the third party 118 was captured in an image associated with the image data. The calculation unit 232 determines the strength of the correlation between the device data and the image data, the region where the image was captured, and other factors, based on one or more factors such as, but not limited to, Probability can be calculated. If this probability exceeds a predetermined threshold, the calculation unit 232 may be configured to indicate that the image may present a privacy issue with respect to the third party 118.

  In some embodiments, the predetermined threshold may be determined by the image sharing network 110. Alternatively or additionally, the third party 118 can determine the predetermined threshold according to the privacy requirements of the third party 118. Alternatively or additionally, the image capture party 119 can determine the predetermined threshold according to the image capture party's 119 privacy requirements. In some embodiments, the third party 118 and / or the image capture party 119 can communicate with the data correlator 230 to send a predetermined privacy threshold over the communication line 250. If more than one party sets a predetermined threshold, the image sharing network 110 can use the lowest value, highest value, average value, median value, or some other variation thereof. is there.

  The image modifier 234 can be configured to modify the image if the probability that a third party has been captured by the image associated with the image data exceeds a predetermined threshold. In some embodiments, before modifying the image, the image modifier 234 can send a notification to the image capture party 119 that the image may have captured a third party. This notification may be sent to the image capture party 119 via the image capture device 130 or via some other electronic device associated with the image capture party 119 and / or an electronic dale address, phone number, or image capture. It can be sent to other addresses associated with party 119. This notice may request image capture party 119 to state that image capture party 119 understands that a third party may have been captured by the image, It is possible to request the image capture party 119 not to share the image. In some embodiments, the image modifier 234 modifies the image and then notifies the image capture party that the image has been modified because the image may have captured a third party image. 119 can be transmitted.

  In some embodiments, the image modifier 234 is configured to access the image in the image store 270 and modify the image by obscuring all or some of the faces in the image. can do. In some embodiments, the face of the image can be obscured by performing mosaic processing. In some embodiments, the image modifier 234 accesses the image in the image storage device 270 and removes the face in the image to prevent public display of the image, or the It is possible to modify an image by labeling the image as private.

  In some embodiments, the image modifier 234 can only obscure or remove the face of the third party 118. The third party 118 can submit an image of the face of the third party 118 to the image modifier 234 via the communication line 250. The image corrector 234 identifies the third party's 118 face in the image based on the image submitted by the third party 118 and obscures or removes only the third party's 118 face. It is possible.

  In some embodiments, the manner in which an image is modified may depend on the probability that the image has captured a third party. For example, in some embodiments, an image with a lower probability can be corrected by obscuring the face of the image, and an image with a higher probability can mark the image as private and Can be corrected by preventing sharing. In some embodiments, the image modifier 234 can correct a copy of the image that can be publicly shown. The original image can be maintained in an unmodified and private state.

  In some embodiments, various components of the image sharing network 110 may be located on another network or combined with another device. For example, in some embodiments, the image storage device 220 may be part of another network that communicates with the image sharing network 110. The image capture device 130 can send images and image data to the image analyzer 222 in the image sharing network 110. The image analyzer 222 can analyze the image and transmit the image to the image storage device 220 on another network. In some embodiments, the image capture device 130 can send the image directly to the image storage device 220 on the other network and send the image data to the image analyzer 222 for analysis. . Having an image storage device 220 on a separate network may allow the image sharing service to partition resources that handle image sharing and resources that handle third party privacy maintenance.

  As another example, in some embodiments, the image capture device 130 can maintain an image and send only image data to the image analyzer 222. If the image sharing network 110 determines that the image needs to be modified, the image modifier 234 can send a request to the image capture device 130 to properly modify the image. As another example, in some embodiments, the mobile device 120 can include a device data analyzer 212 and an analysis data store 214. The mobile device 120 can send the analysis data directly to the data correlator 230 to correlate the device data with the image data.

  As another example, in some embodiments, image sharing network 110 may include only image storage device 220, data correlator 230, calculation unit 232, and image modifier 234. Mobile device 120 may include a device data generator 122, a device data analyzer 212, and an analysis data store 214. The image capture device 130 can include an image analyzer 222 and an analyzed image data storage device 224. The image sharing network 110 receives the analyzed image data and device data from the image capture device 130 and the mobile device 120, correlates the image data with the device data, and converts the image as necessary. It can be modified appropriately.

  In some embodiments, the device data generated by the device data generator 122 and the image data may not be analyzed. Raw device data and image data are received by the image sharing network 110 and correlated by the data correlator 230. In these and other embodiments, the image sharing network 110 may not include the device data analyzer 212 or the image analyzer 222. Those skilled in the art will appreciate that various other configurations and combinations of the components described above with respect to FIGS. 1 and 2 may exist without departing from the essence of the disclosed embodiments.

  FIG. 3 illustrates one exemplary flow diagram of a method 300 that may be implemented within the image management system 100 of FIGS. 1 and 2 configured in accordance with at least some embodiments disclosed herein. Method 300 includes various operations, functions, or activities illustrated by one or more of blocks 302 and / or 304. Method 300 can begin at block 302.

  At block 302, an image capture device, such as the image capture device 130 illustrated in FIGS. 1 and 2, can capture the image. The image capture device is related to the location data that indicates where the image was captured, the time data that indicates when the image was captured, and the party that owns or operates the image capture device or, if not, the image capture device It is possible to include image data indicating a party who performs the process, but it is also possible to generate image data that is not limited thereto. In some embodiments, the image data can be input by the party that owns or operates the image capture device, or otherwise, the party associated with the image capture device. In some embodiments, the image data may be separated from the image or embedded within the image. Block 302 may be followed by block 304.

  At block 304, the image capture device may send the image data to an image sharing network, such as the image sharing network 110 illustrated in FIGS. In some embodiments, the image capture device may send the image to an image sharing network.

  One skilled in the art may appreciate that the functions performed in these processes and methods may be performed in a different order with respect to this and other processes and methods disclosed herein. Further, the outlined steps and operations are provided as examples only, and some of these steps and operations may be optional and combined without departing from the essence of the disclosed embodiments. Fewer steps and actions may be formed, or expanded to additional steps and actions.

  FIG. 4 illustrates one exemplary flow diagram of a method 400 that may be implemented within the image management system 100 of FIGS. 1 and 2 configured in accordance with at least some embodiments described herein. Method 400 includes various operations, functions, or activities as illustrated by one or more of block 402 and / or block 404. The method 400 can begin at block 402.

  In block 402, the mobile device can generate device data for the mobile device. The mobile device can include, for example, the mobile device 120 illustrated in FIGS. Device data can include location data that can indicate the current location and / or past location of the device, and time data that can indicate when the device was at those locations, It is not limited to these. Block 402 may be followed by block 404.

  At block 404, the mobile device can transmit device data to an image sharing network, such as the image sharing network 110 illustrated in FIGS.

  FIG. 5 illustrates one exemplary flow diagram of a method 500 that may be implemented within the image management system 100 of FIGS. 1 and 2 configured in accordance with at least some embodiments described herein. Method 500 may include various operations, functions, or activities as illustrated by one or more of blocks 502, 504, 506, 508, and / or 510. Method 500 can begin at block 502.

  In block 502, the image sharing network may receive image data from the image capture device via the communication network. The image capture device can include, for example, the image capture device 130 of FIGS. The image data can include image position data that can indicate the position at which the image was captured. In some embodiments, the image may be a photograph. In some embodiments, the image may be an image from a video. In some embodiments, the image data can include time data that indicates when the image was captured. Block 502 may be followed by block 504.

  At block 504, the image sharing network may receive device data from a mobile device that is separate from the image capture device via the communication network. The mobile device may be, for example, the mobile device 120 of FIGS. 1 and 2 and / or may be associated with or owned by a third party, such as the third party 118 of FIGS. Also good. The device data can include device location data that can indicate the current location and / or past location of the mobile device. In some embodiments, the device data can also include time data that can indicate when the mobile device was at a location in the device location data. Block 504 may be followed by block 506.

  At block 506, the image data and device data can be correlated. Image data and device data can be correlated by comparing the device position data with the image position data to determine if the image was captured at the approximate location where the device was located It is. In some embodiments, the image is obtained by comparing the device time data with the image time data to determine whether the image was captured in the same location at the approximate time that the device was at the location. Data and device data can be further correlated. Various algorithms can be used to correlate the image data with the device data. Block 506 may be followed by block 508.

  At block 508, the probability that the user of the device will appear in an image that can be captured by the image capture device can be calculated based on correlating the device data with the image data. Block 508 may be followed by block 510.

  At block 510, if the probability that a mobile device user will appear in the image exceeds a threshold, the image can be modified. This threshold can be predetermined by the image sharing network, the owner of the mobile device, the user of the mobile device, the submitter of the image to the image sharing network, or some combination thereof. In some embodiments, the image can be modified by removing the image from the image sharing service. Alternatively or additionally, the image can be modified by removing or obscuring a person's face in the image. Alternatively or additionally, the image can be modified by keeping the image private within the image sharing service.

  The steps and operations outlined in FIG. 5 are provided as examples only, and some of these steps and operations may be optional and combined to form fewer steps and operations, or Additional steps and operations may be extended. For example, in some embodiments, the act of calculating the probability that a user will appear in the image, as represented by block 508, may be omitted. Alternatively, in some embodiments, the operation of modifying the image, as represented by block 510, may be omitted.

  Alternatively or additionally, the method 500 may further include determining when the user of the mobile device is taking an image. Making this determination can also include determining the submitter of the image and whether the submitter of the image is known to the user of the mobile device. If the user of the mobile device is the subject of the image, the image may not be modified to protect the privacy of the user of the mobile device. This situation may occur when a friend or associate of the user of the mobile device gets the user's approval of the mobile device and captures an image of the user of the mobile device. In some embodiments, determining whether an image submitter is known to a user of the mobile device includes networking the user of the mobile device and the image submitter via social media. It may be possible to include determining whether or not.

  In some embodiments, the image sharing network can intermittently receive device data. In some embodiments, the device data may provide only intermittent device locations. As a result, the approximate location of the device during the time interval between intermittent location data may not be available from the device data. In response, the position of the mobile device can be interpolated based on device position data received from the mobile device. Interpolating the position of the mobile device can include interpolating a path between the two positions of the mobile device provided by the device data. This route can be interpolated by using road information or other information.

  An example of method 500 is as follows. A person may desire to protect his privacy by not displaying his image in an image found on a public image sharing service when that person is not the subject of the image. In order to protect his privacy, the person can provide his location data to the image sharing network. A mobile device carried by the person, such as a smartphone, can provide the position of the person to the image sharing service. The image sharing service can store the position data of the person.

  The image sharing network can also receive an image taken by a user of the image sharing service. These images can include image data such as where the image was taken and when the image was taken. The image sharing network can store the image data and compare the image data with data from the person's mobile device to determine if the image was captured at the location where the person was located. . If the image is taken at the location where the person is located, the person may be captured in the image. The image sharing network can modify the image by determining whether the probability that the person was captured in the image exceeds a predetermined threshold. The image can be modified by obscuring the person's face in the image so that the person's privacy is protected.

  FIG. 6 is a block diagram illustrating one exemplary computing device 600 configured to manage captured images in accordance with at least some embodiments described herein. The computing device 600 can include any one of the mobile device 120 and / or the image capture device 130 and / or, for example, the device data analyzer of FIG. 1 or 2 or both. Any one or more of 212, image analyzer 222, data correlator 230, calculation unit 232, or image modifier 234 can be included. In a very basic configuration 602, the computing device 600 typically includes one or more processors 604 and system memory 606. Memory bus 608 can be used to communicate between processor 604 and system memory 606.

  Depending on the desired configuration, processor 604 may be of any type including, but not limited to, a microprocessor (μP), a microcontroller (μC), a digital signal processor (DSP), or any combination thereof. It may be. The processor 604 can include one or more levels of cache, such as a level 1 cache 610 and a level 2 cache 612, a processor core 614, and a register 616. The example processor core 614 may include an arithmetic logic unit (ALU), a floating point unit (FPU), a digital signal processing core (DSP core), or any combination thereof. The example memory controller 618 can also be used with the processor 604, or in some implementations, the memory controller 618 can be an internal component of the processor 604.

  Depending on the desired configuration, the system memory 606 may include any type of volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.), or any combination thereof. It may be a thing. The system memory 606 can include an operating system (OS) 620, one or more applications 622, and program data 624. Application 622 may be a decision application 626 such as a correlation algorithm configured to correlate image data and device data as described herein, or the methods 300, 400, and 500 of FIGS. Other algorithms can be included, including those described with respect to. Application 622 may correspond to, for example, data correlator 230 of FIG. Program data 624 may include decision data 628 that may be useful for calculating the probability that a user of the device was captured in an image, such as device data, as described herein. In some embodiments, application 622 may provide program data 624 on OS 620 such that an implementation for managing an image sharing network as described in method 500 of FIG. 5 may be provided as described herein. Can be configured to work with. This described configuration 602 is illustrated in FIG. 6 by components within an internal dashed line.

  The computing device 600 may have additional features or functionality as well as additional interfaces to facilitate communication between the base configuration 602 and any required devices and interfaces. For example, the bus / interface controller 630 can be used to facilitate communication between the base configuration 602 and one or more data storage devices 632 via the storage interface bus 634. Data storage device 632 may be a removable storage device 636, a non-removable storage device 638, or a combination thereof. Examples of removable and non-removable storage devices include magnetic disk devices such as flexible disk drives and hard disk drives (HDDs), compact disk (CD) drives or digital versatile disks (to name just a few examples). Includes optical disk drives such as DVD) disks, solid state drives (SSD), and tape drives. Exemplary computer storage media include volatile and non-volatile media implemented in any method or technique for storing information, such as computer readable instructions, data structures, program modules, or other data. Possible media and non-removable media can be included.

  System memory 606, removable storage 636, and non-removable storage 638 are examples of computer storage media. Computer storage media can be used to store desired information and can be accessed by computing device 600 such as RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disc ( DVD) or other optical storage device, magnetic cassette, magnetic tape, air conditioned disk storage device or other magnetic storage device, or any other medium. Any such computer storage media may be part of computing device 600.

  Computing device 600 facilitates communication to basic configuration 602 from various interface drives (eg, output device 642, peripheral interface 644, and communication device 646) via bus / interface controller 630. Can also be included. The exemplary output device 642 includes a graphics processing unit 648 and an audio processing unit 650 that may be configured to communicate with various external devices, such as a display or speakers, via one or more A / V ports 652. including. The exemplary peripheral interface 644 may include an input device (eg, keyboard, mouse, pen, voice input device, touch input device, etc.) or other peripheral device (eg, via one or more I / O ports 658). A serial interface controller 654 or a parallel interface controller 656 that may be configured to communicate with an external device, such as a printer, scanner, etc. Exemplary communication device 646 may be configured to facilitate communication with one or more other computing devices 662 via a network communication link via one or more communication ports 664. A controller 660 is included.

  A network communication link may be an example of a communication medium. Communication media typically can be implemented by computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media. is there. A “modulated data signal” may be a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired and acoustic media, radio frequency (RF) media, microwave media, infrared (IR) media, and other wireless media such as a wired network or direct-wired connection. Wireless media can be included. As used herein, the term computer readable media can include both storage media and communication media.

  The computing device 600 can be a small form such as a cell phone, personal digital assistant (PDA), personal media player device, wireless webwatch device, personal headset device, application specific device, or hybrid device that includes any of the functions described above. It can be implemented as part of a small-form factor portable (ie, mobile) electronic device. The computing device 600 can also be implemented as a personal computer, including both laptop and non-laptop computer configurations.

  The present disclosure should not be limited in terms of the particular embodiments described herein, which are intended as illustrations of various aspects. Many modifications and variations can be made without departing from its spirit and scope, as will be apparent to those skilled in the art. In addition to the methods and apparatus listed herein, functionally equivalent methods and apparatuses within the scope of the present disclosure will be apparent to those skilled in the art from the foregoing description. Such modifications and variations are intended to be included within the scope of the appended claims. The present disclosure should be limited only in terms of the appended claims, along with the full scope of equivalents to which such claims are entitled. It should be understood that the present disclosure is, of course, not limited to specific methods, reagents, compound configurations or biological systems that may vary. It is also to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting.

  For the use of substantially all plural and / or singular terms herein, those skilled in the art will recognize from the plural to the singular and / or singular as appropriate to the situation and / or application. You can convert from shape to plural. Various singular / plural permutations can be clearly described herein for ease of understanding.

  In general, the terms used herein, particularly in the appended claims (eg, the body of the appended claims), are intended throughout as “open” terms. Will be understood by those skilled in the art (eg, the term “including” should be construed as “including but not limited to” and the term “having”). Should be interpreted as “having at least,” and the term “includes” should be interpreted as “including but not limited to”. ,Such). Where a specific number of statements is intended in the claims to be introduced, such intentions will be explicitly stated in the claims, and in the absence of such statements, such intentions It will be further appreciated by those skilled in the art that is not present. For example, as an aid to understanding, the appended claims use the introductory phrases “at least one” and “one or more” to guide the claims. May include that. However, the use of such phrases may be used even if the same claim contains indefinite articles such as the introductory phrases “one or more” or “at least one” and “a” or “an”. Embodiments in which the introduction of a claim statement by the indefinite article "a" or "an" includes any particular claim, including the claim description so introduced, is merely one such description. (Eg, “a” and / or “an” should be construed to mean “at least one” or “one or more”). Should be). The same applies to the use of definite articles used to introduce claim recitations. Further, even if a specific number is explicitly stated in the description of the claim to be introduced, it should be understood that such a description should be interpreted to mean at least the number stated. (For example, the mere description of “two descriptions” without other modifiers means at least two descriptions, or two or more descriptions). Further, in cases where a conventional expression similar to “at least one of A, B and C, etc.” is used, such syntax usually means that one skilled in the art would understand the conventional expression. Contemplated (eg, “a system having at least one of A, B, and C” includes A only, B only, C only, A and B together, A and C together, B and C together And / or systems having both A, B, and C together, etc.). In cases where a customary expression similar to “at least one of A, B, or C, etc.” is used, such syntax is usually intended in the sense that one skilled in the art would understand the customary expression. (Eg, “a system having at least one of A, B, or C” includes A only, B only, C only, A and B together, A and C together, B and C together, And / or systems having both A, B, and C together, etc.). Any disjunctive word and / or phrase that presents two or more alternative terms may be either one of the terms, anywhere in the specification, claims, or drawings. It will be further understood by those skilled in the art that it should be understood that the possibility of including either of the terms (both terms), or both of them. For example, it will be understood that the phrase “A or B” includes the possibilities of “A” or “B” or “A and B”.

  In addition, if a feature or aspect of the present disclosure is described using a Markush group, one of ordinary skill in the art will thereby be able to describe the present disclosure using any individual element or subgroup of elements. You will recognize the point.

  As understood by one of ordinary skill in the art, all ranges disclosed herein for any purpose or for any purpose, such as with respect to providing the specification, are intended to cover any conceivable subrange and all ideas. As well as combinations of the subranges as well as combinations of the subranges. Any listed range is easy to explain, making it possible enough to fully explain that the same range is broken down into at least the same half, 1/3, 1/4, 1/5, 1/10, etc. Will be recognized. As a non-limiting example, each range described herein can be easily decomposed into lower 1/3, middle 1/3, upper 1/3, and so forth. As will also be readily understood by those skilled in the art, all terms such as “at most”, “at least”, etc. include the recited numbers and are later broken down into partial ranges as discussed above. Refers to the range to obtain. Ultimately, the range includes each individual element, as will be appreciated by those skilled in the art. Thus, for example, a group having 1-3 cells refers to a group having 1 cell, a group having 2 cells, or a group having 3 cells. Similarly, a group having 1 to 5 cells is a group having 1 cell, a group having 2 cells, a group having 3 cells, a group having 4 cells, or 5 groups. A group having cells, and so on.

  From the foregoing description, various embodiments of the present disclosure have been described herein for purposes of illustration, and various modifications can be made without departing from the scope and spirit of the present disclosure. Let's understand the point. Accordingly, the various embodiments disclosed herein are not intended to be limiting, but are represented by the following claims, along with their true scope and spirit.

Claims (17)

A method for managing captured images, comprising:
Receiving image data from an image transmitting device in an image sharing network, wherein the image data is captured by a first user and the image data includes image position data regarding where the image was captured; To do
Receiving device data from a second user's device in an image sharing network, wherein the device data includes device location data;
Correlating the device data with the image data;
Responsive to the device data being correlated with the image data, calculating a probability that the second user will appear in the image based on a correlation between the device data and the image data;
Marking the image as private in response to the probability that the second user will appear in the image is greater than a first threshold;
Responsive to the probability that the second user will appear in the image is greater than a second threshold and less than or equal to the first threshold, modifying the image by obscuring portions of the image And
Including methods.   The method of claim 1, wherein the image comprises a photographic or video image.   The method of claim 1, wherein modifying the image comprises removing or obscuring the second user's face in the image.   The method of claim 3, further comprising sending a notification to the image sending device that sends the image data, wherein the notification indicates that the image has been modified. Correlating the device data with the image data;
Comparing the device position data with the image position data;
Comparing the device time data with the image time data,
The device time data represents when the device location data was collected;
The method of claim 1, wherein the image time data represents when the image was captured.   By determining the submitter of the image and determining that the submitter of the image is known to the second user of the device, the second user of the device The method according to claim 1, further comprising determining that the subject is a photographing object.   Determining that the submitter of the image is known to the second user of the device means that the second user of the device and the submitter of the image are networked via social media 7. The method of claim 6, comprising determining that   The method of claim 1, wherein the device is a mobile communication device.   The method of claim 1, further comprising receiving the image at an image sharing network, wherein modifying the image occurs after the image is received at the image sharing network.   The method of claim 1, wherein marking the image as private includes maintaining the image private in the image sharing network. A system for managing captured images,
A data correlator configured to correlate image data with device data, wherein the image data includes image position data regarding where the image was captured by a first user, and the device data includes a second A data correlator including device position data relating to the position of the user's device;
In response to the device data being correlated with the image data, a probability that the second user appears in the image is calculated based on a correlation between the device data and the image data, and the second data In response to the probability that the user of the image appears in the image is greater than a first threshold, the image is marked as private, and the probability that the second user appears in the image is greater than the second threshold and An image modifier configured to modify the image by obscuring a portion of the image in response to being less than or equal to the first threshold;
A system comprising:   The system of claim 11, wherein the image comprises a photographic or video image.   The system of claim 11, wherein the image modifier is configured to modify the image by removing or obscuring a second user's face in the image. The data correlator is configured to correlate the image position data with device position data;
Further configured to correlate device time data with image time data;
The device time data represents when the device location data was collected;
The system of claim 11, wherein the image time data represents when the image was captured.   The system of claim 11, wherein the data correlator is configured to determine when the submitter of the image is known to a second user of the device.   16. The data correlator is configured to use data from social media to determine that the submitter of the image is known to the second user of the device. The system described in. The system of claim 11, wherein the image modifier is configured to mark the image as private by classifying the image as private in an image sharing network.

Images