WO2015117672A1 - Processing a time sequence of images, allowing scene dependent image modification - Google Patents
Processing a time sequence of images, allowing scene dependent image modification Download PDFInfo
- Publication number
- WO2015117672A1 WO2015117672A1 PCT/EP2014/052471 EP2014052471W WO2015117672A1 WO 2015117672 A1 WO2015117672 A1 WO 2015117672A1 EP 2014052471 W EP2014052471 W EP 2014052471W WO 2015117672 A1 WO2015117672 A1 WO 2015117672A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- image
- images
- action
- scene recognition
- identifier
- Prior art date
Links
- 230000004048 modification Effects 0.000 title claims abstract description 19
- 238000012986 modification Methods 0.000 title claims abstract description 19
- 238000012545 processing Methods 0.000 title claims abstract description 19
- 230000001419 dependent effect Effects 0.000 title description 4
- 230000009471 action Effects 0.000 claims abstract description 52
- 230000000903 blocking effect Effects 0.000 claims abstract description 19
- 238000000034 method Methods 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 9
- 239000003086 colorant Substances 0.000 claims description 6
- 238000013500 data storage Methods 0.000 claims description 5
- 108010076504 Protein Sorting Signals Proteins 0.000 claims description 4
- 238000001514 detection method Methods 0.000 claims description 4
- 238000003384 imaging method Methods 0.000 claims description 4
- 238000013135 deep learning Methods 0.000 claims description 3
- 238000005065 mining Methods 0.000 claims description 3
- 238000012706 support-vector machine Methods 0.000 claims description 3
- 238000004891 communication Methods 0.000 description 6
- 230000008901 benefit Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000007781 pre-processing Methods 0.000 description 3
- 230000009467 reduction Effects 0.000 description 3
- 241000282472 Canis lupus familiaris Species 0.000 description 2
- 241000235646 Cyberlindnera jadinii Species 0.000 description 2
- 241000656145 Thyrsites atun Species 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 235000013305 food Nutrition 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000010365 information processing Effects 0.000 description 2
- 230000001537 neural effect Effects 0.000 description 2
- 238000003909 pattern recognition Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000001568 sexual effect Effects 0.000 description 2
- 241000196324 Embryophyta Species 0.000 description 1
- 241000593989 Scardinius erythrophthalmus Species 0.000 description 1
- 241000872198 Serjania polyphylla Species 0.000 description 1
- 241000385223 Villosa iris Species 0.000 description 1
- 238000010420 art technique Methods 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 230000009194 climbing Effects 0.000 description 1
- 238000004040 coloring Methods 0.000 description 1
- 230000021615 conjugation Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000000873 masking effect Effects 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 201000005111 ocular hyperemia Diseases 0.000 description 1
- 235000021178 picnic Nutrition 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 238000001454 recorded image Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 239000011435 rock Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/387—Composing, repositioning or otherwise geometrically modifying originals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/35—Categorising the entire scene, e.g. birthday party or wedding scene
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/00127—Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/00127—Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture
- H04N1/00326—Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture with a data reading, recognizing or recording apparatus, e.g. with a bar-code apparatus
- H04N1/00328—Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture with a data reading, recognizing or recording apparatus, e.g. with a bar-code apparatus with an apparatus processing optically-read information
- H04N1/00336—Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture with a data reading, recognizing or recording apparatus, e.g. with a bar-code apparatus with an apparatus processing optically-read information with an apparatus performing pattern recognition, e.g. of a face or a geographic feature
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/00838—Preventing unauthorised reproduction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/61—Control of cameras or camera modules based on recognised objects
- H04N23/611—Control of cameras or camera modules based on recognised objects where the recognised objects include parts of the human body
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
- H04N5/91—Television signal processing therefor
Definitions
- the invention relates to a device for processing a time sequence of images.
- Photographic filters modify recorded images. Sometimes they are used to make only subtle changes to images; other times the image would simply not be possible without them. Colouring filters affect the relative brightness of different colours; red lipstick may be rendered as anything from almost white to almost black with different filters. Others change the colour balance of images, so that photographs under incandescent lighting show colours as they are perceived, rather than with a reddish tinge. There are filters that distort the image in a desired way, diffusing an otherwise sharp image, adding a starry effect, blur or mask an image, etc.
- Photographic filters are well known as they are provided by popular apps like
- Inkwell filter if light and shadow are prominent in image
- Hefe filter if image has vibrant colors (rainbows)
- a photographic filter can be applied to the image in an interactive mode, where the user manually selects the filter that gives the best aesthetic effect. Editing a captured photograph is known for instance from EP1695548.
- An aspect of the invention is to provide new and/or more enhanced use of digital image capturing and/or displaying.
- the invention further and/or in combination allows prevention of capturing and/or displaying of unwanted types of images such as scenes displaying torture or sexual intercourse, and/or capturing and/or displaying types of images displaying child pornography, and/or classified military objects and/or, and/or capturing and/or displaying aesthetically pictures.
- the invention thus provides a device for processing a time sequence of images, said device adapted for retrieving an image from said time sequence of images from a memory, performing scene recognition on said retrieved image, and based upon the result of said scene recognition, perform an action on said image, said action selected from the group consisting of image modification comprising adapting at least part of said image, of modifying said image into a modified image, of blocking storage of said image, of blocking display of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof.
- a display device such as screens, monitors, and the like
- a camera device such as digital cameras
- pointing at a child pornography scene would not be able to record the image.
- it allows automation of image improvement and/or filtering.
- image refers to a digital image.
- image is composed of pixels that each have a digital value representing a quantity of light.
- An image can represent a picture or a photograph. It can be part of a set of subsequent images.
- Another advantage of the invention is that by understanding a scene the user is relieved from the burden where the user has to manually select a photographic filter resulting in an aesthetically improved image or video recording.
- Scene recognition comprises recognition of different types of images or videos. This became possible using computer vision and/or machine learning algorithms. Known algorithms are for example:
- scene recognition relates to processing an image.
- a setting, object, event or a combination thereof is identified.
- a label, identifier or hash is applied to the image.
- such a label or identified relates to or correlates to the result of the scene regignition.
- the algorithms allow for instance recognition of known child sexual abuse images.
- the scene recognition for instance allows:
- Fine grained recognition of leaves from hundreds of plant species or of different dog types such as shephards, afghan hounds, terriers, spaniels, American foxhounds, and so on.
- an action is performed on said image.
- said action is selected from the group consisting of scene modification comprising adapting at least part of said scene, of modifying said image into a modified image, of blocking storage of said image, of blocking display of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof.
- the family of filters describes above, and provided by popular apps, or combinations thereof, can be applied.
- the image modification algorithms can be used in real time to adapt an image. Also or in combination, The image modification algorithms can be used on a time sequence of images, for instance forming a video film being recorded.
- the action of image modification may be performed before an image or sequence of images is displayed or stored.
- the image recognition may be performed on all images that are captured and presented in a live preview, or for instance a subset of the captured images from that time sequence, and the action may be performed on each of images that is displayed in the preview.
- a server may be one server device, for instance a computer device, located at a location.
- a server may refer to at least one server device, connected via one or more data connections, at the same location and/or located at remote, in particular physically/geographically remote locations.
- an image sensor captures an image.
- an image sensor often is a CMOS device, but also other devices may be considered.
- These image sensors may also be referred to as spatial images sensors. These sensors allow capturing of one or more at least two dimensional images.
- a captured image is clocked out or read out of the image sensor, and digitised into a stream of digital values representing a digital pixel image.
- the image recording device may comprise an image processor for providing some basic processing and temporary storage of a captured image. Examples of the pre-processing comprise colour correction, white balancing, noise reduction, and even image conversion for converting and/or compressing an image into a different digital file format.
- an image, set of images or a sequence of images is stored into a memory, and may be converted for allowing to be displayed.
- the image display device may comprise a display screen, for instance an OLED panel, an LCD panel, or the like, or may comprise a projector for projecting a picture or a film on a remote screen.
- the image, set of images or sequence of images is converted.
- the image or at least a subset of the set of images or of the sequence of images is subjected to the scene recognition and identifiers are provided. Based upon an identifier, one of the actions is performed on the image or set of images of sequence of images following and/or including the image that is provided with the specific identifier. In particular, the actions are performed before an image, set of images or sequence of images is presented to a user via the panel or projector.
- Image recording and image display may be combined.
- Many image recording devices also comprise a display that allows a direct view of images while being captured in real-time.
- the display functions as a viewer, allowing a user to compose an image composition.
- the image sensor captures an image or a sequence of images. That image is then pre-processed by the image processor, and stored in a memory. Often, the captured image is also displayed on the display. There, a user may manually apply further image processing, like filtering, red-eye reduction, and the like. Scene recognition and even the action may be preformed before an image or images are provided for preview, displayed, or stored.
- the image recording device may be in a so called 'burst mode', or 'continuous capture mode', allowing a video to be captured.
- a 'burst mode' at a video frame rate images are being captured, providing a film.
- fps frames per second
- the device relates to a time sequence of images.
- An example of a time sequence of images is the recording of a film.
- Another example is a functionally live view though a viewer of a digital camera. In particular when a digital viewer is used, a functionally live sequence of images is displayed via the viewer.
- the device may for instance apply the action on each of the images that are displayed on the viewer.
- the time sequence of images may have a time base.
- the time between the images may be constant, like for instance in a film.
- the time sequence of images may also comprise subsequent bursts of images, each burst having the same of different time between subsequent bursts.
- the action comprises an action on a subset of images from said time sequence of images, said subset including said image.
- the scene recognition may for instance be done on an image.
- images that in time follow or precede the image may be processed using the action.
- the time between images that are subjected to scene recognition is relatively small, for instance small with respect to the vision capabilities of a human, for instance a time interval smaller than 0.2 seconds, and a following set of images between this time interval is processed, then an almost constant visual sequence of images is processed.
- the device is adapted for performing scene recognition on at least a subset of said time sequence of images. For instance a set of continuous images can be subjected to scene recognition. Alternatively, each n-th image can be subjected to scene recognition.
- the device allows the action to be dependent upon the result of the scene recognition.
- the device is adapted for providing an identifier based upon the result of said scene recognition.
- An identifier can be a number of letter.
- An identifier may also be another type of label, for instance allowing the application of a hash function.
- the device if said identifier matches a predefined identifier, based upon the identifier, the device performs an action on said images.
- the scene, recognised object or event changes it may be possible to also change the action in response of the change.
- the action may be selected from the group consisting of image modification comprising adapting at least part of said image, of modifying said image into a modified image, of blocking storage of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof.
- the time sequence of images is selected from the group of a sequence of live images and a sequence of images forming a video film.
- One image or all the images of the entire sequence may be subjected to scene recognition.
- the scene recognition comprises applying an algorithm selected from the group consisting of calculating the unique digital signature of an image and then matching that signature against those of other photos, of discriminative feature mining, of contour-based shape descriptors, of deep Fisher networks, of Bag of Words, of support vector machines, of deep learning, of face detection, of template matching based on the characteristic shapes and colours of objects, and a combination thereof.
- the modifying said image comprises blurring at least a part of said image. For instance, part of a scene, an object that has been recognised, or an event that has been recognised may be blurred. It may thus be possible to blur parts before displaying or before (permanent) storage. Thus, it may be possible to provide an image recorder, digital camera or computer display that cannot store or display unwanted scenes and events and/or objects within scenes.
- the action is image processing by applying photographic filters.
- these filters are filters selected from the group of Rise filter, Hudson filter, Sierra filter, Lo-Fi filter, Sutro filter, Brannan filter, Inkwell filter, Hefe filter, and a combination thereof.
- the device comprises an image sensor adapted for capturing an image, in particular said series of images forming a film, wherein said scene recognition is performed on said image, and said action is performed on said captured image, in particular before a next image is captured.
- the device comprises a data storage, wherein said device is adapted for performing said action is before storage said image in said data storage.
- data storage may comprise a hard disk, solid state disk (SSD), but may also relate to external storage, for instance remote external storage like cloud storage.
- the device comprises a display for displaying said image, wherein said device is adapted for performing said action before displaying said image.
- the invention relates to an imaging system comprising an image sensor for capturing an image, a memory for storing said image, and the device of the invention.
- the invention relates to an image display system, comprising a memory for receiving an image for displaying, a display for displaying said image, and the device of the invention.
- the invention further relates to a computer program comprising software code portions which, when running on a data processor, configure said data processor to:
- image modification comprising adapting at least part of said image, of modifying said image into a modified image, of blocking storage of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof.
- the invention further pertains to a data carrier provided with this computer program.
- the invention further pertains to a signal carrying at least part of this computer program.
- the invention further pertains to a signal sequence representing a program for being executed on a computer, said signal sequence representing this computer program.
- the invention further pertains to a method for processing a live sequence of images, said method comprising performing scene recognition on at least a set of images of said sequence of images, and based upon the result of said scene recognition, perform an action on subsequent images of said sequence of images, said action selected from the group consisting of image modification comprising adapting at least part of said image, of modifying said image into a modified image, of blocking storage of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof.
- the method further comprising providing an identifier based upon the result of said scene recognition.
- the method further comprises if said identifier matches a predefined identifier, based upon the identifier, perform an action on subsequent images of said sequence of images, said action selected from the group consisting of image modification comprising adapting at least part of said image, of modifying said image into a modified image, of blocking storage of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof.
- the invention further pertains to method o device for processing a set of images, said method comprising performing scene recognition on at least a subset of images of said set of images, and based upon the result of said scene recognition, perform an action on subsequent images of said sequence of images, said action selected from the group consisting of image modification comprising adapting at least part of said image, of modifying said image into a modified image, of blocking storage of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof.
- image modification comprising adapting at least part of said image, of modifying said image into a modified image, of blocking storage of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof.
- substantially herein, like in “substantially consists”, will be understood by and clear to a person skilled in the art.
- the term “substantially” may also include embodiments with “entirely”, “completely”, “all”, etc. Hence, in embodiments the adjective substantially may also be removed.
- the term “substantially” may also relate to 90% or higher, such as 95% or higher, especially 99% or higher, even more especially 99.5% or higher, including 100%.
- the term “comprise” includes also embodiments wherein the term “comprises” means "consists of.
- the invention further applies to an apparatus or device comprising one or more of the characterising features described in the description and/or shown in the attached drawings.
- the invention further pertains to a method or process comprising one or more of the characterising features described in the description and/or shown in the attached dr awing s .
- FIG. 1 schematically depicts a device for processing a time sequence of images.
- FIG. 2 schematically depicts an imaging system.
- FIG. 3 schematically depicts a display system.
- FIG. 4 depicts a camera applying a photographic filter on an outdoor scene.
- FIG. 5 depicts a camera applying a photographic filter on a portrait.
- FIG. 6 depicts a camera which blocks the recording of an unwanted event
- FIG. 7 depicts a screen which blocks the scene of an unwanted event
- FIG. 1 schematically depicts a device which receives digitised images through module 201.
- the image or images are a representation of scene 100. These images are stored in a temporary memory 202.
- the image or images are subjected to scene recognition in module 203.
- an identifier 205 may be provided to the images.
- a action alters the images in module 206, and/or identifier 205' prevents the altering of the images and stores the images in a temporary memory 202 which.
- the images are representing scene 100'. In this altered scene 100', parts of the scene may be blurred.
- FIG. 2 schematically depicts an imaging system which captures images through camera 200. These images represent scene 100.
- the images are stored in a temporary memory 202.
- these images are subjected to scene recognition in module 203.
- an identifier 205 may be provided to the images.
- one or more actions may be performed on the images in module 206. For instance, identifier 205' may prevent the altering of the images.
- the images may be stored in a temporary memory 202 and recorded in module 207 where the images, by then, are representing scene 100' .
- FIG. 3 schematically depicts a display system which receives digitised images through module 201. These images represent scene 100.
- the images may be stored in a temporary memory 202.
- a scene recognition is applied in module 203. Based on the result of the scene recognition in module 204 an identifier 205 may be provided to the images.
- An action may be performed on the images in module 206, and/or identifier 205' prevents the altering of the images.
- images may be stored in a temporary memory 202 and displays the images on screen 210. By then, the images may represent a scene 100'.
- FIG. 4 depicts a camera 200 which recognises an outdoor scene 101.
- the camera automatically applies a specific photographic filter on the captured images of scene 101.
- the modified images are then displayed on the viewer of camera 200 which shows the aesthetically enhanced scene 101 ' .
- Camera 200 allows for instance blurring of part of a scene. Unwanted parts of a scene can be blurred functionally life. Thus, a viewer will not be confronted with unwanted scenes. For instance, it can be prevented tha children see threatened details in a film.
- the scene recognition thus in fact each time interprets an image and identifies the unwanted part. In then allows blocking or altering or blurring, for instance, of that unwanted part. Even if such an unwanted part displaces in the scene during playing a movie of film.
- scene recognition provides for instance a interpretation of objects in their surrounding or in events and interprets them in an almost human intelligent way.
- FIG. 5 depicts a camera 200 which recognises a portrait scene 102.
- the camera automatically applies a specific photographic filter on the captured images of scene 102 and displays the modified images on the viewer of camera 200 which shows the aesthetically enhanced scene 102'.
- the camera 200 thus allows an action on a functionally live image or on a sequence of live images.
- FIG. 6 depicts a camera 200 which recognises an unwanted event 103. Next, camera 200 automatically blocks the captured images of event 103 and does not record the event on camera 200.
- FIG. 7 depicts a screen 210 which recognises an unwanted event 103, automatically erases the incoming images of event 103 and does not show the event on screen 210.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Studio Devices (AREA)
Abstract
The invention provides a device for processing a time sequence of images, said device adapted for retrieving an image from said time sequence of images from a memory, performing scene recognition on said retrieved image, and based upon the result of said scene recognition, performs an action on said image, said action selected from the group consisting of image modification comprising adapting at least part of said image, of modifying said image into a modified image, of blocking storage of said image, of blocking display of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof.
Description
PROCESSING A TIME SEQUENCE OF IMAGES, ALLOWING SCENE
DEPENDENT IMAGE MODIFICATION
Field of the invention
The invention relates to a device for processing a time sequence of images.
Background of the invention
The last ten years, capturing, processing and displaying of digital images developed. Currently, most devices allow capturing of digital images at high resolution, capturing and displaying high definition digital video at high frame rates. Most devices comprise image capturing or storing, and comprise an image processor allowing pre-processing of images, like performing noise reduction, colour adjustment, white balancing, image conversion, and other basic pre-processing. In fact, this image processing may be done on images during video recording.
Photographic filters modify recorded images. Sometimes they are used to make only subtle changes to images; other times the image would simply not be possible without them. Colouring filters affect the relative brightness of different colours; red lipstick may be rendered as anything from almost white to almost black with different filters. Others change the colour balance of images, so that photographs under incandescent lighting show colours as they are perceived, rather than with a reddish tinge. There are filters that distort the image in a desired way, diffusing an otherwise sharp image, adding a starry effect, blur or mask an image, etc.
Photographic filters are well known as they are provided by popular apps like
Instagram, Camera+, EyeEm, Hipstamatic, Aviary, and so on. These photographic filters typically adjusting locally or globally in the image the intensity, hue, saturation, contrast, colour curves per red, green or blue colour channel, apply colour lookup tables, overlay one or more masking filters such as a vignetting mask (darker edges and corners), crop the image to adjust the width and height, and add borders to the images thereby generating for example the Polaroid effect, and so on. Different filters are best applied to different types of images in order to obtain an aesthetically pleasing
picture; for instance as published at http://mashable.com/2012/07/19/instagram-filters/. Well known examples of photographic filters provided by Instagram are:
Rise filter: for close-up shots of people
Hudson filter: for outdoor photos of buildings
Sierra filter: for nature outdoor shots
Lo-Fi filter: for shots of food
Sutro filter: for photos of summer events, nights out, BBQ's, picnics
Brannan filter: if image has strong shadows
Inkwell filter: if light and shadow are prominent in image
Hefe filter: if image has vibrant colors (rainbows)
Once a user has snapped an image, a photographic filter can be applied to the image in an interactive mode, where the user manually selects the filter that gives the best aesthetic effect. Editing a captured photograph is known for instance from EP1695548.
Summary of the invention
An aspect of the invention is to provide new and/or more enhanced use of digital image capturing and/or displaying. The invention further and/or in combination allows prevention of capturing and/or displaying of unwanted types of images such as scenes displaying torture or sexual intercourse, and/or capturing and/or displaying types of images displaying child pornography, and/or classified military objects and/or, and/or capturing and/or displaying aesthetically pictures.
The invention thus provides a device for processing a time sequence of images, said device adapted for retrieving an image from said time sequence of images from a memory, performing scene recognition on said retrieved image, and based upon the result of said scene recognition, perform an action on said image, said action selected from the group consisting of image modification comprising adapting at least part of said image, of modifying said image into a modified image, of blocking storage of said image, of blocking display of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof.
By understanding the scene and/or recognising the objects within, also including recognising an event, it can be can prevented that unwanted scenes and/or objects and/or events are being displayed or even being recorded. For example, a display
device (such as screens, monitors, and the like) provided with the invention would not be able to show child pornography even though it would receive an input signal containing these images. In the same manner a camera device (such as digital cameras) pointing at a child pornography scene would not be able to record the image. Furthermore, it allows automation of image improvement and/or filtering.
In this application, image refers to a digital image. Usually, such an image is composed of pixels that each have a digital value representing a quantity of light. An image can represent a picture or a photograph. It can be part of a set of subsequent images.
Another advantage of the invention is that by understanding a scene the user is relieved from the burden where the user has to manually select a photographic filter resulting in an aesthetically improved image or video recording.
Scene recognition comprises recognition of different types of images or videos. This became possible using computer vision and/or machine learning algorithms. Known algorithms are for example:
- Calculating the unique digital signature of an image and then matching that signature against those of other photos [see, for particular embodiments, Microsoft PhotoDNA Fact Sheet December 2009, or Heo et al, "Spherical hashing", in Computer Vision Pattern Recognition Conference. 2012.];
- Discriminative feature mining [see. for particular embodiments. Bangpeng
Yao, Khoshla. Li Fei-Fie, "Combining randomisation and discrimination for finegrained image categorisation", in Computer Vision Pattern Recognition Conference, 2011.] or contour-based shape descriptors [see, for particular embodiments, Hu, Jia, Ling, Huang, "Multiscale Distance Matrix for Fast Plant Leaf Recognition", IEEE Trans, on Image Processing (T-IP), 21(11):4667-4672, 2012],
- Deep Fisher networks [see. for particular embodiments. Simonyan. Vedaldi. Zisserman, "Deep Fisher Networks for Large-Scale Image Classification", in Advances in Neural Information Processing Systems, 2013], Bag of Words/Support vector machines [see. for particular embodiments. Snoek et al, "The MediaMill TRECVID 2012 Semantic Video Search Engine," in Proceedings of the 10th TRECVID Workshop, Gaithersburg, USA, 2012],
- Deep learning [see, for particular embodiments, Krizhevsky, A., Sutskever, I. and Hinton, G. E. ImageNet Classification with Deep Convolutional Neural Networks,
Advances in Neural Information Processing 25, MIT Press, Cambridge, MA],
- Template matching based on the characteristic shapes and colors of objects [see, for particular embodiments, R. Brunelli, Template Matching Techniques in Computer Vision: Theory and Practice, Wiley]
- Face detection [see, for particular embodiments, Viola Jones, Robust Real-
Time Face Detection, International Journal of Computer Vision, 2004] and face recognition [see, for particular embodiments, R. Brunelli and T. Poggio, "Face Recognition: Features versus Templates", IEEE Trans, on PAMI, 1993]
- or a combination thereof [see. for particular embodiments. Snoek et al, "MediaMill at TRECVID 2013: Searching Concepts, Objects, Instances and Events in
Video," in Proceedings of the 11th TRECVID Workshop, Gaithersburg, USA, 2013.].
In this respect scene recognition relates to processing an image. In such processing, a setting, object, event or a combination thereof is identified. In order to process the image or images after scene recognition, in an embodiment a label, identifier or hash is applied to the image. In this respect, in an embodiment such a label or identified relates to or correlates to the result of the scene regignition.
The algorithms allow for instance recognition of known child sexual abuse images.
The scene recognition for instance allows:
Course grained recognition of scenes such as indoor or outdoor, food, people, sunsets, mountains, dogs, and so on.
Fine grained recognition of leaves from hundreds of plant species or of different dog types such as shephards, afghan hounds, terriers, spaniels, American foxhounds, and so on.
Recognition of acts or recognition of relations between objects such as a person changing a car tyre, individuals performing a wedding ceremony, a person making a sandwich, a person cleaning an appliance, a team while rock climbing.
Recognition of book covers or wine labels.
Recognition of known objects such as license plates and traffic signs.
Based upon the results of these recognition algorithms, an action is performed on said image. In an embodiment, said action is selected from the group consisting of scene modification comprising adapting at least part of said scene, of modifying said image into a modified image, of blocking storage of said image, of blocking display of
said image, of erasing said image from said memory, of encrypting said image, and a combination thereof. In an embodiment, the family of filters describes above, and provided by popular apps, or combinations thereof, can be applied. The image modification algorithms can be used in real time to adapt an image. Also or in combination, The image modification algorithms can be used on a time sequence of images, for instance forming a video film being recorded. In other or related embodiments, the action of image modification may be performed before an image or sequence of images is displayed or stored. In this respect, the image recognition may be performed on all images that are captured and presented in a live preview, or for instance a subset of the captured images from that time sequence, and the action may be performed on each of images that is displayed in the preview.
In the application, reference may be made to a server. Such a server may be one server device, for instance a computer device, located at a location. Alternatively, a server may refer to at least one server device, connected via one or more data connections, at the same location and/or located at remote, in particular physically/geographically remote locations.
In an image recording device, an image sensor captures an image. Currently, an image sensor often is a CMOS device, but also other devices may be considered. These image sensors may also be referred to as spatial images sensors. These sensors allow capturing of one or more at least two dimensional images.
In current technology, a captured image is clocked out or read out of the image sensor, and digitised into a stream of digital values representing a digital pixel image. In some case, the image recording device may comprise an image processor for providing some basic processing and temporary storage of a captured image. Examples of the pre-processing comprise colour correction, white balancing, noise reduction, and even image conversion for converting and/or compressing an image into a different digital file format.
In an image display device, an image, set of images or a sequence of images is stored into a memory, and may be converted for allowing to be displayed. The image display device may comprise a display screen, for instance an OLED panel, an LCD panel, or the like, or may comprise a projector for projecting a picture or a film on a remote screen. Often, the image, set of images or sequence of images is converted. In an embodiment of the current invention, the image or at least a subset of the set of
images or of the sequence of images is subjected to the scene recognition and identifiers are provided. Based upon an identifier, one of the actions is performed on the image or set of images of sequence of images following and/or including the image that is provided with the specific identifier. In particular, the actions are performed before an image, set of images or sequence of images is presented to a user via the panel or projector.
Image recording and image display may be combined. Many image recording devices also comprise a display that allows a direct view of images while being captured in real-time. Thus, the display functions as a viewer, allowing a user to compose an image composition. Once the uses selects, for instance shoots a picture, or films a piece of film, the image sensor captures an image or a sequence of images. That image is then pre-processed by the image processor, and stored in a memory. Often, the captured image is also displayed on the display. There, a user may manually apply further image processing, like filtering, red-eye reduction, and the like. Scene recognition and even the action may be preformed before an image or images are provided for preview, displayed, or stored.
In another mode, the image recording device may be in a so called 'burst mode', or 'continuous capture mode', allowing a video to be captured. In this 'burst mode', at a video frame rate images are being captured, providing a film. Often, such a frame rate is at least 20 frames per second (fps), in particular at least 30 fps.
The device relates to a time sequence of images. An example of a time sequence of images is the recording of a film. Another example is a functionally live view though a viewer of a digital camera. In particular when a digital viewer is used, a functionally live sequence of images is displayed via the viewer. The device may for instance apply the action on each of the images that are displayed on the viewer. The time sequence of images may have a time base. The time between the images may be constant, like for instance in a film. The time sequence of images may also comprise subsequent bursts of images, each burst having the same of different time between subsequent bursts.
In an embodiment, the action comprises an action on a subset of images from said time sequence of images, said subset including said image. The scene recognition may for instance be done on an image. Subsequently, images that in time follow or precede the image may be processed using the action. Thus, if the time between
images that are subjected to scene recognition is relatively small, for instance small with respect to the vision capabilities of a human, for instance a time interval smaller than 0.2 seconds, and a following set of images between this time interval is processed, then an almost constant visual sequence of images is processed.
In an embodiment, the device is adapted for performing scene recognition on at least a subset of said time sequence of images. For instance a set of continuous images can be subjected to scene recognition. Alternatively, each n-th image can be subjected to scene recognition.
In an embodiment, the device allows the action to be dependent upon the result of the scene recognition.
In an embodiment, the device is adapted for providing an identifier based upon the result of said scene recognition. An identifier can be a number of letter. An identifier may also be another type of label, for instance allowing the application of a hash function. In a further embodiment, if said identifier matches a predefined identifier, based upon the identifier, the device performs an action on said images. Thus, for instance, if the scene, recognised object or event changes, it may be possible to also change the action in response of the change. The action may be selected from the group consisting of image modification comprising adapting at least part of said image, of modifying said image into a modified image, of blocking storage of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof.
In an embodiment, the time sequence of images is selected from the group of a sequence of live images and a sequence of images forming a video film. One image or all the images of the entire sequence may be subjected to scene recognition.
In an embodiment, the scene recognition comprises applying an algorithm selected from the group consisting of calculating the unique digital signature of an image and then matching that signature against those of other photos, of discriminative feature mining, of contour-based shape descriptors, of deep Fisher networks, of Bag of Words, of support vector machines, of deep learning, of face detection, of template matching based on the characteristic shapes and colours of objects, and a combination thereof.
In an embodiment, the modifying said image comprises blurring at least a part of said image. For instance, part of a scene, an object that has been recognised, or an
event that has been recognised may be blurred. It may thus be possible to blur parts before displaying or before (permanent) storage. Thus, it may be possible to provide an image recorder, digital camera or computer display that cannot store or display unwanted scenes and events and/or objects within scenes.
In an embodiment, the action is image processing by applying photographic filters. As mentioned, examples of these filters are filters selected from the group of Rise filter, Hudson filter, Sierra filter, Lo-Fi filter, Sutro filter, Brannan filter, Inkwell filter, Hefe filter, and a combination thereof.
In an embodiment, the device comprises an image sensor adapted for capturing an image, in particular said series of images forming a film, wherein said scene recognition is performed on said image, and said action is performed on said captured image, in particular before a next image is captured.
In an embodiment, the device comprises a data storage, wherein said device is adapted for performing said action is before storage said image in said data storage. Such data storage may comprise a hard disk, solid state disk (SSD), but may also relate to external storage, for instance remote external storage like cloud storage.
In an embodiment, the device comprises a display for displaying said image, wherein said device is adapted for performing said action before displaying said image.
In an embodiment, the invention relates to an imaging system comprising an image sensor for capturing an image, a memory for storing said image, and the device of the invention.
In an embodiment, the invention relates to an image display system, comprising a memory for receiving an image for displaying, a display for displaying said image, and the device of the invention.
The invention further relates to a computer program comprising software code portions which, when running on a data processor, configure said data processor to:
- retrieve an image from a memory;
- perform scene recognition on said image, and
- based upon the result of said scene recognition performs an action selected from the group consisting of image modification comprising adapting at least part of said image, of modifying said image into a modified image, of blocking storage of said
image, of erasing said image from said memory, of encrypting said image, and a combination thereof.
The invention further pertains to a data carrier provided with this computer program.
The invention further pertains to a signal carrying at least part of this computer program.
The invention further pertains to a signal sequence representing a program for being executed on a computer, said signal sequence representing this computer program.
The invention further pertains to a method for processing a live sequence of images, said method comprising performing scene recognition on at least a set of images of said sequence of images, and based upon the result of said scene recognition, perform an action on subsequent images of said sequence of images, said action selected from the group consisting of image modification comprising adapting at least part of said image, of modifying said image into a modified image, of blocking storage of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof.
In an embodiment, the method further comprising providing an identifier based upon the result of said scene recognition.
In an embodiment, the method further comprises if said identifier matches a predefined identifier, based upon the identifier, perform an action on subsequent images of said sequence of images, said action selected from the group consisting of image modification comprising adapting at least part of said image, of modifying said image into a modified image, of blocking storage of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof.
The invention further pertains to method o device for processing a set of images, said method comprising performing scene recognition on at least a subset of images of said set of images, and based upon the result of said scene recognition, perform an action on subsequent images of said sequence of images, said action selected from the group consisting of image modification comprising adapting at least part of said image, of modifying said image into a modified image, of blocking storage of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof.
Thus, in this embodiment, actions on a large set of images or on a database of images can be automated.
The term "substantially" herein, like in "substantially consists", will be understood by and clear to a person skilled in the art. The term "substantially" may also include embodiments with "entirely", "completely", "all", etc. Hence, in embodiments the adjective substantially may also be removed. Where applicable, the term "substantially" may also relate to 90% or higher, such as 95% or higher, especially 99% or higher, even more especially 99.5% or higher, including 100%. The term "comprise" includes also embodiments wherein the term "comprises" means "consists of.
The term "functionally", when used for instance in "functionally coupled" or "functionally direct communication", will be understood by and clear to a person skilled in the art. The term "substantially" may also include embodiments with "entirely", "completely", "all", etc. Hence, in embodiments the adjective substantially may also be removed. Thus, for instance "functionally direct communication" comprises direct, live communication. It may also comprise communication that, from a perspective of the parties communication, is experienced as "live". Thus, like for instance voice over IP (VOIP), there may be a small amount of time between various data packages comprising digital voice data, but these amounts of time are so small that for users it seems as if there is an open communication line or telephone line available.
Furthermore, the terms first, second, third and the like in the description and in the claims, are used for distinguishing between similar elements and not necessarily for describing a sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances and that the embodiments of the invention described herein are capable of operation in other sequences than described or illustrated herein.
The devices or apparatus herein are amongst others described during operation. As will be clear to the person skilled in the art, the invention is not limited to methods of operation or devices in operation.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In
the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. Use of the verb "to comprise" and its conjugations does not exclude the presence of elements or steps other than those stated in a claim. The article "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the device or apparatus claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.
The invention further applies to an apparatus or device comprising one or more of the characterising features described in the description and/or shown in the attached drawings. The invention further pertains to a method or process comprising one or more of the characterising features described in the description and/or shown in the attached dr awing s .
The various aspects discussed in this patent can be combined in order to provide additional advantages. Furthermore, some of the features can form the basis for one or more divisional applications. Brief description of the drawings
Embodiments of the invention will now be described, by way of example only, with reference to the accompanying schematic drawings in which corresponding reference symbols indicate corresponding parts, and in which:
FIG. 1 schematically depicts a device for processing a time sequence of images. FIG. 2 schematically depicts an imaging system.
FIG. 3 schematically depicts a display system.
FIG. 4 depicts a camera applying a photographic filter on an outdoor scene. FIG. 5 depicts a camera applying a photographic filter on a portrait.
FIG. 6 depicts a camera which blocks the recording of an unwanted event FIG. 7 depicts a screen which blocks the scene of an unwanted event
The drawings are not necessarily on scale.
Description of preferred embodiments
FIG. 1 schematically depicts a device which receives digitised images through module 201. The image or images are a representation of scene 100. These images are stored in a temporary memory 202. Next, the image or images are subjected to scene recognition in module 203. Based on the result of the scene recognition in module 204, an identifier 205 may be provided to the images. A action alters the images in module 206, and/or identifier 205' prevents the altering of the images and stores the images in a temporary memory 202 which. By then, the images are representing scene 100'. In this altered scene 100', parts of the scene may be blurred.
FIG. 2 schematically depicts an imaging system which captures images through camera 200. These images represent scene 100. The images are stored in a temporary memory 202. Next, these images are subjected to scene recognition in module 203. Based on the result of the scene recognition in module 204, an identifier 205 may be provided to the images. Based upon the identifier, one or more actions may be performed on the images in module 206. For instance, identifier 205' may prevent the altering of the images. Next, the images may be stored in a temporary memory 202 and recorded in module 207 where the images, by then, are representing scene 100' .
FIG. 3 schematically depicts a display system which receives digitised images through module 201. These images represent scene 100. The images may be stored in a temporary memory 202. Next, a scene recognition is applied in module 203. Based on the result of the scene recognition in module 204 an identifier 205 may be provided to the images. An action may be performed on the images in module 206, and/or identifier 205' prevents the altering of the images. Next, images may be stored in a temporary memory 202 and displays the images on screen 210. By then, the images may represent a scene 100'.
FIG. 4 depicts a camera 200 which recognises an outdoor scene 101. The camera automatically applies a specific photographic filter on the captured images of scene 101. The modified images are then displayed on the viewer of camera 200 which shows the aesthetically enhanced scene 101 ' . Camera 200 allows for instance blurring of part of a scene. Unwanted parts of a scene can be blurred functionally life. Thus, a viewer will not be confronted with unwanted scenes. For instance, it can be prevented tha children see horrible details in a film. The scene recognition thus in fact each time interprets an image and identifies the unwanted part. In then allows blocking or altering or blurring, for instance, of that unwanted part. Even if such an unwanted part
displaces in the scene during playing a movie of film. Thus, scene recognition provides for instance a interpretation of objects in their surrounding or in events and interprets them in an almost human intelligent way.
FIG. 5 depicts a camera 200 which recognises a portrait scene 102. The camera automatically applies a specific photographic filter on the captured images of scene 102 and displays the modified images on the viewer of camera 200 which shows the aesthetically enhanced scene 102'. The camera 200 thus allows an action on a functionally live image or on a sequence of live images.
FIG. 6 depicts a camera 200 which recognises an unwanted event 103. Next, camera 200 automatically blocks the captured images of event 103 and does not record the event on camera 200.
FIG. 7 depicts a screen 210 which recognises an unwanted event 103, automatically erases the incoming images of event 103 and does not show the event on screen 210.
It will also be clear that the above description and drawings are included to illustrate some embodiments of the invention, and not to limit the scope of protection. Starting from this disclosure, many more embodiments will be evident to a skilled person. These embodiments are within the scope of protection and the essence of this invention and are obvious combinations of prior art techniques and the disclosure of this patent.
Claims
1. A device for processing a time sequence of images, said device adapted for:
- retrieving an image from said time sequence of images from a memory;
- performing scene recognition on said retrieved image, and
- based upon the result of said scene recognition, performs an action on said image, said action selected from the group consisting of image modification comprising adapting at least part of said image, of modifying said image into a modified image, of blocking storage of said image, of blocking display of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof.
2. The device of claim 1, wherein said action comprises an action on a subset of
images from said time sequence of images, said subset including said image.
3. The device of claim 1, wherein said device is adapted for performing scene
recognition on at least a subset of said time sequence of images.
4. The device of claim 1, wherein said device is adapted for:
- providing an identifier based upon the result of said scene recognition, and
- if said identifier matches a predefined identifier, based upon the identifier, perform an action on said images, said action selected from the group consisting of image modification comprising adapting at least part of said image, of modifying said image into a modified image, of blocking storage of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof.
5. The device of claim 1, wherein said time sequence of images is selected from the group of a sequence of live images and a sequence of images forming a video film.
6. The device of claim 1, wherein said scene recognition comprises applying an
algorithm selected from the group consisting of calculating the unique digital signature of an image and then matching that signature against those of other
photos, of discriminative feature mining, of contour-based shape descriptors, of deep Fisher networks, of Bag of Words, of support vector machines, of deep learning, of face detection, of template matching based on the characteristic shapes and colours of objects, and a combination thereof.
7. The device of claim 1, wherein said modifying said image comprises blurring at least a part of said image.
8. The device of claim 1, wherein said action is image processing by applying at least one photographic filter.
9. The device of claim 1, wherein said device comprises an image sensor adapted for capturing an image, in particular said series of images forming a film, wherein said scene recognition is performed on said image, and said action is performed on said captured image, in particular before a next image is captured.
10. The device of claim 1, wherein said device comprises a data storage, wherein said device is adapted for performing said action is before storage said image in said data storage.
11. The device of claim 1, wherein said device comprises a display for displaying said image, wherein said device is adapted for performing said action before displaying said image.
12. An imaging system comprising:
- an image sensor for capturing an image;
- a memory for storing said image, and
- the device of claim 1.
13. An image display system, comprising:
- a memory for receiving an image for displaying;
- a display for displaying said image, and
- the device of claim 1.
14. A computer program comprising software code portions which, when running on a data processor, configure said data processor to:
- retrieve an image from a memory;
- perform scene recognition on said image, and
- based upon the result of said scene recognition performs an action selected from the group consisting of image modification comprising adapting at least part of said image, of modifying said image into a modified image, of blocking storage of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof.
15. The computer program of claim 14, further configured to:
- provide an identifier based upon the result of said scene recognition, and
- if said identifier matches a predefined identifier, based upon the identifier, perform an action on said image, said action selected from the group consisting of image modification comprising adapting at least part of said image, of modifying said image into a modified image, of blocking storage of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof.
16. A data carrier provided with the computer program of claim 14.
17. A signal carrying at least part of said computer program of claim 14.
18. A signal sequence representing a program for being executed on a computer, said signal sequence representing said computer program of claim 14.
19. A method for processing a live sequence of images, said method comprising:
- performing scene recognition on at least a set of images of said sequence of images, and
- based upon the result of said scene recognition, perform an action on subsequent images of said sequence of images, said action selected from the group consisting
of image modification comprising adapting at least part of said image, of modifying said image into a modified image, of blocking storage of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof.
20. The method of claim 19, further comprising:
- providing an identifier based upon the result of said scene recognition, and
- if said identifier matches a predefined identifier, based upon the identifier, perform an action on subsequent images of said sequence of images, said action selected from the group consisting of image modification comprising adapting at least part of said image, of modifying said image into a modified image, of blocking storage of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof.
-o-o-o-o-o-
Priority Applications (10)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/EP2014/052471 WO2015117672A1 (en) | 2014-02-07 | 2014-02-07 | Processing a time sequence of images, allowing scene dependent image modification |
EP14771516.3A EP3103117A1 (en) | 2014-02-07 | 2014-07-03 | Live scene recognition allowing scene dependent image modification before image recording or display |
CN202010152403.3A CN111326183A (en) | 2014-02-07 | 2014-07-03 | System and method for processing a temporal image sequence |
PCT/EP2014/064269 WO2015117681A1 (en) | 2014-02-07 | 2014-07-03 | Live scene recognition allowing scene dependent image modification before image recording or display |
JP2016550545A JP6162345B2 (en) | 2014-02-07 | 2014-07-03 | Raw scene recognition that allows scene-dependent image modification before image recording or display |
KR1020167022241A KR101765428B1 (en) | 2014-02-07 | 2014-07-03 | Live scene recognition allowing scene dependent image modification before image recording or display |
CN201480074872.0A CN106165017A (en) | 2014-02-07 | 2014-07-03 | Allow to carry out the instant scene Recognition of scene associated picture amendment before image record or display |
BR112016018024A BR112016018024A2 (en) | 2014-02-07 | 2014-07-03 | LIVE SCENE RECOGNITION ALLOWS SCENE DEPENDENT IMAGE MODIFICATION BEFORE RECORDING OR IMAGE DISPLAY |
US14/616,634 US9426385B2 (en) | 2014-02-07 | 2015-02-06 | Image processing based on scene recognition |
TW104104278A TWI578782B (en) | 2014-02-07 | 2015-02-09 | Image processing based on scene recognition |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/EP2014/052471 WO2015117672A1 (en) | 2014-02-07 | 2014-02-07 | Processing a time sequence of images, allowing scene dependent image modification |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2015117672A1 true WO2015117672A1 (en) | 2015-08-13 |
Family
ID=50193446
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/EP2014/052471 WO2015117672A1 (en) | 2014-02-07 | 2014-02-07 | Processing a time sequence of images, allowing scene dependent image modification |
Country Status (1)
Country | Link |
---|---|
WO (1) | WO2015117672A1 (en) |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060023077A1 (en) * | 2004-07-30 | 2006-02-02 | Microsoft Corporation | System and method for photo editing |
US20070297641A1 (en) * | 2006-06-27 | 2007-12-27 | Microsoft Corporation | Controlling content suitability by selectively obscuring |
-
2014
- 2014-02-07 WO PCT/EP2014/052471 patent/WO2015117672A1/en active Application Filing
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060023077A1 (en) * | 2004-07-30 | 2006-02-02 | Microsoft Corporation | System and method for photo editing |
EP1695548A2 (en) | 2004-07-30 | 2006-08-30 | Microsoft Corporation | System and method for photo editing |
US20070297641A1 (en) * | 2006-06-27 | 2007-12-27 | Microsoft Corporation | Controlling content suitability by selectively obscuring |
Non-Patent Citations (11)
Title |
---|
"Deep Fisher Networks for Large-Scale Image Classification", ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS, 2013 |
BANGPENG YAO; KHOSHLA, LI FEI-FIE: "Combining randomisation and discrimination for fine- grained image categorisation", COMPUTER VISION PATTERN RECOGNITION CONFERENCE, 2011 |
HEO ET AL.: "Spherical hashing", COMPUTER VISION PATTERN RECOGNITION CONFERENCE, 2012 |
HU, JIA; LING, HUANG: "Multiscale Distance Matrix for Fast Plant Leaf Recognition", IEEE TRANS. ON IMAGE PROCESSING (T-IP, vol. 21, no. 11, 2012, pages 4667 - 4672 |
KRIZHEVSKY, A.; SUTSKEVER, I.; HINTON, G. E.: "ImageNet Classification with Deep Convolutional Neural Networks, Advances in Neural Information Processing 25", MIT PRESS |
MICROSOFT PHOTODNA FACT SHEET DECEMBER, 2009 |
R. BRUNELLI: "Template Matching Techniques in Computer Vision: Theory and Practice", WILEY |
R. BRUNELLI; T. POGGIO: "Face Recognition: Features versus Templates", IEEE TRANS. ON PAMI, 1993 |
SNOEK ET AL.: "MediaMill at TRECVID 2013: Searching Concepts, Objects, Instances and Events in Video", PROCEEDINGS OF THE I ITH TRECVID WORKSHOP, 2013 |
SNOEK ET AL.: "The MediaMill TRECVID 2012 Semantic Video Search Engine", PROCEEDINGS OF THE 10TH TRECVID WORKSHOP, 2012 |
VIOLA JONES: "Robust Real-Time Face Detection", INTERNATIONAL JOURNAL OF COMPUTER VISION, 2004 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9426385B2 (en) | Image processing based on scene recognition | |
KR101765428B1 (en) | Live scene recognition allowing scene dependent image modification before image recording or display | |
US11995530B2 (en) | Systems and methods for providing feedback for artificial intelligence-based image capture devices | |
US9692964B2 (en) | Modification of post-viewing parameters for digital images using image region or feature information | |
KR101688352B1 (en) | Recommending transformations for photography | |
US8212911B2 (en) | Imaging apparatus, imaging system, and imaging method displaying recommendation information | |
US9129381B2 (en) | Modification of post-viewing parameters for digital images using image region or feature information | |
JP6340347B2 (en) | Image processing apparatus, image processing method, program, and recording medium | |
US8675991B2 (en) | Modification of post-viewing parameters for digital images using region or feature information | |
US10728510B2 (en) | Dynamic chroma key for video background replacement | |
US20130301918A1 (en) | System, platform, application and method for automated video foreground and/or background replacement | |
WO2016019770A1 (en) | Method, device and storage medium for picture synthesis | |
US9195880B1 (en) | Interactive viewer for image stacks | |
US20060197851A1 (en) | Positioning a subject with respect to a background scene in a digital camera | |
CN103916597A (en) | Shooting method based on simulated scenario | |
US12010433B2 (en) | Image processing apparatus, image processing method, and storage medium | |
JP2016189507A (en) | Image processing device, image processing method, program and recording medium | |
WO2015117672A1 (en) | Processing a time sequence of images, allowing scene dependent image modification | |
CN106201251B (en) | A kind of content of augmented reality determines method, device and mobile terminal | |
LeGendre et al. | Improved chromakey of hair strands via orientation filter convolution | |
Grogan | Techniques to Detect Modified Video | |
JP2005210366A (en) | Photographing support system and photographing support method | |
CN111164961A (en) | Image processing system and image processing method of intelligent terminal | |
WO2019023966A1 (en) | Photography system and photography method for intelligent terminal | |
Lantz | High Dynamic Range Photography |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 14707664 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 14707664 Country of ref document: EP Kind code of ref document: A1 |