WO2015117672A1 - Processing a time sequence of images, allowing scene dependent image modification - Google Patents

Processing a time sequence of images, allowing scene dependent image modification Download PDF

Info

Publication number
WO2015117672A1
WO2015117672A1 PCT/EP2014/052471 EP2014052471W WO2015117672A1 WO 2015117672 A1 WO2015117672 A1 WO 2015117672A1 EP 2014052471 W EP2014052471 W EP 2014052471W WO 2015117672 A1 WO2015117672 A1 WO 2015117672A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
images
action
scene recognition
identifier
Prior art date
Application number
PCT/EP2014/052471
Other languages
French (fr)
Inventor
Henricus Meinardus Gerardus STOKMAN
Original Assignee
Euclid Vision Technologies B.V.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Euclid Vision Technologies B.V. filed Critical Euclid Vision Technologies B.V.
Priority to PCT/EP2014/052471 priority Critical patent/WO2015117672A1/en
Priority to EP14771516.3A priority patent/EP3103117A1/en
Priority to CN202010152403.3A priority patent/CN111326183A/en
Priority to PCT/EP2014/064269 priority patent/WO2015117681A1/en
Priority to JP2016550545A priority patent/JP6162345B2/en
Priority to KR1020167022241A priority patent/KR101765428B1/en
Priority to CN201480074872.0A priority patent/CN106165017A/en
Priority to BR112016018024A priority patent/BR112016018024A2/en
Priority to US14/616,634 priority patent/US9426385B2/en
Priority to TW104104278A priority patent/TWI578782B/en
Publication of WO2015117672A1 publication Critical patent/WO2015117672A1/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/387Composing, repositioning or otherwise geometrically modifying originals
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/35Categorising the entire scene, e.g. birthday party or wedding scene
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/00127Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/00127Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture
    • H04N1/00326Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture with a data reading, recognizing or recording apparatus, e.g. with a bar-code apparatus
    • H04N1/00328Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture with a data reading, recognizing or recording apparatus, e.g. with a bar-code apparatus with an apparatus processing optically-read information
    • H04N1/00336Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture with a data reading, recognizing or recording apparatus, e.g. with a bar-code apparatus with an apparatus processing optically-read information with an apparatus performing pattern recognition, e.g. of a face or a geographic feature
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/00838Preventing unauthorised reproduction
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60Control of cameras or camera modules
    • H04N23/61Control of cameras or camera modules based on recognised objects
    • H04N23/611Control of cameras or camera modules based on recognised objects where the recognised objects include parts of the human body
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80Camera processing pipelines; Components thereof
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/76Television signal recording
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/76Television signal recording
    • H04N5/91Television signal processing therefor

Definitions

  • the invention relates to a device for processing a time sequence of images.
  • Photographic filters modify recorded images. Sometimes they are used to make only subtle changes to images; other times the image would simply not be possible without them. Colouring filters affect the relative brightness of different colours; red lipstick may be rendered as anything from almost white to almost black with different filters. Others change the colour balance of images, so that photographs under incandescent lighting show colours as they are perceived, rather than with a reddish tinge. There are filters that distort the image in a desired way, diffusing an otherwise sharp image, adding a starry effect, blur or mask an image, etc.
  • Photographic filters are well known as they are provided by popular apps like
  • Inkwell filter if light and shadow are prominent in image
  • Hefe filter if image has vibrant colors (rainbows)
  • a photographic filter can be applied to the image in an interactive mode, where the user manually selects the filter that gives the best aesthetic effect. Editing a captured photograph is known for instance from EP1695548.
  • An aspect of the invention is to provide new and/or more enhanced use of digital image capturing and/or displaying.
  • the invention further and/or in combination allows prevention of capturing and/or displaying of unwanted types of images such as scenes displaying torture or sexual intercourse, and/or capturing and/or displaying types of images displaying child pornography, and/or classified military objects and/or, and/or capturing and/or displaying aesthetically pictures.
  • the invention thus provides a device for processing a time sequence of images, said device adapted for retrieving an image from said time sequence of images from a memory, performing scene recognition on said retrieved image, and based upon the result of said scene recognition, perform an action on said image, said action selected from the group consisting of image modification comprising adapting at least part of said image, of modifying said image into a modified image, of blocking storage of said image, of blocking display of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof.
  • a display device such as screens, monitors, and the like
  • a camera device such as digital cameras
  • pointing at a child pornography scene would not be able to record the image.
  • it allows automation of image improvement and/or filtering.
  • image refers to a digital image.
  • image is composed of pixels that each have a digital value representing a quantity of light.
  • An image can represent a picture or a photograph. It can be part of a set of subsequent images.
  • Another advantage of the invention is that by understanding a scene the user is relieved from the burden where the user has to manually select a photographic filter resulting in an aesthetically improved image or video recording.
  • Scene recognition comprises recognition of different types of images or videos. This became possible using computer vision and/or machine learning algorithms. Known algorithms are for example:
  • scene recognition relates to processing an image.
  • a setting, object, event or a combination thereof is identified.
  • a label, identifier or hash is applied to the image.
  • such a label or identified relates to or correlates to the result of the scene regignition.
  • the algorithms allow for instance recognition of known child sexual abuse images.
  • the scene recognition for instance allows:
  • Fine grained recognition of leaves from hundreds of plant species or of different dog types such as shephards, afghan hounds, terriers, spaniels, American foxhounds, and so on.
  • an action is performed on said image.
  • said action is selected from the group consisting of scene modification comprising adapting at least part of said scene, of modifying said image into a modified image, of blocking storage of said image, of blocking display of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof.
  • the family of filters describes above, and provided by popular apps, or combinations thereof, can be applied.
  • the image modification algorithms can be used in real time to adapt an image. Also or in combination, The image modification algorithms can be used on a time sequence of images, for instance forming a video film being recorded.
  • the action of image modification may be performed before an image or sequence of images is displayed or stored.
  • the image recognition may be performed on all images that are captured and presented in a live preview, or for instance a subset of the captured images from that time sequence, and the action may be performed on each of images that is displayed in the preview.
  • a server may be one server device, for instance a computer device, located at a location.
  • a server may refer to at least one server device, connected via one or more data connections, at the same location and/or located at remote, in particular physically/geographically remote locations.
  • an image sensor captures an image.
  • an image sensor often is a CMOS device, but also other devices may be considered.
  • These image sensors may also be referred to as spatial images sensors. These sensors allow capturing of one or more at least two dimensional images.
  • a captured image is clocked out or read out of the image sensor, and digitised into a stream of digital values representing a digital pixel image.
  • the image recording device may comprise an image processor for providing some basic processing and temporary storage of a captured image. Examples of the pre-processing comprise colour correction, white balancing, noise reduction, and even image conversion for converting and/or compressing an image into a different digital file format.
  • an image, set of images or a sequence of images is stored into a memory, and may be converted for allowing to be displayed.
  • the image display device may comprise a display screen, for instance an OLED panel, an LCD panel, or the like, or may comprise a projector for projecting a picture or a film on a remote screen.
  • the image, set of images or sequence of images is converted.
  • the image or at least a subset of the set of images or of the sequence of images is subjected to the scene recognition and identifiers are provided. Based upon an identifier, one of the actions is performed on the image or set of images of sequence of images following and/or including the image that is provided with the specific identifier. In particular, the actions are performed before an image, set of images or sequence of images is presented to a user via the panel or projector.
  • Image recording and image display may be combined.
  • Many image recording devices also comprise a display that allows a direct view of images while being captured in real-time.
  • the display functions as a viewer, allowing a user to compose an image composition.
  • the image sensor captures an image or a sequence of images. That image is then pre-processed by the image processor, and stored in a memory. Often, the captured image is also displayed on the display. There, a user may manually apply further image processing, like filtering, red-eye reduction, and the like. Scene recognition and even the action may be preformed before an image or images are provided for preview, displayed, or stored.
  • the image recording device may be in a so called 'burst mode', or 'continuous capture mode', allowing a video to be captured.
  • a 'burst mode' at a video frame rate images are being captured, providing a film.
  • fps frames per second
  • the device relates to a time sequence of images.
  • An example of a time sequence of images is the recording of a film.
  • Another example is a functionally live view though a viewer of a digital camera. In particular when a digital viewer is used, a functionally live sequence of images is displayed via the viewer.
  • the device may for instance apply the action on each of the images that are displayed on the viewer.
  • the time sequence of images may have a time base.
  • the time between the images may be constant, like for instance in a film.
  • the time sequence of images may also comprise subsequent bursts of images, each burst having the same of different time between subsequent bursts.
  • the action comprises an action on a subset of images from said time sequence of images, said subset including said image.
  • the scene recognition may for instance be done on an image.
  • images that in time follow or precede the image may be processed using the action.
  • the time between images that are subjected to scene recognition is relatively small, for instance small with respect to the vision capabilities of a human, for instance a time interval smaller than 0.2 seconds, and a following set of images between this time interval is processed, then an almost constant visual sequence of images is processed.
  • the device is adapted for performing scene recognition on at least a subset of said time sequence of images. For instance a set of continuous images can be subjected to scene recognition. Alternatively, each n-th image can be subjected to scene recognition.
  • the device allows the action to be dependent upon the result of the scene recognition.
  • the device is adapted for providing an identifier based upon the result of said scene recognition.
  • An identifier can be a number of letter.
  • An identifier may also be another type of label, for instance allowing the application of a hash function.
  • the device if said identifier matches a predefined identifier, based upon the identifier, the device performs an action on said images.
  • the scene, recognised object or event changes it may be possible to also change the action in response of the change.
  • the action may be selected from the group consisting of image modification comprising adapting at least part of said image, of modifying said image into a modified image, of blocking storage of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof.
  • the time sequence of images is selected from the group of a sequence of live images and a sequence of images forming a video film.
  • One image or all the images of the entire sequence may be subjected to scene recognition.
  • the scene recognition comprises applying an algorithm selected from the group consisting of calculating the unique digital signature of an image and then matching that signature against those of other photos, of discriminative feature mining, of contour-based shape descriptors, of deep Fisher networks, of Bag of Words, of support vector machines, of deep learning, of face detection, of template matching based on the characteristic shapes and colours of objects, and a combination thereof.
  • the modifying said image comprises blurring at least a part of said image. For instance, part of a scene, an object that has been recognised, or an event that has been recognised may be blurred. It may thus be possible to blur parts before displaying or before (permanent) storage. Thus, it may be possible to provide an image recorder, digital camera or computer display that cannot store or display unwanted scenes and events and/or objects within scenes.
  • the action is image processing by applying photographic filters.
  • these filters are filters selected from the group of Rise filter, Hudson filter, Sierra filter, Lo-Fi filter, Sutro filter, Brannan filter, Inkwell filter, Hefe filter, and a combination thereof.
  • the device comprises an image sensor adapted for capturing an image, in particular said series of images forming a film, wherein said scene recognition is performed on said image, and said action is performed on said captured image, in particular before a next image is captured.
  • the device comprises a data storage, wherein said device is adapted for performing said action is before storage said image in said data storage.
  • data storage may comprise a hard disk, solid state disk (SSD), but may also relate to external storage, for instance remote external storage like cloud storage.
  • the device comprises a display for displaying said image, wherein said device is adapted for performing said action before displaying said image.
  • the invention relates to an imaging system comprising an image sensor for capturing an image, a memory for storing said image, and the device of the invention.
  • the invention relates to an image display system, comprising a memory for receiving an image for displaying, a display for displaying said image, and the device of the invention.
  • the invention further relates to a computer program comprising software code portions which, when running on a data processor, configure said data processor to:
  • image modification comprising adapting at least part of said image, of modifying said image into a modified image, of blocking storage of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof.
  • the invention further pertains to a data carrier provided with this computer program.
  • the invention further pertains to a signal carrying at least part of this computer program.
  • the invention further pertains to a signal sequence representing a program for being executed on a computer, said signal sequence representing this computer program.
  • the invention further pertains to a method for processing a live sequence of images, said method comprising performing scene recognition on at least a set of images of said sequence of images, and based upon the result of said scene recognition, perform an action on subsequent images of said sequence of images, said action selected from the group consisting of image modification comprising adapting at least part of said image, of modifying said image into a modified image, of blocking storage of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof.
  • the method further comprising providing an identifier based upon the result of said scene recognition.
  • the method further comprises if said identifier matches a predefined identifier, based upon the identifier, perform an action on subsequent images of said sequence of images, said action selected from the group consisting of image modification comprising adapting at least part of said image, of modifying said image into a modified image, of blocking storage of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof.
  • the invention further pertains to method o device for processing a set of images, said method comprising performing scene recognition on at least a subset of images of said set of images, and based upon the result of said scene recognition, perform an action on subsequent images of said sequence of images, said action selected from the group consisting of image modification comprising adapting at least part of said image, of modifying said image into a modified image, of blocking storage of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof.
  • image modification comprising adapting at least part of said image, of modifying said image into a modified image, of blocking storage of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof.
  • substantially herein, like in “substantially consists”, will be understood by and clear to a person skilled in the art.
  • the term “substantially” may also include embodiments with “entirely”, “completely”, “all”, etc. Hence, in embodiments the adjective substantially may also be removed.
  • the term “substantially” may also relate to 90% or higher, such as 95% or higher, especially 99% or higher, even more especially 99.5% or higher, including 100%.
  • the term “comprise” includes also embodiments wherein the term “comprises” means "consists of.
  • the invention further applies to an apparatus or device comprising one or more of the characterising features described in the description and/or shown in the attached drawings.
  • the invention further pertains to a method or process comprising one or more of the characterising features described in the description and/or shown in the attached dr awing s .
  • FIG. 1 schematically depicts a device for processing a time sequence of images.
  • FIG. 2 schematically depicts an imaging system.
  • FIG. 3 schematically depicts a display system.
  • FIG. 4 depicts a camera applying a photographic filter on an outdoor scene.
  • FIG. 5 depicts a camera applying a photographic filter on a portrait.
  • FIG. 6 depicts a camera which blocks the recording of an unwanted event
  • FIG. 7 depicts a screen which blocks the scene of an unwanted event
  • FIG. 1 schematically depicts a device which receives digitised images through module 201.
  • the image or images are a representation of scene 100. These images are stored in a temporary memory 202.
  • the image or images are subjected to scene recognition in module 203.
  • an identifier 205 may be provided to the images.
  • a action alters the images in module 206, and/or identifier 205' prevents the altering of the images and stores the images in a temporary memory 202 which.
  • the images are representing scene 100'. In this altered scene 100', parts of the scene may be blurred.
  • FIG. 2 schematically depicts an imaging system which captures images through camera 200. These images represent scene 100.
  • the images are stored in a temporary memory 202.
  • these images are subjected to scene recognition in module 203.
  • an identifier 205 may be provided to the images.
  • one or more actions may be performed on the images in module 206. For instance, identifier 205' may prevent the altering of the images.
  • the images may be stored in a temporary memory 202 and recorded in module 207 where the images, by then, are representing scene 100' .
  • FIG. 3 schematically depicts a display system which receives digitised images through module 201. These images represent scene 100.
  • the images may be stored in a temporary memory 202.
  • a scene recognition is applied in module 203. Based on the result of the scene recognition in module 204 an identifier 205 may be provided to the images.
  • An action may be performed on the images in module 206, and/or identifier 205' prevents the altering of the images.
  • images may be stored in a temporary memory 202 and displays the images on screen 210. By then, the images may represent a scene 100'.
  • FIG. 4 depicts a camera 200 which recognises an outdoor scene 101.
  • the camera automatically applies a specific photographic filter on the captured images of scene 101.
  • the modified images are then displayed on the viewer of camera 200 which shows the aesthetically enhanced scene 101 ' .
  • Camera 200 allows for instance blurring of part of a scene. Unwanted parts of a scene can be blurred functionally life. Thus, a viewer will not be confronted with unwanted scenes. For instance, it can be prevented tha children see threatened details in a film.
  • the scene recognition thus in fact each time interprets an image and identifies the unwanted part. In then allows blocking or altering or blurring, for instance, of that unwanted part. Even if such an unwanted part displaces in the scene during playing a movie of film.
  • scene recognition provides for instance a interpretation of objects in their surrounding or in events and interprets them in an almost human intelligent way.
  • FIG. 5 depicts a camera 200 which recognises a portrait scene 102.
  • the camera automatically applies a specific photographic filter on the captured images of scene 102 and displays the modified images on the viewer of camera 200 which shows the aesthetically enhanced scene 102'.
  • the camera 200 thus allows an action on a functionally live image or on a sequence of live images.
  • FIG. 6 depicts a camera 200 which recognises an unwanted event 103. Next, camera 200 automatically blocks the captured images of event 103 and does not record the event on camera 200.
  • FIG. 7 depicts a screen 210 which recognises an unwanted event 103, automatically erases the incoming images of event 103 and does not show the event on screen 210.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Security & Cryptography (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Studio Devices (AREA)

Abstract

The invention provides a device for processing a time sequence of images, said device adapted for retrieving an image from said time sequence of images from a memory, performing scene recognition on said retrieved image, and based upon the result of said scene recognition, performs an action on said image, said action selected from the group consisting of image modification comprising adapting at least part of said image, of modifying said image into a modified image, of blocking storage of said image, of blocking display of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof.

Description

PROCESSING A TIME SEQUENCE OF IMAGES, ALLOWING SCENE
DEPENDENT IMAGE MODIFICATION
Field of the invention
The invention relates to a device for processing a time sequence of images.
Background of the invention
The last ten years, capturing, processing and displaying of digital images developed. Currently, most devices allow capturing of digital images at high resolution, capturing and displaying high definition digital video at high frame rates. Most devices comprise image capturing or storing, and comprise an image processor allowing pre-processing of images, like performing noise reduction, colour adjustment, white balancing, image conversion, and other basic pre-processing. In fact, this image processing may be done on images during video recording.
Photographic filters modify recorded images. Sometimes they are used to make only subtle changes to images; other times the image would simply not be possible without them. Colouring filters affect the relative brightness of different colours; red lipstick may be rendered as anything from almost white to almost black with different filters. Others change the colour balance of images, so that photographs under incandescent lighting show colours as they are perceived, rather than with a reddish tinge. There are filters that distort the image in a desired way, diffusing an otherwise sharp image, adding a starry effect, blur or mask an image, etc.
Photographic filters are well known as they are provided by popular apps like
Instagram, Camera+, EyeEm, Hipstamatic, Aviary, and so on. These photographic filters typically adjusting locally or globally in the image the intensity, hue, saturation, contrast, colour curves per red, green or blue colour channel, apply colour lookup tables, overlay one or more masking filters such as a vignetting mask (darker edges and corners), crop the image to adjust the width and height, and add borders to the images thereby generating for example the Polaroid effect, and so on. Different filters are best applied to different types of images in order to obtain an aesthetically pleasing picture; for instance as published at http://mashable.com/2012/07/19/instagram-filters/. Well known examples of photographic filters provided by Instagram are:
Rise filter: for close-up shots of people
Hudson filter: for outdoor photos of buildings
Sierra filter: for nature outdoor shots
Lo-Fi filter: for shots of food
Sutro filter: for photos of summer events, nights out, BBQ's, picnics
Brannan filter: if image has strong shadows
Inkwell filter: if light and shadow are prominent in image
Hefe filter: if image has vibrant colors (rainbows)
Once a user has snapped an image, a photographic filter can be applied to the image in an interactive mode, where the user manually selects the filter that gives the best aesthetic effect. Editing a captured photograph is known for instance from EP1695548.
Summary of the invention
An aspect of the invention is to provide new and/or more enhanced use of digital image capturing and/or displaying. The invention further and/or in combination allows prevention of capturing and/or displaying of unwanted types of images such as scenes displaying torture or sexual intercourse, and/or capturing and/or displaying types of images displaying child pornography, and/or classified military objects and/or, and/or capturing and/or displaying aesthetically pictures.
The invention thus provides a device for processing a time sequence of images, said device adapted for retrieving an image from said time sequence of images from a memory, performing scene recognition on said retrieved image, and based upon the result of said scene recognition, perform an action on said image, said action selected from the group consisting of image modification comprising adapting at least part of said image, of modifying said image into a modified image, of blocking storage of said image, of blocking display of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof.
By understanding the scene and/or recognising the objects within, also including recognising an event, it can be can prevented that unwanted scenes and/or objects and/or events are being displayed or even being recorded. For example, a display device (such as screens, monitors, and the like) provided with the invention would not be able to show child pornography even though it would receive an input signal containing these images. In the same manner a camera device (such as digital cameras) pointing at a child pornography scene would not be able to record the image. Furthermore, it allows automation of image improvement and/or filtering.
In this application, image refers to a digital image. Usually, such an image is composed of pixels that each have a digital value representing a quantity of light. An image can represent a picture or a photograph. It can be part of a set of subsequent images.
Another advantage of the invention is that by understanding a scene the user is relieved from the burden where the user has to manually select a photographic filter resulting in an aesthetically improved image or video recording.
Scene recognition comprises recognition of different types of images or videos. This became possible using computer vision and/or machine learning algorithms. Known algorithms are for example:
- Calculating the unique digital signature of an image and then matching that signature against those of other photos [see, for particular embodiments, Microsoft PhotoDNA Fact Sheet December 2009, or Heo et al, "Spherical hashing", in Computer Vision Pattern Recognition Conference. 2012.];
- Discriminative feature mining [see. for particular embodiments. Bangpeng
Yao, Khoshla. Li Fei-Fie, "Combining randomisation and discrimination for finegrained image categorisation", in Computer Vision Pattern Recognition Conference, 2011.] or contour-based shape descriptors [see, for particular embodiments, Hu, Jia, Ling, Huang, "Multiscale Distance Matrix for Fast Plant Leaf Recognition", IEEE Trans, on Image Processing (T-IP), 21(11):4667-4672, 2012],
- Deep Fisher networks [see. for particular embodiments. Simonyan. Vedaldi. Zisserman, "Deep Fisher Networks for Large-Scale Image Classification", in Advances in Neural Information Processing Systems, 2013], Bag of Words/Support vector machines [see. for particular embodiments. Snoek et al, "The MediaMill TRECVID 2012 Semantic Video Search Engine," in Proceedings of the 10th TRECVID Workshop, Gaithersburg, USA, 2012],
- Deep learning [see, for particular embodiments, Krizhevsky, A., Sutskever, I. and Hinton, G. E. ImageNet Classification with Deep Convolutional Neural Networks, Advances in Neural Information Processing 25, MIT Press, Cambridge, MA],
- Template matching based on the characteristic shapes and colors of objects [see, for particular embodiments, R. Brunelli, Template Matching Techniques in Computer Vision: Theory and Practice, Wiley]
- Face detection [see, for particular embodiments, Viola Jones, Robust Real-
Time Face Detection, International Journal of Computer Vision, 2004] and face recognition [see, for particular embodiments, R. Brunelli and T. Poggio, "Face Recognition: Features versus Templates", IEEE Trans, on PAMI, 1993]
- or a combination thereof [see. for particular embodiments. Snoek et al, "MediaMill at TRECVID 2013: Searching Concepts, Objects, Instances and Events in
Video," in Proceedings of the 11th TRECVID Workshop, Gaithersburg, USA, 2013.].
In this respect scene recognition relates to processing an image. In such processing, a setting, object, event or a combination thereof is identified. In order to process the image or images after scene recognition, in an embodiment a label, identifier or hash is applied to the image. In this respect, in an embodiment such a label or identified relates to or correlates to the result of the scene regignition.
The algorithms allow for instance recognition of known child sexual abuse images.
The scene recognition for instance allows:
Course grained recognition of scenes such as indoor or outdoor, food, people, sunsets, mountains, dogs, and so on.
Fine grained recognition of leaves from hundreds of plant species or of different dog types such as shephards, afghan hounds, terriers, spaniels, American foxhounds, and so on.
Recognition of acts or recognition of relations between objects such as a person changing a car tyre, individuals performing a wedding ceremony, a person making a sandwich, a person cleaning an appliance, a team while rock climbing.
Recognition of book covers or wine labels.
Recognition of known objects such as license plates and traffic signs.
Based upon the results of these recognition algorithms, an action is performed on said image. In an embodiment, said action is selected from the group consisting of scene modification comprising adapting at least part of said scene, of modifying said image into a modified image, of blocking storage of said image, of blocking display of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof. In an embodiment, the family of filters describes above, and provided by popular apps, or combinations thereof, can be applied. The image modification algorithms can be used in real time to adapt an image. Also or in combination, The image modification algorithms can be used on a time sequence of images, for instance forming a video film being recorded. In other or related embodiments, the action of image modification may be performed before an image or sequence of images is displayed or stored. In this respect, the image recognition may be performed on all images that are captured and presented in a live preview, or for instance a subset of the captured images from that time sequence, and the action may be performed on each of images that is displayed in the preview.
In the application, reference may be made to a server. Such a server may be one server device, for instance a computer device, located at a location. Alternatively, a server may refer to at least one server device, connected via one or more data connections, at the same location and/or located at remote, in particular physically/geographically remote locations.
In an image recording device, an image sensor captures an image. Currently, an image sensor often is a CMOS device, but also other devices may be considered. These image sensors may also be referred to as spatial images sensors. These sensors allow capturing of one or more at least two dimensional images.
In current technology, a captured image is clocked out or read out of the image sensor, and digitised into a stream of digital values representing a digital pixel image. In some case, the image recording device may comprise an image processor for providing some basic processing and temporary storage of a captured image. Examples of the pre-processing comprise colour correction, white balancing, noise reduction, and even image conversion for converting and/or compressing an image into a different digital file format.
In an image display device, an image, set of images or a sequence of images is stored into a memory, and may be converted for allowing to be displayed. The image display device may comprise a display screen, for instance an OLED panel, an LCD panel, or the like, or may comprise a projector for projecting a picture or a film on a remote screen. Often, the image, set of images or sequence of images is converted. In an embodiment of the current invention, the image or at least a subset of the set of images or of the sequence of images is subjected to the scene recognition and identifiers are provided. Based upon an identifier, one of the actions is performed on the image or set of images of sequence of images following and/or including the image that is provided with the specific identifier. In particular, the actions are performed before an image, set of images or sequence of images is presented to a user via the panel or projector.
Image recording and image display may be combined. Many image recording devices also comprise a display that allows a direct view of images while being captured in real-time. Thus, the display functions as a viewer, allowing a user to compose an image composition. Once the uses selects, for instance shoots a picture, or films a piece of film, the image sensor captures an image or a sequence of images. That image is then pre-processed by the image processor, and stored in a memory. Often, the captured image is also displayed on the display. There, a user may manually apply further image processing, like filtering, red-eye reduction, and the like. Scene recognition and even the action may be preformed before an image or images are provided for preview, displayed, or stored.
In another mode, the image recording device may be in a so called 'burst mode', or 'continuous capture mode', allowing a video to be captured. In this 'burst mode', at a video frame rate images are being captured, providing a film. Often, such a frame rate is at least 20 frames per second (fps), in particular at least 30 fps.
The device relates to a time sequence of images. An example of a time sequence of images is the recording of a film. Another example is a functionally live view though a viewer of a digital camera. In particular when a digital viewer is used, a functionally live sequence of images is displayed via the viewer. The device may for instance apply the action on each of the images that are displayed on the viewer. The time sequence of images may have a time base. The time between the images may be constant, like for instance in a film. The time sequence of images may also comprise subsequent bursts of images, each burst having the same of different time between subsequent bursts.
In an embodiment, the action comprises an action on a subset of images from said time sequence of images, said subset including said image. The scene recognition may for instance be done on an image. Subsequently, images that in time follow or precede the image may be processed using the action. Thus, if the time between images that are subjected to scene recognition is relatively small, for instance small with respect to the vision capabilities of a human, for instance a time interval smaller than 0.2 seconds, and a following set of images between this time interval is processed, then an almost constant visual sequence of images is processed.
In an embodiment, the device is adapted for performing scene recognition on at least a subset of said time sequence of images. For instance a set of continuous images can be subjected to scene recognition. Alternatively, each n-th image can be subjected to scene recognition.
In an embodiment, the device allows the action to be dependent upon the result of the scene recognition.
In an embodiment, the device is adapted for providing an identifier based upon the result of said scene recognition. An identifier can be a number of letter. An identifier may also be another type of label, for instance allowing the application of a hash function. In a further embodiment, if said identifier matches a predefined identifier, based upon the identifier, the device performs an action on said images. Thus, for instance, if the scene, recognised object or event changes, it may be possible to also change the action in response of the change. The action may be selected from the group consisting of image modification comprising adapting at least part of said image, of modifying said image into a modified image, of blocking storage of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof.
In an embodiment, the time sequence of images is selected from the group of a sequence of live images and a sequence of images forming a video film. One image or all the images of the entire sequence may be subjected to scene recognition.
In an embodiment, the scene recognition comprises applying an algorithm selected from the group consisting of calculating the unique digital signature of an image and then matching that signature against those of other photos, of discriminative feature mining, of contour-based shape descriptors, of deep Fisher networks, of Bag of Words, of support vector machines, of deep learning, of face detection, of template matching based on the characteristic shapes and colours of objects, and a combination thereof.
In an embodiment, the modifying said image comprises blurring at least a part of said image. For instance, part of a scene, an object that has been recognised, or an event that has been recognised may be blurred. It may thus be possible to blur parts before displaying or before (permanent) storage. Thus, it may be possible to provide an image recorder, digital camera or computer display that cannot store or display unwanted scenes and events and/or objects within scenes.
In an embodiment, the action is image processing by applying photographic filters. As mentioned, examples of these filters are filters selected from the group of Rise filter, Hudson filter, Sierra filter, Lo-Fi filter, Sutro filter, Brannan filter, Inkwell filter, Hefe filter, and a combination thereof.
In an embodiment, the device comprises an image sensor adapted for capturing an image, in particular said series of images forming a film, wherein said scene recognition is performed on said image, and said action is performed on said captured image, in particular before a next image is captured.
In an embodiment, the device comprises a data storage, wherein said device is adapted for performing said action is before storage said image in said data storage. Such data storage may comprise a hard disk, solid state disk (SSD), but may also relate to external storage, for instance remote external storage like cloud storage.
In an embodiment, the device comprises a display for displaying said image, wherein said device is adapted for performing said action before displaying said image.
In an embodiment, the invention relates to an imaging system comprising an image sensor for capturing an image, a memory for storing said image, and the device of the invention.
In an embodiment, the invention relates to an image display system, comprising a memory for receiving an image for displaying, a display for displaying said image, and the device of the invention.
The invention further relates to a computer program comprising software code portions which, when running on a data processor, configure said data processor to:
- retrieve an image from a memory;
- perform scene recognition on said image, and
- based upon the result of said scene recognition performs an action selected from the group consisting of image modification comprising adapting at least part of said image, of modifying said image into a modified image, of blocking storage of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof.
The invention further pertains to a data carrier provided with this computer program.
The invention further pertains to a signal carrying at least part of this computer program.
The invention further pertains to a signal sequence representing a program for being executed on a computer, said signal sequence representing this computer program.
The invention further pertains to a method for processing a live sequence of images, said method comprising performing scene recognition on at least a set of images of said sequence of images, and based upon the result of said scene recognition, perform an action on subsequent images of said sequence of images, said action selected from the group consisting of image modification comprising adapting at least part of said image, of modifying said image into a modified image, of blocking storage of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof.
In an embodiment, the method further comprising providing an identifier based upon the result of said scene recognition.
In an embodiment, the method further comprises if said identifier matches a predefined identifier, based upon the identifier, perform an action on subsequent images of said sequence of images, said action selected from the group consisting of image modification comprising adapting at least part of said image, of modifying said image into a modified image, of blocking storage of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof.
The invention further pertains to method o device for processing a set of images, said method comprising performing scene recognition on at least a subset of images of said set of images, and based upon the result of said scene recognition, perform an action on subsequent images of said sequence of images, said action selected from the group consisting of image modification comprising adapting at least part of said image, of modifying said image into a modified image, of blocking storage of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof. Thus, in this embodiment, actions on a large set of images or on a database of images can be automated.
The term "substantially" herein, like in "substantially consists", will be understood by and clear to a person skilled in the art. The term "substantially" may also include embodiments with "entirely", "completely", "all", etc. Hence, in embodiments the adjective substantially may also be removed. Where applicable, the term "substantially" may also relate to 90% or higher, such as 95% or higher, especially 99% or higher, even more especially 99.5% or higher, including 100%. The term "comprise" includes also embodiments wherein the term "comprises" means "consists of.
The term "functionally", when used for instance in "functionally coupled" or "functionally direct communication", will be understood by and clear to a person skilled in the art. The term "substantially" may also include embodiments with "entirely", "completely", "all", etc. Hence, in embodiments the adjective substantially may also be removed. Thus, for instance "functionally direct communication" comprises direct, live communication. It may also comprise communication that, from a perspective of the parties communication, is experienced as "live". Thus, like for instance voice over IP (VOIP), there may be a small amount of time between various data packages comprising digital voice data, but these amounts of time are so small that for users it seems as if there is an open communication line or telephone line available.
Furthermore, the terms first, second, third and the like in the description and in the claims, are used for distinguishing between similar elements and not necessarily for describing a sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances and that the embodiments of the invention described herein are capable of operation in other sequences than described or illustrated herein.
The devices or apparatus herein are amongst others described during operation. As will be clear to the person skilled in the art, the invention is not limited to methods of operation or devices in operation.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. Use of the verb "to comprise" and its conjugations does not exclude the presence of elements or steps other than those stated in a claim. The article "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the device or apparatus claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.
The invention further applies to an apparatus or device comprising one or more of the characterising features described in the description and/or shown in the attached drawings. The invention further pertains to a method or process comprising one or more of the characterising features described in the description and/or shown in the attached dr awing s .
The various aspects discussed in this patent can be combined in order to provide additional advantages. Furthermore, some of the features can form the basis for one or more divisional applications. Brief description of the drawings
Embodiments of the invention will now be described, by way of example only, with reference to the accompanying schematic drawings in which corresponding reference symbols indicate corresponding parts, and in which:
FIG. 1 schematically depicts a device for processing a time sequence of images. FIG. 2 schematically depicts an imaging system.
FIG. 3 schematically depicts a display system.
FIG. 4 depicts a camera applying a photographic filter on an outdoor scene. FIG. 5 depicts a camera applying a photographic filter on a portrait.
FIG. 6 depicts a camera which blocks the recording of an unwanted event FIG. 7 depicts a screen which blocks the scene of an unwanted event
The drawings are not necessarily on scale.
Description of preferred embodiments FIG. 1 schematically depicts a device which receives digitised images through module 201. The image or images are a representation of scene 100. These images are stored in a temporary memory 202. Next, the image or images are subjected to scene recognition in module 203. Based on the result of the scene recognition in module 204, an identifier 205 may be provided to the images. A action alters the images in module 206, and/or identifier 205' prevents the altering of the images and stores the images in a temporary memory 202 which. By then, the images are representing scene 100'. In this altered scene 100', parts of the scene may be blurred.
FIG. 2 schematically depicts an imaging system which captures images through camera 200. These images represent scene 100. The images are stored in a temporary memory 202. Next, these images are subjected to scene recognition in module 203. Based on the result of the scene recognition in module 204, an identifier 205 may be provided to the images. Based upon the identifier, one or more actions may be performed on the images in module 206. For instance, identifier 205' may prevent the altering of the images. Next, the images may be stored in a temporary memory 202 and recorded in module 207 where the images, by then, are representing scene 100' .
FIG. 3 schematically depicts a display system which receives digitised images through module 201. These images represent scene 100. The images may be stored in a temporary memory 202. Next, a scene recognition is applied in module 203. Based on the result of the scene recognition in module 204 an identifier 205 may be provided to the images. An action may be performed on the images in module 206, and/or identifier 205' prevents the altering of the images. Next, images may be stored in a temporary memory 202 and displays the images on screen 210. By then, the images may represent a scene 100'.
FIG. 4 depicts a camera 200 which recognises an outdoor scene 101. The camera automatically applies a specific photographic filter on the captured images of scene 101. The modified images are then displayed on the viewer of camera 200 which shows the aesthetically enhanced scene 101 ' . Camera 200 allows for instance blurring of part of a scene. Unwanted parts of a scene can be blurred functionally life. Thus, a viewer will not be confronted with unwanted scenes. For instance, it can be prevented tha children see horrible details in a film. The scene recognition thus in fact each time interprets an image and identifies the unwanted part. In then allows blocking or altering or blurring, for instance, of that unwanted part. Even if such an unwanted part displaces in the scene during playing a movie of film. Thus, scene recognition provides for instance a interpretation of objects in their surrounding or in events and interprets them in an almost human intelligent way.
FIG. 5 depicts a camera 200 which recognises a portrait scene 102. The camera automatically applies a specific photographic filter on the captured images of scene 102 and displays the modified images on the viewer of camera 200 which shows the aesthetically enhanced scene 102'. The camera 200 thus allows an action on a functionally live image or on a sequence of live images.
FIG. 6 depicts a camera 200 which recognises an unwanted event 103. Next, camera 200 automatically blocks the captured images of event 103 and does not record the event on camera 200.
FIG. 7 depicts a screen 210 which recognises an unwanted event 103, automatically erases the incoming images of event 103 and does not show the event on screen 210.
It will also be clear that the above description and drawings are included to illustrate some embodiments of the invention, and not to limit the scope of protection. Starting from this disclosure, many more embodiments will be evident to a skilled person. These embodiments are within the scope of protection and the essence of this invention and are obvious combinations of prior art techniques and the disclosure of this patent.

Claims

Claims
1. A device for processing a time sequence of images, said device adapted for:
- retrieving an image from said time sequence of images from a memory;
- performing scene recognition on said retrieved image, and
- based upon the result of said scene recognition, performs an action on said image, said action selected from the group consisting of image modification comprising adapting at least part of said image, of modifying said image into a modified image, of blocking storage of said image, of blocking display of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof.
2. The device of claim 1, wherein said action comprises an action on a subset of
images from said time sequence of images, said subset including said image.
3. The device of claim 1, wherein said device is adapted for performing scene
recognition on at least a subset of said time sequence of images.
4. The device of claim 1, wherein said device is adapted for:
- providing an identifier based upon the result of said scene recognition, and
- if said identifier matches a predefined identifier, based upon the identifier, perform an action on said images, said action selected from the group consisting of image modification comprising adapting at least part of said image, of modifying said image into a modified image, of blocking storage of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof.
5. The device of claim 1, wherein said time sequence of images is selected from the group of a sequence of live images and a sequence of images forming a video film.
6. The device of claim 1, wherein said scene recognition comprises applying an
algorithm selected from the group consisting of calculating the unique digital signature of an image and then matching that signature against those of other photos, of discriminative feature mining, of contour-based shape descriptors, of deep Fisher networks, of Bag of Words, of support vector machines, of deep learning, of face detection, of template matching based on the characteristic shapes and colours of objects, and a combination thereof.
7. The device of claim 1, wherein said modifying said image comprises blurring at least a part of said image.
8. The device of claim 1, wherein said action is image processing by applying at least one photographic filter.
9. The device of claim 1, wherein said device comprises an image sensor adapted for capturing an image, in particular said series of images forming a film, wherein said scene recognition is performed on said image, and said action is performed on said captured image, in particular before a next image is captured.
10. The device of claim 1, wherein said device comprises a data storage, wherein said device is adapted for performing said action is before storage said image in said data storage.
11. The device of claim 1, wherein said device comprises a display for displaying said image, wherein said device is adapted for performing said action before displaying said image.
12. An imaging system comprising:
- an image sensor for capturing an image;
- a memory for storing said image, and
- the device of claim 1.
13. An image display system, comprising:
- a memory for receiving an image for displaying;
- a display for displaying said image, and - the device of claim 1.
14. A computer program comprising software code portions which, when running on a data processor, configure said data processor to:
- retrieve an image from a memory;
- perform scene recognition on said image, and
- based upon the result of said scene recognition performs an action selected from the group consisting of image modification comprising adapting at least part of said image, of modifying said image into a modified image, of blocking storage of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof.
15. The computer program of claim 14, further configured to:
- provide an identifier based upon the result of said scene recognition, and
- if said identifier matches a predefined identifier, based upon the identifier, perform an action on said image, said action selected from the group consisting of image modification comprising adapting at least part of said image, of modifying said image into a modified image, of blocking storage of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof.
16. A data carrier provided with the computer program of claim 14.
17. A signal carrying at least part of said computer program of claim 14.
18. A signal sequence representing a program for being executed on a computer, said signal sequence representing said computer program of claim 14.
19. A method for processing a live sequence of images, said method comprising:
- performing scene recognition on at least a set of images of said sequence of images, and
- based upon the result of said scene recognition, perform an action on subsequent images of said sequence of images, said action selected from the group consisting of image modification comprising adapting at least part of said image, of modifying said image into a modified image, of blocking storage of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof.
20. The method of claim 19, further comprising:
- providing an identifier based upon the result of said scene recognition, and
- if said identifier matches a predefined identifier, based upon the identifier, perform an action on subsequent images of said sequence of images, said action selected from the group consisting of image modification comprising adapting at least part of said image, of modifying said image into a modified image, of blocking storage of said image, of erasing said image from said memory, of encrypting said image, and a combination thereof.
-o-o-o-o-o-
PCT/EP2014/052471 2014-02-07 2014-02-07 Processing a time sequence of images, allowing scene dependent image modification WO2015117672A1 (en)

Priority Applications (10)

Application Number Priority Date Filing Date Title
PCT/EP2014/052471 WO2015117672A1 (en) 2014-02-07 2014-02-07 Processing a time sequence of images, allowing scene dependent image modification
EP14771516.3A EP3103117A1 (en) 2014-02-07 2014-07-03 Live scene recognition allowing scene dependent image modification before image recording or display
CN202010152403.3A CN111326183A (en) 2014-02-07 2014-07-03 System and method for processing a temporal image sequence
PCT/EP2014/064269 WO2015117681A1 (en) 2014-02-07 2014-07-03 Live scene recognition allowing scene dependent image modification before image recording or display
JP2016550545A JP6162345B2 (en) 2014-02-07 2014-07-03 Raw scene recognition that allows scene-dependent image modification before image recording or display
KR1020167022241A KR101765428B1 (en) 2014-02-07 2014-07-03 Live scene recognition allowing scene dependent image modification before image recording or display
CN201480074872.0A CN106165017A (en) 2014-02-07 2014-07-03 Allow to carry out the instant scene Recognition of scene associated picture amendment before image record or display
BR112016018024A BR112016018024A2 (en) 2014-02-07 2014-07-03 LIVE SCENE RECOGNITION ALLOWS SCENE DEPENDENT IMAGE MODIFICATION BEFORE RECORDING OR IMAGE DISPLAY
US14/616,634 US9426385B2 (en) 2014-02-07 2015-02-06 Image processing based on scene recognition
TW104104278A TWI578782B (en) 2014-02-07 2015-02-09 Image processing based on scene recognition

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/EP2014/052471 WO2015117672A1 (en) 2014-02-07 2014-02-07 Processing a time sequence of images, allowing scene dependent image modification

Publications (1)

Publication Number Publication Date
WO2015117672A1 true WO2015117672A1 (en) 2015-08-13

Family

ID=50193446

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2014/052471 WO2015117672A1 (en) 2014-02-07 2014-02-07 Processing a time sequence of images, allowing scene dependent image modification

Country Status (1)

Country Link
WO (1) WO2015117672A1 (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060023077A1 (en) * 2004-07-30 2006-02-02 Microsoft Corporation System and method for photo editing
US20070297641A1 (en) * 2006-06-27 2007-12-27 Microsoft Corporation Controlling content suitability by selectively obscuring

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060023077A1 (en) * 2004-07-30 2006-02-02 Microsoft Corporation System and method for photo editing
EP1695548A2 (en) 2004-07-30 2006-08-30 Microsoft Corporation System and method for photo editing
US20070297641A1 (en) * 2006-06-27 2007-12-27 Microsoft Corporation Controlling content suitability by selectively obscuring

Non-Patent Citations (11)

* Cited by examiner, † Cited by third party
Title
"Deep Fisher Networks for Large-Scale Image Classification", ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS, 2013
BANGPENG YAO; KHOSHLA, LI FEI-FIE: "Combining randomisation and discrimination for fine- grained image categorisation", COMPUTER VISION PATTERN RECOGNITION CONFERENCE, 2011
HEO ET AL.: "Spherical hashing", COMPUTER VISION PATTERN RECOGNITION CONFERENCE, 2012
HU, JIA; LING, HUANG: "Multiscale Distance Matrix for Fast Plant Leaf Recognition", IEEE TRANS. ON IMAGE PROCESSING (T-IP, vol. 21, no. 11, 2012, pages 4667 - 4672
KRIZHEVSKY, A.; SUTSKEVER, I.; HINTON, G. E.: "ImageNet Classification with Deep Convolutional Neural Networks, Advances in Neural Information Processing 25", MIT PRESS
MICROSOFT PHOTODNA FACT SHEET DECEMBER, 2009
R. BRUNELLI: "Template Matching Techniques in Computer Vision: Theory and Practice", WILEY
R. BRUNELLI; T. POGGIO: "Face Recognition: Features versus Templates", IEEE TRANS. ON PAMI, 1993
SNOEK ET AL.: "MediaMill at TRECVID 2013: Searching Concepts, Objects, Instances and Events in Video", PROCEEDINGS OF THE I ITH TRECVID WORKSHOP, 2013
SNOEK ET AL.: "The MediaMill TRECVID 2012 Semantic Video Search Engine", PROCEEDINGS OF THE 10TH TRECVID WORKSHOP, 2012
VIOLA JONES: "Robust Real-Time Face Detection", INTERNATIONAL JOURNAL OF COMPUTER VISION, 2004

Similar Documents

Publication Publication Date Title
US9426385B2 (en) Image processing based on scene recognition
KR101765428B1 (en) Live scene recognition allowing scene dependent image modification before image recording or display
US11995530B2 (en) Systems and methods for providing feedback for artificial intelligence-based image capture devices
US9692964B2 (en) Modification of post-viewing parameters for digital images using image region or feature information
KR101688352B1 (en) Recommending transformations for photography
US8212911B2 (en) Imaging apparatus, imaging system, and imaging method displaying recommendation information
US9129381B2 (en) Modification of post-viewing parameters for digital images using image region or feature information
JP6340347B2 (en) Image processing apparatus, image processing method, program, and recording medium
US8675991B2 (en) Modification of post-viewing parameters for digital images using region or feature information
US10728510B2 (en) Dynamic chroma key for video background replacement
US20130301918A1 (en) System, platform, application and method for automated video foreground and/or background replacement
WO2016019770A1 (en) Method, device and storage medium for picture synthesis
US9195880B1 (en) Interactive viewer for image stacks
US20060197851A1 (en) Positioning a subject with respect to a background scene in a digital camera
CN103916597A (en) Shooting method based on simulated scenario
US12010433B2 (en) Image processing apparatus, image processing method, and storage medium
JP2016189507A (en) Image processing device, image processing method, program and recording medium
WO2015117672A1 (en) Processing a time sequence of images, allowing scene dependent image modification
CN106201251B (en) A kind of content of augmented reality determines method, device and mobile terminal
LeGendre et al. Improved chromakey of hair strands via orientation filter convolution
Grogan Techniques to Detect Modified Video
JP2005210366A (en) Photographing support system and photographing support method
CN111164961A (en) Image processing system and image processing method of intelligent terminal
WO2019023966A1 (en) Photography system and photography method for intelligent terminal
Lantz High Dynamic Range Photography

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14707664

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 14707664

Country of ref document: EP

Kind code of ref document: A1