WO2018098860A1 - 照片合成方法及装置 - Google Patents
照片合成方法及装置 Download PDFInfo
- Publication number
- WO2018098860A1 WO2018098860A1 PCT/CN2016/110975 CN2016110975W WO2018098860A1 WO 2018098860 A1 WO2018098860 A1 WO 2018098860A1 CN 2016110975 W CN2016110975 W CN 2016110975W WO 2018098860 A1 WO2018098860 A1 WO 2018098860A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- face
- photo
- type
- score value
- preset
- Prior art date
Links
- 238000001308 synthesis method Methods 0.000 title abstract 2
- 230000014509 gene expression Effects 0.000 claims abstract description 88
- 238000000034 method Methods 0.000 claims abstract description 45
- 239000002131 composite material Substances 0.000 claims description 65
- 230000002194 synthesizing effect Effects 0.000 claims description 12
- 239000000203 mixture Substances 0.000 claims description 9
- 238000004364 calculation method Methods 0.000 claims description 8
- 230000000694 effects Effects 0.000 abstract description 4
- 238000004904 shortening Methods 0.000 abstract description 2
- 210000000887 face Anatomy 0.000 description 37
- 230000008921 facial expression Effects 0.000 description 12
- 238000004891 communication Methods 0.000 description 10
- 238000012545 processing Methods 0.000 description 10
- 238000010586 diagram Methods 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 4
- 230000005236 sound signal Effects 0.000 description 4
- 238000003860 storage Methods 0.000 description 3
- 230000001133 acceleration Effects 0.000 description 2
- 230000001815 facial effect Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000029553 photosynthesis Effects 0.000 description 2
- 238000010672 photosynthesis Methods 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000008451 emotion Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/211—Selection of the most significant subset of features
- G06F18/2113—Selection of the most significant subset of features by ranking or filtering the set of features, e.g. using a measure of variance or of feature cross-correlation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
- G06V40/167—Detection; Localisation; Normalisation using comparisons between temporally consecutive images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/174—Facial expression recognition
- G06V40/175—Static expression
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/61—Control of cameras or camera modules based on recognised objects
- H04N23/611—Control of cameras or camera modules based on recognised objects where the recognised objects include parts of the human body
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/64—Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/265—Mixing
Definitions
- the present disclosure relates to the field of smart camera technology, and in particular, to a photo synthesis method and apparatus.
- the image capturing apparatus can perform the coincidence operation to achieve the scores of the faces in each of the generated photos in the process of the coincidence, and extract the best-performing faces of each of the photos. synthesis.
- there are many photographs that need to be synthesized when generating a composite photograph which increases the power consumption of the image pickup apparatus and the time for generating a composite photograph.
- the embodiments of the present disclosure provide a photo synthesizing method and apparatus for solving the problem of large power consumption and long time for generating a composite photo.
- a photo composition method which may include:
- the image acquisition component After receiving the instruction to generate the composite photo, the image acquisition component is activated to collect the photo;
- the first type of face is used to indicate that the emoticon score value calculated by the photo taken before the current photo is not greater than a face with a preset score threshold;
- the expression score value of the first type of face in the current photo is greater than the preset score threshold, controlling the image acquisition component to stop collecting photos, and adopting the second type of face in the collected photo
- the splicing method generates a composite photo, and the second type of face is used to represent a face whose emoticon score value is greater than a preset score threshold.
- the method further includes:
- the method further includes:
- a face for generating the composite photograph is determined from the collected photographs, and a composite photograph is generated in a stitching manner.
- determining a face for generating the composite photo from the collected photos includes:
- a second type of face and a face having the highest expression score in the first type of face are selected from the collected photos as a face for generating the composite photo.
- calculating an expression score value of the first type of face in the current photo includes:
- a photo synthesizing apparatus which may include:
- the acquisition module is configured to, after receiving the instruction to generate the composite photo, start the image acquisition component to collect the photo;
- a calculation module configured to calculate an expression score value of the first type of face in the current photo when the current photo is collected, where the first type of face is used to indicate that the photo taken before the current photo has been calculated a face whose expression score is not greater than a preset score threshold;
- a first determining module configured to determine, in the current photo, whether an emoticon score value of each of the first type of faces calculated by the calculating module is greater than the preset score threshold
- a generating module configured to, when the first determining module determines that the expression score value of the first type of face in the current photo is greater than the preset score threshold, control the image collection component to stop collecting photos, and A second type of face in the collected photo is used to generate a composite photo in a stitching manner, and the second type of face is used to represent a face whose expression score value is greater than a preset score threshold.
- the apparatus further includes:
- a second determining module configured to determine, when the first determining module determines that the expression score values of the first type of faces in the current photo are not greater than the preset score threshold, determine whether the number of collected photos is less than Preset number;
- An execution module configured to determine, after the second determining module determines that the number of collected photos is less than a preset number, determine a photo to be collected according to an expression score value of the first type of face in the current photo To calculate the first type of face of the expression score value, the operation of starting the image acquisition component to take a photo is performed.
- the apparatus further includes:
- a third determining module configured to: when the second determining module determines that the number of collected photos is not less than a preset number, determine a face for generating the composite photo from the collected photos, And the splicing method is used to generate a composite photo.
- the third determining module comprises:
- Selecting a sub-module configured to select a second type of face from the collected photos and an expression in the first type of face The face with the highest score value is used as the face for generating the composite photo.
- the computing module comprises:
- An identification submodule configured to identify each of the first type of faces from the current photo
- a calculation submodule configured to calculate a local score value corresponding to the local feature corresponding to each of the first type of faces
- the weighting sub-module is configured to perform weighted summation on each local scoring value of each of the first type of human faces to obtain an emoticon score value of each of the first type of human faces.
- a photo synthesizing apparatus which may include:
- a memory for storing processor executable instructions
- processor is configured to:
- the image acquisition component After receiving the instruction to generate the composite photo, the image acquisition component is activated to collect the photo;
- the first type of face is used to indicate that the emoticon score value calculated by the photo taken before the current photo is not greater than a face with a preset score threshold;
- the expression score value of the first type of face in the current photo is greater than the preset score threshold, controlling the image acquisition component to stop collecting photos, and adopting the second type of face in the collected photo
- the splicing method generates a composite photo, and the second type of face is used to represent a face whose emoticon score value is greater than a preset score threshold.
- the technical solution provided by the embodiment of the present disclosure may include the following effects: after receiving an instruction to generate a composite photo, the image capturing component is activated to collect a photo, and each time a photo is captured, the first type of face in the current photo is calculated.
- the expression score value determines whether the emoticon score value of each first type of face in the current photo is greater than a preset score threshold, and when the emoticon score value of the first type of face in the current photo is greater than the preset score threshold Controlling the image acquisition component to stop collecting photos, and generating a composite photo by splicing the second type of face in the collected photo.
- only one person in the previously generated photo may be calculated each time a photo is generated.
- the facial expression score value of the face with a lower facial expression score value can effectively reduce the number of generated photos and reduce the calculation of facial expression values in the photo on the basis of ensuring a synthetic photograph with good effect. Amount, which effectively shortens the time to generate a composite photo and reduces the power consumption of generating a composite photo.
- the facial expression is determined from various aspects, so that The score on facial expressions is more comprehensive.
- FIG. 1 is a flowchart of a photo composition method according to an exemplary embodiment.
- FIG. 2 is a flowchart of a photo composition method according to an exemplary embodiment.
- FIG. 3 is a flowchart of a method of calculating an expression score value of a human face, according to an exemplary embodiment.
- FIG. 4 is a block diagram of a photo composition apparatus according to an exemplary embodiment.
- FIG. 5 is a block diagram of another photo synthesizing apparatus according to an exemplary embodiment.
- FIG. 6 is a block diagram of still another photo synthesizing apparatus according to an exemplary embodiment.
- FIG. 7 is a block diagram of a device for photo composition, according to an exemplary embodiment.
- FIG. 1 is a flowchart of a photo composition method according to an exemplary embodiment; the photo composition method can be applied to a camera or an electronic device (such as a smart phone or a tablet) including a camera device, as shown in FIG.
- the photo synthesis method includes the following steps:
- step 101 after receiving the instruction to generate a composite photo, the image acquisition component is activated to acquire a photo.
- an instruction to generate a composite photo can be triggered by a touch screen or a physical button.
- the number of photos collected by the image capturing component cannot exceed a preset number, for example, 4, and the number of captured photos is determined by the facial expression score of the face in the first captured photo, for example, if In the first photo, the expression scores of all faces are greater than the preset score threshold, so only one photo can be taken.
- a preset number for example, 4, and the number of captured photos is determined by the facial expression score of the face in the first captured photo, for example, if In the first photo, the expression scores of all faces are greater than the preset score threshold, so only one photo can be taken.
- the preset number may be set by the user or may be preset and stored in the memory by the camera provider.
- step 102 when the current photo is collected, the expression score value of the first type of face in the current photo is calculated.
- the first type of face is used to indicate that a face whose expression score value is not greater than a preset score threshold has been calculated by the photo taken before the current photo. For example, if there are four faces in the photo, namely face A, face B, face C, face D, when the first photo is taken, face A, face B, face C, face D are It can be used as the first type of face, and it is necessary to calculate the expression score value of all faces. If the expression scores of face A, face B, and face C are greater than the preset score threshold in the first photo, then For the two photos, the first type of face includes the face D, and when the second picture is generated, only the expression score of the face D can be calculated.
- the expression score value of each face may be measured by the eyes, mouth, face orientation, face image quality, and the like of the face.
- the expression score value of the face may be calculated by a preset image processing algorithm after each photo is generated.
- the process of calculating the expression score value of the face can be referred to the embodiment shown in FIG. 3, which will not be described in detail herein.
- step 103 in the current photo, it is determined whether the expression score values of each of the first type of faces are greater than a preset score threshold, if the expression scores of the first type of faces in the current photo are greater than the preset score thresholds. Go to step 104.
- the preset score threshold may be a reasonable score, for example, 80 points, and the expression effect reaching the preset score threshold is better, and may be used to generate a composite photo.
- step 104 the image capturing component is controlled to stop capturing photos, and the second type of face in the captured photo is stitched to generate a composite photo.
- the second type of face is used to represent a face whose emoticon score is greater than a preset score threshold.
- the composite photo may be generated by splicing the face of the captured photo with the expression score value greater than the preset score threshold.
- the image capturing component is started to collect the photo, and after each photo is captured, the facial expression score of the first type of face in the current photo is calculated, and each of the current photos is determined. Whether the expression score value of a first type of face is greater than a preset score threshold, and when the expression score value of the first type of face in the current photo is greater than the preset score threshold, controlling the image acquisition component to stop collecting photos,
- the collected photos are generated by splicing, and in the present disclosure, each time a photo is generated, only the expression score of the face with a lower facial expression score in the previously generated photo can be calculated.
- the number of generated photographs can be effectively reduced, and the calculation amount of facial expression values in the photograph can be reduced, thereby effectively shortening the time for generating synthetic photographs and reducing the work of generating synthetic photographs. Consumption.
- the method further includes:
- the method further includes:
- a face for generating a composite photo is determined from the acquired photos, and a composite photo is generated by stitching.
- determining a face for generating a composite photo from the captured photos includes:
- a second type of face and a face having the highest expression score in the first type of face are selected from the collected photos as a face for generating a composite photo.
- calculating an emoticon rating value of the first type of face in the current photo includes:
- Each local score value of each of the first type of faces is weighted and summed to obtain an expression score value of each of the first type of faces.
- FIG. 2 is a flowchart of a method for synthesizing a photo according to an exemplary embodiment of the present invention. This embodiment uses the above method provided by the embodiment of the present disclosure to generate a composite photo as an example, as shown in FIG. 2 . Including the following steps:
- step 201 after receiving the instruction to generate a composite photo, the image acquisition component is activated to collect the photo.
- step 202 when the current photo is collected, the emoticon value of the first type of face in the current photo is calculated.
- the first type of face is used to indicate that a face whose expression score value is not greater than a preset score threshold has been calculated by the photo taken before the current photo.
- step 201 and step 202 can be referred to the description of step 101 and step 102 of the embodiment shown in FIG. 1, and details are not described herein.
- step 203 in the current photo, determining whether the emoticon score value of each of the first type of faces is greater than a preset score threshold, if the emoticon scores of the first type of faces in the current photo are greater than the preset score threshold.
- Step 205 is performed if the expression score values of the first type of faces in the current photo are not all greater than the preset score threshold.
- the preset score threshold may be a reasonable score, such as 80 points. For example, if the current photo is the second photo taken, and only the facial expression A and the facial B's expression score value in the first photo are not greater than the preset score threshold, the face A in the second photo may be calculated. And the expression score value of the face B and determining whether the expression score values of the face A and the face B in the second photo are greater than the preset score threshold.
- step 204 the image capturing component is controlled to stop capturing photos, and the second type of face in the captured photo is stitched to generate a composite photo.
- the second type of face is used to represent a face whose emoticon score is greater than a preset score threshold.
- step 205 it is determined whether the number of collected photos is less than a preset number. If the number of collected photos is less than the preset number, step 206 is performed, and if the number of collected photos is not less than the preset number, step 207 is performed.
- step 206 according to the expression score value of the first type of face in the current photo, the first type of face whose expression is to be calculated is determined by a later collected photo, and step 201 is performed.
- step 203 if only the expression score value of the face A is greater than the preset score threshold in the second photo, it may be determined that only the face B of the third photo is to be calculated.
- step 207 a face for generating a composite photo is determined from the acquired photos, and a composite photo is generated in a stitching manner.
- the second type of face and the expression score of the first type of face may be selected from the collected photos.
- the highest face is used as the face for generating a composite photo.
- the face A, the face C, and the face D in the first photo may be used.
- Determined to be the second type of face that is, the face A, the face C, and the face D in the first photo are the faces used to generate the composite photo; for the face B, the expression score in the first photo
- the emoticon value in the second photo is 72 points
- the emoticon value in the third photo is 75 points
- the emoticon value in the fourth photo is 79 points. If the preset number is 4, Then, the face B in the fourth photo can be selected as the face for generating a composite photo.
- the number of photos for generating a composite photo can be effectively reduced while ensuring the quality of the composite photo; in addition, the expression scores of the face in each photo are In the case where it is not larger than the preset score threshold, the face with the highest expression score value is determined as the face for generating the composite photograph, thereby realizing that the photograph for generating the synthetic photograph is effectively reduced while ensuring the quality of the synthesized photograph.
- the quantity can effectively reduce the number of target photos and increase the speed of generating composite photos.
- FIG. 3 is a flowchart of a method for calculating an expression score value of a human face according to an exemplary embodiment of the present disclosure.
- the embodiment uses the above method provided by the embodiment of the present disclosure to calculate an expression score of a human face as an example.
- An exemplary description, as shown in FIG. 3, includes the following steps:
- each first type of face is identified from the current photo.
- each face in each photo may be identified by an image recognition model such as a convolutional neural network.
- each person's face area can also be identified by other image processing techniques.
- step 302 a local score value corresponding to the local feature corresponding to each of the first type of faces is calculated.
- the local score value corresponding to each local feature value of the face may be preferentially calculated, such as the corner part of the mouth, the part of the human eye, the facial features of the face, the tilt angle of the face, and the like. Local rating value.
- the local score value of the local feature value of the face may also be calculated by using the model trained in advance.
- the local feature value of the face may also be calculated by a preset algorithm. Rating value.
- each local score value of each of the first type of faces is weighted and summed to obtain an expression score value of each of the first type of faces.
- the weight coefficient corresponding to each local score value may be set by a user, or may be preset by an algorithm.
- the multi-faceted determination is realized.
- the facial expression makes the score on the facial expression more comprehensive.
- FIG. 4 is a block diagram of a photo synthesizing apparatus according to an exemplary embodiment. As shown in FIG. 4, the photo synthesizing apparatus includes:
- the acquiring module 410 is configured to, after receiving the instruction to generate the composite photo, start the image collecting component to collect the photo;
- the calculation module 420 is configured to calculate an expression score value of the first type of face in the current photo when the current photo is collected, and the first type of face is used to indicate that the emotion score calculated by the photo taken before the current photo is not calculated. a face larger than a preset score threshold;
- the first determining module 430 is configured to determine, in the current photo, whether the emoticon score value of each first type of face calculated by the calculating module 420 is greater than a preset score threshold;
- the generating module 440 is configured to, when the first determining module 430 determines that the expression score values of the first type of faces in the current photo are greater than the preset score threshold, control the image capturing component to stop collecting photos, and The second type of face generates a composite photo by stitching, and the second type of face is used to represent a face whose expression score is greater than a preset score threshold.
- FIG. 5 is a block diagram of another photo synthesizing apparatus according to an exemplary embodiment. As shown in FIG. 5, based on the embodiment shown in FIG. 4, in an embodiment, the apparatus further includes:
- the second determining module 450 is configured to determine, when the first determining module 430 determines that the expression score values of the first type of faces in the current photo are not greater than the preset score threshold, determine whether the number of collected photos is less than a preset number;
- the execution module 460 is configured to, when the second determining module 450 determines that the number of collected photos is less than the preset number, determine, according to the expression score value of the first type of face in the current photo, to calculate an expression score The first type of face of the value, the operation of starting the image acquisition component to take a photo.
- the apparatus further includes:
- the third determining module 470 is configured to: when the second determining module 450 determines that the number of collected photos is not less than a preset number, determine a face for generating a composite photo from the collected photos, and generate a composite by using a stitching manner photo.
- the third determining module 470 includes:
- the selection sub-module 471 is configured to select a second type of face from the acquired photos and a face with the highest expression score in the first type of face as the face for generating the composite photo.
- FIG. 6 is a block diagram of still another photo synthesizing apparatus according to an exemplary embodiment.
- the calculating module 420 include:
- the identification sub-module 421 is configured to identify each of the first type of faces from the current photo
- the calculating sub-module 422 is configured to calculate a local scoring value corresponding to the local feature corresponding to each first type of face;
- the weighting sub-module 423 is configured to perform weighted summation on each local scoring value of each of the first type of faces to obtain an emoticon rating value of each of the first type of faces.
- the device embodiment since it basically corresponds to the method embodiment, reference may be made to the partial description of the method embodiment.
- the device embodiments described above are merely illustrative, with the single illustrated as separate components
- the elements may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the objectives of the present disclosure. Those of ordinary skill in the art can understand and implement without any creative effort.
- FIG. 7 is a block diagram of a device for photo composition, according to an exemplary embodiment.
- device 700 can be a video camera or other electronic device that includes a camera device.
- apparatus 700 can include one or more of the following components: processing component 702, memory 704, power component 706, multimedia component 708, audio component 710, input/output (I/O) interface 712, sensor component 714, And a communication component 716.
- Processing component 702 typically controls the overall operations of device 700, such as operations associated with display, voice playback, data communication, and recording operations.
- Processing component 702 can include one or more processors 720 to execute instructions to perform all or part of the steps described above.
- processing component 702 can include one or more modules to facilitate interaction between component 702 and other components.
- processing component 702 can include a multimedia module to facilitate interaction between multimedia component 708 and processing component 702.
- Memory 704 is configured to store various types of data to support operation at device 700. Examples of such data include instructions, messages, images, etc. for any application or method operating on device 700.
- Memory 704 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Magnetic Memory, Flash Memory, Disk or Optical Disk.
- SRAM static random access memory
- EEPROM electrically erasable programmable read only memory
- EPROM erasable Programmable Read Only Memory
- PROM Programmable Read Only Memory
- ROM Read Only Memory
- Magnetic Memory Flash Memory
- Disk Disk or Optical Disk.
- Power component 706 provides power to various components of device 700.
- Power component 706 can include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for device 700.
- the multimedia component 708 includes a screen between the device 700 and the user that provides an output interface.
- the screen can include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user.
- the touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensor can sense not only the boundaries of the touch or sliding action, but also the duration and pressure associated with the touch or slide operation.
- the audio component 710 is configured to output and/or input an audio signal.
- audio component 710 includes a microphone (MIC) that is configured to receive an external audio signal when device 700 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode.
- the received audio signal may be further stored in memory 704 or transmitted via communication component 716.
- audio component 710 also includes a speaker for outputting an audio signal.
- the I/O interface 712 provides an interface between the processing component 702 and the peripheral interface module, which may be a keyboard, a click wheel, a button, or the like. These buttons may include, but are not limited to, a home button, a volume button, a start button, and a lock button.
- Sensor assembly 714 includes one or more sensors for providing device 700 with various aspects of status assessment.
- sensor component 714 can detect an open/closed state of device 700, the relative positioning of components, such as a display and a keypad of device 700, and sensor component 714 can also detect a change in position of device 700 or a component of device 700, user The presence or absence of contact with device 700, device 700 orientation or acceleration/deceleration and temperature variation of device 700.
- Sensor assembly 714 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact.
- Sensor component 714 can also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications.
- the sensor component 714 can also include an acceleration sensor, a gyro sensor, a magnetic sensor, a distance sensor, a pressure sensor, or a temperature sensor.
- Communication component 716 is configured to facilitate wired or wireless communication between device 700 and other devices.
- the device 700 can access a wireless network based on a communication standard, such as WIFI, 2G or 3G, or a combination thereof.
- communication component 716 receives broadcast signals or broadcast associated information from an external broadcast management system via a broadcast channel.
- communication component 716 also includes a near field communication (NFC) module to facilitate short range communication.
- NFC near field communication
- the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
- RFID radio frequency identification
- IrDA infrared data association
- UWB ultra-wideband
- Bluetooth Bluetooth
- apparatus 700 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable A gate array (FPGA), controller, microcontroller, microprocessor, or other electronic component implementation that performs the following methods:
- ASICs application specific integrated circuits
- DSPs digital signal processors
- DSPDs digital signal processing devices
- PLDs programmable logic devices
- FPGA field programmable A gate array
- controller microcontroller, microprocessor, or other electronic component implementation that performs the following methods:
- the image acquisition component After receiving the instruction to generate the composite photo, the image acquisition component is activated to collect the photo;
- the first type of face is used to indicate that the person whose expression score value is not greater than the preset score threshold is calculated by the photo taken before the current photo. face;
- the control image acquisition component stops collecting the photo, and the second type of face in the collected photo is stitched to generate a composite photo, and secondly A human face is used to represent a face whose emoticon score is greater than a preset score threshold.
- non-transitory computer readable storage medium comprising instructions, such as a memory 704 comprising instructions executable by processor 720 of apparatus 700 to perform the above method.
- the non-transitory computer readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Oral & Maxillofacial Surgery (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Studio Devices (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
Description
Claims (11)
- 一种照片合成方法,其特征在于,所述方法包括:在接收到生成合成照片的指令后,启动图像采集部件采集照片;在采集当前照片时,计算所述当前照片中第一类人脸的表情评分值,所述第一类人脸用于表示通过所述当前照片之前采集的照片已计算得到表情评分值均不大于预设分数阈值的人脸;在所述当前照片中,确定每一个第一类人脸的表情评分值是否均大于所述预设分数阈值;如果所述当前照片中的第一类人脸的表情评分值均大于所述预设分数阈值,控制所述图像采集部件停止采集照片,将所述已采集的照片中的第二类人脸采用拼接方式生成合成照片,所述第二类人脸用于表示表情评分值大于预设分数阈值的人脸。
- 根据权利要求1所述的方法,其特征在于,所述方法还包括:如果所述当前照片中的第一类人脸的表情评分值没有均大于所述预设分数阈值,则确定已采集的照片数目是否小于预设数目;如果已采集的照片数目小于预设数目,根据所述当前照片中第一类人脸的表情评分值,确定后面采集的一张照片要计算表情评分值的第一类人脸,执行所述启动图像采集部件采集照片的操作。
- 根据权利要求2所述的方法,其特征在于,所述方法还包括:如果已采集的照片数目不小于预设数目,则从所述已采集的照片中确定用于生成所述合成照片的人脸,并且采用拼接方式生成合成照片。
- 根据权利要求3所述的方法,其特征在于,所述从所述已采集的照片中确定用于生成所述合成照片的人脸,包括:从所述已采集的照片中选择第二类人脸以及第一类人脸中表情评分值最高的人脸作为用于生成所述合成照片的人脸。
- 根据权利要求1所述的方法,其特征在于,所述计算所述当前照片中第一类人脸的表情评分值,包括:从所述当前照片中识别出每一个第一类人脸;计算所述每一个第一类人脸对应的局部特征对应的局部评分值;对所述每一个第一类人脸的每一个局部评分值进行加权求和,得到所述每一个第一类人脸的表情评分值。
- 一种照片合成装置,其特征在于,所述装置包括:采集模块,被配置为在接收到生成合成照片的指令后,启动图像采集部件采集照片;计算模块,被配置为在采集当前照片时,计算所述当前照片中第一类人脸的表情评分值,所述第一类人脸用于表示通过所述当前照片之前采集的照片已计算得到表情评分值均不大于预设分数阈值的人脸;第一确定模块,被配置为在所述当前照片中,确定所述计算模块计算的所述每一个第一类人脸的表情评分值是否均大于所述预设分数阈值;生成模块,被配置为在所述第一确定模块确定所述当前照片中的第一类人脸的表情评分值均大于所述预设分数阈值时,控制所述图像采集部件停止采集照片,将所述已采集的照片中的第二类人脸采用拼接方式生成合成照片,所述第二类人脸用于表示表情评分值大于预设分数阈值的人脸。
- 根据权利要求6所述的装置,其特征在于,所述装置还包括:第二确定模块,被配置为在所述第一确定模块确定所述当前照片中的第一类人脸的表情评分值没有均大于所述预设分数阈值时,确定已采集的照片数目是否小于预设数目;执行模块,被配置为在所述第二确定模块确定所述已采集的照片数目小于预设数目时,根据所述当前照片中第一类人脸的表情评分值,确定后面采集的一张照片要计算表情评分值的第一类人脸,执行所述启动图像采集部件采集照片的操作。
- 根据权利要求7所述的装置,其特征在于,所述装置还包括:第三确定模块,被配置为在所述第二确定模块确定所述已采集的照片数目不小于预设数目时,从所述已采集的照片中确定用于生成所述合成照片的人脸,并且采用拼接方式生成合成照片。
- 根据权利要求8所述的装置,其特征在于,所述第三确定模块包括:选择子模块,被配置为从所述已采集的照片中选择第二类人脸以及第一类人脸中表情评分值最高的人脸作为用于生成所述合成照片的人脸。
- 根据权利要求6所述的装置,其特征在于,所述计算模块包括:识别子模块,被配置为从所述当前照片中识别出每一个第一类人脸;计算子模块,被配置为计算所述每一个第一类人脸对应的局部特征对应的局部评分值;加权子模块,被配置为对所述每一个第一类人脸的每一个局部评分值进行加权求和,得到所述每一个第一类人脸的表情评分值。
- 一种照片合成装置,其特征在于,所述装置包括:处理器;用于存储处理器可执行指令的存储器;其中,所述处理器被配置为:在接收到生成合成照片的指令后,启动图像采集部件采集照片;在采集当前照片时,计算所述当前照片中第一类人脸的表情评分值,所述第一类人脸用于表示通过所述当前照片之前采集的照片已计算得到表情评分值均不大于预设分数阈值的人脸;在所述当前照片中,确定每一个第一类人脸的表情评分值是否均大于所述预设分数阈值;如果所述当前照片中的第一类人脸的表情评分值均大于所述预设分数阈值,控制所述图像采集部件停止采集照片,将所述已采集的照片中的第二类人脸采用拼接方式生成合成照片,所述第二类人脸用于表示表情评分值大于预设分数阈值的人脸。
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2017511210A JP6622289B2 (ja) | 2016-11-29 | 2016-12-20 | 写真合成方法、装置、プログラム、及び記録媒体 |
RU2017105737A RU2653233C1 (ru) | 2016-11-29 | 2016-12-20 | Способ и устройство для синтезирования фотографий |
KR1020187032407A KR102116826B1 (ko) | 2016-11-29 | 2016-12-20 | 사진 합성 방법, 장치, 프로그램 및 기록매체 |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201611078279.0 | 2016-11-29 | ||
CN201611078279.0A CN106600530B (zh) | 2016-11-29 | 2016-11-29 | 照片合成方法及装置 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2018098860A1 true WO2018098860A1 (zh) | 2018-06-07 |
Family
ID=58461052
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2016/110975 WO2018098860A1 (zh) | 2016-11-29 | 2016-12-20 | 照片合成方法及装置 |
Country Status (6)
Country | Link |
---|---|
US (1) | US20180150722A1 (zh) |
EP (1) | EP3328062A1 (zh) |
JP (1) | JP6622289B2 (zh) |
KR (1) | KR102116826B1 (zh) |
CN (1) | CN106600530B (zh) |
WO (1) | WO2018098860A1 (zh) |
Families Citing this family (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107507159A (zh) * | 2017-08-10 | 2017-12-22 | 珠海市魅族科技有限公司 | 图片处理方法及装置、计算机装置及可读存储介质 |
CN107895146B (zh) * | 2017-11-01 | 2020-05-26 | 深圳市科迈爱康科技有限公司 | 微表情识别方法、装置、系统及计算机可读存储介质 |
US11222208B2 (en) * | 2018-07-13 | 2022-01-11 | Futurewei Technologies, Inc. | Portrait image evaluation based on aesthetics |
CN109658278A (zh) * | 2018-12-14 | 2019-04-19 | 北京智明星通科技股份有限公司 | 基于微信平台的线上互动方法、互动系统和移动终端 |
CN110852220B (zh) * | 2019-10-30 | 2023-08-18 | 深圳智慧林网络科技有限公司 | 人脸表情的智能识别方法、终端和计算机可读存储介质 |
CN111199210B (zh) * | 2019-12-31 | 2023-05-30 | 武汉星巡智能科技有限公司 | 基于表情的视频生成方法、装置、设备及存储介质 |
CN117795551A (zh) * | 2021-08-11 | 2024-03-29 | 三星电子株式会社 | 用于自动捕捉和处理用户图像的方法和系统 |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101000688A (zh) * | 2007-01-15 | 2007-07-18 | 浙江大学 | 多张人脸照片自动合成方法 |
US20110013038A1 (en) * | 2009-07-15 | 2011-01-20 | Samsung Electronics Co., Ltd. | Apparatus and method for generating image including multiple people |
CN103491299A (zh) * | 2013-09-17 | 2014-01-01 | 宇龙计算机通信科技(深圳)有限公司 | 一种拍照处理方法和装置 |
CN103747180A (zh) * | 2014-01-07 | 2014-04-23 | 宇龙计算机通信科技(深圳)有限公司 | 照片拍摄方法及拍照终端 |
CN106156762A (zh) * | 2016-08-12 | 2016-11-23 | 乐视控股(北京)有限公司 | 拍照处理方法及装置 |
Family Cites Families (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2004046591A (ja) * | 2002-07-12 | 2004-02-12 | Konica Minolta Holdings Inc | 画像評価装置 |
JP2006203600A (ja) * | 2005-01-21 | 2006-08-03 | Konica Minolta Holdings Inc | 撮像装置、画像処理装置、画像処理システム、画像処理方法及びプログラム |
US7659923B1 (en) * | 2005-06-24 | 2010-02-09 | David Alan Johnson | Elimination of blink-related closed eyes in portrait photography |
JP4264663B2 (ja) * | 2006-11-21 | 2009-05-20 | ソニー株式会社 | 撮影装置、画像処理装置、および、これらにおける画像処理方法ならびに当該方法をコンピュータに実行させるプログラム |
JP2010045421A (ja) * | 2008-08-08 | 2010-02-25 | Casio Comput Co Ltd | 撮像装置、画像処理方法、及びプログラム |
JP2010081562A (ja) * | 2008-09-29 | 2010-04-08 | Fujifilm Corp | 撮像装置、方法およびプログラム |
US8515137B2 (en) * | 2010-05-03 | 2013-08-20 | Microsoft Corporation | Generating a combined image from multiple images |
US8559766B2 (en) * | 2011-08-16 | 2013-10-15 | iParse, LLC | Automatic image capture |
US9336583B2 (en) * | 2013-06-17 | 2016-05-10 | Cyberlink Corp. | Systems and methods for image editing |
-
2016
- 2016-11-29 CN CN201611078279.0A patent/CN106600530B/zh active Active
- 2016-12-20 JP JP2017511210A patent/JP6622289B2/ja active Active
- 2016-12-20 KR KR1020187032407A patent/KR102116826B1/ko active IP Right Grant
- 2016-12-20 WO PCT/CN2016/110975 patent/WO2018098860A1/zh active Application Filing
-
2017
- 2017-03-13 EP EP17160519.9A patent/EP3328062A1/en not_active Ceased
- 2017-05-04 US US15/586,281 patent/US20180150722A1/en not_active Abandoned
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101000688A (zh) * | 2007-01-15 | 2007-07-18 | 浙江大学 | 多张人脸照片自动合成方法 |
US20110013038A1 (en) * | 2009-07-15 | 2011-01-20 | Samsung Electronics Co., Ltd. | Apparatus and method for generating image including multiple people |
CN103491299A (zh) * | 2013-09-17 | 2014-01-01 | 宇龙计算机通信科技(深圳)有限公司 | 一种拍照处理方法和装置 |
CN103747180A (zh) * | 2014-01-07 | 2014-04-23 | 宇龙计算机通信科技(深圳)有限公司 | 照片拍摄方法及拍照终端 |
CN106156762A (zh) * | 2016-08-12 | 2016-11-23 | 乐视控股(北京)有限公司 | 拍照处理方法及装置 |
Also Published As
Publication number | Publication date |
---|---|
KR20180133897A (ko) | 2018-12-17 |
US20180150722A1 (en) | 2018-05-31 |
KR102116826B1 (ko) | 2020-06-01 |
JP6622289B2 (ja) | 2019-12-18 |
CN106600530A (zh) | 2017-04-26 |
CN106600530B (zh) | 2019-02-15 |
JP2019504514A (ja) | 2019-02-14 |
EP3328062A1 (en) | 2018-05-30 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2018098860A1 (zh) | 照片合成方法及装置 | |
US10115019B2 (en) | Video categorization method and apparatus, and storage medium | |
US10534972B2 (en) | Image processing method, device and medium | |
WO2016029641A1 (zh) | 照片获取方法及装置 | |
US20170154206A1 (en) | Image processing method and apparatus | |
RU2628494C1 (ru) | Способ и устройство для генерирования фильтра изображения | |
CN105491289B (zh) | 防止拍照遮挡的方法及装置 | |
WO2017016146A1 (zh) | 图像显示方法及装置 | |
WO2017031901A1 (zh) | 人脸识别方法、装置及终端 | |
JP6391708B2 (ja) | 虹彩画像を取得する方法および装置、ならびに虹彩識別機器 | |
WO2017124899A1 (zh) | 一种信息处理方法及装置、电子设备 | |
EP3754459A1 (en) | Method and apparatus for controlling camera, device and storage medium | |
US10248855B2 (en) | Method and apparatus for identifying gesture | |
CN106408603A (zh) | 拍摄方法及装置 | |
EP3258414B1 (en) | Prompting method and apparatus for photographing | |
WO2021047069A1 (zh) | 人脸识别方法和电子终端设备 | |
US20190095746A1 (en) | Method, device and non-transitory storage medium for processing clothes information | |
WO2017140108A1 (zh) | 压力检测方法和装置 | |
KR101701814B1 (ko) | 프레이밍 정보를 디스플레이하기 위한 방법 및 장치 | |
CN106550226A (zh) | 投影画面校正方法及装置 | |
WO2021189927A1 (zh) | 图像处理方法、装置、电子设备及存储介质 | |
US11715234B2 (en) | Image acquisition method, image acquisition device, and storage medium | |
WO2019061118A1 (zh) | 全景拍摄方法及终端 | |
WO2021237744A1 (zh) | 拍摄方法及装置 | |
CN106780296A (zh) | 照片合成方法及装置 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
ENP | Entry into the national phase |
Ref document number: 2017105737 Country of ref document: RU Kind code of ref document: A |
|
ENP | Entry into the national phase |
Ref document number: 2017511210 Country of ref document: JP Kind code of ref document: A |
|
ENP | Entry into the national phase |
Ref document number: 20187032407 Country of ref document: KR Kind code of ref document: A |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 16923064 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 16923064 Country of ref document: EP Kind code of ref document: A1 |