WO2020173211A1 - 图像特效的触发方法、装置和硬件装置 - Google Patents

图像特效的触发方法、装置和硬件装置 Download PDF

Info

Publication number
WO2020173211A1
WO2020173211A1 PCT/CN2019/128744 CN2019128744W WO2020173211A1 WO 2020173211 A1 WO2020173211 A1 WO 2020173211A1 CN 2019128744 W CN2019128744 W CN 2019128744W WO 2020173211 A1 WO2020173211 A1 WO 2020173211A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
semantics
special effect
voice
processing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2019/128744
Other languages
English (en)
French (fr)
Inventor
郑微
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing ByteDance Network Technology Co Ltd
Original Assignee
Beijing ByteDance Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing ByteDance Network Technology Co Ltd filed Critical Beijing ByteDance Network Technology Co Ltd
Priority to SG11202109345WA priority Critical patent/SG11202109345WA/en
Publication of WO2020173211A1 publication Critical patent/WO2020173211A1/zh
Priority to US17/459,739 priority patent/US11595591B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/2621—Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/30—Semantic analysis
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/63—Control of cameras or camera modules by using electronic viewfinders
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80—Camera processing pipelines; Components thereof
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/08—Speech classification or search
    • G10L15/18—Speech classification or search using natural language modelling
    • G10L15/1822—Parsing for meaning understanding
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G10L2015/223—Execution procedure of a spoken command

Definitions

  • the present disclosure relates to the field of image processing, and in particular to a method, device and hardware device for triggering image special effects.
  • smart terminals can be used to listen to music, play games, chat online, and take photos.
  • the camera technology of the smart terminal the camera pixel has reached more than 10 million pixels, with higher definition and the camera effect comparable to professional cameras.
  • the camera software built-in at the factory can be used to achieve the traditional functions of the camera, but also can be downloaded from the network application (Application, referred to as: APP) to achieve the camera effect with additional functions Or add special effects to the video, such as various filter effects.
  • APP Application, referred to as: APP
  • a method for triggering image special effects including:
  • Triggering special effect processing in response to recognizing predetermined semantics from the speech, wherein the predetermined semantics are preset semantics associated with one or more special effect processing;
  • the original image is processed to form and display image special effects.
  • the obtaining the original image from the image source and displaying the original image on the display device includes:
  • the image or video is acquired through the image acquisition device and the image or video is displayed on the display device.
  • the collecting the voice in the environment and recognizing the semantics of the voice includes:
  • the first trigger signal includes:
  • the collecting the voice in the environment and recognizing the semantics of the voice it further includes:
  • the collecting speech in the environment and recognizing the semantics of the speech includes:
  • the triggering special effect processing in response to the recognition of predetermined semantics from the voice, wherein the predetermined semantics are preset semantics associated with one or more special effect processing, including:
  • semantics recognized from the speech includes predetermined semantics, acquiring processing parameters of special effects processing corresponding to the predetermined semantics;
  • the predetermined semantics is the semantics set in advance and corresponding to one or more special effect processing.
  • processing the original image according to the triggered special effect processing to form and display image special effects includes:
  • the original image is processed according to the processing parameters to form an image special effect and the image special effect is displayed on the display device.
  • the special effect processing includes:
  • One or more of filter processing, deformation processing, sticker processing, and animation processing are present.
  • the voice in the environment before collecting the voice in the environment and recognizing the semantics of the voice, it further includes:
  • the method further includes:
  • a correspondence relationship between predetermined semantics and image special effects is set, where the image special effects include one or more special effect processing required to generate the image special effects.
  • a trigger device for image special effects including:
  • Original image acquisition module for acquiring original images
  • the semantic recognition module is used to collect the voice in the environment and recognize the semantics of the voice
  • a special effect processing trigger module configured to trigger special effect processing in response to recognizing predetermined semantics from the voice, wherein the predetermined semantics are preset semantics associated with one or more special effect processing;
  • the image special effect generation module is used to process the original image to form an image special effect according to the triggered special effect processing.
  • the original image acquisition module is also used for:
  • the image or video is acquired through the image acquisition device and the image or video is displayed on the display device.
  • semantic recognition module further includes:
  • the voice collection module is used to collect voice in the environment in response to the first trigger signal
  • the first semantic recognition module is used to recognize the semantics of the voice.
  • the first trigger signal includes:
  • the device further includes:
  • the special effect package loading analysis module is used to load the special effect package and analyze the predetermined semantics that can trigger the special effect in the special effect package.
  • semantic recognition module further includes:
  • Voice data collection module used to collect voice data in the environment
  • a voice data conversion module for converting the voice data into text data
  • the second semantic recognition module is used to perform word segmentation processing on the text data to obtain at least one semantic meaning.
  • the special effect processing trigger module further includes:
  • the processing parameter acquisition module is used to compare the semantics recognized from the voice with the predetermined semantics; if the semantics recognized from the voice includes predetermined semantics, obtain the processing parameters of the special effect processing corresponding to the predetermined semantics; wherein
  • the predetermined semantics is a predetermined semantics corresponding to one or more special effect processing.
  • image special effect generation module is also used for:
  • the original image is processed according to the processing parameters to form an image special effect and the image special effect is displayed on the display device.
  • the special effect processing includes:
  • One or more of filter processing, deformation processing, sticker processing, and animation processing are present.
  • the image processing device may further include:
  • the sampling setting module is used to set the sampling rate and the number of sampling bits for voice collection.
  • the image processing device may further include:
  • the correspondence relationship setting module is used to set the correspondence relationship between predetermined semantics and image special effects, wherein the image special effects include one or more special effect processing required to generate the image special effects. According to another aspect of the present disclosure, the following technical solutions are also provided:
  • An electronic device comprising: a memory for storing non-transitory computer readable instructions; and a processor for running the computer readable instructions so that the processor implements any of the above-mentioned image special effects triggering methods when executed The steps described.
  • a computer-readable storage medium for storing non-transitory computer-readable instructions.
  • the non-transitory computer-readable instructions When executed by a computer, the computer can execute the steps in any of the above methods.
  • the present disclosure discloses a method, device and hardware device for triggering image special effects.
  • the method for triggering the image special effect includes: acquiring an original image from an image source and displaying the original image on a display device; collecting voice in the environment and recognizing the semantics of the voice; A predetermined semantic is recognized, and special effect processing is triggered, where the predetermined semantic is a semantics set in advance and associated with one or more special effect processing; according to the triggered special effect processing, the original image is processed to form and display image special effects .
  • the method for triggering image special effects in the embodiments of the present disclosure triggers image processing effects by recognizing speech semantics, which solves the technical problem of inflexible triggering of image effects in the prior art and the inability to free the user's hands.
  • FIG. 1 is a schematic flowchart of a method for triggering an image special effect according to an embodiment of the present disclosure
  • step S102 of the method for triggering image special effects according to an embodiment of the present disclosure
  • FIG. 3 is a schematic flowchart of yet another further step S102 of the method for triggering image special effects according to an embodiment of the present disclosure
  • FIG. 4 is a further schematic flowchart of a method for triggering image special effects according to an embodiment of the present disclosure
  • FIG. 5 is a schematic structural diagram of an image special effect trigger device according to an embodiment of the present disclosure.
  • Fig. 6 is a schematic structural diagram of an electronic device provided according to an embodiment of the present disclosure.
  • the embodiments of the present disclosure provide a method for triggering image special effects.
  • the method for triggering image special effects provided in this embodiment can be executed by a computing device, the computing device can be implemented as software, or as a combination of software and hardware, and the computing device can be integrated in a server or terminal device. Waiting.
  • the method for triggering the image special effect mainly includes the following steps S101 to S104. among them:
  • Step S101 Obtain an original image from an image source and display the original image on a display device;
  • the image source may be various image sensors, and the original image obtained from the image source may be obtained through an image sensor.
  • the image sensor refers to various devices that can collect images.
  • a typical image sensor is a camera. , Camera, camera, etc.
  • the image sensor may be a camera on a terminal device, such as a front or rear camera on a smart phone, and the image collected by the camera may be directly displayed on the display screen of the mobile phone.
  • the obtaining of the original image from the image source may be obtaining the current image frame of the video collected by the current terminal device. Since the video is composed of multiple image frames, the video is obtained in the embodiment Image, using the video frame image in the video image as the original image. The acquiring of the original image may also be acquiring a picture collected by the current terminal device.
  • the image source is a local or network storage
  • the acquisition of the original image from the image source may be to acquire any form of image from a local storage device or a storage device pointed to by a network address, such as static Pictures, dynamic pictures or videos in various formats, etc., are not restricted here.
  • the original image is displayed on a display device.
  • the display device may be a terminal or server that executes the method for triggering the image special effect, or a terminal or server that receives the original image. Display screen, etc.
  • Step S102 Collect voices in the environment and recognize the semantics of the voices
  • the collecting the speech in the environment and recognizing the semantics of the speech may include: step S201, in response to the first trigger signal, collecting the speech in the environment; step S202: Recognizing the semantics of the voice.
  • the first trigger signal is triggered by a certain condition.
  • a first trigger signal can be generated at this time, and the first trigger signal instructs the terminal device to start collecting voices in the environment.
  • the special effects package Generally, it appears in the form of an interface on the screen, such as a button or icon at a predetermined position.
  • touching a predetermined position on the screen of the terminal device can generate the first trigger signal; optionally, it can also be When a predetermined event is recognized in the original image, the first trigger signal is generated.
  • a predetermined event is recognized in the original image
  • the first trigger signal is generated.
  • a first trigger signal can be generated to instruct the terminal device to start collecting voices in the environment
  • the generation of the first trigger signal is not limited to the above manner, and the above manner is only an example. In practical applications, any suitable signal can be set as the trigger signal of the voice in the collection environment, which will not be repeated here.
  • the voice in the aforementioned environment may further include loading a special effect package and analyzing the semantics that can trigger the special effect in the special effect package.
  • the first trigger signal may be generated by touching a predetermined position on the screen of the terminal device.
  • the user touches the icon of a specific special effect package the first trigger signal is generated.
  • the special effects in the special effects package may include multiple types, and the triggering semantics of the multiple types of special effects may be different.
  • the semantics of "New Year” and “Happy” can be identified in the voice “Happy New Year”, and the new year can correspond to To the firecracker special effects in the special effects package, "happiness” can correspond to the smiley special effects in the special effects package and so on.
  • the collecting the speech in the environment and recognizing the semantics of the speech includes: step S301, collecting speech data in the environment; step S302, converting the speech The data is converted into text data; step S303, word segmentation processing is performed on the text data to obtain at least one semantic meaning.
  • step S301 collecting speech data in the environment
  • step S302 converting the speech The data is converted into text data
  • step S303 word segmentation processing is performed on the text data to obtain at least one semantic meaning.
  • To collect voice data in the environment it is first necessary to collect the voice in the environment through a sound collection device.
  • the typical sound collection device can be a microphone.
  • the voice collected by the sound collection device is a continuous analog signal.
  • Sampling the voice into a PCM (Pulse Code Modulation) signal may also include the step of setting the sampling rate and the number of sampling bits for voice collection.
  • the typical sampling rate and number of sampling bits can be 44.1KHz, 16bit, 16KHz, 16bit, etc.
  • the sampling rate and number of sampling bits in practical applications are not limited to the above examples, if it involves factors such as performance or transmission speed, it can be adjusted down appropriately The sampling rate or number of sampling bits will not be repeated here.
  • the voice data is converted into text data.
  • the voice data is preprocessed first.
  • Typical preprocessing may include converting the voice data into audio files in a specific format, such as WAV (Waveform Audio File Format), FLAC (Free Lossless Audio Codec), etc.; typical preprocessing can also include noise suppression. While collecting the voice in the environment, other sounds in the environment are often collected. In order to reduce the interference of environmental noise, it is necessary to Voice data is processed for noise reduction; typical preprocessing can also include lossless compression.
  • the voice recognition device is an online device and the voice data needs to be transmitted to the line through local equipment for recognition, it can be reduced by lossless compression
  • the volume of voice data saves network resources, while reducing the transmission time of voice data to speed up voice recognition.
  • the aforementioned format conversion, noise suppression processing, and lossless compression processing of the voice data can be implemented in any specific implementation manner, which is not specifically limited in the present disclosure.
  • the voice data is input into the voice recognition device, and the voice is converted into text data.
  • the voice features in the voice data can generally be extracted, compared with the voice features in the voice template, and output the text data of the matched voice, and finally form the text data of the voice data. It is understandable that the above-mentioned speech-to-text method is only an example, and various methods, such as deep learning, can be used to convert speech into text.
  • word segmentation processing is performed on the text data to obtain at least one semantic meaning.
  • the word segmentation here can be divided into different words according to needs. For example, “Happy New Year” can be divided into one semantic, or it can be divided into two semantics, "New Year” and "Happy”.
  • the method of recognizing the semantics of the speech in the step S102 can be any method, such as inputting the speech data into the model through the recognition model to directly recognize the required semantics; in addition, the semantics output in the step , It can be the semantics that can trigger the special effects of the above special effects package. If the special effects of the special effects package cannot be triggered, it does not need to be recognized. At this time, the recognition model needs to be trained so that it can only recognize the predetermined semantics instead of identifying all Semantics.
  • the semantics here can also include words with similar meanings. For example, "New Year” and “Spring Festival” can be recognized as one meaning, etc., which can be realized by synonymous voice templates or training models, which will not be repeated here.
  • the steps can be performed locally or online, that is to say, the speech semantic recognition system can be performed locally or online, and different settings can be made in different application scenarios. This disclosure does not do this. limit.
  • Step S103 in response to recognizing predetermined semantics from the voice, trigger special effect processing, where the predetermined semantics are preset semantics associated with one or more special effect processing;
  • the predetermined semantics is preset and can trigger the semantics of the special effects. If the above steps include the steps of loading the special effects package and parsing the semantics that can trigger the special effects in the special effects package, then the predetermined semantics here Is the semantics parsed in the loading step.
  • the triggering special effect processing in response to the recognition of predetermined semantics from the voice includes: comparing the recognized semantics from the voice with the predetermined semantics; if the recognized semantics from the voice It contains predetermined semantics, and the processing parameters of the special effect processing corresponding to the predetermined semantics are obtained; wherein the predetermined semantics are preset semantics corresponding to one or more special effect processing. If the semantics includes predetermined semantics, the special effect processing parameters corresponding to the predetermined semantics are acquired.
  • the semantics may include multiple semantics, such as the user saying "Congratulations, Happy New Year", at this time, through step S102, the semantics of the voice can be recognized as “Congratulations” and "Everyone” , “New Year”, “Happy”, if the predetermined semantics is "New Year", the semantics contains predetermined semantics, then proceed to the next step, if the predetermined semantics is "eat", then the next step is not executed.
  • the semantics include predetermined semantics, it is necessary to further obtain special effect processing parameters corresponding to the predetermined semantics.
  • the special effect processing here may be filter processing, deformation processing, sticker processing, and animation One or more of the processing.
  • the special effect processing corresponding to the predetermined semantics can be obtained from the special effect package, and the corresponding relationship between the semantics and the special effect can be obtained when the special effect package is parsed, and the special effect corresponds to one or several special effect processing, so that the semantics corresponds to the special effect processing.
  • the special effect processing parameter may be the type of special effect processing and the resources and characteristic parameters required to execute the special effect processing.
  • the type is a sticker
  • the resource is a picture or a picture used by the sticker.
  • the characteristic parameters are the display position and display time of the sticker.
  • the special effect processing is not limited to the above examples. In fact, any special effect processing can be used in the present disclosure.
  • the processing parameters of the special effect processing are not limited to the above processing parameters, and the present disclosure does not specifically limit this. .
  • Step S104 Process the original image according to the triggered special effect processing to form and display an image special effect.
  • the original image is processed according to the processing parameters of the special effect processing to form an image special effect.
  • animation processing Take animation processing as an example.
  • the animation processing in the special effects package is triggered to perform the original image Processing, adding an animation of lighting firecrackers to a predetermined position of the original image, and playing the background sound of the firecrackers, to obtain an image with special effects of the firecracker animation.
  • the filter Take the filter as an example again.
  • the semantics recognizes "good-looking”.
  • the beauty filter in the special effects package will be triggered.
  • the face in the image undergoes beautification processing to obtain a face image after beautification.
  • the image special effect is displayed on a display device.
  • the display device may be a terminal or a server that executes the method for triggering the image special effect, or a terminal that receives the special effect image, a display screen of the server, etc. .
  • step S401 may be further included: setting the correspondence between semantics and image special effects, where the image special effects include special effect processing required to generate the image special effects.
  • the steps can be executed when the special effect package is generated, that is, the trigger condition of the image special effect can be set by setting the corresponding relationship between the semantics and one or more image special effects in the special effect package. It is understandable that the corresponding relationship can be stored in
  • the configuration file of the special effect package may include, in addition to the above-mentioned correspondence, the image special effects contained in the special effect package, the storage location of the material contained in the image special effects, and the special effects required by the image special effects.
  • any attributes of the special effects of the special effect package such as the display position, display time, display mode, and processing intensity of the special effect, can be configured, and no examples are given here.
  • the corresponding relationship between the semantics and the image special effects set here can be the corresponding relationship between the custom semantics and the image special effects when the user uses it.
  • the user can record the voice by himself
  • the semantics of the voice is used as the triggering condition of a certain image effect.
  • the present disclosure discloses a method, device and hardware device for triggering image special effects.
  • the method for triggering the image special effect includes: acquiring an original image from an image source and displaying the original image on a display device; collecting voice in the environment and recognizing the semantics of the voice; A predetermined semantic is recognized, and special effect processing is triggered, where the predetermined semantic is a semantics set in advance and associated with one or more special effect processing; according to the triggered special effect processing, the original image is processed to form and display image special effects .
  • the method for triggering image special effects in the embodiments of the present disclosure triggers image processing effects by recognizing speech semantics, which solves the technical problem of inflexible triggering of image effects in the prior art and the inability to free the user's hands.
  • the device embodiments of the present disclosure can be used to perform the steps implemented by the method embodiments of the present disclosure.
  • the embodiment of the present disclosure provides an image processing device.
  • the device can execute the steps described in the above-mentioned embodiment of the method for triggering image special effects.
  • the device 500 mainly includes: an original image acquisition module 501, a semantic recognition module 502, a special effect processing trigger module 503, and an image special effect generation module 504. among them,
  • An original image acquisition module 501 which acquires an original image from an image source and displays the original image on a display device;
  • the semantic recognition module 502 is used to collect voices in the environment and recognize the semantics of the voices;
  • the special effect processing trigger module 503 is configured to trigger special effect processing in response to recognizing predetermined semantics from the voice, where the predetermined semantics are preset semantics associated with one or more special effect processing;
  • the image special effect generation module 504 is configured to process the original image according to the triggered special effect processing to form and display the image special effect.
  • the original image acquisition module 501 is also used for:
  • the image or video is acquired through the image acquisition device and the image or video is displayed on the display device.
  • semantic recognition module 502 further includes:
  • the voice collection module is used to collect voice in the environment in response to the first trigger signal
  • the first semantic recognition module is used to recognize the semantics of the voice.
  • the first trigger signal includes:
  • the device 500 further includes:
  • the special effect package loading analysis module is used to load the special effect package and analyze the predetermined semantics that can trigger the special effect in the special effect package.
  • semantic recognition module 502 further includes:
  • Voice data collection module used to collect voice data in the environment
  • a voice data conversion module for converting the voice data into text data
  • the second semantic recognition module is used to perform word segmentation processing on the text data to obtain at least one semantic meaning.
  • the special effect processing trigger module 503 further includes:
  • the processing parameter acquisition module is used to compare the semantics recognized from the voice with the predetermined semantics; if the semantics recognized from the voice includes predetermined semantics, obtain the processing parameters of the special effect processing corresponding to the predetermined semantics; wherein
  • the predetermined semantics is a predetermined semantics corresponding to one or more special effect processing.
  • image special effect generation module 504 is also used for:
  • the original image is processed according to the processing parameters to form an image special effect and the image special effect is displayed on the display device.
  • the special effect processing includes:
  • One or more of filter processing, deformation processing, sticker processing, and animation processing are present.
  • the image processing apparatus 500 may further include:
  • the sampling setting module is used to set the sampling rate and the number of sampling bits for voice collection.
  • the image processing apparatus 500 may further include:
  • the correspondence relationship setting module is used to set the correspondence relationship between predetermined semantics and image special effects, wherein the image special effects include one or more special effect processing required to generate the image special effects.
  • the device shown in FIG. 5 can execute the methods of the embodiments shown in FIG. 1, FIG. 2, FIG. 3, and FIG. 4.
  • FIG. 1, FIG. 2, FIG. 3, and FIG. Description of the embodiment please refer to FIG. 1, FIG. 2, FIG. 3, and FIG. Description of the embodiment.
  • FIG. 6 shows a schematic structural diagram of an electronic device 600 suitable for implementing embodiments of the present disclosure.
  • the electronic devices in the embodiments of the present disclosure may include, but are not limited to, mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablets), PMPs (portable multimedia players), vehicle-mounted terminals (e.g. Mobile terminals such as car navigation terminals) and fixed terminals such as digital TVs, desktop computers, etc.
  • the electronic device shown in FIG. 6 is only an example, and should not bring any limitation to the function and scope of use of the embodiments of the present disclosure.
  • the electronic device 600 may include a processing device (such as a central processing unit, a graphics processor, etc.) 601, which can be loaded into a random access device according to a program stored in a read-only memory (ROM) 602 or from a storage device 608.
  • the program in the memory (RAM) 603 executes various appropriate actions and processing.
  • the RAM 603 also stores various programs and data required for the operation of the electronic device 600.
  • the processing device 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604.
  • An input/output (I/O) interface 605 is also connected to the bus 604.
  • the following devices can be connected to the I/O interface 605: including input devices 606 such as touch screen, touch panel, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; including, for example, liquid crystal display (LCD), speakers, An output device 607 such as a vibrator; a storage device 608 such as a magnetic tape, a hard disk, etc.; and a communication device 609.
  • the communication device 609 may allow the electronic device 600 to perform wireless or wired communication with other devices to exchange data.
  • FIG. 6 shows an electronic device 600 having various devices, it should be understood that it is not required to implement or have all the illustrated devices. It may alternatively be implemented or provided with more or fewer devices.
  • an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart.
  • the computer program may be downloaded and installed from the network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602.
  • the processing device 601 the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.
  • the aforementioned computer-readable medium in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two.
  • the computer-readable storage medium may be, for example, but not limited to, an electric, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable Programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.
  • a computer-readable storage medium may be any tangible medium that contains or stores a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device.
  • a computer-readable signal medium may include a data signal propagated in a baseband or as a part of a carrier wave, and a computer-readable program code is carried therein. This propagated data signal can take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.
  • the computer-readable signal medium may also be any computer-readable medium other than the computer-readable storage medium, and the computer-readable signal medium may send, propagate, or transmit for use by or in combination with the instruction execution system, apparatus, or device. program.
  • the program code contained on the computer-readable medium can be transmitted by any suitable medium, but not limited to: wire, optical cable, RF (Radio Frequency), etc., or any suitable combination of the above.
  • the above-mentioned computer-readable medium may be included in the above-mentioned electronic device; or it may exist alone without being assembled into the above-mentioned electronic device.
  • the computer-readable medium described above carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device: obtains the original image from the image source and displays the original image on the display device ; Collect the voice in the environment and recognize the semantics of the voice; in response to recognizing predetermined semantics from the voice, trigger special effects processing, wherein the predetermined semantics are preset and related to one or more special effects processing The semantics of the connection; according to the triggered special effect processing, the original image is processed to form and display image special effects.
  • the computer program code used to perform the operations of the present disclosure may be written in one or more programming languages or a combination thereof.
  • the above-mentioned programming languages include object-oriented programming languages-such as Java, Smalltalk, C++, and also conventional Procedural programming language-such as "C" language or similar programming language.
  • the program code can be executed entirely on the user's computer, partly on the user's computer, executed as an independent software package, partly on the user's computer and partly executed on a remote computer, or entirely executed on the remote computer or server.
  • the remote computer can be connected to the user's computer through any kind of network including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (for example, using an Internet service provider to pass Internet connection).
  • LAN local area network
  • WAN wide area network
  • each block in the flowchart or block diagram may represent a module, program segment, or part of code, and the module, program segment, or part of code contains one or more logic for implementing prescribed Function executable instructions.
  • the functions marked in the block may also occur in a different order from the order marked in the drawings. For example, two blocks shown in succession can actually be executed substantially in parallel, or they can sometimes be executed in the reverse order, depending on the functions involved.
  • each block in the block diagram and/or flowchart, and the combination of the blocks in the block diagram and/or flowchart can be implemented by a dedicated hardware-based system that performs the specified functions or operations Or it can be realized by a combination of dedicated hardware and computer instructions.
  • the units involved in the embodiments described in the present disclosure may be implemented in a software manner, or may be implemented in a hardware manner. Among them, the name of the unit does not constitute a limitation on the unit itself under certain circumstances.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Acoustics & Sound (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

本公开公开一种图像特效的触发方法、装置和硬件装置。其中，所述图像特效的触发方法包括：从图像源获取原始图像并在显示装置上显示所述原始图像；采集环境中的语音并对所述语音的语义进行识别；响应于从所述语音中识别出预定语义，触发特效处理，其中所述预定语义为预先设置的、跟一个或多个特效处理相关联的语义；根据触发的特效处理，对所述原始图像进行处理以形成并显示图像特效。本公开实施例的图像特效的触发方法，通过对语音语义的识别来触发对图像的处理效果，解决了现有技术图像效果触发不灵活，无法解放用户的双手的技术问题。

Description

图像特效的触发方法、装置和硬件装置
相关申请的交叉引用
本申请要求于2019年02月28日提交的，申请号为201910151013.1、发明名称为“图像特效的触发方法、装置和硬件装置”的中国专利申请的优先权，该申请的全文通过引用结合在本申请中。
技术领域
本公开涉及图像处理领域，特别是涉及一种图像特效的触发方法、装置和硬件装置。
背景技术
随着计算机技术的发展，智能终端的应用范围得到了广泛的提高，例如可以通过智能终端听音乐、玩游戏、上网聊天和拍照等。对于智能终端的拍照技术来说，其拍照像素已经达到千万像素以上，具有较高的清晰度和媲美专业相机的拍照效果。
目前在采用智能终端进行拍照时，不仅可以使用出厂时内置的拍照软件实现传统功能的拍照效果，还可以通过从网络端下载应用程序(Application，简称为：APP)来实现具有附加功能的拍照效果或者给视频加上特殊效果，例如可以实现各种滤镜效果。
然而现有技术中的效果添加需要用户手动选择，比如选择一个贴纸或者滤镜，特别是当用户的手上都有东西时，不方便用手操作，现有技术无法解放用户的双手。
发明内容
根据本公开的一个方面，提供以下技术方案：
一种图像特效的触发方法，包括：
从图像源获取原始图像并在显示装置上显示所述原始图像；
采集环境中的语音并对所述语音的语义进行识别；
响应于从所述语音中识别出预定语义，触发特效处理，其中所述预定语义为预先设置的、跟一个或多个特效处理相关联的语义；
根据触发的特效处理，对所述原始图像进行处理以形成并显示图像特效。
进一步的，所述从图像源获取原始图像并在显示装置上显示所述原始图像包括：
通过图像采集装置获取图片或者视频并将所述图像或视频显示在显示装置上。
进一步的，所述采集环境中的语音并对所述语音的语义进行识别包括：
响应于第一触发信号，采集环境中的语音；
对所述语音的语义进行识别。
进一步的，所述第一触发信号包括：
对终端设备的屏幕的预定位置的触摸所产生的信号或者在所述原始图像中识别出预定的事件所产生的信号中的任一个。
进一步的，在所述采集环境中的语音并对所述语音的语义进行识别之前，还包括：
加载特效包并解析能触发特效包中的特效的预定语义。
进一步的，所述采集环境中的语音并对所述语音的语义进行识别，包括：
采集环境中的语音数据；
将所述语音数据转换成文本数据；
对所述文本数据进行分词处理得到至少一个语义。
进一步的，所述响应于从所述语音中识别出预定语义，触发特效处理，其中所述预定语义为预先设置的、跟一个或多个特效处理相关联的语义，包括：
比较从语音中识别出的语义和预定语义；
如果所述从语音中识别出的语义中包含预定语义，获取与所述预定语义所对应的特效处理的处理参数；
其中所述预定语义为预先设置的、跟一个或多个特效处理相对应的语义。
进一步的，所述根据触发的特效处理，对所述原始图像进行处理以形成并显示图像特效，包括：
根据所述处理参数对原始图像进行处理以形成图像特效并将所述图像特效显示在所述显示装置上。
进一步的，所述特效处理包括：
滤镜处理、形变处理、贴纸处理和动画处理中的一个或多个。
进一步的，在采集环境中的语音并对所述语音的语义进行识别之前，还包括：
设置语音采集的采样率以及采样位数。
进一步的，在响应于从所述语音中识别出预定语义，触发特效处理之前，还包括：
设置预定语义和图像特效的对应关系，其中所述图像特效包括产生所述图像特效所需要的一个或多个特效处理。
根据本公开的另一个方面，还提供以下技术方案：
一种图像特效的触发装置，包括：
原始图像获取模块，用于获取原始图像；
语义识别模块，用于采集环境中的语音并对语音的语义进行识别；
特效处理触发模块，用于响应于从所述语音中识别出预定语义，触发 特效处理，其中所述预定语义为预先设置的、跟一个或多个特效处理相关联的语义；
图像特效生成模块，用于根据触发的特效处理，对所述原始图像进行处理以形成图像特效。
进一步的，所述原始图像获取模块，还用于：
通过图像采集装置获取图片或者视频并将所述图像或视频显示在显示装置上。
进一步的，所述语义识别模块，还包括：
语音采集模块，用于响应于第一触发信号，采集环境中的语音；
第一语义识别模块，用于对所述语音的语义进行识别。
进一步的，所述第一触发信号包括：
对终端设备的屏幕的预定位置的触摸所产生的信号或者在所述原始图像中识别出预定的事件所产生的信号中的任一个。
进一步的，所述装置，还包括：
特效包加载解析模块，用于加载特效包并解析能触发特效包中的特效的预定语义。
进一步的，所述语义识别模块，还包括：
语音数据采集模块，用于采集环境中的语音数据；
语音数据转换模块，用于将所述语音数据转换成文本数据；
第二语义识别模块，用于对所述文本数据进行分词处理得到至少一个语义。
进一步的，所述特效处理触发模块，还包括：
处理参数获取模块，用于比较从语音中识别出的语义和预定语义；如果所述从语音中识别出的语义中包含预定语义，获取与所述预定语义所对应的特效处理的处理参数；其中所述预定语义为预先设置的、跟一个或多个特效处理相对应的语义。
进一步的，所述图像特效生成模块，还用于：
根据所述处理参数对原始图像进行处理以形成图像特效并将所述图像特效显示在所述显示装置上。
进一步的，所述特效处理包括：
滤镜处理、形变处理、贴纸处理和动画处理中的一个或多个。
进一步的，所述图像的处理装置，还可以包括：
采样设置模块，用于设置语音采集的采样率以及采样位数。
进一步的，所述图像的处理装置，还可以包括：
对应关系设置模块，用于设置预定语义和图像特效的对应关系，其中所述图像特效包括产生所述图像特效所需要的一个或多个特效处理。根据本公开的又一个方面，还提供以下技术方案：
一种电子设备，包括：存储器，用于存储非暂时性计算机可读指令；以及处理器，用于运行所述计算机可读指令，使得所述处理器执行时实现上述任一图像特效的触发方法所述的步骤。
根据本公开的又一个方面，还提供以下技术方案：
一种计算机可读存储介质，用于存储非暂时性计算机可读指令，当所述非暂时性计算机可读指令由计算机执行时，使得所述计算机执行上述任一方法中所述的步骤。
本公开公开一种图像特效的触发方法、装置和硬件装置。其中，所述图像特效的触发方法包括：从图像源获取原始图像并在显示装置上显示所 述原始图像；采集环境中的语音并对所述语音的语义进行识别；响应于从所述语音中识别出预定语义，触发特效处理，其中所述预定语义为预先设置的、跟一个或多个特效处理相关联的语义；根据触发的特效处理，对所述原始图像进行处理以形成并显示图像特效。本公开实施例的图像特效的触发方法，通过对语音语义的识别来触发对图像的处理效果，解决了现有技术图像效果触发不灵活，无法解放用户的双手的技术问题。
上述说明仅是本公开技术方案的概述，为了能更清楚了解本公开的技术手段，而可依照说明书的内容予以实施，并且为让本公开的上述和其他目的、特征和优点能够更明显易懂，以下特举较佳实施例，并配合附图，详细说明如下。
附图说明
图1为根据本公开一个实施例的图像特效的触发方法的流程示意图；
图2为根据本公开一个实施例的图像特效的触发方法的步骤S102进一步的流程示意图；
图3为根据本公开一个实施例的图像特效的触发方法的步骤S102又一进一步的流程示意图；
图4为根据本公开一个实施例的图像特效的触发方法的进一步的流程示意图；
图5为根据本公开一个实施例的图像特效的触发装置的结构示意图；
图6为根据本公开实施例提供的电子设备的结构示意图。
具体实施方式
以下通过特定的具体实例说明本公开的实施方式，本领域技术人员可由本说明书所揭露的内容轻易地了解本公开的其他优点与功效。显然，所 描述的实施例仅仅是本公开一部分实施例，而不是全部的实施例。本公开还可以通过另外不同的具体实施方式加以实施或应用，本说明书中的各项细节也可以基于不同观点与应用，在没有背离本公开的精神下进行各种修饰或改变。需说明的是，在不冲突的情况下，以下实施例及实施例中的特征可以相互组合。基于本公开中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本公开保护的范围。
需要说明的是，下文描述在所附权利要求书的范围内的实施例的各种方面。应显而易见，本文中所描述的方面可体现于广泛多种形式中，且本文中所描述的任何特定结构及/或功能仅为说明性的。基于本公开，所属领域的技术人员应了解，本文中所描述的一个方面可与任何其它方面独立地实施，且可以各种方式组合这些方面中的两者或两者以上。举例来说，可使用本文中所阐述的任何数目个方面来实施设备及/或实践方法。另外，可使用除了本文中所阐述的方面中的一或多者之外的其它结构及/或功能性实施此设备及/或实践此方法。
还需要说明的是，以下实施例中所提供的图示仅以示意方式说明本公开的基本构想，图式中仅显示与本公开中有关的组件而非按照实际实施时的组件数目、形状及尺寸绘制，其实际实施时各组件的型态、数量及比例可为一种随意的改变，且其组件布局型态也可能更为复杂。
另外，在以下描述中，提供具体细节是为了便于透彻理解实例。然而，所属领域的技术人员将理解，可在没有这些特定细节的情况下实践所述方面。
本公开实施例提供一种图像特效的触发方法。本实施例提供的所述图像特效的触发方法可以由一计算装置来执行，所述计算装置可以实现为软件，或者实现为软件和硬件的组合，所述计算装置可以集成设置在服务器、终端设备等中。如图1所示，所述图像特效的触发方法主要包括如下步骤 S101至步骤S104。其中：
步骤S101：从图像源获取原始图像并在显示装置上显示所述原始图像；
在所述实施例中，所述图像源可以是各种图像传感器，从图像源获取原始图像可以是通过图像传感器获取，所述图像传感器指可以采集图像的各种设备，典型的图像传感器为摄像机、摄像头、相机等。在所述实施例中，所述图像传感器可以是终端设备上的摄像头，比如智能手机上的前置或者后置摄像头，摄像头采集的图像可以直接显示在手机的显示屏上。
在一个实施例中，所述从图像源获取原始图像，可以是获取当前终端设备所采集到的视频的当前图像帧，由于视频是由多个图像帧组成的，在所述实施例中获取视频图像，将所述视频图像中的视频帧图像作为原始图像。所述获取原始图像，还可以是获取当前终端设备所采集到的图片。
在一个实施例中，所述图像源为本地或者网络中的存储器，所述从图像源获取原始图像可以是从本地存储设备或者网络地址所指向的存储设备中获取任何形式的图像，如静态的图片、动态的图片或者各种格式的视频等等，在此不做限制。
在步骤中，获取到原始图像后还将所述原始图像显示在显示装置上，所述显示装置可以是执行所述图像特效的触发方法的终端、服务器或者接收所述原始图像的终端、服务器的显示屏等。
步骤S102：采集环境中的语音并对所述语音的语义进行识别；
如图2所示，在所述步骤的一个实施方式中，所述采集环境中的语音并对语音的语义进行识别可以包括：步骤S201，响应于第一触发信号，采集环境中的语音；步骤S202，对所述语音的语义进行识别。在一个实施例中，所述第一触发信号，由一定的条件触发。可选的，当用户打开特效应用程序，选择特定的特效包，此时可以产生第一触发信号，第一触发信号 指示终端设备开始采集环境中的语音，在所述可选方式中，特效包一般以屏幕上的接口的形式出现，如预定位置上的按钮或者图标等等，因此对终端设备的屏幕的预定位置的触摸可以产生所述第一触发信号；可选的，也可以在当在所述原始图像中识别出预定的事件时，产生所述的第一触发信号，如当所述原始图像中识别出人脸，即可以产生第一触发信号，指示终端设备开始采集环境中的语音；可以理解的是，第一触发信号的产生不限于上述方式，上述方式仅仅是举例，在实际应用中可以设置任何合适的信号作为采集环境中的语音的触发信号，在此不再赘述。
可选的，在上述采集环境中的语音之前，还可以包括加载特效包并解析能触发特效包中的特效的语义。在所述实施方式中，所述第一触发信号可以由对终端设备的屏幕的预定位置的触摸产生，特别的，当用户触摸特定特效包的图标时，产生第一触发信号，此时在采集环境中的语音之前，首先需要加载特效包并解析出能触发所述特效包中所包含的特效的语义。可选的，所述特效包中的特效可以包括多种，其中所述多种特效的触发语义可以不同，比如语音“新年快乐”中可以识别出语义“新年”和“快乐”，新年可以对应到特效包中的鞭炮特效，“快乐”可以对应到特效包的笑脸特效等等。
如图3所示，在所述步骤的一个实施方式中，所述采集环境中的语音并对语音的语义进行识别，包括：步骤S301，采集环境中的语音数据；步骤S302，将所述语音数据转换成文本数据；步骤S303，对所述文本数据进行分词处理得到至少一个语义。采集环境中的语音数据，首先需要通过声音收集装置收集环境中的语音，典型的所述声音收集装置可以是麦克风等，通过声音收集装置收集到的语音是连续的模拟信号，此时可以通过采样将语音采样成PCM(Pulse Code Modulation，脉冲编码调制)信号，可选的，在采集环境中的语音并对语音的语义进行识别之前还可以包括步骤：设置 语音采集的采样率以及采样位数。典型的采样率和采样位数可以为44.1KHz、16bit，16KHz、16bit等等，实际应用中的采样率和采样位数不限于上述例子，如涉及到性能或者传输速度等因素，可以适当调低采样率或采样位数，在此不再赘述。
在采集到语音数据之后，将所述语音数据转换成文本数据。可选的，在所述步骤中，为了转换的精确，首先对语音数据进行预处理，典型的预处理可以包括将语音数据转换为特定格式的音频文件，如WAV(Waveform Audio File Format)、FLAC(Free Lossless Audio Codec)等等；典型的预处理还可以包括噪声抑制，在采集环境中的语音的同时，往往会采集到环境中的其他声音，为了减少环境噪音的干扰，需要对采集到的语音数据进行降噪处理；典型的预处理还可以包括无损压缩，如果语音的识别装置为线上装置，需要通过本地设备将语音数据传输到线上进行识别，则可以通过无损压缩的方式减小语音数据的体积，节省网络资源，同时可以减少语音数据的传输时间，以加快语音识别的速度。上述的语音数据的格式转换、噪声抑制处理以及无损压缩处理可以使用任何具体的实现方式来实现，本公开不做具体限制。在预处理之后，将语音数据输入语音识别装置，将语音转换成文本数据。在所述步骤中，一般可以提取语音数据中的语音特征，与语音模板中的语音特征进行比对，输出匹配的语音的文本数据，最终形成语音数据的文本数据。可以理解的是，上述语音转文本的方式仅仅是举例，实际上可以使用各种方式，如深度学习等方式，将语音转换成文本。
在得到语音的文本数据之后，对所述文本数据进行分词处理得到至少一个语义。此处的分词可以根据需要进行不同的分词，如“新年快乐”可以分为一个语义，也可以分为“新年”和“快乐”两个语义。
可以理解的是，所述步骤S102中识别语音的语义的方式，可以使用任何方式，如通过识别模型将语音数据输入模型，直接识别出需要的语义； 此外，在所述步骤中所输出的语义，可以是可以触发上述特效包的特效的语义，如果无法触发特效包的特效，则可以不识别，此时需要对识别模型进行训练，使其只能识别出预定语义，而不是识别出所有的语义。此处的语义也可以包括意思近似的词，如“新年”和“春节”可以被识别为一个意思等等，可以通过同义词语音模板或者训练模型实现，在此不再赘述。
可以理解的是，所述步骤可以在本地执行也可以在线上执行，也就是说语音语义识别系统可以在本地也可以在线上，在应用场景不同时可以进行不同的设置，本公开对此不做限制。
步骤S103：响应于从所述语音中识别出预定语义，触发特效处理，其中所述预定语义为预先设置的、跟一个或多个特效处理相关联的语义；
在所述步骤中，所述预定语义为预先设置的，可以触发特效的语义，如果在上述步骤中包括加载特效包并解析能触发特效包中的特效的语义的步骤，则此处的预定语义为在所述加载步骤中解析出来的语义。该
在一个具体的实施方式中，所述响应于从所述语音中识别出预定语义，触发特效处理，包括：比较从语音中识别出的语义和预定语义；如果所述从语音中识别出的语义中包含预定语义，获取与所述预定语义所对应的特效处理的处理参数；其中所述预定语义为预先设置的、跟一个或多个特效处理相对应的语义。如果所述语义中包含预定语义，获取与所述预定语义所对应的特效处理参数。在所述实施方式中，所述语义可以包含多个语义，如用户说出“恭贺大家，新年快乐”，此时通过步骤S102，可以将所述语音的语义识别成“恭贺”、“大家”、“新年”、“快乐”，如果预定语义为“新年”，则语义中包含预定语义，则接着进行下一步骤，如果预定语义为“吃饭”，则不执行下一步骤。在所述具体实施方式中，当所述语义包含预定语义，则需要进一步获取与所述预定语义对应的特效处理参数，此处的特效处理，可以为滤镜处理、形变处理、贴纸处理和动画处理中的一个或多个。预定语义对应的特效处理可以从特效包中获取，在解析特效包时可以得到语义与特效的对应关系，而特效又对应于某种或某几种特效 处理，这样语义就与特效处理对应起来。可以理解的，所述的特效处理参数可以为特效处理的类型以及执行所述特效处理时所需要的资源以及特征参数，如在贴纸处理中，其类型为贴纸，资源为贴纸所使用的图片或帧，特征参数为贴纸的显示位置、显示时间等等。
可以理解的，所述特效处理不限于上述所举例子，实际上任何特效处理都可以用于本公开中，所述特效处理的处理参数，也不限于上述处理参数，本公开不对此做具体限定。
步骤S104：根据触发的特效处理，对所述原始图像进行处理以形成并显示图像特效。
在获取了与所述预定语义所对应的特效处理之后，根据所述特效处理的处理参数对原始图像进行处理以形成图像特效。以动画处理为例，当用户说出“恭贺大家，新年快乐”时，语义识别出“新年”，如果特效包中的预定语义包含“新年”，则触发特效包中的动画处理对原始图像进行处理，在原始图像的预定位置上加入点燃鞭炮的动画，并播放鞭炮的背景声音，得到带有放鞭炮动画特效的图像。再以滤镜为例，当用户说出“我怎么这么好看”，语义识别出“好看”，如果特效包中的预定语义包含“好看”，则触发特效包中的美颜滤镜，对原始图像中的人脸进行美颜处理得到美颜之后的人脸图像。在生成上述图像特效之后，将所述图像特效显示在显示装置上，所述显示装置可以是执行所述图像特效的触发方法的终端、服务器或者接收所述特效图像的终端、服务器的显示屏等。
为了增强灵活性，如图4所示，在步骤S103之前，还可以包括步骤S401：设置语义和图像特效的对应关系，其中所述图像特效包括产生所述图像特效所需要的特效处理。所述步骤可以在生成特效包时执行，即可以通过设置语义与特效包中的一个或多个图像特效的对应关系，来设置图像特效的触发条件，可以理解的，所述对应关系可以保存在特效包的配置文件中， 所述特效包的配置文件除了包括上述对应关系之外，还可以包括特效包中所包含的图像特效、图像特效所包含的素材的存储位置、图像特效所需要的特效处理，特效处理的处理参数等等，通过所述配置文件可以配置特效包的特效的任何属性如特效的显示位置、显示时间、显示方式以及处理强度等等，在此不再一一举例。另外为了增强用户的使用灵活性，此处的设置语义和图像特效的对应关系，可以是在用户使用时，自定义语义和图像特效的对应关系，此时，用户可以自己录制语音而将所述语音的语义作为某个图像特效的触发条件。
本公开公开一种图像特效的触发方法、装置和硬件装置。其中，所述图像特效的触发方法包括：从图像源获取原始图像并在显示装置上显示所述原始图像；采集环境中的语音并对所述语音的语义进行识别；响应于从所述语音中识别出预定语义，触发特效处理，其中所述预定语义为预先设置的、跟一个或多个特效处理相关联的语义；根据触发的特效处理，对所述原始图像进行处理以形成并显示图像特效。本公开实施例的图像特效的触发方法，通过对语音语义的识别来触发对图像的处理效果，解决了现有技术图像效果触发不灵活，无法解放用户的双手的技术问题。
在上文中，虽然按照上述的顺序描述了上述方法实施例中的各个步骤，本领域技术人员应清楚，本公开实施例中的步骤并不必然按照上述顺序执行，其也可以倒序、并行、交叉等其他顺序执行，而且，在上述步骤的基础上，本领域技术人员也可以再加入其他步骤，这些明显变型或等同替换的方式也应包含在本公开的保护范围之内，在此不再赘述。
下面为本公开装置实施例，本公开装置实施例可用于执行本公开方法实施例实现的步骤，为了便于说明，仅示出了与本公开实施例相关的部分，具体技术细节未揭示的，请参照本公开方法实施例。
本公开实施例提供一种图像的处理装置。所述装置可以执行上述图像 特效的触发方法实施例中所述的步骤。如图5所示，所述装置500主要包括：原始图像获取模块501、语义识别模块502、特效处理触发模块503和图像特效生成模块504。其中，
原始图像获取模块501，从图像源获取原始图像并在显示装置上显示所述原始图像；
语义识别模块502，用于采集环境中的语音并对所述语音的语义进行识别；
特效处理触发模块503，用于响应于从所述语音中识别出预定语义，触发特效处理，其中所述预定语义为预先设置的、跟一个或多个特效处理相关联的语义；
图像特效生成模块504，用于根据触发的特效处理，对所述原始图像进行处理以形成并显示图像特效。
进一步的，所述原始图像获取模块501，还用于：
通过图像采集装置获取图片或者视频并将所述图像或视频显示在显示装置上。
进一步的，所述语义识别模块502，还包括：
语音采集模块，用于响应于第一触发信号，采集环境中的语音；
第一语义识别模块，用于对所述语音的语义进行识别。
进一步的，所述第一触发信号包括：
对终端设备的屏幕的预定位置的触摸所产生的信号或者在所述原始图像中识别出预定的事件所产生的信号中的任一个。
进一步的，所述装置500，还包括：
特效包加载解析模块，用于加载特效包并解析能触发特效包中的特效 的预定语义。
进一步的，所述语义识别模块502，还包括：
语音数据采集模块，用于采集环境中的语音数据；
语音数据转换模块，用于将所述语音数据转换成文本数据；
第二语义识别模块，用于对所述文本数据进行分词处理得到至少一个语义。
进一步的，所述特效处理触发模块503，还包括：
处理参数获取模块，用于比较从语音中识别出的语义和预定语义；如果所述从语音中识别出的语义中包含预定语义，获取与所述预定语义所对应的特效处理的处理参数；其中所述预定语义为预先设置的、跟一个或多个特效处理相对应的语义。
进一步的，所述图像特效生成模块504，还用于：
根据所述处理参数对原始图像进行处理以形成图像特效并将所述图像特效显示在所述显示装置上。
进一步的，所述特效处理包括：
滤镜处理、形变处理、贴纸处理和动画处理中的一个或多个。
进一步的，所述图像的处理装置500，还可以包括：
采样设置模块，用于设置语音采集的采样率以及采样位数。
进一步的，所述图像的处理装置500，还可以包括：
对应关系设置模块，用于设置预定语义和图像特效的对应关系，其中所述图像特效包括产生所述图像特效所需要的一个或多个特效处理。
图5所示装置可以执行图1、图2、图3和图4所示实施例的方法，本实施例未详细描述的部分，可参考对图1、图2、图3和图4所示实施例的 相关说明。所述技术方案的执行过程和技术效果参见图1、图2、图3和图4所示实施例中的描述，在此不再赘述。
下面参考图6，其示出了适于用来实现本公开实施例的电子设备600的结构示意图。本公开实施例中的电子设备可以包括但不限于诸如移动电话、笔记本电脑、数字广播接收器、PDA(个人数字助理)、PAD(平板电脑)、PMP(便携式多媒体播放器)、车载终端(例如车载导航终端)等等的移动终端以及诸如数字TV、台式计算机等等的固定终端。图6示出的电子设备仅仅是一个示例，不应对本公开实施例的功能和使用范围带来任何限制。
如图6所示，电子设备600可以包括处理装置(例如中央处理器、图形处理器等)601，其可以根据存储在只读存储器(ROM)602中的程序或者从存储装置608加载到随机访问存储器(RAM)603中的程序而执行各种适当的动作和处理。在RAM 603中，还存储有电子设备600操作所需的各种程序和数据。处理装置601、ROM 602以及RAM 603通过总线604彼此相连。输入/输出(I/O)接口605也连接至总线604。
通常，以下装置可以连接至I/O接口605：包括例如触摸屏、触摸板、键盘、鼠标、图像传感器、麦克风、加速度计、陀螺仪等的输入装置606；包括例如液晶显示器(LCD)、扬声器、振动器等的输出装置607；包括例如磁带、硬盘等的存储装置608；以及通信装置609。通信装置609可以允许电子设备600与其他设备进行无线或有线通信以交换数据。虽然图6示出了具有各种装置的电子设备600，但是应理解的是，并不要求实施或具备所有示出的装置。可以替代地实施或具备更多或更少的装置。
特别地，根据本公开的实施例，上文参考流程图描述的过程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品， 其包括承载在计算机可读介质上的计算机程序，所述计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，所述计算机程序可以通过通信装置609从网络上被下载和安装，或者从存储装置608被安装，或者从ROM 602被安装。在所述计算机程序被处理装置601执行时，执行本公开实施例的方法中限定的上述功能。
需要说明的是，本公开上述的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本公开中，计算机可读存储介质可以是任何包含或存储程序的有形介质，所述程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本公开中，计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，所述计算机可读信号介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：电线、光缆、RF(射频)等等，或者上述的任意合适的组合。
上述计算机可读介质可以是上述电子设备中所包含的；也可以是单独存在，而未装配入所述电子设备中。
上述计算机可读介质承载有一个或者多个程序，当上述一个或者多个程序被所述电子设备执行时，使得所述电子设备：从图像源获取原始图像并在显示装置上显示所述原始图像；采集环境中的语音并对所述语音的语义进行识别；响应于从所述语音中识别出预定语义，触发特效处理，其中所述预定语义为预先设置的、跟一个或多个特效处理相关联的语义；根据触发的特效处理，对所述原始图像进行处理以形成并显示图像特效。
可以以一种或多种程序设计语言或其组合来编写用于执行本公开的操作的计算机程序代码，上述程序设计语言包括面向对象的程序设计语言-诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言-诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络——包括局域网(LAN)或广域网(WAN)-连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。
附图中的流程图和框图，图示了按照本公开各种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，所述模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
描述于本公开实施例中所涉及到的单元可以通过软件的方式实现，也可以通过硬件的方式来实现。其中，单元的名称在某种情况下并不构成对所述单元本身的限定。
以上描述仅为本公开的较佳实施例以及对所运用技术原理的说明。本领域技术人员应当理解，本公开中所涉及的公开范围，并不限于上述技术特征的特定组合而成的技术方案，同时也应涵盖在不脱离上述公开构思的情况下，由上述技术特征或其等同特征进行任意组合而形成的其它技术方案。例如上述特征与本公开中公开的(但不限于)具有类似功能的技术特征进行互相替换而形成的技术方案。

Claims (14)

  1. 一种图像特效的触发方法，包括：
    从图像源获取原始图像并在显示装置上显示所述原始图像；
    采集环境中的语音并对所述语音的语义进行识别；
    响应于从所述语音中识别出预定语义，触发特效处理，其中所述预定语义为预先设置的、跟一个或多个特效处理相关联的语义；
    根据触发的特效处理，对所述原始图像进行处理以形成并显示图像特效。
  2. 如权利要求1所述的图像特效的触发方法，其中所述从图像源获取原始图像并在显示装置上显示所述原始图像包括：
    通过图像采集装置获取图片或者视频并将所述图像或视频显示在显示装置上。
  3. 如权利要求1所述的图像特效的触发方法，其中所述采集环境中的语音并对所述语音的语义进行识别包括：
    响应于第一触发信号，采集环境中的语音；
    对所述语音的语义进行识别。
  4. 如权利要求3所述的图像特效的触发方法，其中所述第一触发信号包括：
    对终端设备的屏幕的预定位置的触摸所产生的信号或者在所述原始图像中识别出预定的事件所产生的信号中的任一个。
  5. 如权利要求3所述的图像特效的触发方法，其中在所述采集环境中的语音并对所述语音的语义进行识别之前，还包括：
    加载特效包并解析能触发特效包中的特效的预定语义。
  6. 如权利要求1所述的图像特效的触发方法，其中所述采集环境中的语音并对所述语音的语义进行识别，包括：
    采集环境中的语音数据；
    将所述语音数据转换成文本数据；
    对所述文本数据进行分词处理得到至少一个语义。
  7. 如权利要求1所述的图像特效的触发方法，其中所述响应于从所述语音中识别出预定语义，触发特效处理，其中所述预定语义为预先设置的、跟一个或多个特效处理相关联的语义，包括：
    比较从语音中识别出的语义和预定语义；
    如果所述从语音中识别出的语义中包含预定语义，获取与所述预定语义所对应的特效处理的处理参数；
    其中所述预定语义为预先设置的、跟一个或多个特效处理相对应的语义。
  8. 如权利要求7所述的图像特效的触发方法，其中所述根据触发的特效处理，对所述原始图像进行处理以形成并显示图像特效，包括：
    根据所述处理参数对原始图像进行处理以形成图像特效并将所述图像特效显示在所述显示装置上。
  9. 如权利要求1所述的图像特效的触发方法，其中所述特效处理包括：
    滤镜处理、形变处理、贴纸处理和动画处理中的一个或多个。
  10. 如权利要求1所述的图像特效的触发方法，其中在采集环境中的语音并对所述语音的语义进行识别之前，还包括：
    设置语音采集的采样率以及采样位数。
  11. 如权利要求1所述的图像特效的触发方法，其中在响应于从所述语音中识别出预定语义，触发特效处理之前，还包括：
    设置预定语义和图像特效的对应关系，其中所述图像特效包括产生所述图像特效所需要的一个或多个特效处理。
  12. 一种图像特效的触发装置，包括：
    原始图像获取模块，用于获取原始图像；
    语义识别模块，用于采集环境中的语音并对语音的语义进行识别；
    特效处理触发模块，用于响应于从所述语音中识别出预定语义，触发特效处理，其中所述预定语义为预先设置的、跟一个或多个特效处理相关 联的语义；
    图像特效生成模块，用于根据触发的特效处理，对所述原始图像进行处理以形成图像特效。
  13. 一种电子设备，包括：
    存储器，用于存储非暂时性计算机可读指令；以及
    处理器，用于运行所述计算机可读指令，使得所述处理器执行时实现根据权利要求1-11中任意一项所述的图像特效的触发方法。
  14. 一种计算机可读存储介质，用于存储非暂时性计算机可读指令，当所述非暂时性计算机可读指令由计算机执行时，使得所述计算机执行权利要求1-11中任意一项所述的图像特效的触发方法。
PCT/CN2019/128744 2019-02-28 2019-12-26 图像特效的触发方法、装置和硬件装置 Ceased WO2020173211A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
SG11202109345WA SG11202109345WA (en) 2019-02-28 2019-12-26 Method and apparatus for triggering special image effects and hardware device
US17/459,739 US11595591B2 (en) 2019-02-28 2021-08-27 Method and apparatus for triggering special image effects and hardware device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910151013.1 2019-02-28
CN201910151013.1A CN111629156A (zh) 2019-02-28 2019-02-28 图像特效的触发方法、装置和硬件装置

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/459,739 Continuation US11595591B2 (en) 2019-02-28 2021-08-27 Method and apparatus for triggering special image effects and hardware device

Publications (1)

Publication Number Publication Date
WO2020173211A1 true WO2020173211A1 (zh) 2020-09-03

Family

ID=72239043

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/128744 Ceased WO2020173211A1 (zh) 2019-02-28 2019-12-26 图像特效的触发方法、装置和硬件装置

Country Status (4)

Country Link
US (1) US11595591B2 (zh)
CN (1) CN111629156A (zh)
SG (1) SG11202109345WA (zh)
WO (1) WO2020173211A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112950735A (zh) * 2021-03-04 2021-06-11 爱昕科技(广州)有限公司 简图与语音指令结合生成图像的方法、计算机可读存储介质及显示设备
US11778263B2 (en) * 2019-12-09 2023-10-03 Shanghai Hode Information Technology Co., Ltd. Live streaming video interaction method and apparatus, and computer device

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111667831B (zh) * 2020-06-08 2022-04-26 中国民航大学 基于管制员指令语义识别的飞机地面引导系统及方法
US12354602B2 (en) * 2020-09-10 2025-07-08 Harman International Industries, Incorporated Virtual conversation agent for controlling multiple vehicular intelligent virtual assistants
CN115101057A (zh) * 2021-03-03 2022-09-23 Oppo广东移动通信有限公司 图像的语音标注及使用方法与装置、电子装置及存储介质
CN115082490B (zh) * 2022-08-23 2022-11-15 腾讯科技（深圳）有限公司 异常预测方法、异常预测模型的训练方法、装置及设备
CN119522564A (zh) * 2023-02-16 2025-02-25 深圳引望智能技术有限公司 一种控制方法、装置和运载工具

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104780338A (zh) * 2015-04-16 2015-07-15 美国掌赢信息科技有限公司 一种即时视频中的表情特效动画加载方法和电子设备
CN106339201A (zh) * 2016-09-14 2017-01-18 北京金山安全软件有限公司 贴图处理方法、装置和电子设备
CN106791370A (zh) * 2016-11-29 2017-05-31 北京小米移动软件有限公司 一种拍摄照片的方法和装置
CN107820017A (zh) * 2017-11-30 2018-03-20 广东欧珀移动通信有限公司 图像拍摄方法、装置、计算机可读存储介质和电子设备
CN108986017A (zh) * 2018-06-29 2018-12-11 北京微播视界科技有限公司 图像特效处理方法、装置和计算机可读存储介质

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5677650B2 (ja) * 2012-11-05 2015-02-25 三菱電機株式会社 音声認識装置
CN104780459A (zh) * 2015-04-16 2015-07-15 美国掌赢信息科技有限公司 一种即时视频中的特效加载方法和电子设备
US20190189019A1 (en) * 2015-06-08 2019-06-20 Novel Effect, Inc. System and Method for Integrating Special Effects with a Text Source
US10249205B2 (en) * 2015-06-08 2019-04-02 Novel Effect, Inc. System and method for integrating special effects with a text source
CN105872838A (zh) * 2016-04-28 2016-08-17 徐文波 即时视频的媒体特效发送方法和装置
US10015546B1 (en) * 2017-07-27 2018-07-03 Global Tel*Link Corp. System and method for audio visual content creation and publishing within a controlled environment
CN109391792B (zh) * 2017-08-03 2021-10-29 腾讯科技（深圳）有限公司 视频通信的方法、装置、终端及计算机可读存储介质
CN108363557B (zh) * 2018-02-02 2020-06-12 刘国华 人机交互方法、装置、计算机设备和存储介质
WO2019168920A1 (en) * 2018-02-28 2019-09-06 Novel Effect, Inc. System and method for integrating special effects with a text source
CN207867892U (zh) * 2018-03-07 2018-09-14 佛山市云米电器科技有限公司 一种具有人脸识别功能的语音控制终端
US11501769B2 (en) * 2018-08-31 2022-11-15 Google Llc Dynamic adjustment of story time special effects based on contextual data

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104780338A (zh) * 2015-04-16 2015-07-15 美国掌赢信息科技有限公司 一种即时视频中的表情特效动画加载方法和电子设备
CN106339201A (zh) * 2016-09-14 2017-01-18 北京金山安全软件有限公司 贴图处理方法、装置和电子设备
CN106791370A (zh) * 2016-11-29 2017-05-31 北京小米移动软件有限公司 一种拍摄照片的方法和装置
CN107820017A (zh) * 2017-11-30 2018-03-20 广东欧珀移动通信有限公司 图像拍摄方法、装置、计算机可读存储介质和电子设备
CN108986017A (zh) * 2018-06-29 2018-12-11 北京微播视界科技有限公司 图像特效处理方法、装置和计算机可读存储介质

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11778263B2 (en) * 2019-12-09 2023-10-03 Shanghai Hode Information Technology Co., Ltd. Live streaming video interaction method and apparatus, and computer device
CN112950735A (zh) * 2021-03-04 2021-06-11 爱昕科技(广州)有限公司 简图与语音指令结合生成图像的方法、计算机可读存储介质及显示设备

Also Published As

Publication number Publication date
US20210392276A1 (en) 2021-12-16
CN111629156A (zh) 2020-09-04
SG11202109345WA (en) 2021-09-29
US11595591B2 (en) 2023-02-28

Similar Documents

Publication Publication Date Title
US11595591B2 (en) Method and apparatus for triggering special image effects and hardware device
US10811005B2 (en) Adapting voice input processing based on voice input characteristics
WO2021004481A1 (zh) 一种媒体文件推荐方法及装置
US10783884B2 (en) Electronic device-awakening method and apparatus, device and computer-readable storage medium
US20140046660A1 (en) Method and system for voice based mood analysis
WO2020098115A1 (zh) 字幕添加方法、装置、电子设备及计算机可读存储介质
CN106570106A (zh) 一种输入过程中将语音信息转化为表情的方法和装置
CN103442130A (zh) 语音操控方法、移动终端装置及语音操控系统
CN114120992B (zh) 语音生成视频的方法、装置、电子设备及计算机可读介质
CN108711428B (zh) 指令执行方法、装置、存储介质及电子设备
CN105100672A (zh) 显示装置及其视频通话执行方法
CN107808007A (zh) 信息处理方法和装置
WO2021212985A1 (zh) 声学网络模型训练方法、装置及电子设备
CN110246502A (zh) 语音降噪方法、装置及终端设备
WO2020102943A1 (zh) 手势识别模型的生成方法、装置、存储介质及电子设备
US11818491B2 (en) Image special effect configuration method, image recognition method, apparatus and electronic device
JP7823992B2 (ja) ビデオ生成方法、装置、端末機器および記憶媒体
CN116009864A (zh) 联机帮助语句生成方法、装置和设备
CN112114770A (zh) 基于语音交互的界面引导方法、装置及设备
CN110379406A (zh) 语音评论转换方法、系统、介质和电子设备
CN108922523A (zh) 位置提示方法、装置、存储介质及电子设备
CN107343101B (zh) 一种实现定向录音的方法、装置、设备及存储介质
CN112542157A (zh) 语音处理方法、装置、电子设备及计算机可读存储介质
CN111666068A (zh) 一种平台开发框架、构建方法、设备及介质
CN112218137B (zh) 一种多媒体数据采集方法、装置、设备及介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19917315

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 26/01/2022)

122 Ep: pct application non-entry in european phase

Ref document number: 19917315

Country of ref document: EP

Kind code of ref document: A1