EP3764631B1 - Procédé de commande d'assistant numérique et dispositif terminal - Google Patents

Procédé de commande d'assistant numérique et dispositif terminal Download PDF

Info

Publication number
EP3764631B1
EP3764631B1 EP18912032.2A EP18912032A EP3764631B1 EP 3764631 B1 EP3764631 B1 EP 3764631B1 EP 18912032 A EP18912032 A EP 18912032A EP 3764631 B1 EP3764631 B1 EP 3764631B1
Authority
EP
European Patent Office
Prior art keywords
terminal device
photographing mode
preview interface
user
scene
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP18912032.2A
Other languages
German (de)
English (en)
Other versions
EP3764631A1 (fr
EP3764631A4 (fr
Inventor
Lei Song
Xin Guo
Xiaomeng Chen
Yunchao Zhang
Xin Chen
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Publication of EP3764631A1 publication Critical patent/EP3764631A1/fr
Publication of EP3764631A4 publication Critical patent/EP3764631A4/fr
Application granted granted Critical
Publication of EP3764631B1 publication Critical patent/EP3764631B1/fr
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80—Camera processing pipelines; Components thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/64—Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/617—Upgrading or updating of programs or applications for camera control
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/62—Control of parameters via user interfaces
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/63—Control of cameras or camera modules by using electronic viewfinders
    • H04N23/631—Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/63—Control of cameras or camera modules by using electronic viewfinders
    • H04N23/631—Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
    • H04N23/632—Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/66—Remote control of cameras or camera parts, e.g. by remote control devices
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/667—Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/67—Focus control based on electronic image sensor signals

Definitions

  • This disclosure relates to the field of image photographing technologies, and in particular, to an intelligent assistant control method and a terminal device.
  • US 2014/240531 A1 relates to an image capture apparatus that controls photographing according to a photographic scene, a display control method, and a program (storage medium).
  • US 2009/237548 A1 relates to a camera, storage medium having stored therein camera control program, and camera control method for performing photography under set photography conditions.
  • a consumer manually selects a plurality of photographing modes. After trying several photographing modes, the user selects, from memory, a preferred photographing mode from the plurality of tried photographing modes for photographing. However, in this manner, the user needs to manually try a plurality of photographing modes, and the operation is cumbersome.
  • This disclosure provides an intelligent assistant control method and a terminal device, to recommend a photographing mode to a user, improve an image photographing effect, facilitate a user operation, and enhance user experience.
  • the present invention is defined according to the independent claims.
  • the dependent claims recite advantageous embodiments of the invention.
  • an embodiment of the present invention provides an intelligent assistant control method, wherein the method comprises: starting, by a terminal device, a lens; displaying, by the terminal device, a preview interface on a display screen, wherein the preview interface comprises a to-be-photographed object; identifying, by the terminal device , at least two types of scene information comprised in the to-be-photographed object; determining a first photographing mode according to a priority policy and a first mapping relationship between scene information and a photographing mode, the photographing mode corresponding to each of the at least two types of scene information, wherein the priority policy comprises priorities of the at least two photographing modes; and displaying, by the terminal device, first prompt information of the first photographing mode in the preview interface if the terminal device determines that the to-be-photographed object in the preview interface has been stably displayed for preset duration, wherein the first prompt information is used to recommend a first photographing mode to a user; wherein if the user selects another photographing
  • the terminal device recommends a proper photographing mode to the user only after determining that the preview interface is stably output. Therefore, the following case can be avoided:
  • the first prompt information is constantly displayed in the preview interface and shields a part of the preview interface, preventing a user from better searching for a photographing angle.
  • the first prompt information is constantly displayed in the preview interface, and a scenario in which the first prompt information is switched in the preview interface in real time is presented to the user, easily arousing disgust in the user.
  • the terminal device after determining that the preview interface is stably output, the terminal device considers that the user has determined a photographing angle, and then recommends an optimal photographing mode to the user. In this way, intelligence is improved, and user experience is enhanced.
  • the user may select the photographing mode recommended by the terminal device, or may select another photographing mode. If the user selects the another photographing mode, the terminal device may record a quantity of times the user selects the another photographing mode. After the quantity reaches a preset quantity of times, the terminal device updates a mapping relationship between a photographing mode and scene information, or updates the priority policy. In other words, the terminal device may record a usage habit of the user, and adjust a recommendation policy based on the usage habit of the user, thereby enhancing user experience.
  • the terminal device collects n frames of preview interfaces, where n is an integer greater than or equal to 1.
  • the terminal device identifies at least one type of scene information of the to-be-photographed object included in each of the n frames of preview interfaces.
  • the terminal device determines, based on the at least one type of scene information, a photographing mode matching each frame of preview interface.
  • the outputting, by the terminal device, first prompt information if the preview interface has been stably displayed for preset duration includes: if the preview interface has been stably displayed for the preset duration, displaying, by the terminal device, the first prompt information in the preview interface, where the first prompt information is used to recommend the first photographing mode to the user, and the first photographing mode is a photographing mode matching the last frame of preview interface in the n frames of preview interfaces.
  • the terminal device may collect a plurality of frames of preview interfaces in real time, and match each frame of preview interface with a proper photographing mode to obtain a plurality of photographing modes. After the preview interface has been stably displayed for the preset duration, the terminal device displays one of the plurality of photographing modes.
  • the terminal device after determining that the preview interface is stably output, the terminal device considers that the user has determined a photographing angle, and then recommends an optimal photographing mode to the user. In this way, intelligence is improved, and user experience is enhanced.
  • the determining, by the terminal device, whether the preview interface has been stably displayed for the preset duration includes: collecting, by the terminal device, at least two frames of preview interfaces within the preset duration; and if a similarity between the at least two frames of preview interfaces is higher than a preset similarity, determining that the preview interface has been stably displayed for the preset duration; or detecting, by the terminal device by using a motion sensor, whether the terminal device is in a still state within the preset duration; and if the terminal device is in a still state within the preset duration, determining that the preview interface has been stably displayed for the preset duration.
  • the terminal device determines, in at least two manners, whether the preview interface is stably displayed.
  • Manner 1 is determining whether the similarity between the at least two frames of preview interfaces collected within the preset duration is higher than the preset similarity.
  • Manner 2 is determining whether the terminal device is in a still state within the preset duration.
  • the first prompt information includes a first icon
  • the first icon is used to identify the first photographing mode
  • the outputting, by the terminal device, first prompt information includes: displaying, by the terminal device, the first icon in a first display region in the preview interface, where the terminal device loads, in response to a first operation on the first icon, a parameter corresponding to the first photographing mode.
  • the first prompt information is an icon corresponding to the recommended photographing mode.
  • An icon display area may be usually set to be relatively small, to avoid shielding the preview interface.
  • the user is relatively sensitive to the icon, and displaying the icon helps enhance user experience.
  • the user may start the first photographing mode by triggering the icon, for example, tapping the icon, so that a user operation is facilitated.
  • the terminal device displays the first parameter set in a second display region in the preview interface.
  • the terminal device displays a modified second parameter set in response to a second operation, where the second parameter set is a parameter set obtained after a first parameter in the first parameter set is modified.
  • the terminal device loads the second parameter set.
  • the terminal device may display the parameter, and the user may modify a value of the parameter, thereby facilitating a user operation and enhancing user experience.
  • the terminal device if the first parameter is modified more than a preset quantity of times, the terminal device establishes a third mapping relationship between the second parameter set and the first photographing mode; and the terminal device adds the third mapping relationship to the second mapping relationship.
  • the terminal device may record a quantity of times the user modifies the first parameter in the first parameter set. When the quantity of modification times is greater than the preset quantity of times, the terminal device establishes the third mapping relationship between the second parameter set and the first photographing mode. If the first photographing mode is recommended to the user next time, the second parameter set is used. In this manner, the terminal device may recommend an optimal photographing mode to the user based on a usage habit of the user, so that an image that relatively meets a requirement of the user can be photographed, thereby enhancing user experience.
  • the terminal device displays first information in the preview interface in response to the second operation on the first icon, where the first information is text information describing the first photographing mode.
  • the user may trigger, for example, double-tap, the icon of the photographing mode displayed in the preview interface, and the terminal device may display the detailed description of the recommended photographing mode, to help the user know the recommended photographing mode and enhance user experience.
  • the terminal device displays second prompt information in the preview interface, where the second prompt information is used to remind the user that the terminal device is currently in the first photographing mode.
  • the terminal device may display the second prompt information, to remind the user that the terminal device has entered the photographing mode. This manner can help the user determine whether the recommended photographing mode is successfully enabled, thereby enhancing user experience.
  • an embodiment of the present invention provides a terminal device, wherein the terminal device comprises a processor, a lens, a display screen, and a memory, wherein the processor is configured to start the lens; the display screen is configured to display a preview interface; and the processor invokes an instruction in the memory, to perform the method according to the first aspect.
  • an embodiment of the present invention further provides a computer readable storage medium, where the computer readable storage medium includes a computer program, and when the computer program is run on a terminal device, the terminal device is enabled to perform the method according to any one of the first aspect and the possible designs of the first aspect.
  • FIG. 2 is a flowchart of an intelligent assistant control method according to an embodiment of this disclosure. As shown in FIG. 2 , the method includes the following steps.
  • the terminal device 100 starts a lens 170, and displays a preview interface on a display unit 160.
  • the preview interface includes a to-be-photographed object.
  • a photographing apparatus may be an application (APP) with an image photographing function in the terminal device 100, for example, a camera included in the terminal device 100.
  • the photographing apparatus may be a third-party application installed in the terminal device 100, for example, an application with an image photographing function, such as Sweet Camera, Photo Editor & Filters, Beauty Camera, WeChat, Alipay, Tencent chat software (QQ), DingTalk, photo sharing (instagram), Kakao Talk, or Line (Line).
  • the camera included in the terminal device 100 is used as an example.
  • a camera icon displayed on the terminal device 100 may be triggered to start the camera.
  • a processor 130 of the terminal device 100 runs the camera, starts the lens 170, and displays the preview interface on a display panel 161 of the display unit 160.
  • the to-be-photographed object is displayed in the preview interface.
  • the terminal device 100 identifies a scene of the to-be-photographed object, and determines a proper photographing mode based on identified scene information.
  • the scene information includes but is not limited to a character image scene, a landscape scene, a night scene, a green plant scene, a snow scene, and the like.
  • the terminal device 100 may identify the to-be-photographed object, and may further identify the scene in which the to-be-photographed object is located.
  • the to-be-photographed object identified by the terminal device 100 includes food, a flower, a pet, and a transportation tool.
  • the scene in which the to-be-photographed object is located and that is identified by the terminal device may include a mountain or a lake or a common urban facility: a cinema, a library, a hospital, a stadium, a station, or the like.
  • the terminal device 100 may identify the scene of the to-be-photographed object by collecting the preview interface.
  • the terminal device 100 may identify the scene information of the to-be-photographed object by identifying feature information in the collected preview interface.
  • the feature information may be the to-be-photographed object, namely, a character, an animal, or a plant, included in the preview interface, or the feature information may be the scene in which the to-be-photographed object is located, for example, a building or natural scenery.
  • the terminal device 100 determines that the scene information of the to-be-photographed object includes the character image scene.
  • the terminal device 100 determines that the scene information of the to-be-photographed object includes the landscape scene.
  • the terminal device 100 may refine different scenes.
  • the character image scene may fall into a common character image scene and a group image scene.
  • the landscape scene may fall into a plant scene, a natural scenery scene, a sunrise/sunset scene, and the like.
  • Each refined scene has a corresponding determining condition. Therefore, after identifying the feature information in the preview interface, the terminal device 100 may further determine whether the feature information meets a preset determining condition. If the feature information meets the preset determining condition, the terminal device 100 determines a scene corresponding to the determining condition.
  • the feature information in the preview interface includes a character.
  • the terminal device 100 may further determine whether a quantity of characters included in the preview interface is less than a preset quantity, or whether a proportion of an area occupied by characters in the preview interface to a total area of the preview interface is greater than a preset proportion.
  • the terminal device 100 determines that the quantity of characters is less than or equal to the preset quantity such as 2, and the proportion of the area occupied by the characters in the preview interface to the total area of the preview interface is less than the preset proportion such as 10%
  • the terminal device 100 determines that the scene information of the to-be-photographed object includes a common character image scene.
  • the terminal device 100 determines that the quantity of characters in the preview interface is greater than 2
  • the terminal device 100 determines that the scene information of the to-be-photographed object includes a group image scene.
  • the feature information in the preview interface includes a flower or a green plant.
  • the terminal device 100 may further determine a saturation value of the flower or the green plant in the preview interface. When the saturation value is greater than a preset value, the terminal device 100 determines that the scene information of the to-be-photographed object is a high-saturation plant scene.
  • the terminal device 100 may further identify the scene information of the to-be-photographed object by determining a distance between the to-be-photographed object and the terminal device 100. When the distance between the to-be-photographed object and the terminal device 100 is less than or equal to a preset distance, the terminal device 100 determines that the scene information of the preview interface is a close shot scene.
  • the terminal device 100 may identify the scene information of the to-be-photographed object in another manner. For example, the foregoing two manners are combined.
  • the terminal device 100 may obtain, based on a mapping relationship between scene information and a photographing mode, the photographing mode matching the identified scene information.
  • Table 1 lists a possible mapping relationship between scene information and a photographing mode.
  • Table 1 Scene information Photographing mode Common character image scene Character image mode Group image scene Group image mode Landscape scene Landscape mode Close shot scene Macro mode Flower/Green plant scene Vivid mode
  • the terminal device 100 may determine, according to Table 1, the photographing mode corresponding to the identified scene information.
  • the terminal device 100 determines, according to Table 1, a photographing mode corresponding to the scene information.
  • the terminal device 100 determines a plurality of photographing modes according to Table 1.
  • the terminal device 100 may select a better photographing mode from the plurality of photographing modes based on a preset priority policy.
  • the terminal device 100 may count a quantity of times or a frequency of using various photographing modes by the user, set a photographing mode frequently used by the user to be relatively high, and set a photographing mode infrequently used by the user to be relatively low.
  • the preset priority policy includes three priority policies, and there is a one-to-one correspondence between these three priority policies and identified scene types. For details, refer to Table 2.
  • Table 2 Scene type Priority policy Close shot type Micro mode, vivid mode Object type Character > Animal > Text > Food > Flower > Green plant > Autumn Leaf Scene type Stage > Beach > Waterfall > Super night scene > Sunrise/Sunset > Blue sky > Snow scene
  • the terminal device 100 may determine a proper photographing mode based on different environment conditions. For example, if light intensity of an environment is greater than preset intensity, the vivid mode is selected; or if light intensity is less than preset intensity, the macro mode is selected.
  • the terminal device 100 may determine a type of each scene. Specifically, the terminal device 100 may determine the type of the scene based on a type of the feature information in the preview interface, or may determine the type of the scene based on the distance between the terminal device 100 and the to-be-photographed object.
  • the terminal device 100 determines that the preview interface includes three types of feature information, namely, a character, a dog, and a snow scene.
  • the terminal device 100 determines three photographing modes, namely, the character image mode, an animal mode, and the landscape mode, according to Table 1.
  • the terminal device 100 determines a type of each type of feature information.
  • the character and the dog belong to the object type, and the snow scene belongs to the scene type.
  • the terminal device 100 determines that a priority of the object type is higher than a priority of the scene type, and determines, according to Table 2, that a priority of the character in the object type is higher than a priority of the animal. Therefore, the terminal device 100 may determine that a better photographing mode in the three photographing modes is the character image mode.
  • the photographing mode may be understood as a set of parameters that are set when the terminal device 100 photographs the to-be-photographed object.
  • the set may include an aperture, a shutter, ISO, white balance, and the like.
  • a proper value is set for each parameter in a parameter set.
  • a manufacturer of the terminal device 100 stores a set of preset parameters and a value of each parameter in the terminal device 100 based on some typical scenes such a night scene, a character image, a snow scene, and a beach. For example, the manufacturer may obtain the set of preset parameters and the value of each parameter through an experiment, or may obtain the set of preset parameters and the value of each parameter in another manner.
  • the user may also manually adjust a value of a parameter in a photographing mode.
  • a default value of an aperture is a first value.
  • the user may also adjust the aperture, and a value of the adjusted aperture is a second value.
  • the value of the aperture in the character image mode may be set to the second value.
  • the terminal device 100 determines whether the to-be-photographed object has been continuously and stably displayed in the preview interface for preset duration. If yes, S204 is performed; or if no, S202 is performed.
  • the to-be-photographed object in the preview interface changes dynamically. For example, after starting the photographing apparatus in the terminal device 100, the user usually searches for an optimal photographing angle, photographing range, and the like. In the search process, the to-be-photographed object in the preview interface changes in real time.
  • the terminal device 100 considers that the user is currently searching for a photographing angle, and the terminal device 100 may perform S202.
  • the terminal device 100 may identify the scene of the to-be-photographed object in real time, and determine the proper photographing mode based on the identified scene information. Specifically, the terminal device 100 can collect the preview interface in real time, and then identify the scene information of the to-be-photographed object by identifying the feature information in the collected preview interface (refer to the description of S202).
  • FIG. 3 is a schematic diagram of periodically collecting a preview interface by a terminal device.
  • a cycle of collecting the preview interface by the terminal device 100 is T.
  • the terminal device 100 collects an n th frame of preview interface at a moment t1, then identifies scene information of the to-be-photographed object by identifying feature information in the n th frame of preview interface, and determines that a photographing mode corresponding to the identified scene information is a first photographing mode.
  • the terminal device 100 collects an (n+m) th frame of preview interface at a moment t1+T, then identifies scene information of the to-be-photographed object by identifying feature information in the (n+m) th frame of preview interface, and determines that a photographing mode corresponding to the identified scene information is a second photographing mode.
  • the terminal device 100 collects an (n+2m) th frame of preview interface at a moment tl+2T, then identifies scene information of the to-be-photographed object by identifying feature information in the (n+2m) th frame of preview interface, and determines that a photographing mode corresponding to the identified scene information is a third photographing mode.
  • the preset duration is greater than T.
  • the terminal device 100 determines a similarity between the n th frame of preview interface, an (n+1) th frame of preview interface, and an (n+2) th frame of preview interface through comparison. If the similarity is higher than a preset similarity, it is considered that the current preview interface is stably displayed. In this case, the terminal device 100 outputs first prompt information. Because the preview interface has been stably displayed for the preset duration, there is a relatively large possibility that the first photographing mode, the second photographing mode, and the third photographing mode are the same. Therefore, the first prompt information output by the terminal device 100 may be used to recommend any one of the three photographing modes.
  • the terminal device 100 may recommend the finally determined photographing mode, namely, the third photographing mode.
  • the preset duration is greater than the cycle of collecting the preview interface, only for ease of understanding by the reader.
  • a person skilled in the art may set, based on an actual requirement, a relationship between the preset duration and the cycle of collecting the preview interface.
  • the terminal device 100 detects, by using a motion sensor in the terminal device 100, that the terminal device 100 is in a still state within the preset duration. In this case, the terminal device 100 determines that the preview interface is stably displayed. The terminal device 100 may output first prompt information. Likewise, because the preview interface has been stably displayed for the preset duration, there is a relatively large possibility that the first photographing mode, the second photographing mode, and the third photographing mode are the same. Therefore, the first prompt information output by the terminal device 100 may be used to recommend any one of the three photographing modes. Certainly, preferably, the terminal device 100 may recommend the finally determined photographing mode, namely, the third photographing mode.
  • the motion sensor in the terminal device 100 may be a speed sensor, an acceleration sensor, an angular velocity sensor, a gyroscope, or the like.
  • the terminal device 100 After determining that the preview interface has been stably displayed for the preset duration, the terminal device 100 considers that the user completes the process of searching for a photographing angle, in other words, the to-be-photographed object displayed in the current preview interface is a scene that the user expects to photograph.
  • the terminal device 100 may output first prompt information.
  • the first prompt information is used to recommend a better photographing mode obtained by the terminal device 100 through matching to the user.
  • a new object may enter a photographing range.
  • a new object may occur in the preview interface (the new object enters a photographing range), in other words, the to-be-photographed object in the preview interface changes. Therefore, after continuously and stably displaying the preview interface, the terminal device 100 may also identify scene information of the to-be-photographed object in real time, and recommend a proper photographing mode to the user.
  • FIG. 4A and FIG. 4B are schematic diagrams in which a new object enters a photographing range after a preview interface of a terminal device is stably output.
  • the terminal device 100 determines that the preview interface has been stably displayed for the preset duration at a moment t2, the terminal device 100 detects that the terminal device 100 recommends a first photographing mode to the user at the moment t2. If a new object enters the photographing range at the moment t2 (or another moment, where in FIG. 4A , a moment between t2 and t3 is used as an example), in other words, the to-be-photographed object in the preview interface changes, the terminal device 100 may perform scene identification on the new to-be-photographed object. If a priority of a second photographing mode corresponding to the new object is lower than that of the first photographing mode, the terminal device 100 continues to recommend the first photographing mode to the user, for example, at the moment t3 in FIG. 4A .
  • the preview interface displayed on the terminal device 100 includes a character and a snow scene, and the preview interface has been stably displayed for the preset duration.
  • the terminal device 100 recommends the character image mode to the user according to the priority policy. If the user does not trigger a photographing control, and a dog enters the photographing range, in other words, the companion of the user, the dog, and the snow scene are displayed in the preview interface, when detecting that the to-be-photographed object changes, the terminal device 100 may continue to identify a scene of the to-be-photographed object. After the identification, new scene information includes a character image scene, an animal scene, and a snow scene.
  • the terminal device 100 determines, according to the priority policy, that a priority of the character image mode is higher than that of the animal mode and that of a snow scene mode. Therefore, at the moment t3, the terminal device 100 continues to recommend the character image mode to the user. In this manner, because the terminal device 100 has determined that the priority of the animal mode is lower than that of the character image mode, the terminal device 100 does not need to detect whether the dog has been stably displayed in the preview interface for the preset duration, so that computing resources are reduced.
  • the terminal device 100 determines whether the new object has been stably displayed in the preview interface for the preset duration. If yes, the terminal device 100 recommends the second photographing mode corresponding to the new object to the user, for example, at the moment t4 in FIG. 4A .
  • the terminal device 100 identifies that the scene information includes the animal scene and the snow scene. It can be learned from the priority policy that the priority of the animal mode is higher than that of the snow scene mode. Therefore, if a preview interface including the dog and the snow scene has been stably displayed for the preset duration, the terminal device 100 recommends the animal mode to the user.
  • the terminal device 100 determines that the preview interface has been stably displayed for the preset duration at a moment t2, the terminal device 100 recommends a first photographing mode to the user. If a new object enters the photographing range at the moment t2 (or another moment, where in FIG. 4A , a moment between t2 and t3 is used as an example), in other words, the to-be-photographed object in the preview interface changes, the terminal device 100 may perform scene identification on the new to-be-photographed object.
  • the terminal device 100 If a priority of a second photographing mode corresponding to the new object is higher than that of the first photographing mode, but the new object has been in the preview interface for a relatively short time, in other words, the new object has not been stably displayed in the preview interface for the preset duration, the terminal device 100 still recommends the first photographing mode to the user, for example, at the moment t3 in FIG. 4B . If a priority of the second photographing mode corresponding to the new object is higher than that of the first photographing mode, and the new object has been stably displayed in the preview interface for the preset duration, the terminal device 100 recommends the second photographing mode to the user at a moment t4, for example, at the moment t4 in FIG. 4B .
  • the terminal device 100 recommends the first photographing mode, namely, the animal mode, to the user according to the priority policy. If at a moment during t2 and t3, a character enters the photographing range, in other words, the character occurs in the preview interface, because the to-be-photographed object changes, the terminal device 100 continues to identify a new to-be-photographed object. After the identification, the terminal device 100 determines that the new to-be-photographed object includes a human image scene, an animal scene, and a snow scene.
  • the terminal device 100 determines, according to the priority policy, that the character image mode has the highest priority, and the animal scene has the second highest priority. Therefore, if the character has been in the preview interface for a relatively short time that is less than the preset duration, the terminal device 100 continues to recommend the animal mode to the user. If the character has been stably displayed in the preview interface for the preset duration, the terminal device 100 recommends the character image mode to the user. Afterward, if the character exits the photographing range, in other words, only the dog and the snow scene are left in the preview interface, the terminal device 100 determines whether the dog and the snow scene have been stably displayed in the preview interface for the preset duration. If yes, the terminal device 100 continues to recommend the animal mode to the user according to the priority policy.
  • the terminal device recommends a proper photographing mode to the user after determining that the preview interface has been stably displayed for the preset duration.
  • the terminal device may further perform scene recognition on a new to-be-photographed object in real time. If the new to-be-photographed object includes a plurality of scenes, the terminal device may re-determine an optimal photographing mode according to the priority policy. If the new to-be-photographed object has been stably displayed in the preview interface for the preset duration, the terminal device recommends the determined photographing mode to the user.
  • a preview interface changes in real time in a process in which the user searches for a photographing angle.
  • first prompt information is constantly displayed in the preview interface, and the first prompt information usually shields a part of the preview interface. Therefore, the user is prevented from better searching for a photographing angle, and user experience is reduced.
  • it can be learned from the foregoing content that when preview interfaces are different, determined photographing modes may be different, in other words, first prompt information may be different. If the first prompt information is constantly displayed in the preview interface, a scenario in which the first prompt information is switched in the preview interface in real time is presented to the user, easily arousing disgust in the user.
  • the terminal device 100 displays the first prompt information only after determining that the preview interface has been stably displayed for the preset duration.
  • the terminal device 100 recommends an optimal photographing mode to the user, so that no prompt information occurs frequently and interferes with a normal photographing activity of the user, and prompt information matching a mode can also be provided in a timely manner.
  • the user can perform optimized photographing based on the prompt information to obtain a better photographing result. Therefore, in this solution, intelligence of human-machine interaction can be improved, and quality of an output image can be improved.
  • the terminal device 100 outputs first prompt information, where the first prompt information is used to notify the user of a matched photographing mode.
  • the terminal device 100 after determining an optimal photographing mode, the terminal device 100 outputs the first prompt information.
  • the first prompt information is used to recommend the matched photographing mode to the user.
  • the terminal device 100 may display the first prompt information in the preview interface.
  • the first prompt information may be in a text form, an image form, an icon form, or the like.
  • the first prompt information may be audio information.
  • the terminal device 100 determines whether the user selects the recommended photographing mode. If no, S206 is performed; or if yes, the current recommendation process of the photographing mode is completed.
  • the user may select the photographing mode recommended by using the first prompt information, or may select another photographing mode.
  • the terminal device 100 records another photographing mode selected by the user, and establishes a mapping relationship between the another photographing mode and the identified scene information when a quantity of times the user selects the another photographing mode exceeds a preset quantity of times.
  • the terminal device 100 may identify one or more types of scene information of the to-be-photographed object. When there is only one type of scene information, the terminal device 100 determines, according to Table 1, a photographing mode corresponding to the scene information, and then recommends the determined photographing mode to the user. If the user does not use the recommended photographing mode, but selects another photographing mode, in this scene, when a quantity of times the user selects the another photographing mode exceeds a preset quantity of times, the terminal device 100 establishes a mapping relationship between the scene information and the another photographing mode, and updates Table 1 based on the mapping relationship. When it is identified next time that the to-be-photographed object includes the scene, the terminal device 100 recommends the another photographing mode to the user.
  • the terminal device selects a better photographing mode from the plurality of photographing modes according to the priority policy, and recommends the selected photographing mode to the user. If the user does not use the recommended photographing mode, but selects another photographing mode, the terminal device 100 may keep a record. For example, the terminal device 100 may count a quantity of times the user selects the another photographing mode. After the counted quantity exceeds a preset quantity of times, the terminal device 100 considers that the user prefers the another photographing mode in this scene. In this case, the terminal device 100 increases a priority of the another photographing mode, and updates Table 2.
  • the terminal device 100 recommends the character image mode to the user according to the priority policy. If the user selects the snow scene mode, the terminal device 100 may record that the user selects the snow scene mode once when the to-be-photographed object includes a snow scene. A higher quantity of times the user selects the snow scene mode indicates that the snow scene mode has a higher priority.
  • the terminal device 100 may update Table 2, to be specific, increase the priority of the snow scene mode. For example, the priority of the snow scene mode is higher than that of the blue sky mode, and even a priority of the scene type is higher than that of the animal mode.
  • the terminal device 100 recommends the snow scene mode to the user according to the new priority policy.
  • the terminal device 100 may record a usage habit of the user, and recommend a photographing mode that meets the usage habit of the user to the user, so that user experience is enhanced.
  • the terminal device 100 identifies the scene information of the to-be-photographed object in real time, and determines the photographing mode matching the identified scene information. After detecting that the preview interface has been stably displayed for the preset duration, the terminal device 100 recommends the matched photographing mode to the user.
  • the terminal device 100 may identify the scene information of the to-be-photographed object only after determining that the preview interface has been stably displayed for the preset duration, and determine the photographing mode matching the identified scene information. Then the terminal device 100 recommends the determined photographing mode to the user.
  • FIG. 5 is a flowchart of another intelligent assistant control method according to an embodiment of this disclosure.
  • a terminal device 100 starts a lens 170, and displays a preview interface on a display unit 160.
  • the preview interface includes a to-be-photographed object.
  • the terminal device 100 determines that the to-be-photographed object in the preview interface has been continuously and stably displayed for preset duration.
  • the terminal device 100 may collect three frames of preview interfaces within the preset duration. If a similarity between the three frames of preview interfaces is higher than a preset similarity, it is considered that the preview interface is stably displayed.
  • the terminal device 100 may select one frame of preview interface from the three frames of preview interfaces, for example, select a collected last frame of preview interface, namely, an (n+2) th frame of preview interface. Then the terminal device 100 identifies scene information of the to-be-photographed object by identifying feature information in the selected (n+2) th frame of preview interface.
  • the terminal device 100 detects, by using a motion sensor in the terminal device 100, that the terminal device 100 is in a still state within the preset duration. In this case, the terminal device 100 considers that the preview interface is stably displayed.
  • the terminal device 100 may collect one or more frames of preview interfaces at a moment t2. Then the terminal device 100 identifies the scene information of the to-be-photographed object by using feature information in the collected preview interface. Specifically, if the terminal device 100 collects a frame of preview interface at the moment t2, the terminal device 100 identifies the scene information of the to-be-photographed object by using feature information in the preview interface.
  • the terminal device 100 may select a frame of preview interface with a better effect from the plurality of preview interfaces, for example, may select a frame of preview interface with relatively high definition from the plurality of preview interfaces. Then the terminal device 100 identifies the scene information of the to-be-photographed object by using feature information in the selected preview interface.
  • the terminal device 100 does not need to collect the preview interface in real time.
  • the terminal device 100 collects the preview interface only after determining that the preview interface is stably displayed.
  • the terminal device 100 identifies the scene information of the to-be-photographed object by using the feature information in the collected preview interface, and further determines a proper photographing mode based on the scene information. In this manner, memory resources are reduced, computing resources and a computing time are reduced, and efficiency is improved.
  • the terminal device 100 identifies a scene of the to-be-photographed object, and determines a proper photographing mode based on the identified scene information.
  • the scene information includes but is not limited to a character image scene, a landscape scene, a night scene, a green plant scene, a snow scene, and the like.
  • the terminal device 100 after determining that the preview interface has been continuously and stably displayed for the preset duration, the terminal device 100 considers that a user completes a process of searching for a photographing angle, in other words, the to-be-photographed object displayed in the current preview interface is a scene that the user expects to photograph. Therefore, the terminal device 100 identifies the scene information of the to-be-photographed object, and determines the proper photographing mode based on the identified scene information. For details, refer to the description of S202 in the embodiment shown in FIG. 2 .
  • the terminal device 100 does not need to perform scene identification on the to-be-photographed object in real time. Only after determining that the preview interface is stable, the terminal device 100 performs processes of performing scene identification and determining a photographing mode, so that computing resources and a computing time are reduced, and computing efficiency is improved.
  • a manner in which the terminal device 100 determines the proper photographing mode based on the identified scene information is similar to that in S202 in the embodiment shown in FIG. 2 . For brevity of this specification, details are not described herein again.
  • the terminal device 100 outputs first prompt information, where the first prompt information is used to recommend a matched photographing mode to the user.
  • the terminal device 100 determines whether the user selects the recommended photographing mode. If no, S506 is performed; or if yes, the current recommendation process of the photographing mode is completed.
  • the terminal device 100 records another photographing mode selected by the user, and establishes a mapping relationship between the another photographing mode and the identified scene information when a quantity of times the user selects the another photographing mode exceeds a preset quantity of times.
  • FIG. 7A to FIG. 7E are schematic diagrams of a display status of a display screen when a terminal device photographs an image.
  • the terminal device is a mobile phone.
  • a user starts a camera in the mobile phone 700 and starts a lens.
  • the user aligns the started lens with Alice for photographing.
  • An image of Alice is displayed in a preview interface 701 of the camera.
  • the mobile phone 700 determines that scene information is a character image scene, and determines that a photographing mode corresponding to the character image scene is a character image mode.
  • For outputting first prompt information by the mobile phone 700 refer to FIG. 7B .
  • the mobile phone 700 displays the first prompt information in the preview interface 701, and the first prompt information is displayed in a form of an icon 702.
  • the icon 702 may be an image corresponding to the recommended photographing mode. Because the recommended photographing mode is the character image mode, the displayed icon 702 is an icon of the character image mode.
  • the first prompt information may be displayed in another form such as a text or an image.
  • a display location of the first prompt information in the preview interface 701 is not specifically limited in this embodiment of this disclosure.
  • the first prompt information may be displayed near a photographing control 703, so that the user can easily observe the first prompt information, and the user can perform an operation on the first prompt information.
  • the mobile phone 700 displays the icon 702 in the preview interface 701
  • the user may trigger the icon 702 by performing a specific trigger operation, to open detailed information of the recommended photographing mode. Refer to FIG. 7C .
  • the mobile phone 700 opens an introduction card 704 of the character image photographing mode, so that the user knows details of the photographing mode recommended by the mobile phone 700. This can help the user determine whether to use the recommended photographing mode.
  • the user may close the introduction card 704 by using a close control in the introduction card, or the user may close the introduction card 704 by touching any location in the preview interface.
  • the mobile phone 700 may still display the icon 702. If the user wants to use the recommended photographing mode, the user may trigger the icon 702 by performing another trigger operation. For example, the user may double-tap the icon 702, and then the mobile phone 700 enables the character image mode.
  • the mobile phone 700 may highlight the icon 702 of the character image mode in the preview interface. The user may determine, by viewing whether the icon 702 is highlighted, whether the character image mode is successfully entered.
  • the mobile phone 700 may hide the icon 702 in the preview interface 701, and display the highlighted icon 702 of the character image mode in an icon display bar 705.
  • the user may select a control by triggering a specific photographing mode, or a finger of the user or a stylus slides to the right at any location in the preview interface 701, and the mobile phone 700 displays a photographing mode selection interface 706 in the preview interface 701. Refer to FIG. 7E .
  • the photographing mode selection interface 706 is displayed in the preview interface 701 of the mobile phone 700.
  • the user may select a desired photographing mode from the photographing mode selection interface 706.
  • the photographing mode may be understood as a set of parameters that are set when the mobile phone photographs an image.
  • the mobile phone 700 may further display parameters related to the photographing mode. As shown in FIG. 7F , the mobile phone 700 displays a display window 707, and the display window 707 displays parameters corresponding to a photographing mode currently used by the mobile phone 700. In FIG. 7F , an aperture and brightness are used as examples.
  • the display window 707 further includes an add control 708.
  • the user may trigger the add control 708 to display icons of all parameters related to photographing in the mobile phone 700, and the user may select icons of several parameters from the icons of all the parameters and add them to the display window 707.
  • the mobile phone 700 loads the parameter. In other words, the mobile phone 700 considers that the user adjusts a set of parameters in the photographing mode, for example, adds or removes a parameter.
  • the user may further tap an icon of a parameter in the display window 707, and then the mobile phone 700 displays an input box, where a value of the parameter is displayed in the input box, and the user may modify the value, for example, increase or decrease the value.
  • the mobile phone 700 saves the value obtained after the modification of the user, and loads the parameter by using the modified value.
  • the mobile phone 700 detects that a quantity of times the user modifies a value of a parameter in a same photographing mode is greater than a preset quantity of times, the mobile phone 700 saves the parameter value obtained after the modification of the user. When this photographing mode is used next time, the parameter is automatically loaded by using the modified value.
  • the terminal device may include a hardware structure and/or a software module, and implement the functions in a form of the hardware structure, the software module, or a combination of the hardware structure and the software module. Whether a specific function in the functions is performed by using the hardware structure, the software module, or the combination of the hardware structure and the software module depends on a specific application and a design constraint of the technical solution.
  • FIG. 8 shows a terminal device 800 according to this disclosure.
  • the terminal device 800 may be a terminal device having an image photographing capability.
  • the terminal device 800 includes a processing unit 801 and a display unit 802.
  • the processing unit 801 is configured to start a lens in the terminal device 800.
  • the display unit 802 is configured to display a preview interface, where the preview interface includes a to-be-photographed object.
  • the display unit is further configured to display first prompt information of a photographing mode if the processing unit 801 determines that the preview interface has been stably displayed for preset duration, where the first prompt information is used to recommend a first photographing mode to a user, and the first photographing mode is determined by the processing unit 801 based on scene information of the to-be-photographed object.
  • the processing unit 801 is further configured to: identify at least one type of scene information included in the to-be-photographed object; when the to-be-photographed object includes one type of scene information, determine, based on the scene information and a first mapping relationship between scene information and a photographing mode, the first photographing mode corresponding to the scene information; or when the to-be-photographed object includes at least two types of scene information, determine, based on the scene information and a first mapping relationship between scene information and a photographing mode, a photographing mode corresponding to each of the at least two types of scene information, to obtain at least two photographing modes; and select the first photographing mode from the at least two photographing modes according to a priority policy, where the priority policy includes priorities of the at least two photographing modes.
  • the processing unit 801 is further configured to: collect n frames of preview interfaces, where n is an integer greater than or equal to 1; identify at least one type of scene information of the to-be-photographed object included in each of the n frames of preview interfaces; and determine, based on the at least one type of scene information, a photographing mode matching each frame of preview interface.
  • the display unit 802 is specifically configured to display the first prompt information in the preview interface, where the first prompt information is used to recommend the first photographing mode to the user, and the first photographing mode is a photographing mode matching the last frame of preview interface in the n frames of preview interfaces.
  • the processing unit 801 when the processing unit 801 is configured to determine whether the preview interface has been stably displayed for the preset duration, the processing unit 801 is specifically configured to: collect at least two frames of preview interfaces within the preset duration; if a similarity between the at least two frames of preview interfaces is higher than a preset similarity, determine that the preview interface has been stably displayed for the preset duration; or detect, by using a motion sensor, whether the processing unit 801 is in a still state within the preset duration; and if the terminal device is in a still state within the preset duration, determine that the preview interface has been stably displayed for the preset duration.
  • the first prompt information includes a first icon, the first icon is used to identify the first photographing mode, and the display unit is specifically configured to display the first icon in a first display region in the preview interface.
  • the processing unit 801 loads, in response to a first operation on the first icon, a parameter corresponding to the first photographing mode.
  • the display unit 802 is further configured to display the first parameter set in a second display region in the preview interface.
  • the display unit 802 is further configured to display a modified second parameter set in response to a second operation, where the second parameter set is a parameter set obtained after a first parameter in the first parameter set is modified.
  • the processing unit 801 is further configured to load the second parameter set.
  • the processing unit 801 is further configured to establish a third mapping relationship between the second parameter set and the first photographing mode.
  • the processing unit 801 is further configured to add the third mapping relationship to the second mapping relationship.
  • the display unit 802 is further configured to display first information in the preview interface after the processing unit 801 responds to the second operation on the first icon, where the first information is text information describing the first photographing mode.
  • the display unit 802 is further configured to display second prompt information in the preview interface, where the second prompt information is used to remind the user that the terminal device is currently in the first photographing mode.
  • the processing unit 801 is further configured to: determine whether a quantity of times the user selects the another photographing mode exceeds a preset quantity of times; and when the quantity of times the user selects the another photographing mode exceeds the preset quantity of times, if there is only one type of scene information of the to-be-photographed scene, update the first mapping relationship by using a second mapping relationship between the another photographing mode and the scene information; or if there are at least two types of scene information of the to-be-photographed scene, set a first priority of the another photographing mode to a second priority, where the second priority is greater than the first priority.
  • the terminal device updates the priority policy by using the second priority of the another photographing mode.
  • the terminal device 800 is further implemented by using a structure shown in FIG. 9 .
  • the terminal device 900 includes a memory 901, a processor 902, and a bus 903.
  • the memory 901 and the processor 902 are connected by using the bus 903.
  • the memory 901 is configured to store a computer-executable instruction.
  • the processor 902 executes the computer-executable instruction stored in the memory 901, so that the terminal device 900 implements the intelligent assistant control method provided in the embodiment shown in FIG. 2 or FIG. 5 .
  • the intelligent assistant control method refer to the foregoing descriptions and related descriptions in the accompanying drawings. For brevity of this specification, details are not described herein again.
  • the embodiments of this disclosure may be implemented through hardware, firmware, or a combination thereof.
  • the foregoing functions may be stored in a computer-readable medium or transmitted as one or more instructions or code in a computer-readable medium.
  • the computer-readable medium includes a computer storage medium and a communications medium.
  • the communications medium includes any medium that enables a computer program to be transmitted from one place to another place.
  • the storage medium may be any available medium accessible to a computer.
  • the computer-readable medium may include a RAM, a ROM, an electrically erasable programmable read-only memory (electrically erasable programmable read-only memory, EEPROM), a compact disc read-only memory (compact disc read-Only memory, CD-ROM) or another compact disc storage, a magnetic disk storage medium or another magnetic storage device, or any other medium that can be configured to carry or store desired program code in a form of an instruction or a data structure and that can be accessed by a computer.
  • any connection may be appropriately defined as a computer-readable medium.
  • the coaxial cable, the optical fiber cable, the twisted pair, the DSL, or the wireless technologies such as the infrared technology, the radio technology, and the microwave technology are included in fixation of a medium to which they belong.
  • a disk (disk) and disc (disc) used in the embodiments of this disclosure include a compact disc (compact disc, CD), a laser disc, an optical disc, a digital video disc (digital video disc, DVD), a floppy disk, and a Blu-ray disc.
  • the disk generally reproduces data magnetically, and the disc reproduces data optically by using laser.
  • the foregoing combination should also fall within the protection scope of the computer-readable medium.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Human Computer Interaction (AREA)
  • Software Systems (AREA)
  • User Interface Of Digital Computer (AREA)
  • Studio Devices (AREA)
  • Telephone Function (AREA)
  • Image Analysis (AREA)

Claims (10)

  1. Procédé de commande d'un assistant intelligent, où le procédé comprend les étapes suivantes :
    mettre en marche (S501), par un dispositif terminal (100, 700), un objectif (170) ; afficher (S501), par le dispositif terminal (100, 700), une interface de prévisualisation (701) sur un écran d'affichage, où l'interface de prévisualisation (701) comprend un objet à photographier ;
    identifier, par le dispositif terminal (100, 700), au moins deux types d'informations de scène comprises dans l'objet à photographier ;
    déterminer un premier mode de prise de vue en fonction d'une politique de priorité et d'une première relation de mise en correspondance entre les informations de scène et un mode de prise de vue, qui correspond à chacun des au moins deux types d'informations de scène, où la politique de priorité comprend les priorités des au moins deux modes de prise de vue ; et
    afficher (S504), par le dispositif terminal (100, 700), des premières informations d'invite du premier mode de prise de vue dans l'interface de prévisualisation (701) si le dispositif terminal (100, 700) détermine (S502) que l'objet à photographier dans l'interface de prévisualisation (701) a été affiché de manière stable pendant une durée prédéfinie, où les premières informations d'invite sont utilisées pour recommander le premier mode de prise de vue à un utilisateur ;
    où, si l'utilisateur sélectionne un autre mode de prise de vue que le premier mode de prise de vue recommandé à l'aide des premières informations d'invite, le procédé comprend en outre les étapes suivantes :
    déterminer, par le dispositif terminal (100, 700), si le nombre de fois où l'utilisateur sélectionne l'autre mode de prise de vue dépasse un nombre de fois prédéfini ; et,
    lorsque le nombre de fois où l'utilisateur sélectionne l'autre mode de prise de vue dépasse le nombre de fois prédéfini,
    définir, par le dispositif terminal (100, 700), une première priorité de l'autre mode de prise de vue à une deuxième priorité, où la deuxième priorité est supérieure à la première priorité ; et mettre à jour, par le dispositif terminal (100, 700), la politique de priorité en utilisant la deuxième priorité de l'autre mode de prise de vue.
  2. Procédé selon la revendication 1, dans lequel, avant que l'interface de prévisualisation (701) n'ait été affichée de manière stable pendant la durée prédéfinie, le procédé comprend en outre les étapes suivantes :
    collecter, par le dispositif terminal (100, 700), n trames d'interfaces de prévisualisation (701), où n est un nombre entier supérieur ou égal à 1 ;
    identifier, par le dispositif terminal (100, 700), au moins un type d'informations de scène de l'objet à photographier compris dans chacune des n trames d'interfaces de prévisualisation (701) ; et
    déterminer, par le dispositif terminal (100, 700), sur la base de l'au moins un type d'informations de scène, un mode de prise de vue correspondant à chaque trame d'interface de prévisualisation (701) ; et
    l'émission, par le dispositif terminal (100, 700), de premières informations d'invite si l'objet à photographier dans l'interface de prévisualisation (701) a été affiché de manière stable pendant une durée prédéfinie comprend l'étape suivante :
    si l'interface de prévisualisation (701) a été affichée de manière stable pendant la durée prédéfinie, afficher, par le dispositif terminal (100, 700), les premières informations d'invite dans l'interface de prévisualisation (701), les premières informations d'invite étant utilisées pour recommander le premier mode de prise de vue à l'utilisateur, et le premier mode de prise de vue étant un mode de prise de vue correspondant à la dernière trame de l'interface de prévisualisation (701) dans les n trames des interfaces de prévisualisation (701).
  3. Procédé selon la revendication 1 ou la revendication 2, dans lequel la détermination par le dispositif terminal (100, 700) de la durée prédéfinie pendant laquelle l'objet à photographier de l'interface de prévisualisation (701) a été affiché de manière stable comprend les étapes suivantes :
    collecter, par le dispositif terminal (100, 700), au moins deux trames de l'interface de prévisualisation (701) au cours de la durée prédéfinie ;
    si une similarité entre les au moins deux trames de l'interface de prévisualisation (701) est supérieure à une similarité prédéfinie, déterminer que l'objet à photographier dans l'interface de prévisualisation (701) a été affiché de manière stable pendant la durée prédéfinie ; ou
    détecter, par le dispositif terminal (100, 700) à l'aide d'un capteur de mouvement, si le dispositif terminal (100, 700) est dans un état immobile au cours de la durée prédéfinie ; et
    si le dispositif terminal (100, 700) est dans un état immobile au cours de la durée prédéfinie, déterminer que l'objet à photographier dans l'interface de prévisualisation (701) a été affiché de manière stable pendant la durée prédéfinie.
  4. Procédé selon l'une quelconque des revendications 1 à 3, dans lequel les premières informations d'invite comprennent une première icône (702), la première icône (702) est utilisée pour identifier le premier mode de prise de vue, et l'émission, par le dispositif terminal (100, 700), des premières informations d'invite comprend l'étape suivante :
    afficher, par le dispositif terminal (100, 700), la première icône (702) dans une première région d'affichage de l'interface de prévisualisation (701), où le dispositif terminal (100, 700) charge, en réponse à une première opération sur la première icône (702), un premier ensemble de paramètres correspondant au premier mode de prise de vue sur la base d'une deuxième relation de mise en correspondance entre un mode de prise de vue et un ensemble de paramètres.
  5. Procédé selon la revendication 4, dans lequel le procédé comprend en outre les étapes suivantes :
    afficher, par le dispositif terminal (100, 700), le premier ensemble de paramètres dans une deuxième région d'affichage de l'interface de prévisualisation (701) ;
    afficher, par le dispositif terminal (100, 700), un deuxième ensemble de paramètres modifié en réponse à une deuxième opération, où le deuxième ensemble de paramètres est un ensemble de paramètres obtenu après la modification d'un premier paramètre du premier ensemble de paramètres ; et
    charger, par le dispositif terminal (100, 700), le deuxième ensemble de paramètres.
  6. Procédé selon la revendication 5, dans lequel le procédé comprend en outre les étapes suivantes :
    si le premier paramètre est modifié plus d'un nombre prédéfini de fois, établir, par le dispositif terminal (100, 700), une troisième relation de mise en correspondance entre le deuxième ensemble de paramètres et le premier mode de prise de vue ; et
    ajouter, par le dispositif terminal (100, 700), la troisième relation de mise en correspondance à la deuxième relation de mise en correspondance.
  7. Procédé selon l'une quelconque des revendications 4 à 6, dans lequel le procédé comprend en outre l'étape suivante :
    afficher, par le dispositif terminal (100, 700), des premières informations dans l'interface de prévisualisation (701) en réponse à la deuxième opération sur la première icône (702), où les premières informations sont des informations textuelles décrivant le premier mode de prise de vue.
  8. Procédé selon l'une quelconque des revendications 4 à 7, dans lequel le procédé comprend en outre l'étape suivante : afficher, par le dispositif terminal (100, 700), des deuxièmes informations d'invite dans l'interface de prévisualisation (701), où les deuxièmes informations d'invite sont utilisées pour rappeler à l'utilisateur que le dispositif terminal (100, 700) est actuellement dans le premier mode de prise de vue.
  9. Dispositif terminal (100, 700), dans lequel le dispositif terminal (100, 700) comprend un processeur (130, 801, 902), un objectif (170), un écran d'affichage (160) et une mémoire (140, 901), où le processeur (130, 801, 902) est configuré pour mettre en marche l'obj ectif (170) ;
    l'écran d'affichage (160) est configuré pour afficher une interface de prévisualisation (701) ; et
    le processeur (130, 801, 902) invoque une instruction dans la mémoire (140, 901), pour exécuter le procédé selon l'une quelconque des revendications 1 à 8.
  10. Support de stockage lisible par ordinateur, où le support de stockage lisible par ordinateur comprend un programme informatique et, lorsque le programme informatique est exécuté sur un dispositif terminal (100, 700), le dispositif terminal (100, 700) est activé pour exécuter le procédé selon l'une quelconque des revendications 1 à 8.
EP18912032.2A 2018-03-26 2018-03-26 Procédé de commande d'assistant numérique et dispositif terminal Active EP3764631B1 (fr)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/080555 WO2019183775A1 (fr) 2018-03-26 2018-03-26 Procédé de commande d'assistant numérique et dispositif terminal

Publications (3)

Publication Number Publication Date
EP3764631A1 EP3764631A1 (fr) 2021-01-13
EP3764631A4 EP3764631A4 (fr) 2021-03-17
EP3764631B1 true EP3764631B1 (fr) 2023-06-21

Family

ID=68062428

Family Applications (1)

Application Number Title Priority Date Filing Date
EP18912032.2A Active EP3764631B1 (fr) 2018-03-26 2018-03-26 Procédé de commande d'assistant numérique et dispositif terminal

Country Status (7)

Country Link
US (1) US11503208B2 (fr)
EP (1) EP3764631B1 (fr)
JP (1) JP7072667B2 (fr)
KR (1) KR102540450B1 (fr)
CN (1) CN111630836B (fr)
RU (1) RU2748303C1 (fr)
WO (1) WO2019183775A1 (fr)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102598109B1 (ko) * 2018-08-08 2023-11-06 삼성전자주식회사 이미지 분석에 기반하여, 디스플레이를 통해 표시된 영상과 메모리에 저장된 영상과 관련하여 알림을 제공하는 전자 장치 및 방법
CN114339054B (zh) * 2019-12-31 2024-01-09 北京华为数字技术有限公司 拍照模式的生成方法、装置和计算机可读存储介质
KR102358898B1 (ko) * 2020-08-24 2022-02-07 민팃(주) 객체 촬영 장치 및 방법
CN113030852B (zh) * 2021-03-01 2022-01-07 嘉兴太和信息技术有限公司 一种定位方法及定位系统
CN113727017B (zh) * 2021-06-16 2022-07-05 荣耀终端有限公司 拍摄方法、图形界面及相关装置
CN113810602B (zh) * 2021-08-12 2023-07-11 荣耀终端有限公司 一种拍摄方法及电子设备
CN117979153A (zh) * 2022-10-26 2024-05-03 荣耀终端有限公司 一种录像模式的推荐方法、电子设备及可读存储介质
CN115883957B (zh) * 2022-11-22 2023-10-20 荣耀终端有限公司 一种拍摄模式推荐方法
CN116233599B (zh) * 2022-11-22 2024-04-12 荣耀终端有限公司 一种录像模式推荐方法和电子设备
CN115802144B (zh) * 2023-01-04 2023-09-05 荣耀终端有限公司 视频拍摄方法及相关设备

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070065137A1 (en) * 2005-09-21 2007-03-22 Sony Corporation Photographic device, method of processing information, and program
US20070171282A1 (en) * 2006-01-18 2007-07-26 Casio Computer Co., Ltd. Camera apparatus and imaging method

Family Cites Families (40)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2004254256A (ja) 2003-02-24 2004-09-09 Casio Comput Co Ltd カメラ装置、表示方法及びプログラム
JP2005062370A (ja) * 2003-08-08 2005-03-10 Ricoh Co Ltd 撮像装置
JP2005348298A (ja) 2004-06-07 2005-12-15 Canon Inc デジタルカメラ
EP1802096B1 (fr) * 2004-09-30 2019-10-23 FUJIFILM Corporation Dispositif de traitement d'images, procede et programme de traitement d'images
JP4775029B2 (ja) 2006-02-22 2011-09-21 セイコーエプソン株式会社 撮影シーンの決定
KR100762640B1 (ko) * 2006-07-18 2007-10-01 삼성전자주식회사 촬영 모드를 자동 선택하는 휴대 단말기 및 그 방법
JP2008273171A (ja) 2007-04-04 2008-11-13 Seiko Epson Corp 情報処理方法、情報処理装置及びプログラム
JP5061188B2 (ja) 2007-07-09 2012-10-31 パナソニック株式会社 デジタル一眼レフカメラ
JP5039786B2 (ja) 2007-07-23 2012-10-03 パナソニック株式会社 撮像装置
WO2009090992A1 (fr) 2008-01-17 2009-07-23 Nikon Corporation Appareil photo électronique
JP4544332B2 (ja) * 2008-04-07 2010-09-15 ソニー株式会社 画像処理装置、画像処理方法、プログラム
JP4702401B2 (ja) 2008-06-02 2011-06-15 カシオ計算機株式会社 カメラ、カメラ制御プログラム及びカメラ制御方法
JP2010050798A (ja) 2008-08-22 2010-03-04 Sanyo Electric Co Ltd 電子カメラ
JP5331552B2 (ja) 2009-04-16 2013-10-30 富士フイルム株式会社 デジタルカメラ及びその制御方法
JP5287465B2 (ja) * 2009-04-21 2013-09-11 ソニー株式会社 撮像装置、撮影設定方法及びそのプログラム
KR101599886B1 (ko) * 2009-09-29 2016-03-04 삼성전자주식회사 장면 모드 표시에 따른 영상 처리 장치 및 방법
CN102457673A (zh) * 2010-10-26 2012-05-16 宏达国际电子股份有限公司 影像撷取方法及系统
JP2012109866A (ja) 2010-11-18 2012-06-07 Canon Inc 撮像装置およびその制御方法
JP5921080B2 (ja) 2011-04-28 2016-05-24 キヤノン株式会社 撮像装置及びその制御方法
CN103188423A (zh) 2011-12-27 2013-07-03 富泰华工业（深圳）有限公司 摄像装置及摄像方法
US9143679B2 (en) 2012-01-26 2015-09-22 Canon Kabushiki Kaisha Electronic apparatus, electronic apparatus control method, and storage medium
JP5975656B2 (ja) 2012-01-26 2016-08-23 キヤノン株式会社 電子機器、電子機器の制御方法、プログラム及び記憶媒体
JP5729407B2 (ja) * 2013-02-28 2015-06-03 カシオ計算機株式会社 撮像装置、表示制御方法及びプログラム
CN103293825B (zh) * 2013-06-26 2014-11-26 深圳市中兴移动通信有限公司 多重曝光方法和装置
KR102146856B1 (ko) * 2013-12-31 2020-08-21 삼성전자주식회사 렌즈 특성 기반의 촬영 모드 제시 방법, 상기 방법을 기록한 컴퓨터 판독 가능 저장매체 및 디지털 촬영 장치.
CN103841324A (zh) * 2014-02-20 2014-06-04 小米科技有限责任公司 拍摄处理方法、装置和终端设备
CN104243822A (zh) 2014-09-12 2014-12-24 广州三星通信技术研究有限公司 拍摄图像的方法及装置
EP3018892A1 (fr) * 2014-10-31 2016-05-11 Canon Kabushiki Kaisha Appareil de capture d'images et son procédé de commande correspondant
CN104660908A (zh) * 2015-03-09 2015-05-27 深圳市中兴移动通信有限公司 拍摄装置及其拍摄模式的自动匹配方法
CN104717366B (zh) 2015-03-31 2017-11-14 小米科技有限责任公司 联系人头像的推荐方法及装置
CN104853092A (zh) 2015-04-30 2015-08-19 广东欧珀移动通信有限公司 一种拍照方法及装置
JP6861342B2 (ja) * 2015-09-18 2021-04-21 パナソニックＩｐマネジメント株式会社 撮像装置および撮像装置とサーバとを含むシステム
JP6635816B2 (ja) 2016-02-05 2020-01-29 キヤノン株式会社 撮像装置、撮像装置の制御方法およびプログラム
CN105681675A (zh) 2016-03-22 2016-06-15 珠海格力电器股份有限公司 移动终端及其拍照模式设置方法和装置
CN107229625A (zh) 2016-03-23 2017-10-03 北京搜狗科技发展有限公司 一种拍摄处理方法和装置、一种用于拍摄处理的装置
CN106101536A (zh) 2016-06-22 2016-11-09 维沃移动通信有限公司 一种拍照方法及移动终端
CN107155060A (zh) 2017-04-19 2017-09-12 北京小米移动软件有限公司 图像处理方法及装置
CN107105165A (zh) 2017-05-24 2017-08-29 深圳天珑无线科技有限公司 一种拍照模式的切换方法及装置
CN107333004A (zh) 2017-07-25 2017-11-07 维沃移动通信有限公司 一种拍照方法及移动终端
CN107566529B (zh) * 2017-10-18 2020-08-14 维沃移动通信有限公司 一种拍照方法、移动终端及云端服务器

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070065137A1 (en) * 2005-09-21 2007-03-22 Sony Corporation Photographic device, method of processing information, and program
US20070171282A1 (en) * 2006-01-18 2007-07-26 Casio Computer Co., Ltd. Camera apparatus and imaging method

Also Published As

Publication number Publication date
EP3764631A1 (fr) 2021-01-13
WO2019183775A1 (fr) 2019-10-03
US11503208B2 (en) 2022-11-15
CN111630836B (zh) 2022-03-29
EP3764631A4 (fr) 2021-03-17
JP7072667B2 (ja) 2022-05-20
US20210014420A1 (en) 2021-01-14
KR102540450B1 (ko) 2023-06-05
JP2021517413A (ja) 2021-07-15
RU2748303C1 (ru) 2021-05-21
KR20200128151A (ko) 2020-11-11
CN111630836A (zh) 2020-09-04

Similar Documents

Publication Publication Date Title
EP3764631B1 (fr) Procédé de commande d'assistant numérique et dispositif terminal
KR102597680B1 (ko) 맞춤형 화질 이미지를 제공하는 전자 장치 및 그 제어 방법
CN105049726B (zh) 一种移动终端拍照方法及移动终端
WO2021244295A1 (fr) Procédé et dispositif d'enregistrement vidéo
US20220417417A1 (en) Content Operation Method and Device, Terminal, and Storage Medium
CN104902177A (zh) 一种智能拍照方法及终端
US20220343648A1 (en) Image selection method and electronic device
CN105183837B (zh) 一种移动终端的缩略图列表视图显示方法及系统
US11996123B2 (en) Method for synthesizing videos and electronic device therefor
CN112135046A (zh) 视频拍摄方法、视频拍摄装置及电子设备
CN108737739A (zh) 一种预览画面采集方法、预览画面采集装置及电子设备
US9262062B2 (en) Method of providing thumbnail image and image photographing apparatus thereof
US20230345113A1 (en) Display control method and apparatus, electronic device, and medium
CN110633380B (zh) 图片处理界面的控制方法、装置、电子设备及可读介质
EP3062513A1 (fr) Appareil vidéo et procédé de photographie de celui-ci
CN113873161A (zh) 拍摄方法、装置及电子设备
CN104967778A (zh) 一种对焦提示方法及终端
CN111601012A (zh) 图像处理方法、装置及电子设备
CN111866379A (zh) 一种图像处理方法、图像处理装置、电子设备和存储介质
CN112637515A (zh) 拍摄方法、装置和电子设备
WO2024109224A1 (fr) Procédé de recommandation de mode photographique
US9100577B2 (en) Method of providing user interface and image photographing apparatus applying the same
US11622078B2 (en) Method and apparatus for image formation using preview images
CN118042274B (zh) 一种拍照方法和设备
JP7520535B2 (ja) 画像処理装置

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20201005

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

A4 Supplementary search report drawn up and despatched

Effective date: 20210211

RIC1 Information provided on ipc code assigned before grant

Ipc: H04N 5/232 20060101AFI20210208BHEP

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20220504

REG Reference to a national code

Ref document number: 602018052274

Country of ref document: DE

Ref country code: DE

Ref legal event code: R079

Free format text: PREVIOUS MAIN CLASS: H04N0005232000

Ipc: H04N0023630000

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

RIC1 Information provided on ipc code assigned before grant

Ipc: H04N 23/667 20230101ALI20230112BHEP

Ipc: H04N 23/60 20230101ALI20230112BHEP

Ipc: H04N 23/63 20230101AFI20230112BHEP

INTG Intention to grant announced

Effective date: 20230214

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602018052274

Country of ref document: DE

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 1581756

Country of ref document: AT

Kind code of ref document: T

Effective date: 20230715

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: NL

Ref legal event code: FP

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG9D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230621

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230921

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1581756

Country of ref document: AT

Kind code of ref document: T

Effective date: 20230621

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230621

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230621

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230621

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230621

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230922

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230621

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230621

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230621

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20231021

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230621

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230621

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230621

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20231023

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20231021

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230621

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230621

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230621

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230621

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230621

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602018052274

Country of ref document: DE

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: NL

Payment date: 20240214

Year of fee payment: 7

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230621

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20240130

Year of fee payment: 7

Ref country code: GB

Payment date: 20240201

Year of fee payment: 7

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230621

26N No opposition filed

Effective date: 20240322

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230621

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230621