JP6684883B2 - Method and system for providing camera effects - Google Patents

Description

  The following description relates to a technique for providing an AR (Augmented Reality) camera effect.

  In recent years, mobile data communication services using personal information terminals such as smartphones, tablets, and wearable devices have been widely used.

  Among such mobile data communication services, one of the most popular fields is a wireless broadcasting service field that provides various contents to personal information terminals.

  As an example, Patent Document 1 discloses a technique of providing shooting information to a mobile terminal via Internet broadcasting, wireless broadcasting, or terrestrial broadcasting.

Korean Published Patent No. 10-2003-0029556

  Provided is a method and system capable of rapidly providing a camera effect that matches emotions with respect to a camera image.

  A method of providing an effect in an electronic device realized by a computer, wherein a step of determining an emotion with respect to the camera image as a reference for selecting an effect to be reflected in a camera image of the electronic device; There is provided an effect providing method including a step of immediately reflecting an effect randomly selected from effects showing the corresponding feeling on the camera image based on the feeling.

  According to one aspect, the determining step may determine emotion selected by a user of the electronic device as an effect selection criterion.

  According to another aspect, the determining step may determine the emotion recognized from the facial expression of the camera image as an effect selection criterion.

  According to another aspect, the effect providing method further comprises collecting context information related to a user of the electronic device, and the reflecting step includes an effect corresponding to the determined emotion. Randomly selecting an effect that matches the situation information from among the above.

  According to another aspect, the collecting step includes: time information associated with the camera image, location information associated with the camera image, color information associated with the camera image, shooting mode information associated with the camera image, Collecting at least one situation information of history information about effects used by the user, feedback information about the camera image, target information related to the camera image, and facial expression information recognized from the camera image. You may

  According to another aspect, in the selecting step, a weighted value based on the situation information is given to the effect corresponding to the determined emotion, and an effect to be reflected in the camera image based on the weighted value is added. You may choose.

  According to another aspect, the selecting step may select different situation information according to the determined emotion and use it as an effect selection criterion.

  According to another aspect, the selecting step assigns different weights to the effects corresponding to the determined emotions according to the situation information according to the determined emotions. The effect to be reflected on the camera image may be selected based on

  According to another aspect, the determining step includes providing an emotion selection menu including a plurality of emotion icons and determining the emotion selected from the emotion selection menu as an effect selection criterion.

  According to still another aspect, in the reflecting step, the effect selection target to be reflected in the camera image is different from the effects corresponding to the determined emotion based on the user gesture type for the emotion selection menu. You may choose.

  A computer program for causing a computer to execute the effect providing method is provided.

  A computer-readable recording medium storing the computer program is provided.

  An effect providing system in an electronic device implemented by a computer, the system including at least one processor implemented to execute a computer-readable instruction, the at least one processor displaying a camera image of the electronic device. As a criterion for selecting an effect to be reflected, an emotion determination unit that determines an emotion with respect to the camera image, and an effect randomly selected from among the effects showing the corresponding emotion based on the determined emotion are described above. Provided is an effect providing system including an effect selection unit that immediately reflects on a camera image.

  It is possible to quickly provide a camera effect that matches the emotion of the camera image.

It is a figure showing the example of the network environment in one embodiment of the present invention. It is a block diagram for explaining an electronic device and an internal configuration of a server in one embodiment of the present invention. FIG. 6 is a diagram showing an example of components that can be included in a processor of an electronic device according to an embodiment of the present invention. 6 is a flowchart showing an example of an effect providing method that can be executed by an electronic device according to an embodiment of the present invention. It is a figure for demonstrating the situation information utilized as an effect selection reference | standard in one Embodiment of this invention. FIG. 6 is an exemplary diagram illustrating a process of providing an effect corresponding to emotions according to an exemplary embodiment of the present invention. FIG. 6 is an exemplary diagram illustrating a process of providing an effect corresponding to emotions according to an exemplary embodiment of the present invention. FIG. 6 is an exemplary diagram illustrating a process of providing an effect corresponding to emotions according to an exemplary embodiment of the present invention. FIG. 6 is an exemplary diagram illustrating a process of providing an effect corresponding to emotions according to an exemplary embodiment of the present invention. FIG. 6 is an exemplary diagram illustrating a process of providing an effect corresponding to emotions according to an exemplary embodiment of the present invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

  Embodiments of the present invention relate to a technique for providing a camera effect, and more particularly, to a method and a system capable of quickly providing a camera effect that matches an emotion of a camera image.

  Embodiments, including those specifically disclosed herein, can rapidly provide camera effects during video footage or live broadcast, thereby providing speed, efficiency, versatility, entertainment elements, etc. A considerable advantage can be achieved in the aspect of.

  FIG. 1 is a diagram showing an example of a network environment in an embodiment of the present invention. The network environment in FIG. 1 shows an example including a plurality of electronic devices 110, 120, 130, 140, a plurality of servers 150, 160, and a network 170. Such FIG. 1 is merely an example for explaining the invention, and the number of electronic devices and the number of servers are not limited as in FIG.

  The plurality of electronic devices 110, 120, 130, 140 may be fixed terminals or mobile terminals realized by computer devices. Examples of the plurality of electronic devices 110, 120, 130, 140 include smartphones, mobile phones, tablets, navigation, PCs (personal computers), notebook PCs, digital broadcasting terminals, PDAs (Personal Digital Assistants), and PMPs (Portables). Multimedia Player). For example, the first electronic device 110 may communicate with other electronic devices 120, 130, 140 and / or servers 150, 160 via the network 170 using a wireless or wired communication scheme.

  The communication method is not limited, and not only a communication method that utilizes a communication network (for example, a mobile communication network, a wired Internet, a wireless Internet, a broadcasting network) that the network 170 can include, but also a short distance between devices. Wireless communication may be included. For example, the network 170 includes a PAN (personal area network), a LAN (local area network), a CAN (campus area network), a MAN (a metropolitan area network), a WAN (broadband network), and a WAN (wideBear network). It may include any one or more of the networks. Further, the network 170 may include any one or more of network topologies including, but not limited to, bus networks, star networks, ring networks, mesh networks, star-bus networks, tree or hierarchical networks, and the like. It will not be done.

  Each of the servers 150 and 160 may be realized by one or more computer devices that communicate with the electronic devices 110, 120, 130 and 140 via the network 170 to provide instructions, codes, files, contents, services and the like. For example, the server 160 may provide a file for installing an application to the first electronic device 110 connected via the network 170. In this case, the first electronic device 110 may use the file provided by the server 160 to install the application. The service provided by the server 150 is connected to the server 150 under the control of an operating system (OS: Operating System) included in the first electronic device 110 and at least one program (for example, a browser or an installed application). Content may be provided. For example, when the first electronic device 110 sends a service request message to the server 150 via the network 170 under the control of the application, the server 150 may send a code corresponding to the service request message to the first electronic device 110, The first electronic device 110 may provide the content to the user by configuring and displaying a screen based on the code under the control of the application.

  FIG. 2 is a block diagram for explaining internal configurations of the electronic device and the server according to the embodiment of the present invention. In FIG. 2, the internal configuration of the first electronic device 110 will be described as an example for the electronic device, and the internal configuration of the server 150 will be described as an example for the server. The other electronic devices 120, 130, 140 and the server 160 may also have the same or similar internal configuration.

  The first electronic device 110 and the server 150 may include memories 211, 221, processors 212, 222, communication modules 213, 223, and input / output interfaces 214, 224. The memories 211 and 221 are computer-readable recording media, and may include a permanent mass storage device such as a random access memory (RAM), a read only memory (ROM), and a disk drive. In addition, the memories 211 and 221 may store an operating system and at least one program code (for example, a code for an application installed and executed in the first electronic device 110). Such software components may be loaded from a computer-readable recording medium other than the memories 211 and 221. Such another computer-readable recording medium may include a computer-readable recording medium such as a floppy (registered trademark) drive, a disk, a tape, a DVD / CD-ROM drive, and a memory card. In another embodiment, the software components may be loaded into the memories 211 and 221 through the communication modules 213 and 223 which are not computer-readable recording media. For example, at least one program is a program installed by a file provided by a file distribution system (for example, the server 160 described above) that distributes an installation file of a developer or an application via the network 170 (for example, the above described It may be loaded into the memory 211, 221 based on (application).

  Processors 212, 222 may be configured to process instructions of computer programs by performing basic arithmetic, logic, and input / output operations. The instructions may be provided to the processors 212, 222 by the memory 211, 221, or the communication modules 213, 223. For example, the processors 212 and 222 may be configured to execute the received instructions according to the program code recorded in the recording device such as the memories 211 and 221.

  The communication modules 213 and 223 may provide a function for the first electronic device 110 and the server 150 to communicate with each other via the network 170, or other electronic device (for example, the second electronic device 120). Alternatively, a function for communicating with another server (as an example, the server 160) may be provided. As an example, a request (for example, a search request) generated by the processor 212 of the first electronic device 110 according to a program code recorded in a recording device such as the memory 211 is transmitted via the network 170 under the control of the communication module 213. May be transmitted to the server 150. On the contrary, control signals, commands, contents, files, etc. provided according to the control of the processor 222 of the server 150 pass through the communication module 223 and the network 170, and through the communication module 213 of the first electronic device 110, the first electronic device 110. It may be received by the device 110. For example, control signals and commands of the server 150 received through the communication module 213 may be transmitted to the processor 212 and the memory 211, and content and files may be recorded in a recording medium that the first electronic device 110 may further include. May be done.

  Input / output interface 214 may be a means for interfacing with input / output device 215. For example, the input device may include a device such as a keyboard or a mouse, and the output device may include a device such as a display for displaying the communication session of the application. As another example, the input / output interface 214 may be a means for interfacing with a device having integrated functions for input and output, such as a touch screen. As a more specific example, when the processor 212 of the first electronic device 110 processes the instructions of the computer program loaded in the memory 211, the processor 212 is configured by using the data provided by the server 150 or the second electronic device 120. Service screens and content may be displayed on the display via the input / output interface 214. The input / output interface 224 may similarly output information configured using the data provided by the server 150 when the processor 222 of the server 150 processes the instructions of the computer program loaded in the memory.

  Moreover, in another embodiment, the first electronic device 110 and the server 150 may include more components than the components of FIG. 2. For example, the first electronic device 110 may be configured to include at least a part of the input / output device 215 described above, and may include a transceiver, a GPS (Global Positioning System) module, a camera, various sensors, a database, and the like. It may further include other components such as. As a more specific example, when the first electronic device 110 is a smartphone, an acceleration sensor or a gyro sensor that is generally included in the smartphone, a camera, various physical buttons, a button using a touch panel, an input / output port. It will be appreciated that various components such as a vibrator for vibration may be further included in the first electronic device 110. However, most prior art components need not be explicitly shown in the figures.

  FIG. 3 is a block diagram showing an example of components that can be included in the processor of the electronic device according to the embodiment of the present invention, and FIG. 4 is executed by the electronic device according to the embodiment of the present invention. It is the flowchart which showed the example of the possible method.

  An effect providing system realized by a computer may be configured in the first electronic device 110 according to the present embodiment. The effect providing system may be composed of a PC-based program or an application dedicated to a mobile terminal. The effect providing system according to the present embodiment may be implemented in the form of a program that operates independently, or may be configured in the form of an in-app of a specific application (for example, a camera function). It may be realized so that it can operate on a specific application. The effect providing system is realized in the form of an application installed in the first electronic device 110, and a random effect suitable for emotion can be immediately applied to the camera during live broadcasting by linking with the server 150.

  For example, the effect providing system implemented in the first electronic device 110 may execute the effect providing method based on a command provided by the application installed in the first electronic device 110. In order to execute the effect providing method according to FIG. 4, the processor 212 of the first electronic device 110 includes, as components, an emotion determination unit 310, a situation collection unit 320, and an effect selection unit 330, as illustrated in FIG. You may be prepared. In some embodiments, components of processor 212 may optionally be included in or excluded from processor 212. Also, in some embodiments, components of processor 212 may be separate or merged for representation of the functionality of processor 212.

  The processor 212 and the components of the processor 212 may control the first electronic device 110 to perform steps 410 to 440 included in the effect providing method of FIG. 4. For example, the processor 212 and components of the processor 212 may be implemented to execute instructions in the code of the operating system and the code of the at least one program included in the memory 211.

  Here, the components of the processor 212 are executed by the processor 212 according to an instruction provided by a program code recorded in the first electronic device 110 (as an example, an instruction provided by an application executed in the first electronic device 110). Can be different representations of different functions of the processor 212. For example, the emotion determination unit 310 may be used as a functional expression of the processor 212 that controls the first electronic device 110 according to the above-described instruction so that the first electronic device 110 determines the emotion for the camera image.

  At step 410, the processor 212 may read the necessary instructions from the memory 211 loaded with the instructions related to the control of the first electronic device 110. In this case, the instructions read may include instructions for controlling processor 212 to perform steps 420-440 described below.

  According to the embodiment of the present invention, an AR image is displayed on a camera image in a streaming environment in which a video image captured by the camera of the first electronic device 110 is sent out in real time via an internet line, such as live broadcasting and internet telephone (VoIP). As a result, it is possible to combine a predetermined effect and provide it as a single image. In particular, instead of directly specifying the effect to be reflected in the camera image, in consideration of the environmental condition of sending the camera image in real time, It enables to select effects automatically and provide them more quickly. In addition to the live streaming environment described above, effects may be automatically selected and provided for camera images including video image capturing environments.

  In operation 420, the emotion determination unit 310 may determine an emotion for the camera image as a reference for selecting an effect to be reflected in the camera image of the first electronic device 110. In the present specification, an effect may be a content composed of an image object and including all filters, stamps, pictograms, and the like that can be combined with a camera image. At this time, not only a fixed object but also a moving image object to which flash or animation is applied can be realized. Such effects indicate emotional information and may be classified in advance according to emotions. In other words, multiple emotions (eg, 6 emotions: happy, love, sad, angry, cool, crazy) may be predefined and each emotion may be grouped and managed according to the emotion. . As an example, the emotion determination unit 310 enables the user of the first electronic device 110 to select any one specific emotion from among a plurality of predefined emotions, and the emotion determination unit 310 selects the specific emotion for the camera image as the emotion selected by the user. You may decide your feelings. As another example, the emotion determination unit 310 may determine the emotion for the camera image by extracting emotion information from the facial expression based on the camera image. Facial expression is indicated by the contraction of facial muscles that occur when facial elements such as eyebrows, eyes, nose, mouth, and skin are deformed, and the strength of facial expression is the geometrical change of facial features or muscle expression. May be determined by the density of For example, the emotion determination unit 310 extracts a region of interest (for example, an eye region, an eyebrow region, a nose region, a mouth region, etc.) for extracting a feature based on a facial expression, and then extracts a feature point from the region of interest to extract a feature point. May be used to determine certain feature values. The feature value corresponds to a specific numerical value indicating a human facial expression, such as the distance between the feature points. The emotion determination unit 310 determines a certain intensity value according to the degree of the numerical value with respect to the feature value shown in the image, and applies the prepared mapping value in order to apply the determined feature value to the emotion reaction value model. A table is used to determine a constant intensity value that matches the numerical value of each specific value. The mapping table is prepared in advance based on the emotion reaction value model. The emotion determination unit 310 may map the emotion reaction value model and the strength value, and extract the determined emotion type and strength based on the result of applying the corresponding strength value to the emotion reaction value model. The above-described technique for extracting emotion information is merely an example, and the present invention is not limited to this, and other well-known techniques can be used.

  Therefore, the emotion determination unit 310 may determine the emotion selected by the user or the emotion automatically recognized from the image as one of the criteria for selecting the effect to be reflected in the camera image.

  In operation 430, the situation collecting unit 320 may collect at least one situation information related to the user of the first electronic device 110. In other words, the situation collection unit 320 may collect the situation information of the user as an additional criterion for selecting the effect to be reflected in the camera image. As an example, the situation collection unit 320 collects time information associated with a camera image, and may collect time information including the date and time acquired from the first electronic device 110 at the time of image capturing. As another example, the situation collection unit 320 collects location information related to a camera image, and collects position information such as GPS coordinates from the first electronic device 110 at the time of image capturing, for example. Good. In addition, as another example, the situation collection unit 330 collects color information related to a camera image, analyzes color values included in the camera screen at the time of image capturing, and collects color information in the camera image. The color information may be collected by recognizing the skin tone of the face, the camera illumination color, or the like. As another example, the situation collection unit 330 collects shooting mode information related to a camera image, and includes, for example, an in-camera shooting mode / out-camera shooting mode or a person shooting mode / background shooting mode. Such mode information on which the image was captured may be collected. As another example, the situation collection unit 330 collects history information about an effect used by the user of the first electronic device 110. For example, the situation collection unit 330 is similar to an effect that the user usually uses or similar to this effect. Information about effects may be collected. In addition, as another example, the situation collection unit 330 collects feedback information of a viewer with respect to a camera image, and includes a dialogue exchanged between the user and the viewer, comments and reactions registered by the viewer, and the like. The context of may be collected. In addition, as another example, the situation collection unit 330 collects target information related to a camera image. For example, when the user selects a specific community and provides the camera image, the name of the community is displayed. Information about the community may be collected, such as categories and categories. As still another example, the situation collection unit 330 collects facial expression information including emotional strength (degree) from the camera image, and collects not only the emotion type but also the emotional strength from the facial expression in the camera image. You may Not only the type of emotion but also the effect that can express the intensity may be targeted and the actual facial expression in the image may be grasped and used as the effect selection criterion.

  Therefore, the situation collection unit 320 uses the user's situation, that is, time information, location information, color information, shooting mode information, history information, feedback information, and target information as an additional criterion for selecting an effect to be reflected in the camera image. , And at least one of facial expression information may be collected.

  In step 440, the effect selection unit 330 may randomly select an effect to be reflected in the camera image from the plurality of effects classified as emotions determined in step 420. For example, if the user selects "happy" from a plurality of predefined emotions (for example, 6 types of emotions: happy, love, sad, anger, cool, crazy), it is classified as "happy". A specific effect may be randomly selected from within the effect group and immediately applied to the camera image. Therefore, the effect of the corresponding emotion can be randomly selected based on the emotion selected by the user, and can be immediately provided to the image sent in real time.

  Furthermore, the effect selection unit 330 may randomly select, as an effect to be reflected in the camera image, an effect matching the situation information collected in step 430 from effects corresponding to the emotion determined in step 420. . Such a random effect is selected by using weight value modeling based on at least one of situation information among time information, location information, color information, shooting mode information, history information, feedback information, target information, and facial expression information. May be done. As shown in FIG. 5, the effect selecting unit 330 uses the emotion selected by the user or the emotion 501 automatically recognized from the image as a criterion for selecting an effect, but the user's context is considered. The status information (at least one of time information, location information, color information, shooting mode information, history information, feedback information, target information, and facial expression information) 502 may be used as an additional selection criterion. Each effect may include metadata, various tags, information about design elements, and the like as attribute information related to the effect. Accordingly, the effect selection unit 330 may compare the situation information with the attribute information, give a weighted value to the effect for each situation, and provide the effect with the highest score calculated as the weighted value. An effect randomly selected from effects having a score calculated as a weighted value or higher may be provided.

  As an example of a method of giving a weighted value for each situation, the effect selection unit 330 may use all the situation information 502 as additional selection criteria regardless of the emotion 501 and give a weighted value for each situation information 502. As another example, the effect selection unit 330 may select and use different situation information 502 as the additional selection criterion according to the emotion 501. For example, in the case of "happy", the time information and the location information of the situation information 502 are used as additional selection criteria, and in the case of "sad", the location information of the situation information 502 is used as additional selection criteria. You may Further, as another example, the effect selection unit 330 may give different weights to the situation information 502 according to the emotion 501. For example, in the case of "happy", a higher weight is given to the time information and the place information of the situation information 502, and in the case of "sad", a higher weight is given to the place information of the situation information 502. May be given.

  In the case of the time information of the situation information 502 which is the additional selection criterion: a weighted value may be given to the effect specialized for the user's shooting date or shooting time zone, for example, Christmas, Valentine's day, late night time zone. An effect specialized for a specified time such as may be selected.

  In the case of location information in the situation information 502 that is the additional selection criterion: a specific location may be recognized from the user's shooting location or camera image, and a weight value may be given to the effect specialized for the location, for example, You may select effects that are specific to a specific location, such as Eiffel Tower, Namsan, and Myeongdong.

  In the case of the color information of the situation information 502 which is the additional selection criterion: By analyzing the color value included in the camera screen or recognizing the skin tone or illumination color of the person in the image, Weighting may be added to the effect suitable for the color.For example, if the overall background color in the camera is extremely bright, avoid similar colors and select a camera effect with a contrasting color. Good.

  In the case of the shooting mode information of the situation information 502 serving as the additional selection criteria: the video shooting mode such as the in-camera shooting mode / out-camera shooting mode, or the person shooting mode / background shooting mode is recognized and suitable for the corresponding mode. A weighted value may be added to the effect. For example, in self-shooting mode using the in-camera, an effect centered on a mask is selected, and when shooting multiple people, multiple people such as a face swap effect are selected. It is also possible to select a camera effect of a form applicable to, and select a special effect for the background place in the background shooting mode using the out-camera.

  In the case of history information of the situation information 502 which is the additional selection criterion: a weight value may be given to an effect frequently used by the user or an effect similar to this, for example, an effect used recently by the user, or an effect recently used by the user. It is possible to select an effect or the like that has been used a lot within a certain period of time.

  In the case of the feedback information of the situation information 502 which is the additional selection criterion: The context such as the interaction between the user and the viewer or the reaction of the viewer is analyzed, and the weighted value is given to the effect matching the corresponding content. Of course, for example, an effect suitable for the title of the dialogue exchanged between the user and the viewer may be selected.

  In the case of the target information in the situation information 502 which is the additional selection criterion: A weight value may be given to an effect suitable for the characteristic of the target object for which the user intends to provide a camera image, and for example, a scuba diving club community group When providing a live broadcast to, a camera effect related to scuba diving may be selected, and when providing a live broadcast to a dog community group, a camera effect related to an animal may be selected.

  In the case of facial expression information of the situation information 502 serving as an additional selection criterion: a weighted value may be given to an effect suitable for the facial expression in the image, for example, a smiling expression when a person in the image is laughing. You may select an effect that is specialized for. As another example, the emotion intensity may be analyzed from the facial expressions in the image, and the effect specialized for the intensity may be selected. For example, when the user selects “happy” and the emotional strength “3” is recognized from the video, an effect suitable for the emotional strength “3” may be selected from the effects belonging to the “happy” group.

  6 to 10 are exemplary views for explaining a process of providing an effect corresponding to an emotion according to an exemplary embodiment of the present invention.

  FIG. 6 is a diagram exemplifying a camera image screen 600 provided as a live broadcast. The camera image screen 600 may include an emotion selection menu 601 for selecting an effect group classified into a plurality of emotions. For example, an effect group may be classified into 6 kinds of emotions, that is, happy, love, sad, anger, cool, and crazy, and is composed of 6 emotion icons so that the emotions can be selected as an effect selection criterion. The displayed emotion selection menu 601 may be displayed on the camera image screen 600.

  Referring to FIG. 7, when the user selects the “happy” icon from the emotion selection menu 601 on the camera image screen 600, an effect is randomly selected from effects belonging to the “happy” group and immediately applied to the camera. As a result, it is possible to provide the live image of the image 710 on which the effect suitable for the emotion “happy”, for example, the heart mark effect is reflected.

  On the other hand, as shown in FIG. 8, when the user selects the “sad” icon from the emotion selection menu 601 on the camera image screen 600, an effect is randomly selected from the effects belonging to the “sad” group, and the selected effect is displayed on the camera. By immediately applying it, it is possible to provide an image suitable for emotion "sad", for example, a video 810 in which a tear mark effect is reflected in live broadcasting.

  As described above, as a criterion for selecting an effect, it is possible to select a random effect suitable for the user's situation by additionally using at least one situation information together with the emotion selected by the user. Become. Therefore, from the effects belonging to the emotion group selected by the user, an effect more suitable for the user's actual situation can be selected and provided in live broadcasting.

  Further, the effect selection target reflected on the camera may be applied differently according to the user gesture type for the emotion selection menu 601. For example, a user gesture for the emotion selection menu 601 is divided into a single tap, a double tap, and a long tap. In the case of a single tap, an effect randomly selected from a specific emotion group is provided, and in the case of a double tap, May provide recently used effects, and in the case of long tap, an effect selected directly by the user from the effect list recommended by a predetermined recommendation criterion may be provided.

  When the user single-tapes the "happy" icon in the emotion selection menu 601 on the camera image screen 600, as described with reference to FIG. 7, an effect randomly selected from the "happy" group, for example, heart. The video 710 in which the mark effect is reflected may be provided as a live broadcast.

  Assuming that the star mark effect is the most recently used effect in the "happy" group, the user double taps the "happy" icon from the emotion selection menu 601 on the camera image screen 600, as shown in FIG. In this case, apply the star mark effect, which is a recently used effect instead of a random effect, to the camera image from within the emotion group, so that the recently used effect of the “happy” group effect is reflected. The captured image 910 will be provided as a live broadcast.

  Referring to FIG. 10, when the user long-tapes the “happy” icon in the emotion selection menu 601 on the camera image screen 600, some effects in the “happy” group that meet predetermined conditions are recommended in the effect list. It may be configured and provided as 1011. As an example, the recommended effect may be selected on the basis of user history information for the effect, and for example, an effect frequently used by the user may be recommended. The user may directly select the effect to be reflected on the camera image from the recommended effect list 1011 composed of the effects frequently used in the “happy” group, and not the random effect (for example, the effect selected directly by the user). A video 1010 that reflects the lip mark effect) may be provided live.

  As described above, according to the embodiments of the present invention, it is possible to quickly provide a camera effect without interrupting the flow of the broadcast in consideration of the live broadcast situation, and automatically recognize the emotion and the image selected by the user during the live broadcast. It is possible to provide a camera effect suitable for the given emotion. In addition, according to an embodiment of the present invention, a camera effect is provided by using weighted value modeling in consideration of a user's situation, among camera effects that match an emotion selected by a user and an emotion automatically recognized from a video. As a result, it is possible to provide an effect that maximizes and expresses the corresponding emotion.

  The apparatus described above may be implemented by hardware components, software components, and / or a combination of hardware and software components. For example, the devices and components described in the embodiments include a processor, a controller, an ALU (arithmetic logic unit), a digital signal processor, a microcomputer, a FPGA (field programmable gate array), a PLU (programmable logic unit), a microprocessor, Alternatively, it may be implemented utilizing one or more general purpose or special purpose computers, such as various devices capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and one or more software applications running on the OS. The processing device may also access, record, manipulate, process, and generate data in response to executing the software. For ease of understanding, a single processing device may be described as being used, but those skilled in the art will appreciate that the processing device may include multiple processing elements and / or multiple types of processing elements. You can understand. For example, the processing unit may include multiple processors or a processor and a controller. Other processing configurations are also possible, such as parallel processors.

  Software may include computer programs, code, instructions, or combinations of one or more of these to configure the processing units to operate as desired, or to instruct the processing units independently or collectively. You can The software and / or data may be embodied in any type of machine, component, physical device, computer storage media or device for interpreting or providing instructions or data to the processing device. Good. The software may be distributed and recorded or executed in a distributed manner on computer systems connected by a network. The software and data may be recorded on one or more computer-readable recording media.

  The method according to the embodiment may be embodied in the form of program instructions executable by various computer means and recorded on a computer-readable medium. At this time, the medium may continuously record the program executable by the computer, or may temporarily record the program for execution or download. Further, the medium may be various recording means or storage means in a form in which single or plural pieces of hardware are combined, but the medium is not limited to the medium directly connected to a certain computer system, and it may be distributed over a network. And may exist. Examples of the medium include a magnetic medium such as a hard disk, a floppy (registered trademark) disc, and a magnetic tape, an optical medium such as a CD-ROM and a DVD, a magneto-optical medium such as a floppy disk, And ROM, RAM, flash memory, etc., and may be configured to record program instructions. Further, as another example of the medium, there are an application store for distributing an application, a site for supplying or distributing various other software, and a recording medium or a storage medium managed by a server or the like.

  Although the embodiments have been described above based on the limited embodiments and the drawings, those skilled in the art will be able to make various modifications and variations from the above description. For example, the described techniques may be performed in a different order than the described methods, and / or components of the described systems, structures, devices, circuits, etc. may be implemented in a different manner than the described methods. Appropriate results can be achieved whether combined or combined, or apposed or replaced by other components or equivalents.

  Therefore, even in the case of different embodiments, if they are equivalent to the claims, they belong to the appended claims.

212: Processor 310: Emotion determination unit 320: Situation collection unit 330: Effect selection unit

Claims (14)

A method for providing an effect in an electronic device realized by a computer,
As a reference for selecting the effect to be reflected to the camera image of the electronic device, determining step of determining an emotion that corresponds to the camera image,
A collection step of collecting situation information related to the user of the electronic device, and a reflection of immediately selecting an effect randomly selected from among effects showing the corresponding emotion on the camera image based on the determined emotion. viewing including the stage,
In the reflecting step, different weights are assigned to the situation information according to the determined emotions from among the effects indicating the determined emotions, and reflected in the camera image based on the weighted values. An effect providing method including a step of selecting an effect to be performed . The determining step includes
The emotion selected by the user of the electronic device is determined as an effect selection criterion,
The effect providing method according to claim 1. The determining step includes
The emotion recognized from the facial expression of the camera image is determined as an effect selection criterion,
The effect providing method according to claim 1. The status information is
Time information associated with the camera image, location information associated with the camera image, color information associated with the camera image, shooting mode information associated with the camera image, history information regarding effects used by the user, the camera image feedback information for the target information associated with the camera image, and at least one of the camera image recognized facial expression information from,
The effect providing method according to claim 1 . The step of selecting is
According to the determined emotion, different situation information is selected and used as an effect selection criterion,
The effect providing method according to claim 4 . The determining step includes
The effect providing method according to claim 1, further comprising: providing an emotion selection menu configured with a plurality of emotion icons; and determining an emotion selected from the emotion selection menu as an effect selection criterion. The reflecting step is
Based on a user gesture type for the emotion selection menu, different effect selection targets to be reflected in the camera image are selected from among the effects corresponding to the determined emotion.
The effect providing method according to claim 6 . Computer program for executing the effect providing method according to the computer in any one of claims 1-7. A computer-readable recording medium recording the computer program according to claim 8 . A system for providing effects in electronic devices realized by a computer,
Including at least one processor implemented to execute computer readable instructions;
The at least one processor is
The basis for selecting the effect to be reflected to the camera image of the electronic device, the emotion determiner that determines an emotion that corresponds to the camera image,
Based on the determined emotion, the situation collecting unit that collects the situation information related to the user of the electronic device, and the effect randomly selected from the effects showing the corresponding emotion are immediately reflected in the camera image. Equipped with an effect selection section ,
The effect selection unit assigns a different weight value for each of the situation information items according to the determined emotion from among the effects indicating the determined emotion, and based on the weight value, the camera image is added to the camera image. An effect providing system that selects the effect to be reflected . The emotion determination unit is
The emotion selected by the user of the electronic device or the emotion recognized from the facial expression of the camera image is determined as the effect selection criterion.
The effect providing system according to claim 10 . The status information is
Time information associated with the camera image, location information associated with the camera image, color information associated with the camera image, shooting mode information associated with the camera image, history information regarding effects used by the user, the camera image feedback information for the target information associated with the camera image, and at least one of the camera image recognized facial expression information from,
The effect providing system according to claim 10 . The effect selection unit,
According to the determined emotion, different situation information is selected and used as an effect selection criterion,
The effect providing system according to claim 12 . The emotion determination unit is
An emotion selection menu composed of a plurality of emotion icons is provided, and the emotion selected from the emotion selection menu is determined as an effect selection criterion,
The effect selection unit,
Based on a user gesture type for the emotion selection menu, different effect selection targets to be reflected in the camera image are selected from among the effects corresponding to the determined emotion.
The effect providing system according to claim 10 .

Images