WO2024085422A1

WO2024085422A1 - Electronic device, method for operating electronic device, and image editing system

Info

Publication number: WO2024085422A1
Application number: PCT/KR2023/012793
Authority: WO
Inventors: 프라이모스트카안드리; 젤레즈니아코브드미트로; 자이체브빅토르; 체르네하아나스타시아; 라디보넨코올가
Original assignee: 삼성전자 주식회사
Priority date: 2022-10-18
Filing date: 2023-08-29
Publication date: 2024-04-25
Also published as: KR20240054111A

Abstract

An electronic device (100) for editing an image displayed on a display (160) according to an embodiment disclosed herein comprises: memory (120) including one or more instructions; and one or more processors (130), wherein the one or more processors (130) can execute the one or more instructions. An electronic device (100) according to one embodiment can receive an editing signal associated with the editing of the image displayed on the display (160) from an external device (200). An electronic device (100) according to one embodiment can use the received editing signal to generate an edited image in which virtual light is synthesized with the image. An electronic device (100) according to one embodiment can display the generated edited image on the display (160).

Description

Electronic device, method of operating the electronic device, and image editing system

The disclosed embodiments relate to an electronic device, a method of operating the electronic device, and an image editing system. More specifically, it relates to an electronic device that edits images in conjunction with an external device, a method of operating the electronic device, and an image editing system.

As the performance of cameras installed in mobile devices such as smartphones improves, methods for controlling the light irradiated to subjects in various environments have been developed. For example, there is a method of installing a light source on a mobile device and shining light on the subject in real time, or a method of correcting the light after the fact in an image that has already been taken.

However, the above-mentioned conventional methods have the disadvantage of increasing the weight and cost of the device because a separate light source is installed in the device, and since the image is corrected after the fact, the edited image has the problem of not being natural.

The electronic device 100 for editing an image displayed on the display 160 according to the disclosed embodiment includes a memory 120 including one or more instructions and one or more processors 130, and the one or more processors 130 include one The above instructions can be executed.

The electronic device 100 according to one embodiment may receive an editing signal associated with editing an image displayed on the display 160 from the external device 200.

The electronic device 100 according to one embodiment may generate an edited image in which virtual light is synthesized into the image using the received editing signal.

The electronic device 100 according to one embodiment may display the generated edited image on the display 160.

A method of operating the electronic device 100 for editing an image displayed on the display 160 according to the disclosed embodiment may include receiving an editing signal associated with editing an image displayed on the display from an external device (step 410). .

A method of operating the electronic device 100 according to an embodiment may include a step 440 of generating an edited image in which virtual light is synthesized into an image using a received editing signal.

A method of operating the electronic device 100 according to an embodiment may include displaying the generated edited image on a display (step 450).

The image editing system 10 according to the disclosed embodiment may include an external device 200 that generates a plurality of editing signals for an image and an electronic device 100 that receives a plurality of editing signals and edits the image.

The electronic device 100 of the image editing system according to one embodiment may include an image generator 110 that generates an edited image by combining virtual light irradiated on the image with the image.

The electronic device 100 of the image editing system according to one embodiment may include a memory 120 including one or more instructions.

The electronic device 100 of the image editing system according to one embodiment includes one or more processors 130, and the one or more processors 130 may execute one or more instructions based on image editing signals.

The electronic device 100 of the image editing system according to one embodiment may generate virtual light and control the image generator 110 to adjust the characteristics of the generated virtual light.

FIG. 1 illustrates the configuration of an image editing system 10 according to an exemplary embodiment disclosed.

Figure 2 is a block diagram of an electronic device 100 according to one disclosed embodiment.

Figure 3 is a block diagram of the processor 130 according to the disclosed embodiment.

FIG. 4 is a flowchart of a method of operating the electronic device 100 according to the disclosed embodiment.

FIG. 5 is a flowchart illustrating a process of selecting an image in a method of operating the electronic device 100 according to an exemplary embodiment of the present disclosure.

FIG. 6 is a flowchart illustrating a process of changing the characteristics of light in a method of operating an electronic device 100 according to an exemplary embodiment of the present disclosure.

FIG. 7 is a flowchart illustrating a process of adding a parameter in a method of operating the electronic device 100 according to an embodiment of the disclosed disclosure.

FIG. 8 illustrates that a virtual light source 200a is controlled in conjunction with an external device 200 according to an exemplary embodiment of the present disclosure.

FIGS. 9A and 9B illustrate controlling a virtual light source 200a in the electronic device 100 according to an exemplary embodiment of the present disclosure. FIG. 10 is a block diagram of an external device 200 according to an embodiment of the disclosed technology.

FIG. 10 is a block diagram of an external device 200 according to an embodiment of the disclosed technology.

The terms used in this specification will be briefly explained, and the present invention will be described in detail.

The terms used in the present invention are general terms that are currently widely used as much as possible while considering the functions in the present invention, but this may vary depending on the intention or precedent of a person working in the art, the emergence of new technology, etc. In addition, in certain cases, there are terms arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the relevant invention. Therefore, the terms used in the present invention should be defined based on the meaning of the term and the overall content of the present invention, rather than simply the name of the term.

When it is said that a part "includes" a certain element throughout the specification, this means that, unless specifically stated to the contrary, it does not exclude other elements but may further include other elements. In addition, terms such as "... unit" and "module" used in the specification refer to a unit that processes at least one function or operation, which may be implemented as hardware or software, or as a combination of hardware and software. .

Below, with reference to the attached drawings, embodiments will be described in detail so that those skilled in the art can easily implement the present invention. However, the present invention may be implemented in many different forms and is not limited to the embodiments described herein. In order to clearly explain the present invention in the drawings, parts unrelated to the description are omitted, and similar parts are given similar reference numerals throughout the specification.

In embodiments herein, the term “user” refers to a person who controls a system, function, or operation, and may include a developer, administrator, or installer.

Referring to FIG. 1 , an image editing system 10 according to an embodiment may include an electronic device 100 and an external device 200.

The electronic device 100 according to one embodiment includes at least one camera or a sensor that detects the external environment, and may display the external environment. For example, the electronic device 100 includes a smart phone, laptop computer, tablet PC, digital camera, e-book terminal, digital broadcasting terminal, Personal Digital Assistants (PDA), and Portable Multimedia Player (PMP). ), navigation, MP3 player, etc., but are not limited to this. For example, the electronic device 100 may include a wearable device that can be worn by a user.

The external device 200 may generate an editing signal for an image displayed on the electronic device 100 and transmit the generated editing signal to the electronic device 100. The external device 200 according to one embodiment may generate editing signals for images based on the operation of the external device 200 itself. For example, the external device 200 may generate editing signals for the image based on the angle, rotational speed, acceleration, movement, or position of the external device 200 with respect to the electronic device 100. The external device 200 according to one embodiment may be a stylus pen including a plurality of sensors, but is not limited thereto, and may include a plurality of devices that can be used in conjunction with the electronic device 100. You can.

Referring to FIG. 2, the electronic device 100 according to the disclosed embodiment includes an image generator 110, a memory 120, a processor 130, a storage 140, a camera 150, or a display 160. can do.

The image generator 110 may generate virtual light based on image editing signals (Edit_Sig), adjust the characteristics of the generated virtual light, and synthesize the virtual light into the image to create an edited image. For example, for a photo taken in an environment where there is actually insufficient light or no light desired by the user, the image generator 110 synthesizes virtual light and edits the photo as if it were taken in an environment where a virtual light source exists. Images can be created. The image generator 110 according to one embodiment may generate light desired by the user by generating virtual light with adjusted parameters. For example, parameters that can be adjusted by the user may include, but are not limited to, light intensity, location of the light source, direction of light, radius of light, or illuminance, and may include various parameters that can represent the characteristics of light. there is.

The memory 120 may include at least one instruction for generating virtual light and controlling characteristics of the virtual light. For example, memory 120 may include instructions for parameters that control virtual light.

The processor 130 executes the instructions stored in the memory 120, receives an edit signal (Edit_Sig) associated with editing the image displayed on the display 160 from the external device 200, and uses the received edit signal to edit the image. An edited image can be created by combining virtual light. Additionally, the processor 130 may display the generated edited image on the display 160. For example, the processor 130 may receive an image editing signal (Edit_Sig) and generate virtual light based on the image editing signal (Edit_Sig). Virtual light does not exist in the actual captured image, but refers to light generated based on the image editing signal (Edit_Sig). For example, when a user uses the electronic device 100 to take a photo in a state of insufficient light, the processor 130 generates virtual light and creates an edited image using the photo taken in a state of sufficient light. can do. According to one embodiment, the processor 130 generates virtual light and reduces the intensity of light when the user uses the electronic device 100 to take a picture in a state where there is excessive light. By doing this, you can create an edited image from a photo taken under appropriate lighting conditions.

The processor 130 according to one embodiment may adjust the characteristics of virtual light based on the position, direction, or operation of the external device 200. For example, the processor 130 may adjust at least one of the position, direction, intensity, or area of the virtual light source. The processor 130 according to one embodiment may determine the angle between the external device 200 and the electronic device 100 as the angle of the light source for the captured image. The processor 130 according to one embodiment may determine the location of the light source based on the location of the external device 200. For example, the processor 130 may determine the location of the virtual light source in proportion to the distance between the external device 200 and the electronic device 100. The processor 130 may determine the ratio of the distance between the external device 200 and the electronic device 100 and the distance between the virtual light source and the object present in the image by user definition.

When a plurality of objects exist in the captured image, the processor 130 according to one embodiment receives a selection signal for the object from the external device 200, and virtual light is emitted based on the received selection signal. You can select an object. For example, when a first object and a second object exist in a captured image, and the external device 200 approaches the first object, the processor 130 may determine the first object as the object to which light is irradiated. there is.

The processor 130 according to an embodiment may receive a signal to change the object to which virtual light is irradiated from the external device 200, and change the object to which virtual light is irradiated based on the received change signal. For example, when the external device 200 moves away from the first object and approaches the second object, the processor 130 uses the change in distance to the object within the image as a change signal to irradiate the second object with light. It can be changed to an object. The first object and the second object according to the disclosed embodiment are referred to for convenience when a plurality of objects exist within the captured image, and when there are three or more objects in the captured image, the third object or the nth object is also referred to. A similar operation can be performed.

When an object moves in a captured image, the processor 130 according to an embodiment may reset at least one of the position, direction, intensity, or area of the virtual light source according to the movement of the object. For example, when a user captures a video using the electronic device 100, the characteristics of virtual light may change according to the movement of the object. The processor 130 according to one embodiment may adjust the characteristics of the virtual light so that when an object moves opposite to the direction in which the light is irradiated, the intensity of the virtual light gradually becomes weaker and the area to which the light is irradiated becomes narrower. Conversely, the processor 130 according to one embodiment may adjust the characteristics of the virtual light so that when the object moves in the direction in which the light is irradiated, the intensity of the virtual light becomes stronger and the area to which the light is irradiated expands. Additionally, the processor 130 according to the disclosed embodiment may adjust the characteristics of the light source to have the same effect as the light source moving from left to right when an object within the image moves from right to left.

However, the disclosed embodiments are not limited to this, and the processor 130 can adjust the characteristics of virtual light according to the movement of objects within the image and generate a natural edited image.

The processor 130 according to one embodiment may adjust the characteristics of virtual light based on the operation of the external device 130. Parameters may include, but are not limited to, intensity of light, location of the light source, direction of light, radius of light, or illuminance, and may include various parameters that can represent the characteristics of light. For example, when the user rotates the external device 200 clockwise about a predetermined axis, the processor 130 can adjust parameters for illuminance and increase the illuminance of virtual light. When the user rotates the external device 200 counterclockwise about a predetermined axis, the processor 130 can adjust parameters for illuminance and lower the illuminance of virtual light.

When the external device 200 according to an embodiment sets a border by a user's action, the processor 130 may determine the set border as the radius at which light is irradiated. For example, when a user sets a circular border on an image using the external device 200, the processor 130 may generate an edited image in which virtual light is irradiated to the set circular border. However, the radius and form in which light is irradiated are not limited to this and may exist in various forms depending on the radius set by the user using the external device 200.

The processor 130 may receive an image editing signal (Edit_Sig) for an image by communicating with the external device 200. For example, the processor 130 can perform all short-distance communications. The processor 130 includes a Bluetooth communication unit, a Bluetooth Low Energy (BLE) communication unit, a Near Field Communication unit (NFC), a WLAN (Wi-Fi) communication unit, a Zigbee communication unit, an infrared data association (IrDA) communication unit, It may include, but is not limited to, a Wi-Fi Direct (WFD) communication unit, an ultra wideband (UWB) communication unit, and an Ant+ communication unit.

Storage 140 may store characteristics or parameters of the generated light. For example, the processor 130 may convert the characteristics of virtual light generated by the operation of the external device 200 into big data and store it in the storage 140. Storage 140 may include volatile memory or non-volatile memory. By storing the characteristics of virtual light generated in various environments as big data, the electronic device 100 according to the disclosed embodiment can generate natural virtual light.

The camera 150 photographs the external environment, and the display 160 displays the captured image of the external environment on the electronic device 100. The camera 150 according to one embodiment may acquire at least one image. Additionally, the camera 150 is expressed as a singular number, but may include a plurality of cameras.

For example, the camera 150 may include a plurality of cameras having different angles of view. Here, having different angles of view may mean that the angles of view of different cameras are different by setting the angles of view differently. Alternatively, having different angles of view may mean that the ranges of angles of view of the two different cameras are different. For example, the camera 150 may include a standard camera (e.g., a camera with an angle of view ranging from 40 degrees to 60 degrees), a telephoto camera (e.g., a camera with an angle of view of 40 degrees or less), and a wide-angle camera (e.g. For example, a camera with an angle of view in the range of 60 to 80 degrees), a camera capable of optical zoom (for example, a camera whose angle of view can be adjusted within the range of 60 to 80 degrees by physically adjusting the lens of the camera) It may include at least one of a camera capable of digital zoom (e.g., a camera capable of digitally adjusting the shooting range). Additionally, the standard camera may be a camera set to 1x zoom. , may be referred to as a 'general camera'. In addition, telephoto cameras, optical cameras, cameras capable of optical zoom, and cameras capable of digital zoom may also exist in a wide variety depending on design and/or product specifications. Since the functions and detailed configuration of and digital zoom are self-evident in the image sensor field, detailed descriptions are omitted.

As another example, the camera 150 may include one camera whose angle of view can be adjusted. Here, the angle of view may correspond to the shooting range of the image acquired by the camera 150. Additionally, adjusting the angle of view may mean adjusting the focal distance of the camera 150.

The display 160 displays an image or video captured by the camera 150 under the control of the processor 130. The display 160 according to one embodiment may output an image stored in the storage 140 under the control of the processor 130. In addition, the display 160 includes a motion UI (e.g., a user motion guide for motion recognition) to perform a motion recognition task corresponding to motion recognition in order to perform a task corresponding to the operation of the external device 200. ) can be displayed. In the disclosed embodiment, the display 160 may display content used when executing an image editing application.

Figure 3 is a block diagram of the processor 130 according to one disclosed embodiment.

Referring to FIG. 3, the processor 130 includes a motion controller 131, a motion detection unit 132, an image selection unit 133, a camera controller 134, a parameter creation unit 135, or an image editing unit 136. It can be included.

The motion controller 131 may receive an image editing signal (Edit_Sig) generated based on the operation of the external device 200 and control the operation of the virtual light based on the received image editing signal (Edit_Sig). For example, the motion controller 131 may reduce or enlarge the operating radius of the external device 200 by a preset ratio and convert it into an operating radius of virtual light. According to one embodiment, the motion controller 131 may adjust at least one of the position, direction, intensity, or area of the virtual light source. The motion controller 131 according to an embodiment may determine the angle between the external device 200 and the electronic device 100 as the angle of the light source for the captured image. The motion controller 131 according to one embodiment may determine the location of the light source based on the location of the external device 200. For example, the motion controller 131 may determine the position of the virtual light source in proportion to the distance between the external device 200 and the electronic device 100. The motion controller 131 may determine the ratio of the distance between the external device 200 and the electronic device 100 and the distance between the virtual light source and the object present in the image by user definition.

The motion detection unit 132 may detect the motion of the external device 200 and generate data for generating virtual light. For example, when the external device 200 moves in the air at a certain distance from the electronic device 100, the motion detection unit 132 detects the motion of the external device 200 and generates virtual light. Create data. When the external device 200 according to one embodiment draws a circular radius in the air around the electronic device 100, the motion detection unit 132 detects the presence of a light source in the external device 200 with respect to the electronic device 100. You can decide to do this and generate virtual light with a circular radius. However, the above-described example is only one disclosed embodiment, and the motion detection unit 132 may generate image editing data corresponding to various operations of the external device 200.

The image selection unit 133 may select an object to be edited from among objects present in the image. An image according to the disclosed embodiment may include a single object or a plurality of objects. The image selection unit 133 according to the disclosed embodiment may determine the external device 200's approach to the object as a selection signal. For example, when a first object and a second object exist in a captured image, and the external device 200 approaches the first object, the image selection unit 133 selects the first object as the object to which light is irradiated. You can decide.

The image selection unit 133 according to one embodiment may receive a signal for changing the object to which virtual light is irradiated from the external device 200, and change the object to which virtual light is irradiated based on the received change signal. there is. For example, when the external device 200 moves away from the first object and approaches the second object, the image selection unit 133 uses the change in distance to the object within the image as a change signal to display the second object as light. This can be changed to the object being investigated. The first object and the second object according to the disclosed embodiment are referred to for convenience when a plurality of objects exist within the captured image, and when there are three or more objects in the captured image, the third object or the nth object is also referred to. A similar operation can be performed.

The camera controller 134 may receive image data from the camera 150 and apply parameters for image editing to the image data. For example, the camera controller 134 may receive data of an image captured by the camera 150, adjust the brightness of the image, contrast, hue, or saturation according to virtual light, and generate edited image data. . The camera controller 134 according to the disclosed embodiment can adjust brightness, contrast, hue, or saturation according to virtual light for not only still images but also moving images and generate edited image data.

The parameter generator 135 may generate parameters for image editing and adjust the parameters based on the operation of the external device 200. Parameters to be adjusted may include, but are not limited to, brightness, brightness, saturation, black and white, horizontality, or hue, and may include various effects that can be applied to the image. For example, when a user moves the external device 200 left or right, the parameter generator 135 can change the type of parameter being generated. When the user slides the external device 200 to the left, the parameter generator 135 according to one embodiment can change the parameter to be edited from a brightness adjustment parameter to a hue adjustment parameter. However, the types of parameters to be changed are not limited to this, and the user can select various parameters by moving the external device 200 in a sliding manner.

For example, when a user moves the external device 200 up and down, the parameter generator 135 can change the degree of the applied parameter. The parameter generator 135 according to one embodiment can apply many parameters when the user slides the external device 200 upward. Conversely, when the user slides the external device 200 downward, fewer parameters can be applied.

The image editing unit 136 may apply virtual light to an object selected from a captured image and generate an edited image to which various parameters are applied. For example, the image editing unit 136 may synthesize virtual light into the original image and create an edited image in which at least one parameter of brightness, brightness, saturation, black and white, horizontality, or hue is adjusted. Additionally, the image editing unit 136 may transmit data to the display 160 so that the edited image is displayed on the display 160.

Referring to FIG. 4, the electronic device 100 may receive an edit signal (Edit_Sig) associated with editing an image displayed on the display of the electronic device 100 from the external device 200 (410). For example, the electronic device 100 may receive a plurality of image editing signals (Edit_Sig) for an image from the external device 200. The electronic device 100 may receive a signal that generates a virtual light source for an image and a signal that adjusts parameters of the virtual light emitted from the generated virtual light source. Parameters according to one embodiment may be brightness, intensity, radius, saturation, luminance, brightness, or color tone for virtual light, but are not limited thereto. For example, the external device 200 according to one embodiment may adjust parameters for virtual wind rather than parameters for virtual light. When the parameters for the virtual wind are adjusted, the image editing signal (Edit_Sig) for the image can have the same effect as a natural wind blowing on an object inside the image.

In one embodiment, the electronic device 100 may use a received editing signal to generate an edited image in which virtual light is synthesized with the image (420). For example, when virtual light is generated and the characteristics of the light are adjusted, the electronic device 100 can synthesize the virtual light into the original image. In one embodiment, the electronic device 100 may synthesize virtual light to which a plurality of parameters are applied to an original image and generate an edited image. Additionally, the processor 100 according to one embodiment may synthesize the generated virtual light into an original image and store the characteristics of the virtual light. For example, the electronic device 100 can store the characteristics of virtual light and the situation in which the virtual light was generated as big data. By turning data about virtual light into big data, the electronic device 100 can create natural edited images in various environments that may occur in the future.

For example, the electronic device 100 may generate an edited image in which virtual light is synthesized using a received editing signal. In one embodiment, the electronic device 100 may generate virtual light irradiated to the image when a plurality of image editing signals (Edit_Sig) for the image are received. For example, the electronic device 100 may receive a plurality of image editing signals (Edit_Sig) generated based on the operation of the external device 200 and generate virtual light. As described above, virtual light may be generated by setting the angle between the external device 200 and the electronic device 100 as the angle between the virtual light source and the object within the image.

In one embodiment, the electronic device 100 may generate virtual light and then adjust the characteristics of the virtual light irradiated to the image. For example, the user can select a parameter to be applied to the image based on the left-right movement of the external device 200 and adjust the degree of the applied parameter based on the up-and-down movement of the external device 200. Parameters according to one embodiment may be brightness, intensity, radius, saturation, luminance, brightness, or color tone for virtual light, but are not limited thereto.

In one embodiment, the electronic device 100 may display the generated edited image on the display (430). The edited image according to the disclosed embodiment may be an image in which virtual light is reflected. When virtual light is reflected, the disclosed embodiment has the effect of naturally correcting images taken in an environment with insufficient light or an environment with excessive light based on the operation of the external device 200.

5 to 7 below describe a process in which a user selects an image and adjusts virtual light using the external device 200. Hereinafter, the case where the external device 200 is a stylus pen is described, but the external device 200 is not limited thereto and may include various external devices that can perform short-distance communication in conjunction with the electronic device 100.

Referring to FIG. 5 , when a user presses a button on a pen, the electronic device 100 according to one embodiment enters the image selection mode (510).

Upon entering the image selection mode, the electronic device 100 may determine whether the pen is further away from the image than a preset distance (520). The electronic device 100 according to one embodiment may determine whether the image and the pen are close to each other as a selection signal for the image. For example, the electronic device 100 may determine that the image and the pen are getting closer when the electronic device 100 and the pen are closer than the preset distance, and when the electronic device 100 and the pen are farther than the preset distance, the electronic device 100 may determine that the image and the pen are closer. This can be judged by the distance between the image and the pen. The preset distance according to the disclosed embodiment may be determined by user definition, and whether the pen is approaching may be determined based on a change in the speed at which the pen approaches the electronic device 100 by an acceleration sensor installed in the pen.

If the pen moves further away from the image than a preset distance, the electronic device 100 according to one embodiment may deselect the image (530). However, if it is determined that the pen is not further away from the image than a preset distance, the electronic device 100 according to one embodiment may select the image (540). For example, if it is determined that the pen and the electronic device 100 are not further apart than a preset distance, the electronic device 100 may select a new image and maintain a previously performed image selection operation.

Referring to FIG. 6 , an object to be edited may be selected from an image displayed on the electronic device 100 according to an embodiment (650).

When an object is selected in an image, the electronic device 100 may adjust the characteristics of virtual light based on the operation of the pen.

The electronic device 100 according to the disclosed embodiment may determine whether the position of the pen has changed (620). For example, the electronic device 100 may determine the location of a virtual light source based on the location of the pen. If it is determined that the position of the pen has changed, the electronic device 100 according to one embodiment may determine the position of the virtual light source based on the position of the pen and perform another operation to adjust the characteristics of the light. However, if it is determined that the position of the pen does not change, the electronic device 100 according to one embodiment can maintain the characteristics of light (670).

The electronic device 100 according to the disclosed embodiment may determine whether the direction of the pen has changed (630). For example, the electronic device 100 may determine the direction of virtual light based on the position of the pen. If it is determined that the direction of the pen has changed, the electronic device 100 according to one embodiment may determine the direction in which virtual light is irradiated to the object based on the direction of the pen and perform other operations to adjust the characteristics of the light. there is. However, if it is determined that the direction of the pen does not change, the electronic device 100 according to one embodiment can maintain the characteristics of light (670).

The electronic device 100 according to the disclosed embodiment may determine whether the speed of the pen has changed (640). For example, the electronic device 100 may determine the intensity of virtual light based on the speed of the pen. If it is determined that the speed of the pen has changed, the electronic device 100 according to one embodiment may determine the intensity of virtual light based on the speed of the pen and perform another operation to adjust the characteristics of the light. For example, when the speed of the pen increases, the electronic device 100 may set the intensity of light to be strong, and when the speed of the pen decreases, the electronic device 100 may set the intensity of light to be weak. However, if it is determined that the position of the pen does not change, the electronic device 100 according to one embodiment can maintain the characteristics of light (670).

The electronic device 100 according to the disclosed embodiment may determine whether the movement radius of the pen has changed (650). For example, the electronic device 100 may determine the radius at which the virtual light shines based on the movement radius. If it is determined that the movement radius of the pen has changed, the electronic device 100 according to one embodiment may determine the range where the virtual light shines based on the movement radius of the pen. However, if it is determined that the movement radius of the pen does not change, the electronic device 100 according to one embodiment can maintain the characteristics of light (670).

As described above, the electronic device 100 emits light when it is determined that the position of the pen has changed, the direction of the pen has changed, the speed of the pen has changed, or the movement radius of the pen has changed. Characteristics can be changed (660). However, the characteristics of light that change based on the operation of the pen are not limited to this and may further include other characteristics of light that may affect the image.

Referring to FIG. 7, the electronic device 100 according to one embodiment may detect the movement of the pen (710). For example, the electronic device 100 can determine whether the pen moves up/down, left/right, or spins.

When movement of the pen is detected, the electronic device 100 may determine whether the movement of the pen matches a preset parameter selection signal (720). For example, when the pen moves up and down, the electronic device 100 may judge it as a parameter selection signal, and when the pen moves left or right or rotates, the electronic device 100 may judge it as a parameter adjustment signal.

If the movement of the pen matches the preset parameter selection signal, the electronic device 100 can add a new parameter to the image (730). The parameters may be, but are not limited to, brightness, intensity, radius, saturation, luminance, brightness, or hue for virtual light.

However, if it is determined that the movement of the pen does not match the preset parameter selection signal, the electronic device 100 may maintain the existing parameters (740). For example, the electronic device 100 may maintain an existing state if a previously applied parameter exists, and may maintain an original image state if a previously applied parameter does not exist.

Referring to FIG. 8, the external device 200 and the virtual light source 200a according to one embodiment may perform linked operations on three-dimensional coordinates based on a preset ratio. Here, the preset ratio can be determined by user definition. For example, when a user draws a circular radius with an external device 200 in the form of a stylus pen, the virtual light source 200a is a circular shape with a diameter three times the radius drawn by the external device 200. You can draw a radius. When the virtual light source 200a is linked to the external device 200, the radius of light irradiated to the image may be linked to the radius at which the external device 200 moves.

The electronic device according to the disclosed embodiment includes a first interlocking variable (q) determined based on the first formula, a second interlocking variable (Φ) determined based on the second formula, and the second formula, as described below. The radius at which the virtual light source irradiates light can be determined based on the third interlocking variable (θ) determined by . The coordinates of the external device 200 according to the disclosed embodiment may be (x, y, z), and the coordinates of the virtual light source may be (x', y', z'). Additionally, α, β, and γ may be interlocking coefficients for the x-axis, y-axis, and z-axis, respectively. Additionally, τ may be a ratio of movements of the external device 200 and the virtual light source 200a as a result of applying interlocking coefficients for the x, y, and z axes.

first formula

2nd formula

Third formula

4th formula

The electronic device 100 according to one embodiment may determine the first interlocking variable (q) based on the first equation and the second interlocking variable (Φ) based on the second formula. The first interlocking variable (q) may be information for determining the direction of rotation based on the x-coordinate or coordinates of the external device 200. The second interlocking variable Φ may be information for determining the rotation rate of the external device 200. Once the first interlocking variable (q) and the second interlocking variable (Φ) are determined, the electronic device 100 may determine the third interlocking variable (θ) based on the third equation. The third interlocking variable θ may be a variable for the rotation angle of the external device 200. When the first interlocking variable (q), the second interlocking variable (Φ), and the third interlocking variable (θ) are determined, the electronic device 100 determines the coordinates (x`, y`) of the virtual light source based on the fourth equation. , z`) can be determined.

FIGS. 9A and 9B illustrate controlling a virtual light source 200a in the electronic device 100 according to an exemplary embodiment of the present disclosure.

9A and 9B, when the external device 200 is displayed in a spherical coordinate system based on the plane of the electronic device 100, the angle α formed by the external device 200 with the electronic device is a virtual It may be equal to the angle α formed between the virtual light generated from the light source 200a and the image. The electronic device 100 according to one embodiment may determine the position of the virtual light source 200a based on the angle with the external device 200 and determine the angle of the virtual light emitted from the virtual light source 200a. .

Referring to FIG. 10, the external device 200 includes a power source 210, a motion detection sensor 220, an acceleration detection sensor 230, a gyroscope 240, a magnetic sensor 250, a communication unit 260, and a button ( 270).

The power source 210 may supply power to the external device 200. The power source 210 can be charged in conjunction with the electronic device 100. For example, the power source 210 may be charged based on a wired/wireless charging method while electrically interconnected with the electronic device 100.

The motion detection sensor 220 can detect the motion of the external device 200. As described above, the external device 200 can perform up and down movement, left and right movement, or rotation movement by the user's manipulation, and the operation of the external device 200 can be used to generate a signal that determines the parameter. .

The acceleration sensor 230 can detect an increase or decrease in the speed of the external device 200. As described above, the speed of the external device 200 can be increased or decreased by the user's manipulation, and the acceleration of the external device 200 can be used to determine the degree of application of the parameter.

The gyroscope 240 can detect the position or rotation radius of the external device 200. As described above, the external device 200 can rotate along a predetermined radius by the user's manipulation and can be positioned at a predetermined angle with the electronic device 100. The position or rotation of the external device 200 can be used to determine the position of the virtual light source and the radius on which the virtual light shines.

The magnetic sensor 250 can detect whether the external device 200 and the electronic device 100 are approaching. For example, the magnetic sensor 250 can detect whether the external device 200 is approaching the electronic device 100, and whether the external device 200 and the device 100 are approaching generates an image selection signal. It can be a criterion for judgment.

The communication unit 260 may exchange data about the operation of the electronic device 100 and the external device 200. For example, the communication unit 260 can perform all short-distance communications. The communication unit 260 includes a Bluetooth communication unit, a Bluetooth Low Energy (BLE) communication unit, a Near Field Communication unit (NFC), a WLAN (Wi-Fi) communication unit, a Zigbee communication unit, an infrared data association (IrDA) communication unit, It may include, but is not limited to, a Wi-Fi Direct (WFD) communication unit, an ultra wideband (UWB) communication unit, and an Ant+ communication unit.

Button 270 may receive user input. For example, when the user presses the button 270 of the external device 200, the external device 200 transmits the user's input to the electronic device 100, and the electronic device 100 performs the image editing mode. can do. However, the method of receiving the user's input is not limited to this and may further include various operations of the external device 200 that can be performed by user definition.

In order to perform the above-described operation, the electronic device 100 for editing an image displayed on the display 160 according to the disclosed embodiment includes a memory 120 including one or more instructions and one or more processors 130, and one The processor 130 may execute one or more instructions.

One or more processors 130 according to an embodiment may adjust virtual light based on the location or direction of the external device 200.

The electronic device 100 according to an embodiment further includes a storage 140 that stores the characteristics of virtual light in big data, and one or more processors 130 according to an embodiment monitor the movement of the external device 200. can be detected, and the characteristics of the virtual light generated based on the movement of the external device 200 can be stored in the storage 140.

One or more processors 130 according to an embodiment may receive a selection signal for one of the objects present in the image and determine an object to which virtual light is irradiated based on the received selection signal.

One or more processors 130 according to an embodiment may adjust at least one of the position, direction, intensity, or area of a light source of virtual light based on the movement of the external device 200.

If it is determined that an object present in the image is moving, the one or more processors 130 according to one embodiment may reset at least one of the position, direction, intensity, or area of the virtual light source based on the movement of the object.

When there are multiple objects present in the image, one or more processors 130 according to an embodiment receive a change signal for one of the objects and change the object to which virtual light is irradiated based on the received change signal. You can.

One or more processors 130 according to an embodiment may adjust parameters that adjust the characteristics of virtual light based on the movement of the external device 200.

The external device 200 according to one embodiment includes a motion detection sensor 220, an acceleration detection sensor 230, or a gyroscope 240, and includes a motion detection sensor 220, an acceleration detection sensor 230, or a gyroscope 240. It may include a communication unit 260 that transmits a plurality of editing signals for an image generated based on a detection result of at least one of the gyroscopes 240 to the electronic device 100.

Generating virtual light according to an embodiment may include adjusting virtual light based on the location or direction of an external device.

The method of operating the electronic device 100 according to an embodiment further includes the step of converting and storing the characteristics of virtual light into big data, and the step of generating virtual light according to an embodiment includes detecting the movement of an external device, , It may include storing the characteristics of virtual light generated based on the movement of an external device in storage.

The step of generating virtual light according to an embodiment may include receiving a selection signal for one of the objects present in the image and determining the object to which the virtual light is irradiated based on the received selection signal. there is.

Generating virtual light according to an embodiment may include adjusting at least one of the position, direction, intensity, or area of the virtual light source based on the movement of the external device.

The step of generating virtual light according to an embodiment may include resetting at least one of the position, direction, intensity, or area of the virtual light source based on the movement of the object when it is determined that the object present in the image is moving. You can.

When there are a plurality of objects present in an image according to the disclosed embodiment, the step of generating virtual light according to an embodiment includes receiving a change signal for one of the objects, and generating virtual light based on the received change signal. This may involve changing the object being examined.

Generating virtual light according to an embodiment may include adjusting parameters that adjust the characteristics of virtual light based on the movement of the external device 200.

Receiving an editing signal according to an embodiment may include receiving an editing signal generated based on at least one of operation information, acceleration information, and rotation information of the external device 200.

A method of operating the electronic device 100 according to an embodiment may be stored in a computer-readable recording medium on which a program for execution by a computer is recorded.

A storage medium that can be read by a device may be provided in the form of a non-transitory storage medium. Here, 'non-transitory storage medium' only means that it is a tangible device and does not contain signals (e.g. electromagnetic waves). This term refers to cases where data is semi-permanently stored in a storage medium and temporary storage media. It does not distinguish between cases where it is stored as . For example, a 'non-transitory storage medium' may include a buffer where data is temporarily stored.

According to one embodiment, methods according to various embodiments disclosed in this document may be provided and included in a computer program product. Computer program products are commodities and can be traded between sellers and buyers. A computer program product may be distributed in the form of a machine-readable storage medium (e.g. compact disc read only memory (CD-ROM)) or through an application store or between two user devices (e.g. smartphones). It may be distributed in person or online (e.g., downloaded or uploaded). In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) is stored on a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server. It can be temporarily stored or created temporarily.

Claims

In the electronic device 100 for editing an image displayed on the display 160,

a memory 120 containing one or more instructions; and

Comprising one or more processors 130, wherein the one or more processors 130 execute the one or more instructions,

Receiving an editing signal associated with editing an image displayed on the display 160 from an external device 200,

Generating an edited image in which virtual light is synthesized with the image using the received editing signal,

An electronic device that displays the generated edited image on the display (160).
According to paragraph 1,

The one or more processors 130,

An electronic device that adjusts the virtual light based on at least one of the position or direction of the external device (200).
According to claim 1 or 2,

It further includes a storage 140 that converts the characteristics of the virtual light into big data and stores it,

The one or more processors 130,

An electronic device that detects the movement of the external device (200) and stores the characteristics of virtual light generated based on the movement of the external device (200) in the storage (140).
According to at least one of claims 1 to 3,

The one or more processors 130,

An electronic device that receives a selection signal for one of the objects present in the image and determines an object to which the virtual light is irradiated based on the received selection signal.
According to at least one of claims 1 to 4,

The one or more processors 130

An electronic device that adjusts at least one of the position, direction, intensity, or area of the virtual light source based on the movement of the external device (200).
According to at least one of claims 1 to 5,

The one or more processors 130

An electronic device that, when it is determined that an object in the image is moving, resets at least one of the position, direction, intensity, or area of the virtual light source based on the movement of the object.
According to at least one of claims 1 to 6,

If there are multiple objects present in the image,

The one or more processors 130,

An electronic device that receives a change signal for one of the objects and changes the object to which the virtual light is irradiated based on the received change signal.
According to at least one of claims 1 to 7,

The one or more processors 130,

An electronic device that adjusts parameters for controlling characteristics of the virtual light based on the movement of the external device (200).
According to at least one of claims 1 to 8,

The external device 200,

Includes a motion detection sensor 220, an acceleration detection sensor 230, or a gyroscope 240,

Transmitting a plurality of editing signals for the image generated based on the detection result of at least one of the motion detection sensor 220, the acceleration detection sensor 230, or the gyroscope 240 to the electronic device 100. An electronic device including a communication unit (260).
In a method of operating the electronic device 100 for editing an image displayed on the display 160,

Receiving an editing signal associated with editing an image displayed on the display from an external device (410);

Generating an edited image in which virtual light is synthesized with the image using the received editing signal (440); and

A method of operating the electronic device 100, including displaying the generated edited image on the display (450).
According to claim 10,

The step 440 of generating the edited image is,

A method of operating the electronic device 100, including adjusting the virtual light based on at least one of the position or direction of the external device.
According to claim 11,

It further includes converting the characteristics of the virtual light into big data and storing it,

The step 440 of generating the edited image is,

A method of operating an electronic device (100) including detecting movement of the external device (200) and storing characteristics of virtual light generated based on the movement of the external device in the storage (140).
The method of at least one of claims 10 to 12,

The step 440 of generating the edited image is,

A method of operating the electronic device 100, comprising receiving a selection signal for one of the objects present in the image and determining an object to which the virtual light is irradiated based on the received selection signal.
A computer-readable recording medium on which a program for performing the methods of claims 10 to 13 is recorded on a computer.
An external device 200 that generates a plurality of editing signals for an image; and

In an image editing system (10) including an electronic device (100) that receives the plurality of editing signals and edits an image displayed on a display (160),

The electronic device 100,

an image generator 110 that generates an edited image by combining virtual light irradiated on the image with the image;

a memory 120 containing one or more instructions; and

Comprising one or more processors 130, wherein the one or more processors 130 execute the one or more instructions based on the image editing signals,

Receiving an editing signal associated with editing an image displayed on the display 160 from the external device 200,

Generating an edited image in which virtual light is synthesized with the image using the received editing signal,

An image editing system that displays the generated edited image on the display (160).