KR102380574B1 - Alternative method for new image using mosaic processing - Google Patents

Abstract

One embodiment relates to an alternative method for a new image using mosaic processing. According to the present invention, an alternative method comprises the steps of: acquiring an image; displaying a notification indicating that the mosaic is processed on an image; and setting a first value of the characteristic of the mosaic. The purpose of the present invention to provide an electronic device and an operating method for improving availability for mosaic processing.

Description

Alternative method for new image using mosaic processing

The present application relates to a replacement method for a new image using mosaic processing.

Recently, as various services are provided, the use of 　video and content is increasing. Video content produced through multiple sources contains sensitive private information due to indiscriminate collection and exposure of personal information, and video content is indiscriminately shared without deliberation.

As such, it is necessary to apply and distribute a mosaic of video and content containing private information. However, since there are so many video and content, it is not easy for an administrator to play back video and content for all video and content to determine the area to process mosaics. In addition, it takes a lot of processing time because it is necessary to determine the area where the mosaic should be processed by analyzing numerous video and content.

Therefore, a method that can apply the 　 mosaic more accurately and quickly is required.

According to an embodiment of the present application, one task is to automatically recognize a plurality of objects to be mosaiced included in images uploaded to a server and process the mosaic, thereby improving the usability for mosaic processing and the operation It's about providing a way.

According to an embodiment of the present application, another object is to use big data accumulated in a server to automatically process mosaics on objects to which a mosaic is to be applied to a specific image, thereby improving the usability for mosaic processing, and It is to provide a method of operation.

According to an embodiment of the present application, another task is to automatically edit the corresponding image when there is a large amount of mosaic in a specific time period of the edited image during mosaic processing, so that the quality of the image is not deteriorated due to the mosaic processing. An object of the present invention is to provide an apparatus and a method of operation.

The problem to be solved by the present application is not limited to the above-mentioned problems, and the problems not mentioned will be clearly understood by those of ordinary skill in the art to which the present application belongs from the present specification and the accompanying drawings. .

According to one embodiment, there is provided a method of operating a server, comprising: providing an interface for uploading an image and editing an image to the electronic device when the electronic device is connected to the server, wherein the interface plays the image acquiring the uploaded image when an image photographed by the electronic device is uploaded on the interface, the method comprising: When a menu for processing the image mosaic is selected from among a plurality of menus on the interface, a plurality of objects included in the image are identified, and a type selected by the electronic device from among the plurality of identified objects is selected. identifying 1 objects; wherein the selected type includes at least one of a sign, a signboard, a license plate of a vehicle, or a human face, mosaicing first areas of the identified first objects, and the displaying a notification indicating that the mosaic is processed on an image displayed on an interface; and when an object for storing the image including the mosaic-processed first regions is selected on the interface, storing the image; and, according to a request received from the electronic device, the stored image A method of operation, which is registered with this video sharing server, may be provided.

According to one embodiment, there is provided a server comprising: communication circuitry; and at least one processor, wherein the at least one processor provides an interface for uploading an image and editing an image to the electronic device when the electronic device is connected to a server, wherein the interface plays the image and a plurality of menus for editing an image, and when an image photographed by the electronic device is uploaded on the interface, the uploaded image is acquired, and the image is selected from among a plurality of menus on the interface. When a menu for processing the mosaic of an image is selected, a plurality of objects included in the image are identified, first objects of a type selected by the electronic device among the identified plurality of objects are identified, and the selected type includes at least one of a sign, a signboard, a license plate of a vehicle, or a human face, and mosaics the first areas of the identified first objects, and a notification indicating that the mosaic is processed on an image displayed on the interface and when an object for storing the image including the mosaic-processed first regions is selected on the interface, the image is set to be stored, and the stored image is set according to a request received from the electronic device A server, which is registered as this video sharing server, may be provided.

Solutions of the problems are not limited to the above-described solutions, and solutions not mentioned will be clearly understood by those of ordinary skill in the art to which the present application belongs from the present specification and the accompanying drawings. .

According to an embodiment of the present application, an electronic device and an operating method therefor improve availability for mosaic processing by automatically recognizing and mosaicing a plurality of objects to be mosaiced included in images uploaded to a server. can

According to an embodiment of the present application, an electronic device and an operating method thereof use big data accumulated in a server to automatically mosaic objects to which a mosaic is to be applied to a specific image, thereby improving the usability for mosaic processing can do it

According to an embodiment of the present application, an electronic device and an operating method thereof automatically edit the corresponding image when the amount of mosaic in a specific time period of the edited image is large during mosaic processing, so that the quality of the image is not deteriorated due to the mosaic processing. can prevent it

Effects are not limited to the above-described effects, and effects not mentioned will be clearly understood by those of ordinary skill in the art to which the present application belongs from the present specification and the accompanying drawings.

1 is a view for explaining an example of devices included in a smart image editing system according to an embodiment of the present application.
2 is a block diagram illustrating an example of a configuration of a management server according to an embodiment of the present application.
3 is a block diagram illustrating an example of the configuration of an electronic device according to an embodiment of the present application.
4 is a flowchart illustrating an example of an operation of a smart image editing system according to an embodiment of the present application.
5 is a diagram for explaining an example of operations of a management server and an electronic device according to an embodiment of the present application.
6 is a diagram for explaining an example of an operation of editing an image through an interface provided to an electronic device of a management server according to an embodiment of the present application.
7 is a diagram for explaining an example of an operation of obtaining information on a type of sensitive information to be processed through an interface provided to an electronic device of a management server according to an embodiment of the present application.
FIG. 8 is a diagram for explaining an example of an operation of adjusting a characteristic of a mosaic through an interface provided to an electronic device of a management server according to an embodiment of the present application.
9 is a flowchart for explaining another example of the operation of the smart image editing system according to an embodiment of the present application.
10 is a diagram for explaining an example of an operation of learning an artificial intelligence model based on accumulated information of a smart image editing system according to an embodiment of the present application.
11 is a view for explaining an example of the operation of mosaic processing a specific image based on the artificial intelligence model of the smart image editing system according to an embodiment of the present application.
12 is a diagram for explaining an example of the operation of mosaic processing a specific image on the interface of the smart image editing system according to an embodiment of the present application.
13 is a flowchart for explaining another example of the operation of the smart image editing system according to an embodiment of the present application.
14 is a diagram for explaining an example of an operation of editing an image of a specific time period by identifying a mosaic area in an image of a management server according to an embodiment of the present application.

According to one embodiment, there is provided a method of operating a server, comprising: providing an interface for uploading an image and editing an image to the electronic device when the electronic device is connected to the server, wherein the interface plays the image acquiring the uploaded image when an image photographed by the electronic device is uploaded on the interface, the method comprising: When a menu for processing the image mosaic is selected from among a plurality of menus on the interface, a plurality of objects included in the image are identified, and a type selected by the electronic device from among the plurality of identified objects is selected. identifying 1 objects; wherein the selected type includes at least one of a sign, a signboard, a license plate of a vehicle, or a human face, mosaicing first areas of the identified first objects, and the displaying a notification indicating that the mosaic is processed on an image displayed on an interface; and when an object for storing the image including the mosaic-processed first regions is selected on the interface, storing the image; and, according to a request received from the electronic device, the stored image A method of operation, which is registered with this video sharing server, may be provided.

According to an embodiment, when the notification indicating that the mosaic is selected is selected, displaying at least one object for adjusting the value of the characteristic of the mosaic at a position associated with the mosaic-processed first areas; Including, wherein the characteristic of the mosaic includes a degree of the mosaic and a pattern type of the mosaic, and based on the at least one object, setting a first value of the characteristic of the mosaic; A method of operation may be provided.

According to an embodiment, the method includes: obtaining information about a plurality of images stored in the server and information related to a mosaic corresponding to the plurality of images, each of the information on the plurality of images includes: , the type of image, the length of time of the image, information on the types of objects included in the image, or information on the image sharing server to which the image is uploaded, each of the information related to the mosaic is the image includes information on the types of objects for which a mosaic is set, or information on the characteristics of the mosaic set in the image, using at least one of information on the plurality of images as input data, and using information related to the mosaic as output data to generate an artificial intelligence model; including, wherein the artificial intelligence model responds to receiving information on a specific image, weights for the types of specific objects for which a mosaic is to be set in the specific image, or outputting weights for the values of the characteristic of the mosaic, and receiving a message for a request for setting a mosaic of a first image from the electronic device; In response to inputting information about the first image into the artificial intelligence model, in the first image, the types of second objects for which a mosaic is to be set and first weights for the values of the characteristic of the mosaic are identified. step; and identifying the second objects from among a plurality of objects included in the first image based on the first weights, and mosaicing and providing second regions of the second objects. A method of operation may be provided.

According to an embodiment, at least one outline is recognized based on a value for each pixel for each of the plurality of frame images constituting the first image, and the recognized at least one outline is pre-stored in the server. Identifying the second objects based on the comparison with the information on the outline for each type; further comprising, an operating method may be provided.

According to one embodiment, there is provided a server comprising: communication circuitry; and at least one processor, wherein the at least one processor provides an interface for uploading an image and editing an image to the electronic device when the electronic device is connected to a server, wherein the interface plays the image and a plurality of menus for editing an image, and when an image photographed by the electronic device is uploaded on the interface, the uploaded image is acquired, and the image is selected from among a plurality of menus on the interface. When a menu for processing the mosaic of an image is selected, a plurality of objects included in the image are identified, first objects of a type selected by the electronic device among the identified plurality of objects are identified, and the selected type includes at least one of a sign, a signboard, a license plate of a vehicle, or a human face, and mosaics the first areas of the identified first objects, and a notification indicating that the mosaic is processed on an image displayed on the interface and when an object for storing the image including the mosaic-processed first regions is selected on the interface, the image is set to be stored, and the stored image is set according to a request received from the electronic device A server, which is registered as this video sharing server, may be provided.

According to various embodiments to those of ordinary skill in the art to which the present application belongs, the embodiments described in this specification are for clearly explaining the idea, so that the present application is limited by the embodiments described in this specification No, according to various embodiments, the range should be construed to include modifications or variations without departing from the spirit according to various embodiments.

The terms used in this specification have been selected as widely used general terms as possible in consideration of the functions in the present application, but they may vary depending on the intention, custom, or emergence of new technology of those of ordinary skill in the art to which this application belongs. can However, if a specific term is defined and used in an arbitrary sense, the meaning of the term will be separately described. Therefore, the terms used in this specification should be interpreted based on the actual meaning of the terms and the contents of the entire specification, rather than the names of simple terms.

The drawings attached to this specification are for easy explanation of the present application, and according to various embodiments, the shapes shown in the drawings may be exaggerated as needed to help understanding, so that the present application is limited by the drawings. it is not

In the present specification, when it is determined that a detailed description of a known configuration or function related to the present application may obscure the subject matter according to various embodiments, a detailed description thereof will be omitted if necessary.

The various embodiments of this document and terms used therein are not intended to limit the technical features described in this document to specific embodiments, but it should be understood to include various modifications, equivalents, or substitutions of the embodiments. In connection with the description of the drawings, like reference numerals may be used for similar or related components. The singular form of the noun corresponding to the item may include one or more of the item, unless the relevant context clearly dictates otherwise. In this document, "A or B", "at least one of A and B", "at least one of A or B," "A, B or C," "at least one of A, B and C," and "A , B, or C" each may include any one of the items listed together in the corresponding one of the phrases, or all possible combinations thereof. Terms such as "first", "second", or "first" or "second" may be used simply to distinguish the element from other elements in question, and may refer to elements in other aspects (e.g., importance or order) is not limited. It is said that one (eg, first) component is "coupled" or "connected" to another (eg, second) component, with or without the terms "functionally" or "communicatively". When referenced, it means that one component can be connected to the other component directly (eg by wire), wirelessly, or through a third component.

As used herein, the term “module” may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as, for example, logic, logic block, component, or circuit. A module may be an integrally formed part or a minimum unit or a part of the part that performs one or more functions. For example, according to an embodiment, the module may be implemented in the form of an application-specific integrated circuit (ASIC).

According to various embodiments of the present document, one or more instructions stored in a storage medium (eg, internal memory 136 or external memory 138) readable by a machine (eg, electronic device 101) may be implemented as software (eg, the program 140) including For example, the processor (eg, the processor 320 ) of the device (eg, the electronic device 101 ) may call at least one command among one or more commands stored from a storage medium and execute it. This makes it possible for the device to be operated to perform at least one function according to the called at least one command. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' only means that the storage medium is a tangible device and does not contain a signal (eg, electromagnetic wave), and this term is used in cases where data is semi-permanently stored in the storage medium and It does not distinguish between temporary storage cases.

According to one embodiment, the method according to various embodiments disclosed in this document may be provided as included in a computer program product. Computer program products may be traded between sellers and buyers as commodities. The computer program product is distributed in the form of a device-readable storage medium (eg compact disc read only memory (CD-ROM)), or through an application store (eg Play Store™) or on two user devices (eg, It can be distributed (eg downloaded or uploaded) directly or online between smartphones (eg: smartphones). In the case of online distribution, at least a part of the computer program product may be temporarily stored or temporarily created in a machine-readable storage medium such as a memory of a server of a manufacturer, a server of an application store, or a relay server.

According to various embodiments, each component (eg, a module or a program) of the above-described components may include a singular or a plurality of entities. According to various embodiments, one or more components or operations among the above-described corresponding components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (eg, a module or a program) may be integrated into one component. In this case, the integrated component may perform one or more functions of each component of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. . According to various embodiments, operations performed by a module, program, or other component are executed sequentially, in parallel, repeatedly, or heuristically, or one or more of the operations are executed in a different order, or omitted. or one or more other operations may be added.

1. Smart video editing system

Hereinafter, a smart image editing system according to an embodiment of the present application will be described.

In this specification, a smart image editing system may be defined as a system that acquires (or stores) a captured image and automatically recognizes and edits sensitive information in the acquired image. The smart image editing system may automatically extract an object corresponding to sensitive information in a photographed image and process a mosaic on the extracted object. In this case, when the object is moved in the image, the smart image editing system may perform an operation of processing the mosaic by tracking the position of the moving object. The smart video editing system may obtain an edited video with sensitive information in a mosaic, and upload the edited video to a server such as a video sharing server (eg, youtube) according to a user's request.

Hereinafter, the smart image editing system will be described in more detail.

2. Configuration of smart video editing system

1 is a diagram for explaining an example of devices included in a smart image editing system according to an embodiment of the present application.

Referring to FIG. 1 , a smart image editing system may include a management server 110 , a video sharing server 120 , and an electronic device 130 used by users. However, the present invention is not limited to that shown in FIG. 1 , and the smart video editing system may be implemented to include fewer or more devices than the illustrated devices.

According to an embodiment of the present application, the management server 110 includes images 150 of various types (eg, documentaries, interview images, dramas, movies, etc.) captured from the user's electronic devices 130 . ), and by editing sensitive information (eg, a sign, a vehicle license plate, a human face, etc.) included in the acquired images 150 , the edited image 160 may be accumulated. For example, the management server 110 provides an interface for editing the image 150 captured by the electronic device 130 connected to the management server 110 , and the image 150 captured on the provided interface is displayed. When uploaded, a function for editing the image 150 may be provided on the interface. In this case, the management server 110 may provide (eg, download, or distribute) a separate application (or program) for accessing the management server 110 to the electronic device 130 , and the electronic device 130 . may access the management server 110 based on the execution of the separate application.

According to an embodiment of the present application, the video sharing server 120 may be a kind of platform server for sharing an uploaded image (eg, an edited image 160). For example, the video sharing server may be a video sharing platform server such as a YouTube server. The video sharing server 120 uploads an image (eg, an edited image 160 ) according to a request from the management server 110 or the electronic device 130 , and allows other users using the video sharing server 120 to view the image. (eg, the edited image 160) may be made viewable.

According to an embodiment of the present application, the electronic device 130 may include various types of electronic devices available to a user. The electronic device 130 may include personal terminals (eg, a smart phone, etc.), electronic devices (eg, a PC, etc.) provided at a fixed location, and a portable personal notebook computer. For example, as shown in FIG. 1 , the electronic device 130 acquires the image 150 by performing photographing using a camera, and is included in the image 150 acquired through the management server 110 . The edited image 160 may be acquired by editing sensitive information (eg, a sign, a license plate of a vehicle, a human face, etc.).

2.1. An example of configurations of a smart video editing system

Hereinafter, an example of components for performing operations of components included in the smart image editing system will be described. Meanwhile, since the smart image editing system may be implemented as a system type or an on-device type depending on the implementation purpose, this will be described later in "2.2 Table of Contents".

2.1.1 An example of the configuration of the management server 110

First, an example of the configuration of the management server 110 will be described.

2 is a block diagram illustrating an example of the configuration of the management server 110 according to an embodiment of the present application. On the other hand, not limited to the bar shown in FIG. 2 , the management server 110 may include fewer or more components than the illustrated components.

Referring to FIG. 2 , according to an embodiment, the management server 110 includes a first communication circuit 111 , an image acquisition module 113 , an image editing module 114 , and a recommendation module 115 . a control circuit 112 , and a first database 116 .

The first communication circuit 111 may communicate with an external device (eg, the image sharing server 120 or the electronic device 130 ). For example, the first communication circuit 111 is connected to a network through wireless communication or wired communication to establish communication with an external device (eg, the image sharing server 120 , the electronic device 130 ) to perform the set communication. information and/or data may be exchanged through The wireless communication is, for example, LTE, LTE Advance (LTE-A), CDMA (code division multiple access), WCDMA (wideband CDMA), UMTS (universal mobile telecommunications system), WiBro (Wireless Broadband), or GSM ( Global System for Mobile Communications) and the like may include cellular communication using at least one. According to one embodiment, wireless communication is, for example, wireless fidelity (WiFi), Bluetooth, Bluetooth low energy (BLE), Zigbee (Zigbee), near field communication (NFC), magnetic secure transmission (Magnetic Secure Transmission), radio It may include at least one of a frequency (RF) or a body area network (BAN). According to one embodiment, the wireless communication may include GNSS. The GNSS may be, for example, a Global Positioning System (GPS), a Global Navigation Satellite System (Glonass), a Beidou Navigation Satellite System (hereinafter, “Beidou”) or Galileo, the European global satellite-based navigation system. Hereinafter, in this document, "GPS" may be used interchangeably with "GNSS". Wired communication may include, for example, at least one of universal serial bus (USB), high definition multimedia interface (HDMI), recommended standard232 (RS-232), power line communication, or plain old telephone service (POTS). there is. The network 162 may include at least one of a telecommunication network, for example, a computer network (eg, LAN or WAN), the Internet, or a telephone network.

The first control circuit 112 may control the overall operation of the management server 110 . To this end, the first control circuit 112 may perform calculation and processing of various types of information and may control the operation of components (eg, the first communication circuit 111 ) of the management server 110 . The first control circuit 112 may be implemented in a computer or similar device according to hardware, software, or a combination thereof. In terms of hardware, the first control circuit 112 may be provided in the form of an electronic circuit (eg, CPU, etc.) that processes electrical signals to perform a control function, and in terms of software, the first control circuit 112 in hardware 112) may be provided in the form of a program for driving. The modules (eg, the image acquisition module 113 , the image editing module 114 , and the recommendation module 115 ) included in the first control circuit 112 of the management server 110 described below are the first The control circuit 112 may control to perform an operation associated with the module. In other words, the modules are implemented with a program, computer readable code, process or instructions for controlling to perform an operation associated with the module, wherein the modules are executed by the first control circuit 112 . In this case, it is possible to control the first control circuit 112 to perform an operation related to the module. Meanwhile, in the following description, unless otherwise specified, the operation of the management server 110 may be interpreted as being performed under the control of the first control circuit 112 .

The first control circuit 112 includes an image acquisition module 113 , and the image acquisition module 113 may acquire an image to be edited from the electronic device 130 . For example, the image acquisition module 113 provides an interface for uploading an image to the electronic device 130 connected to the management server 110 , and receives the image uploaded by the electronic device 130 through the provided interface. can be obtained As will be described later, the electronic device 130 may access the management server 110 based on the execution of the application, and may receive an interface for uploading the above-described image from the management server 110 .

The first control circuit 112 includes an image editing module 114 , and the image editing module 114 may edit an image provided from the electronic device 130 through an interface. For example, the image editing module 114 may extract objects included in the image, and if the extracted objects correspond to sensitive information, mosaic processing the objects. For example, the objects corresponding to the sensitive information may be objects representing private information such as a sign, a vehicle license plate, and a human face, but it is not limited to the description and various sensitive information may be set according to the user's settings. there is. The image editing module 113 identifies an outline and/or appearance of each object based on a pixel-specific value for each of a plurality of image frames constituting an image, and the identified outline and/or appearance is pre-stored as sensitive information. When it corresponds to the set outline and/or appearance, the object may be determined as sensitive information. In addition, as at least part of the operation of processing the mosaic, the image editing module 113 manipulates the values of pixels constituting the object identified as sensitive information to be invisible (eg, reduces the area of the object at a certain rate). and then enlarging and changing the pixel value), but the present invention is not limited thereto, and the mosaic processing operation may be implemented as well-known techniques.

The first control circuit 112 includes a recommendation module 115 , the recommendation module 115 stores and accumulates images collected from the electronic device 130 and information related to mosaic setting, and uses the accumulated information. Based on that, when a mosaic editing request for a specific image is received by the electronic device 130 thereafter, a mosaic processing operation for the specific image may be automatically performed. This will be described in more detail later in "Second Embodiment".

The first database 116 may store various types of information (eg, information related to collected images and mosaic setting, and/or an artificial intelligence model to be described later). The first database 116 may temporarily or semi-permanently store data. For example, in the first database 116 of the management server 110 , an operating program (OS) for driving the management server 110, data or programs or applications (eg, for hosting a web site) , web application) and the like may be stored. Examples of the first database 116 include a hard disk (HDD), a solid state drive (SSD), a flash memory, a read-only memory (ROM), and a random access (RAM). memory), etc. Such a database may be provided in a built-in type or a detachable type.

2.1.2 An example of the configuration of the electronic device 130

Hereinafter, an example of the configuration of the electronic device 130 will be described.

3 is a block diagram illustrating an example of the configuration of the electronic device 130 according to an embodiment of the present application. Meanwhile, without being limited to that shown in FIG. 3 , the electronic device 130 may include fewer or more components than the illustrated components.

First, configurations of the electronic device 130 will be described below.

Referring to FIG. 3 , according to an embodiment, the electronic device 140 includes a second communication circuit 131 , a second control circuit 132 , an input device 133 , a display 134 , and a photographing device 135 . may include

The second communication circuit 131 may communicate with an external device (eg, the management server 110 and the video sharing server 120 ). For example, the second communication circuit 131 is connected to a network through wireless communication or wired communication to establish communication with an external device (eg, the management server 110 and the video sharing server 120 ), thereby performing the established communication. information and/or data may be exchanged through Since the second communication circuit 131 may be implemented as the first communication circuit 111 of the management server 110 described above, a redundant description will be omitted.

The second control circuit 132 may control the overall operation of the electronic device 130 . To this end, the second control circuit 132 may perform calculation and processing of various types of information and may control operations of components (eg, the second communication circuit 131 ) of the electronic device 130 . Since the second control circuit 132 may be implemented as the first control circuit 112 of the management server 110 described above, a redundant description will be omitted.

The input device 133 may receive information from users (eg, receive user input (eg, hand writing, drawing, various types of touch, text typing, etc.) on an interface). The input device 143 may be various interfaces or connection ports that receive a user input or output information to the user. The input device 143 may be divided into an input module and an output module, and the input module receives a user input from a user. The user input may be made in various forms including a key input, a touch input, and a voice input. Examples of input modules that can receive such user input include a traditional keypad, keyboard, and mouse, as well as a touch sensor that detects a user's touch, a microphone that receives a voice signal, a camera that recognizes gestures through image recognition, A proximity sensor composed of an illuminance sensor or infrared sensor that detects a user's approach, a motion sensor that recognizes a user's motion through an acceleration sensor or a gyro sensor, and other various types of input means for sensing or receiving various types of user input It is a comprehensive concept that includes all Here, the touch sensor may be implemented as a piezoelectric or capacitive touch sensor for detecting a touch through a touch panel or a touch film attached to the display panel, an optical touch sensor for detecting a touch by an optical method, and the like. In this case, the input device 133 may be implemented in a display 134 to be described later to receive a user's touch input. In other words, the display 134 may include a touch screen as the input device 143 , and may receive, for example, a touch, gesture, proximity, or hovering input using an electronic pen or a part of the user's body. can In addition, the input device 133 may be implemented in the form of an input interface (USB port, PS/2 port, etc.) for connecting an external input device that receives a user input instead of a device that detects a user input by itself.

The display 134 may include a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, or a microelectromechanical system (MEMS) display, or an electronic paper display. . The display 134 may display an interface provided from the management server 110 .

The photographing device 135 may photograph an image and/or an image. The photographing device 250 includes a plurality of image sensors composed of a charge-coupled device (CCD) or a complementary metal oxide semiconductor (CMOS), receives light during photographing, and converts the received light amount into electrical information for each pixel to perform photographing can do.

2.3. Implementation example of smart video editing system

The smart video editing system may be implemented as a system type or an on-device type as described above.

According to an embodiment, as described above, when the management server 110 receives predetermined information from the client server 120 and performs an operation to provide, the smart video editing system may be defined as a system type. there is.

According to another embodiment, when the above-described components are implemented in one physical device, the smart image editing system may be defined as an on-device type. For example, the components of the management server 110 (eg, the image acquisition module 113 , the image editing module 114 , and the recommendation module 115 ) may all be implemented in the electronic device 130 , in this case The smart image editing system may be defined as an on-device type. In this case, an application (or program) implemented to provide an interface for image upload and image editing from the management server 110 to the electronic device 130 is provided, and the electronic device 130 provides an application (or program) of the application (or program). Depending on execution, functions for the above-described components (eg, the image acquisition module 113 , the image editing module 114 , and the recommendation module 115 ) may be provided.

Also, without being limited thereto, the smart video editing system may be implemented as a hybrid type in which a system type and an on-device type are combined as shown. For example, when at least some components of the management server 110 are maintained in the management server 110 and the remaining components are implemented in the electronic device 130 , the corresponding smart image editing system may be defined as a hybrid type.

3. Operation of Smart Video Editing System

Hereinafter, various examples of operations of devices constituting the smart image editing system (eg, the management server 110 , the image sharing server 120 , and the electronic device 130 ) will be described.

3.1. 1st embodiment <Example of mosaic processing of sensitive information in an image>

According to an embodiment, the management server 110 identifies at least one object corresponding to sensitive information included in an image uploaded from the electronic device 130 through an interface to be described later, and mosaics the identified at least one object. can be processed In this case, the management server 110 may receive a type of sensitive information to be mosaicked from the electronic device 130 (or a user), and may mosaic-process an object corresponding to the input type. In addition, the management server 110 may provide an editing function of the characteristics of the mosaic (eg, mosaic degree, mosaic pattern type).

4 is a flowchart illustrating an example of an operation of a smart image editing system according to an embodiment of the present application. According to various embodiments, the operation of the smart image editing system is not limited to the order of the operations illustrated in FIG. 4 , and may be performed in an order different from the illustrated order. In addition, according to various embodiments, more operations or fewer operations than those of the smart image editing system shown in FIG. 4 may be performed, or at least one operation may be performed. Hereinafter, FIG. 4 will be described with reference to FIGS. 5 to 6 .

5 is a diagram for explaining an example of operations of the management server 110 and the electronic device 130 according to an embodiment of the present application. 6 is a diagram for explaining an example of an operation of editing an image through an interface provided to the electronic device 130 of the management server 110 according to an embodiment of the present application. 7 is a diagram for explaining an example of an operation of acquiring information on the type of sensitive information to be processed through an interface provided to the electronic device 130 of the management server 110 according to an embodiment of the present application . FIG. 8 is a diagram for explaining an example of an operation of adjusting a characteristic of a mosaic through an interface provided to the electronic device 130 of the management server 110 according to an embodiment of the present application.

According to an embodiment, the electronic device 130 may capture an image in operation 401 and transmit the captured image to the management server 110 in operation 402 . For example, the electronic device 130 may perform photographing using the photographing device 135 as illustrated in 501 of FIG. 5 . The photographed subject may include various types of objects such as vehicles, people, and buildings. The electronic device 130 may transmit the captured image to the management server 110 through the interface as shown in 502 of FIG. 5 . For example, the electronic device 130 executes an application (or program) for uploading an image and editing an image distributed by the management server 110 , and as shown in 601 of FIG. 6 , user authentication (eg, ID) /Password input) to connect to the management server 110 . The management server 110 may provide an interface for uploading an image and editing an image to the connected electronic device 130 as shown in 602 of FIG. 6 . The interface is an execution screen produced by the management server 110 and may be an execution screen implemented to provide an image upload function and an image editing function. The electronic device 130 may upload the captured image 621 on the interface (eg, drag & drop, or attach an image), and the management server 110 may receive the image uploaded by the electronic device 130 . there is. On the other hand, without being limited to the description and/or shown, a web site for providing the interface is implemented instead of an application, and the above-described interface for uploading an image is provided based on the electronic device 130 accessing the web site. may be

According to an embodiment, the interface includes a screen on which the uploaded image 621 is reproduced, function menus (or graphic objects) 622 for editing the image 621 , and a menu 623 for adding a mosaic. , a menu for deleting an image, or a menu 633 for storing an image may be included. The screen on which the image 621 is reproduced may reproduce image frames corresponding to a viewpoint of a timeline below the screen among a plurality of image frames constituting the image. In this case, when a menu for editing is selected from the function menus 622 for editing the image 621 , image frames displayed on the screen may be edited. For example, the function menus 622 for editing the image 621 may include editing functions such as color editing (eg, brightness, contrast) and animation addition, and are limited to the description and/or shown. It may further include well-known editing functions. When the menu 623 for adding mosaic is selected, a mosaic processing operation may be performed on objects corresponding to sensitive information of the image 621 .

According to an embodiment, the management server 110 may identify specific objects from the image in operation 403 and image-process regions in which the specific objects of the image are located in operation 404 . For example, when a menu 623 for adding a mosaic is selected in the electronic device 130 as shown in 602 of FIG. 6 , the management server 110 may obtain a request for adding a mosaic through an interface. . The management server 110 may identify objects corresponding to sensitive information from the image and mosaic the identified objects based on obtaining the request. For example, the objects corresponding to the sensitive information may be objects representing private information such as a sign, a license plate of a vehicle, or a person's face, but it is not limited to what is described and various sensitive information may be set according to the user's settings. can As described above, the management server 110 identifies an outline and/or an appearance of each object based on a value for each pixel included in each of a plurality of image frames constituting an image, and the identified outline and/or appearance When it corresponds to an outline and/or an appearance set as sensitive information (eg, a sign, a license plate of a vehicle, a human face) pre-stored in the management server 110 , the corresponding object may be determined as sensitive information. In addition, as at least part of the operation of processing the mosaic, the image editing module 113 manipulates the values of pixels constituting the object identified as sensitive information to be invisible (eg, reduces the area of the object at a certain rate). and then enlarging and changing the pixel value), but the present invention is not limited thereto, and the mosaic processing operation may be implemented as well-known techniques. In addition, as the time of the above-described timeline elapses, when the position of the object corresponding to the sensitive information in the image is moved, the management server 110 tracks the position of the object and processes the mosaic on the tracked object. action can be performed. Accordingly, the management server 110 may mosaic the area of the object (eg, vehicle license plate) 631 corresponding to the sensitive information of the image 621 as shown in 603 of FIG. 6 . In this case, the management server 110 may provide a notification 632 indicating that the mosaic is processed, as shown in 603 of FIG. 6 , so that the user can easily recognize the mosaic-processed portion in the image 621 .

According to an embodiment, the management server 110 may mosaic-process objects of a type set as sensitive information by a user. For example, the management server 110 may provide a menu 700 for receiving the type of sensitive information as shown in FIG. 7 . The menu 700 may include graphic objects corresponding to selectable types (eg, license plates, signs, and human faces) as sensitive information. When specific objects (eg, an object representing each of a license plate and a sign) are selected from among the objects on the interface, the management server 110 includes objects corresponding to the specific objects from the image (eg, a sign 710); It is possible to identify the license plate 631) and mosaic the area of the identified objects.

According to an embodiment, the management server 110 may set the characteristics of the mosaic. For example, when a notification 632 indicating that a mosaic is set on the interface is selected as shown in 801 of FIG. 8 , the management server 110 , as shown in 802 of FIG. You can provide objects (such as bars and buttons) for setting the mosaic degree). For example, the characteristics of the mosaic are not limited to the substrate and/or shown, and may include various types of visual characteristics, such as a pattern type of the mosaic, in addition to the degree of mosaic. When the value of the characteristic of the mosaic (eg, degree of mosaic) is set using the object, the management server 110 may determine the characteristic of the mosaic with the corresponding value and reflect it in the image.

According to an embodiment, the management server 110 may store an image in which regions are image-processed in operation 405 . For example, when the menu 633 for storage is selected on the interface as shown in 603 of FIG. 6 , the management server 110 receives a request to save the edited image so far through the interface, and You can save the edited video up to.

According to an embodiment, as shown in 503 of FIG. 5 , the management server 110 transmits the edited image to the image sharing server 120 at the request of the electronic device 130 and/or automatically for editing. It is possible to upload the video to the image sharing server 120 .

3.2 Second embodiment <Example of performing mosaic processing on a specific image based on accumulated information>

Since the operations of the above-described smart image editing system can be applied mutatis mutandis to the second embodiment, a redundant description will be omitted.

According to an embodiment, the management server 110 generates an artificial intelligence model for applying the mosaic of a specific image, based on information about the images accumulated in the management server 110 and information related to the mosaic set in the images. can learn The artificial intelligence model learned by the management server 110, in response to receiving information about the image, information about the type of sensitive information to be processed as a mosaic in the image (or information about objects) And/or it may be implemented to output information about the characteristics of the mosaic.

9 is a flowchart for explaining another example of the operation of the smart image editing system according to an embodiment of the present application. According to various embodiments, the operation of the smart image editing system is not limited to the order of the operations illustrated in FIG. 9 , and may be performed in an order different from the illustrated order. Also, according to various embodiments, more operations or fewer operations than those of the smart image editing system illustrated in FIG. 8 may be performed, or at least one operation less may be performed. Hereinafter, FIG. 9 will be described with reference to FIGS. 10 to 12 .

10 is a diagram for explaining an example of an operation of learning an artificial intelligence model based on accumulated information of a smart image editing system according to an embodiment of the present application. 11 is a view for explaining an example of the operation of mosaic processing a specific image based on the artificial intelligence model of the smart image editing system according to an embodiment of the present application. 12 is a view for explaining an example of the operation of mosaic processing a specific image on the interface of the smart image editing system according to an embodiment of the present application.

According to an embodiment, the management server 110 may acquire a plurality of pieces of information related to the mosaic setting accumulated in the management server in operation 901 . For example, the management server 110 completes the operation of mosaicing the images uploaded from the electronic device 130 and objects corresponding to sensitive information in the image as described above (eg, a save menu is selected on the interface) Accordingly, information about the image and information related to the mosaic set in the image may be accumulated in the database 116 . The information about the image is at least one of the type of image, the length of time of the image, the types of objects included in the image (eg, signs, license plates, human faces), or the platform (or image sharing server) to which the image is uploaded. may include information about, and may include information related to various types of images without being limited to the descriptions. In this case, when the image is accumulated in the database 116 , the management server 110 may extract information on objects included in the image and store information on the types of objects included in the image. . Also, the information related to the mosaic may include information on at least one of the types of objects for which a mosaic is finally set in the image, or characteristics of the set mosaic (eg, a degree of mosaic, a type of a mosaic pattern, etc.). The management server 110 may store information about the image and information related to a mosaic corresponding to the image in a form related to each other.

According to an embodiment, the management server 110 may learn an artificial intelligence model based on a plurality of pieces of information in operation 902 . In one embodiment, the management server 110 first, as shown in 1001 of FIG. 10, to generate the artificial intelligence model 1000 for mosaic processing for a specific image, information accumulated in the database 116 (eg: Information about the above-described image and information related to the mosaic stored in a related form) may be extracted. In an embodiment, the management server 110 processes the information (eg, information about the above-described image and information related to the mosaic stored in a related form, etc.) into a learnable form as shown in 1002 of FIG. 10 . can For example, the management server 110 may perform operations (eg, Allocation, Wrangling) such as pre-processing each piece of information, converting it into a matrix value capable of matrix operation, and arranging the converted matrix values to enable operation. Since the operation of pre-processing the data is a well-known technique, a detailed description thereof will be omitted.

In an embodiment, the management server 110 performs learning by using the processed information (eg, information about the image and information related to the mosaic stored in a related form, etc.) as training data as training data. A model 1000 may be created. For example, as shown in 1003 of FIG. 10 , the management server 110 includes information about an image (eg, information about an image and information related to a mosaic stored in a related form, etc.) Input data (information about at least one of the length of time of the video, the types of objects included in the video (eg, signs, license plates, human faces), or the platform (or video sharing server) to which the video is uploaded) input data) and, accordingly, information related to the mosaic corresponding to the image (eg, the types of objects for which the mosaic is finally set in the image, or the value of the set mosaic characteristics (eg, the degree of mosaic, the type of mosaic pattern, etc.) One information) may be set as output data, and learning (eg, parameter (eg, weight) learning 1030) may be performed based on various types of artificial intelligence learning algorithms. Since the artificial intelligence learning algorithm may include various types of well-known machine learning and deep learning algorithms, a detailed description thereof will be omitted. At this time, according to the learning, the generated artificial intelligence model 1000 responds to input of information 1041 on a specific image, by object type (eg, sign, license plate, human face) to be mosaiced in a specific image. It may be implemented to finally output the information 1042 on the weight for each value of the weight and the characteristic of the mosaic (eg, the degree of mosaic, the type of mosaic pattern). The artificial intelligence model 1000 is information about the input image (eg, the type of image, the length of the image, the types of objects included in the image (eg, signs, license plates, human faces), or a platform to which the image is uploaded. (or, information on at least one of the image sharing server), including layers including weight values for calculating at least a part, and finally specific It may be implemented to output information 1042 on a weight for each type of object to be mosaiced in the image (eg, a sign, a license plate, a human face) and a weight for each value of a characteristic of a mosaic (eg, a degree of mosaic, a type of a mosaic pattern).

According to an embodiment, the management server 110 receives a request for mosaic generation for a specific image in operation 903 , inputs information on a specific image to the artificial intelligence model in operation 904 , and based on the output information to acquire characteristic information for setting mosaic, and to process mosaic for a specific image based on the acquired characteristic information. For example, as illustrated in 1101 of FIG. 11 , the management server 110 may receive a request for mosaic setting for a specific image (eg, image A) from the electronic device 130 . For example, as shown in 1201 of FIG. 12 , when a menu for mosaic processing is selected while a specific image (eg, image A) is uploaded on the interface, the management server 110 sends a specific image ( Example: A request for mosaic setting for A video) may be received. In an embodiment, the management server 110 may receive information on the specific image (eg, image A) through the interface as at least part of an operation of receiving the specific image (eg, image A). For example, the information about the input specific image (eg, image A) may include the type of image, the length of time of the image, and a platform (or image sharing server) to which the image is uploaded. When the specific video (eg, video A) is uploaded on the interface, a screen for entering information on the specific video (eg, video A) is displayed and the specific video (eg, video A) is displayed on the screen. As the information is input, the information may be transmitted to the management server 110 . In another embodiment, the management server 110 analyzes the uploaded specific image (eg, image A) as at least part of an operation of receiving a specific image (eg, image A) to analyze a specific image (eg, image A). information can be entered. For example, the management server 110 analyzes pixel values of a plurality of image frames reproduced for the entire time of the specific image and compares the pixel values with pre-stored information on the types of objects, the object included in the specific image types (eg, signs, license plates, human faces) can be obtained. Accordingly, the management server 110 may process information on the acquired image (eg, image A) into a form that can be input to the artificial intelligence model 1000 . As shown in 1102 of FIG. 11 , the management server 110 responds to input of information 1121 about an image (eg, image A) into the artificial intelligence model 1000, an object to be mosaiced in a specific image. Information on weights for each type (eg, sign, license plate, human face) and for each characteristic of the mosaic (eg, degree of mosaic, type of mosaic pattern) may be acquired. Management server 110 in the order of the weight of the weight of each type of object to be mosaiced (eg, sign, license plate, human face) in a specific image and information about the weight for each characteristic of the mosaic (eg, degree of mosaic, type of mosaic) It is possible to select a preset number (eg, two) of types of objects to be mosaiced (eg, signs, license plates) and values of characteristics of mosaics (eg, first values and second values). The management server 110 identifies the type of object (eg, sign, license plate) included in the image based on the selected information as shown in 1202 of FIG. 12 , and the type of the identified object (eg, a sign ( 1221), the mosaic is automatically processed in the area of the license plate 1222), but the mosaic can be provided by processing the mosaic with the value (eg, first value, second value) of the characteristic (eg, mosaic degree).

According to an embodiment, when the value of the mosaic property for a specific object among the objects automatically mosaiced on the interface is changed, the management server 110 may also automatically change the value of the mosaic property for the remaining objects. . For example, if the management server 110 identifies that the value of the mosaic characteristic (eg, mosaic degree) of the sign 1221 is reduced, the mosaic characteristic of the remaining automatically mosaicized license plate 1222 (eg, mosaic) degree) can also be reduced. In this case, the management server 110 may set the reduction degree as shown in [Equation 1] below.

[Equation 1]

According to one embodiment, the management server 110 automatically removes mosaic processing of a specific object from among the mosaic-processed objects on the interface, an object having a lower weight (eg, a human face) and a value of the mosaic characteristic Based on (eg, a third value), an object (eg, a human face) in the image may be mosaicked.

3.3 Third embodiment <Example of editing video sections in which mosaics are frequent in images in which mosaics are automatically set>

Since the operations of the above-described smart image editing system can be applied mutatis mutandis to the third embodiment, a redundant description will be omitted.

According to an embodiment, the management server 110 automatically deletes and edits the mosaic-processed image based on the information output from the artificial intelligence model 1000, when an image section with frequent mosaics is identified, so that an image with high visibility is displayed. can provide Alternatively, without being limited thereto, the management server 110 may release mosaic processing of objects having a low weight ranking from the corresponding section when an image section in which mosaic is frequent is identified.

13 is a flowchart for explaining another example of the operation of the smart image editing system according to an embodiment of the present application. According to various embodiments, the operation of the smart image editing system is not limited to the order of the operations illustrated in FIG. 13 , and may be performed in an order different from the illustrated order. In addition, according to various embodiments, more operations or fewer operations than those of the smart image editing system shown in FIG. 13 may be performed, or at least one operation may be performed. Hereinafter, FIG. 13 will be described with reference to FIG. 14 .

14 is a diagram for explaining an example of an operation of editing an image of a specific time period by identifying a mosaic area in an image of the management server 110 according to an embodiment of the present application.

According to an embodiment, the management server 110 may identify an area of a mosaic in a specific time period of an image in which a plurality of set mosaics are generated in operation 1301 . For example, the management server 110, as described above, based on the information obtained using the artificial intelligence model 1000 (eg, the type of object to be mosaiced and the value of the characteristic), among the objects included in the image. Some areas (eg signs and license plates) can be mosaicked. At this time, the management server 110 , as shown in 1401 of FIG. 14 , the mosaics 1411 , 1412 , 1413 , 1414 , 1415 included in the image frame constituting the image of a specific time period (eg, the time period of t1 to t2). ) can be identified, and the total mosaic area can be calculated by summing the identified areas.

According to an embodiment, the management server 110 compares the identified area of the mosaic with a preset first area in operation 1302, and when the identified area of the mosaic is greater than the first area, in operation 1303, a specific time period You can create an edited video by automatically deleting the video of For example, the preset first area may be preset to 80% of the area of the image. When the area of the mosaic is greater than 80% of the area of the image, the management server 110 removes and edits the image of a specific time period (eg, time period of t1 to t2) as shown in 1402 of FIG. 14 . image can be obtained. Accordingly, a high-quality image may be provided by removing an image portion having an excessively large area of the mosaic.

According to an embodiment, when the area of the identified mosaic is smaller than the first area, the management server 110 compares the area of the identified mosaic with the designated second area in operation 1304, and the area of the identified mosaic is When the area is larger than the second area, mosaic processing of some of the objects of the image of a specific time period may be released based on the weights for each mosaic-processed object in operation 1305 . For example, the preset second area may be preset to 40% of the area of the image. In this case, the non-mosaic image portion may be sufficiently secured by releasing mosaic processing of some of the mosaic objects (eg, signs and license plates). In one embodiment, the management server 110 may release the mosaic of the object with the lowest priority (eg, license plate) among the mosaic-processed objects. After the mosaic release, the management server 110 identifies the area of the mosaic, and when the identified area of the mosaic is still greater than 40% of the area of the image, it may release the mosaic of an object of a higher priority.

Claims (1)

As an alternative method to a new image using mosaic processing performed on a server,
and providing an interface for uploading an image and editing an image to the electronic device when the electronic device is connected to the server, wherein the interface includes a screen for playing an image and a plurality of images for editing the image including menus,
acquiring the uploaded image when the image photographed by the electronic device is uploaded on the interface;
When a menu for processing the image mosaic is selected from among a plurality of menus on the interface, a plurality of objects included in the image are identified, and a type selected by the electronic device from among the identified plurality of objects is selected. 1 step of identifying objects; including, wherein the selected type includes at least one of a sign, a signboard, a license plate of a vehicle, or a person's face,
mosaicing the first regions of the identified first objects, and displaying a notification indicating that the mosaic is processed on an image displayed on the interface; and
and storing the image when an object for storing the image including the mosaic-processed first areas is selected on the interface, wherein the stored image is saved according to a request received from the electronic device Registered as a video sharing server,
Displaying at least one object for adjusting a value of a characteristic of a mosaic at a position associated with the mosaic-processed first areas when the notification indicating that the mosaic is processed is selected; further comprising, the mosaic The characteristics of include the degree of the mosaic and the pattern type of the mosaic,
Based on the at least one object, setting a first value of the characteristic of the mosaic; further comprising,
Further comprising; obtaining information about a plurality of images stored in the server, and information related to a mosaic corresponding to the plurality of images, each of the information on the plurality of images includes: a type of image; It includes the length of time of the image, information on the types of objects included in the image, or information on the image sharing server to which the image is uploaded, and each of the information related to the mosaic is an object for which a mosaic is set in the image. Including information about the type of the mosaic, or the characteristics of the mosaic set in the image,
Using at least one of the information on the plurality of images as input data, and using the information related to the mosaic as output data, generating an artificial intelligence model; further comprising, wherein the artificial intelligence model includes: In response to receiving information, it is implemented to output weights for the types of specific objects for which a mosaic is to be set in the specific image, or weights for values of the characteristic of the mosaic,
receiving a message for a request for setting a mosaic of a first image from the electronic device;
In response to inputting the information on the first image into the artificial intelligence model, in the first image, the types of second objects for which the mosaic is to be set and the first weights for the values of the characteristic of the mosaic are identified. step; and
The method further includes: identifying the second objects from among a plurality of objects included in the first image based on the first weights, and mosaicing and providing second regions of the second objects;
At least one outline is recognized based on a value for each pixel for each of the plurality of frame images constituting the first image, and the recognized at least one outline is pre-stored in the server Information on the outline for each type of object and identifying the second objects based on comparison with
The values of the characteristic of the mosaic associated with the first weights include a first value of the characteristic for some of the second objects and a second value of the characteristic for other of the second objects and,
When the value of the characteristic of the mosaic of some of the second objects is decreased, reducing the value of the characteristic of the mosaic of the remaining objects of the second objects based on the following equation; further including,

,
When the mosaic of some of the second objects is released, releasing the mosaic of the remaining objects; further comprising,
identifying an area of a mosaic in a specific time period of the first image in which the mosaic is set in the second areas; and
Comparing the identified area and a designated first area; further comprising, wherein the designated first area is an area of 80% of the area of the first image,
When the identified area is larger than the designated first area, obtaining the edited first image by deleting the image of the specific time period;
Alternative method to new image using mosaic processing.

Images