WO2024029675A1 - System for providing private video management service using qr codes - Google Patents

Abstract

Provided is a system for providing a private video management service using QR codes, comprising: a user terminal which registers a QR code on a pre-purchased sticker, shoots a video, and requests the generation of shared QR codes; a terminal to which the video is shared, which scans a shared QR code to gain access to the video shot by the user terminal and streams the shot video; and a management service provision server which comprises a purchase confirmation unit for registering the QR code on a pre-purchased sticker from the user terminal, an upload unit for receiving a shot video when the user terminal shoots a video, a shared link generation unit for generating a preset number of shared QR codes according to the rating of the user terminal when the user terminal requests the generation of shared QR codes, and an access grant unit for granting access to the shot video for the terminal to which the video is shared, if the terminal scans the shared QR code.

Description

Private video management service provision system using QR code

The present invention relates to a system for providing a private video management service using a QR code, and provides a platform through which videos captured by a user can be delivered to the recipient using a shared QR code.

In order to think about the contemporary media environment, it is necessary to pay attention to the phenomenon of personalization of video. The most used services in the personalized media environment are video streaming services such as YouTube and social networking services (SNS) such as Facebook and Instagram. Audiences, who were in the position of unilaterally accepting completed content, take on a more active role in the YouTube and SNS paradigm, uploading their own content as creators, and at the same time finding and watching the content they want as active users. Particularly in terms of creativity, a single media creator completes a video shot with a single camera in a private space using editing software and then publishes it online, or transmits the video in real time on site using a small camera such as a GoPro. do.

At this time, a method of sharing a video based on a QR code or accessing and printing content by scanning a QR code was researched and developed. In this regard, the prior art, Korea Registered Patent No. 10-2009231 (August 9, 2019) Announcement) and Korean Patent No. 10-1527492 (announced on June 10, 2015), there is a configuration for recognizing a QR code, processing an image, storing the processed image into a database, and executing an application on a user terminal. When you select the button to create or scan a QR code, the URL information associated with the QR code is extracted, the server is accessed through the URL, the storage within the server is accessed, the information is detected, and the information is detected through the speaker or screen of the user terminal. Each configuration for outputting content is disclosed.

However, in the former case, you must go through the process of extracting the QR code with a microscope to output the sample image, and in the latter case, rather than sharing private images based on social relationships, you simply share the URL through the QR code. It's just starting. Profile pages and social relationships are made visible through social media, but public communication through SNS, streaming, likes, and comments has been initiated. Although a privacy function has recently been added, tagging or making it private so that only specific people can view it is being implemented. There is no video communication that can share private videos other than dropping them. Accordingly, research and development of a platform that can deliver and manage private images through QR codes according to social relationships is required.

One embodiment of the present invention allows the user terminal to select Heart Messenger as a delivery option when purchasing a product at a shopping mall and sending it as a gift, and the shopping mall processes delivery by attaching a QR code based on Heart Messenger in the delivery box, The user terminal can upload a video message, and the shared QR code can be printed out by purchasing a sticker QR code on the user terminal, uploading the captured video, generating a shared QR code, and delivering it to the shared terminal. , Deliver and manage private images so that you can access the captured images by scanning the shared QR code on the shared terminal, and edit the images captured on the user terminal manually or using an editing tool that is already mapped and stored in categories according to social relationships. It allows automatic editing, and when sending a gift from the user terminal, a shared QR code is sent along with the gift so that private videos accumulated in time series can be shared, stored, and printed according to the social relationship between the user and the recipient. It is possible to provide a private video management service provision system using QR code. However, the technical challenge that this embodiment aims to achieve is not limited to the technical challenges described above, and other technical challenges may exist.

As a technical means for achieving the above-described technical problem, one embodiment of the present invention is a user terminal that registers a previously purchased sticker QR code, takes an image, and requests creation of a shared QR code, and scans the shared QR code. A purchase confirmation unit that registers a QR code for a sticker already purchased from a shared terminal and a user terminal that receives streaming of the captured video by granting access rights to the video captured in the user terminal, and that records the captured video when a video is captured in the user terminal. Receiving upload unit, when the user terminal requests the generation of a shared QR code, a shared link generator that generates a preset number of shared QR codes according to the level of the user terminal, and when the shared QR code is scanned by the shared terminal. It includes a management service providing server that includes an access right grant status that grants access rights to images captured with a shared terminal.

In addition, when purchasing a product from at least one shopping mall on the user terminal, when the option of the sticker QR code or the shared QR code is selected, the sticker QR code or the shared QR code is included in the delivery option together with the product to deliver the product. When delivered, the sticker QR code or shared QR code is printed, attached, or included in a delivery box.

The management service providing server further includes a gift delivery unit that processes an order to attach and transmit the shared QR code to a box of the product when the user terminal purchases a product to be sent together with the captured video.

The management service providing server determines the type and number of editing tools applied to the captured video according to the grade of the user terminal, the number and capacity to generate the shared QR code, and the number of accesses to the video shared with the shared QR code. and a grade classification unit that sets the period differentially.

The management service providing server uploads and stores the video captured by the user terminal to a preset cloud server, and when the shared QR code is scanned by the shared terminal, the video captured by the user terminal in the preset cloud server is stored. It further includes a cloud management unit that grants access to the video.

The management service providing server builds at least one editing tool applied according to at least one social relationship for each category, and when the user terminal selects a target to transmit the shared QR code, pre-stored in a category according to the target It further includes a social relationship reflection unit that provides at least one editing tool.

The management service providing server further includes a crowdsourcing unit that constructs a database by uploading a template applied according to the at least one social relationship from at least one designer terminal.

According to one of the above-described means of solving the problem of the present invention, the user terminal allows the heart messenger to be selected as a delivery option when purchasing a product at a shopping mall and sending it as a gift, and the shopping mall uses a QR code based on the heart messenger in the delivery box. It is delivered to be attached, and the user terminal can upload a video message, and the shared terminal can print it. After purchasing a sticker QR code on the user terminal, uploading the captured video and creating a shared QR code When delivered to the shared terminal, the private video is delivered and managed so that the captured video can be accessed after scanning the shared QR code on the shared terminal, and the video captured on the user terminal is already mapped to categories according to social relationships. It allows manual or automatic editing with the saved editing tool, and when sending a gift from the user terminal, a shared QR code is sent along with the gift to share private videos accumulated in time series according to the social relationship between the user and the recipient. Make it available for storage and printing.

1 is a diagram illustrating a system for providing a private video management service using a QR code according to an embodiment of the present invention.

FIG. 2 is a block diagram illustrating a management service providing server included in the system of FIG. 1.

Figures 3 and 4 are diagrams for explaining an embodiment in which a private video management service using a QR code is implemented according to an embodiment of the present invention.

Figure 5 is an operation flowchart for explaining a method of providing a private video management service using a QR code according to an embodiment of the present invention.

As a technical means for achieving the above-described technical problem, one embodiment of the present invention is a user terminal that registers a previously purchased sticker QR code, takes an image, and requests creation of a shared QR code, and scans the shared QR code. A purchase confirmation unit that registers a QR code for a sticker already purchased from a shared terminal and a user terminal that receives streaming of the captured video by granting access rights to the video captured in the user terminal, and that records the captured video when a video is captured in the user terminal. Receiving upload unit, when the user terminal requests the generation of a shared QR code, a shared link generator that generates a preset number of shared QR codes according to the level of the user terminal, and when the shared QR code is scanned by the shared terminal. It includes a management service providing server that includes an access right grant status that grants access rights to images captured with a shared terminal.

Below, with reference to the attached drawings, embodiments of the present invention will be described in detail so that those skilled in the art can easily implement the present invention. However, the present invention may be implemented in many different forms and is not limited to the embodiments described herein. In order to clearly explain the present invention in the drawings, parts that are not related to the description are omitted, and similar parts are given similar reference numerals throughout the specification.

Throughout the specification, when a part is said to be "connected" to another part, this includes not only the case where it is "directly connected," but also the case where it is "electrically connected" with another element in between. . In addition, when a part is said to "include" a certain component, this does not mean excluding other components unless specifically stated to the contrary, but may further include other components, and one or more other features. It should be understood that it does not exclude in advance the presence or addition of numbers, steps, operations, components, parts, or combinations thereof.

The terms "about", "substantially", etc. used throughout the specification are used to mean at or close to that value when manufacturing and material tolerances inherent in the stated meaning are presented, and are used to enhance the understanding of the present invention. Precise or absolute figures are used to assist in preventing unscrupulous infringers from taking unfair advantage of stated disclosures. The term “step of” or “step of” as used throughout the specification of the present invention does not mean “step for.”

In this specification, 'part' includes a unit realized by hardware, a unit realized by software, and a unit realized using both. Additionally, one unit may be realized using two or more pieces of hardware, and two or more units may be realized using one piece of hardware. Meanwhile, '~ part' is not limited to software or hardware, and '~ part' may be configured to reside in an addressable storage medium or may be configured to reproduce one or more processors. Therefore, as an example, '~ part' refers to components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, and procedures. , subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided within the components and 'parts' may be combined into a smaller number of components and 'parts' or may be further separated into additional components and 'parts'. Additionally, components and 'parts' may be implemented to regenerate one or more CPUs within a device or a secure multimedia card.

In this specification, some of the operations or functions described as being performed by a terminal, apparatus, or device may instead be performed on a server connected to the terminal, apparatus, or device. Likewise, some of the operations or functions described as being performed by the server may also be performed in a terminal, apparatus, or device connected to the server.

In this specification, some of the operations or functions described as mapping or matching with the terminal mean mapping or matching the terminal's unique number or personal identification information, which is identifying data of the terminal. It can be interpreted as

Hereinafter, the present invention will be described in detail with reference to the attached drawings.

1 is a diagram illustrating a system for providing a private video management service using a QR code according to an embodiment of the present invention. Referring to FIG. 1, the private video management service providing system 1 using a QR code may include at least one user terminal 100, a management service providing server 300, and at least one shared terminal 400. You can. However, since the private video management service providing system 1 using the QR code of FIG. 1 is only an embodiment of the present invention, the present invention is not limitedly interpreted through FIG. 1.

At this time, each component of FIG. 1 is generally connected through a network (Network, 200). For example, as shown in FIG. 1, at least one user terminal 100 may be connected to the management service providing server 300 through the network 200. In addition, the management service providing server 300 may be connected to at least one user terminal 100 and at least one shared terminal 400 through the network 200. Additionally, at least one shared terminal 400 may be connected to the management service providing server 300 through the network 200.

Here, the network refers to a connection structure that allows information exchange between each node, such as a plurality of terminals and servers. Examples of such networks include a local area network (LAN) and a wide area network (WAN). Wide Area Network, Internet (WWW: World Wide Web), wired and wireless data communication network, telephone network, wired and wireless television communication network, etc. Examples of wireless data communication networks include 3G, 4G, 5G, 3rd Generation Partnership Project (3GPP), 5th Generation Partnership Project (5GPP), Long Term Evolution (LTE), World Interoperability for Microwave Access (WIMAX), and Wi-Fi. , Internet, LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), RF (Radio Frequency), Bluetooth network, NFC ( It includes, but is not limited to, Near-Field Communication (Near-Field Communication) network, satellite broadcasting network, analog broadcasting network, and DMB (Digital Multimedia Broadcasting) network.

In the following, the term at least one is defined as a term including singular and plural, and even if the term at least one does not exist, each component may exist in singular or plural, and may mean singular or plural. This should be self-explanatory. In addition, whether each component is provided in singular or plural form may be changed depending on the embodiment.

At least one user terminal 100 purchases a sticker QR code using a web page, app page, program, or application related to a private video management service using a QR code, takes a video, uploads it, and shares the QR code with a member. It may be a terminal that issues the number corresponding to the grade and transmits the shared QR code to the shared terminal 400 via the management service providing server 300.

Here, at least one user terminal 100 may be implemented as a computer capable of accessing a remote server or terminal through a network. Here, the computer may include, for example, a laptop equipped with a navigation system and a web browser, a desktop, a laptop, etc. At this time, at least one user terminal 100 may be implemented as a terminal capable of accessing a remote server or terminal through a network. At least one user terminal 100 is, for example, a wireless communication device that guarantees portability and mobility, and includes navigation, personal communication system (PCS), global system for mobile communications (GSM), personal digital cellular (PDC), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet) ) It may include all types of handheld-based wireless communication devices such as terminals, smartphones, smartpads, and tablet PCs.

The management service providing server 300 may be a server that provides a private video management service web page, app page, program, or application using a QR code. In addition, the management service providing server 300 may be a server that generates a preset number of shared QR codes for each grade when an image is captured by the user terminal 100 that scans the sticker QR code. Additionally, the management service providing server 300 may be a server that grants access rights to access images captured by the user terminal 100 when the shared QR code is scanned by the shared terminal 400. In addition, when the user terminal 100 wants to use an editing tool or a template according to the social relationship between the user and the shared recipient, the management service providing server 300 is free according to the level of the user terminal 100 or whether it is a paid member or not. It may be a server that differentially grants access rights depending on membership.

Here, the management service providing server 300 may be implemented as a computer that can connect to a remote server or terminal through a network. Here, the computer may include, for example, a laptop equipped with a navigation system and a web browser, a desktop, a laptop, etc.

At least one shared terminal 400 scans the shared QR code using a web page, app page, program, or application related to a private video management service using a QR code, and then accesses and streams the video of the user terminal 100. It may be the share recipient's terminal that receives and outputs the video.

Here, at least one shared terminal 400 may be implemented as a computer capable of accessing a remote server or terminal through a network. Here, the computer may include, for example, a laptop equipped with a navigation system and a web browser, a desktop, a laptop, etc. At this time, at least one shared terminal 400 may be implemented as a terminal capable of accessing a remote server or terminal through a network. At least one shared terminal 400 is, for example, a wireless communication device that guarantees portability and mobility, such as navigation, personal communication system (PCS), global system for mobile communications (GSM), and personal digital cellular (PDC). , PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband) It may include all types of handheld-based wireless communication devices such as Internet) terminals, smartphones, smartpads, and tablet PCs.

Figure 2 is a block diagram for explaining the management service providing server included in the system of Figure 1, and Figures 3 and 4 are an implementation of a private video management service using a QR code according to an embodiment of the present invention. This is a drawing to explain an example.

Referring to Figure 2, the management service providing server 300 includes a purchase confirmation unit 310, an upload unit 320, a sharing link creation unit 330, an access rights grant unit 340, a gift delivery unit 350, It may include a rating division unit 360, a cloud management unit 370, a social relationship reflection unit 380, and a crowdsourcing unit 390.

The management service providing server 300 according to an embodiment of the present invention or another server (not shown) operating in conjunction uses a QR code with at least one user terminal 100 and at least one shared terminal 400. When transmitting a private video management service application, program, app page, web page, etc., at least one user terminal 100 and at least one shared terminal 400, a private video management service application, program using a QR code , you can install or open app pages, web pages, etc. Additionally, a service program may be run on at least one user terminal 100 and at least one shared terminal 400 using a script executed in a web browser. Here, a web browser is a program that allows the use of web (WWW: World Wide Web) services and refers to a program that receives and displays hypertext written in HTML (Hyper Text Mark-up Language), for example, Netscape. , Explorer, Chrome, etc. Additionally, an application refers to an application on a terminal and includes, for example, an app running on a mobile terminal (smartphone).

Referring to FIG. 2, the purchase confirmation unit 310 may receive a previously purchased sticker QR code from the user terminal 100. The user terminal 100 may register a previously purchased sticker QR code, capture a video, and request generation of a shared QR code. In one embodiment of the present invention, two types of QR codes are used. One is a sticker QR code that allows the user to purchase offline and edit and transmit the video through the platform of the present invention, and the other is a sticker QR code that allows the user to upload the video captured in the platform of the present invention, for example, an application, and then send it to the share recipient terminal 400. Another shared QR code saved as [Video-URL-Shared QR Code] for sharing. A method of using only one rather than both can be applied, and the platform of the present invention can be built in various embodiments.

At this time, the ultimate purpose of the service according to an embodiment of the present invention is to use the heart messenger (QR code) as a delivery option when delivering to the shareee of the shareee terminal 400 directly at the shopping mall or by a seller in the shopping mall. When the user terminal 100 pays a preset fee and selects the heart messenger option while sending the product as a gift to the recipient, the shopping mall or seller attaches a sticker with a QR code printed on the product or sends it to the delivery box. It is delivered by including it within the user terminal 100, and the user terminal 100 uploads the desired video message within the application. After the platform according to an embodiment of the present invention is commercialized, a delivery option service can be provided in partnership with at least one shopping mall. Accordingly, when the user terminal 100 selects the option of a sticker QR code or a shared QR code when purchasing a product at at least one shopping mall, the sticker QR code or the shared QR code is included in the delivery option together with the product so that the product is delivered. When delivered, a sticker QR code or shared QR code can be printed, attached, or included in the shipping box.

The upload unit 320 may receive the captured image when the user terminal 100 captures the image. You can shoot a video on the spot, upload a video that has already been shot, or select the image you want and create a video by adding subtitles or background music using the editing tool described later.

When the user terminal 100 requests generation of a shared QR code, the shared link generator 330 may generate a preset number of shared QR codes according to the level of the user terminal 100. For example, templates that can be automatically created, edited, or used by the platform of the present invention, such as free members and paid members, or subscription members and general members, or services or editing tools provided by the platform of the present invention according to the level of membership, etc. The number and type of can be differentiated. In addition, since the number of shared QR codes can be sent to the number of shared QR codes, i.e., how many shared QR codes can be generated, can be set differentially, the sharing link generator 330 determines the user's level level, membership You can proceed with the process so that a preset number of shared QR codes can be created by determining whether or not you are subscribed or not.

The access right management unit 340 may grant access to the image captured by the shared terminal 400 when the shared QR code is scanned. The shared terminal 400 can scan the shared QR code to be granted access to the video captured by the user terminal 100 and receive streaming of the captured video. At this time, if anyone who scans the shared QR code can access it, there is a risk that anyone who receives the shared QR code link can access the private video. However, it is also very cumbersome to manually map the shared QR code to each recipient's mobile phone number and go through authentication, and especially for the IT vulnerable, accessibility is reduced and barriers are erected. Accordingly, token-based authentication that allows even the IT vulnerable to easily access and authenticate with the other party, that is, the shareee's terminal 400, so that only the shareee can scan the shared QR code, authenticate, and access the video. , for example, you can use a JWT (JSON Web Token) token.

At this time, when the shareee accesses the URL logged in after registering as a member, the management service providing server 300 verifies through a filter whether the token exists and is valid. If the authentication process is completed, the shareee, who is the client, verifies the existence and validity of the token through a filter. A cookie containing a JWT value is sent to the terminal (400). If authentication fails, a '401 Unauthorized' error can be returned, and when accessing a resource that requires specific permission after successful authentication, the corresponding permission does not exist. If not, you can return a '403 Forbidden' error. In terms of security for maintaining login, apply security to prevent access to cookies through JavaScript code by applying the HTTP ONLY option to cookies to prevent XSS attacks. can do.

<Token-based authentication>

The token-based authentication method issues a token to the shared recipient terminal 400 requesting authentication. Afterwards, when the shared recipient terminal 400 makes a request to the management service providing server 300, a token is transmitted and validity is checked. In this way, the management service providing server 300 can process tasks only with requests coming from the client, that is, the sharing recipient terminal 400, without maintaining the shared recipient's authentication information in the management service providing server 300 or the session. In other words, unlike server-based authentication systems, it does not maintain state and therefore has a stateless structure. Numerous problems can be solved through this token-based authentication method. For example, the system can be easily expanded without worrying about whether the recipient is logged in or not. Additionally, for security purposes, cookies are no longer transmitted when the client sends a request to the management service providing server 300, so vulnerabilities caused by the use of cookies disappear.

<JWT>

JWT (JSON Web Token) is a web standard (RFC 7519) and is a token authentication method that transmits information lightly and safely using a JSON object between two entities. Additionally, since JWT is also a token authentication-based method, it proceeds in the same way as the previously mentioned token authentication-based method. The characteristic of JWT is that it is self-contained, and the issued token includes information about the shareee, information about the token itself, and signature.

AAAAAA	BBBBBB	CCCCCC
Header	Content (Payload)	Signature

As shown in Table 1, the structure is largely divided into three types, and each structure is explained. First, the Header contains two pieces of information: the first specifies the type of token and the second specifies the hashing algorithm. The hashing algorithm is usually HMAC SHA256 or RSA, and this algorithm is used in the Signature part used to verify the token. The following is the part containing the information to be included in the payload. A piece of information contained here is called a claim, and it consists of a Name/Value pair. There are three main types of claims: Registered Claim, Public Claim, and Private Claim. Registered Claims are claims whose names have already been determined to contain information about tokens, not information required for the service. The use of registered claims is all optional, and the claim names included are shown in Table 2 below.

property	explanation
iss	Token issuer
sub	Token subject
aud	Token audience
exp	Token expiration time
nbf	Not Before
iat	Time the token was issued (issued at)
jti	Unique identifier for JWT

The following Public Claim has a collision-resistant name, and the claim name is stored in URI (Uniform Resource Identifier) format to prevent collisions. Lastly, Private Claim is a claim name used under agreement between two participants (client↔server). The simple payload format is shown in Table 3 below.

{ "iss": testlcom", "exp": "11244230000000", "https://testcom/jwt_claims/is_admin": true, "userId": "10044230000000", "username": "test" }

The final Signature is created by combining the encoding value of the header and the encoding value of the information and then hashing it with the given secret key. The main advantage of this JWT is that all information required for sharing recipient authentication is included in the token itself, so there is no need for a separate authentication storage. Therefore, it is safer than methods that use server scalability or cookies. Of course, it goes without saying that the authentication of the recipient can be maintained using various methods in addition to the JWT described above.

When purchasing a product to be sent along with a video captured on the user terminal 100, the gift delivery unit 350 may place an order for transmission by attaching a shared QR code to the box of the product.

The rating classification unit 360 determines the type and number of editing tools applied to the captured video according to the rating of the user terminal 100, the number of shared QR codes to be created, the capacity, and the number of accesses to the video shared with the shared QR code. and periods can be set differentially.

The cloud management unit 370 uploads and stores the video captured in the user terminal 100 to a preset cloud server, and when the shared QR code is scanned by the shared terminal 400, the user terminal (100) in the preset cloud server ) can be granted access to the recorded video. At this time, in addition to DaaS (Data as a Service) functions, the cloud can provide highlighting functions, editing functions, and viewer functions like SaaS (Software as a Service). The highlighting engine is a module that uses multimedia analysis technology to analyze the contents of image, video, and audio content and then extracts available data. The editing function may be a type of editing tool that automatically provides edited videos to the user from the management service providing server 300 and the user terminal 100 or allows the user to edit them directly. The viewer can show the relationship with the user and life log. The highlighting function is performed in the intelligent analysis highlighting engine, and the editing and viewer functions are performed in the user terminal 100 and the management service providing server 300.

<Intelligent Analysis Highlighting Engine>

The highlighting module can largely be composed of a client and a cloud server. The client may be a user terminal 100 or a shared receiver terminal 400, and may acquire multimedia data and perform low-level image analysis. The functions performed by the client are image analysis, video analysis, and audio analysis, and the image analysis includes face detection, expression recognition, and metadata analysis. Video analysis may be analysis that extracts metadata included in a video file. Audio analysis may be analysis that extracts tags included in an audio file. Cloud servers can also perform image analysis, video analysis, and audio analysis functions, and most functions are performed on the cloud server. Detailed technologies performed on the cloud server include image classification, attention map generation, object detection, motion analysis, cut detection, keyframe analysis, action recognition, facial expression recognition, and audio segmentation. Most technologies can be implemented using deep learning technology.

<Image analysis module>

The function of the image analysis module is to analyze the input image data, classify the image data, recognize objects included in the image, and generate an attention map. The resulting data is saved as metadata, and a video is created using the classified data in the video creation module. The image analysis module can increase accuracy by using a CNN-based learning model. Pretrained learning models can use ImageNet, and performance can be improved by additionally learning target image category data.

<Image classification>

The information needed to create a video is not only image classification information and object information included in the content, but also an attention map is needed to create a video containing motion based on various themes. The attention map is calculated using Local Image Patch and uses N basis functions. Basis functions are obtained by calculating from multiple natural images using the Infomax ICA (Independent Component Analysis) technique. All regions of the input image are multiplied by the Basis function to obtain N feature maps. Through kernel density estimation, a probability distribution can be obtained using each coefficient value of the feature map, and using this, the likelihood value of each pixel can be calculated. The combined likelihood is calculated using the likelihood values of the same pixel location in each feature map, and this can be converted into a self-information value using Shannon's theorem. This Self-Information value becomes the attention value of each pixel location.

<Object Search>

The object search technique in the image can remove the fully connected layer from the CNN structure consisting of a convolutional layer and a fully connected layer and use only the 2D feature map of the last convolutional layer. Only the highest values are extracted from the feature map to create a K-dimensional feature vector. These feature vectors are calculated for the query and reference images respectively, and the reference image closest to the query is found using cosine similarity. This method can use the learned model for image classification without additional learning.

<Video analysis module>

Analyze the motion of the video and use this to estimate the movement of the video camera. Camera movement can be classified as no movement, zoom in, zoom out, camera movement, and object-centered camera movement. First, movement can be estimated by calculating optical flow between video frames. Among the estimated motions, motions smaller than a certain size are determined as no motion, and the remaining motions are quantized into representative directions by dividing 360° into 8 directions. The movements converted into these 9 values can be accumulated for each mode to obtain a movement histogram. The method for estimating camera movement through the motion histogram may be as shown in Table 4 below.

One	no movement	When the histogram value of Mode 0 is the largest
2	Zoom in, zoom out	When not in the case of -1 -If all histogram values in 8 directions have values above a certain percentage
2-1	zoom in	When the image is divided into two along the vertical axis and the sum of the values of Mode 1~3 in the upper area and Mode 5~7 in the lower area is greater than the sum of the values of Mode 5~7 in the upper area and Mode 1~3 in the lower area.
2-2	zoom out	If not 2-1
3	Camera movement, object-centered camera movement	1, 2, none of them apply
3-1	Object-centered camera movement	When the ratio of the sum of the 8-way histogram values and the Mode 0 value is above a certain level
3-2	Camera movement	If not 3-1

<Scene change detection and key frame detection>

A keyframe refers to a representative frame in a video and contains a summary of the video content. By using keyframes, you can express the main content of video data. Eigenvalue can be obtained to calculate dissimilarity in consecutive frames of fixed size. The input image can be converted using an SVD-based image compression algorithm, using the image data matrix and left and right singular vectors.

<Facial expression recognition module>

Facial expression recognition is a representative method of extracting emotional expressions included in multimedia data, and is information that can be used to select themes for providing content. The facial expression recognition module operates in the management service providing server 300 and the user terminal 100, and may be optimally designed for operation in the user terminal 100. For facial expression data augmentation, the number of learning datasets can be increased by converting face angles into multiple angles. A basic learning model can be designed using Alexnet. When driving the facial expression recognition module using the Alexnet model, a relatively long operation time is required for operation on the user terminal 100, which is a small terminal. Therefore, model optimization is essential, so the structure of Alexnet can be optimized for each layer.

<Behavior recognition module>

Various behaviors within multimedia content are analyzed and inter-object behavior information is used for content analysis and video creation. The structure for action recognition can be a 3D CNN structure using the Long Term Recurrent Convolutional Network technique. For example, a 2D CNN structure can be transformed into a 3D CNN and used. The number of consecutive frames input to an independent 3D CNN can be defined as N, and the number of steps referring to the values coming out of each 3D CNN can be defined as T. In this case, N × T frames are divided from consecutive frames. When selecting, the interval between starting frames is expressed as Stride.

<Audio module>

Audio files can be used by synchronizing the start and end of scene changes or effects when creating a video. To extract video bits, you can use Mel-Frequency Cepstral Coefficient and Harmonic Patch ClassProfile. Afterwards, F1 accuracy can be extracted by dividing the audio data through Smoothing and Normalizing techniques.

The social relationship reflection unit 380 builds at least one editing tool for each category that is applied according to at least one social relationship, and when selecting a target to transmit a shared QR code in the user terminal 100, a category according to the target At least one editing tool already stored in can be provided.

The crowdsourcing unit 390 may build a database by uploading a template applied according to at least one social relationship from at least one designer terminal. At this time, after improving the accuracy of shot size classification through data learning, patterns for each social relationship can be built into a visualization model and a characteristic template database for each social relationship can be built. The following process can be further performed so that the image or captured video stored in the user terminal 100 can be randomly selected and matched with a template with social relationship characteristics to be automatically generated without an editing process.

<Data learning>

Finally, in order to provide an automatic video editing application with the necessary social relationship characteristics, it is important to secure N class image data classified by shot size. Korea's unique funeral will and the characteristics of each relationship, for example, whether it is a video sent by a granddaughter to her grandfather, a video sent by a grandfather to his granddaughter, a video sent between relatives of the same age, a video sent between sisters or brothers, and a video sent by a child to his parents. Class classification training can be performed using the CNN-based VGG16 model for image classification depending on whether the image is being sent. The reason for using VGG16's convolution layer is that it supports diversity of image location information and size. It is necessary to inspect the optimizer and the functions used to prevent overfitting and improve learning speed due to data and class expansion. As a multi-class activation function, Relu and Softmax can be used instead of Siagmod, and overfitting can be prevented with dropout. Accuracy can be measured by adjusting the optimizer value and steps (stepsize), and additionally, AlexNet's binary classification model can be used.

<Visualization of characteristic patterns by social relationship>

You can use the Firebase real-time database for a matching system that reflects video characteristics by social relationship. In the Firebase database, the attribute values of images can be converted into metadata, and the type of each social relationship characteristic can be specified. These designated models can be loaded first, or the user can be asked to first select a model that applies social relationship characteristics. After selecting the image that the user wants to make into a video among the videos or photos taken by the user, the user can obtain a video result with social relationship characteristics by selecting one of M patterns.

<Application>

It can be used as a CoreData method that supports a pattern model (graph) format to directly store data within the application, but if the computing resources or networking resources of the user terminal 100 do not meet the specifications of the platform of the present invention Speed may slow down significantly. Accordingly, it can be built with a Firebase real-time database that supports storage. CSV files can be converted to JSON files in the Firebase Back-End database environment and then stored in database storage, organized and stored according to patterns by social relationship. When the user presses the capture or photo upload button on the application screen, the photos taken or selected by the user are uploaded to a designated folder in Firebase and can be connected to a pattern model with characteristics for each social relationship stored in Firebase.

Hereinafter, the operation process according to the configuration of the management service providing server of FIG. 2 described above will be described in detail using FIGS. 3 and 4 as an example. However, it will be apparent that the embodiment is only one of various embodiments of the present invention and is not limited thereto.

The platform (tentative name, Heart Messenger) of the present invention as shown in Figure 3a, captures a video at the user terminal 100 as shown in Figure 3b and sends a gift containing a shared QR code to the address of the shared QR code, and sends a gift to the shared QR code to the shared recipient's terminal 400. ) receives streaming video uploaded from the user terminal 100 by scanning the shared QR code. At this time, as shown in Figure 3c, the sticker QR code (tentative name, heart messenger sticker) can be sold in the form of a sticker through offline in B2B and B2C markets, and can be linked to various online platforms as shown in Figure 3d. To summarize, it can be designed so that it can be provided as shown in Figure 3e, in the case of free services as shown in Figure 3f, and in the case of basic and premium paid services as shown in Figure 3g. Of course, the types of services are not limited to those described above and may change depending on the embodiment.

In addition, quarterly, seasonal, and limited edition sticker QR codes can be issued as shown in Figure 3h, and time-series private videos can be stored, played, and accessed through the platform of the present invention, taking social networks into account, as shown in Figures 3i and 3j. You can. The membership registration process may be as shown in Figure 4A, and a shared QR code may be generated as shown in Figure 4B. The payment process for paid members may be as shown in FIGS. 4C and 4D.

Matters that are not explained about the method of providing a private video management service using the QR code of FIGS. 2 to 4 are the same as or have not been explained previously with respect to the method of providing a private video management service using a QR code through FIG. 1. Since it can be easily inferred from the content, the description below will be omitted.

FIG. 5 is a diagram illustrating a process in which data is transmitted and received between components included in the private video management service providing system using the QR code of FIG. 1 according to an embodiment of the present invention. Hereinafter, an example of the process of transmitting and receiving data between each component will be described with reference to FIG. 5, but the present application is not limited to this embodiment, and the process shown in FIG. 5 according to the various embodiments described above It is obvious to those skilled in the art that the process of transmitting and receiving data can be changed.

Referring to FIG. 5, the management service providing server registers a previously purchased sticker QR code from the user terminal (S5100) and receives the captured image when the user terminal captures the image (S5200).

In addition, when the management service providing server requests the creation of a shared QR code from the user terminal, it generates a preset number of shared QR codes according to the level of the user terminal (S5300) and scans the shared QR code from the shared terminal. In this case, access rights are granted to the video captured by the shared terminal (S5400).

The sequence between the above-described steps (S5100 to S5400) is only an example and is not limited thereto. That is, the order between the above-described steps (S5100 to S5400) may change, and some of the steps may be executed simultaneously or deleted.

Matters that are not explained regarding the method of providing a private video management service using the QR code of FIG. 5 are the same as or have not been explained previously with respect to the method of providing a private video management service using a QR code through FIGS. 1 to 4. Since it can be easily inferred from the content, the description below will be omitted.

The method of providing a private video management service using a QR code according to an embodiment described with reference to FIG. 5 is also implemented in the form of a recording medium containing instructions executable by a computer, such as an application or program module executed by a computer. It can be. Computer-readable media can be any available media that can be accessed by a computer and includes both volatile and non-volatile media, removable and non-removable media. Additionally, computer-readable media may include all computer storage media. Computer storage media includes both volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data.

The method of providing a private video management service using a QR code according to an embodiment of the present invention described above includes an application installed by default on a terminal (this may include programs included in a platform or operating system, etc., installed by default on the terminal). It may be executed by, or may be executed by an application (i.e., program) installed by the user directly on the master terminal through an application providing server such as an application store server, an application, or a web server related to the service. In this sense, the method of providing a private video management service using a QR code according to an embodiment of the present invention described above is implemented as an application (i.e., a program) installed by default in the terminal or directly installed by the user and is installed on a computer such as the terminal. It can be recorded on a readable recording medium.

The description of the present invention described above is for illustrative purposes, and those skilled in the art will understand that the present invention can be easily modified into other specific forms without changing the technical idea or essential features of the present invention. will be. Therefore, the embodiments described above should be understood in all respects as illustrative and not restrictive. For example, each component described as single may be implemented in a distributed manner, and similarly, components described as distributed may also be implemented in a combined form.

The scope of the present invention is indicated by the claims described below rather than the detailed description above, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be construed as being included in the scope of the present invention. do.

Claims (7)

1. A user terminal that registers a previously purchased sticker QR code, takes a video, and requests creation of a shared QR code;

   A shared terminal that scans the shared QR code to receive access to the video captured by the user terminal and streams the captured video; and

   A purchase confirmation unit that registers a previously purchased sticker QR code from the user terminal, an upload unit that receives the captured video when the user terminal captures a video, and a request for creation of a shared QR code on the user terminal, the user A shared link generator that generates a preset number of shared QR codes according to the level of the terminal, and an access right unit that grants access to the captured video to the shared terminal when the shared QR code is scanned by the shared terminal. Management service provider servers, including whether;

   A private video management service provision system using QR code including.
2. According to claim 1,

   When purchasing a product from at least one shopping mall on the user terminal, if the option of the sticker QR code or the shared QR code is selected, the product is delivered by including the sticker QR code or the shared QR code in the delivery option along with the product. A system for providing a private video management service using a QR code, wherein the sticker QR code or shared QR code is printed, attached, or included in a delivery box when delivered.
3. According to claim 1,

   The management service providing server is,

   When purchasing a product to be sent along with the captured video at the user terminal, a gift delivery unit that processes an order to attach the shared QR code to a box of the product and transmit it;

   A system for providing a private video management service using a QR code, further comprising:
4. According to claim 1,

   The management service providing server is,

   Depending on the grade of the user terminal, the type and number of editing tools applied to the captured video, the number and capacity to generate the shared QR code, and the number and period of access to the video shared with the shared QR code are differentially set. grade classification unit;

   A system for providing a private video management service using a QR code, further comprising:
5. According to claim 1,

   The management service providing server is,

   The video captured by the user terminal is uploaded and stored on a preset cloud server, and when the shared QR code is scanned by the shared terminal, access rights to the captured video of the user terminal within the preset cloud server are granted. Cloud management department;

   A system for providing a private video management service using a QR code, further comprising:
6. According to claim 1,

   The management service providing server is,

   At least one editing tool applied according to at least one social relationship is constructed for each category, and when the user terminal selects a target to transmit the shared QR code, at least one editing tool pre-stored in a category according to the target is provided. Social Relations Reflection Department;

   A system for providing a private video management service using a QR code, further comprising:
7. According to claim 5,

   The management service providing server is,

   A crowdsourcing unit that builds a database by uploading a template applied according to the at least one social relationship from at least one designer terminal;

   A system for providing a private video management service using a QR code, further comprising:

Images