KR102645143B1 - Method, apparatus and system for generating and providing responsive user manual contents based on user interaction - Google Patents

Abstract

A device according to an embodiment acquires a first image of a product in a first state, a second image of a product in a second state, and a third image of a product in a third state. Output the first image, receive a user input for the first image, and if the user input is the first user input, convert the first image to the second image and output it, and if the user input is the second user input In this case, the first image can be converted into a third image and output.

Description

Method, device, and system for generating and providing responsive user manual content based on user interaction {METHOD, APPARATUS AND SYSTEM FOR GENERATING AND PROVIDING RESPONSIVE USER MANUAL CONTENTS BASED ON USER INTERACTION}

The following embodiments relate to technology for generating and providing responsive user manual content based on user interaction.

The product's manual is intended to provide visual guidance on how to use the product or explain its functions. By showing the various functions and usage methods of the product, it increases understanding of the product's functions and usage methods and allows users to operate more smoothly. You can use the product.

When purchasing a product, consumers are provided with picture and text manuals to guide them on how to assemble, install, and operate the product. However, existing manuals had the problem of being difficult to understand intuitively.

Accordingly, in order to provide the manual more intuitively, a method of providing guidance through video or audio was researched and developed, but there was a limitation in that the manual could not be provided according to the user's level of understanding or response by unilaterally providing video or audio to the user. .

Accordingly, there is a need for the development of technology for creating and providing responsive user manual content based on user interaction.

Republic of Korea Patent No. 10-2261828 (announced on 2021.06.08) Republic of Korea Patent Publication No. 10-2022-0096808 (published on July 7, 2022) Republic of Korea Patent No. 10-1699491 (announced on January 24, 2017) Republic of Korea Patent Publication No. 10-2021-0076775 (published on June 24, 2021)

Embodiments seek to provide a method for creating and providing responsive user manual content based on user interaction.

Embodiments attempt to minimize visual differences by generating a responsive image by converting an image of a product based on user input and generating an intermediate image in the process of converting the image.

Embodiments seek to provide additional product information in a pop-up form based on additional user input.

According to one embodiment, a method performed by an apparatus includes acquiring a first image of a product in a first state; Obtaining a second image of the product in a second state; Obtaining a third image of the product in a third state; outputting the first image; Receiving user input for the first image; If the user input is a first user input, converting the first image into the second image and outputting it; and, when the user input is a second user input, converting the first image into the third image and outputting the image.

The second state is a state in which at least a part of the product is visually distinguished from the first state, and the third state may include a state in which at least a part of the product is enlarged or reduced compared to the first state. You can.

In the step of converting and outputting the first image into the second image, in order to minimize the visual difference between the first image and the second image, the first image and the second image are applied to a pre-trained artificial intelligence model. Inputting an image and outputting an intermediate image, extracting a common part and a difference part between the first image and the second image and the intermediate image, and maintaining the common part in the intermediate image and the difference part The step of modifying the intermediate image by blurring may further be included.

The intermediate image compares the first image and the second image, calculates the degree of difference between the first image and the second image, and determines the number of intermediate images to be output based on the degree of difference, and the user input Extract keywords from the sound input, check the number of product keywords related to the product among the keywords, and apply a weight determined to be proportional to the number of product keywords while being inversely proportional to the number output at a preset reference time. The output maintenance time may be determined.

After converting the first image into the third image and outputting it, when additional user input for the third image is received, outputting the product manual image in a pop-up form at a first size; If the touch intensity of the additional user input is more than a preset standard, the part information of the part included in the part of the product corresponding to the touch position and the sales page of the part are output in the form of a pop-up of a second size different from the first size. steps; And if the number of touches of the additional user input is more than a preset reference number, the method may further include outputting a product-related customer center connection information message.

The method performed by the device further includes the step of checking the user's understanding of the product's manual, and the step of checking the understanding includes obtaining a user image including the product and the user from a camera. , obtaining the user's gaze information from the user image, confirming the speed of change of the gaze, and dividing the part of the product that appears in the manual video that has changed more than a preset standard into a focused part. Determining, confirming the position of the gaze, determining whether the position of the gaze matches the position of the focus portion corresponding to the product, calculating the gaze time during which the position of the gaze remains at the position of the focus portion A step of determining gaze understanding based on the change speed, whether the location matches, and the gaze time, Obtaining expression information of the user from the user image, Affirming the user's expression corresponding to a positive state Classifying a negative expression corresponding to an expression and a negative state, calculating a positive duration for which a positive expression lasts and a negative duration for which the negative expression lasts from the facial expression information;

Confirming the positive intensity, which is the intensity of the positive facial expression, and the negative intensity, which is the intensity of the negative facial expression, from the facial expression information; calculating a positive value by applying a positive weight determined based on the positive intensity to the positive duration time; Calculating a negativity value by applying a negativity weight determined based on the negativity intensity to the negativity duration, determining facial expression understanding based on a ratio of the positive value and the negative value, and voice information of the user from the user image. Obtaining, confirming the intensity of sound from the voice information, extracting a keyword from the voice information, confirming the number of product keywords related to the product among the keywords, the intensity of the sound and the product keyword determining a voice understandability based on the number of steps, determining the user's understanding level based on the gaze understanding level, the facial expression understanding level, and the voice understanding level, and the length of the manual video and the number of steps according to the understanding level. It may include a step of adjusting the user's level of understanding by applying a weight determined based on .

The method performed by the device further includes the step of generating manual content for the product based on the level of understanding, and the step of generating the manual content for the product includes reproducing the manual content in inverse proportion to the level of gaze understanding. determining the speed, determining the number of repetitions of the manual content in inverse proportion to the facial expression understandability, determining the intensity of a voice explaining the manual content based on the voice understandability, comparing the product and the user image to the user image. It may include checking the distance between users, and determining an enlargement ratio of the product in the manual content in inverse proportion to the distance between the product and the user.

The method performed by the apparatus includes: confirming sales revenue of the product; Approving NFT production for the manual video of the product based on the sales revenue being more than a preset standard; Clipping a section with a high viewership per minute of the manual video to create an image for NFT production; Creating an NFT using the image; And it may further include uploading the NFT to the market and distributing a preset ratio of the sales revenue of the NFT to the seller of the product and the producer who created the manual video.

The device according to one embodiment may be combined with hardware and controlled by a computer program stored in a medium to execute any one of the above-described methods.

Embodiments may provide methods for creating and providing responsive user manual content based on user interaction.

Embodiments may generate a responsive image by converting an image taken of a product based on user input, and generate an intermediate image in the process of converting the image to minimize visual differences.

Embodiments may provide additional product information in the form of a pop-up based on additional user input.

1 is a diagram for explaining the configuration of a system according to an embodiment.
Figure 2 is a flowchart illustrating a process for creating and providing responsive user manual content based on user interaction according to an embodiment.
Figure 3 is a flowchart for explaining the process of converting a first image into a second image and outputting it, according to an embodiment.
Figure 4 is a flowchart for explaining the process of determining the output number of intermediate images and the thrust maintenance time according to one embodiment.
Figure 5 is a flowchart for explaining a process of converting a first image into a third image and outputting it, according to an embodiment.
Figure 6 is a flowchart for explaining the process of determining gaze understanding according to an embodiment.
Figure 7 is a flowchart for explaining the process of determining facial expression understanding according to one embodiment.
Figure 8 is a flowchart for explaining the process of checking the user's understanding of the voice understanding and the product manual according to one embodiment.
Figure 9 is a flow chart to explain the process of generating manual content for a product based on understanding according to an embodiment.
Figure 10 is a flow chart to explain the process of distributing NFT sales revenue according to an embodiment.
11 is an exemplary diagram of the configuration of a device according to an embodiment.

Hereinafter, embodiments will be described in detail with reference to the attached drawings. However, various changes can be made to the embodiments, so the scope of the patent application is not limited or limited by these embodiments. It should be understood that all changes, equivalents, or substitutes for the embodiments are included in the scope of rights.

Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only and may be modified and implemented in various forms. Accordingly, the embodiments are not limited to the specific disclosed form, and the scope of the present specification includes changes, equivalents, or substitutes included in the technical spirit.

Terms such as first or second may be used to describe various components, but these terms should be interpreted only for the purpose of distinguishing one component from another component. For example, a first component may be named a second component, and similarly, the second component may also be named a first component.

When a component is referred to as being “connected” to another component, it should be understood that it may be directly connected or connected to the other component, but that other components may exist in between.

The terms used in the examples are for descriptive purposes only and should not be construed as limiting. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by a person of ordinary skill in the technical field to which the embodiments belong. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and unless explicitly defined in the present application, should not be interpreted in an ideal or excessively formal sense. No.

In addition, when describing with reference to the accompanying drawings, identical components will be assigned the same reference numerals regardless of the reference numerals, and overlapping descriptions thereof will be omitted. In describing the embodiments, if it is determined that detailed descriptions of related known technologies may unnecessarily obscure the gist of the embodiments, the detailed descriptions are omitted.

Embodiments may be implemented in various types of products such as personal computers, laptop computers, tablet computers, smart phones, televisions, smart home appliances, intelligent vehicles, kiosks, and wearable devices.

1 is a diagram for explaining the configuration of a system according to an embodiment.

The system according to one embodiment may include a device 30 capable of communicating through a communication network.

First, a communication network can be configured regardless of the communication mode, such as wired or wireless, and can be implemented in various forms to enable communication between servers and between servers and terminals.

The singular expressions recited in the claims may be understood to include the plural. For example, the user in a claim may refer to one user or two or more users.

The device 30 may be its own server owned by a person or organization that provides services using the device 30, a cloud server, or a p2p (peer-to-peer) collection of distributed nodes. It may be possible. The device 30 may be configured to perform all or part of the calculation function, storage/reference function, input/output function, and control function of a typical computer.

According to one embodiment, device 30 may include a display for generating output related to vision, hearing, or tactile sensation. The display may be implemented as a touch screen, for example, by forming a layered structure or integrated with the touch sensor, but is not limited thereto. This touch screen may perform the function of providing a user input interface between the device 30 and the user, and may provide an output interface between the device 30 and the user.

Device 30 may provide responsive user manual content through a device that includes a display. The device 30 may be equipped with a display, but is not limited to this, and the display may be configured separately from the device 30.

At this time, a device including a display may mean any electronic device equipped with a display, for example, a smartphone, laptop, tablet PC, touch screen, etc.

The device 30 may be configured to communicate wired or wirelessly with a device including a display, control the operation of the device including the display, and control which information to display on the screen of the device including the display. .

Device 30 may generate and provide responsive user manual content based on interaction with the user. At this time, the process of creating responsive user manual content will be described later with reference to FIG. 2.

The following effects can be achieved through the process of creating and providing responsive user manual content based on user interaction.

The device 30 can provide manual content customized to the needs and preferences of individual users by generating manual content through interaction with the user, and the user can find information tailored to his or her situation and needs and solve problems. You can obtain a solution and receive an efficient experience.

Additionally, responsive user manual content provides information through user interaction, allowing users to obtain necessary information in an intuitive and easy manner. For example, when a user wants to perform a specific task, he or she can easily find a description of that task.

Manual content created through user interaction can be updated in real time, and when there is a product upgrade, new function addition, or change in problem solving method, it can be quickly reflected to the user to provide the latest information. .

Manual content created through interaction with users can help users better understand and learn the product, and users can experience the product's functions and operation methods through direct participation and easily obtain the necessary information. It can be acquired.

Interaction-based manual content can promote user participation and improve users' problem-solving skills. This has the effect of allowing users to utilize the product more effectively by directly interacting with each other, solving problems, and utilizing the information provided in the manual.

In other words, the device 30 can increase user satisfaction through these effects and improve the efficiency and convenience of product use, and improve the product or manual content based on user data obtained through interaction with the user. The quality of product manual content can be improved.

According to one embodiment, a database may be provided within the device 30, but the present invention is not limited to this, and the database may be configured separately from the device 30. Device 30 may include multiple artificial neural networks for performing machine learning algorithms.

In the present invention, artificial intelligence (AI) refers to technology that imitates human learning ability, reasoning ability, perception ability, etc. and implements this with a computer, and includes concepts such as machine learning and symbolic logic. may include. Machine Learning (ML) is an algorithmic technology that classifies or learns the characteristics of input data on its own. Artificial intelligence technology is a machine learning algorithm that analyzes input data, learns the results of the analysis, and makes judgments or predictions based on the results of the learning. Additionally, technologies that mimic the functions of the human brain, such as cognition and judgment, using machine learning algorithms can also be understood as the category of artificial intelligence. For example, technical fields such as verbal understanding, visual understanding, reasoning/prediction, knowledge representation, and motion control may be included.

Machine learning can refer to the process of training a neural network model using experience processing data. Machine learning can mean that computer software improves its own data processing capabilities. A neural network model is built by modeling the correlation between data, and the correlation can be expressed by a plurality of parameters. A neural network model extracts and analyzes features from given data to derive correlations between data. Repeating this process to optimize the parameters of the neural network model can be called machine learning. For example, a neural network model can learn the mapping (correlation) between input and output for data given as input-output pairs. Alternatively, a neural network model may learn the relationships by deriving regularities between given data even when only input data is given.

An artificial intelligence learning model or neural network model may be designed to implement the human brain structure on a computer, and may include a plurality of network nodes with weights that simulate neurons of a human neural network. A plurality of network nodes may have a connection relationship with each other by simulating the synaptic activity of neurons in which neurons exchange signals through synapses. In an artificial intelligence learning model, multiple network nodes are located in layers of different depths and can exchange data according to convolutional connection relationships. The artificial intelligence learning model may be, for example, an artificial neural network model (Artificial Neural Network), a convolution neural network (CNN) model, etc. As an example, an artificial intelligence learning model may be machine-learned according to methods such as supervised learning, unsupervised learning, and reinforcement learning. Machine learning algorithms for performing machine learning include Decision Tree, Bayesian Network, Support Vector Machine, Artificial Neural Network, and Ada-boost. , Perceptron, Genetic Programming, Clustering, etc. can be used.

Among them, CNN is a type of multilayer perceptrons designed to use minimal preprocessing. CNN consists of one or several convolution layers and general artificial neural network layers on top of them, and additionally utilizes weights and pooling layers. Thanks to this structure, CNN can fully utilize input data with a two-dimensional structure. Compared to other deep learning structures, CNN shows good performance in both video and audio fields. CNNs can also be trained via standard back propagation. CNNs have the advantage of being easier to train and using fewer parameters than other feedforward artificial neural network techniques.

Convolutional networks are neural networks that contain sets of nodes with bound parameters. The increasing size of available training data and the availability of computational power, combined with algorithmic advances such as piecewise linear unit and dropout training, have led to significant improvements in many computer vision tasks. For extremely large data sets, such as those available for many tasks today, overfitting is not critical, and increasing the size of the network improves test accuracy. Optimal use of computing resources becomes a limiting factor. For this purpose, distributed, scalable implementations of deep neural networks can be used.

The embodiments described above may be implemented with hardware components, software components, and/or a combination of hardware components and software components. For example, the devices, methods, and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, and a field programmable gate (FPGA). It may be implemented using one or more general-purpose or special-purpose computers, such as an array, programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. A processing device may execute an operating system (OS) and one or more software applications that run on the operating system. Additionally, a processing device may access, store, manipulate, process, and generate data in response to the execution of software. For ease of understanding, a single processing device may be described as being used; however, those skilled in the art will understand that a processing device includes multiple processing elements and/or multiple types of processing elements. It can be seen that it may include. For example, a processing device may include a plurality of processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are possible.

Figure 2 is a flowchart illustrating a process for creating and providing responsive user manual content based on user interaction according to an embodiment.

Referring to FIG. 2, first, in step S201, the device 30 may acquire a first image of a product in a first state.

At this time, the first state may mean the basic state of the product. For example, if the product is a desk, the first state may include a state in which assembly of the desk has been completed and all parts are correctly combined, and the first image may refer to an image of the entire desk with complete assembly. However, this is only an example and is not limited to this.

According to one embodiment, the device 30 may receive a first image of a product in a first state from a user terminal of a user who wants to assemble and use the product based on a manual. According to another embodiment, the device 30 may collect a first image of a product in a first state from a website where the product is sold. At this time, the process of acquiring the first image is not limited to this and can be performed in various ways.

In step S202, the device 30 may acquire a second image of the product in the second state.

At this time, the second state may include a state in which at least a portion of the product is visually distinguished from the first state. For example, if the product is a desk, the second image may represent an image of only the legs of the desk, but this is only an example and is not limited thereto.

According to one embodiment, the device 30 may receive a second image of a product in a second state from a user terminal of a user who wants to assemble and use the product based on a manual. According to another embodiment, the device 30 may collect a second image of a product in a second state from a website where the product is sold. At this time, the process of acquiring the second image is not limited to this and can be performed in various ways.

In step S203, the device 30 may acquire a third image of the product in the third state.

At this time, the third state may include a state in which at least part of the product is enlarged or reduced compared to the first state. For example, if the product is a desk, the third image may be an image taken by enlarging the legs of the desk, but this is only an example and is not limited thereto.

According to one embodiment, the device 30 may receive a third image of a product in a third state from a user terminal of a user who wants to assemble and use the product based on a manual. According to another embodiment, the device 30 may collect a third image of a product in a third state from a website where the product is sold. At this time, the process of acquiring the third image is not limited to this and can be performed in various ways.

At this time, the description from the first state to the third state is only an example, and the state of the product is not limited to this, and the state of the product may be further divided depending on the characteristics of the product, the complexity of assembly, etc. For example, the state of the product may include a first state to a tenth state.

In step S204, the device 30 may output the first image.

Device 30 may output the first image through a device including a display. The device 30 may be equipped with a display, but is not limited to this, and the display may be configured separately from the device 30. According to one embodiment, a device including a display may include a user's terminal.

For example, the device 30 may control the first image to be output on the screen of the user's terminal.

In step S205, the device 30 may receive user input.

At this time, the user input may include at least one of touch input, proximity input, and sound input. At this time, the device 30 may store the received user input in the database.

In step S206, if the user input is the first user input, the device 30 may convert the first image into a second image and output it.

At this time, the first user input may include at least one of a touch input, a proximity input, and a sound input, and the first user input may be set differently depending on the embodiment.

For example, touch input includes tap (touching the screen briefly), swipe (moving a finger across the screen), pinch (reducing or enlarging the screen with two fingers), and drag. It may include drag (moving an object by sliding your finger across the screen), long press (long pressing the screen), double tap (double tap: quickly tapping twice), etc. It is not limited.

For example, proximity input may include, but is not limited to, an action of placing a hand close to a device including a display.

For example, sound input may include a case where a user provides a command or input by voice. The device 30 can recognize the user's voice through voice recognition technology and process the corresponding command.

For example, when the first user input is a tap among touch inputs, the device 30 may convert the first image into a second image and output it. If the product is a desk and the user input is the first user input, the device 30 converts the first image, which is an image of the entire assembled desk, into a second image, which is an image of the entire assembled desk, and outputs it. You can.

When the user input is the first user input, the device 30 converts the first image into a second image and outputs it, making it easier for the user to recognize and understand changes in the state of the product, such as product assembly or function changes. It can convey information clearly, improve user convenience, and help users utilize the product properly.

In step S207, if the user input is a second user input, the device 30 may convert the first image into a third image and output it.

For example, when the second user input is a long press among touch inputs, the device 30 may convert the first image into a third image and output it. If the product is a desk and the user input is a second user input, the device 30 converts the first image, which is an image of the entire assembled desk, into a third image, which is an image taken by enlarging the legs of the desk. Can be printed.

According to one embodiment, when the second user input is a long press, the device 30 checks the time for pressing the screen, and when the time for pressing the screen is longer than a preset reference time, the device 30 checks each part of the leg portion of the desk. The second image can be converted into a third image including an exploded perspective view in which the disassembled image is captured and output. At this time, the preset reference time may be set differently depending on the embodiment.

According to one embodiment, when the second user input is a double tap, the device 30 may convert the second image into a third image including an image of an enlarged view of a part of a desk leg and output the converted image. there is.

When the user input is a second user input, the device 30 can provide more information to the user by showing various states of the product through the process of converting the first image into a third image and outputting it. For example, by enlarging or reducing the first image and converting it to a third image, a specific part of the product can be emphasized or details expressed in more detail. This can help users understand the various functions or characteristics of the product and obtain the necessary information efficiently.

Figure 3 is a flowchart for explaining the process of converting a first image into a second image and outputting it, according to an embodiment.

Referring to FIG. 3, first, in step S301, the device 30 inputs the first image and the second image into a pre-trained artificial intelligence model to minimize the visual difference between the first image and the second image. An intermediate image can be output.

At this time, the device 30 may train an artificial intelligence model in advance to output an intermediate image representing an intermediate state between the first image and the second image in order to minimize the visual difference between the first image and the second image.

The device 30 can configure learning data using a pair of a first image and a second image to learn an artificial intelligence model, and each image is preprocessed to an appropriate size and prepared to be input into the model. At this time, the device 30 may collect learning data by collecting a pair of the first image and the second image from a sales site that sells a specific product and a specific product including a manual image of the specific product. The process is not limited to this and can be performed in various ways.

The device 30 can train an artificial intelligence model using prepared training data, and the learned model can receive the first image and the second image as input and generate an intermediate image. At this time, the artificial intelligence model can be trained to generate an intermediate image in a way that minimizes the visual difference between the first image and the second image.

For example, the artificial intelligence model may be set to at least one of an autoencoder, a generative adversarial network (GAN), and a variational autoencoder (VAE), but the artificial intelligence model may be different. Models can be used, and various models can be selected as artificial intelligence models depending on the characteristics of the learning data, purpose of the conversion process, performance requirements, etc.

An autoencoder is a neural network model consisting of an encoder that compresses input data and a decoder that restores the compressed expression. The first image can be input to the encoder and the encoded expression can be input to the decoder to generate the second image. The encoder generates an intermediate image while preserving the information of the input image as much as possible, so it can be used to minimize visual differences.

GAN is a model composed of a generator and a discriminator. The generator receives the first image as input and creates a second image, and the discriminator plays the role of distinguishing the generated image from the actual second image. The generator and discriminator are adjusted to generate a visually realistic second image through competitive learning.

VAE is an extended form of autoencoder and is a model that reconstructs the input image using latent variables. VAE receives the first image as input, generates latent variables, and can use them to generate the second image. Since it performs sampling in a limited latent space, it can generate various intermediate images.

The device 30 may generate an intermediate image by providing the first image and the second image as input to the learned artificial intelligence model. The artificial intelligence model can generate an intermediate image by identifying the visual characteristics and correlations of the input image, and can adjust the intermediate image to minimize the visual difference between the first image and the second image.

After generating the intermediate image, device 30 may perform additional post-processing steps to minimize visual differences. For example, the device 30 may adjust the color, brightness, contrast, etc. of the image to increase consistency between the first image and the second image.

By outputting the generated intermediate image to the user, the device 30 can minimize the visual difference between the first image and the second image and provide a natural change to the user.

In step S302, the device 30 may extract a common part and a difference part between the first image, the second image, and the middle image.

The device 30 may extract common and different parts of the first image and the second image using image processing and analysis techniques. For example, device 30 may utilize image comparison and difference detection techniques, for example, to compute pixel-level differences, or may use algorithms to detect differences in specific features or patterns, but may also utilize image comparison and difference detection techniques to determine differences in common features or patterns. The process of extracting the part is not limited to this and can be performed in various ways.

In step S303, the device 30 may modify the intermediate image by maintaining the common portion in the intermediate image and blurring the difference portion.

The device 30 can generate an intermediate image showing natural changes by maintaining common parts in the intermediate image and blurring the difference parts. The device 30 provides smooth and consistent visual changes to the user through the process of modifying the intermediate image, and can more clearly communicate changes in the state of the product.

The intermediate image can enable the user to understand and understand the conversion process, allow the user to smoothly perceive the change from the first image to the second image, and recognize changes in the state of the product.

Intermediate images visually convey changes in the state of the product and help users understand the changes. Users can recognize the various states of the product and obtain necessary information. The meaning of the middle image allows for the effective creation of responsive user manual content.

Figure 4 is a flowchart for explaining the process of determining the output number of intermediate images and the thrust maintenance time according to one embodiment.

Referring to FIG. 4 , in step S401, the device 30 may compare the first image and the second image and calculate the degree of difference between the first image and the second image.

For example, the device 30 may calculate the difference in pixels between the first image and the second image. Device 30 may calculate the difference by comparing the pixel values of each image, calculating the difference in RGB (or grayscale) values in each pixel of the first image and the second image, dividing this by the total number of pixels to calculate the average difference. The degree of difference between the first image and the second image can be calculated by obtaining .

In step S402, the device 30 may determine the number of intermediate images to be output based on the degree of difference.

According to one embodiment, the device 30 may determine whether the degree of difference between the first image and the second image is greater than or equal to a preset standard degree. At this time, the preset standard degree may be set differently depending on the embodiment.

If the difference between the first image and the second image is less than the standard, the device 30 can generate and output one intermediate image. If the difference between the first image and the second image is greater than the standard, an intermediate image can be generated and output. Multiple images can be created.

If the degree of difference between the first image and the second image is greater than or equal to a standard degree, the device 30 may determine the number of intermediate images to be output in proportion to the degree of difference.

If the device 30 determines that the difference between the first image and the second image is less than the standard level, it can generate and output one intermediate image, which is an intermediate image because the difference between the first image and the second image is relatively small. Applicable when the state is clearly defined. The process of creating and printing one intermediate image can naturally show the conversion process to the user and allow them to recognize changes in the state of the product.

On the other hand, the device 30 can generate and output a plurality of intermediate images when the degree of difference between the first image and the second image is greater than the standard level. This is because the change in state of the product is large and complex, so a single intermediate image is sufficient. It can be applied when it is difficult to express. The process of generating and outputting multiple intermediate images can show the user various stages of change and convey detailed state changes. This allows users to perceive and understand product changes in more detail.

Additionally, the process of generating and outputting multiple intermediate images can naturally convey changes in the state of the product while maintaining the continuity of transformation. Through these various intermediate images, users can visually understand the product's change process and determine whether they can proceed to the next step as needed. This can have the effect of improving the user's experience and increasing understanding of product use.

In step S403, the device 30 may extract a keyword from the sound input of the user.

The device 30 may collect the user's voice input from sound input during the user's input, and output keywords from the user's sound input through a process of converting the collected voice input into text. At this time, since the process of extracting keywords from voice input is self-evident to a person skilled in the art, a detailed explanation will be omitted.

In step S404, the device 30 may check the number of product keywords related to the product among the keywords.

At this time, product keywords related to the product may include, but are not limited to, the product name, function keyword, use keyword, technology keyword, attribute keyword, accessory keyword, related keyword, color keyword, etc.

Function keywords may refer to words or phrases that represent the main functions performed by the product, and may include, but are not limited to, automation, wireless, timer, security, power saving, sensor, touch, etc.

Use keywords may refer to words or phrases that express the purpose or purpose of use of a product, and may include, but are not limited to, cooking, cleaning, travel, office, exercise, etc.

Technology keywords may refer to words or phrases that indicate specific technologies or technical features used in a product, and may include, but are not limited to, AI, Bluetooth, waterproof, high resolution, touch screen, etc.

Attribute keywords may refer to words or phrases that express the physical characteristics or attributes of a product, and may include, for example, light and easy to carry, durable, comfortable, high performance, etc., but are not limited thereto.

Accessory keywords may refer to words or phrases that represent accessories or accessories provided with a product, and may include, but are not limited to, chargers, cases, cables, remote controls, adapters, screws, bolts, etc.

Related keywords may refer to words or phrases that represent specific concepts, fields, industries, etc. related to the product, and may include, but are not limited to, smart home, automobiles, music, health, environment, etc. .

A color keyword may refer to a word or phrase that represents the color of a product, and may include, for example, black, white, silver, blue, pink, etc., but is not limited thereto.

In step S405, the device 30 may determine the output maintenance time of the intermediate image by applying a weight determined to be proportional to the number of product keywords and inversely proportional to the number output at a preset reference time.

According to one embodiment, the device 30 sets the weight lower as the number of intermediate images output increases and the number of product keywords decreases, and as the number of intermediate images output decreases and the number of product keywords increases, the weight is set lower. can be set high.

The device 30 may determine the output maintenance time of the intermediate image by applying a weight to a preset reference time. At this time, the preset reference time may be set differently depending on the embodiment.

The output maintenance time of the intermediate image should be set to an appropriate time for the user to recognize and understand the change while maintaining the naturalness of the conversion. The output maintenance time must be adjusted.

The device 30 may measure the number of keywords related to a product extracted from the user input and proportionally adjust the output retention time of the intermediate image according to the number of product keywords. As the number of keywords increases, the output retention time of the intermediate image can be set longer so that the user can fully recognize and understand the information about the keyword.

The greater the number of keywords, the longer the intermediate image output remains, allowing users to recognize and understand changes without missing important information.

The device 30 can prevent images from switching too quickly by setting the output maintenance time of the intermediate image to be inversely proportional to the number of intermediate images. Too fast image transitions can make it difficult for users to perceive and understand changes, so visual reduction can be prevented by appropriately adjusting the output retention time of intermediate images.

The device 30 can increase the user's interest and concentration in each intermediate image by setting the output maintenance time of the intermediate image to be inversely proportional to the number of intermediate images. When a small number of intermediate images are output, the user can focus on the image during that time and can observe and understand the content and changes in the image in detail.

Figure 5 is a flowchart for explaining a process of converting a first image into a third image and outputting it, according to an embodiment.

Referring to FIG. 5 , in step S501, when an additional user input for the third image is received, the device 30 may output the product manual image in the form of a pop-up of the first size.

The device 30 may receive additional user input for the third image after converting the first image into the third image and outputting it.

At this time, additional user input may include touch input. At this time, the device 30 may store the received additional user input in the database.

At this time, the additional user input may include a touch input, and the additional user input may be set differently depending on the embodiment.

According to one embodiment, when the device 30 receives an additional user input including a touch input for the third image after converting the first image into the third image and outputting the third image, the device 30 displays the device including the display. The product's manual video can be output in the form of a pop-up of the first size. At this time, the first size may be set differently depending on the embodiment.

At this time, the device 30 can collect the product's manual video from a website that sells the product and store it in a database. However, the process of collecting the product's manual video is not limited to this and can be performed in various ways.

At this time, the product's manual video is a video that visually shows how to use or function of the product, and may include information on product introduction, product assembly and installation, adjustment methods, maintenance methods, etc., and the device 30 is provided with a manual video. From the video, you can check the total length of the manual video and each manual step for product assembly and installation.

When the device 30 receives an additional user input for the third image, it can immediately provide related information to the user by outputting a manual image in the form of a pop-up. Because pop-ups appear as small windows on the screen or in specific locations, users can easily focus their visual attention.

In addition, manual videos provided in the form of pop-ups can provide users with an intuitive interface, and by providing clear and focused information in a small window, users can quickly understand the operation, functions, and usage of the product.

The manual video provided in the form of a pop-up provides greater visibility to the user's field of view and allows the user to view the product and the manual video at the same time, making it easier to understand the relationship between the actual product operation and the manual.

Pop-up-type manual videos can provide a personalized user experience according to the user's needs and desires. The size and location of the pop-up and the content of the video can be adjusted to suit the user's preferences, allowing users to use the manual video to suit their needs.

In step S502, if the touch intensity of the additional user input is greater than or equal to a preset standard, the device 30 displays the part information of the part included in the part of the product corresponding to the touch position and the sales page of the part in the form of a pop-up of a second size. Can be printed. At this time, the preset standard may be set differently depending on the embodiment. At this time, the second size may be set differently depending on the embodiment.

If the additional user input is a touch input, the device 30 checks the touch intensity of the additional user input, and if the touch intensity is greater than or equal to a preset standard, the device 30 provides component information and component information of the part included in the part of the product corresponding to the touch location. A sales page selling a product may be output on the screen of a device including a display in the form of a pop-up of a second size.

The device 30 can collect product part information and link information of a sales page selling parts from a website selling the product and store them in a database, but the process of collecting part information and link information of a sales page is limited to this. It can be done in a variety of ways.

When the user strongly touches a specific part of the product, the device 30 can quickly provide information about the parts included in that part, and the user can immediately check the information about the necessary parts by directly touching the product. .

When the touch intensity is high, the device 30 can accurately recognize and identify the product part corresponding to the touch location and provide part information and sales page tailored to that part, and the user can obtain detailed information about the desired part. make it possible

The device 30 outputs part information and a sales page in a pop-up form of a second size, allowing the user to immediately purchase the part, and a link to move to the part's sales page is provided, so the user can conveniently go through the purchasing process. can be carried out and sales of product parts can be induced.

By providing parts information and sales pages in a pop-up format, users can see comprehensive information about product-related parts at a glance, improving users' understanding of the product and making it easier to find the parts they need.

In step S503, if the number of touches of the additional user input is more than a preset standard number, the device 30 may output a product-related customer center connection information message. At this time, the reference number of times may be set differently depending on the embodiment.

According to one embodiment, after converting the first image into a third image and outputting the device 30, if the number of touches of an additional user input including a touch input for the third image is more than a preset reference number, A product-related customer center connection information message can be displayed on the screen of a device that includes a display.

The device 30 may collect information about the product-related customer center from a website selling the product and store it in a database, but the process of collecting information about the product-related customer center is not limited to this and can be performed in various ways. You can. At this time, information about the product-related customer center may include, but is not limited to, information about the phone number, email, fax number, etc. of the product-related customer center on the website selling the product.

When a user is experiencing a problem or question related to a product, the device 30 can provide quick problem-solving support by outputting a customer center connection information message if the number of additional touches is more than the standard number. Users can receive help with their problems by connecting to the customer center, which can improve user satisfaction. Additionally, it can improve customer satisfaction and create a positive user experience. Customer trust can be increased by providing appropriate support even in situations where users cannot solve problems on their own.

Figure 6 is a flowchart for explaining the process of determining gaze understanding according to an embodiment.

Referring to FIG. 6, first, in step S601, the device 30 may acquire a user image including a product and a user from a camera.

A device including a display may include a camera, and the device 30 may acquire a user image including a product and a user and store the user image in a database.

In step S602, the device 30 may obtain the user's gaze information from the user's image. The device 30 may identify the position of the user's eyes from the user image and estimate the direction of the user's gaze by tracking the movement of the pupil.

In step S603, the device 30 can check the change speed of gaze.

In step S604, the device 30 may determine the part of the product that appears in the manual video that has changed more than a preset standard as the focused part. At this time, the preset standard may be set differently depending on the embodiment.

In step S605, the device 30 can check the position of the gaze.

In step S606, the device 30 may determine whether the position of the gaze matches the position of the focused portion corresponding to the product.

In step S607, the device 30 may calculate the gaze time during which the position of the gaze remains at the position of the focused portion.

In step S608, the device 30 may determine gaze understanding based on change speed, location coincidence, and gaze time.

According to one embodiment, the device 30 determines the gaze understanding to be higher as the change speed is slower, if the positions match, and the gaze time is longer, and as the change speed is slower, if the positions are mismatched, the gaze time is shorter. The lower the level of gaze understanding, the lower the level of gaze understanding can be determined.

At this time, gaze understanding may refer to a number representing the user's understanding of the product based on the user's gaze information. The higher the gaze understanding, the higher the understanding of the product, and the lower the gaze understanding, the lower the understanding of the product. It can mean something.

Figure 7 is a flowchart for explaining the process of determining facial expression understanding according to one embodiment.

Referring to FIG. 7, first, in step S701, the device 30 may obtain the user's facial expression information from the user image.

The device 30 may obtain information on the user's facial expression from the user's image using an algorithm for determining the facial expression or a deep learning-based facial expression recognition model. At this time, since the process of acquiring facial expression information from an image is self-evident to a person skilled in the art, a detailed explanation will be omitted.

In step S702, the device 30 may classify the user's expression into a positive expression corresponding to a positive state and a negative expression corresponding to a negative state.

The device 30 may classify the user's facial expression into a positive expression corresponding to a positive state and a negative expression corresponding to a negative state based on the user's facial expression information.

In step S703, the device 30 may calculate a positive duration for which a positive expression lasts and a negative duration for which a negative expression lasts from facial expression information.

In step S704, the device 30 may check the positive intensity, which is the intensity of a positive facial expression, and the negative intensity, which is the intensity of a negative facial expression, from the facial expression information.

The device 30 can determine the intensity of emotion by extracting features related to facial expressions. For example, facial expression features such as mouth angle, eye size, eyebrow movement, etc. can be extracted from the facial expression information of the user image, and emotional intensity can be determined based on the corresponding features. This feature analysis can be performed using statistical methods or machine learning techniques.

For example, the larger the angle of the mouth, the size of the eyes, and the movement of the eyebrows, the higher the positive and negative intensity corresponding to each emotional state can be determined.

In step S705, the device 30 may calculate a positivity value by applying a positivity weight determined based on the positivity intensity to the positivity duration.

The device 30 may determine the positivity weight in proportion to the positivity intensity and calculate the positivity value by applying the positivity weight to the positivity duration.

In step S706, the device 30 may calculate a denial value by applying a denial weight determined based on the denial intensity to the denial duration.

The device 30 can determine the denial weight in proportion to the denial intensity and calculate the denial value by applying the denial weight to the denial duration.

In step S707, the device 30 may determine the level of facial expression understanding based on the ratio of positive and negative values.

The device 30 can calculate the ratio of negative values to positive values. The lower the ratio of negative values to positive values, the higher the facial expression understanding is determined. The higher the positive numerical value to negative numerical ratio is, the lower the facial expression understanding is determined. You can.

At this time, facial expression understanding can refer to a number representing the user's understanding of the product based on the user's facial expression information. The higher the facial expression understanding, the higher the understanding of the product, and the lower the facial expression understanding, the lower the understanding of the product. It can mean something.

Figure 8 is a flowchart for explaining the process of checking the user's understanding of the voice understanding and the product manual according to one embodiment.

Referring to FIG. 8, first, in step S801, the device 30 may obtain the user's voice information from the user's image.

In step S802, the device 30 can check the intensity of the sound from the voice information.

In step S803, the device 30 may extract keywords from voice information.

In step S804, the device 30 may check the number of product keywords related to the product among the keywords.

In step S805, the device 30 may determine speech understandability based on the intensity of sound and the number of product keywords.

According to one embodiment, the device 30 may determine the speech understanding to be higher as the intensity of the sound is smaller and the number of product keywords is greater, and determine the speech understanding to be lower as the intensity of the sound is greater and the number of product keywords is smaller. there is.

At this time, voice understanding can refer to a value that represents the user's understanding of the product based on the user's voice information. The higher the voice understanding, the higher the understanding of the product, and the lower the voice understanding, the lower the understanding of the product. It can mean something.

In step S806, the device 30 may determine the user's understanding level based on gaze understanding, facial expression understanding, and voice understanding.

According to one embodiment, the device 30 may determine the user's understanding level by adding the values of gaze understanding, facial expression understanding, and voice understanding. The device 30 may determine that the higher the summed value, the higher the user's understanding of the product, and the lower the summed value, the lower the user's understanding of the product.

In step S807, the device 30 may adjust the user's level of understanding by applying a weight determined based on the length and number of steps of the manual image to the level of understanding.

The device 30 can check the length of the manual image and the number of manual steps from the manual image.

The device 30 may determine a weight in inverse proportion to the length of the manual image and the number of manual steps, and may adjust the user's level of understanding by applying the determined weight to the level of understanding.

The device 30 may adjust the user's level of understanding based on the length of the manual projection and the number of manual steps, reflecting that it will be relatively difficult to understand the product if the manual video is long or consists of many manual steps.

Figure 9 is a flow chart to explain the process of generating manual content for a product based on understanding according to an embodiment.

Referring to FIG. 9, first, in step S901, the device 30 may determine the playback speed of manual content in inverse proportion to gaze comprehension.

The device 30 can provide a learning experience tailored to each user's learning speed and level of understanding by adjusting the playback speed of manual content in inverse proportion to the level of gaze comprehension. This can help users understand and acquire content more effectively.

If the user's gaze comprehension is low, the device 30 lowers the playback speed of the manual content so that the user can more easily digest and understand the information, improve the user's learning efficiency, and help minimize boredom or confusion. You can.

The device 30 adjusts the playback speed of manual content in inverse proportion to gaze comprehension, allowing the user to focus more on the content. If the playback speed is appropriately adjusted according to the user's gaze and understanding, it is not too fast or too slow, maintaining the user's interest and creating an environment where they can focus on learning.

Additionally, by adjusting the playback speed of manual content according to the user's level of understanding, information tailored to individual needs and levels can be delivered.

In other words, through the above effect, the device 30 can provide a personalized learning experience tailored to the user's individual needs and learning level by adjusting the playback speed of the manual content in inverse proportion to the gaze comprehension, and can improve the user's learning efficiency and It can help improve understanding.

In step S902, the device 30 may determine the number of repetitions of the manual content in inverse proportion to the level of facial expression understanding.

According to one embodiment, when facial expression understanding is low, the device 30 determines that the user may not understand the manual content or is having difficulty, and provides the manual content repeatedly to help the user understand the content. I can give it. Increasing the number of repetitions allows users to have more opportunities to view and understand the content again, thereby improving comprehension.

Conversely, if the level of facial expression understanding is high, the device 30 may determine that the user quickly understands the manual content and move to the next step by reducing the number of repetitions. Since it is determined that the user has already fully understood, the number of repetitions can be reduced to provide the user with a more efficient learning experience.

Therefore, the process of determining the number of repetitions of manual content in inverse proportion to the understanding of facial expressions can have the effect of increasing the user's learning efficiency by adjusting the appropriate number of repetitions in consideration of the user's learning state.

In step S903, the device 30 may determine the intensity of the voice explaining the manual content based on the voice understandability.

According to one embodiment, when speech understanding is low, the intensity of the voice is increased to help the user hear clearly, and when speech understanding is high, the intensity of the voice is lowered to maintain clear speech transmission without causing inconvenience to the user. It can be adjusted to do so. Through this, users can conveniently understand manual content provided by voice and accurately grasp the information they need.

In step S904, the device 30 can check the distance between the product and the user from the user image.

In step S905, the device 30 may determine the enlargement ratio of the product in the manual content in inverse proportion to the distance between the product and the user.

Since the product may appear small when the device 30 is far away, the product can be displayed more clearly by increasing the magnification ratio of the product in the manual content, allowing the user to better understand the detailed information of the product.

Conversely, if the distance is close, the product may appear large, so reduce the enlargement ratio of the product in the manual content to prevent the product from exceeding the user's field of view, and the user can observe and operate the product more comfortably.

Therefore, the process of determining the enlargement ratio of the product in inverse proportion to the distance between the product and the user can improve the user's visual convenience and the clarity of product information.

Figure 10 is a flow chart to explain the process of distributing NFT sales revenue according to an embodiment.

According to one embodiment, the device 30 issues an NFT for a product's manual video, and distributes the NFT's sales revenue to the seller of the product and the producer who produced the product's manual video based on the sales revenue of the product. there is.

Referring to FIG. 10, first, in step S1001, the device 30 can check the sales revenue of the product.

According to one embodiment, the device 30 collects information on product sales volume, sales price, profit, etc. from a seller selling the product or a sales page selling the product, and uses the collected information to generate sales revenue for the product. can confirm. Additionally, information about the revenue generated by a product can be obtained directly from the seller, but the method of checking the sales revenue of the product is not limited to this.

In step S1002, the device 30 may approve NFT production for the product's manual video based on the sales revenue being greater than or equal to a preset standard. At this time, the preset standard may be set differently depending on the embodiment.

In the present invention, a non-fungible token (NFT) is a cryptocurrency in which it is impossible to replace one token with another. A unique ID and a unique ID are included in the content so that the unique identification code can be identified without changing even after several transactions. It may be an asset to which metadata information has been assigned.

According to one embodiment, creating an encrypted digital asset includes generating a unique encrypted digital asset code having an address, a public key, and a private key pair.

Non-Fungible Token (NFT) in the present invention is a digital asset certified with a unique bit that permanently records its authenticity and is stored in blockchain, a distributed ledger system that is the basis of cryptocurrency such as Ethereum. do.

It is similar to fungible cryptocurrency in that it is tokenized (or assetized) using blockchain technology. However, while cryptocurrency can be used by anyone like real money, NFT allows each digital asset to have a unique recognition value. Because it has it, it has irreplaceable characteristics.

Due to the nature of blockchain technology, these NFTs cannot be deleted or forged once created, and their ownership and transaction history are clearly stated, so they can be used as a type of digital certificate or ownership certificate. For example, ERC-721 on the Ethereum network is a non-fungible token (NFT).

The device 30 may compare sales revenue with a preset standard and approve NFT production for the product's manual video if the sales revenue is greater than or equal to the preset standard.

In step S1003, the device 30 may clip a section with a high viewership per minute of the product's manual video and create an image for NFT production.

In step S1004, the device 30 can create an NFT using an image.

According to one embodiment, the device 30 may obtain image registration information for issuing an NFT of an image. At this time, the image registration information may include at least one of the name of the product, the seller, the producer who produced the manual video, and the product's manual video. The device 30 may issue an NFT of an image from image registration information and generate NFT issuance information for the NFT of the issued image.

Here, the process of generating NFT issuance information involves issuing an NFT for image registration information that includes at least one of the product's name, seller, producer who produced the manual video, and the product's manual video. The process of issuing an NFT A small fee may be charged for NFT issuance.

At this time, the NFT issuance information may include, but is not limited to, information such as the name of the product for the image, the seller, the producer who produced the manual video, a description of the product's manual video, and the price of the NFT.

When the device 30 issues an NFT of an image and generates NFT issuance information, it can generate the NFT of the image as an encrypted digital asset. According to one embodiment, creating an encrypted digital asset includes generating a unique encrypted digital asset code having an address, a public key, and a private key pair.

According to one embodiment, the encrypted digital asset includes an identifier, which is a unique number assigned to the image. Therefore, encrypted digital assets can be matched one-to-one with images.

The device 30 can generate additional revenue from the sale of NFT in addition to the revenue earned from existing product sales by issuing the product's manual video as NFT. Since NFT has unique and limited characteristics, when demand is high Its value may rise.

In addition, the device 30 can create a new profit model by issuing a product manual video as an NFT because NFT represents ownership of a digital asset. For example, profits can be generated in various ways, such as providing special benefits to NFT owners or forming a community through the exchange of rights between NFT holders.

Additionally, the device 30 can improve the value of a company or brand selling a product by issuing a product manual video as an NFT. Because NFTs can have value as digital art, they can emphasize the uniqueness and differentiation of products and brands, and can have a positive impact on brand awareness and image building.

In other words, the device 30 can create additional value for products and brands through the process of issuing product manual videos as NFTs and help build a differentiated business model.

In step S1005, the device 30 may upload the NFT to the market and distribute a preset percentage of the NFT sales revenue to the seller selling the product and the producer who created the product's manual video. At this time, the preset ratio may be set differently depending on the embodiment.

The device 30 can register NFT issuance information on the NFT transaction site. At this time, the device 30 can enable the NFT of the product's manual video to be traded by registering the NFT issuance information on the NFT trading site.

To this end, the device 30 may be connected to a server that provides an NFT transaction site, or may be implemented including a server that provides an NFT transaction site.

The device 30 may register the issuance information of the NFT on the NFT transaction site and store the transaction history of the issuance information of the traded NFT in the database each time the NFT is traded.

The device 30 can provide additional incentives to sellers selling products and creators of product manual videos by distributing a certain percentage of the NFT sales revenue to them. This can encourage better collaboration and creation by providing rewards for NFT issuance and sales activities to sellers and creators of products.

In addition, the device 30 can strengthen the cooperative relationship between the seller selling the product and the producer by distributing the sales revenue of the NFT to the product seller and the producer, and by sharing mutual benefits and establishing closer cooperation, a better It can have an impact on product development and marketing.

The device 30 can increase the brand value of the product through NFT. By enabling a direct connection between creators and consumers, NFTs can emphasize the reliability and value of the product brand, which can strengthen the product's brand image and improve competitiveness by providing additional value and benefits to consumers. It can have an effect. Furthermore, it can stimulate product sales.

Figure 11 is an exemplary diagram of the configuration of the device 30 according to an embodiment.

Device 30 according to one embodiment includes a processor 31 and memory 32. The device 30 according to one embodiment may be the server or terminal described above. The processor 31 may include at least one device described above with reference to FIGS. 1 to 10 or may perform at least one method described with reference to FIGS. 1 to 10 . The memory 32 may store information related to the above-described method or store a program in which the above-described method is implemented. Memory 32 may be volatile memory or non-volatile memory.

The processor 31 can execute programs and control the device 30. The code of the program executed by the processor 31 may be stored in the memory 32. The device 30 is connected to an external device (eg, a personal computer or a network) through an input/output device (not shown) and can exchange data.

The embodiments described above may be implemented with hardware components, software components, and/or a combination of hardware components and software components. For example, the devices, methods, and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, and a field programmable gate (FPGA). It may be implemented using one or more general-purpose or special-purpose computers, such as an array, programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. A processing device may execute an operating system (OS) and one or more software applications that run on the operating system. Additionally, a processing device may access, store, manipulate, process, and generate data in response to the execution of software. For ease of understanding, a single processing device may be described as being used; however, those skilled in the art will understand that a processing device includes multiple processing elements and/or multiple types of processing elements. It can be seen that it may include. For example, a processing device may include a plurality of processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are possible.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on the medium may be specially designed and configured for the embodiment or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -Includes optical media (magneto-optical media) and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing unit to operate as desired, or may be processed independently or collectively. You can command the device. Software and/or data may be used on any type of machine, component, physical device, virtual equipment, computer storage medium or device to be interpreted by or to provide instructions or data to a processing device. , or may be permanently or temporarily embodied in a transmitted signal wave. Software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

Although the embodiments have been described with limited drawings as described above, those skilled in the art can apply various technical modifications and variations based on the above. For example, the described techniques are performed in a different order than the described method, and/or components of the described system, structure, device, circuit, etc. are combined or combined in a different form than the described method, or other components are used. Alternatively, appropriate results may be achieved even if substituted or substituted by an equivalent.

Therefore, other implementations, other embodiments, and equivalents of the claims also fall within the scope of the following claims.

Claims (3)

In a method performed by a device,
Obtaining a first image of a product in a first state;
Obtaining a second image of the product in a second state;
Obtaining a third image of the product in a third state;
outputting the first image;
Receiving user input for the first image;
If the user input is a first user input, converting the first image into the second image and outputting it; and
When the user input is a second user input, converting the first image into the third image and outputting it,
The second state is,
At least a portion of the product is in a state that is visually distinguishable from the first state,
The third state is,
Including a state in which at least a part of the product is enlarged or reduced compared to the first state,
In the step of converting and outputting the first image into the second image,
In order to minimize the visual difference between the first image and the second image, inputting the first image and the second image into a pre-trained artificial intelligence model and outputting an intermediate image;
extracting common portions and difference portions of the first image, the second image, and the intermediate image, and
Further comprising modifying the intermediate image by maintaining the common portion in the intermediate image and blurring the difference portion,
The middle image is,
Compare the first image and the second image to calculate the degree of difference between the first image and the second image,
The number to be output is determined based on the degree of difference,
Extracting keywords from the sound input of the user input,
Check the number of product keywords related to the product among the keywords,
The output maintenance time is determined by applying a weight determined to be inversely proportional to the number output at a preset reference time and proportional to the number of product keywords,
Method for creating and providing responsive user manual content based on user interaction. delete According to paragraph 1,
After converting the first image into the third image and outputting it,
When an additional user input for the third image is received, outputting a manual image of the product in a pop-up form of a first size;
If the touch intensity of the additional user input is more than a preset standard, the part information of the part included in the part of the product corresponding to the touch position and the sales page of the part are output in the form of a pop-up of a second size different from the first size. steps; and
If the number of touches of the additional user input is more than a preset reference number, outputting a product-related customer center connection information message,
Method for creating and providing responsive user manual content based on user interaction.