KR102388581B1 - Artificial Intelligence deep learning applied NFT-based image processing method and its chapter - Google Patents

Abstract

An AI deep learning applied NFT-based image processing device according to the present invention is a convergence device of blockchain and AI deep learning for NFT asset transaction. The device comprises: an image quality improvement filter part that performs similarity analysis using CNN in order to improve image clarity for easily identifying an image, and performs image quality improvement on the image before the image similarity analysis; an image similarity analysis part for image similarity analysis; a token issuing part for issuing tokens; an image copyright registration part for registering video image works; an image copyright transaction part for trading image works; a smart contract part for transaction contracts; and a Klay transmitter for Klay transmission. The image quality improvement filter part includes Pre-Denoising, Denoising, color adjustment, RESIZE, and Cooler readjustment that are sequentially performed. The Pre-Denoising includes a TTempSmoothF filter and a VagueDenoiser filter. The Denoising includes a 2DCLEAN filter, an FFT3D filter, and a DFTTEST filter. The color adjustment includes a ColorMatrix filter and a CONVERTORGB32 filter. The RESIZE includes a SPLINE36RESIZE filter. The Color readjustment includes a ConverToRGB32 filter.

Description

Artificial Intelligence deep learning applied NFT-based image processing method and its chapter}

The present invention relates to an NFT-based image processing method and apparatus for applying artificial intelligence (hereinafter referred to as "AI") deep learning. More specifically, it is a technology related to the visual processing of artificial intelligence, which is an ICT-based technology in the 4th industrial revolution. In the technology related to visual processing among the convergence devices of block chain and AI deep learning for NFT asset transaction, NFT-based image processing method and It's about the device.

NFT (Non-Fungible Token) means a non-fungible token. NFT technology using blockchain’s cryptocurrency technology stores digital files in tokens and is issued only once, and since all transactions are recorded on the blockchain, counterfeiting and falsification are impossible, so the originality of digital files is recognized and ownership is granted do. In this way, it is possible to create a system that can protect the rights (eg, copyright, etc.) of digital files on the Internet.

The task of existing digital content has a problem in that it is easy to copy and modify.

Although there is a copyright management technology called 'DRM', it costs a lot of money, ownership cannot be transferred, and in most cases, the buyer only gets the right to use it. In other words, it is a structure that cannot be controlled by secondary distribution rights holders (authors, etc.) and distributors, and there is a problem of very low liquidity. In response to these problems of existing digital content, NFT is inexpensive, and the ownership is recorded on the block chain and can be transferred to the purchaser, and the design is so that the right holder (author, etc.) can earn profits even during secondary distribution. It has the advantage of being able to expect market expansion by adopting standard specifications and realizing secondary distribution.

Such NFT is a kind of digital data using block chain and is characterized by being ‘Non Fungible’. Digital data with non-counterfeitable appraisal and ownership certificates can be traded as assets.

Most of them are issued in the Ethereum (ETH) standard <ERC-721>, a type of block chain, and FT (ERC-20) and NFT, which act as the Ethereum (ETH) currency, are issued, traded, and stored.

In order to be recognized as a photographic work containing the author's individuality and creativity, various techniques such as subject, light direction and angle, and exposure time must be considered. In other words, even if it is a subject lit by the same amount of light at the same time, if it is recognized that there is a personality according to the angle of the light illuminating the subject, copyright can be granted as a work.

As a background technology of the present invention, Registered Patent No. 10-2343025 "non-replaceable token-based digital art transaction method and product transaction method using the same" (hereinafter referred to as "prior art 1") is disclosed. In the prior art 1, in a non-fungible token-based digital art transaction method, a digital art management server receives a registration request including a division number to divide an original file of digital art and ownership of digital art from a creator terminal, and uses a smart contract to create a divided number of non-fungible tokens that have a unique token ID corresponding to digital art and have the address of the creator of the digital art as the owner address, and the number of non-fungible tokens for digital art sold from the creator's terminal Receive an auction request including receiving, determining the buyer corresponding to the buyer terminal that has transmitted the highest bid amount among the plurality of buyer terminals as the successful bidder, and using a smart contract It is transmitted to the address, and the history of changing the ownership of the sold number of non-fungible tokens from the creator's address to the successful bidder's address is stored in the block chain.

Registered Patent No. 10-2340588, "object management system supporting object management service using blockchain-based NFT" (hereinafter referred to as "prior art 2") has been disclosed.

Prior art 2 is an object management system that supports an object management service using blockchain-based NFT. The object management system determines the degree of management of objects based on information about objects registered in the platform by the object owner terminal, and , the object management server that issues the block chain-based NFT for the object reflecting the determined management level and provides it to the object owner terminal and the object management server required for NFT issuance in response to the request for creation of the object received from the object management server It includes a blockchain management server that acquires information from an object management server, creates an NFT for an object based on the obtained information, and transmits it to the object management server.

Registered Patent No. 10-2022-0014764 "an electronic device and a method for generating training data of an artificial intelligence learning model of an electronic device" (hereinafter referred to as "prior art 3") is disclosed. Prior art 3 may include a communication circuit, a memory, and a processor operatively coupled to the memory. According to an embodiment, the memory, when executed, causes the processor to acquire a first image, generate a virtual light source image, insert the virtual light source image into the first image to generate a first output image, , After inserting the virtual light source image into the first image, a first filter process is performed to generate a first input image, and an artificial intelligence learning model for compensating an image for the first input image and the first output image. Instructions for generating the first training data set may be stored.

Registered Patent No. 10-2319492 "Personal information processing system using AI deep learning and personal information processing method using the same" (hereinafter referred to as "prior art 4") is disclosed.

In the prior art 4, a preprocessing unit extracts an image file from a target file, detects a region of interest that is determined to contain personal information from the extracted image file, and performs processing to improve the image recognition rate for the region of interest; The personal information extraction step of extracting the processing target information including personal information from the region of interest detected from the pre-processing unit by the personal information region extraction unit using the CNN neural network, and the pattern analysis unit of the personal information extracted by the personal information region extraction unit A pattern analysis step of extracting personal information of the processing target information by comparing and analyzing the pattern with a preset personal information pattern, and a de-identification processing step in which the de-identification processing unit de-identifies the personal information extracted by the pattern analysis unit It is possible to provide a personal information processing system and personal information processing method using deep learning, characterized in that it comprises a. According to the above, after analyzing the text and image information included in the image file using an artificial intelligence neural network, the personal information (personal identification number information or fingerprint information, etc.) included therein is detected and extracted and retained by an individual or institution. You can effectively protect personal information by effectively detecting and de-identifying your personal information.

(0001) Republic of Korea Patent Publication No. 10-2343025 (0002) Republic of Korea Patent Publication No. 10-2340588 (0003) Republic of Korea Patent Publication No. 10-2022-0014764 (0004) Republic of Korea Patent Publication No. 10-2319492

An object of the present invention is to provide a NFT-based image processing method applied with artificial intelligence deep learning that can provide a clear image with respect to image analysis essential in a product transaction system through NFT.

Another object of the present invention is to build a similarity analysis modeling using artificial intelligence deep learning to prevent copyright granting of imitation images and imitation photos, and artificial intelligence deep learning to grant copyrights only to images and photo works that have passed the similarity check An object of the present invention is to provide an applied NFT-based image processing apparatus.

The NFT-based image processing method applied with artificial intelligence deep learning of the present invention is an AI visual processing among the convergence devices of blockchain and deep learning for NFT asset transaction. Recognition unit 210, image similarity analysis unit 220, application model unit (CNN model) 221, image data unit 222, token issuing unit 230, smart contract writing/distributing unit 231, NFT Consists of a issuing unit 232, an image work registration unit 240, an image work transaction unit 250, a smart contract unit 260 and a Klay transmission unit 270, and image similarity analysis using a CNN (Convolutional Neural Network) (20), but before performing the image similarity analysis (20), the image 10 through the image quality improvement filter (100); The step of passing through the image quality improvement filter 100 includes a Pre-Denoising step 110 using a TTempSmoothF filter 111 and a VagueDenoiser filter 112; 2DCLEAN filter 121, FFT3D filter 122, and denoising step 120 using a combination of DFTTEST filter 123;

a color adjustment step 130 using the ColorMatrix filter 131 and the CONVERTORGB32 filter 132; RESIZE step 140 for resizing using the SPLINE36RESIZE filter 141;

a color readjustment step 150 of re-adjusting the color using the ConverToRGB32 filter 132; and other image data 22 for the image similarity analysis 20 also pass through the image quality improvement filter 100 performing the image similarity analysis (20); issuing tokens (30) by performing NFT issuance (32) together with smart contract creation/distribution (31) for the image (10) that has passed the image similarity analysis (20);

performing image work registration (40); Image copyright transaction (50) step; It is characterized in that the smart contract (60) step and the Klay transmission (70) step are sequentially performed.

}, 정규화 상수를 λ라 할 때, SVM 함수(L)는, In addition, in the NFT-based image processing method applied with artificial intelligence deep learning of the present invention, the identifier for identifying the image is for image recognition, and SVM (Support Vector Machine) is used, and the SVM function ( L ) is [expression below] It is characterized according to [Equation below] y = w ^T x for the linear identifier of the weight w, and the labeled training image pair {

}, when the regularization constant is λ, the SVM function ( L ) is

In addition, the artificial intelligence deep learning applied NFT-based image processing apparatus of the present invention.

In AI (artificial intelligence) visual processing for a convergence device of blockchain and deep learning for NFT work transaction, image similarity analysis using CNN to improve image clarity so that images can be easily identified (20) However, before performing the image similarity analysis 20, the image quality improvement filter unit 1000 for image quality improvement with respect to the image 10, the image similarity analysis unit 220 for performing the image similarity analysis 20, Token issuance unit 230 for issuing tokens, image work registration unit 240 for registering image image works, image work transaction unit 250 for trading image works, smart contract unit 260 for transaction contracts , consists of a Klay transmission unit 270 for Klay transmission,

The image quality improvement filter unit 1000 includes that Pre-Denoising (110), Denoising (120), color adjustment (130), RESIZE (140), and cooler re-adjustment (150) are sequentially performed, and the Pre-Denoising Reference numeral 110 includes a TTempSmoothF filter 111 and a VagueDenoiser filter 112, and the Denoising 120 includes a 2DCLEAN filter 121, an FFT3D filter 122, and a DFTTEST filter 123, and the color Adjustment 130 includes a ColorMatrix filter 131 and a CONVERTORGB32 filter 132 , the RESIZE 140 includes a SPLINE36RESIZE filter 141 , and the color readjustment 150 includes a ConverToRGB32 filter 151 . In addition, as a virtual asset trading service, the Coinchain unit (2) operates a direct purchase store and a trading order matching exchange between customers, and the Coinchain NFT unit ( 3), Coinchain IEO (Initial Exchange Offering), a platform that supports financing through token issuance (4), and Coinchain that automatically accumulates a certain amount of virtual assets every month to provide stable virtual asset investment In response to the questionnaire, the ACC (Accumulation) Service Department (5), the Coinchain VAL (Virtual Asset Loan) Service Department (6), which provides a service to borrow virtual assets to the coin chain and obtain a fee after the contract period expires, It is characterized in that it includes a Coinchain ENQ Service unit (7) that provides a service that allows the accumulated points to be exchanged for virtual assets, and a Sharely Service unit (8) that provides a service that enables companies to hold a general shareholder meeting online.

According to the present invention, there is an advantage of being able to provide a clear image in AI visual processing in a convergence device of blockchain and deep learning for NFT asset transaction.

In addition, according to the present invention, there is an effect of preventing copyright granting of imitation images and imitation photos by constructing similarity analysis modeling using learning, and granting copyright only to images and photographic works that have passed the similarity test.

1 is a structural diagram of an NFT according to an embodiment of the present invention.
2 is a flowchart of an NFT transaction according to an embodiment of the present invention.
3 is a picture of the electronic seal service 'Block Record'.
4 is a picture of the fact authentication service 'Block Record'.
5 is a picture of 'Block Record', a birth record fact authentication service.
6 is a diagram for explaining authenticity (anti-counterfeiting) of NFT transactions.
7 is a diagram illustrating the concept of a virtual asset transaction service.
8 is a schematic configuration diagram of an NFT unit of the present invention.
9 is a flowchart showing AI deep learning applied to NFT-based image image transaction.
10 is a flow chart for using an image quality enhancement filter;
11 is a block diagram of a device applying AI deep learning to NFT-based image image transaction.
12 is a conceptual diagram of an automatic encoder of the present invention.
13 is a diagram illustrating the concept of a collapsible layer image filter and a pooling layer.
Fig. 14 is a diagram showing pre-training of the three-layer net.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

1 is a diagram of an NFT structure according to an embodiment of the present invention, and FIG. 2 is a flowchart of an NFT transaction according to an embodiment of the present invention. 3 is a picture of the electronic seal service 'Block Record',

4 is a photo of the fact authentication service 'Block Record'. 5 is a picture of 'Block Record', a birth record fact authentication service. 6 is a diagram for explaining authenticity (anti-counterfeiting) of an NFT transaction, and FIG. 7 is a diagram illustrating the concept of a virtual asset transaction service.

8 is a schematic configuration diagram of the NFT unit of the present invention, and FIG. 9 is a flowchart illustrating AI deep learning applied to NFT-based image image trading. 10 is a flowchart for using an image quality improvement filter, and FIG. 11 is a block diagram of a device applying AI deep learning to NFT-based image image transaction. 12 is a conceptual diagram of an autoencoder of the present invention, FIG. 13 is a diagram illustrating the concept of a collapsible layer image filter and a pooling layer, and FIG. 14 is a diagram illustrating pre-training of the three-layer net.

The present invention is a technology related to the visual processing of AI, which is an ICT-based technology, and aims to provide a clear image with respect to image analysis, which is essential in a system for goods transaction through NFT. In addition, by establishing a similarity analysis modeling using AI deep learning, it is intended to prevent copyright granting of imitation images and imitation photos, and to grant copyrights only to images and photographic works that have passed the similarity test.

NFT characteristics include uniqueness, tradability, interoperability, and program usefulness.

NFT uniqueness is unique digital data that can be identified from other digital data and has property properties. In addition, meta information such as the number of issuance, creation date, and identification number floating in the data is specified and stored on the block chain that cannot be altered. In NFT tradability, the ownership of digital data becomes clear and it has uniqueness and asset properties, so it can be a transaction target. In addition, transactions with high stability are possible because anyone can view and verify the transaction details on the block chain. Regarding NFT interoperability, NFTs issued and distributed in a common standard can, in principle, be used on multiple wallets or marketplaces. The usefulness of the NFT program allows the design of additional functions such as transaction volume restrictions and fees for secondary distribution.

1 is a description of the NFT structure. 1 illustrates a structure that connects an online chain and an offline chain. In the offline chain, the service consists of web application, information retention, and display screen provision. The online chain consists of a smart contract as a block chain (Ethereum), ETH remittance by the buyer / NFT transmission by the trader, distributed ledger, and ETH storage / NFT storage by users.

The characteristics of NFT are digital contents that have uniqueness, express contents as tokens, and require entity management. Features of the token include programmability such as transaction volume control, automatic transaction processing for values and rights expressed in tokens, digital currency (coin), and digital security (ST). The characteristics of the block chain are that copying is virtually impossible, recorded information is virtually indestructible, recorded information verification is easy, history management (tracking), and value as virtual assets (BTC, ETH, etc.) .

<Table 1> summarizes this in a table.

characteristic what can be realized purpose NFT Possess uniqueness
(exactly matching tokens do not exist) Representing content as tokens Digital content requiring object management token Programmable, such as trading volume control value expressed in tokens;
Automatic processing of transactions for rights Digital currency (coins)
Digital Securities (ST) blockchain Copying is virtually impossible.
Recorded information is virtually indestructible Easy verification of recorded information History management (tracking)
Virtual assets (BTC, ETH, etc.)

2 shows the flow of an NFT transaction.

① The content author records the information floating in the content (content storage location, etc.) on the blockchain and issues the NFT. ② NFTs are purchased from the marketplace (exchange) and the buyer's address is recorded on the block chain (the NFT owner moves).

③ NFT buyers have a structure in which content creators can earn profits even when they are secondaryly distributed in the marketplace. ④ NFT payment is made with the virtual asset Ethereum, and ETH transaction information is also recorded on the block chain.

3 shows an electronic seal authentication service.

The NFT seal is imprinted on the electronic document with the seal holder and the NFT-ized seal information, and the seal record is recorded in an immutable block chain, leaving clear evidence of ‘when, who sealed what.’

4 shows an example of the fact authentication service 'Block Record'.

The fact-checking service ‘Block Record’ can be used to record traffic accident scene photos, workplace harassment videos/photos, natural disaster photos, and videos.

Another application is a picture image, a photograph image, a contract image, a testamentary data image, a notice/confirmation letter, etc.

Figure 5 shows the birth record as an example of the birth record fact authentication service 'Block Record'. As another use of the fact authentication service ‘Block Record’, it is possible to verify the fact of birth records in the record-post-certificate issuance stage.

6 shows authenticity authentication (anti-counterfeiting).

It provides a service to determine whether a product is a genuine product or a counterfeit product. Product information is recorded on the block chain, and the consumer can read the code given to the product with a terminal to prove authenticity, which indicates whether the actual product is genuine, and the certificate of authenticity, which proves the buyer's ownership.

In this way, by utilizing blockchain technology and proprietary algorithms, counterfeit products and counterfeit products can be eradicated and authenticity can be verified. New products, used products, and online and offline are all available. In addition, by issuing certificates such as premium products from the manufacturer, it is possible to provide a service that can prove the owner by judging the authenticity of limited premium products with a block chain. The effect of being able to prevent imitation and counterfeiting occurs by authenticating brand products.

Consumers can check whether the product is genuine before purchase, digital storage of the warranty is possible safely, and there is no risk of loss by registering the warranty.

7 is a virtual asset trading service, showing the operation of a trading order matching exchange between customers. In other words, it represents the operation of a store that can be directly purchased as a marketplace where NFTs and virtual assets can be exchanged between customers.

Exhibitors submit their items to the NFT Marketplace through an NFT dedicated wallet. The buyer pays to the virtual asset wallet through the Coinchain virtual asset account, and then the exhibitor generates sales from the virtual asset wallet through the Coinchain virtual asset account.

After that, the buyer receives the exhibitor's goods to the NFT marketplace through an NFT dedicated wallet.

8 is a schematic configuration diagram of an NFT unit, and the NFT unit is as follows.

Coinchain is a virtual asset trading service that operates an exchange that allows direct purchase and a matching exchange between customers.

Coinchain NFT is a marketplace where NFT transactions are possible, and one-stop purchases are possible through integration with the virtual asset exchange.

Coinchain IEO (Initial Exchange Offering) is a platform that supports financing through coin issuance.

Coinchain ACC (Accumulation) Service is a service that enables stable virtual asset investment by automatically accumulating a certain amount of virtual assets every month.

Coinchain VAL (Virtual Asset Loan) Service is a service that allows customers to borrow virtual assets owned by the coin chain to obtain usage fees after the contract period expires.

Coinchain ENQ Service is a service that can exchange points collected in response to survey company questionnaires into virtual assets.

Sharely Service: This is a service that allows companies to hold general shareholders' meetings online.

The NFT unit (1) of the invention is a virtual asset trading service, and the Coinchain unit (2), which operates a direct purchase store, a trading order matching exchange between customers, and a marketplace where NFT transactions are possible, is a one-stop purchase integrated with the virtual asset exchange. The Coinchain NFT unit (3) that enables Coinchain ACC (Accumulation) Service Department (5), which provides possible services, and Coinchain VAL (Virtual Asset Loan) Service Department (6), which provides services that allow users to obtain usage fees after the contract period expires by lending virtual assets to the coin chain and Coinchain ENQ Service (7), which provides a service where points collected in response to questionnaires can be exchanged for virtual assets, and Sharely Service (8), which provides a service that allows companies to hold shareholder meetings online to provide an NFT-based image trading device applied with AI deep learning.

9 is a conceptual diagram of a system in which AI deep learning is applied to the NFT-based image image transaction of the present invention. NFTracer is a non-fungible token tracking proof of concept, using NFTracer to build an art auction and real estate auction that solves the problem of a centralized system. The NFTracer system is used to verify the authenticity of works of art by cryptographic signatures during the transfer of ownership. That is, NFTracer is used to improve transparency by providing an immutable record of ownership.

Ethereum-based ownership transaction that implements ownership of artworks as tokens and processes transactions with smart contracts is off-chain by registering artworks, uploading artwork transactions to Ethereum, and uploading the rest of the data to off-chain p2p data storage. problems such as loss of However, Ethereum implements a complex cryptocurrency wallet, which makes users uncomfortable.

In the present invention, in order to solve the NFT copyright problem, in the convergence of blockchain and AI deep learning, the sharpness of the image is improved through a filter so that the copyright can be violated or the image image can be easily identified.

As shown in FIG. 13, the image copy detection system based on a convolutional neural network (CNN) model also detects images that are duplicated by rotation, scaling, and other content manipulation. In image pre-processing, after converting the original image into 44 formats, extracting the value of each filtered image based on Inception V3 and deep learning the image copy detection model.

Inception V3 has better AI deep learning training time and accuracy than ResNet_v2, MobileNet_v2, and NASNet large. A method for improving the sharpness of a video image is shown in FIG. 10 .

The concept of applying deep learning to the NFT-based image image transaction shown in FIG. 9 will be described in more detail.

Perform image similarity analysis (20) using CNN model or VGG-16 (Visual Geometry Group - 16 layers) model and AlexNet model (21), but before analyzing image (10) similarity, image quality improvement filter (100) Also, the image data 22 for similarity analysis is also subjected to the image similarity analysis 20 through the image quality improvement filter 100 in the same way. Afterwards, smart contract creation/distribution 31 and NFT issuance 32 are performed for the image 10 that has passed the image similarity analysis 20, token issuance 30, and image work registration 40 are performed. After that, when the image work transaction 50 and the smart contract 60 are completed, the work can be traded by Klay transmission 70 .

That is, to build an AI deep learning model for similar image detection, the features of the image data 22 are analyzed using a CNN model, a Visual Geometry Group (16 layers) model, or an AlexNet model 21. For image similarity analysis (20), the image 10 for work transaction is improved through the image quality improvement filter 100, and also image data 22 for image similarity analysis (20) Also, image similarity analysis (20) is performed by improving the image quality through the image quality improvement filter (100). Thereafter, the copyright holder completes the image work registration (40) and conducts an image work transaction (50).

In order to trade blockchain NFT-based image works, the copyright holder issues NFT 32 when he wants to sell his work, that is, the image 10. The copyright holder creates a smart contract 60 containing contents related to his/her image work transaction, and the smart contract 60 can be changed according to the sales conditions of the copyright holder. It does not include security information such as identity information or account password. If the created smart contract 60 is then distributed on the blockchain network, Klay transmission 70, ie, virtual currency, is transmitted through the smart contract 60, and NFT ownership can be traded.

When a buyer wants to trade image assets, the transaction proceeds through the smart contract 60 distributed on the block chain. When the transaction conditions, which are the contents of the smart contract 60, are satisfied, the contract is automatically concluded to become the virtual currency of the block chain, that is, Klay transmission 70, and the NFT of the copyright holder is delivered to the purchaser.

10 is a diagram illustrating the use of an image quality improvement filter.

Each step of the image quality improvement filter 100 will be described in detail.

< Pre-Denoising step (110)>

In order to remove the sharpening noise and speckles from the image with the denoise filter from the back, the value is very large, and the side effects are also large. However, if sharpening is performed after removing noise and stains in advance, stains and noise can also be removed with a small value, and side effects are reduced that much.

In other words, the purpose of Pre-Denoising is to weaken the noise intensity for Dfttest of Post-Denoising (denoising for weak noise), but not completely eliminating noise. The Pre-Denoising step uses the TTempSmoothF filter 111 and the VagueDenoiser filter 112 .

The TTempSmoothF filter 111 is a smooth-based filter, and relieves slight stains. The VagueDenoiser filter is used when the DFTTEST filter does not solve the problem, and the value must be set higher than DFTTEST for the effect to occur properly. Even if the DFTTEST filter also removes noise, image detail is lost when used. The reason for using the TTempSmoothF filter 111 and the VagueDenoiser filter 112 together is to remove stains.

In the VagueDenoiser filter 112, the filter strength threshold is 2.5, the filtering processing method to be used by the filter is 1, nsteps is 6, chromaT is -1, and WIENER is False. The higher the threshold, the more clips are filtered, and 2.5 is assigned as a value in the present invention. Method is the filtering processing method to be used by the filter. A value of 1 is most preferable, but it is adjusted by giving a difference of 0.5 value. If it is given more than 0.5, be careful as fine adjustment is not well done. nsteps is the number of times the wavelet decomposes the photo, the photo cannot be decomposed beyond a certain point, the higher the nsteps, the higher the accuracy, but slower. In the present invention, a value of 6 is assigned. chromaT is a chroma-related option, and its value is -1. It's slow, but it gives good results. If set to 0 or less, Chroma denoising is disabled, and it becomes the default mode. Wiener is an option for stronger denoising, and the value of Wiener is set to false.

< Denoise step (120) >

The filter used in the denoise step 120 after the Pre-Denoise step 110 is used by combining the 2DCLEAN filter 121 , the FFT3D filter 122 , and the DFTTEST filter 123 . [2DCLEAN filter 121] is used to gently press the image surface to remove stains, etc., and is light and has a good effect. It is used when there are chronic problems such as stains in the sauce and cannot be resolved. Since there may be color change, carefully adjust the value so that color change does not occur. If you set the value above sharpening, the stain will not disappear and the effect will decrease. If the 2DCLEAN filter 121 is used in the YV12 color space, it should be noted that 100% error occurs in the modified AVISYNTH filter of the MT version filter. [FFT3D filter 122] is very powerful, so you have to be careful about crushing details, and it is preferable to adjust the SIGMA value within the range of 1.2 to 2.0. Note that if the FFT3D filter 122 is placed behind DFTTEST, the effect will be lost. If the FFT3D filter 122 and the DFTTEST filter combo also leave stains, consider using the +SORA_2DCLEAN filter. [DFTTEST filter 123] is used when some sources are stained. The SIGMA value is 4, the TBSIZE value is 2, the TMODE value is 1, and the LSB value is FALSE. Finely adjust the sigma value to fine-tune the detail.

< Color adjustment step (130) >

When the denoise is finished, the color filter is used to adjust the color of the image or video. The color filters used in the present invention are [ColorMatrix filter 131] and [CONVERTORGB32 filter 132]. [ColorMatrix filter 131] is a filter for matching standardized color key values in advance. Generally, the source is 601 and Blu-ray is 709. 601 is warm and 709 is cold. If you give a value to a location other than this location, be careful because ringing occurs.

[CONVERTORGB32 filter 132] can remove overlapping colors. However, be careful as color breakage may occur if it deviates from the color series or if there is a significant difference in values. [CONVERTORGB32 filter 132] uses RGBAdjust, HSVAdjust, HSLAdjust, ConvertToYV12, ColorYUV2, AutoGain_VD, and RGBAdjustment (133) adjusts white balance. RGBA (A is alpha channel) provides adjustment values according to Gamma and Bias as well as simple RGB adjustment. The HSVAdjust (134) is concerned only with the color density regardless of the color saturation. HSLAdjust(135) is concerned with color intensity along with color saturation. ConvertToYV12 (136) is for adjusting the image to look natural, and it is preferable to do YV12 only once. This is because, when YV12 conversion is executed multiple times, the loss is quite large, and a color close to gray is generated around the line. ColorYUY2 (137) mainly adjusts brightness, gamma, and contrast. Since there are many corresponding option values, you must adjust the real-time previewing in VirtualDub to use it.

Although not shown in the drawing, AutoGain_VD is a filter that automatically finds GAIN values by analyzing frames. It is used as a possible sub-filter in the color adjustment step 130, and it is preferable not to use it as a main filter. When used as a main filter, side effects occur in some scenes. If you use the given default value, it will cause a very slow and flickering flick. In other words, if you use 1.5 as the default value for Dark_limit and Bright_limit, the corresponding option will flicker, and if you set it as the minimum value of 1.0, the filter effect becomes 0, so it is preferable to set it to 1.1.

< RESIZE step (140) >

[RESIZE] should be done after adjusting the color of an image or video using a color filter. The RESIZE step 140 uses [SPLINE36RESIZE filter 141]. In the case of using the SPLINE36RESIZE filter 141, it must be warp down when down-scaling, and when up-scale or the same vertical value, it must be warp up. If it is the original scale, it should be given at the bottom. If you don't do this, and apply over sharpening or denoise, blurring and line thinning may cause a slight weeping phenomenon.

< Color readjustment step (150) >

Finally, all residual banding is removed by processing again with [ConverToRGB32 filter 151]. The method of using [ConverToRGB32 filter 151] is the same as the adjustment of [ConverToRGB32 filter 132] in the color adjustment step 130 .

11 is a block diagram of a convergence device of blockchain and AI deep learning for NFT asset transaction. This is a block diagram for the application of blockchain and AI deep learning convergence to NFT-based image image transaction. The block diagram of the convergence device of block chain and AI deep learning for NFT work transaction is image recognition unit 210, image similarity analysis unit 220, applied model unit (CNN model or VGG-16 model or AlexNet model) (221) , image data unit 222, token issuance unit 230, smart contract writing/distributing unit 231, NFT issuing unit 232, image asset registration unit 240, image asset transaction unit 250, smart contract unit 260 , and a Klay transmission unit 270 .

In more detail, in order to improve the image clarity so that the image can be easily identified, image similarity analysis is performed using a convolutional neural network (CNN), but the image similarity analysis unit 220 performs image similarity analysis before image An image quality improvement filter unit 1000 for image quality improvement for the image recognized by the recognition unit 210, a token issuing unit 230 for issuing a token, and an image work registration unit 240 for registering an image image work ), an image work transaction unit 250 for transacting image works, a smart contract unit 260 for a transaction contract, and a Klay transmission unit 270 for Klay transmission. The image quality improvement filter unit 1000 includes a Pre-Denoising unit, a Denoising unit, a Color adjustment unit, a RESIZE unit, and a Color re-adjustment unit, and the Pre-Denoising unit includes a TTempSmoothF filter 111 and a VagueDenoiser filter 112, The denoising unit includes a 2DCLEAN filter 121, an FFT3D filter 122, and a DFTTEST filter 123, the RESIZE unit includes a SPLINE36RESIZE filter 141, and the color readjustment unit includes a ConverToRGB32 filter 151 .

Meanwhile, [CONVERTORGB32 filter 132] consists of an RGBAdjustment unit 133 for adjusting white balance, an HSVAdjust unit 134 for adjusting color depth, and an HSLAdjust unit 135 for adjusting color density along with color saturation. It consists of a ConvertToYV12 (136) unit that adjusts the image to look natural, and a ColorYUY2 unit (137) that adjusts brightness, gamma, and contrast.

The application of AI deep learning and image filters to learn coding, identifiers, and module groups at once in FIGS. 12, 13, and 14 will be described.

Body identification refers to a task of predicting a category of an object in one input image, and object detection refers to a task of discovering an object of a given category in an image, including the area of the target object. In addition, there is also a task of simultaneously performing object identification and object detection that predicts the category and area of an object for an image whose image is not known, and this complex task is very important for image recognition.

The present invention is to provide an image filter that is very important in image recognition.

In the image recognition structure below, an approach has been used that considers each module as a separate problem and designs each module using machine learning. The present invention is an AI deep learning structure of object identification that can learn from the initial stage to the final stage at once by combining the identifier module at the end through several stages of the module. Object detection is realized by extracting a group of image regions serving as object candidates from an input image and applying the object identification to each image region through the filter of the present invention.

Deformable Part Model (DPM), an object identification technique, can handle a deformable object model. Using the deformation state of the object as a hidden variable, LSVMs (Latent SVMs; Support Vector Machines) are applied to learn the identifier. In the module for extracting the region group of the object candidate in the image, the image of the image is an important part that determines the degree and speed of object detection. As a slow but reliable method, it is a technique of extracting a candidate area while shifting a local area of a certain size for each pixel. In this process, it is very important to improve the accuracy of the image through the filter. In image recognition, AI deep learning is a technique for learning the weights from the lowest layer to the highest layer at once by joining with vectors that are superimposed on multiple layers.

As a coding technique for local features, Bag of Features (BoF) is widely used. As shown in FIG. 13, BoF is a histogram of how many representative local features appear in an image by extracting several representative local features from the training set. BoF is also known as Bag of Visual Words (BoVW). The calculation process of BoF is performed as follows.

i) Select K representative local features from all local features. A typical local feature is called a code word, and a set of selected code words is called a codebook. Codewords include w1, … , wk is labeled, and K-means is used. ii) Map all local features to codewords. iii) Compute the histogram for the codeword. That is, count wk and the number of labeled local features. Let the histogram of the codeword be the feature vector of the image. As described above, BoF is a technique for calculating a histogram of local features, and it is possible to improve BoF by using a mixed Gaussian distribution as shown in the following equation.

는 혼합요소이며, 평균μ _K 와 분산 Σ _K 를 가진다. ΠK 는 혼합 계수이다.here

is a mixing element, and has a mean μ _K and a variance Σ _K . ΠK is the mixing coefficient.

}라 하면 SVM의 코스트 함수는 다음 식과 같다. By using the mixed Gaussian distribution, since each Gaussian distribution constituting the mixed Gaussian distribution has a common variance, it is possible to use a distance metric considering the common variance. In addition, although only one codeword is assigned to a local feature in BoF, in a mixed Gaussian distribution, the relationship between local features and many codewords can be expressed, and information about the location of local features in the feature space can be encoded. . The identifier is the image 10, that is, for object recognition, and uses a support vector machine (SVM), and online learning is used for batch learning of large-scale data. In the image 10 object recognition, online learning in which the SGD (Stichastic Gradient Descent) technique is applied to the SVM is used. Let y=w ^T x be the linear identifier of weight w, and the labeled training image pair as {

}, the cost function of SVM is as follows.

where λ is the normalization constant. When SGD is applied to this cost function, the weight expression is as follows.

In standard SGD, since the procedure takes time, speed is achieved by applying the average to the weights, and it is possible to apply to large-scale data by online learning. The extraction of local features from the input image is called local feature, and hundreds to tens of thousands of local features can be obtained from one image. The operation of organizing into vectors is called pooling. Pooling includes either calculating the average of a target vector (average pooling) or computing the maximum value of each element of a vector (maximum pooling). As a result of this pooling, when a vector representing one image is obtained, it is called an image feature vector. A module that classifies the feature vector of an image into categories such as people, dogs, and cats is called an identifier, and object identification is completed through this module.

In this system, in the pooling, average pooling is used to reduce the image to ¼ size. The effect of fine distortion and movement is reduced by the pulling operation. As a method to overcome this when Deepnets cause scientific learning during learning, when learning Deepnets using the error backpropagation method, it is most common to initialize the network at random. However, since science learning may occur because the method of determining the initial value is not appropriate, if the initial value is set in advance, science learning can be avoided. This method divides the deepnet into two layers as shown in FIG. 14, and trains each of the two layers separately in order one by one. Each layer of training is executed by using an automatic encoder. The automatic encoder is a system whose ideal is that the output to the input is as close to the input as possible. When the layer under consideration calculates the output y from the input x (Fig. 12(a)), it is calculated in the opposite direction to this, that is, it is calculated to return the output y to the input side x'. This series of calculations, i.e., the calculation of x → y → x', is expressed by a two-layer net in the same structure as in Fig. 12(b), in which the original net is returned as an output. Decide so that these two layers four represent the output as close as possible to each input. In autoencoders, the input is projected onto the representation in the hidden layer and onto the representation output in the more hidden layer. It is an autoencoder that learns to make the input and output as equal as possible. Let the input vector be x, the projection to the hidden layer is as follows, and this manipulation is called coding.

y=f(W ₁ x + b ₁ )

Here, the function f is a sigmoid function, etc., where y is the expression of x in the hidden layer, b ₁ is the bias, and W ₁ is the weight matrix from the input layer to the hidden layer. Reconstruction in the hidden layer is as follows, and this operation is called decoding.

z=f(W ₂ y + b ₂ )

Here, z is the output vector, W ₂ is the weight matrix from the hidden layer to the output layer, and b ₂ is the bias, and we often set the W ₂ =W ₁ constraint. Weights W ₁ and W ₂ are learned so that the reconstruction error between the input x and the output z is small. SGD is mainly used for parameter learning. For the input used in learning, a learning sample (input value) that is finally to be treated as a deepnet is used. For a plurality of learning samples (input values), the normal error backpropagation method is executed so that the difference between the output and the input when each input value is input is small, and the weights of the two layers are determined. As for the automatic encoder, the greater the number of units in the middle layer, the higher the degree of freedom and expressive ability. On the other hand, if there are too many degrees of freedom, the input becomes the output as it is, and the identity image is learned. Here, when the auto-encoder reproduces each input, it learns with restrictions so that only a small number of units in the middle layer are used. Returning to deepnet training, the learning of each layer by the above automatic encoding starts from the first layer, where data is input, and the second and third layers are repeated. For the input of the first layer, the same thing as the learning sample for the deepnet described above is used as it is. For the input of the second layer, as shown in FIG. 14 , an output of an input value obtained by using the weight determined after learning of the first layer is used. After that, it keeps repeating itself. The weight of each layer determined in this way becomes the initial value of the weight of the layer corresponding to the target deepnet. After that, learning, which is the original teacher, is performed by the error backpropagation method. In addition, with the exception of the output layer at the end, pre-training is not targeted, and the weights are initialized randomly.

13 shows an overview of a collapsible layer image filter and a pooling layer.

A convolutional neural network or convolutional network (CNN, Convolutional Neural Network, or Convolutional Network) is a deepnet having a structure in which two types of layers called a folding layer and a pooling layer are folded and overlapped. Convolutional network is a very important technology in AI deep learning image recognition application. In image image processing, convolution of filters, that is, small-sized image images are two-dimensionally folded into the input image to blur or border the image image. Same as emphasizing By convolution of this image filter, extraction of local pattern features is achieved. It represents the features extracted by the convolutional image filter. The pooling layer takes the output of the convolution layer as an input, and aggregates many input channel images into a small area, that is, 5x5 values into one value. In summary, it is an operation to downsample the image picture of many input channels. Thereby, it is possible to obtain a certain degree of invariant characteristic to the position shift, that is, a characteristic that does not change even if the position is slightly shifted by ignoring the characteristic minute position change in the image. A convolutional network has a structure in which two pairs are repeated several times by combining a folding layer and a pooling layer. By using the folding layer and the pulling layer as a pair, local feature extraction and invariance to positional displacement are realized. The collapsible layer and the pooling layer are each expressed as a network of two structures connected for their computations. Either side is connected to one unit in the output layer, so the units in the input layer are located on the image as a small number.

The present invention is a technology related to the visual processing of artificial intelligence, which is an ICT-based technology in the 4th industrial revolution. More specifically, in the technology related to visual processing among the convergence devices of block chain and AI (artificial intelligence) AI deep learning for NFT work transaction, AI (artificial intelligence) for improving the image clarity so that the image can be easily identified ) AI deep learning applied NFT-based image processing method and technology related to the device are provided, so it is a useful invention with industrial applicability.

1: NFT Department 2: Coinchain Department
3: Coinchain NFT Department 4: Coinchain IEO (Initial Exchange Offering)
5: Coinchain ACC (Accumulation) Service Department
6: Coinchain VAL (Virtual Asset Loan) Service Department
7: Coinchain ENQ Service Department 8: Sharely Service Department
10: Image 20: Image similarity analysis
21: CNN model or VGG-16 or AlexNet model
22: image data 30: token issuance
31: Smart contract creation/distribution 32: NFT issuance
40: Image copyright registration 50: Image copyright transaction
60: Smart contract 70: Klay transmission
100: Image quality improvement filter 110: Pre-Denoising
111: TTempSmoothF 112: VagueDenoiser
120: Denoising 121: 2DCLEAN
122: FFT3D 123: DFTTEST
130: Color adjustment 131: ColorMatrix
132: CONVERTORGB32 133: RGBAdjustment
134: HSVAdjust 135: HSLAdjust
136: ColorYUY2 137: ColorYUY2
140: RESIZE 141: SPLINE36RESIZE
150: Color readjustment 151: CONVERTORGB32
210: image recognition unit 220: image similarity analysis unit
221: applied model unit 222: image data unit
230: token issuing unit 231: smart contract writing/distributing unit
232: NFT issuer 240: image copyright registration
250: Image copyright transaction unit 260: Smart contract unit
270: Klay transmission unit 1000: image quality improvement filter unit

Claims (3)

In the convergence method of blockchain and deep learning for NFT asset transaction, in order to improve the image clarity so that the image can be easily identified,
An image similarity analysis (20) is performed using a convolutional neural network (CNN),
subjecting the image 10 to an image quality improvement filter 100 before performing the image similarity analysis 20;
The step of passing through the image quality improvement filter 100 includes: a pre-denoising step 110 using a TTempSmoothF filter 111 and a VagueDenoiser filter 112;
2DCLEAN filter 121, FFT3D filter 122, and denoising step 120 using a combination of DFTTEST filter 123;
a color adjustment step 130 using the ColorMatrix filter 131 and the CONVERTORGB32 filter 132;
RESIZE step 140 for resizing using the SPLINE36RESIZE filter 141;
a color re-adjustment step 150 of re-adjusting the color using the ConverToRGB32 filter 132;
performing the image similarity analysis (20) on other image data 22 for the image similarity analysis (20) also through the image quality improvement filter (100);
issuing tokens (30) by performing NFT issuance (32) together with smart contract creation/distribution (31) for the image (10) that has passed the image similarity analysis (20);
image work registration (40) step for registering an image image work;
Image asset transaction (50) step for transacting image assets;
Smart contract (60) step for a transaction contract;
Klay transmission (70) step for Klay transmission;
Deep learning applied NFT-based image trading method, characterized in that it is sequentially performed with
The method according to claim 1, wherein the identifier for identifying the image 10 is for image recognition and uses a Support Vector Machine (SVM), and the SVM function ( L ) is characterized in that according to [Equation below]. AI (Artificial Intelligence) deep learning applied NFT-based image processing method.

[Formula below]
Let y=w ^T x be the linear identifier of weight w, and the labeled training image pair as {

}, when the regularization constant is λ, the SVM function ( L ) is

In the convergence device of blockchain and deep learning for NFT asset transaction, in order to improve the image clarity so that the image can be easily identified,
An image similarity analysis (20) is performed using a convolutional neural network (CNN),
Before the image similarity analysis (20), the image quality improvement filter unit 1000 for image quality improvement with respect to the image 10, the image similarity analysis unit 220 for performing the image similarity analysis 20,
Token issuance unit 230 for issuing tokens, image work registration unit 240 for registering image image works, image work transaction unit 250 for trading image works, smart contract unit 260 for transaction contracts , consisting of a Klay transmission unit 270 for Klay transmission,
The image quality improvement filter unit 1000 includes that Pre-Denoising (110), Denoising (120), color adjustment (130), RESIZE (140), and color re-adjustment (150) are sequentially performed,
The Pre-Denoising 110 includes a TTempSmoothF filter 111 and a VagueDenoiser filter 112,
The Denoising 120 includes a 2DCLEAN filter 121, an FFT3D filter 122, and a DFTTEST filter 123,
The color adjustment 130 includes a ColorMatrix filter 131 and a CONVERTORGB32 filter 132,
The RESIZE 140 includes a SPLINE36RESIZE filter 141,
The color readjustment 150 includes a ConverToRGB32 filter 151
Deep learning applied NFT-based image trading device, characterized in that.

Images