KR102646316B1 - Method and apparatus for identifying genuine article of seized movable property, and electronic auction system using the same - Google Patents

Abstract

A method of operating a genuine product verification device, comprising the steps of receiving one or more images containing preliminary seized personal property from a linked terminal, inputting the image into a learned genuine product classification model, and determining whether the preliminary seized personal property is genuine or counterfeit, and the basis for judgment. Step of acquiring the image area corresponding to, if the preliminary seizure property is genuine, input the image into the learned market price estimation model, calculate the loss rate of the image's preliminary seizure property, and apply the loss rate to the used market price for the preliminary seizure property. The step of calculating the used price by applying, and if the preliminary seized property is a counterfeit product, the result determined as a counterfeit product and the image area are provided to the terminal together, and if the preliminary seized property is a genuine product, the results determined as a genuine product and the calculated and providing the used price to the terminal.

Description

Method and device for verifying the authenticity of seized property and electronic public auction system using the same {METHOD AND APPARATUS FOR IDENTIFYING GENUINE ARTICLE OF SEIZED MOVABLE PROPERTY, AND ELECTRONIC AUCTION SYSTEM USING THE SAME}

The present invention relates to a technology for verifying the authenticity of seized property.

In order to ensure the soundness of local finances, we are strengthening efforts to dispose of large amounts of delinquent debt or habitual delinquency, and accordingly, we are tracking hidden assets and holding public auctions of seized properties.

In particular, in the case of goods with asset value, such as expensive luxury goods, they are classified as seized personal property, but it is difficult to confirm whether the goods in question actually have economic value as seized personal property. In fact, counterfeit or similar products are being distributed for expensive luxury goods, and unless you are a relevant expert, it is difficult to know the difference between luxury goods and imitations or similar products.

Therefore, an expert must check whether it is a luxury item at the seizure site and classify it as a seized property. However, in practice, it is difficult for experts to be deployed to all seizure sites, so arbitrarily classified seized properties are later evaluated by an expert.

Accordingly, counterfeit or similar goods with no economic value may be classified as seized property and then excluded by experts, and the public auction of seized property may be delayed due to the expert's appraisal process to determine the economic value of the seized property.

Therefore, technology is required so that even non-experts can immediately confirm the authenticity of luxury goods and confirm their economic value on site.

The problem to be solved is a method and device for verifying the authenticity of seized property by inputting images taken of preliminary seized property into a learned artificial intelligence model, calculating the authenticity and used price according to the loss rate, and providing it in real time. It provides a public auction system.

A method of operating a genuine product confirmation device according to an embodiment of the present invention, comprising the steps of receiving one or more images containing preliminary seized personal property from a linked terminal, inputting the image into a learned genuine product classification model to determine whether the preliminary seized personal property is genuine or counterfeit. A step of acquiring the image area corresponding to the recognition judgment result and judgment basis. If the preliminary seizure property is genuine, the image is input into the learned market price estimation model to calculate the loss rate of the preliminary seizure property, and the collected preliminary seizure property is A step of calculating the used price by applying the loss rate to the second-hand market price, and if the preliminary seized property is a counterfeit product, the result and image area judged to be a counterfeit product are provided to the terminal, and if the preliminary seized property is a genuine product, the judged genuine product is provided to the terminal. It includes providing the results and the calculated used price to the terminal.

The step of acquiring the image area is to analyze the image to determine the type and brand name of the preliminary seizure property, and to identify one or more of the logo, symbol, pattern, letter, metal decoration, stitching, texture, color, and material for the preliminary seizure property. By extracting characteristic points, it is possible to determine whether the preliminary seized property is genuine or fake.

The step of calculating the used price is to compare the image with the reference image of the same product as the preliminary seized property, set the different parts as point areas, and analyze the corresponding status among scratches, contamination, presence of accessories, damage, and damage for each point area. Thus, the loss rate according to the analyzed state information can be calculated.

The genuine product classification model and the price estimation model can be visualized and output in RGB format for feature points on the image that serve as the standard for the results output by applying the GRAD-CAM algorithm.

When judgment information on preliminary seized property is requested from the linked electronic public auction server, a step of providing an image area, used price calculation data, and a point area may be further included.

Set the collected fake images or used genuine images as learning images, match the results of fake or genuine products to the learning images to create learning data, and then input the learning images into the genuine classification model and output the matched results. A step of learning the genuine product classification model as much as possible, and matching the image captured for the appraised used genuine product with the damage area and loss rate of the used genuine product to generate learning data, and inputting the captured image into the price estimation model A step of training a price estimation model to output a matched loss rate may be further included.

An authenticity verification device according to an embodiment of the present invention, comprising a memory containing instructions and a processor that executes the instructions to confirm authenticity in an input image and calculate a used price, and the processor detects preliminary seizure property from a linked terminal. Upon receiving one or more images containing, the image is input into the learned genuine product classification model to obtain the result of determining whether the preliminary seized property is genuine or counterfeit and the image area corresponding to the basis for judgment, and if the preliminary seized property is a counterfeit product, The judgment result and image area are provided to the terminal, and if the preliminary seized property is genuine, the used price for the preliminary seized property calculated using the learned market price estimation model is provided to the terminal.

The learned genuine product classification model extracts one or more characteristic points from the logo, symbol, pattern, text, metal decoration, stitching, texture, color, and material of the preliminary seized property to determine whether the property is genuine or counterfeit, By applying the GRAD-CAM algorithm, the characteristic points that serve as the basis for determining whether a product is genuine or counterfeit can be visualized and output in RGB format.

The learned market price estimation model sets the different parts as point areas by comparing them with the reference image set based on the type and brand name of the preliminary seized property, and for each point area, the corresponding points among scratches, contamination, presence of accessories, damage, and damage are assessed. By analyzing the state, the loss rate is calculated according to the analyzed state information, and by applying the GRAD-CAM algorithm, the point area that is the standard for calculating the loss rate can be visualized and output in RGB format.

The processor can calculate the loss rate of preliminary seized property from the learned market price estimation model and calculate the used price by applying the loss rate to the used market price for the preliminary seized property.

A method of operating an electronic public auction system according to an embodiment of the present invention, which includes basic information including the product name, brand name, and seizure case number of the seized property determined to be genuine from a linked terminal, the used price set for the seized property, and the seized property. Receiving one or more images including, assigning an identification number to the seized property in response to the seizure case number, requesting judgment information about the seized property from the linked authenticity verification device to establish the basis for determining that the seized property is genuine Receiving the image area, receiving the point area where the loss rate of the seized property is calculated and the used price calculation data with the loss rate applied to the used market price, generating a 3D image of the seized property, and detailing the image area and the point area Step of creating product information of the seized property by setting it as an image and specifying the calculated used price as the basic price; posting the product information of the seized property online in response to the identification number; and seizing it by bidding method according to the designated public auction period. It includes a step of selecting a successful bidder for the movable property, and when receipt of the seized movable property is confirmed, a step of guiding the handover process of the seized movable property, and the image area contains the seized movable property entered into the learned genuine classification model. The image area represents the image area in which the feature points that serve as the basis for judging the product as genuine or counterfeit are visualized in RGB format, and the point area indicates scratches, contamination, presence of accessories, and damage in the image containing the seized property entered into the learned market price estimation model. The characteristic points that serve as the basis for the loss rate calculated by analyzing the corresponding state among the damages are displayed in an area visualized in RGB format.

According to an embodiment of the present invention, images of luxury goods taken on site are analyzed using a neural network to confirm authenticity, thereby excluding counterfeit goods or similar goods with low economic value and securing seized property.

According to an embodiment of the present invention, by quickly determining whether the seized property is genuine and the used price, the time and cost required for the appraisal of the seized property can be saved by minimizing the face-to-face item appraisal step by an expert.

According to an embodiment of the present invention, by providing a 3D image of the seized property and a basis for setting the genuine or used price of the seized property at the public auction stage of the seized property, more accurate product status information of the seized property is provided in the public auction procedure conducted non-face-to-face. can be provided.

Figure 1 is a network configuration diagram of an electronic public auction system for seized property according to an embodiment.
Figure 2 is a configuration diagram of an authenticity verification device according to an embodiment.
Figure 3 is a flowchart showing a method for verifying authenticity according to an embodiment.
Figure 4 is an example diagram showing and providing an area that serves as the basis for determining authenticity according to one embodiment.
Figure 5 is a flowchart of conducting an electronic public auction for seized property according to an embodiment.
Figures 6 and 7 are exemplary diagrams showing emotional points according to one embodiment.

Below, with reference to the attached drawings, embodiments of the present disclosure will be described in detail so that those skilled in the art can easily practice them. However, the present disclosure may be implemented in many different forms and is not limited to the embodiments described herein. In order to clearly explain the present disclosure in the drawings, parts that are not related to the description are omitted, and similar parts are given similar reference numerals throughout the specification.

In the specification, when a part “includes” a certain element, this means that it may further include other elements rather than excluding other elements, unless specifically stated to the contrary. The devices that make up the network may be implemented as hardware, software, or a combination of hardware and software.

In addition, terms such as "...unit", "...unit", and "...module" used in the specification refer to a unit that processes at least one function or operation, which is hardware, software, or a combination of hardware and software. It can be implemented as:

The devices described in the present invention are composed of hardware including at least one processor, a memory device, a communication device, etc., and a program that is executed in conjunction with the hardware is stored in a designated location. The hardware has a configuration and performance capable of executing the method of the present invention. The program includes instructions that implement the operating method of the present invention described with reference to the drawings, and executes the present invention by combining it with hardware such as a processor and memory device.

In this specification, “transmission or provision” may include not only direct transmission or provision, but also indirect transmission or provision through another device or using a circuitous route.

In this specification, expressions described as singular may be interpreted as singular or plural, unless explicit expressions such as “one” or “single” are used.

In this specification, terms including ordinal numbers, such as first, second, etc., may be used to describe various components, but the components are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, a first component may be referred to as a second component, and similarly, the second component may be referred to as a first component without departing from the scope of the present disclosure.

In the flowcharts described herein with reference to the drawings, the order of operations may be changed, several operations may be merged, certain operations may be divided, and certain operations may not be performed.

Figure 1 is a network configuration diagram of an electronic public auction system for seized property according to an embodiment.

As shown in FIG. 1, a user terminal 100 and an authenticity confirmation device that receives a captured image containing preliminary seized property from the user terminal 100 and provides authenticity confirmation or calculated used price for the preliminary seized property ( 200) is connected to the network and transmits and receives data to each other. In addition, the user terminal 100 and the authenticity verification device 200 are connected to a network between the electronic public auction system 300 and can register seized property and transmit and receive information on seizure climbing.

Here, the network may include any type of communication network that transmits data, such as a wired network, a short-range or long-distance wireless communication network, or a mixture of these networks.

The user terminal 100 is a general term for terminals equipped with computational processing capabilities by each having a memory means and a processor, and refers to a terminal of an administrator who wishes to seize the personal property of a tax delinquent.

The user terminal 100 includes a personal computer, a handheld computer, a personal digital assistant (PDA), a mobile phone, a smart device, and a tablet.

The user terminal 100 may access the authenticity verification device 200 or the electronic public auction system 300 after performing user authentication. For example, when registering a seized or impounded property, the ID of an administrator with authority may be entered to perform authentication, and then the authenticity verification device 200 or the electronic public auction system 300 may be accessed.

The user terminal 100 is equipped with a camera module, and after taking pictures of the preliminary seized property, the captured image can be transmitted to the authenticity confirmation device 200, and then the authenticity confirmation results can be received in real time.

Here, preliminary seized movable assets are movable assets before being designated as seized movable assets, which are expensive products exceeding a certain amount such as bags, watches, clothing, jewelry, miscellaneous goods, electronic products, and decorations, or luxury goods such as products of pre-designated brands. Indicates the goods being classified. Brands classified as luxury goods, types of goods, standard prices treated as expensive, etc. can be easily set and changed by the administrator.

The authenticity verification device 200 inputs the image received from the user terminal 100 into a trained authenticity verification model to determine whether the preliminary seized property in the image is genuine.

The authenticity confirmation device 200 can learn an authenticity confirmation model using image data of items treated as luxury goods at a previous point in time, and the authenticity confirmation model includes a plurality of artificial intelligence models corresponding to the type and brand name of the product. It can be composed of:

The authenticity verification model is an algorithm that directly learns by finding patterns to analyze images and classifies images based on the learned results, and can be implemented as a suitable artificial intelligence model. For example, implemented with Convolution Neural Network (CNN) algorithm, Recurrent Neural Network (RNN), Long Short-Term Memory (LSTM), Transformer based on attention mechanism, etc. It can be, and is not limited to any one.

In addition, the authenticity verification device 200 can calculate a used price using a market price estimation model learned for a preliminary seized property confirmed to be a luxury product.

Here, the market price estimation model compares the received image with the saved original image to calculate the ratio for scratches, contamination, accessories, damage or damage, etc., and calculates the ratio calculated for the used market price according to the year of production or sale of the product of the image. It can be implemented as an appropriate artificial intelligence model by applying an algorithm to estimate the market price of used goods. For example, implemented with Convolution Neural Network (CNN) algorithm, Recurrent Neural Network (RNN), Long Short-Term Memory (LSTM), Transformer based on attention mechanism, etc. It can be, and is not limited to any one.

Below, the case where the authenticity confirmation model and the price estimation model are implemented with a convolution neural network (CNN) algorithm is explained.

The authenticity verification device 200 applies the Grad-cam (Gradient-weighted cam) algorithm to the authenticity verification model and the market price estimation model to visualize and output the basis area judged for the result output from the input image.

For example, in an image input in RGB format, an area with a high degree of relevance to the basis for authenticity confirmation or a basis for loss rate estimation may be displayed in red, and a region with a low degree of relevance may be displayed in blue.

The electronic public auction system 300 refers to a system for conducting a non-face-to-face public auction. When the seized property is registered based on the delinquent's unique number by the user terminal 100, information on the seized property is sent to the authenticity verification device ( 200) and post the seized property for public auction on the Internet.

The electronic public auction system 300 can limit the posting period of seized properties according to the public auction schedule, and can provide 3D images of seized properties taken with a 3D stereo camera or combine images of seized properties from various angles to provide a 3D image. You can.

In addition, the electronic public auction system 300 can select the successful bidder through bidding, in which the person who applied for the higher amount is successful based on the authenticity and second-hand price of the seized property.

In this way, the electronic public auction system 300 allows the user terminal 100 to immediately register the seized personal property confirmed as genuine through the authenticity verification device 200 during the on-site seizure process as the seized personal property.

Figure 2 is a configuration diagram of an authenticity verification device according to an embodiment.

As shown in FIG. 2, the authenticity verification device 200 includes a reception module 210, a learning module 220, an authenticity verification module 230, and a price measurement module 240.

For explanation purposes, they are referred to as a receiving module 210, a learning module 220, an authenticity verification module 230, and a price measurement module 240, but they may be operated by at least one processor. Here, the receiving module 210, the learning module 220, the authenticity verification module 230, and the price measurement module 240 may be distributed and implemented in separate computing devices, and if distributedly implemented, through a communication interface. can communicate with each other.

For example, the genuine product classification model (A) and the price estimation model (B) that have been trained in the learning module 220 can be linked with the genuine product confirmation module 230 or the price measurement module 240, so the genuine product classification model ( The learning module 220 that trains A) and the price estimation model (B) does not need to be implemented together with the authenticity verification module 230 or the price measurement module 240.

In addition, since the authenticity verification module 230 can transmit the authenticity of the preliminary seized property included in the image to the market price measurement module 240, the authenticity verification module 230 and the market price measurement module 240 are separate computers connected to a network. It can be implemented as a device.

The receiving module 210 receives a request to confirm authenticity and a request to measure used goods along with a specific image from the linked user terminal 100. The receiving module 210 transmits the received image to the learning module 220, the authenticity verification module 230, or the price measurement module 240.

The learning module 220 uses the Grad-CAM algorithm to learn the genuine product classification model (A) and the price estimation model (B) using corresponding learning data.

The learning module 220 displays the basis area for the results output from each genuine product classification model (A) and price estimation model (B) in RGB format through the Grad-CAM algorithm.

Here, the learning data refers to luxury product appraisal images based on the type of luxury product, brand name, etc., and in detail, images of new product luxury products, images of used luxury products, images of counterfeit products, or used market prices are matched with the appraisal results, respectively.

In addition, preliminary seizure properties can be entrusted to the Appraisal Agency and the results of a precise appraisal conducted using a light microscope can be used as learning data.

Precision appraisal not only determines whether the product is genuine or counterfeit, but also calculates the used market price for the preliminary seized property, so it is used as training data for the genuine product classification model (A) and as training data for the market price estimation model (B) according to each judgment standard. It can be.

For example, in order to determine whether a product is genuine or counterfeit, image data that analyzes emotional points such as leather patterns, stitching, engravings, engravings, logo zippers, etc. through precise appraisal can be used as learning data for the genuine product classification model (A). . In addition, image data analyzed for the state of damage, wear, leather (outer skin/inner skin), coating condition, dyeing state, logo (plating) condition, etc. of the preliminary seized property, as well as production information or used goods according to the year of sale of the preliminary seized property. The market price can be used as learning data for the price estimation model (B).

In this way, the learning module 220 inputs the images of the learning data into the genuine classification model (A) and repeatedly trains them to output the emotion results.

The learning module 220 analyzes one or more characteristic points such as logos, symbols, patterns, letters, metal decorations, stitching, texture, color, and material of the preliminary seized property, and provides an appraisal to classify it as genuine or fake based on the analyzed characteristic points. The genuine classification model (A) is trained repeatedly so that the results are output.

In addition, the learning module 220 uses learning data that matches the loss rate calculated based on the reference image for each used luxury product image, and outputs the matched loss rate when an image of a used luxury product is input, as well as production information, production/sales of the used luxury product. The price estimation model (B) is repeatedly trained so that the loss rate is applied to the transaction prices collected based on the year and the used price of used luxury goods is output.

In detail, a standard image with a loss rate of 0 is selected for each type of luxury product, the reference image and the used luxury product image are compared to extract areas with differences, and the corresponding area is analyzed to create a used product with the loss rate and the loss rate reflected compared to the same area of the reference image. The price estimation model (B) is trained repeatedly to calculate the price.

At this time, the area where there is a difference represents the area where scratches, contamination, presence of accessories, damage, damage, etc. were detected in the used luxury product image compared to the reference image.

The learning module 220 generally designates an area where the sense of use is noticeable or a point area that is easily contaminated when using the luxury product, and then calculates the loss rate by quickly comparing only the point area in the reference image and the point area in the image of the used luxury product. can do.

In addition, the learning module 220 collects the transaction price based on the production information and year of production/sale of the used luxury product from the image of the used luxury product, and applies the calculated loss rate to the collected transaction price to derive the used market price for the used luxury product. Learn as repeatedly as possible.

In this way, the learning module 220 can repeatedly train the genuine product classification model (A) and the price estimation model (B) so that the accuracy of the genuine product classification model (A) and the price estimation model (B) exceed each set threshold based on the learning data, and classifies the genuine product at regular intervals or at random cycles. The model (A) and the price estimation model (B) can be retrained.

The authenticity verification module 230 uses the learned authenticity classification model (A) to determine whether the preliminary seized property included in the input image is genuine in real time.

The authenticity verification module 230 may perform preprocessing on the input image according to the input format of the authenticity classification model (A). Pre-processing operations refer to image size adjustment, resolution adjustment, enlargement of a specific area, etc., and the authenticity confirmation module 230 inputs the pre-processed image into the authenticity classification model (A), and the area that serves as the basis for judgment in the image as to whether it is genuine or not. is displayed in red and output.

The market price measurement module 240 uses the learned market price estimation model (B) to measure the genuine product in the genuine product verification module 230, or when the separately input precise appraisal result determines that the product is genuine, in the point area of the image input in real time. Calculate the loss rate for each. At this time, an image displayed in red for the area on which the loss rate is calculated for each point area is output along with the loss rate.

And the price measurement module 240 uses the learned price estimation model (B) to calculate the used price by applying the loss rate to the point area. The price measurement module 240 can calculate the amount by depreciation according to the loss rate for each point area, and in this process, information on actual used used price prices can be collected and referenced.

Figure 3 is a flowchart showing a method for verifying authenticity according to an embodiment.

As shown in FIG. 3, the authenticity verification device 200 receives an image of a preliminary seized property (S110).

The authenticity verification device 200 may receive one or more images of preliminary seized property based on the case number of the seizure execution from the user terminal 100 that has performed user authentication.

Here, the image may include the entire composition of the preliminary seizure property or may be a close-up shot of a specific area, such as the logo area.

The authenticity verification device 200 sets minimum standards such as resolution, size, and brightness of the received image, and if it does not meet the set standards, it can request a rephotographed image from the user terminal 100.

Next, the authenticity verification device 200 inputs the received image into the learned authenticity classification model (S120).

The authenticity verification device 200 preprocesses the received image according to the input format of the trained genuine product classification model, and inputs the preprocessed image into the trained genuine product classification model.

And the authenticity verification device 200 outputs whether the product is genuine from the learned authenticity classification model (S130).

The authenticity verification device 200 outputs an image displayed in the area that serves as a standard for determining authenticity in the input image, along with the authenticity status.

At this time, the authenticity verification device 200 may store the input image, authenticity, and an image displaying the judgment criteria area in a linked database by linking the case number of the corresponding seizure execution and the certified administrator ID.

The authenticity verification device 200 checks whether the preliminary seized property is genuine (S140) and, if it is determined to be a counterfeit product, provides the result to the user terminal 100 (S150).

If the product is a counterfeit product rather than a genuine product, that is, a counterfeit product or a similar product, the public auction procedure cannot be carried out for the product. Therefore, the authenticity verification device 200 directly transmits the fact to the user terminal 100 without separately calculating the used price. do.

On the other hand, if the preliminary seized personal property is genuine, the authenticity confirmation device 200 collects the precise appraisal results of the same preliminary seized personal property and confirms whether the precise appraisal results are genuine (S160).

In other words, the authenticity verification device 200 performs a primary appraisal based on the learned genuine product classification model (A), and can finally determine whether the product is genuine or counterfeit through the collected secondary precision appraisal results by an expert. .

And, if the product is judged to be counterfeit as a result of the precise appraisal, the authenticity verification device 200 collects the determined image as learning data (S170).

If the genuine product confirmation device 200 determines the product to be genuine based on the learned genuine product classification model, but determines the product to be counterfeit based on the detailed appraisal result, priority is given to the precise appraisal result.

Accordingly, the authenticity verification device 200 matches the image determined to be a counterfeit product with the precision appraisal result, collects it as learning data, and retrains the genuine product classification model based on the collected training data.

Next, if the authenticity verification device 200 determines that the product is genuine as a result of the precise appraisal, the authenticity verification device 200 performs preprocessing on the image input in step S110 based on the input format of the market price estimation model, and Input to the learned price estimation model.

The authenticity verification device 200 calculates the used price according to the used market price based on the loss rate output from the learned market price estimation model (S190).

The authenticity verification device 200 extracts one or more differences between a reference image with a loss rate set to 0 and an input image, sets a point area including the extracted differences, and then analyzes the state of each point area to calculate a loss rate.

At this time, the weight for the loss rate may be set differently for each point area.

For example, in the case of a bag, if there are identical scratches in the point area including the handle and the point area located on the front of the bag, the scratch loss rate in the front point area may be set to be greater than in the handle point area. In detail, even if the same scratch is found, the loss rate may be set differently depending on the cost, technology, and materials required to compensate for the scratch.

The authenticity verification device 200 outputs an image displayed in an area that serves as a standard for determining authenticity in the input image along with a loss rate calculated for each point area.

And the authenticity verification device 200 measures the loss rate according to each state, such as scratches, contamination, presence of accessories, damage, and damage, for each point area.

Specifically, the authenticity verification device 200 can measure the loss rate by analyzing conditions such as damage, damage, wear, or contamination by analyzing the leather (outer skin/inner skin), coating state, dyeing state, and logo/plating state.

And, based on the obtained loss rate, the authenticity verification device can calculate the used price by applying the depreciation triangle to the price of the used product corresponding to the same production information and sales year in which the loss rate is 0. Here, production information may include brand name, production location, production year, production volume, etc.

The authenticity verification device 200 can store the input image, the loss rate for each point area, the image displaying the judgment standard area, and the calculated used price in a linked database by linking the case number of the corresponding seizure execution and the certified administrator ID. there is.

Next, the authenticity verification device 200 provides the used price of the preliminary seized property judged to be genuine (S200).

The authenticity verification device 200 may provide the calculated used price to the user terminal 100 or the electronic public auction system 300 along with the authenticity of the preliminary seized property for the received image.

In this way, the authenticity verification device 200 is explained as checking the authenticity of the preliminary seized property of the received image, and if it is genuine, calculating the used price and providing it together. However, it is implemented differently according to the request in the user terminal 100. It can be.

For example, if the user terminal 100 only requests authenticity, the authenticity verification device 200 may provide information by only checking whether the preliminary seizure property is genuine, and if only the used price of the preliminary seizure property is requested, The authenticity verification device 200 can calculate and provide only the used price.

Figure 4 is an example diagram showing and providing an area that serves as the basis for determining authenticity according to one embodiment.

This shows an image that determines the authenticity of the luxury logo in Figure 4. Real or fake is calculated for the predicted result based on the actual input image, and the result is judged as genuine or fake. You can see that areas with high relevance are marked in red, and areas with low relevance are marked in blue.

Accordingly, managers can check whether the preliminary seized property is genuine or fake based on the area marked in red.

Below, the process of the user terminal 100 accessing the authenticity verification device 200 on-site, requesting prices for genuine and used products, registering seized property on-site, and conducting an electronic public auction will be described in detail.

Figure 5 is a flowchart of conducting an electronic public auction for seized property according to an embodiment.

As shown in Figure 5, the user terminal 100 photographs the preliminary seized property (S201).

The user terminal 100 can take one or more images of the preliminary seizure property at the site, and can capture an entire image of the preliminary seizure property, an enlarged image of a specific area, etc.

Next, the user terminal 100 transmits the captured image to the authenticity verification device 200 and requests authenticity verification (S203).

The user terminal 100 can select one or more captured images and transmit them to the authenticity verification device 200, and can only request authenticity confirmation, request genuine and used prices, or only use price requests.

Below, it is explained that in the case of a genuine product, the used price is also requested.

Then, the authenticity verification device 200 confirms the authenticity of the preliminary seized property based on the received image (S205). At this time, the authenticity confirmation device 200 can first check the authenticity based on the received image through the authenticity classification model (A), and then secondarily confirm the authenticity based on a precise appraisal by an expert. In other words, the authenticity verification device 200 can determine that the preliminary seized property is genuine when both the first and second appraisal results are determined to be genuine.

And the authenticity verification device 200 calculates the used price of the preliminary seized property (S207).

The authenticity verification device 200 uses the authenticity classification model (A) and the market price estimation model (B) to determine whether the preliminary seized property included in the image is genuine and, if it is genuine, to calculate the used price. At this time, the authenticity verification device 200 links and stores the analysis result data (authenticity, used price) corresponding to the requested image.

Next, the authenticity verification device 200 provides the used price for the preliminary seized property that is genuine to the user terminal 100 (S209).

In the case of a counterfeit product, the authenticity verification device 200 may provide only the fact that the product is a counterfeit product to the user terminal 100, and in the case of a genuine product, it may also provide a used price for the preliminary seized property that is a genuine product.

Then, the user terminal 100 designates the preliminary seized property as a seized property (S211).

At this time, the user terminal 100 designates the seized property based on the arrears of the delinquent person, and if the used price is set at a price greater than the required cost, the user terminal 100 can designate the preliminary seized property as the seized property.

The required cost refers to the minimum amount allocated to the object of seizure based on the cost of issuance, such as the second-hand price of the preliminary seized property, the cost of seizing, the cost of storing the goods, and the cost of conducting a public auction. can be easily changed and set by the administrator.

In addition, if the used price is set lower than the cost, it may not be designated as a seized property, and may or may not be designated as a seized property based on the target amount of arrears.

Next, the user terminal 100 transmits the designated seized property to the electronic public auction system 300 and proceeds with registration of the seized property (S213).

The user terminal 100 registers the preliminary seized property whose authenticity has been confirmed as a seized property in the electronic public auction system 300. At this time, the case number of the seizure execution entered to confirm the authenticity or basic information of the preliminary seized property (brand name, product category name, date of purchase, etc.) and one or more captured images can be transmitted to the public auction system 300.

The electronic public auction system 300 assigns an identification number to the seized property (S215).

The electronic public auction system 300 assigns an identification number to the registered seized property and stores the assigned identification number in connection with the case number of the seizure execution.

Then, the electronic public auction system 300 requests judgment information about the seized property from the authenticity verification device 200 (S217). The electronic public auction system 300 may request judgment information on stored seized property based on the case number of the corresponding seizure execution.

Then, the authenticity verification device 200 provides judgment information about the seized property to the electronic public auction system 300 (S219).

The electronic public auction system 300 can process the received judgment information about seized property into information posted for public auction.

For example, the electronic public auction system 300 converts the area marked in red, which is the basis for judgment in the image, into a detailed image in order to set a used price. Here, the detailed image does not display the corresponding red color, but the status of the product can be provided by providing an image of the corresponding area.

Next, the electronic public auction system 300 posts 3D-based seized property item information online (S221).

The electronic public auction system 300 not only clearly provides bidding participants with scratches, contamination, accessories, damage or damage to the seized property through detailed images of the seized property. By creating 3D images, information on seized property can be posted online based on identification numbers.

This can be separately captured using a 3D stereo camera to create a 3D image, or a 3D image can be created by combining 2D images from various angles received from the user terminal 100.

Next, the electronic public auction system 300 selects a successful bidder using a bidding method (S223).

The electronic public auction system 300 can select a successful bidder through a bidding method in which the previously set used price is used as the lowest price and the person who wishes to purchase the item at the highest desired price is selected. The method of selecting a successful bidder is not limited to the bidding method, and various methods can be applied, such as a method of selecting the person who desired the price as the successful bidder on a first-come-first-served basis.

Next, the electronic public auction system 300 guides the transfer process after receiving the seized property (S225).

The electronic public auction system 300 may provide a virtual account to the successful bidder to store the winning bid amount. A virtual account can be opened with the name of the successful bidder and the winning bid amount specified, and can be set to be valid only for a certain period of time.

When the deposit of the winning bid is confirmed, the electronic public auction system 300 may provide a location for delivering the seized property to the successful bidder or guide certain procedures to deliver it to the location desired by the successful bidder.

Meanwhile, after the user terminal 100 first verifies the authenticity of the preliminary seized personal property from the authenticity verification device 200, hands it over to an expert for a detailed appraisal, and if the preliminary seized personal property is authentic in the detailed appraisal, an electronic public auction is conducted. If the preliminary seized property is fake, the procedure for delivering it to the delinquent person can be carried out.

Figures 6 and 7 are exemplary diagrams showing emotional points according to one embodiment.

Figures 6 and 7 show light microscope points for precise appraisal, and the authenticity verification device 200 can analyze the image by enlarging the received image to a resolution that can be confirmed with a light microscope.

In detail, Figure 6 (a) shows emotional point images for the stitch part, and (b) shows emotional point images for the engraved part on the product.

Additionally, Figure 7 (a) shows emotional point images for feature points of the zipper loop portion, and (b) shows emotional point images for the leather pattern.

In addition, areas with high usage, such as logos or handles, can be designated as emotional points and the image can be analyzed.

In this way, the authenticity verification device 200 performs enlargement and pre-processing of the image received by the user terminal 100 into an image that can be confirmed by a light microscope, and analyzes the image based on a precise appraisal procedure to identify a genuine or counterfeit product. You can judge and measure the used market price based on the loss rate.

In addition, as shown in FIGS. 6 and 7, images of precise appraisal points secured by an actual light microscope are collected as learning data, and the authenticity verification device 200 uses an authentic product classification model (A) and a market price estimation model (B). Can be learned and relearned.

According to the present invention, by analyzing images of luxury goods taken on site using a neural network to confirm authenticity, it is possible to exclude counterfeit goods or similar goods with low economic value and secure seized property.

In addition, by quickly determining the authenticity and used price of seized property, the time and cost required for appraisal of seized property can be saved by minimizing the face-to-face item appraisal step by experts.

In addition, by providing a 3D image of the seized property and the basis for setting the real or used price of the seized property at the public auction stage of the seized property, more accurate information on the condition of the seized property can be provided in the public auction process conducted non-face-to-face.

The embodiments of the present disclosure described above are not only implemented through devices and methods, but may also be implemented through programs that implement functions corresponding to the configurations of the embodiments of the present disclosure or recording media on which the programs are recorded.

Although the embodiments of the present disclosure have been described in detail above, the scope of the rights of the present disclosure is not limited thereto, and various modifications and improvements made by those skilled in the art using the basic concept of the present disclosure defined in the following claims are also possible. It falls within the scope of rights.

Claims (11)

As a method of operating an authenticity verification device,
Receiving one or more images containing preliminary seizure property from a linked terminal,
Inputting the image into a learned genuine classification model to obtain an image area corresponding to the result and basis for determining whether the preliminary seized property is genuine or fake,
If the preliminary seizure property is genuine, the image is input into the learned market price estimation model to calculate the loss rate of the preliminary seizure property, and the loss rate is applied to the collected used market price for the preliminary seizure property to determine the used price. steps to calculate, and
If the preliminary seizure property is a counterfeit product, the result of the judgment as a counterfeit product and the image area are provided to the terminal, and if the preliminary seizure property is a genuine product, the result of the judgment as a genuine product and the calculated used price are provided to the terminal. steps to do
Including,
The genuine product classification model and the market price estimation model are,
By applying the GRAD-CAM algorithm, the feature points on the image that serve as the standard for the output results are visualized and output in RGB format.
In the image, an area with a high degree of relevance to the result determined to be a genuine or counterfeit product is displayed in a first color, an area with a low degree of correlation to the result determined to be a genuine product or a counterfeit product is displayed in a second color, and the first color Based on the area marked with, it is confirmed whether the preliminary seized property is genuine or fake,
If the preliminary seizure movable property is genuine, the authenticity verification device collects the results of a precise appraisal of the preliminary seizure movable property by an expert, and checks whether the precise appraisal result is genuine,
The authenticity confirmation device judges the preliminary seized personal property as a counterfeit product according to the precise appraisal result when the genuine product is determined to be genuine in the genuine product classification model and the precise appraisal result is a counterfeit product,
The authenticity verification device collects the precise appraisal results as learning data and retrains the authenticity classification model,
The terminal determines whether to designate the preliminary seized property as a seized property based on the amount of arrears of the delinquent person, and if the second-hand price is set at a price greater than the required cost, the terminal designates the preliminary seized property as a seized property, and If the second-hand price is set at a price lower than the cost, the preliminary seizure property is not designated as a property for seizure, and the costs include the cost of seizing the property, storage costs, and public auction. . In paragraph 1:
The step of acquiring the image area is,
Analyze the image to determine the type and brand name of the preliminary seized property,
An operation method for determining whether the preliminary seizure property is genuine or counterfeit by extracting one or more characteristic points among the logo, symbol, pattern, letter, metal decoration, stitching, texture, color, and material of the preliminary seizure property. In paragraph 1:
The step of calculating the used price is,
Compare the image with a reference image of the same product as the preliminary seizure property and set the different parts as point areas,
An operation method for analyzing the corresponding state among scratches, contamination, presence of accessories, damage, and damage for each point area, and calculating a loss rate according to the analyzed state information. delete In paragraph 3,
An operation method further comprising providing the image area, calculation data for the used price, and the point area when receiving a request for judgment information on the preliminary seized property from an linked electronic public auction server. In paragraph 1:
Set the collected fake images or used genuine images as learning images, match the results of fake or genuine products to the learning images to create learning data, and then input the learning images into the genuine classification model to obtain the matched results. A step of training the genuine product classification model to be output, and
For an appraised used genuine product, the image captured is matched with the damage area and loss rate of the used genuine product to generate learning data, and the captured image is input into the market price estimation model to estimate the market price so that the matched loss rate is output. An operation method further comprising the step of training a model. As an authenticity verification device,
memory containing instructions, and
It includes a processor that executes the above commands to check the authenticity of the input image and calculate the used price,
The processor is
When one or more images containing preliminary seizure property are received from the linked terminal, the image is input into a learned genuine classification model to determine whether the preliminary seizure property is genuine or counterfeit, and the image area corresponding to the basis for judgment is obtained. ,
If the preliminary seizure property is a fake, the judgment result and the image area are provided to the terminal, and if the preliminary seizure property is genuine, the used price for the preliminary seizure property calculated using the learned market price estimation model is provided to the terminal. Provided through the terminal,
The genuine product classification model and the market price estimation model are,
By applying the GRAD-CAM algorithm, the feature points on the image that serve as the standard for the output results are visualized and output in RGB format.
In the image, an area with a high degree of relevance to the result determined to be a genuine or counterfeit product is displayed in a first color, an area with a low degree of correlation to the result determined to be a genuine product or a counterfeit product is displayed in a second color, and the first color Based on the area marked with, it is confirmed whether the preliminary seized property is genuine or fake,
If the preliminary seizure movable property is genuine, the authenticity verification device collects the results of a precise appraisal of the preliminary seizure movable property by an expert, and checks whether the precise appraisal result is genuine,
The authenticity confirmation device judges the preliminary seized personal property as a counterfeit product according to the precise appraisal result when the genuine product is determined to be genuine in the genuine product classification model and the precise appraisal result is a counterfeit product,
The authenticity verification device collects the precise appraisal results as learning data and retrains the authenticity classification model,
The terminal determines whether to designate the preliminary seized property as a seized property based on the amount of arrears of the delinquent person, and if the second-hand price is set at a price greater than the required cost, the terminal designates the preliminary seized property as a seized property, and If the second-hand price is set at a lower price than the cost, the preliminary seizure property is not designated as a seized property, and the costs required to verify the authenticity include the cost of seizing the preliminary seizure property, the cost of storing it, and the cost of public auction. Device. In paragraph 7:
The learned authenticity classification model is,
An authenticity verification device that extracts one or more characteristic points from the logo, symbol, pattern, letter, metal decoration, stitching, texture, color, and material of the preliminary seizure property to determine whether the preliminary seizure property is genuine or counterfeit. In paragraph 7:
The learned price estimation model is,
Compare with the standard image set based on the type and brand name of the preliminary seized property, set the different parts as point areas, and analyze the corresponding status among scratches, contamination, presence of accessories, damage, and damage for each point area. An authenticity verification device that calculates the loss rate based on status information. In paragraph 9:
The processor,
An authenticity verification device that calculates the loss rate for the preliminary seizure property from the learned market price estimation model and calculates a used price by applying the loss rate to the used market price for the preliminary seizure property. As a method of operating an electronic public auction system,
Receiving basic information including the product name, brand name, and seizure case number of the seized property determined to be genuine from the linked terminal, a used price set for the seized property, and one or more images including the seized property,
assigning an identification number to the seized property in response to the seizure case number;
Request judgment information about the seized property from the linked authenticity verification device, receive the image area that serves as the basis for judging the seized property as genuine, and apply the loss rate to the point area where the loss rate of the seized property is calculated and the used market price. receiving used price calculation data;
Creating a 3D image of the seized property, setting the image area and the point area as a detailed image, and specifying the calculated used price as a basic price to generate product information of the seized property,
Posting product information of the seized property online in response to the identification number, and selecting a successful bidder of the seized property through a bidding method according to a designated public auction period, and
When receipt of the seized movable property is confirmed, it includes the step of guiding the transfer process of the seized movable property,
The image area is,
In the image containing the seized property inputted into the learned genuine product classification model, the feature points that serve as the basis for judgment as genuine or counterfeit are shown in the image area visualized in RGB format,
The point area is,
In the image containing the seized property inputted into the learned price estimation model, the characteristic points that serve as the basis for the loss rate calculated by analyzing the corresponding states among scratches, contamination, presence of accessories, and damage and damage are displayed in an area visualized in RGB format,
In the image, an area with a high degree of relevance to the result determined to be a genuine or counterfeit product is displayed in a first color, an area with a low degree of correlation to the result determined to be a genuine product or a counterfeit product is displayed in a second color, and the first color Based on the area marked with, it is confirmed whether the preliminary seized property is genuine or fake,
If the preliminary seizure movable property is genuine, the authenticity verification device collects the results of a precise appraisal of the preliminary seizure movable property by an expert, and checks whether the precise appraisal result is genuine,
The authenticity confirmation device judges the preliminary seized personal property as a counterfeit product according to the precise appraisal result when the genuine product is determined to be genuine in the genuine product classification model and the precise appraisal result is a counterfeit product,
The authenticity verification device collects the precise appraisal results as learning data and retrains the authenticity classification model,
The terminal determines whether to designate the preliminary seizure movable property as the seized movable property based on the delinquent person's arrears, and if the used price is set at a price greater than the required cost, designates the preliminary seizure movable property as the seized movable property, If the second-hand price is set at a price lower than the above-mentioned cost, the preliminary seizure property is not designated as the above-mentioned property for seizure, and the cost includes the cost of seizing, storage, and public auction of the preliminary seizure property. How to operate.