JP2022173741A - Information processing device, information processing system and information processing method - Google Patents

Abstract

To enable the evaluation of an article from a plurality of different viewpoints while reducing user's work about an information processing device for evaluating the article on the basis of photographed image data.SOLUTION: A server 3 includes an evaluation part 13 for acquiring photographed image data generated by photographing an object article by an object user, performing an image analysis of an image of the object article recorded in the photographed image data to recognize a state about the external appearance of the object article, executing at least any one of processing for performing character recognition of an image of characters included in the image of the object article as an object, processing for performing an image analysis of the image of the object image to recognize a model number of the object article, and processing for performing the image analysis of the image of the object article to determine the authenticity of the object article, and evaluating the object article by reflecting at least one of the recognized state about the external appearance of the object article, the recognized characters, the recognized model number and a result of authenticity determination.SELECTED DRAWING: Figure 2

Description

The present invention relates to an information processing device that evaluates an article based on captured image data, an information processing system that includes the information processing device and a terminal, and an information processing method using the information processing device.

2. Description of the Related Art Conventionally, there is known a system for evaluating an article by using photographed image data generated by photographing the article. Regarding this type of system, for example, in Patent Document 1, after a server obtains a panorama photograph (photographed image) generated by photographing a used car, the panorama photograph is stored, and after the fact, the condition of the used car's scratches, etc. It describes a system that enables confirmation by panoramic photographs.

JP 2018-97679 A

In general, when evaluating an article, it is required to evaluate the article not only from a single point of view but also from a plurality of different points of view. Normally, there is not only one element of a product that affects the evaluation, but by evaluating the product from multiple different perspectives, it becomes possible to evaluate the product by taking into account multiple factors, and the validity of the evaluation results is high. Because it becomes In addition, in conventional systems, prior to evaluating an article, the user takes a picture of the article and provides information necessary for evaluating the article (for example, inputting information used for evaluation into a given user interface). ), etc., but this work was troublesome for the user. Based on the above, there is a need to be able to evaluate articles from a plurality of different viewpoints while reducing the user's work.

SUMMARY OF THE INVENTION The present invention has been made to solve such problems, and an object of the present invention is to enable evaluation of an article from a plurality of different points of view while reducing user's work.

In order to solve the above-described problems, the present invention obtains, from a terminal via a network, photographed image data generated by photographing a target article by a target user. Then, the image of the target article recorded in the photographed image data is image-analyzed to recognize the state of the appearance of the target article, and the image of the character included in the image of the target article is processed for character recognition, and the image of the target article is processed. image analysis to recognize the model number of the target article, and image analysis of the image of the target article to determine the authenticity of the target article. Then, the target product is evaluated by reflecting at least one of the recognized appearance state of the target product, the recognized characters, the recognized model number, and the result of authenticity determination.

According to the present invention configured as described above, in addition to the state related to the appearance of the article, characters recognized by character recognition targeting character images and model numbers recognized by image analysis targeting image images of articles , and at least one of the result of authenticity determination performed on the image of the product is reflected in the evaluation of the product, so the product can be evaluated from a plurality of different viewpoints. At this time, at least one of the recognition of the appearance of the article, the recognition of characters by character recognition, the recognition of the model number by image analysis, and the determination of authenticity is performed based on common photographed image data. Therefore, the user only needs to take a picture of the object to evaluate the object, and does not need to separately perform the work of inputting the characters to be recognized, the input of the model number, and the work of providing information to be used for authenticity determination. do not have. That is, according to the present invention, an article can be evaluated from a plurality of different viewpoints while reducing user's work.

1 is a diagram showing a configuration example of an information processing system according to an embodiment of the present invention; FIG. 1 is a block diagram showing an example of functional configurations of a terminal and a server according to one embodiment of the present invention; FIG. It is a figure which shows an example of an article classification tree. It is a figure which shows an example of the screen displayed on a terminal. It is a figure which shows an example of the system relevant information regarding a notebook computer. It is a figure which shows an example of the screen displayed on a terminal. It is a figure which shows an example of the system relevant information which concerns on a smart phone. It is a figure which shows an example of the screen displayed on a terminal. It is a figure which shows an example of the screen displayed on a terminal. 4 is a flowchart showing an operation example of the information processing apparatus according to one embodiment of the present invention;

An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram showing a configuration example of an information processing system 1 according to this embodiment. The information processing system 1 includes a terminal 2 and a server 3 as shown in FIG. Both the terminal 2 and the server 3 are connectable to a network N including the Internet, telephone networks and other communication networks.

The server 3 is a server device that uses the terminal 2 as one of its clients. In FIG. 1, the server 3 is represented by one block, but the server 3 does not have to be a single server device, and may be composed of, for example, a plurality of server devices. There may be. The server 3 is managed and operated by a service provider company (predetermined entity), and provides services to be described later. The server 3 corresponds to an "information processing device" in the claims.

The terminal 2 is a smart phone provided with a touch screen 4 (display panel + touch panel) over a wide area on the entire surface of a tablet-type housing. In particular, the terminal 2 according to the present embodiment is equipped with a camera 5 and has a function of generating captured image data based on the result of capturing by the camera 5 . A predetermined application (hereinafter referred to as “dedicated application”) is installed in the terminal 2 in advance using an application download system. Functions of the dedicated application will be described later. Note that FIG. 1 exemplarily shows one of the innumerable terminals 2 connected to the network N. As shown in FIG.

FIG. 2 is a block diagram showing a functional configuration example of the terminal 2 and the server 3. As shown in FIG. As shown in FIG. 2, the terminal 2 includes a terminal-side control section 10 and a terminal-side communication section 11 as functional configurations. The server 3 also includes a server-side communication unit 12 and an evaluation unit 13 as functional configurations. Each of the functional blocks 10 to 12 can be configured by hardware, DSP (Digital Signal Processor), or software. For example, when configured by software, each of the functional blocks 10 to 12 is actually configured with a computer CPU, RAM, ROM, etc., and a program stored in a recording medium such as RAM, ROM, hard disk, or semiconductor memory. is realized by the operation of In particular, the terminal-side control unit 10 performs various processes by reading and executing a dedicated application (including an OS, an API provided by the OS, and other accompanying programs) stored in a storage medium by hardware such as a CPU. is executable. The server 3 also includes a server-side storage unit 14 as a storage medium.

The terminal-side communication unit 11 communicates with devices connected to the network N according to a predetermined protocol. In the following description, communication between the terminal 2 and devices connected to the network N is assumed to be appropriately performed by the terminal-side communication unit 11, and detailed description thereof will be omitted. Also, the server-side communication unit 12 communicates with devices connected to the network N according to a predetermined protocol. In the following description, communication between the server 3 and devices connected to the network N is assumed to be properly performed by the server-side communication unit 12, and detailed description thereof will be omitted.

Here, the server 3 provides the following services (hereinafter referred to as "this service") to the user. In other words, assume a user who wants to earn money based on the goods he/she owns. Examples of items owned by users include branded bags, branded accessories, and other branded items, and examples include notebook computers, smartphones, game consoles, digital cameras, and other electronic devices (however, these is just an example). The user takes a picture of the article using the terminal 2 , and the taken image data based on the photographed result is transmitted from the terminal 2 to the server 3 . The server 3 derives the assessed value of the article based on the photographed image data, and provides information indicating the assessed value to the user. Note that in the following description, the term "assess" may be used as appropriate to refer to performing various processes and finally deriving the assessed amount.

Although detailed explanation is omitted, after the appraisal value of the item is derived, if the user agrees, the service provider purchases the item at the appraisal value and provides the user with money equivalent to the appraisal value. be done. Articles purchased by the service provider are then leased to the user on the condition of payment of the lease fee. At the time of leasing, a lease contract is concluded between the user and the service provider company, and the user is obliged to pay the lease fee according to the lease contract. Therefore, the user can obtain funds equivalent to the appraisal value while keeping the goods at hand without handing over the goods to the service provider company.

The server 3 cooperates with the terminal 2 to systematically implement the provision of the above services. In particular, the server 3 according to the present embodiment has a unique configuration that will be described below, and while reducing the user's work, performs assessment from a plurality of different viewpoints and derives the assessment amount. The operations of the server 3 and the terminal 2 when providing this service will be described below.

Here, before explaining the processing of the server 3 and the terminal 2, the "article classification" and "the deepest classification" in this embodiment will be explained. In this embodiment, an item classification tree having a tree structure (hereinafter referred to as an "item classification tree") is defined in advance for items that can be assessed in this service. FIG. 3 is a diagram showing an example of an article classification tree. The article classification tree will be described below with reference to FIG. However, the article classification tree illustrated in FIG. 3 is a very simplified one for convenience of explanation. In the example of FIG. 3, three product categories are defined in the hierarchy one level below the root node for the purpose of roughly classifying products: notebook computers (product category), smartphones, and bags (product category). In the following description, when representing the hierarchical state of the product classification, the root node may be omitted, assuming that the root node naturally exists. Also, in this embodiment, the bag is assumed to be a brand-name product.

Two product categories, Windows (product category) and MacOS (product category), are defined by focusing on the OS in the hierarchy one level below the notebook computer (product category) (both Windows and MacOS are registered trademarks). ). Further, in the layer one level below Windows (article classification), focusing on the manufacturer, two article classifications of maker MK1 (article classification) and maker MK2 (article classification) are defined. In addition, two product categories, iOS (product category) and Android (product category), are defined by focusing on the OS in the hierarchy one level below the smartphone (product category) (both iOS and Android are registered trademarks). ). Also, in the hierarchy one level below the bag (goods classification), focusing on the brand of the bag, three product classifications, brand B1 (goods classification), brand B2 (goods classification), and brand B3 (goods classification) are defined. It is

In the above example of FIG. 3, the product classification corresponding to the leaf node of the product classification tree corresponds to the "deepest classification". For example, "Laptop (product classification) > Windows (product classification) > Manufacturer MK1 (product classification)" is one independent "deepest classification", and "Bag (product classification) > Brand B3 (product classification)" is also independent. is one "deepest classification". The deepest classification corresponds to "classification" in claims. Further, as is clear from the above description, the "article classification" means each classification when classifying the upper hierarchy into two or more in a certain hierarchy.

When using this service, the user activates a dedicated application on the terminal 2 . In response to activation of the dedicated application, the terminal-side control unit 10 of the terminal 2 and the evaluation unit 13 of the server 3 cooperate to appropriately perform user authentication. to recognize Hereinafter, the user of the terminal 2 (the user who intends to use this service) will be particularly referred to as the "target user", and the item that the target user will be subject to assessment will be particularly referred to as the "target item".

Next, the target user performs a predetermined operation on the user interface provided by the function of the dedicated application to display the second layer article classification selection screen G4A (FIG. 4A) on the touch screen 4 of the terminal 2. instruct. In response to the instruction, the terminal-side control unit 10 of the terminal 2 and the evaluation unit 13 of the server 3 cooperate to execute deepest classification recognition processing. The deepest classification recognition processing will be described in detail below.

In the deepest classification recognition process, the terminal-side control unit 10 of the terminal 2 requests the server 3 for drawing data for displaying the second layer article classification selection screen G4A. The request is made, for example, as an HTTP request. In response to the request, the evaluation unit 13 of the server 3 generates drawing data and responds to the terminal-side control unit 10 of the terminal 2 . The terminal-side control unit 10 of the terminal 2 receives the drawing data, and displays the second layer article classification selection screen G4A on the touch screen 4 based on the drawing data.

Generation/transmission of drawing data by the evaluation unit 13 of the server 3, detection of input to various screens displayed based on the drawing data, acquisition of information input to various screens, dynamic change of various screens, etc. , are appropriately executed using CGI, JavaScript (registered trademark), Ajax, DHTML, and other existing technologies, and detailed description thereof will not be given. The evaluation unit 13 of the server 3 transmits drawing data to the terminal-side control unit 10 of the terminal 2 and displays a screen based on the drawing data on the touch screen 4 of the terminal 2. It may be expressed as "the unit 10 displays the XX screen on the touch screen 4" or simply "the evaluation unit 13 displays the XX screen".

FIG. 4A is an example of the second layer article classification selection screen G4A. The second layer product classification selection screen G4A is a screen displaying a list of icons indicating each of the second layer product classifications in the product classification tree. Each icon can be selected by touch operation. The second layer of product classification roughly classifies the products that can be assessed by this service, and sufficient types are prepared so that the products that can be assessed by this service belong to any product category. It is The target user selects the product category by touching the icon corresponding to the product category of the target product.

When one of the product categories is selected on the second layer product category selection screen G4A, the evaluation unit 13 causes the terminal-side control unit 10 to display an icon indicating each of the product categories one level below the selected product category. is displayed on the touch screen 4. The target user selects an article classification by touching any one of the icons displayed on the screen. As described above, when the target user selects an article classification on a certain screen, a screen displaying a list of article classifications one level below the selected article classification is displayed on the touch screen 4, and the target user When an article classification is selected, a screen displaying a list of article classifications one level below the selected article classification is displayed on the touch screen 4. This process is repeatedly executed until the target user selects the deepest classification. be done.

For example, assume that the hierarchical structure of the product classification conforms to the product classification tree illustrated in FIG. 3, and the target product is a “smartphone whose OS is iOS”. In this case, referring to FIG. 4, the target user selects the icon B4A indicating the smart phone (article classification) on the second layer article classification selection screen G4A (FIG. 4A), and according to this, the article classification related to the OS is selected. A screen G4B (FIG. 4B) for selecting is displayed, and the target user selects an icon B4B indicating iOS (item classification) on this screen G4B. As described above, the target user selects "smartphone (item classification)>iOS (item classification)" as the deepest classification.

The evaluation unit 13 recognizes the deepest classification (the deepest classification of the target article) selected by the target user as described above. Hereinafter, the deepest classification of the target article will be referred to as "the deepest target classification". The above processing is the deepest classification recognition processing.

After the deepest classification recognition processing, the evaluation unit 13 and the terminal-side control unit 10 cooperate to execute the captured image data acquisition processing. The photographed image data acquisition process will be described in detail below.

As described above, in this embodiment, the deepest classification is defined according to the article classification tree, and an article belongs to any of the deepest classifications. In this embodiment, for each deepest classification, it is determined in advance by what method the photographed image data is required to be captured and generated. The mode of processing is different. Hereinafter, a method for photographing an article will be referred to as an "article photographing method". As will become clear later, the photographed image data is used when deriving the appraisal value of the target article. The method of photographing an article for a certain deepest classification shall be determined from the viewpoint of preparing data with appropriate content as photographed image data to be used when deriving the appraised value of an article belonging to the deepest classification. It is

Below, assuming that the product classification tree follows the tree shown in FIG. The photographed image data acquisition process will be described using the case of "bag (item classification)>brand B1" as an example.

In the following description, "model number" means information for identifying the type, type, type, or model of an article. The model number is sometimes called a manufacturer's model number, part number, product number, model name, model, model number, model, model number, model, type, product, and the like. A "unique number" means unique information for each article. The unique number is sometimes called a manufacturing number, a manufacturing number, a serial number, a serial number, or the like. Naturally, even articles with the same model number always have different unique numbers. Particularly for smartphones, the IMEI corresponds to a unique number.

<When the target deepest classification is "Laptop PC (article classification) > Windows (article classification) > Maker MK1 (article classification)">
First, the photographed image data acquisition process when the target deepest classification is "notebook computer (article classification)>Windows (article classification)>manufacturer MK1 (article classification)" will be described. In this case, the target article belongs to notebook computers as a broad classification, and notebook computers generally have the following characteristics.
(1) It has a display on which system-related information including at least the model number and specs of a given element can be displayed.
(2) Even if the model number is the same, the specifications may differ.
(3) Degradation occurring in a portion including the front of the display affects assessment (evaluation).
In other words, the notebook computer has a display, and is capable of displaying system-related information including at least the model number and specs of predetermined elements on the display; , and deterioration occurring in a portion including the front of the display affects the evaluation.

Regarding (1), system-related information is various types of information related to electronic devices, such as hardware and software specs, specifications, settings, statuses, attributes, and the like. System-related information is what is called system information in Windows-based systems. In general, smartphones, general-purpose notebook computers, general-purpose tablet terminals, general-purpose laptop terminals, and general-purpose desktop terminals are configured so that system-related information can be displayed on the display by performing a predetermined operation according to the installed OS. ing. In addition, there are dedicated devices such as digital watches that are configured to display system-related information on their displays.

FIG. 5 is a diagram showing a typical example of system-related information that can be displayed on a notebook computer. As shown in FIG. 5, system-related information related to a notebook computer usually includes model number, unique number, processor type and total number of memories. The types of processors and the total number of memories correspond to "specifications of predetermined elements" in claims. Note that the model number, unique number, processor type, and total number of memories in FIG. For example, the processor type may be expressed as a processor name, and the total number of memories may be simply expressed as memory.

Regarding (2), for example, notebook computers may allow users to choose the type of processor at the time of purchase, and even notebook computers with the same model number may have different processor types (=different specifications). Also, for example, in some notebook computers, memory (main memory) can be added later, and even notebook computers with the same model number have different total memory (capacity of main memory) (= different specifications). Sometimes. Regarding laptops, differences in specs affect the assessment.

With respect to (3), the portion including the front of the display is specifically the display and keyboard that are exposed at the same time when the notebook computer is opened. Deterioration of the display includes cracks, scratches, stains, dents, and the like. Deterioration of the keyboard includes loss of parts, scratches, stains, wear, and the like. Deterioration of the display and keyboard affects usability of the notebook computer, so it has a stronger impact on the assessment compared to deterioration in other parts.

Now, if the target deepest classification is "notebook computer (article classification) > Windows (article classification) > maker MK1 (article classification)", the evaluation unit 13 first refers to the article photography method database stored in the server-side storage unit 14. to obtain the article photographing method-related data corresponding to the object deepest classification. The article photographing method database is a database in which article photographing method-related data is registered for each deepest classification. The article photographing method-related data includes drawing data of various screens that the evaluation unit 13 causes the terminal-side control unit 10 to display in the photographed image data acquisition process. The processing of the evaluation unit 13 in the photographed image data acquisition processing described below is executed based on the article photographing method-related data corresponding to the object deepest classification.

The evaluation unit 13 causes the terminal-side control unit 10 to display the screen G6A of FIG. 6(A) on the touch screen 4 . Screen G6A instructs the user to open the notebook computer, display system-related information on the display, and take a picture so that the entire area of the display and keyboard (including the front of the display) is photographed. This is a screen for guidance (explanation). In particular, on the screen 6A, the user is instructed to display the model number, unique number, processor type, and total number of memories included in the system-related information on the display. This is because, as will become clear later, this information will be used in the assessment.

As shown in FIG. 6A, at the top of the screen 6A, the notebook computer is opened and system-related information (expressed as "system information" in FIG. 6A) is displayed on the display, and the display and keyboard are displayed. A text is displayed to instruct the user to take a picture so that the In the screen G6A, an image representing an open notebook computer is displayed below the text. This image is an image for giving the target user an image of what method should be taken, and is of course not limited to the content illustrated in FIG. 6(A).

On the screen G6A, an explanation for explaining the photographing procedure is displayed below this image. Although omitted in FIG. 6(A), the explanation includes the system-related information (especially the model number, unique number, processor type and total memory number included in the system-related information) displayed on the display. and information that the target user should recognize and keep in mind in order to capture the entire keyboard. For example, the explanation includes information indicating a procedure for displaying system-related information on the display. For example, when the procedure differs for each OS version, the procedure for each OS version is shown in the information indicating the procedure. Also, the description reflects that the OS of the notebook computer is Windows and that the manufacturer of the notebook computer is MK1.

The screen G6A is defined in the scope of claims as "a screen that guides the character of the target article to be photographed together with the appearance of the target article" and "a portion including the front of the display is photographed while system-related information is displayed on the display." It corresponds to the screen that guides you to That is, when the target user takes a picture of the target article, the evaluation unit 13 recognizes the classification of the target article prior to photographing, and the recognized classification is the classification to which the electronic device having the above-described three features belongs. In the case of , the terminal 2 displays a screen that guides the user to take an image of a part including the front of the display while the system-related information is displayed on the display.

A button B6A for instructing the start of photographing is displayed on the screen G6A of FIG. 6A, and the target user operates the button B6A when preparation for photographing is completed. In response to the operation of the button B6A, the terminal-side control unit 10 activates the camera function of the terminal 2, cooperates with the evaluation unit 13, and selects a frame in which the target article should be placed when photographing, as shown in FIG. 6B. A screen G6B displaying a line and precautions for photographing is displayed. The target user takes a picture while the screen G6B is displayed, and the taken image data is generated according to the picture taken. The terminal-side control unit 10 transmits the captured image data to the evaluation unit 13, and the evaluation unit 13 acquires the captured image data.

As described above, regarding the captured image data acquisition process, when the target deepest classification is "notebook computer (article classification) > Windows (article classification) > manufacturer MK1 (article classification)", the system-related information is displayed on the display. , and a portion including the front of the display (more specifically, display + keyboard) is guided by the evaluation unit 13 to be photographed. Then, when the target user takes a photograph according to the guidance, one photographed image data is generated in which an image of the notebook computer (target article) with system-related information shown on the display is recorded, and the evaluation unit 13 evaluates this one. Acquire two captured image data.

In the above, the photographed image data acquisition processing when the target deepest classification is "notebook computer (article classification) > Windows (article classification) > maker MK1 (article classification)" has been described. , the photographed image data acquisition process is executed in a similar manner. Whether or not it has the same characteristics as a notebook computer is determined based on the deepest classification. An article having characteristics similar to a notebook computer is, for example, a laptop computer.

<When the target deepest classification is "smartphone (article classification) > iOS (article classification)">
Next, photographed image data acquisition processing in the case where the target deepest classification is "smartphone (item classification)>iOS (item classification)" will be described. In this case, the target article belongs to smartphones as a broad classification, and smartphones generally have the following characteristics.
(1) A touch screen (display) is provided in a large front area, and system-related information including at least the model number and IMEI (unique number) can be displayed on the display.
(2) A terminal may be purchased by installment (payment in installments).
(3) Deterioration of the touch screen (display) on the front of the housing affects the assessment.
In other words, the smartphone has a display in a large area on the front, and system-related information including at least the model number and unique number can be displayed on the display. It is a tablet terminal with three characteristics that the deterioration occurring in the part containing affects the evaluation.

Regarding (1), FIG. 7 is a diagram showing a typical example of system-related information that can be displayed on a smartphone. As shown in FIG. 7, system-related information relating to a smartphone typically includes a model number (expressed as a model number in FIG. 7) and an IMEI (unique number). Regarding (2), a service that enables smartphones to be purchased by installment payment is widely available, and terminals may be purchased by installment payment through this service. Regarding (3), the deterioration of the touch screen affects the usability of the smartphone, so it has a stronger impact on the assessment compared to the deterioration of other parts. With smartphones in particular, cracks can occur in the touchscreen, and in general, if the touchscreen is cracked, the smartphone's appraisal value will be significantly lower.

Now, when the target deepest classification is "smartphone (article classification)>iOS (article classification)", the evaluation unit 13 acquires the article photography method-related data corresponding to the target deepest classification from the article photography method database, Based on the method-related data, the following processes are performed. That is, the evaluation unit 13 causes the terminal-side control unit 10 to display the screen G8 of FIG. 8 on the touch screen 4 instead of the screen G6A of FIG. 6(A). The screen G8 is a screen for instructing shooting so as to capture the entire front surface of the housing with the system-related information displayed on the touch screen, and for guiding (explaining) the shooting procedure. In particular, on the screen G8, the user is instructed to display the model number and IMEI (unique number) included in the system-related information on the touch screen. This is because, as will become clear later, this information will be used in the assessment.

As shown in FIG. 8, system-related information (expressed as "system information" in FIG. 8) is displayed on the touch screen at the top of the screen G8. There is a text indicating that Further, on the screen G8, below the text, an image showing the smartphone viewed from the front is displayed. This image is an image for giving the target user an image of what method should be taken, and of course the content is not limited to that illustrated in FIG. 8 .

On the screen G8, an explanation for explaining the photographing procedure is displayed below this image. Although it is omitted in FIG. 8, the description includes the system-related information (especially the model number and IMEI) displayed on the touch screen, and the target user to capture the entire front surface of the housing. contains information that should be recognized and heeded. For example, the explanation includes information indicating a procedure for displaying system-related information on the touch screen. For example, when the procedure differs for each OS version, the procedure for each OS version is shown in the information indicating the procedure. In addition, the description reflects that the OS of the smartphone is iOS.

The screen G8 is "a screen that guides the character of the target article to be photographed together with the appearance of the target article" and "the housing is photographed from the front with the system-related information displayed on the display." It corresponds to the screen that guides you to That is, when the target user takes a picture of the target article, the evaluation unit 13 recognizes the classification of the target article prior to photographing, and the recognized classification is the classification to which the tablet terminal having the above-described three characteristics belongs. In the case of , the terminal 2 displays a screen that guides the user to photograph the housing from the front while the system-related information is displayed on the display.

When the button B8 on the screen G8 is operated, a screen corresponding to FIG. 6B is displayed, and the target user takes an image. Photographed image data is generated according to photographing, the photographed image data is transmitted from the terminal-side control unit 10 to the evaluation unit 13, and the evaluation unit 13 acquires the photographed image data.

As described above, in the captured image data acquisition process, when the target deepest classification is "smartphone (item classification) > iOS (item classification)", the system-related information is displayed on the touch screen, and the housing is viewed from the front. Guided to be photographed. Then, when the target user takes an image according to the instruction, a piece of captured image data is generated in which an image of the smartphone (target article) with the system-related information displayed on the touch screen is recorded, and the evaluation unit 13 to get

In the above, the photographed image data acquisition processing when the target deepest classification is "smartphone (item classification) > iOS (item classification)" has been described. Data acquisition processing is executed. An example of an article that has similar features to a smartphone is a tablet terminal other than a smartphone (however, it can be purchased on an installment basis).

<When the target deepest classification is "Bag (article classification) > Brand B1 (article classification)">
Next, photographed image data acquisition processing in the case where the target deepest classification is "bag (item classification)>brand B1 (item classification)" will be described. In this case, the target article belongs to the bag as a broad classification, and the bag generally has the following features.
(1) While there are cases where the model number as text is not recorded on the article, the model number can be determined from the appearance of the article.
(2) It may be a counterfeit, and it is important to check whether the item is genuine.
In other words, while there are cases where the model number is not recorded as letters on the item, the bag has the characteristic that the model number can be determined from the appearance of the item, and there are cases where it is a counterfeit. It is an article that has two characteristics of what is important.

Regarding (1), the model number may be recorded on the bag by engraving, printing, or other means, but it may not be recorded. In particular, depending on the brand, there are cases where the model number is not uniformly recorded on the bags provided by the brand. On the other hand, in general, the model number of a bag can be identified by its external features when the bag is viewed from the front. With regard to (2), there are cases in which the bag is counterfeit, and it is important to ascertain the authenticity of the bag when making an assessment. In general, authenticity determination is performed using an authenticity determination object represented in a bag. The object of authenticity determination is, for example, a logo, pattern, mark, characteristic two-dimensional or three-dimensional shape, etc. recorded on the article. In some cases, not a part of the article but the entire article (for example, the three-dimensional shape of the entire article) is subject to authenticity determination.

When the target deepest classification is "bag (article classification)>brand B1 (article classification)", the evaluation unit 13 acquires article photography method-related data corresponding to the target deepest classification from the article photography method database, The following processing is executed based on the imaging method-related data. That is, the evaluation unit 13 causes the terminal-side control unit 10 to display the screen G9 of FIG. 9 on the touch screen 4 instead of the screen G6A of FIG. 6A. The screen G9 is a screen for instructing to shoot so that the entire front of the bag is captured, and for guiding (explaining) the procedure for shooting. The reason for asking the target user to photograph the entire front of the bag is that the appearance of the bag when photographed from the front can be most effectively used to identify the model number of the bag, and the bag is photographed from the front. This is because the elements related to the bag, which are sometimes photographed, can be most effectively used to determine the authenticity of the bag.

As shown in FIG. 9, at the top of the screen G9 is displayed a text instructing to shoot so that the entire front of the bag is captured. Further, on the screen G9, below the text, an image showing the bag viewed from the front is displayed. This image is an image for giving the user an image of what method should be taken, and is of course not limited to the content illustrated in FIG. 9 .

On the screen G9, an explanation for explaining the photographing procedure is displayed below this image. Although omitted in FIG. 9, the explanation contains information that the user should recognize and pay attention to in order to capture the entire front of the bag. In addition, the description reflects that the brand of the bag is brand B1.

The screen G9 is "a screen for guiding the appearance of the target article to be photographed so that the generated photographed image data can be used for recognition of the model number", "the generated photographed image data A screen that guides you to photograph the appearance of the target item so that the data can be used to determine the authenticity of the target item" and "The captured image data that is generated can be used to recognize the model number and to determine the authenticity of the target item. It corresponds to "a screen that guides you to photograph the appearance of the target article so that it can be used as possible data." That is, when the target user takes a picture of the target article, the evaluation unit 13 recognizes the classification of the target article prior to photographing. 2, the terminal 2 is caused to display a screen for guiding the external appearance of the target article to be photographed so that the generated photographed image data can be used for recognition of the model number and for judgment of authenticity.

When the button B9 on the screen G9 is operated, a screen corresponding to FIG. 6B is displayed, and the user takes an image. Photographed image data is generated according to photographing, the photographed image data is transmitted from the terminal-side control unit 10 to the evaluation unit 13, and the evaluation unit 13 acquires the photographed image data.

As described above, in the photographed image data acquisition process, when the target deepest classification is "bag (item classification)>brand B1 (item classification)", the evaluation unit 13 guides so that the entire front of the bag is photographed. . Then, when the target user has taken a photograph according to the instruction, a piece of photographed image data in which an image of the front of the bag is recorded is generated, and the evaluation unit 13 acquires this. The photographed image data acquisition processing when the target deepest classification is "bag (item classification) > brand B1 (item classification)" has been described above. Image data acquisition processing is executed. Articles having features similar to bags are, for example, shoes and other branded items other than bags.

The photographed image data acquisition processing executed by the evaluation unit 13 and the terminal-side control unit 10 has been described above. As described above, as a result of the photographed image data acquisition process, the article photographing method corresponding to the deepest target classification is guided, and (ideally) the target article is photographed by the method corresponding to the deepest target classification. Image data is generated, and the evaluation unit 13 acquires the generated photographed image data.

After the photographed image data acquisition process, the evaluation unit 13 executes an assessment process. The assessment process will be described in detail below. Regarding assessment processing, the evaluation unit 13 executes assessment processing according to the target deepest classification. Based on this, below, the case where the deepest target classification is "Laptop PC (article classification) > Windows (article classification) > Manufacturer MK1 (article classification)" and the case where the target deepest classification is "Smartphone (article classification) > iOS (article classification) )” and the case where the deepest target classification is “bag (goods classification)>brand B1”.

<When the target deepest classification is "Laptop PC (article classification) > Windows (article classification) > Maker MK1 (article classification)">
First, the processing of the evaluation unit 13 when the target deepest classification is "notebook computer (article classification)>Windows (article classification)>manufacturer MK1 (article classification)" will be described. As described above, in this case, the evaluation unit 13 acquires captured image data generated by capturing an image of the entire display and keyboard while the system-related information is being displayed on the display.

First, the evaluation unit 13 executes expected image determination processing. The expected image determination process determines whether or not the captured image data acquired by the evaluation unit 13 is appropriate for the normal execution of subsequent processes using the captured image data. It is a process to For example, when the target article is not captured in the expected manner (for example, when the target article is not captured from the expected viewpoint, or when the image of the target article is abnormally small) It is determined that the content of the image data is not appropriate. Further, for example, the evaluation unit 13 determines that the content of the captured image data is not appropriate when the image of the captured image data is blurred to such an extent that subsequent processing cannot be performed normally.

The expected image determination process is performed using, for example, the following models prepared for each type of deepest classification. The model inputs captured image data, performs various image processing on the input captured image data, and determines whether the content of the input captured image data is normal or not. good). All existing image processing techniques can be applied to the model, and the model may be learned by a predetermined machine learning method (for example, deep learning).

When determining that the content of the captured image data is not appropriate, the evaluation unit 13 executes corresponding processing. For example, as corresponding processing, the evaluation unit 13 causes the terminal-side control unit 10 to display a screen on the touch screen 4 prompting the user to retake the image of the target article.

On the other hand, when determining that the contents of the photographed image data are appropriate, the evaluation unit 13 executes character recognition processing. The character recognition process is a process of recognizing characters on an image of characters (image of system-related information in this example) included in an image of a target product. In the character recognition process, the evaluation unit 13 analyzes the photographed image data by image processing, and identifies the image of the target article from the images recorded in the photographed image data. Next, the evaluation unit 13 analyzes the photographed image data by image processing, and identifies an image of system-related information (an image of characters included in the image of the target article recorded in the photographed image data) from the image of the target article. Next, the evaluation unit 13 analyzes the image of the system-related information by optical character recognition processing, and performs character recognition on the character group indicated by the image of the system-related information.

Next, the evaluation unit 13 executes model number recognition processing. The model number recognition process is a process of recognizing a model number by recognizing characters in an image of system-related information. In the model number recognition process, the evaluation unit 13 identifies a character string indicating the model number from the character group indicated by the image of the system-related information based on the result of the character recognition process, and recognizes the model number. For example, usually in system-related information, combinations of item names and item values are arranged in a list. Based on this, the evaluation unit 13 identifies a character string (for example, system model, model, model, etc.) used as the item name of the model number from the character group indicated by the image of the system-related information, and identifies the identified character string The item value associated with is recognized as the model number. For example, a character string is specified by character string search using text data as a key.

Next, the evaluation unit 13 executes unique number recognition processing. The unique number recognition process is a process of recognizing a model number by performing character recognition on an image of system-related information. In the unique number recognition process, the evaluation unit 13 identifies a character string indicating the unique number from the character group indicated by the image of the system-related information based on the result of the character recognition process, and recognizes the unique number. For example, the evaluation unit 13 identifies a character string (for example, serial number or serial number) used as the item name of the unique number from among the character group indicated by the image of the system-related information, and associates it with the identified character string. Recognizes the item value that is specified as a unique number.

Next, the evaluation unit 13 executes spec recognition processing. The spec recognition process is a process of recognizing specs of predetermined elements (processor type and total number of memories in this example) by performing character recognition on an image of system-related information. In the specification recognition process, the evaluation unit 13 identifies character strings indicating the processor type and the total number of memories from among characters indicated by the image of the system-related information based on the result of the character recognition process, and recognizes them. For example, the evaluation unit 13 identifies character strings used as item names of the processor type and the total number of memories from among the character group indicated by the image of the system-related information, and each character string associated with each identified character string. Recognize item values as processor type and total memory count.

Next, the evaluation unit 13 executes state recognition processing. The state recognition processing is a process of recognizing the state of the appearance of the target article by image analysis of the image of the target article recorded in the photographed image data. This is a process of analyzing the image and recognizing the state of the appearance of the part including the front of the display of the target article. In particular, in this embodiment, the evaluation unit 13 derives the article condition level of the target article based on the photographed image data (that is, deriving the article condition level corresponds to recognizing the appearance state of the target article). . The item condition level refers to the degree to which the condition of the appearance of the item is close to new, taking into consideration the conditions of factors related to deterioration such as scratches, stains, dents, wear, and missing parts. It is a level indicating whether deterioration is small). In the present embodiment, the value of the article condition level ranges from level L5, which indicates the state closest to new (least deteriorated state), to level indicating the state farthest from new (state with the greatest deterioration). Five values up to L1 are prepared.

In particular, in this example, the target article is a notebook computer, and the photographed image data includes images of the front of the display and the keyboard of the notebook computer. The evaluation unit 13 derives the article condition level by reflecting the status of elements related to deterioration such as scratches (including cracks in the display), dirt, dents, wear, missing parts, etc. for the display and keyboard. Note that the evaluation unit 13 derives the article condition level by existing image processing. The data necessary for performing the item condition level is prepared in advance. Necessary data is, for example, template data for each model number when pattern matching is executed using template data corresponding to the model number of an article, and image processing based on the background subtraction method corresponding to the model number is executed, for example. background image data for each model number. Also, at least part of the state recognition processing may be performed using a machine-learned model.

Next, the evaluation unit 13 executes fraud possibility investigation processing. The fraud possibility investigation process is a process of investigating the possibility of fraudulent use of this service using individual numbers. In the fraud possibility investigation process, the evaluation unit 13 refers to the usage history database. The usage history database is a database that manages the usage history of this service (not limited to the usage history of the target user), and each record contains at least the user identification information of the user who used this service and the subject of assessment. and the unique number of the item that was purchased. That is, in the usage history database, user identification information and a unique number are associated and managed for each case of using this service in the past. The usage history database includes "data for managing the unique number of an article in association with user identification information of a user who owned the article at least at one time in the past" and "previously evaluated articles data for managing the unique number of the item and the user identification information of the user who owned the item when the evaluation was performed in association with each other.

After referring to the usage history database in the fraud possibility investigation process, the evaluation unit 13 determines whether or not a record having the same unique number as that of the target article exists in the usage history database. There may be no such records, or there may be one or more. The presence of such a record in the usage history database means that the service has been used for the target article in the past. If there is no such record in the usage history database, the evaluation unit 13 terminates the fraud possibility investigation process and executes the subsequent assessed amount derivation process.

On the other hand, if one or more such records exist in the usage history database, the evaluation unit 13 determines whether or not the user identification information of the target user matches the user identification information of each record. If the user identification information of a certain record is different from the user identification information of the target user, it means that a user different from the target user has used this service with the target product as the target of assessment in the past, and the target user You may be trying to use the Service for items owned by others.

Note that if there are a plurality of records having the same unique number as the unique number of the target article in the usage history database, the evaluation unit 13 determines that even one of the records has a different value from the user identification information of the target user. If there is a record containing the target user, it is determined that the user identification information of the target user does not match the user identification information of each record. However, when the value of the user identification information included in the most recently generated record is different from the value of the user identification information of the target user, the evaluation unit 13 evaluates the user identification information of the target user and the user identification information of each record. may be determined as not matching.

When the user identification information of the target user and the user identification information of each record do not match, the evaluation unit 13 executes corresponding processing. For example, the evaluation unit 13 executes a process of notifying the operator as a corresponding process. Also, for example, the evaluation unit 13 controls the terminal-side control unit 10 to notify that a person other than the target user has used this service for the same item in the past, and determines whether the target item belongs to the target user. I will check what it is. However, the illustrated process is merely an example, and the evaluation unit 13 may naturally perform other processes.

In this embodiment, the usage history database is a database that manages the usage history of this service, but it may be a database that also manages the usage history of similar services other than this service. Further, although the usage history database is stored in the server 3, the usage history database may be stored in a device other than the server 3. FIG.

Next, the evaluation unit 13 executes an assessment amount derivation process. The assessed amount derivation process is a process of deriving an assessed amount. In the assessed value derivation process, the evaluation unit 13 refers to the assessed value table corresponding to the target deepest classification (in this example, "notebook computer (article classification)>Windows (article classification)>maker MK1 (article classification)"). The appraisal value table for the target deepest classification includes values that model numbers can take, values that article condition levels can take (levels L1 to L5 in this embodiment), values that processor types can take, and the total number of memories. It is a table in which a reference appraisal amount is associated with each combination with a value to be obtained.

Hereinafter, the combination of the model number, article condition level, processor name and total number of memories will be referred to as "evaluation combination". In the assessment price table, the value of the reference assessment price associated with a certain combination for assessment is a value reflecting the value of each element of that combination for assessment. The reference assessment price is basically determined by the model number, and this reference amount is lower the lower the item condition level (= the greater the deterioration of the target item), and the lower the processor performance. , is adjusted so that it becomes lower as the total number of memories becomes smaller. Of course, the standard amount of money may be adjusted so that the higher the article condition level (=the smaller the deterioration of the target article), the higher the higher the processor performance, and the higher the total number of memories.

The evaluation unit 13 acquires the reference assessed value associated with the combination for evaluation of the target article in the assessed value table, and uses this as the assessed value.

The above is the assessment process executed by the evaluation unit 13 when the target deepest classification is "notebook computer (article classification)>Windows (article classification)>maker MK1 (article classification)". Note that the order of processing in the assessment process is not limited to the order illustrated in this embodiment. As described above, according to the present embodiment, in addition to the state of the appearance of the target article, characters (model number, unique number, spec ) is reflected in the assessment (evaluation of the article), the target article can be assessed from a plurality of different viewpoints. In particular, in the present embodiment, considering that the specs affect the assessed value (affect the evaluation), the specs are reflected in the assessment of the target article by the evaluator 13. You can appraise the target goods at

At that time, the recognition of the state of the appearance of the target article and the recognition of the characters (model number, spec, unique number) by character recognition are performed based on common photographed image data. Therefore, the target user can take a picture of the target product in order to evaluate the target product. For example, the user inputs the model number, the unique number, and the specifications, or selects the model number, the unique number, and the specifications in a predetermined user interface. No need. In particular, in this embodiment, the evaluation unit 13 performs recognition of the appearance state of the target article and recognition of characters (model number, spec, unique number) by character recognition based on the common photographed image data. For this reason, the user takes photographs to generate photographed image data used to recognize the state of the appearance of the target item (notebook computer), and to generate photographed image data used to recognize the model number, unique number, and specifications. Since there is no need to separate the image capturing from the image capturing, the user's work is reduced in this respect as well. That is, according to the present embodiment, it is possible to evaluate an article from a plurality of different viewpoints while reducing user's work.

<When the target deepest classification is "smartphone (article classification) > iOS (article classification)">
Next, the processing of the evaluation unit 13 when the target deepest classification is "smartphone (item classification)>iOS (item classification)" will be described. As described above, in this case, the evaluation unit 13 acquires captured image data generated by capturing an image of the housing from the front while the system-related information is displayed on the touch screen.

The evaluation unit 13 sequentially executes expected image determination processing, character recognition processing, model number recognition processing, unique number recognition processing, state recognition processing, and fraud possibility investigation processing. These processes are performed in the same manner as in the case where <target deepest classification is "notebook computer (article classification)>Windows (article classification)>maker MK1 (article classification)". Especially in the unique number acquisition process, the evaluation unit 13 acquires the IMEI as the unique number. In addition, regarding the state recognition processing, in this example, the target article is a smartphone, and the image of the front of the housing including the entire touch screen is recorded in the photographed image data. The evaluation unit 13 derives the article condition level by reflecting the conditions of elements related to deterioration such as scratches (including cracks in the touch screen), dirt, dents, wear, missing parts, etc. on the front of the housing.

In this example, the model number acquisition process is "a process of recognizing a model number by performing character recognition on an image of system-related information." Further, the unique number recognition processing is "a process of recognizing a unique number by performing character recognition on an image of system-related information". The state recognition processing is "a process of analyzing the image of the target article recorded in the photographed image data and recognizing the appearance state of the portion including the front of the display of the target article".

After the fraud possibility investigation process, the evaluation unit 13 executes the payment status recognition process. The payment status recognition process is "a process of making an inquiry using the recognized unique number and recognizing the payment status of the target product". The payment status confirmation process will be described in detail below. In the following description, the IMEI (unique number) acquired in the unique number acquisition process is particularly referred to as "target IMEI".

In the payment status recognition process, the evaluation unit 13 inquires of the payment status management server connected to the network N about the payment status of the target IMEI. The inquiry is made by transmitting data in a predetermined format including the target IMEI (for example, request data including the target IMEI in the query). In this embodiment, "payment status" means whether the target item (smartphone in this example) was purchased by installment payment, and if it was purchased by installment payment, whether payment is being made at this time, and whether payment is being made at this time. means whether there have been any past delays in payments, whether or not The payment status management server is a server device that manages the payment status together with the IMEI in a database, and has the function of responding to payment status information indicating the payment status of an IMEI in response to an inquiry about the payment status of an IMEI. ing.

After inquiring about the payment status of the target IMEI, the evaluation unit 13 acquires payment status information about the target IMEI from the payment status management server and recognizes the payment status. That is, the evaluation unit 13 determines whether or not the smartphone as the target product was purchased by installment payment, if it was purchased by installment payment, whether payment is currently being made, and regardless of whether or not payment is being made at this time, the payment to recognize if there was a delay in

Next, the evaluation unit 13 executes an assessment amount derivation process. In this example, the evaluation unit 13 refers to the assessed amount table corresponding to the target deepest classification. The assessed value table corresponding to the target deepest classification is a table in which a reference assessed value is associated with each combination of the value that the model number can take and the value that the item condition level can take. The evaluation unit 13 acquires the reference assessed value corresponding to the combination of the model number and the item condition level of the target item in the assessed value table. Next, the evaluation unit 13 adjusts the obtained reference appraisal value based on the payment status of the target article, and uses the adjusted amount as the final appraisal value. Adjustments are made so that if the target item (smartphone in this example) is purchased by installment payment and is currently being paid for, the assessed amount will be lower. It is done so that the assessment amount will be low. This takes into account the possibility of delays in the payment of lease payments. However, the adjustment method is not limited to the one exemplified in this embodiment, and for example, a configuration may be adopted in which adjustment is made in consideration of only past payment delays.

The above is the assessment process executed by the evaluation unit 13 when the target deepest classification is "smartphone (item classification)>iOS (item classification)". Note that the order of processing in the assessment process is not limited to the order illustrated in this embodiment. As described above, according to the present embodiment, characters (model number, IMEI) recognized by character recognition targeting system-related information images (character images) are reflected in addition to the appearance of the target article. Since the assessment (assessment of the article) is performed after the evaluation, the target article can be assessed from a plurality of different viewpoints. In particular, in this embodiment, considering that the target product may have been purchased by installment payment, the evaluation unit 13 evaluates the target product by reflecting the payment status. Goods can be appraised.

At that time, the recognition of the state of the appearance of the target article and the recognition of characters (model number, unique number) by character recognition are performed based on common photographed image data. Therefore, the target user only needs to take a picture of the target product when evaluating the target product, and does not need to input the model number and IMEI or select the model number and IMEI on a predetermined user interface, for example. In particular, in this embodiment, the evaluation unit 13 recognizes the state of the appearance of the target article and recognizes characters (model number, IMEI) by character recognition based on the common photographed image data. For this reason, the user needs to perform two types of photographing, one for generating photographed image data used for recognizing the state of the appearance of the target article (smartphone), and the other for generating photographed image data used for recognizing the model number and unique number. There is no need to do this separately, and the user's work is reduced in that respect as well. That is, according to the present embodiment, it is possible to evaluate an article from a plurality of different viewpoints while reducing user's work.

<When the target deepest classification is "Bag (article classification) > Brand B1">
Next, the processing of the evaluation unit 13 when the target deepest classification is "bag (item classification)>brand B1" will be described. As described above, in this case, the evaluation unit 13 acquires captured image data generated by capturing an image of the entire front surface of the bag.

First, the evaluation unit 13 executes expected image determination processing. Next, the evaluation unit 13 executes image analysis processing for model number recognition. The image analysis processing for model number recognition corresponds to "process of analyzing the image of the target article and recognizing the model number of the target article". Image analysis here means image processing other than "character recognition for character images". An example of image analysis processing for model number recognition will be described below.

First, the evaluation unit 13 recognizes that the target product is a bag and the brand of the bag based on the deepest target classification (the deepest target product classification). The server-side storage unit 14 stores a database in which template data for each model number (an image showing a bag viewed from the front and template data to be compared) is registered for each bag brand. The evaluation unit 13 performs template matching using each of the template data corresponding to the brand of the target article for the photographed image data to specify and recognize the model number.

Note that template matching is performed after pre-processing including noise removal and size/angle correction for the captured image data (however, size/angle correction may be performed on the template data). (eg template matching using template data holding pixel values, chamfer matching, active search using histogram information). However, the method of identifying the model number using the photographed image data is not limited to the exemplified method. For example, a method using feature quantity matching or a combination of template matching and feature quantity matching may be used. Alternatively, for example, a model may be used in which photographed image data is used as one of inputs and a corresponding model number (or the possibility of corresponding to each model number) is output. In this case, the model can be a machine-learned model using a predetermined machine-learning technique (eg, deep learning).

Next, the evaluation unit 13 executes authenticity determination processing. The authenticity determination process is "a process of image-analyzing the image of the target article to determine the authenticity of the target article". An example of the authenticity determination process will be described below. In the authenticity determination process, for example, the evaluation unit 13 executes the following processes. For example, when a logo image is included in the image of the front of the bag recorded in the photographed image data, the evaluation unit 13 determines that the logo recorded as an image in the photographed image data is the model number ( (model number of the target article) is analyzed from the viewpoint of authenticity as the logo of the bag, and it is determined whether or not the target article is genuine. Analysis of captured image data is performed from various points of view. For example, the analysis of the photographed image data is performed by determining the validity of the shape of the logo, the validity of the color of the logo, the validity of the position of the logo, and the like.

The validity of the logo position means whether or not the position where the logo is attached matches that of the real product. The validity of the shape and position of the logo can be determined, for example, by pattern matching using template data corresponding to the real logo prepared in advance, or feature amount matching. Further, the validity of the color of the logo can be determined, for example, by analyzing the histogram of the color components or analyzing the distribution of each color component in the image. It should be noted that the determination of whether or not the target article is genuine based on the analysis of the photographed image data may be performed by, for example, taking the photographed image data as one of the inputs and providing information indicating whether or not the photographed image data is genuine (or the possibility that the article is genuine). A model for outputting the information indicating is prepared, and this model is used for execution. In this case, the model can be a machine-learned model using a predetermined machine-learning technique (eg, deep learning).

Although the authenticity determination process using the logo has been described above, various existing techniques can be used for authenticity determination. In particular, instead of or in addition to the logo, a pattern, a mark, a characteristic two-dimensional or three-dimensional shape, or the like may be used as an object of authenticity determination, and the authenticity determination process may be performed. In addition, in the authenticity determination process for items other than bags, image analysis is performed on the photographed image data by an appropriate method according to the article and the object to be authenticated, and it is determined whether or not the item is genuine.

When it is determined that the target article is not genuine in the authenticity determination process, the evaluation unit 13 executes the corresponding process. For example, the evaluation unit 13 notifies the operator that the target item may not be genuine. According to this, the operator can take measures such as confirming the photographed image data in more detail. In the present embodiment, when it is determined that the target article is not genuine in the authenticity determination process, the evaluation unit 13 does not derive the following assessment amount. However, the evaluation unit 13 may be configured to derive an appraisal value as a counterfeit. In either case, the evaluation unit 13 assesses (evaluates) the target product by reflecting the result of the authenticity determination.

Next, the evaluation unit 13 executes state recognition processing. Regarding the state recognition processing, in this example, the target article is a bag, and an image of the entire front surface of the bag is recorded in the photographed image data. The evaluation unit 13 performs image processing on the photographed image data, and derives an article condition level by reflecting factors related to deterioration such as scratches, stains, dents, wear, missing parts, etc. on the front of the bag.

Next, the evaluation unit 13 executes an assessment amount derivation process. In the assessed value derivation process according to this example, the evaluation unit 13 refers to the assessed value table corresponding to the target deepest classification. The assessed value table corresponding to the target deepest classification is a table in which a reference assessed value is associated with each combination of the value that the model number can take and the value that the item condition level can take. The evaluation unit 13 acquires the reference assessed value associated with the combination of the model number and the article condition level of the target article in the assessed value table, and uses this as the assessed value.

The above is the assessment process executed by the evaluation unit 13 when the target deepest classification is "bag (article classification)>brand B1 (article classification)". Note that the order of processing in the assessment process is not limited to the order illustrated in this embodiment. As described above, according to the present embodiment, in addition to the state related to the appearance of the target article, the model number recognized by the image analysis targeting the image of the target article and the image of the target article are subjected to Since the target product is assessed (evaluated) by reflecting the result of the determination of authenticity, the target product can be assessed from a plurality of different viewpoints. In particular, in the present embodiment, in consideration of the fact that authenticity is important for assessment, the target article is subjected to authenticity determination, and the evaluation unit 13 assesses the target article based on the results of the authenticity determination. Therefore, in that respect, the target article can be assessed more accurately.

At that time, the recognition of the state of the appearance of the target article, the recognition of the model number by image analysis, and the authenticity determination by image analysis are performed based on common photographed image data. Therefore, the target user only needs to take a picture of the target product when assessing the target product, and does not need to input the model number or provide information for authenticity determination in a predetermined user interface, for example. In particular, in this embodiment, the target user performs photography for generating photographed image data used for recognizing the state of the appearance of the target article (bag) and photographing for generating photographed image data used for recognizing the model number. There is no need to separately perform photographing for generating photographed image data used for authenticity determination, and the user's work is reduced in this respect as well. That is, according to the present embodiment, it is possible to evaluate an article from a plurality of different viewpoints while reducing user's work.

The assessment process has been described above for a plurality of cases. After the appraisal amount is derived by the appraisal process, as described above, with the consent of the target user, the service provider purchases the target item at the appraised amount, and provides the target user with money equivalent to the appraisal amount. Subject to payment of a lease fee, the target item is leased to the target user.

Next, an information processing method by the server 3 (information processing device) will be described using a flowchart. FIG. 10 is a flow chart showing an operation example of the server 3 . As shown in FIG. 10, the evaluation unit 13 of the server 3 acquires, from the terminal 2 via the network N, photographed image data generated by photographing the target article by the target user (step SA1).

Next, the evaluation unit 13 of the server 3 performs image analysis on the image of the target article recorded in the photographed image data to recognize the state of the appearance of the target article, and the character image included in the image of the target article. At least one of a recognition process, a process of analyzing the image of the target article to recognize the model number of the target article, and a process of image-analyzing the image of the target article to determine the authenticity of the target article is executed. (step SA2). Next, the evaluation unit 13 of the server 3 evaluates the target article by reflecting at least one of the recognized appearance state of the target article, the recognized characters, the recognized model number, and the authentication result (step SA3). .

Although one embodiment of the present invention has been described above, the above-described embodiment merely shows an example of specific implementation of the present invention. It should not be done. Thus, the invention may be embodied in various forms without departing from its spirit or essential characteristics.

For example, in the above-described embodiment, the present invention can be applied to a system that simply presents the derived valuation price instead of a system that purchases the target article with the valuation price derived by the evaluation unit 13 . Further, for example, the present invention can be applied to a system in which the target article purchased by the service provider company is handed over from the target user to the service provider company by a predetermined means, instead of being leased and kept at the user's hand. .

Moreover, the evaluation of the target product is not limited to deriving the monetary value, and for example, the evaluation unit 13 may rank the target product.

Further, the information to be recognized as characters by character recognition from the photographed image data is not limited to the one exemplified in this embodiment. The information to be recognized as characters from the photographed image data by character recognition may be anything as long as it affects the evaluation of the target article by the evaluation unit 13 . For example, information that shows when the target goods were manufactured (manufacturing date), information that shows how the target goods were used, information that shows the environment in which the target goods were used, etc. There may be.

Further, in the above embodiment, if the target deepest classification is "notebook computer (article classification) > Windows (article classification) > manufacturer MK1 (article classification)", if "smartphone (article classification) > iOS (article classification)", In addition, in each case of "bag (item classification)>brand B1 (item classification)", the evaluation unit 13 acquires one shot image data through the shot image data acquisition process. However, this does not mean that the evaluation unit 13 is limited to acquiring "one" photographed image data when evaluating (assessing in the above embodiment) the target article.

For example, when the target deepest classification is "laptop (article classification) > Windows (article classification) > maker MK1 (article classification)", the evaluation unit 13 captures the surface of the notebook computer when it is closed. Image data may be further acquired, and the captured image data may be further used to execute the state recognition processing. Further, for example, when the target deepest classification is "smartphone (item classification)>iOS (item classification)", the evaluation unit 13 further acquires captured image data generated by capturing an image of the rear surface of the housing, and this captured image data may be used to execute the state recognition process. Further, for example, when the target deepest classification is "bag (item classification)>brand B1 (item classification)", the evaluation unit 13 further acquires photographed image data generated by photographing the back of the bag, and this photographed image data may be used to perform at least one of the state recognition process, image analysis process for model number recognition, and authenticity determination process.

In addition, in the system-related information (which may of course be other information), the information to be photographed is spaced apart, and in order to photograph the information to be photographed, photographing must be performed two or more times. If necessary, two or more shots are naturally taken, and the evaluation unit 13 acquires two or more pieces of shot image data.

Various screens displayed on the touch screen 4 of the terminal 2 have been described, but the contents of each screen are not limited to the contents described.

Also, when the target deepest classification is "Laptop PC (article classification) > Windows (article classification) > Maker MK1 (article classification)" and "Smartphone (article classification) > iOS (article classification)", the evaluation department 13 may not execute the fraud possibility investigation process. Conversely, if the target deepest classification is "bag (item classification)>brand B1 (item classification)" and the individual number can be obtained, the evaluation unit 13 may be configured to execute the fraud possibility investigation process.

In the above-described embodiment, the terminal 2 is the medium on which the evaluation unit 13 displays various screens. In this respect, the evaluation unit 13 may be configured to display various screens on a device having a display other than the terminal 2, the display being visible to the target user.

Also, when the target deepest classification is "notebook computer (article classification)>Windows (article classification)>maker MK1 (article separation)" (including, of course, equivalent cases), the evaluation unit 13 cannot execute the specification recognition process. A configuration may be adopted in which the assessment is performed without reflecting the specifications. For example, if the target product is an electronic dictionary whose specifications are assumed to be the same as long as the model number is the same, the evaluation unit 13 does not need to perform the processing related to the specifications. In addition, when the target deepest classification is "smartphone (item classification)>iOS (item classification)" (including, of course, equivalent cases), the evaluation unit 13 does not execute the payment status recognition process and reflects the payment status. The configuration may be such that the assessment is performed without In these cases, the target article has a display, and is characterized in that system-related information including at least the model number can be displayed on the display, and in which deterioration occurring in a part including the front of the display affects the evaluation. It is an electronic device having the following two characteristics. Then, the evaluation unit 13 acquires photographed image data generated by photographing the appearance of the target article including the front of the display while the system-related information is displayed on the display. The evaluation unit 13 analyzes the image of the target article recorded in the photographed image data to recognize the state of the appearance of the part including the front of the display of the target article, and the system recorded in the photographed image data. A processing for recognizing the model number is executed by performing character recognition on the image of the related information. The evaluation unit 13 evaluates the target product by reflecting the appearance state of the recognized target product including the front of the display and the recognized model number. Furthermore, when the target user takes a picture of the target article, the evaluation unit 13 recognizes the classification of the target article (the deepest classification) prior to the photographing, and the electronic If the equipment belongs to the category, the terminal or a device that can be visually recognized by the target user displays a screen that guides a part including the front of the display to be photographed while the system-related information is displayed on the display.

Also, when the target deepest classification is "bag (item classification) > brand B1 (item classification)" (including, of course, equivalent cases), the evaluation unit 13 may perform assessment without executing the authenticity determination process. good. In this case, the target product is a product having one feature that the model number can be determined from the appearance of the product, while the model number may not be recorded as characters on the product. acquisition of photographed image data generated by photographing the appearance of the object, image analysis of the image of the target article recorded in the photographed image data, processing to recognize the state of the appearance of the target article, and processing of recognizing the image of the target article A process of analyzing the image and recognizing the model number of the target article is executed, and the target article is evaluated by reflecting the recognized appearance state of the target article and the recognized model number. Furthermore, when the target user takes a picture of the target article, the evaluation unit 13 recognizes the classification of the target article prior to the photographing, and if the recognized classification is the classification to which the article having one feature belongs. 2, a screen is displayed on the terminal 2 or a device visible to the target user so that the appearance of the target article is captured so that the generated captured image data can be used for recognizing the model number.

Further, in the above-described embodiment, the evaluation unit 13 is configured to derive the assessed amount using the assessed amount table, but the method for deriving the assessed amount is not limited to the exemplified method. As an example, the evaluation unit 13 may be configured to derive the assessed value using a calculation formula in which various elements used for deriving the assessed value are used as parameters.

Further, in the above embodiment, the object to be subjected to character recognition is basically system-related information, but the object to be subjected to character recognition is not limited to system-related information. For example, a nameplate label (sometimes called a nameplate sticker) attached to an electric appliance or the like may be an object to be subjected to character recognition. In addition, character information recorded on a target article or an object attached to the target article by engraving, printing, distribution of a medium, or other means may be used as an object to be subjected to character recognition.

In the above embodiment, the terminal 2 is a smart phone, but the terminal 2 is not limited to a smart phone. For example, the terminal 2 may be a tablet terminal or a notebook computer that does not have terminal functions.

1 information processing system 2 terminal 3 server (information processing device)
13 Evaluation Department

Claims (19)

Acquiring from a terminal via a network photographed image data generated by photographing the target article by the target user,
image analysis of the image of the target article recorded in the photographed image data to recognize the state of the appearance of the target article;
Character recognition processing for an image of characters included in the image of the target product, processing for image analysis of the image of the target product to recognize the model number of the target product, and image analysis of the image of the target product performing at least one of the processes for determining the authenticity of the target article,
an evaluation unit that evaluates the target article by reflecting at least one of the recognized appearance state of the target article, the recognized characters, the recognized model number, and the result of authenticity determination. Device. the image of characters includes an image of characters indicating the model number of the target product;
The evaluation unit analyzes the image of the target article recorded in the photographed image data to recognize the appearance state of the target article, and performs character recognition on the image of characters included in the image of the target article. 2. The information processing according to claim 1, wherein the model number of the target product is recognized by performing the above-mentioned processing, and the target product is evaluated by reflecting the recognized state of the appearance of the target product and the recognized model number. Device. The image of characters includes an image of characters indicating a unique number of the target article,
The evaluation unit analyzes the image of the target article recorded in the photographed image data to recognize the appearance state of the target article, and performs character recognition on the image of characters included in the image of the target article. to recognize the unique number of the target product, and evaluate the target product by reflecting the recognized state of the appearance of the target product and the recognized unique number. The information processing device described. The evaluation unit executes a process of deriving an assessed value of the target item as a process of evaluating the target item,
After the appraisal amount is derived by the evaluation unit, the subject article is purchased at the appraisal amount by a predetermined entity with the consent of the target user, and money equivalent to the appraisal amount is provided to the target user, and a lease fee is paid. said subject item is leased to said subject user subject to payment of
The target goods are goods that may be purchased by installment,
When deriving the appraisal value of the target product, the evaluation unit performs an inquiry using the recognized unique number, recognizes the payment status of the target product, recognizes the appearance status of the target product, and 4. The information processing apparatus according to claim 3, wherein the assessed value of the target article is derived by reflecting the recognized payment status. With respect to the item, the evaluation unit utilizes data for managing the item's unique number in association with user identification information of a user who owned the item at least at one time in the past, based on the recognized unique number. 4. The information processing apparatus according to claim 3, wherein the possibility that the target article is not the property of the target user is investigated by using the method. The evaluation unit recognizes an item that has been evaluated in the past by using data managed by associating the unique number of the item with the user identification information of the user who owned the item when the evaluation was performed. 6. The information processing apparatus according to claim 5, wherein a possibility that the target article is not owned by the target user is investigated based on the unique number. The evaluation unit
When the user takes a picture of the target article, the classification of the target article is recognized prior to photographing, and according to the classification of the target article,
When performing character recognition processing on an image of characters included in the image of the target article, a screen guiding the character of the target article to be photographed together with the appearance of the target article is displayed on the terminal or the target. displayed on a device visible to the user,
When performing image analysis of the image of the target article to recognize the model number of the target article, the external appearance of the target article is adjusted so that the generated photographed image data becomes data that can be used for model number recognition. is displayed on the terminal or a device visible to the target user,
When the image of the target article is subjected to image analysis to determine the authenticity of the target article, the photographed image data to be generated is data that can be used to determine the authenticity of the target article. 7. The information processing apparatus according to any one of claims 1 to 6, wherein the terminal or a device that can be viewed by the target user displays a screen that guides the user to photograph the appearance of the target article. The target article has a display, and has two features: the feature that system-related information including at least the model number can be displayed on the display, and the degradation that occurs in the part including the front of the display affects the evaluation. It is an electronic device with characteristics,
The evaluation unit
Acquiring photographed image data generated by photographing the appearance of the target article including the front of the display while the system-related information is displayed on the display;
A process of image analysis of the image of the target article recorded in the photographed image data to recognize the appearance state of the portion including the front of the display of the target article, and the system-related information recorded in the photographed image data. Executes the process of recognizing the model number by character recognition for the image of
2. The information processing apparatus according to claim 1, wherein the target article is evaluated by reflecting the appearance state of the recognized part including the front of the display of the target article and the recognized model number. The evaluation unit recognizes a classification of the target article prior to photographing when the target user takes an image of the target article, and the electronic device having the two characteristics belongs to the recognized classification. In the case of classification, displaying a screen on the terminal or a device visible to the target user while the system-related information is displayed on the display, and guides the user to take an image of a part including the front of the display. 9. The information processing apparatus according to claim 8, characterized by: The target article has a display, and is capable of displaying system-related information including at least the model number and specs of a predetermined element on the display; , an electronic device having three characteristics that deterioration occurring in a part including the front of the display affects the evaluation,
The evaluation unit
Acquiring photographed image data generated by photographing the appearance of the target article including the front of the display while the system-related information is displayed on the display;
Image analysis of the image of the target article recorded in the photographed image data to recognize the appearance state of the part including the front of the display of the target article, and the image of the system-related information recorded in the photographed image data and a process of recognizing the specifications of a predetermined element by performing character recognition on the image of the system-related information recorded in the photographed image data. ,
1. The target product is evaluated by reflecting the appearance state of the recognized part including the front of the display of the target product, the recognized model number, and the specifications of the recognized predetermined elements. The information processing device according to . The evaluation unit recognizes the classification of the target article prior to photographing when the target user takes a photograph of the target article, and the electronic device having the three characteristics belongs to the recognized classification. In the case of classification, displaying a screen on the terminal or a device visible to the target user while the system-related information is displayed on the display, and guides the user to take an image of a part including the front of the display. 11. The information processing apparatus according to claim 10, characterized by: The evaluation unit executes a process of deriving an assessed value of the target item as a process of evaluating the target item,
After the appraisal amount is derived by the evaluation unit, the subject article is purchased at the appraisal amount by a predetermined entity with the consent of the target user, and money equivalent to the appraisal amount is provided to the target user, and a lease fee is paid. said subject item is leased to said subject user subject to payment of
The target product is provided with a display in a wide area on the front surface, and has the feature that system-related information including at least the model number and the unique number can be displayed on the display, the feature that it may be purchased by installment, and the display A tablet terminal with three characteristics that deterioration occurring in parts including the front affects the evaluation,
The evaluation unit
Acquiring photographed image data generated by photographing the casing of the target article, which is a tablet terminal, from the front while the system-related information is displayed on the display;
Image analysis of the image of the target article recorded in the photographed image data to recognize the appearance state of the part including the front of the display of the target article, and the image of the system-related information recorded in the photographed image data processing for recognizing the model number by character recognition for the target, processing for recognizing the unique number by subjecting the image of the system-related information recorded in the captured image data to character recognition, and using the recognized unique number perform a process of recognizing the payment status of the target goods by making an inquiry,
The appraisal value of the target product is derived by reflecting the appearance status of the recognized part including the front of the display of the target product, the recognized model number, and the recognized payment status. The information processing device described. When the target user takes a picture of the target article, the evaluation unit recognizes the classification of the target article prior to photographing, and the tablet terminal having the three features belongs to the recognized classification. In the case of classification, displaying a screen on the terminal or a device that can be visually recognized by the target user while the system-related information is displayed on the display, and guides the user to take a picture of the housing from the front. 13. The information processing apparatus according to claim 12. The target article is an article having one feature that the model number as a character may not be recorded on the article, but the model number can be determined from the appearance of the article,
The evaluation unit
Acquiring photographed image data generated by photographing the appearance of the target article,
A process of image-analyzing the image of the target article recorded in the photographed image data to recognize the appearance state of the target article, and a process of image-analyzing the image of the target article and recognizing the model number of the target article. and run
2. The information processing apparatus according to claim 1, wherein the target product is evaluated by reflecting the recognized appearance state of the target product and the recognized model number. The evaluation unit recognizes a classification of the target article prior to photographing when the target user takes an image of the target article, and the recognized classification is the classification to which the article having the one feature belongs. In the case of , the terminal or the target user can visually recognize a screen that guides the appearance of the target article to be captured so that the generated captured image data becomes data that can be used for model number recognition. 15. The information processing apparatus according to claim 14, wherein the display is performed on the . The target product may not have a model number recorded as characters, but may have the feature that the model number can be determined from the appearance of the product, and may be a counterfeit, and the product is genuine in the evaluation. An article with two characteristics, the characteristic being that it is important whether
The evaluation unit
Acquiring photographed image data generated by photographing the appearance of the target article,
a process of image-analyzing the image of the target article recorded in the photographed image data to recognize the appearance state of the target article, a process of image-analyzing the image of the target article and recognizing the model number of the target article, and , performing image analysis of the image of the target product to determine the authenticity of the target product;
2. The information processing apparatus according to claim 1, wherein the target article is evaluated by reflecting the recognized appearance state of the target article, the recognized model number, and the result of authenticity determination. The evaluation unit recognizes the classification of the target article prior to photographing when the target user takes an image of the target article, and the recognized classification is the classification to which the article having the two characteristics belongs. In this case, the terminal or the said 17. The information processing apparatus according to claim 16, wherein the information is displayed on a device that can be viewed by the target user. An information processing system comprising a terminal and an information processing device connected to the terminal via a network,
The information processing device is
Acquiring photographed image data generated by photographing a target article by a target user from the terminal via a network;
image analysis of the image of the target article recorded in the photographed image data to recognize the state of the appearance of the target article;
Character recognition processing for an image of characters included in the image of the target product, processing for image analysis of the image of the target product to recognize the model number of the target product, and image analysis of the image of the target product performing at least one of the processes for determining the authenticity of the target article,
an evaluation unit that evaluates the target article by reflecting at least one of the recognized appearance state of the target article, the recognized characters, the recognized model number, and the result of authenticity determination. system. a step in which an evaluation unit of an information processing device acquires captured image data generated by capturing a target article by a target user from a terminal via a network;
The evaluation unit of the information processing device analyzes the image of the target article recorded in the photographed image data to recognize the appearance state of the target article, and the image of characters included in the image of the target article. character recognition processing, image analysis of the image of the target product to recognize the model number of the target product, and image analysis of the image of the target product to determine the authenticity of the target product. performing at least any one process;
the evaluation unit of the information processing device evaluating the target product by reflecting at least one of the recognized appearance state of the target product, the recognized characters, the recognized model number, and the authentication result; An information processing method comprising:

Images