KR102040525B1 - Artificial intelligence based part search system - Google Patents

Abstract

An artificial intelligence based part search system is disclosed. An AI-based part search system according to an embodiment may include a client requesting a part search together with a photographed part image by capturing a part image or acquiring a photographed part image, and performing AI-based machine learning. Learning model management device that creates and stores a number of parts image learning models through the model, and when a part search request is received from a client, the learning model stored in the learning model management device searches for a part image that matches the photographed part image. It includes an image search device for providing a client with a search result including the part name.

Description

Artificial intelligence based part search system {Artificial intelligence based part search system}

The present invention relates to data analysis and retrieval technology.

In general, one machine consists of thousands of parts. So there are no engineers who know every part name. When a problem occurs in a part of the equipment to be used, in most cases, in order to find the correct part name, the parts part book in text form as shown in FIG. 1 is checked one by one. In some limited online parts manuals, part names are searched through keyword search using a part name, a navigation method (FIG. 2), or an existing image search method. However, all of these methods are indirect part searching activities performed by requesting parts specialists to search for parts rather than part searching activities performed by general users directly at the trouble spot. Either way, knowing the exact part name out of thousands to tens of thousands of pieces of equipment requires expertise and search time.

Korean Unexamined Patent Publication No. 10-2016-0123485 (published Oct. 26, 2016) Korean Unexamined Patent Publication No. 10-2013-0124767 (published Nov. 15, 2013)

According to an embodiment of the present disclosure, an artificial intelligence-based part search system optimized for an equipment part domain by introducing artificial intelligence (AI) -based deep learning technology to a machine part search is proposed.

An AI-based part search system according to an embodiment may include a client requesting a part search together with a photographed part image by capturing a part image or acquiring a photographed part image, and performing AI-based machine learning. Learning model management device that creates and stores a number of parts image learning models through the model, and when a part search request is received from a client, the learning model stored in the learning model management device searches for a part image that matches the photographed part image. It includes an image search device for providing a client with a search result including the part name.

The client selects a part from the user by providing a part photographing unit which photographs a part image or obtains a photographed part image, and a user interface for selecting an image to be searched for a part among the photographed part images acquired by the part photographing unit. The image selection unit may include an input image selection unit, a search request and a result receiving unit which transmits a component search request together with a photographed component image or a selected input component image to the image retrieval device and receives a component search result from the image retrieval device. The search request and the result receiving unit may designate and request the number of result values to be included in the component search result of the image search apparatus when the search request is made.

The client may include a pre-feedback unit configured to limit a search area of the image search apparatus according to a feedback received from the user through a keyword search function before the search of the image search apparatus, and each search when receiving a plurality of search result values from the image search apparatus. The apparatus may further include a post-feedback unit which provides the component part information together with the component image and receives feedback to finally select a component that the user wants to find among the search components. If the client has a large number of parts in the photographed part image, the client recommends a main part object in the photographed part image, but provides a part recommendation part for selecting the user by providing the name and probability value of the main part in the photographed part image. It may further include.

The learning model management apparatus includes a part video collecting unit for acquiring a part video for learning, and a part for extracting a part image from a part video acquired through the part video collecting unit and performing image processing and editing on the extracted part image. The image processor may include a deep learning unit configured to learn a part image edited through the component image processor based on deep learning to generate a learning model.

The part image processing unit extracts the parts images necessary for learning from the obtained video, the part image extracting unit which extracts the part image after designating the number of extracts per frame and the part name according to the screen conversion, and the extracting through the part image extracting unit. Part image controllers that perform edits that include at least one of selecting, deleting, grouping, labeling, and cropping the edited part images, and dumping the training data to match the data format required for training training with the edited part images. The component image dump unit may be included.

The deep learning learner may select a predetermined deep learning learning method from among a plurality of deep learning learning methods, and may verify whether learning has been normally performed during learning using at least one of accuracy and cross entropy. The deep learning unit may learn at least one of a bulk training and an incremental training on the part image.

The image search apparatus may include: a search request receiver configured to receive a part search request together with a photographed part image from a client; a search unit configured to search for a part image matching the photographed part image in a learning model stored in the learning model management device; and a search part image. And a search result providing unit for providing a search result including the search part name to the client and providing the search results in a similar order to the photographing part image requested to be searched by the client.

The image retrieval apparatus includes a parts parts book providing unit which provides the parts part information together with the searched parts image, a parts ordering unit providing an order service for the searched parts, and a statistical information system providing statistical information about the searched parts. It may further include at least one of the study.

Through the AI-based parts search system according to an embodiment, even a non-expert can easily search for a part name having a problem in the field. By using client terminals, such as smartphones, which are the most common everyday, the search time problem caused by the non-real time search interface method can be solved, and the deep learning AI technique can improve the accuracy of the part image search.

When building an AI-based learning model, the accuracy of the training data can be improved by capturing parts as moving images and extracting images for learning from the captured moving images. Dedicated parts image management programs for editing images for learning can provide more sophisticated images for learning. We can build a learning model with high image retrieval accuracy through bulk training and incremental training on the extracted images.

After taking a picture of the part with the client terminal, the part can be searched in real time in the field. Real-time part image retrieval enables end users and experts to identify part names faster than ever before, giving you flexibility in responding to failures (eg, ordering parts, requesting an AS engineer), and more You can proceed. Furthermore, the search accuracy can be improved by referring to the parts part book.

1 is a reference diagram showing an example of a parts part book used for a general part search;
2 is a reference diagram illustrating an example of a keyword search method using a part name in an online parts manual used for a general part search;
3 is a conceptual diagram of an artificial intelligence based parts search system according to an embodiment of the present invention;
4 is a block diagram of an artificial intelligence-based component search system according to an embodiment of the present invention;
5 is a block diagram of a learning model management apparatus according to an embodiment of the present invention;
6 is a detailed configuration diagram of a component image processing unit of FIG. 5 according to an embodiment of the present disclosure;
7 is a conceptual diagram illustrating an example of building a learning model of the deep learning unit of FIG. 5 according to an embodiment of the present invention;
8 is a reference diagram showing an accuracy change transition parameter required for learning verification according to an embodiment of the present invention;
9 is a reference diagram showing an accuracy change transition parameter required for learning verification according to an embodiment of the present invention;
10 is a block diagram of an image search apparatus according to an embodiment of the present invention;
11 is a block diagram of a client according to an embodiment of the present invention;
12 is a detailed configuration diagram of the pre-feedback unit of FIG. 11 according to an embodiment of the present disclosure;
FIG. 13 is a screen view illustrating a client's image cropping according to an embodiment of the present invention; FIG.
14 is a reference diagram illustrating a client screen to show a result of a part image search according to an embodiment of the present disclosure;
15 is a reference diagram for explaining a part image retrieval process considering a similar part according to an embodiment of the present invention;
16 and 17 are reference diagrams for describing an image retrieval process in consideration of similar parts according to an exemplary embodiment.

Advantages and features of the present invention, and methods for achieving them will be apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but can be implemented in various forms. It is provided to fully convey the scope of the invention to those skilled in the art, and the present invention is defined only by the scope of the claims. Like reference numerals refer to like elements throughout.

In describing the embodiments of the present disclosure, when it is determined that a detailed description of a known function or configuration may unnecessarily obscure the subject matter of the present disclosure, the detailed description thereof will be omitted, and the following terms are used in the embodiments of the present disclosure. Terms are defined in consideration of the function of the may vary depending on the user or operator's intention or custom. Therefore, the definition should be made based on the contents throughout the specification.

Combinations of each block of the accompanying block diagrams and respective steps of the flowcharts may be performed by computer program instructions (execution engines), which may be executed on a processor of a general purpose computer, special purpose computer or other programmable data processing equipment. As such, instructions executed through a processor of a computer or other programmable data processing equipment create means for performing the functions described in each block of the block diagram or in each step of the flowchart.

These computer program instructions may also be stored in a computer usable or computer readable memory that can be directed to a computer or other programmable data processing equipment to implement functionality in a particular manner, and thus the computer usable or computer readable memory. The instructions stored therein may also produce an article of manufacture containing instruction means for performing the functions described in each block of the block diagram or in each step of the flowchart.

And computer program instructions can also be mounted on a computer or other programmable data processing equipment, such that a series of operating steps can be performed on the computer or other programmable data processing equipment to create a computer-implemented process that can be executed by the computer or other programmable data. Instructions for performing data processing equipment may also provide steps for performing the functions described in each block of the block diagram and in each step of the flowchart.

In addition, each block or step may represent a portion of a module, segment or code that includes one or more executable instructions for executing specific logical functions, and in some alternative embodiments referred to in blocks or steps It should be noted that the functions may occur out of order. For example, the two blocks or steps shown in succession may, in fact, be performed substantially concurrently, or the blocks or steps may be performed in the reverse order of the corresponding function, as required.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, embodiments of the present invention illustrated in the following may be modified in many different forms, the scope of the present invention is not limited to the embodiments described in the following. Embodiments of the present invention are provided to more fully explain the present invention to those skilled in the art.

3 is a conceptual diagram of an artificial intelligence-based part search system according to an exemplary embodiment.

Referring to FIG. 3, the AI-based part search system 1 provides a part search service for a user who wants to know an accurate part name for an equipment part. The component to be searched is a module constituting the equipment, and may be extended to equipment composed of a plurality of components. Equipment includes machinery, equipment, and equipment installed on site. The site may be a daily living site such as a home or a house or an industrial site such as a factory or a building. The user may receive the parts image search service using the client 10 carried by the user, and may obtain information about the corresponding parts without the help of an expert.

In an embodiment of the artificial intelligence-based part search system 1, a user photographs a real part image of the front surface through a client 10 and requests a part search from the image retrieval device 14 along with the captured part image. Receiving the part search request, the image search apparatus 14 searches for a part image matching the photographed part image from pre-built AI-based learning model data and provides a search result to the client 10. The search results include the found part image and the corresponding part name. In this case, the parts manual for the found part image may be provided together with the parts manual book.

4 is a block diagram of an artificial intelligence-based part searching system according to an embodiment of the present invention.

Referring to FIG. 4, the AI-based part retrieval system 1 includes a client 10, a learning model management device 12, and an image retrieval device 14. In the example of FIG. 4, the client 10, the learning model management device 12, and the image retrieval device 14 are physically separated, but at least some of the functions may be integrated into one device according to the function implementation. For example, at least some functions of the learning model management device 12 and the image retrieval device 14 may be integrated. As another example, at least some functions of the learning model managing apparatus 12 and the image searching apparatus 14 may be performed by the client 10.

The learning model management apparatus 12 according to an embodiment includes an image acquisition server 120 and a training server 122. The image acquisition server 120 collects part images that are subject to machine learning training. The training server 122 performs AI-based machine learning on the part images collected through the image acquisition server 120 to generate and store learning model data.

The image retrieval apparatus 14 according to an embodiment includes a service server 140, a web server 142, and a main server 144. The service server 140 connects to the training server 122, searches for a part image matching the part image requested by the user in the learning model data stored in the training server 122, and provides a search result. The web server 142 is a device that connects the service server 140 and the client 10 on the web. It processes the network connection of the client 10 and transmits the search request of the client 10 to the service server 140, and provides the search result received from the service server 140 to the client 10 via the network. The main server 144 is connected to the web server 142 and provides an additional service. For example, a parts parts book providing server, a parts order server, and a statistics server.

Servers constituting each of the devices 12 and 14 of FIG. 4 may be computer hardware or computer programs that provide services to the client 10. Servers constituting each of the devices 12 and 14 of FIG. 4 are classified according to functional aspects, and may be achieved through one device integrating at least two functions.

The client 10 may be a terminal possessed by a user, or may be a computer program that receives a service from a server. The client 10 may be a fixed terminal such as a desktop or a portable terminal such as a smartphone. The client 10 is compatible with an operating system (OS), for example, iOS, Android, Windows, and the like. The client 10 may access the web to the web server 142 through the web.

When the AI-based parts retrieval system 1 according to an embodiment is classified according to a process, it is divided into a learning model construction process 400 and a parts image retrieval process 410. In the learning model building process 400, a learning model of various parts images is constructed. In the part image retrieval process 410, a search result for the part image requested by the user is provided to the client 10 using the constructed learning model.

5 is a block diagram of a learning model management apparatus according to an embodiment of the present invention.

4 and 5, the learning model management apparatus 12 includes a part video collecting unit 125, a part image processing unit 126, and a deep learning learning unit 127. The component video collector 125 and the component image processor 126 may be implemented through the image acquisition server 120 of FIG. 4, and the deep learning learner 127 is implemented through the training server 122 of FIG. 4. Can be.

The parts video collecting unit 125 obtains a parts video for learning. Due to the nature of deep learning, a few hundred to as many ceiling images are needed per piece. Therefore, the parts video collecting unit 125 obtains the captured parts video instead of the image of each part.

The component image processor 126 extracts a component image from the component video acquired through the component video collector 125 and performs image processing and editing on the extracted component image. The detailed configuration of the component image processing unit 126 will be described later with reference to FIG. 6.

The deep learning learner 127 generates a learning model by learning a part image edited through the part image processor 126 based on deep learning. An embodiment of the deep learning learning of the deep learning learner 127 will be described later with reference to FIG. 7.

6 is a detailed block diagram of the component image processor of FIG. 5 according to an exemplary embodiment.

4 to 6, the component image processor 126 includes a component image extractor 1260, a component image adjuster 1262, and a component image dumper 1264.

The component image extractor 1260 extracts component images necessary for learning from the video acquired through the component video collector 125. For example, extract images from the video as needed for learning. At this time, the number of extracts per frame can be specified, and the part name according to the screen conversion can be specified.

The component image controller 1262 edits the component image extracted by the component image extractor 1260. Examples of part image editing include selecting, deleting, grouping, labeling, and cropping. For example, in the extracted part image, a picture containing an object (eg, a human hand) unnecessary for learning, a picture out of focus, and the like are removed. For example in crop, just cut out the part you want to find.

When image editing for all parts is finished through the part image adjusting unit 1262, the part image dumping unit 1264 dumps the training data in accordance with the data format required for the training training. Accordingly, data suitable for the training standard can be output.

7 is a conceptual diagram illustrating an example of building a learning model of the deep learning unit of FIG. 5 according to an exemplary embodiment.

4 to 7, the deep learning learner 127 trains the training image 700 through the deep learning learning 710 to generate the learning model 720. In this case, the learning model 720 may be generated by selecting a suitable one among various published deep learning methods (for example, VGG16, VGG19, RestNet50, and Inceoption V3). That is, by providing a variety of deep learning learning method, it is possible to select the optimal learning method suitable for the part characteristics.

The deep learning learner 127 according to an embodiment verifies whether learning is normally performed during learning. In this case, verification may be performed using at least one of accuracy and cross entropy. An exemplary embodiment of the deep learning learner 127 will be described later with reference to FIGS. 8 and 9.

The deep learning learner 127 according to an embodiment may learn at least one of bulk training and incremental training on a part image. There are two methods for constructing a learning model: bulk training method and incremental training method. Bulk training is used as a batch when the model is first created from an entire image. The learning target is a full image, the model update method is a full update, and the processing method is a batch process. However, since the model generation time is burdensome to perform the bulk training every time, there is a need for a method of partially updating only the update information in the first generated model. What is used for this is the incremental training method. The learning target is an update image, the model updating method is partial updating, and the processing method is interactive processing. Since only the updated information is updated, the model generation time is significantly shorter than that of bulk training, and it operates in an interactive manner. However, version management and evaluation history management of models should be accompanied in case of reflecting wrong information.

8 and 9 are reference diagrams showing parameters required for learning verification according to an embodiment of the present invention. In detail, FIG. 8 is a graph showing a change in accuracy, and FIG. 9 is a cross entropy ( Cross Entropy) is a graph showing the change trend.

Learning can be progressed by setting the necessary parameters, using the change of accuracy (Fig. 8) and cross-entropy (Fig. 9) as a means to check whether the learning is normally performed during the learning. Trends in accuracy and cross-entropy can stop learning at the most appropriate moment.

10 is a block diagram of an image retrieval apparatus according to an embodiment of the present invention.

4 and 10, the image search apparatus 14 may include a search request receiver 145, a searcher 146, and a search result provider 147. The ordering unit 149 and the statistical information providing unit 150 may be further included. The search request receiver 145 and the search result provider 147 may be implemented through the web server 142 of FIG. 4. The search unit 146 may be implemented through the service server 140 of FIG. 4. The component part book providing unit 148, the component ordering unit 149, and the statistical information providing unit 150 may provide a service in cooperation with the main server 144 of FIG. 4.

The search request receiving unit 145 receives a part search request from the client 10. At this time, the part image captured by the client 10 is received together. The search unit 146 searches for a part image that matches the photographed part image in the learning model stored in the learning model management apparatus 12. The search result provider 147 provides the client 10 with a search result including the search part image and the corresponding part name. In this case, the top several search results may be provided in the order most similar to the photographing part image requested to be searched. The number of search result values may be many, which may be determined in consideration of the processing capacity of the client, the terminal specification, and the like. For example, it provides up to the top four search result values by default, reflecting the screen size of the smartphone. As another example, the client user may directly specify the number of search result values.

In order to increase the accuracy of image retrieval accuracy, the parts part book providing unit 148 provides the parts information together with the parts part book providing server, which is a main system, so that the search result provided by the client user in the search unit 146 It provides the function to check if it is correct. An embodiment of providing component information associated with a component part book will be described later with reference to FIG. 14. The parts ordering unit 149 enables the ordering of parts by interworking with the parts ordering server, which is a main system, thereby enabling everything in one process, from photographing parts to ordering parts. The statistical information providing unit 150 may provide statistical information in association with a statistical server, which is a main system. Statistical information can be classified by category, product model, and part.

11 is a block diagram of a client according to an embodiment of the present invention.

4 and 11, the client 10 includes a component photographing unit 100, an image selecting unit 102, a search request and a result receiving unit 104, and includes a pre-feedback unit 106 and a post-feedback unit. 108 and the component recommendation unit 109 may be further included.

The component photographing unit 100 captures an actual component image for which the user wants to know the component name through the camera of the client 10. Alternatively, you can select an image of a part you have already taken and use it to search for the part.

The image selector 102 provides a user interface that allows the user to select an image for which the user wants to search for a part among the photographed part images acquired by the part photographing part 100. The user interface may be an image cropping interface, for example. Due to the nature of the part, the photographing part image includes various parts. Accordingly, the image selector 102 provides an image cropping interface (see FIG. 13) for selecting an image region suitable for the user's intention.

The search request and result receiver 104 is a module that transmits and receives data to and from the image search apparatus 14. For example, the part search request message is transmitted to the image search apparatus 14 together with the photographed part image acquired through the part photographing unit 100 or the part image selected and input through the image selecting unit 102. In this case, the search request and result receiver 104 may transmit the area information selected by the user in the image selector 102 to the image search apparatus 14. Further, the search request and result receiving unit 104 may specify and request the number of result values to be included in the part search result. When the client 10 requests the image retrieval device 14 to search for parts, the number of result values may be specified in consideration of the size of the screen of the client 10 (for example, the default value is 4). The device 14 may request by specifying a desired number of result values.

The image retrieval apparatus 14 retrieves a part image matching the photographed component image by using the previously built AI-based learning model data, and the search request and result receiver 104 retrieves the search result from the image retrieval apparatus 14. Receive The search results include matching part images and part name information of the corresponding part images.

The pre-feedback unit 106 limits the search area by the user feedback before the search of the image search apparatus 14. There are a variety of similar parts in the equipment. Even if they are the same shape, they will have different names if they differ in size depending on where they are used. Accordingly, the prior feedback unit 106 narrows the scope of the search before performing the image search by using the feedback (confirmation process) using the user's knowledge.

The dictionary feedback unit 106 provides a keyword search function to the user. In this case, the keyword search unit may be configured in the order of category> product model> main part> parts. A detailed configuration of the pre-feedback unit 106 will be described later with reference to FIG. 12, and a pre-feedback embodiment of the pre-feedback unit 106 will be described later with reference to FIG. 15.

When the post feedback unit 108 receives a plurality of search result values from the image search apparatus 14, the search parts may be provided by linking the parts part book with each search part image and providing part part information on the corresponding search part. Feedback to the user to finally select the part they want to find. A post feedback embodiment of the post feedback unit 108 will be described later with reference to FIG. 15.

The component recommendation unit 109 recommends the main component object in the photographed part image to the user when there are a plurality of parts in the photographed part image. For example, Faster R-CNN is used to provide the user with the name and probability value of the main part in the photographed part image. Faster R-CNN is a fast R-CNN to solve R-CNN's speed problem. It does not extract features from the input image, but it is spatial on a feature map through a convolutional neural network (CNN). Extracting features using RoI Pooling, a special form of Pyramid Pooling. If a component intended by the user is not recommended, the image selection unit 102 may further select a search area using a user interface such as a crop interface. A component recommendation embodiment of the component recommendation unit 109 will be described later with reference to FIG. 17.

12 is a detailed block diagram of the pre-feedback unit of FIG. 11 according to an exemplary embodiment.

11 and 12, the pre-feedback unit 106 includes a category selector 1060, an equipment model selector 1062, and a main component selector 1064.

The category selector 1060 selects and inputs a category from a client user, the equipment model selector 1062 receives a model input from a selected category, and the main component selector 1064 receives and inputs a main component from the selected model. . The configuration of the pre-feedback unit 106 is performed before the client requests the component image search from the image retrieval apparatus. For example, the pre-feedback unit 106 may be performed before or after the user's image capture.

FIG. 13 is a client screen illustrating an example of cropping (cropping) an image of a client according to an exemplary embodiment. FIG.

Referring to FIG. 13, a client user selects a component area to be searched using a cropping interface from among photographed images displayed on the client screen.

14 is a reference diagram illustrating a client screen to show a result of a part image search according to an exemplary embodiment.

Referring to FIG. 14, in response to a search request from a client, a part image search result is displayed on a client screen. At this time, the top search results (for example, four) are output on the screen. When the user selects each part image in the top search result, the user can check the part information of the part part book associated with the corresponding part image and confirm the search reliability through this part information. The part information of the part part book may be a photorealistic image (or a virtual image). It can also be implemented in 2D or 3D images.

15 is a reference diagram for explaining a part image retrieval process considering a similar part according to an embodiment of the present invention.

There are a variety of similar parts in the equipment. Even if they are the same shape, they will have different names if they differ in size depending on where they are used. Accordingly, the present invention seeks to increase accuracy through user feedback before and after searching for parts. In the example of the user feedback before the search, the scope of the search is narrowed before performing the image search by using the feedback (confirmation process) using the user's knowledge. As an example of the user feedback after the search, the user feedback for verifying whether the part name is correct through linkage with the parts part book after performing the image search is used.

In an embodiment, as shown in FIG. 15, after the client user performs category selection, model selection, and main part selection (pre-feedback), the client user requests an image search for the selected part. Upon receiving a plurality of search result values, the client selects each search part image to check the part part information of the selected search part image and finally selects the part that the user wants to find (post-feedback).

16 and 17 are reference diagrams for describing an image retrieval process in consideration of similar parts according to an exemplary embodiment.

When performing a part search using a client, for example a smartphone, it is virtually difficult to photograph only one part. Therefore, there is a need for selecting a part that the user wants to find. According to an embodiment of the present disclosure, when several parts are photographed in one photograph, a cropping interface may be provided to allow a user to select an area suitable for his / her intention (see FIG. 16). Further, after performing the image search, the user feedback for verifying the correct part name through linkage with the part part book may be used.

As another example, the user recommends a main part object in the photographed part image. For example, Faster R-CNN may be used to provide the user with the name and probability value of the main part in the photographed part image to be selected by the user (see FIG. 17). Faster R-CNN is a network structure that performs deep learning-based object detection. 17 illustrates an example of recommending three main component objects in an image, and a user may select a predetermined component object among them. If a user intended part is not provided, an additional cropping interface may be provided to allow the user to select a search area. Further, after performing the image search, the user feedback for verifying the correct part name through linkage with the part part book may be used.

As described above with reference to FIGS. 1 to 17, artificial technology, which has been remarkably developed recently, may be applied to parts search of equipment. Due to the characteristics of deep learning, the more the training data, the more the accuracy is improved. In the present invention, the parts are captured by a video and an image necessary for learning is extracted from the captured video. In addition, a dedicated parts image management program is provided for more sophisticated images for learning. We construct a learning model with high accuracy of image retrieval through bulk training and incremental training. Based on this learning model, we build a real-time parts image retrieval system that can be photographed on a smartphone and then searched in the field. The built-in real-time parts image retrieval system enables both end users and experts to identify part names faster than ever before, making it easier to order faults and perform other tasks.

So far, the present invention has been described with reference to the embodiments. Those skilled in the art will appreciate that the present invention can be implemented in a modified form without departing from the essential features of the present invention. Therefore, the disclosed embodiments should be considered in descriptive sense only and not for purposes of limitation. The scope of the present invention is shown in the claims rather than the foregoing description, and all differences within the scope will be construed as being included in the present invention.

1: AI based part search system 10: client
12: learning model management device 14: image search device
100: part photographing unit 102: image selection unit
104: search request and result receiving unit 106: dictionary feedback unit
108: post-feedback part 109: parts recommendation part
120: image acquisition server 122: training server
125: parts video collection unit 126: parts image processing unit
127: deep learning unit 140: service server
142: Web Server 144: Periodic Server
145: search request receiving unit 146: search unit
147: search result provider 148: parts part book provider
149: parts ordering unit 150: statistical information providing unit
1060: Category selector 1062: Equipment model selector
1064: main part selector 1260: part image extractor
1262: part image adjusting unit 1264: part image dumping unit

Claims (11)

A client for capturing a part image for obtaining part information or obtaining a photographed part image and requesting a part search together with the photographed part image;
A learning model management device for generating and storing a plurality of parts image learning models through artificial intelligence based machine learning; And
Upon receiving a part search request from the client, an image search device which searches a part image matching a photographed part image in a learning model stored in the learning model management device and provides a search result including the found part image and the corresponding part name to the client. ; Including;
Clients
If a large number of parts are included in the shot part image, it provides a user interface that allows the user to select the desired image for the part search before the part search, and if a large number of parts are found after the part search, the shot part image for user selection. AI-based parts search system characterized in that it provides the name and probability value of the main parts. The method of claim 1, wherein the client,
A component photographing unit which photographs a component image or acquires a photographed component image;
An image selector configured to provide a user interface for selecting an image for which a component is to be searched from among the photographed component images acquired by the component photographing unit and to receive a selection input from a user; And
A search request and a result receiving unit which transmits a part search request together with a photographed part image or a selected input part image to the image search apparatus and receives a part search result from the image search apparatus;
Artificial intelligence-based component search system comprising a. The method of claim 2, wherein the search request and result receiving unit
When the search request, AI-based parts search system, characterized in that the request by specifying the number of results to be included in the parts search results of the image search apparatus. The method of claim 2, wherein the client,
Prior to the search of the image search device, the pre-feedback unit for limiting the search area of the image search device in response to feedback from the user through the keyword search function; And
When receiving a plurality of search result values from the image search apparatus, the post-feedback unit for providing feedback to the user to finally select the parts to be searched among the search parts by providing the parts part information with each search part image;
Artificial intelligence-based parts search system further comprises. delete According to claim 1, The learning model management device,
Part video collection unit for obtaining a part video for learning;
A component image processor extracting a component image from the component video acquired through the component video collector and performing image processing and editing on the extracted component image; And
A deep learning learner configured to learn a part image edited through the part image processor based on deep learning to generate a learning model;
Artificial intelligence-based component search system comprising a. The method of claim 6, wherein the component image processing unit,
A part image extracting unit extracting a part image from the obtained moving image;
A part image adjusting unit providing a user interface for editing including at least one of selecting, deleting, grouping, labeling, and cropping the part image extracted by the part image extracting unit; And
A part image dump unit which dumps the training data according to a data format required for training training by fitting the part image edited by the user interface;
Artificial intelligence-based component search system comprising a. The method of claim 6, wherein the deep learning learning unit,
An AI-based component search system comprising selecting a predetermined deep learning method from among a plurality of deep learning methods, and verifying whether learning is normally performed using at least one of accuracy and cross entropy. The method of claim 6, wherein the deep learning learning unit,
An AI-based component search system for learning a part image using at least one of bulk training and incremental training. The apparatus of claim 1, wherein the image retrieval device is
A search request receiver configured to receive a part search request together with a photographed part image from a client;
A search unit for searching for a part image matching the photographed part image in the learning model stored in the learning model management device; And
A search result providing unit which provides a search result including a search part image and a search part name to a client, and provides search results in an order similar to the photographing part image requested to be searched by the client;
Artificial intelligence-based component search system comprising a. The method of claim 10, wherein the image search apparatus,
A parts part book provider for providing parts part information together with a search part image;
A parts ordering unit providing an ordering service for the searched parts; And
Statistical information providing unit for providing statistical information about the search component;
AI-based component search system further comprises at least one of.

Images