KR20170081348A - Method and System for Intelligent Mining of Digital Image Big-Data - Google Patents

Abstract

The present invention provides a high-speed automatic object extraction technique for objects in a large-scale image and a graph-division-based intelligent image data mining technique for extracted image metadata, (eg, useful background information such as a background set / object set of images) that has been overlooked in relation to the image data itself or related to the unsorted large-scale image data, The present invention relates to an intelligent mining method and a processing system for a video big data which can be presented to a user and help a user to make a decision in the future.

Description

TECHNICAL FIELD [0001] The present invention relates to an intelligent mining method and a processing system,

The present invention relates to a method and system for processing image data, and more particularly, to an efficient and effective automatic object extraction for large-scale image big data, and an intelligent mining method and processing system for extracted data.

For the processing of image data, object segmentation / extraction in the field of 'image processing', 'computational vision', 'machine learning', 'data mining' image segmentation / extraction, image abstraction, and graph paritioning are considered. There are various application contents using image data mining, image retrieval, and grouping and clustering as application fields.

However, due to problems such as automation solution, storage space, data modeling, etc., processing technology for large-scale image big data is insufficient. Therefore, efficient and effective processing technology for large-scale image big data is needed.

SUMMARY OF THE INVENTION The present invention has been made in order to solve the above problems, and it is an object of the present invention to provide a high-speed automatic object extraction technique for a large-scale image object and a graph - It provides segmentation-based intelligent image data mining technology. In application aspect, it analyzes data of unsorted large-scale image data completely automatically and extracts useful advanced information that has been overlooked in relation to image data itself (For example, useful relationship information such as a background set / object set of images) to a user, thereby providing an intelligent mining method and a processing system for the video big data, which can help a user's decision in the future .

The technical problems of the present invention are not limited to the above-mentioned technical problems, and other technical problems which are not mentioned can be understood by those skilled in the art from the following description.

According to an aspect of the present invention, there is provided an image big data processing system for visualizing and providing relationship information of images through mining of large-scale image big data, An interest object extracting unit that receives large-scale image big data including an image set and separates a background and an object image in units of an image set, and collects an object image set and a background image set; And extracting a characteristic of each image from the object image set and the background image set, acquiring a relationship between the image and the characteristic, mining through visual analysis, and visualizing the relationship information of the images as graphical information And a big data mining unit.

Wherein the interest object extracting unit comprises: an input unit for receiving the large-scale image big data and storing the large-sized image big data in a storage unit; A data distribution processor for distributing the large-scale image big data in units of predetermined image sets; And a controller for processing each of the plurality of image sets, wherein each controller separates a background and an object image from each image with respect to the image to be processed, and extracts the object image set and the background image set .

Wherein the image big data mining unit includes a mining unit for extracting a feature of each image from the object image set and the background image set and generating a feature relationship map that links the relation of one or more images to each extracted feature, Wherein the mining unit groups clusters of images through a spectral analysis of each image in the feature relationship map, wherein clustering is performed between image groups of a clustered object image set, between image groups of a background image set, And the image group of the set of background images.

The image big data mining unit includes a visualization unit for providing an intuitive user interface by visualizing the feature relationship map and the relationship information as the graphical information on a screen of a display device.

In another aspect of the present invention, there is provided a method for visualizing and providing relation information of images through mining of large-scale image big data, comprising the steps of: receiving large-scale image big data including a plurality of image sets; Collecting an object image set and a background image set by separating a background and an object image in units of image sets; Extracting a characteristic of each image from the object image set and the background image set, acquiring a relation between the image and the characteristic, mining through visualization and visualizing the object image, and visualizing the relationship information of the images as graphical information .

The collecting of the object image set and the background image set may include: receiving the large-scale image big data and storing the large image big data in the storage means; Distributing the large-scale image big data in units of a predetermined image set; And collecting the set of object images and the set of background images by separating the background and object images from each image for the set of images to be processed in the respective controllers using respective controllers for processing each of the plurality of sets of images .

Wherein the visualizing step comprises the steps of: extracting features of each image for the object image set and the background image set, and creating a feature relationship map for linking the relationships of one or more images to each extracted feature; And clustering the images through a spectral analysis on each image in the feature relationship map, wherein the images are grouped into a group of image groups of a clustered object image set, a group of image sets of the background image set, And estimating relationship information between the groups of images of the set.

The visualizing step includes visualizing the feature relationship map and the relationship information as the graphical information on a screen of the display device to provide an intuitive user interface.

According to the intelligent mining method and processing system of video big data according to the present invention, large-scale image big data is input, and background and object images are separately extracted and collected in units of image sets, and feature extraction and relational modeling are performed and spectral analysis is performed By providing efficient and effective automatic object extraction for large-scale image big data and intelligent mining processing of extracted data by providing advanced information such as mining processing and visualization and useful relationship information such as background set / object set of images, It can help the user to make a decision.

In addition, the present invention can intelligently preprocess large-scale image-based application services through process automation in the processing of large-scale image big data. The automation technology of the video object extraction of the present invention can be extended as a core function in most systems using image input (for example, application contents including object recognition, annotation, compression, search, robot vision, education, game, etc.).

In addition, the present invention enables high-level and practical use of analysis results through intelligent mining of image big data. The semantic and relational analysis between images can be made through the application of the video big data of the present invention. In addition, the results of this analysis can be extended to decision makers and consumers as well as general users, as well as applications in help-related application services.

Further, the present invention can be utilized as a recommendation system of customized, formula, and the like through intelligent mining of image big data. In other words, it is possible to extend the application of clothing-related, clothing-related, cultural-art-related information recommendation by using images existing on the personalized web. For example, it is possible to extract a portrait object and a background separately from an image collected from on-line or off-line, extract background-related characteristic information such as weather and a location from the background image, and extract body, clothes size, Color, and so on. "In any place and weather, A type user prefers B type C, D color clothes." Can be derived. This information can be used as highly useful information for the user (consumer) or producer selection, sale, and item recommendation.

1 is a view for explaining a video big data processing system according to an embodiment of the present invention.
FIG. 2 is a block diagram for explaining the interest object extracting unit of FIG. 1; FIG.
FIG. 3 is a diagram for explaining a specific example of the controller of FIG. 2. FIG.
FIG. 4 is a block diagram for explaining a video big data mining unit of FIG. 1; FIG.
5 is a diagram for explaining an example of the processing result in the mining unit in Fig.
6A is an example of a user interface according to the processing result in the controller of FIG.
6B is an example of a user interface according to the processing result in the mining unit of FIG.
7 is a diagram for explaining an example of a method of implementing a video big data processing system according to an embodiment of the present invention.

Hereinafter, some embodiments of the present invention will be described in detail with reference to exemplary drawings. It should be noted that, in adding reference numerals to the constituent elements of the drawings, the same constituent elements are denoted by the same reference symbols as possible even if they are shown in different drawings. In the following description of the embodiments of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the difference that the embodiments of the present invention are not conclusive.

In describing the components of the embodiment of the present invention, terms such as first, second, A, B, (a), and (b) may be used. These terms are intended to distinguish the constituent elements from other constituent elements, and the terms do not limit the nature, order or order of the constituent elements. Also, unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the relevant art and are to be interpreted in an ideal or overly formal sense unless explicitly defined in the present application Do not.

First, a video object extraction technique, a video big data processing technique, and an image data mining technique can be used for processing the video big data. First, video object extraction technology is considered as a very important subject in the field of computer vision. It is called 'object-of-interest image segmentation', 'object recognition', 'object annotation' 'Is a core module for providing application services such as' adaptive image compression', 'image retrieval', 'image-based content creation (eg, image synthesis, non-photorealistic rendering, etc.)' . However, video object extraction technology is inherently an ill-posed problem of underconstrained conditions, making it difficult to develop a fully automated solution.

In addition, due to the rapid development of Web technology and data collection technology, it is possible to efficiently and effectively display large video data having characteristics of 'complexity', 'diversity' and 'scalability' Processing issues are important issues. For such large-scale data processing, specialized algorithms for large-scale efficient matrix computation and software and hardware systems (eg, MapReduce and GPU (Graphic Processing Unit) technology) for distributed-parallel processing are required. Particularly, in the case of image data, there is a difficulty in memory space requirement and data processing because of the characteristic of generally having a large data dimension as compared with other data. For example, a dense similarity matrix element for a full HD class pixel of 10 ⁶ or more requires 10 ¹² or more, thus requiring 8 TB of storage space for a double data instance.

Data mining is a technique for deriving new data models and patterns that have not been known in a given data, extracting useful information in the future, and using the information for future decision making. In the present invention, ) Video Big data is targeted. For such large data mining, detailed modeling, partitioning, and co-clustering techniques for large-scale graph-based data processing are required.

1 is a diagram for explaining a video big data processing system 100 according to an embodiment of the present invention.

1, the image data processing system 100 according to an exemplary embodiment of the present invention includes a Fast & Fully-Automatic Salient Object Collection (FFA-SOC) And an image big data mining unit 120 for the Big Object Data Mining (hereinafter, abbreviated as B-VDM).

The interest object extracting unit 110 receives the large-scale image big data and separates and extracts the background image and the object image through the core module (FFA-SOC Core Module) in units of image sets to extract the object image set (s) Are collected. Large-scale image big data can be obtained through a user terminal, the Internet web, and the like. The user terminal may be a terminal that works through a wired / wireless network supporting wired Internet communication, wireless Internet communication such as WiFi, WiBro, WCDMA, LTE mobile communication or WAVE (Wireless Access in Vehicular Environment) wireless communication. For example, the user terminal may be a wireless terminal such as a smart phone, a wearable device capable of voice / video phone communication, a tablet PC, a notebook PC, and the like, and may also include a wired terminal such as a desktop PC or other communication exclusive terminal.

The image big data mining unit 120 extracts the characteristics of each image for the collected object image set (s) and the background image set (s) using a core module (B-VDM Core Module) And minutiae processing and visualization through spectral analysis to provide advanced information such as useful information such as object set and background set of images.

FIG. 2 is a block diagram for describing the interest object extraction unit 110 of FIG. 1 in detail.

2, the interest object extracting unit 110 includes an input unit 111 for inputting large-scale image big data, a data distribution processing unit 112 for distributing input image data, And an extracting unit 113 for extracting the image.

The input unit 111 can receive large-scale image big data obtained through a user terminal, the Internet web, and the like, and store the large-sized image big data in a predetermined storage means (e.g., a memory or the like).

The data distribution processing unit 112 distributes the large-scale image big data as a predetermined image set unit to receive and process the large-sized image big data. The data distribution processing unit 112 may extract the large-scale image big data of the storage means according to a predetermined execution command and distribute the large-sized image big data in units of image sets. The image set unit may be a set of images for a predetermined frame or a set of images within a predetermined frame that can be classified by terminal or website.

The extracting unit 113 includes respective controllers for processing each of the image sets distributed by the data distribution processing unit 112. Each of the controllers has a predetermined Algorithms such as saliency map generation and adptive matting are performed to separate the background image and the object image from each of the included images to obtain a collection of extracted image sets Object Images) and a collection of Extracted Background Images (see FIG. 6A).

FIG. 4 is a block diagram for explaining the video big data mining unit 120 of FIG. 1 in detail.

Referring to FIG. 4, the image big data mining unit 120 includes a mining unit 121 and a visualization unit 122.

The mining unit 121 receives the processing result from the interest object extraction unit 110, that is, the extracted object image set (s) and the background image set (s) The spectral analysis method of the image data mining process is performed.

That is, the mining unit 121 extracts characteristics of each image (e.g., background type information such as weather and place) with respect to the object image set (s) Object type information such as body, clothes size, color of top and bottom, etc.) and extracts the characteristics of one or more images belonging to each extraction feature through a mass data parallel processing based on Map Reduce framework You can create a relationship map (eg, a k-partite graph structure).

In addition, the mining unit 121 can perform a co-clustering function by mining processing based on a spectroscopic analysis. The mining unit 121 clusters (or groups) images having similar spectral characteristics through spectral analysis (spectral analysis) of the respective images in the feature relationship map generated as described above. As shown in FIG. 5, (Or grouped) image groups, i.e., between image groups of the object image set (s), between image groups of the background image set (s), between image groups of the object image set (S) of image groups. For example, it is possible to estimate useful relationship information for recommendation such as customized, expression, and example, so that information related to clothing, food, and culture art related to an individual can be recommended using images existing on a personalized web have.

6A and 6B, in order to intuitively provide the user with the meaning according to the processing result in the interest object extracting unit 110 and the image big data mining unit 120, the visualizing unit 122 displays an intuitive UI Interface or the like to provide graphical information based on two-dimensional or three-dimensional information on a screen of a display device.

The visualization unit 122 has a window for selecting a menu, an input / extracted image set list window (Extracted Image Set # List)), two main windows, a data processing related message (Print Messages) can be provided. According to the user's selection on the screen provided as described above, the visualization unit 122 displays the results before and after the processing in the controller of the extraction unit 113, that is, And the object image set (s) after extraction and the background image set (s) can be selectively displayed as graphical information. In addition, as shown in FIG. 6B, the result of the processing before and after the processing in the mining unit 121, (Eg, the k-partite graph structure) (eg, the left main window) that links the relationships of one or more objects / background images belonging to the extracted feature to the selected group and useful relationship information obtained through simultaneous clustering (E.g., the right main window) can be visualized and provided as graphical information.

7 is a view for explaining an example of a method of implementing the video big data processing system 100 according to an embodiment of the present invention. The image big data processing system 100 according to an exemplary embodiment of the present invention may be implemented by hardware, software, or a combination thereof. For example, the video big data processing system 100 may be implemented in the computing system 1000 as shown in FIG.

The computing system 1000 includes at least one processor 1100, a memory 1300, a user interface input device 1400, a user interface output device 1500, a storage 1600, And an interface 1700. The processor 1100 may be a central processing unit (CPU) or a memory device 1300 and / or a semiconductor device that performs processing for instructions stored in the storage 1600. Memory 1300 and storage 1600 may include various types of volatile or non-volatile storage media. For example, the memory 1300 may include a ROM (Read Only Memory) 1310 and a RAM (Random Access Memory)

Thus, the steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by processor 1100, or in a combination of the two. The software module may reside in a storage medium (i.e., memory 1300 and / or storage 1600) such as a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, You may. An exemplary storage medium is coupled to the processor 1100, which can read information from, and write information to, the storage medium. Alternatively, the storage medium may be integral to the processor 1100. [ The processor and the storage medium may reside within an application specific integrated circuit (ASIC). The ASIC may reside within the user terminal. Alternatively, the processor and the storage medium may reside as discrete components in a user terminal.

As described above, in the image big data processing system 100 according to the present invention, large-scale image big data is input, and a background image and object image are separately extracted and collected in units of an image set, and feature extraction and relational modeling are performed, And intelligent mining processing of extracted data by providing efficient and effective automatic object extraction for large-scale image big data and providing advanced information such as useful background information such as background set / object set of images, It can help the decision of the future user. In addition, in the present invention, preprocessing of a large-scale image-based application service can be made intelligent through process automation in the processing of large-scale image big data. The automation technology of the video object extraction of the present invention can be extended as a core function in most systems using image input (for example, application contents including object recognition, annotation, compression, search, robot vision, education, game, etc.).

Further, in the video big data processing system 100 according to the present invention, it is possible to make high-level and practical use of analysis results through intelligent mining of video big data. The semantic and relational analysis between images can be made through the application of the video big data of the present invention. In addition, the results of this analysis can be extended to decision makers and consumers as well as general users, as well as applications in help-related application services. Further, the present invention can be utilized as a recommendation system of customized, formula, and the like through intelligent mining of image big data. In other words, it is possible to extend the application of clothing-related, clothing-related, cultural-art-related information recommendation by using images existing on the personalized web. For example, it is possible to extract a portrait object and a background separately from an image collected from on-line or off-line, extract background-related characteristic information such as weather and a location from the background image, and extract body, clothes size, Color, and so on. "In any place and weather, A type user prefers B type C, D color clothes." Can be derived. This information can be used as highly useful information for the user (consumer) or producer selection, sale, and item recommendation.

The foregoing description is merely illustrative of the technical idea of the present invention, and various changes and modifications may be made by those skilled in the art without departing from the essential characteristics of the present invention.

Therefore, the embodiments disclosed in the present invention are intended to illustrate rather than limit the scope of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents should be construed as falling within the scope of the present invention.

The interest object extraction unit 110 extracts,
In the video big data mining unit 120,
The input unit 111,
The data distribution processor 112,
The extracting unit 113 extracts,

Claims (1)

1. An image big data processing system for visualizing and providing relation information of images through mining of large-scale image big data,
An interest object extracting unit that receives large-scale image big data including a plurality of image sets and separates the background and object images in units of image sets, and collects a set of object images and a set of background images; And
The object image set and the background image set are extracted by extracting the characteristics of each image, acquiring the relation between the image and the characteristic, mining through spectral analysis, visualizing the relation information, and visualizing the relation information of the images as graphical information. And a data mining unit.

Images