KR102169700B1 - Apparatus for processing image and method for the same - Google Patents

Abstract

According to the present disclosure, an image processing apparatus may be provided. The image processing device corresponds to an image management unit that manages a divided image obtained by dividing an input image into a predetermined semantic unit, a query processing unit that provides a query term and receives answer information for the query term, and answer information for the query term. An image search processing unit that searches for the divided image to be used, an image recommendation processing unit that determines at least one slice image that can be used for a reproduction image from the divided image searched by the image search processing unit, and the at least one slice image are combined. Thus, it may include an image reproducing unit for generating and outputting the reproduced image.

Description

Image processing apparatus and method {APPARATUS FOR PROCESSING IMAGE AND METHOD FOR THE SAME}

The present disclosure relates to image processing technology, and more particularly, to a method and apparatus for detecting a sub image to be used for image combination.

With the recent development of the Internet, an environment that can provide a variety of contents to users has been established, and an environment that can play a role as a provider that creates and supplies contents has been built away from simply using contents.

Furthermore, as mobile devices are popularized and various social media services are provided, users of mobile devices are actively creating and sharing various contents without region or time constraints.

As described above, various contents, especially contents related to images, are provided freely, but since they are shared without accurately reflecting the user's request or intention, there is a problem that the user may be hindered from finding the desired contents.

On the other hand, video content is being used only as a one-time consumer product that can deliver simple messages repeatedly, and it is not utilizing all of the potential strengths of video content. Since the video content has an understanding of 30 times or more and a recognition speed of 60,000 times or more compared to text data, the accumulated video information can be used as an intuitive information delivery means.

The present disclosure has been devised in consideration of the above points, and an object thereof is to provide a method and an apparatus capable of generating new content by recombining video content.

Another technical problem of the present disclosure is to provide a method and apparatus for dividing video content to compose new content by recombining the video content, and detecting the divided video suitable for a user's intention.

Another technical problem of the present disclosure is to provide a method and apparatus capable of more effectively detecting a segmented image suitable for a user's intention.

The technical problems to be achieved in the present disclosure are not limited to the technical problems mentioned above, and other technical problems that are not mentioned will be clearly understood by those of ordinary skill in the technical field to which the present disclosure belongs from the following description. I will be able to.

According to an aspect of the present disclosure, an image processing apparatus may be provided. The device includes an image management unit that manages a divided image obtained by dividing an input image into a predetermined semantic unit, a query processing unit that provides a query term and receives answer information for the query term, and the response information corresponding to the query term. An image search processing unit that searches for a divided image, an image recommendation processing unit that determines at least one slice image that can be used for a reproduced image from the divided image searched by the image search processing unit, and the at least one slice image are combined, and the It may include an image reproducing unit that generates and outputs a reproduced image.

Features briefly summarized above with respect to the present disclosure are only exemplary aspects of the detailed description of the present disclosure described below, and do not limit the scope of the present disclosure.

According to the present disclosure, a method and an apparatus capable of generating new content by recombining video content may be provided.

In addition, according to the present disclosure, there is provided a method and apparatus for dividing and managing video content into predetermined semantic units, and composing a new content by combining the divided and managed images.

In addition, according to the present disclosure, a method and apparatus capable of more accurately detecting an image required to compose new content from an image divided into a predetermined semantic unit and managed may be provided.

The effects that can be obtained in the present disclosure are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those of ordinary skill in the art from the following description. will be.

1 is a diagram illustrating a computing environment in which an image processing apparatus according to an embodiment of the present disclosure is implemented.
2 is a block diagram illustrating a configuration of an image processing apparatus according to an embodiment of the present disclosure.
3 is a diagram illustrating that an image input by an image management unit provided in an image processing apparatus according to an exemplary embodiment is divided into semantic units.
4 is a diagram illustrating a divided image processed by an image processing apparatus according to an embodiment of the present disclosure.
5A to 5D are diagrams illustrating a fragment image and a reproduction image processed by an image processing apparatus according to an exemplary embodiment of the present disclosure.
6 is a flowchart illustrating a procedure of an image processing method according to an embodiment of the present disclosure.
7 is a flowchart illustrating a detailed configuration of an image recommendation step included in an image processing method according to an exemplary embodiment of the present disclosure.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art may easily implement the embodiments. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein.

In describing an embodiment of the present disclosure, if it is determined that a detailed description of a known configuration or function may obscure the subject matter of the present disclosure, a detailed description thereof will be omitted. In addition, parts not related to the description of the present disclosure in the drawings are omitted, and similar reference numerals are attached to similar parts.

In the present disclosure, when a component is said to be "connected", "coupled" or "connected" with another component, it is not only a direct connection relationship, but an indirect connection relationship in which another component is present It can also include. In addition, when a certain component "includes" or "have" another component, it means that other components may be further included rather than excluding other components unless otherwise stated. .

In the present disclosure, terms such as first and second are used only for the purpose of distinguishing one component from other components, and do not limit the order or importance of the components unless otherwise stated. Accordingly, within the scope of the present disclosure, a first component in one embodiment may be referred to as a second component in another embodiment, and similarly, a second component in one embodiment is a first component in another embodiment. It can also be called.

In the present disclosure, components that are distinguished from each other are intended to clearly describe each feature, and do not necessarily mean that the components are separated. That is, a plurality of components may be integrated to be formed in one hardware or software unit, or one component may be distributed in a plurality of hardware or software units. Therefore, even if not stated otherwise, such integrated or distributed embodiments are also included in the scope of the present disclosure.

In the present disclosure, components described in various embodiments do not necessarily mean essential components, and some may be optional components. Accordingly, an embodiment consisting of a subset of components described in the embodiment is also included in the scope of the present disclosure. In addition, embodiments including other components in addition to the components described in the various embodiments are included in the scope of the present disclosure.

Terms used in the present disclosure may be defined as follows.

The image may include a still image referred to as a normal image captured at any one time or a video in which a plurality of images captured at a plurality of times are combined.

The input image refers to a commonly used image and indicates an original image that is not divided or combined.

The split image is an image formed by dividing a partial region of the input image, and may indicate an image divided into at least one object unit included in the input image.

The fragment image may indicate an image searched or detected from the divided image.

The reproduced image may indicate an image formed by combining at least one fragment image.

Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings.

1 is a diagram illustrating a computing environment in which an image processing apparatus according to an embodiment of the present disclosure is implemented.

The following description related to FIG. 1 is for illustrating a computing environment in which the image processing apparatus of the present disclosure may be implemented. The illustrated computing environment of FIG. 1 is an example of an operating environment in which an image processing apparatus may be executed, and does not limit the scope of use or function of the present disclosure.

Examples of computing systems, environments and/or configurations that can be suitably used in the image processing apparatus according to an embodiment of the present disclosure include personal computers, server computers, hand-held or laptop devices, multiprocessor systems, and microprocessor-based systems. , Programmable home appliances, network personal computers, mini computers, main frame computers, distributed computing environments including any of the above systems or devices, and the like.

An image processing apparatus according to an embodiment of the present disclosure will generally be described in connection with computer executable instructions such as program modules executed by one or more computers or other devices. In general, program modules may include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. Typically, the functions of the program modules can be combined or distributed according to various environments.

Referring to FIG. 1, the computing device 100 typically includes at least one processing unit 102 and a memory 104.

Depending on the configuration and type of the computing device, the memory may be a volatile memory such as RAM, a nonvolatile memory such as ROM or flash memory, or a combination of them. Further, the computing device 100 may include additional (eg, removable and/or fixed) storage devices such as magnetic or optical disks or tapes. Additional storage devices may include a removable storage device 108 and a fixed storage device 110.

The memory 104, the removable storage device 108, and the fixed storage device 110 may all be provided as computer storage media, and RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, DVD (digital versatile disk) or other optical disk storage devices, magnetic cassettes, magnetic tapes, magnetic disk storage devices or other magnetic storage devices, or any other medium that can be accessed by the device 100 and store desired information. .

The computing device 100 may include a communication unit 112 that enables communication with other devices such as other computing devices through a wired or wireless network. The communication unit 112 generally implements a computer-readable command, a data structure, a program module, or other data on a modulated data signal such as a carrier wave or other transport mechanism. Includes information delivery media. The term "modulated data signal" means a signal in which one or more of the characteristics of the signal are set or changed to encode information in the signal. For example, the communication unit 112 may include a wired medium such as a wired network or a direct-wired connection, and a wireless medium such as sound, radio frequency, infrared, and other wireless media.

The computing device 100 may include an input device 114 such as a keyboard, a mouse, a pen, a voice input device, a touch input device, a laser range finder, an infrared camera, and a video input device. Further, the computing device 100 may include an output device 116 such as a display, a speaker, and a printer.

At least one processing unit 102 included in the computing device 100 may process an operation corresponding to the image processing apparatus and method according to an embodiment of the present disclosure. In particular, the processing unit 102 of the computing device 100 according to an embodiment of the present disclosure stores and manages the input image, divides the input image into semantic units, and divides the divided image region (hereinafter, referred to as'division Image'), an operation of searching and extracting at least one segmented image with high similarity based on a query input from a user, and an extracted segmented image (hereinafter referred to as'sculpted image') Combining and reproducing operations can be handled.

2 is a block diagram illustrating a configuration of an image processing apparatus according to an embodiment of the present disclosure. Referring to FIG. 2, an image processing apparatus according to an embodiment of the present disclosure includes an image management unit 210, an image DB 215, a query processing unit 220, an image search processing unit 230, an image recommendation processing unit 240, And an image reproducing unit 250.

The image manager 210 may store and manage the input image in the image DB 215. The input image may include information for distinguishing an object included in the image. For example, information for distinguishing an object included in the image may include data in a metadata format. To this end, when an image is input, the image management unit 210 checks the data of the input image, extracts information for distinguishing an object included in the input image, and stores the extracted information together with the input image and the image DB 215 Can be saved on.

In particular, the image management unit 210 according to an exemplary embodiment of the present disclosure may divide and manage an input image into meaning units.

3 is a diagram illustrating that an input image is divided into semantic units by an image management unit provided in an image processing apparatus according to an exemplary embodiment of the present disclosure.

Referring to FIG. 3, the image manager 210 may construct a divided image 320 by dividing the input image 310 into semantic units, and store the divided image in the image DB 215. In this case, the image management unit 210 may store information 330 indicating a semantic unit obtained by dividing the divided image 320, and preferably, the information indicating the semantic unit is stored in the image DB 215 in a meta data format. Can be saved.

In an embodiment of the present disclosure, the semantic unit may indicate a unit capable of classifying an object included in the input image 310. For example, if the input image is an image (refer to 400, FIG. 4) to capture a bicycle and a car passing through the road, the meaning unit is an object 401, 402, such as a bicycle, a car, or a road included in the image. 403) can be illustrated. Classification of these semantic units can be processed using various image recognition algorithms. In addition, information indicating the semantic unit (eg, meta data) may be identified by a user input or an image recognition algorithm. To this end, the image manager 210 may be provided to process an image recognition algorithm capable of classifying a semantic unit. In addition, the image management unit 210 may be provided to check information (eg, meta data) indicating a semantic unit by a user input or an image recognition algorithm.

Meanwhile, the query processing unit 220, the image search processing unit 230, the image recommendation processing unit 240, and the image reproducing unit 250 configure a reproduced image by combining fragment images selected from divided images according to a user's request. And can be output.

In order to confirm the user's request, the query processing unit 220 may provide an environment in which a query word is constructed and provided to the user, and an answer to the query word is received from the user. For example, the query processing unit 220 may display a query word and display an input window for inputting an answer to the query word. In response, the user may input the user's answer through an input device provided in the computing environment. Through such an operation, the query processing unit 220 may receive an answer to a query term from a user, and provide the received information (ie, answer information to the query term) to the search processing unit 230. At this time, the query terms are who, 何人, when, 何時, where, 何處, what is based on the sixth principle (hereinafter referred to as '5W1H'). , 何事), how, 如何, and why (何故) may include at least one of the information.

The image search processing unit 230 may receive answer information for a query term from the query processing unit 220 and may search for an image corresponding thereto from the image DB 215. Preferably, the image search processing unit 230 may search for a segmented image corresponding to answer information for a query word, targeting the segmented image stored in the image DB 215.

The image recommendation processor 240 may check a query conformance for the searched segmented image, detect a segmented image in consideration of the confirmed query conformance, and detect at least one of the detected segmented images as a slice image. For example, the image recommendation processor 240 may detect an image having a relatively high query conformance as a fragment image, or an image having a query conformance greater than or equal to a predetermined threshold as a fragment image.

In addition, the query processing unit 220, the image search processing unit 230, the image recommendation processing unit 240, and the image reproducing unit 250 compose a reproduced image by combining fragment images according to a user's request. 230 may search for each divided image to be employed as a fragment image to be used in the reproduction image. In response to this, the image recommendation processing unit 240 may respectively detect divided images to be respectively employed as fragment images of the reproduced image. For example, when the reproduced image is composed of first to fourth slice images, the image search processing unit 230 is based on the answer information (501 (refer to FIG. 5A )) for query words provided from the query processing unit 220, A plurality of divided images 502 corresponding to a one-piece image may be searched and provided to the image recommendation processing unit 240. In addition, the image recommendation processing unit 240 may check a query conformance for the searched segmented image 502 and detect 503 a segmented image in consideration of the confirmed query conformance. In addition, the image recommendation processing unit 240 detects an image having a relatively high query conformance as a first slice image 504 included in the playback image, or detects an image having a query conformance equal to or higher than a predetermined threshold. It can be detected by the image 504. In addition, as another example, the image recommendation processing unit 240 may provide the detected divided image 503 to the user through a display or the like, and may set the first sliced image 504 by the user's selection.

Next, the image search processing unit 230 searches for a plurality of divided images 512 corresponding to the second slice image based on the answer information 511 (refer to FIG. 5B) for the query word provided from the query processing unit 220. Thus, it may be provided to the image recommendation processing unit 240. Correspondingly, the image recommendation processing unit 240 may check a query conformance for the searched segmented image 512 and detect the segmented image 513 in consideration of the confirmed query conformance. In this case, the image recommendation processing unit 240 may be provided to check the similarity with the divided image 513 corresponding to the first slice image 504 and detect the recommended divided image 513 based on the checked similarity. have. In addition, the image recommendation processor 240 detects an image having a relatively high query conformance and/or similarity as the second fragment image 514 included in the playback image, or a query conformance equal to or greater than a predetermined threshold and/or An image having a similarity may be detected as a second fragment image 514. In addition, as another example, the image recommendation processing unit 240 may provide the detected recommended segmented image 513 to the user through a display or the like, and may set the second segmented image 514 according to the user's selection.

Further, the image search processing unit 230 and the image recommendation processing unit 240 repeat the same operation as the operation of detecting the second piece image 514, so that a plurality of images corresponding to the second piece image are performed, as illustrated in FIG. 5C. The segmented image 522 may be searched, the recommended segmented image 523 may be detected, and a third segmented image 524 may be detected. Then, as illustrated in FIG. 5D, a plurality of divided images 532 corresponding to a fourth slice image are searched, a recommended divided image 533 from the plurality of divided images 532 is detected, and finally, a fourth image A fragment image 534 may be detected.

The image reproducing unit 250 may configure a reproduced image by combining the first to fourth fragmented images 504, 514, 524, and 534 recommended through the image recommendation processing unit 240.

Meanwhile, the image recommendation processing unit 240 provided in the image processing apparatus according to an embodiment of the present disclosure calculates a query conformance based on answer information (hereinafter referred to as'answer information') for a query word, and input The query conformance can be calculated by taking into account proper nouns, important words, the frequency of input terms, and excluded words.

For example, the query conformance may be checked based on the accuracy of the answer information and information matched to the searched image.

The video recommendation processing unit 240 checks whether the answer information includes a proper noun by using the predefined vocabulary list data. In addition, when the answer information includes a proper noun and the information (meta data) matched with the searched divided image includes the proper noun, the image recommendation processing unit 240 increases the search accuracy by a predetermined value (eg, 2). Let it.

If the answer information includes a proper noun, but there is no proper noun corresponding to the information (meta data) matched to the searched divided image, the image recommendation processing unit 240 excludes the divided image from the recommendation target.

In addition, when the answer information does not include a proper noun, the video recommendation processing unit 240 checks whether the answer information includes an important word. As the answer information includes an important word, the image recommendation processing unit 240 determines the search accuracy to a predetermined value (eg, 1) when the information (meta data) matched to the searched divided image is the same as the important word included in the answer information. Increase by Furthermore, a plurality of important words may be included in the answer information. In consideration of this, the image recommendation processing unit 240 determines the number of important words in which the plurality of important words included in the answer information and the information (meta data) matched to the searched divided images are matched (hereinafter referred to as'important word matching frequency'). It can be checked, and search accuracy can be increased by reflecting the frequency of matching important words. For example, the image recommendation processing unit 240 may increase the search accuracy by the number of key word matching frequencies.

On the other hand, if the answer information includes a negative word and there is a negative word corresponding to information (meta data) matched to the searched divided image, the image recommendation processing unit 240 determines the accuracy of the searched divided image corresponding to the excluded word. It may be set to a predetermined value (eg, 0), or the divided image image may be excluded from the recommendation target.

The image recommendation processing unit 240 may process the above-described operation for each query word, and check the accuracy of the entire query word for each of the searched divided images.

Furthermore, the image recommendation processing unit 240 may reflect a weight for each morpheme of terms included in the answer information in calculating the query conformance.

For example, the image recommendation processing unit 240 may extract words included in answer information based on 5W1H. The image recommendation processing unit 240 may increase the number of appearances of the corresponding word by 1 when the extracted word is a word that has previously appeared in the log, and may set the number of appearances to 1 when the word appears for the first time. Furthermore, the image recommendation processing unit 240 may manage the count in units of 5W1H according to the morpheme of the word. That is, if the corresponding word is a word representing WHO, the number of appearances of the WHO unit morpheme may be increased by 1 and the total number of words may be increased by 1. The image recommendation processing unit 240 may repeatedly perform such an operation to accumulate the number of appearances of 5W1H morpheme units and the total number of appearances of words for all extracted words.

In addition, the image recommendation processing unit 240 accumulates the number of appearances for each word, the number of appearances in morpheme units, and the total number of words with respect to the answer information, and then calculates the appearance frequency for each word, and a word list including the word name and appearance frequency ( Predefined Word List) can be saved and managed. In addition, the image recommendation processing unit 240 may calculate the appearance frequency of the morpheme unit and use it as a weight when calculating query matching.

Furthermore, since the image recommendation processing unit 240 is for providing a fragment image that can be used in the reproduced image, the fragment image should not have different characteristics from each other in the reproduced image. Accordingly, when determining the fragment image, the image recommendation processing unit 240 may determine a fragment image by reflecting a similarity with another fragment image. For example, in a state in which detection of the first slice image is completed, the image recommendation processing unit 240 may detect a divided image for the second slice image. In this process, the image recommendation processing unit 240 checks the URI for the first fragment image and compares it with URIs for the plurality of images searched for the second fragment image. URI for the first slice image An image having the same URI is excluded from the segmented image searched for the second slice image. Through such an operation, the searched segmented image may be removed, and the similarity may be determined by checking a color histogram of the remaining segmented image. Specifically, a color histogram (hereinafter referred to as'first color histogram') for a predetermined number of frames (eg, the last 5 frames) of the first slice image is calculated, and a predetermined number of remaining segment images are calculated. Calculate a color histogram (hereinafter referred to as'second color histogram') for a frame (eg, starting 5 frames). Further, the image recommendation processing unit 240 may calculate a similarity between the first color histogram and the second color histogram, and determine whether to use the divided image as a second slice image based on the calculated similarity. For example, the image recommendation processing unit 240 uses a divided image whose calculated similarity exceeds a predetermined threshold value as a second slice image, or uses the divided image having the calculated similarity highest value as a second slice image. Can be used.

Hereinafter, an image processing method according to an embodiment of the present disclosure will be described. An image processing method according to an embodiment of the present disclosure may be performed by the image processing apparatus according to an embodiment of the present disclosure described above.

6 is a flowchart illustrating a procedure of an image processing method according to an embodiment of the present disclosure.

Referring to FIG. 6, in step S601, the image processing apparatus may store an input image in an image DB. The input image may include information for distinguishing an object included in the image. For example, information for distinguishing an object included in the image may include data in a metadata format. When an image is input, data of the input image may be checked, information for distinguishing an object included in the image may be extracted, and the extracted information may be stored in an image DB along with the input image.

In particular, in step S601, the image processing apparatus may divide and manage the input image into semantic units. The input image 310 (refer to FIG. 3) may be divided into semantic units to form a divided image 320, and the divided image may be stored in the image DB. In this case, the image processing apparatus may store information 330 indicating a semantic unit obtained by dividing the divided image 320, and preferably, information indicating the semantic unit may be stored in an image DB in a meta data format.

Although, in an embodiment of the present disclosure, the operation of storing and managing the image input in step S601 is illustrated, in the image processing method according to an embodiment of the present disclosure, step S601 processes steps S602 to S607 to be described later. It may be a pretreatment step for Therefore, when the divided image is stored and managed in the image DB so that the steps S602 to S607 described later can be processed, the aforementioned step S601 may be omitted.

Next, in step S602, in order to confirm the user's request, the image processing apparatus may provide an environment in which a query word is constructed and provided to the user, and an answer to the query word is received from the user. For example, in step S602, while displaying the query word, an input window for receiving an answer to the query word may be displayed, and an answer may be input from the user. Through such an operation, the image processing apparatus may receive an answer to a query word from a user and provide the received information (ie, answer information) to the search processor 230. At this time, the image processing apparatus uses the query language based on the sixth principle (hereinafter referred to as '5W1H'), who (何人), when (何時), where (where, 何處), It can be configured to include at least one of information about what (何事), how (how, 如何), and why (what, 何故).

When answer information is input, the image processing device may search for an image corresponding thereto. The image processing apparatus may detect a divided image corresponding to the answer information from the divided image stored in the image DB.

Since the image processing method according to the exemplary embodiment of the present disclosure is for constructing a reproduction image by combining a plurality of fragment images, a plurality of fragment images to be included in the reproduction image must be detected. Accordingly, in step S603, the image processing apparatus may designate a region in which a fragment image to be included in the reproduction image is to be employed.

Next, in step S604, the image processing apparatus may search for a segmented image corresponding to the specified segmented image as a target of the segmented image stored in the image DB.

Then, in step S605, the image processing apparatus may check a query conformance for the searched segmented image, and detect the fragment image in consideration of the confirmed query conformance. For example, the image processing apparatus may detect a segmented image having a relatively high query conformance as a fragment image, or may detect an image having a query conformance greater than or equal to a predetermined threshold as a slice image.

In addition, as another example, the image processing apparatus may configure a reproduced image by combining fragment images according to a user's request. To this end, the image processing apparatus may each detect divided images to be employed in each of a plurality of fragment images to be used in the reproduction image. For example, if the reproduced image is composed of the first to fourth slice images, in step S603, the image processing apparatus may designate a slice image to be included in the reproduced image as the first slice image. Then, in step S604, the image processing apparatus may search for a plurality of divided images corresponding to the first slice image. Further, the image processing apparatus may check a query conformance for the searched segmented image, and may detect a segmented image to be used as the first segmented image in consideration of the confirmed query conformance (S605). The image processing apparatus may detect a segmented image having a relatively high query conformance as a first slice image to be included in the playback image, or may detect a segmented image having a query conformance greater than or equal to a predetermined threshold as a first slice image. In addition, as another example, the image processing apparatus may provide a segmented image that can be used as a first sliced image to a user through a display or the like, and set the segmented image as the first sliced image by user's selection.

Next, the image processing apparatus may check whether all of the fragment images to be used for the reproduction image have been detected (S606). When all the fragment images to be used in the playback image are detected (S606-Yes), the image processing device proceeds to step S607, and when all the fragment images to be used in the playback image are not detected (S606-No), another fragment image to be used in the playback image Steps S603 to S605 may be repeatedly performed in order to detect.

In the above-described operation of the embodiment of the present disclosure, since the image processing apparatus exemplifies detecting the first fragment image, the image processing apparatus proceeds to step S603, and the image processing apparatus converts the fragment image to be included in the reproduced image into a second fragment image. Can be specified. In addition, in steps S604 and S605, the image processing apparatus may set a divided image to be used for the second fragment image.

The image processing apparatus may repeatedly perform steps S603 to S605 described above to set a divided image to be used for the third and fourth slice images.

Meanwhile, in step S607, the image processing apparatus may configure a reproduced image by combining the first to fourth fragment images and output the same.

Hereinafter, a detailed operation of the image recommendation step included in the image processing method according to an embodiment of the present disclosure will be described.

7 is a flowchart illustrating a detailed configuration of an image recommendation step included in an image processing method according to an exemplary embodiment of the present disclosure.

In step S701, an answer (ie, answer information) for the query word may be input. The answer information may correspond to step S602 described above.

In step S602, a query word is generated for each morpheme unit of 5W1H, and an input window for receiving an answer (i.e., answer information) is individually generated to receive the answer information in the morpheme unit of 5W1H or the morpheme of 5W1H. By generating and outputting all query words for a unit, and generating an input window for receiving all answers (ie, answer information), the answer information in the morpheme unit of 5W1H can be input.

The image processing apparatus may individually provide the answer information checked in step S602, that is, answer information in morpheme units of 5W1H, to step S701.

In step 702, the image processing apparatus uses the predefined vocabulary list data to check whether the input answer information includes a proper noun. Then, if the answer information includes a proper noun, step S703 is performed.

In step S703, the image processing apparatus checks the image DB and checks whether a divided image having information (meta data) matching the proper noun included in the answer information exists. When a divided image having information (meta data) matching the proper noun exists, the image processing apparatus increases the search accuracy by a predetermined value (eg, 2) (S704). On the other hand, when there is no divided image having information (meta data) matching the proper noun, the image processing apparatus excludes the corresponding answer information from the recommendation target (S705).

Also, if the answer information does not include a proper noun, the image processing apparatus proceeds to step S706.

In step S706, the image processing apparatus checks whether the answer information includes an important word. As the answer information includes an important word, the image processing apparatus increases the search accuracy by a predetermined value (e.g., 1) when the information (meta data) matched to the divided image is the same as the important word included in the answer information ( S708).

Furthermore, a plurality of important words may be included in the answer information. In consideration of this, in step S708, the image processing apparatus determines the number of important words (hereinafter referred to as'important word matching frequency') in which a plurality of important words included in the answer information and information (meta data) matched to the divided image are matched. It can be checked, and search accuracy can be increased by reflecting the frequency of matching important words. For example, the image processing apparatus may increase the search accuracy by the number of key word matching frequencies.

Meanwhile, if the answer information does not include an important word, the image processing apparatus proceeds to step S709. In step S709, the image processing apparatus may check whether the answer information includes a negative word. When the answer information includes the negative word, the image processing apparatus may proceed to step S710, and when the answer information includes the excluded word, the image processing apparatus may proceed to step S712.

In step S710, the image processing apparatus may check whether there is an exclusive word corresponding to the information (meta data) matched to the divided image (S710), and there is an exclusive word corresponding to the information (meta data) matched to the divided image. In case (S710-Yes), the image processing apparatus may set the accuracy of the divided image corresponding to the exclusion word to a predetermined value (eg, 0) (S711). As another example, in step S711, the image processing apparatus may exclude the divided image from the recommendation target.

The image processing apparatus may repeatedly perform the above-described steps S701 to S711 until all answer information in morpheme units of 5W1H is input (S712).

In step S713, the image processing apparatus may cumulatively calculate the accuracy of all answer information in the morpheme unit of 5W1H, and calculate the query conformance.

Further, in calculating the query conformance in the image processing method according to an embodiment of the present disclosure, a weight for each morpheme of a term included in answer information may be reflected.

For example, the image processing apparatus may extract words included in answer information based on 5W1H. For the extracted word, the image processing apparatus may increase the number of occurrences of the corresponding word if it is a word that has previously appeared in the log, and may set the number of appearances as an initial value (eg, 1) if the word appears for the first time. Furthermore, the image processing apparatus may manage the count in units of 5W1H according to the morpheme of the word. That is, if the corresponding word is a word representing WHO, the number of appearances of the WHO unit morpheme may be increased by 1 and the total number of words may be increased by 1. The image processing apparatus may repeatedly perform such an operation to accumulate the number of appearances in 5W1H morpheme units and the total number of appearances of words for all extracted words.

In addition, the image processing device accumulates the number of appearances for each word, the number of appearances in morpheme units, and the total number of words with respect to the answer information, calculates the appearance frequency for each word, and calculates a word list including the word name and appearance frequency ) Can be stored and managed. In addition, the image processing apparatus may calculate the appearance frequency of the morpheme unit and use it as a weight when calculating query matching.

Furthermore, since the image processing method according to an embodiment of the present disclosure is for providing a fragment image that can be used for a reproduced image, the slice image provided through recommendation should not have different characteristics from each other in the reproduced image. Accordingly, in the image processing method according to an exemplary embodiment of the present disclosure, in the step of recommending an image, the image may be recommended by reflecting a similarity with another fragment image. For example, in a state in which the detection of the first slice image is completed, the image processing apparatus may detect the divided image to be used as the second slice image. In this process, the image processing apparatus checks the URI of the first fragment image and compares it with the URIs of the plurality of images searched for the second fragment image. URI for the first slice image An image having the same URI is excluded from the image searched for the second slice image. Through such an operation, the searched segmented image may be removed, and the similarity may be determined by checking a color histogram of the remaining segmented image. Specifically, a color histogram (hereinafter referred to as'first color histogram') for a predetermined number of frames (eg, the last 5 frames) of the first slice image is calculated, and a predetermined number of remaining segment images are calculated. Calculate a color histogram (hereinafter referred to as'second color histogram') for a frame (eg, starting 5 frames). In addition, the image processing apparatus may calculate a similarity between the first color histogram and the second color histogram, and determine whether to use the divided image as the second fragment image based on the calculated similarity. For example, the image processing apparatus may use a segmented image whose calculated similarity exceeds a predetermined threshold value as the second slice image, or may use a segmented image having the highest calculated similarity value as the second slice image. .

Video can deliver information intuitively, but it was difficult for anyone to easily produce video and deliver information. However, as the amount of accumulated image information increases rapidly, a number of images that can meet the various needs of users have already been produced, so if you use them efficiently, you can create images even if you are not an expert with expensive equipment. Was prepared. In addition, since the technology for analyzing and automatically recognizing images is rapidly developing, it is becoming easier to divide images into semantic units so that users can easily use them.

An image processing apparatus and method according to an exemplary embodiment of the present disclosure may provide a new method of producing an image by dividing and storing images into semantic units, and reusing only an area required by a user.

The image processing apparatus and method according to the exemplary embodiment of the present disclosure do not simply recommend a search result or an image reflecting the user's preference, but meet the user's search intention and have heterogeneity in the process of combining fragment images to be included in the reproduced image. It is implemented to detect the lowest image.

As described above, according to an image processing apparatus and method according to an exemplary embodiment of the present disclosure, a new reproduced image is more conveniently generated by using the recommended or detected slice image by being configured to recommend or detect a fragment image to be used for a reproduced image. Configurable.

The exemplary methods of the present disclosure are expressed as a series of operations for clarity of description, but this is not intended to limit the order in which steps are performed, and each step may be performed simultaneously or in a different order if necessary. In order to implement the method according to the present disclosure, the illustrative steps may include additional steps, other steps may be included excluding some steps, or may include additional other steps excluding some steps.

Various embodiments of the present disclosure are not listed in all possible combinations, but are intended to describe representative aspects of the present disclosure, and matters described in the various embodiments may be applied independently or may be applied in combination of two or more.

In addition, various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof. For implementation by hardware, one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), general purpose It may be implemented by a processor (general processor), a controller, a microcontroller, a microprocessor, or the like.

The scope of the present disclosure is software or machine-executable instructions (e.g., operating systems, applications, firmware, programs, etc.) that allow an operation according to a method of various embodiments to be executed on a device or computer, and such software or It includes a non-transitory computer-readable medium (non-transitory computer-readable medium) which stores instructions and the like and is executable on a device or a computer.

Claims (10)

In the image processing device,
An image management unit that manages divided images obtained by dividing the input image into predetermined semantic units,
A query processing unit that provides a query term and receives answer information for the query term;
An image search processing unit for searching for the divided image corresponding to the answer information for the query term;
An image recommendation processing unit that determines at least one piece image that can be used for a reproduced image from the divided image searched by the image search processing unit,
An image processing apparatus comprising an image reproducing unit that combines the at least one fragment image to generate and output the reproduced image. The method of claim 1, wherein the image management unit,
Configure the divided image corresponding to the input image,
Storing the split image in an image database,
And storing information indicating the semantic unit corresponding to the divided image. The method of claim 2, wherein the information indicating the semantic unit,
An image processing apparatus comprising a metadata format and being stored in the image database. The method of claim 1, wherein the image management unit,
An image recognition algorithm capable of distinguishing the semantic units may be provided,
The image processing apparatus, characterized in that, through the image recognition algorithm, the information indicating the semantic unit corresponding to the divided image is checked. The method of claim 1, wherein the video recommendation processing unit,
Checking a query conformance to the searched segmented image, detecting the segmented image in consideration of the checked query conformance, and determining at least one of the detected segmented images as the at least one segmented image. Image processing device. The method of claim 1, wherein the video recommendation processing unit,
Determining the segmented image having a relatively high query conformance as the at least one slice image, or determining the segmented image having a query conformance greater than or equal to a predetermined threshold as the at least one slice image Image processing device. The method of claim 6, wherein the video recommendation processing unit,
An image processing apparatus comprising: calculating a query conformance in consideration of at least one of a proper noun, an important word, a frequency of an input term, and a negative word. The method of claim 6, wherein the video recommendation processing unit,
An image processing apparatus comprising: calculating a query conformance by sequentially considering proper nouns, important words, frequency of input terms, and excluded words. The method of claim 6, wherein the video recommendation processing unit,
After accumulating the number of appearances for each word, the number of appearances in morpheme units, and the total number of words, calculate the appearance frequency for each word, store and manage a Predefined Word List including word names and frequency of appearance,
An image processing apparatus comprising using at least one of the number of appearances for each word, the number of appearances in a morpheme unit, the frequency of appearances for each word, and a word list as a weight for calculating a query match. In the image processing method,
The process of managing divided images obtained by dividing the input image into predetermined semantic units,
The process of providing a query word and receiving answer information for the query word,
The process of searching for the segmented image corresponding to the answer information for the query term, and
A process of determining at least one fragment image that can be used for a reproduction image from the searched divided image,
And generating and outputting the reproduced image by combining the at least one fragment image.

Images