JP2013239006A - Object extraction method, object extraction device, and object extraction program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To extract with high accuracy an object included in an image.SOLUTION: An object extraction device which extracts an object included in an acquired image, comprises: image analysis means which analyzes the image to acquire a position and content of invisible information; object extraction means which extracts an object included in the image on the basis of the invisible information acquired by the image analysis means; invisible information corresponding processing means which performs predetermined processing on the object extracted by the object extraction means according to a pattern of the invisible information; and output means which outputs an image acquired by the invisible information corresponding processing means.

Description

  The present invention relates to an object extraction method, an object extraction device, and an object extraction program, and more particularly, to an object extraction method, an object extraction device, and an object extraction program for performing object extraction with high accuracy.

  Conventionally, using a technique such as chroma key, an image area of an object portion is extracted by focusing on a color difference between a photographed object (including a person, an object, character information, etc.) and a background. There is a method. However, for example, when the object and the background image have the same color, the object cannot be appropriately extracted from the image.

  Therefore, conventionally, for example, an object (object) including the same color as the background is extracted by shooting in front of a checkered pattern background of two colors and using the checkered pattern area information in addition to the color information. There exists a method (see, for example, Non-Patent Document 1). Conventionally, there is a method to extract the contour of an object by using an active contour model for an object extraction image using a striped background, and systematically divide the object and the background. (For example, refer nonpatent literature 2).

  Further, the above-described conventional method is a case where the object and the background are clearly separated, and the region can be extracted by setting the checkerboard pattern and the stripe to be dark in the background.

Hiroki Tsuji, Satoshi Yamashita, Toru Kaneko, “Area Extraction Using Chroma Keys Using Checkered Background”, The Institute of Electronics, Information and Communication Engineers, IEICE Tech. Reports CS 2005-62, IE 2005-117 (2005-12) Satoshi Yamashita, Hiroki Tsuji, Toru Kaneko, “Striped Chroma Key Using an Active Contour Model”, Journal of the Institute of Image Information and Television Engineers, Vol. 61, No, 2, pp. 101-109 (2007)

  Here, for example, in the case of image editing or painting software or application such that the user freely draws a picture on the background or pastes a predetermined image stored in advance in a memory or the like, the user's An object or a predetermined image drawn by handwriting or the like is drawn on the background color. Further, when an image edited with the above-described software or the like is printed on a print medium such as paper, the electronic information for identifying the background and the object is lost. Therefore, in order to extract only the object from such a print medium, it is conceivable to set a checkered pattern or a stripe deep in the background as in the conventional method described above.

  However, for example, even if the object area or image area drawn by the user is colored, depending on the color density, transmittance, etc., a dark checkered pattern or stripe set as the background may appear inside the object area or image area. May be displayed. Therefore, even if an object can be extracted using a dark checkered pattern or stripe, the checkered pattern or stripe can be visually recognized.

  Furthermore, conventionally, checkerboard patterns and stripes are merely used for extracting objects, and some operations and processes are performed on the extracted objects according to their types. Did not exist.

  The present invention has been made in view of the above-described problems, and an object thereof is to provide an object extraction method, an object extraction apparatus, and an object extraction program for performing object extraction with high accuracy.

  In order to solve the above-described problems, the present invention employs means for solving the problems having the following characteristics.

  The present invention provides an object extraction device that extracts an object included in an acquired image, an image analysis unit that analyzes the image and acquires the position and content of the non-visualization information, and the non-visualization information obtained by the image analysis unit An object extraction means for extracting an object included in the image based on the image, and a non-visualization information corresponding process for performing a predetermined process on the object extracted by the object extraction means in correspondence with the pattern of the non-visualization information Means and an output means for outputting an image obtained by the invisible information corresponding processing means.

  According to another aspect of the present invention, there is provided an object extraction method for extracting an object included in an acquired image, an image analysis step for analyzing the image to acquire a position and content of non-visualization information, An object extraction step for extracting an object included in the image based on visualization information, and non-visualization information for performing predetermined processing on the object extracted by the object extraction step in correspondence with the pattern of the non-visualization information It has a correspondence processing step and an output step for outputting an image obtained by the non-visualization information correspondence processing step.

  Further, the present invention is an object extraction program for causing a computer to function as each unit included in the object extraction device described above.

  In addition, what applied the component, expression, or arbitrary combination of the component of this invention to a method, an apparatus, a system, a computer program, a recording medium, a data structure, etc. is also effective as an aspect of this invention.

  According to the present invention, an object can be extracted with high accuracy.

It is a figure which shows an example of a function structure of the object extraction apparatus in this embodiment. It is a figure which shows an example of the hardware constitutions which can implement | achieve the object extraction process in this embodiment. It is a flowchart which shows an example of the object extraction process sequence in this embodiment. It is a figure which shows an example of a high frequency pattern. It is a figure for demonstrating the content of filtering. It is a figure which shows an example of the extraction result of the object at the time of applying a conventional method and this embodiment. It is a figure for demonstrating the example of an image in this embodiment, and the operation example of an object.

<About the present invention>
In the present invention, for example, a pattern such as a stripe or checkered pattern using invisible information is formed on the background, and the above-described invisible information is used for an object drawn or pasted on the background. Highly accurate object extraction. In this embodiment, an object is drawn on a background in which invisible information is embedded by software, an application, or the like, and a printing medium (for example, a paper medium) on which the drawn image is printed is used as an imaging unit such as a camera. This example shows how to extract the object part included in the image with high accuracy using the image information captured using the image information or the image information without the electronic position information of the object related to the background image. The form is not limited to this.

  Furthermore, in the present invention, a predetermined operation or process is performed on the object extracted in accordance with the pattern type of the invisible information described above. In the present invention, an operation and processing corresponding to an object can be set in combination with the background color.

  The non-visualized information is information formed by using, for example, a high-frequency pattern described later, and can be analyzed on software using a computer and is difficult (or cannot be recognized) by the user's visual (visual). Means information. The object includes, for example, a person, an object, character information, and the like, and includes a photographed image and an image drawn by a user on software.

  Hereinafter, preferred embodiments of an object extraction method, an object extraction apparatus, and an object extraction program according to the present invention will be described with reference to the drawings. Note that the image described below includes one frame (frame image) of a video.

<Object extraction device: functional configuration example>
Here, a functional configuration example of the object extraction device in the present embodiment will be described with reference to the drawings. FIG. 1 is a diagram illustrating an example of a functional configuration of an object extraction device according to the present embodiment.

  An object extraction apparatus 10 shown in FIG. 1 includes an input unit 11, an output unit 12, a storage unit 13, an image acquisition unit 14, an image analysis unit 15, an object extraction unit 16, and a non-visualization information correspondence processing unit 17. And a transmission / reception means 18 and a control means 19.

  The input means 11 is an input for starting / ending instructions for various processing such as image acquisition processing, image analysis processing, object extraction processing, invisible information handling processing, transmission / reception processing, etc., input of various setting information, etc. Accept. The input unit 11 includes a pointing device such as a keyboard and a mouse in a general-purpose computer such as a PC (Personal Computer), and includes operation buttons and a touch panel in a portable terminal or a game machine. The input unit 11 also has a function of inputting an image or video captured by an imaging unit such as a digital camera. Note that the above-described imaging unit may be provided in the object extraction device 10 or may have an external functional configuration.

  The output unit 12 outputs the content input by the input unit 11 and the content executed based on the input content. Specifically, the output unit 12 displays a screen display and a setting screen display of processing results in each component of the object extraction device 10 such as an acquired image, an image analysis result, an object extraction result, and a non-visualization information corresponding processing result. Audio output corresponding to the display content is performed. The output unit 12 includes a display, a speaker, and the like.

  Further, the output unit 12 may have a printing function such as a printer, and can print the above-described output contents on a printing medium such as paper and provide it to a user or the like.

  The storage unit 13 stores various information required in the present embodiment, various data at the time of execution of the object extraction process, and various data after execution. Specifically, the storage unit 13 stores, for example, one or a plurality of images acquired by the image acquisition unit 14. The storage unit 13 executes the present embodiment as a result of analysis by the image analysis unit 15, an object extracted by the object extraction unit 16, and a result of processing by the non-visualization information correspondence processing unit 17. Various setting information and the like are stored. Moreover, the memory | storage means 13 can read the various information memorize | stored as needed.

  The image acquisition unit 14 acquires an image or the like from which an object is extracted. The image is an image obtained by an imaging means such as a camera, for example, and the image is not limited to an image obtained from simple paper, and a poster, a photograph, a card, a sticker, a TV screen, etc. is photographed. Although the obtained image etc. are included, it is not limited to this. The image acquisition unit 14 can also acquire an image taken by an external device connected to the communication network via the transmission / reception unit 18, an image stored in an external database, or the like. It is also possible to acquire an image actually taken by a user or the like via a camera.

  The image analysis unit 15 analyzes the image acquired by the image acquisition unit 14 and analyzes the contents included in the image. Specifically, what part of the invisible information is embedded in which part (for example, position and area) in the image is analyzed. Note that the image analysis unit 15 performs a predetermined filtering process or the like on the image in order to extract the invisible region. Further, the image analysis means 15 analyzes the color information of the image (for example, which area is which color). In addition, as an example of the non-visualization information mentioned above, although there exists a high frequency pattern, for example, it is not limited to this. The high frequency will be described later.

  The object extraction unit 16 extracts an object included in the image based on the non-visualization information obtained by the image analysis unit 15. Specifically, the object extraction unit 16 extracts, for example, an area that does not include the invisible information as an object area. One or a plurality of object areas are included in the image, but all the areas can be extracted, but the present invention is not limited to this.

  For example, in the present embodiment, only the largest or smallest object among the objects included in the image may be extracted, and the object at a predetermined position (for example, the center or the right side) in the image is extracted. Alternatively, the shapes of the objects may be compared to extract an object having a shape close to a predetermined shape (for example, a rectangle or a circle). Note that what kind of object is to be extracted can be set in advance by a user, a software creator, or the like, and the set information is stored in the storage unit 13 and used when extracting the object.

  Further, when the object areas overlap, the object extraction means 16 may extract the whole as one object, for example, or based on the color information for each object analyzed by the image analysis means 15. Each object may be extracted based on a predetermined luminance difference.

  Further, the non-visualized information correspondence processing unit 17 performs predetermined processing on the image output by the output unit 12 or the like in correspondence with the contents (pattern or orientation) of the non-visualized information obtained by the image acquisition unit 14. I do. For example, the non-visualization information correspondence processing unit 17 generates an image in which a predetermined operation is performed on the object extracted by the object extraction unit 16 based on the contents of the surrounding non-visualization information, and the output unit 12 is displayed on the screen. The predetermined operation means, for example, at least one of movement (eg, up / down, left / right, etc.), rotation, enlargement / reduction, vertical vibration, horizontal vibration, swing, fade-in, and fade-out. However, the present invention is not limited to this. In the present embodiment, for example, processing such as changing the operation speed or changing the color of the object may be performed according to the operation pattern of the object. Further, when a plurality of operations are performed, different operations may be performed one by one successively or may be performed simultaneously. Further, the number of times of operation may be one or may be repeated a plurality of times.

  Further, the non-visualization information correspondence processing unit 17 may perform image processing on objects other than objects. For example, the non-visualization information correspondence processing unit 17 changes at least one of the background color and causes at least one of a movement, rotation, enlargement / reduction, vertical vibration, and horizontal vibration. Further, the non-visualized information correspondence processing means 17 may output a voice (for example, a speech of an object, a cry, a BGM, etc.) based on the non-visualized information, and a character at a predetermined position such as a telop or a balloon. Etc. may be displayed.

  Further, the invisible information correspondence processing means 17 may perform a predetermined operation corresponding to the background color of the position of the object, or perform a predetermined operation by combining the content of the above-described invisible information and the background color. Also good. Further, the non-visualization information correspondence processing unit 17 may perform the same processing on a plurality of objects based on the obtained non-visualization information, or may perform image processing in which a plurality of objects are linked. The linked image processing includes, for example, processing in which objects move to the center and collide with each other, display / non-display is alternately switched, and rotation speed is alternately changed, but is not limited thereto. is not.

  In this embodiment, when an object is operated, the processing operation is performed in accordance with the contents of the invisible information around the object. However, the present invention is not limited to this, and for example, in the image The object may be caused to perform a predetermined action in correspondence with a combination of the number and type of objects.

  When the object is moved with respect to the background image, a background area that has not been displayed by the object until now is displayed. Based on the color information or the like of the background image that has been displayed around the area, the background area is interpolated. Therefore, in this embodiment, when an object on a light blue background is moved, light blue is displayed by the above-described interpolation processing for the background area that is not displayed by the object.

  The transmission / reception means 18 is an interface for acquiring a photographed image and the like, an execution program for realizing the object extraction processing in the present embodiment, and the like from an external device that can be connected using a communication network or the like. The transmission / reception means 18 can transmit various information such as images and processing results obtained in the object extraction device 10 to an external device.

  The control means 19 controls the entire components of the object extraction device 10. Specifically, the control unit 19 performs control of each process such as image acquisition, image analysis, object extraction, and invisible information handling process based on an instruction from the input unit 11 by a user or the like.

  In the present embodiment, an object can be extracted with high accuracy based on the invisible information embedded in the image described above. Further, in the present embodiment, it is possible to provide a highly accurate image with excellent added value with respect to an acquired object or the like using the invisible information.

<Object Extraction Device 10: Hardware Configuration>
Here, in the object extraction device 10 described above, an execution program (object extraction program) capable of causing a computer to execute each function is generated, and for example, a general-purpose personal computer, a server, or a portable terminal (for example, a smartphone or a mobile phone) The object extraction process and the invisible information recognition process in the present invention can be realized by installing the execution program in a telephone, a notebook PC, a tablet terminal, etc.), a game machine or the like. Here, a hardware configuration example of a computer capable of realizing the object extraction processing in the present embodiment will be described with reference to the drawings.

  FIG. 2 is a diagram illustrating an example of a hardware configuration capable of realizing the object extraction process according to the present embodiment. 2 includes an input device 21, an output device 22, a drive device 23, an auxiliary storage device 24, a memory device 25, a CPU (Central Processing Unit) 26 for performing various controls, and a network connection device. 27, and these are connected to each other by a system bus B.

  The input device 21 has a pointing device such as a keyboard and a mouse operated by a user or the like, and inputs various operation signals such as execution of a program from the user or the like. The input device 21 may include an image input unit that inputs an image taken from an imaging unit such as a camera.

  The output device 22 has a display for displaying various windows and data necessary for operating the computer main body for performing the processing according to the present invention, and displays the program execution progress, results, and the like by the control program of the CPU 26. can do.

  Here, the execution program installed in the computer main body in the present invention is provided by, for example, a portable recording medium 28 such as a USB (Universal Serial Bus) memory or a CD-ROM. The recording medium 28 on which the program is recorded can be set in the drive device 23, and the execution program included in the recording medium 28 is installed in the auxiliary storage device 24 from the recording medium 28 via the drive device 23.

  The auxiliary storage device 24 is a storage means such as a hard disk, and stores an execution program according to the present invention, a control program provided in a computer, and the like, and can perform input / output as necessary.

  The memory device 25 stores an execution program read from the auxiliary storage device 24 by the CPU 26. The memory device 25 includes a ROM (Read Only Memory), a RAM (Random Access Memory), and the like.

  The CPU 26 controls processing of the entire computer, such as various operations and input / output of data with each hardware component, based on a control program such as an OS (Operating System) and an execution program stored in the memory device 25. Thus, each process in the object extraction can be realized. Various information necessary during the execution of the program can be acquired from the auxiliary storage device 24, and an execution result or the like can be stored.

  The network connection device 27 acquires an execution program from another terminal connected to the communication network by connecting to a communication network or the like, or an execution result obtained by executing the program or an execution in the present invention The program itself can be provided to other terminals.

  With the hardware configuration as described above, the object extraction processing according to the present invention can be executed. Also, by installing the program, the object extraction processing in the present invention can be easily realized by a general-purpose personal computer or the like.

  Next, the object extraction process will be specifically described.

<Object extraction processing procedure>
First, the object extraction processing procedure in this embodiment will be described. FIG. 3 is a flowchart showing an example of an object extraction processing procedure in the present embodiment.

  In the object extraction process shown in FIG. 3, first, an image to be extracted from an object photographed by an imaging means such as a camera is read (S01), and filtering is performed on the read image to extract a non-visible region ( (S02), an object is extracted from the result (S03). In the process of S02, a low-pass filter may be applied to the image acquired in S01, or a difference image using a predetermined image may be extracted from the image acquired in S01. However, the present invention is not limited to this. is not. In the process of S03, a part (region) without the invisible information is extracted as an object.

  Next, the object extraction processing determines the pattern (eg, high frequency pattern) of the invisible information included in the image from the filtering result obtained in S02 (S04), and further determines the frequency content of the high frequency pattern. (S05). Here, the frequency of the high frequency pattern is, for example, whether the content (for example, pattern) formed by the high frequency is a stripe such as a horizontal stripe or a vertical stripe, an oblique stripe pattern, or a checkered pattern. Information is determined as a frequency pattern. Further, the determination in S05 also determines area information such as which pattern is in which area in the image.

  Next, the object extraction process determines an object motion pattern corresponding to the frequency determination result of the high-frequency pattern obtained in S05 (S06). In the process of S06, for example, a predetermined motion pattern corresponding to the high frequency pattern is determined for the object surrounded by the high frequency pattern (invisible information) from which the frequency determination result is obtained. The operation pattern corresponding to the frequency determination result (high frequency pattern) is set in advance by, for example, a software creator or a user, and the operation pattern for the frequency determination result is determined with reference to the data.

  In the object extraction process, the operation speed corresponding to the operation pattern is determined based on the frequency determination result (S07), and the image is displayed based on the determined content (S08).

  In the above example, the high-frequency pattern is used as the invisible information, but the present invention is not limited to this. In the above-described example, the object included in the image is operated based on the invisible information. However, the present embodiment is not limited to this. For example, the background color is changed or the printing process is performed. It is also possible to perform various controls such as printing, printing only the object at the time of printing, or not printing the object. Further, in the present embodiment, the background color information is acquired together with the filtering process in the above-described process of S02, and the process of determining the operation pattern of the object in the process of S06 by combining the acquired color information and the invisible information. You may go.

<About high frequency>
Here, the high frequency in this embodiment mentioned above is demonstrated. Usually, the frequency includes a frequency for time (temporal frequency) and a frequency for spatial position (spatial frequency). In the present embodiment, the frequency means a spatial frequency unless otherwise specified. The spatial frequency is defined as “the reciprocal of the period of the pixel value with respect to the unit length”.

  That is, the high frequency in the present embodiment means a region where the bright part and the dark part are periodically repeated. Although the frequency in this embodiment is not specifically limited, For example, although it is set in the range of 0.2-2 [cycle / pixel] in a high frequency part, it is not limited to this.

  For example, a predetermined pixel region (for example, 4 × 4 pixels) formed in a high-frequency part only needs to periodically repeat a bright part and a dark part, for example, vertical stripes, horizontal stripes, checkered pattern, etc. It is done. In addition, the brightness difference between the bright part and the dark part at that time may be 10 or more, preferably 50 or more, and more preferably 100 or more. The brightness difference may be set in advance as a fixed value, and can be arbitrarily changed according to the background color, the content of the object, and the like. Therefore, for example, the increase / decrease value of the brightness can be determined by determining from the brightness of the background. Thereby, in this embodiment, more appropriate de-visualization information can be embedded according to the background color, the content of the object, or the like.

  Here, the high frequency pattern embedded in the background image will be described. In the high-frequency pattern, a predetermined pattern is generated in the background image (grid) region. For example, when a horizontal striped high-frequency pattern is formed, the lightness of the even rows of the grid is high and the lightness of the odd rows is low.

<Specific examples of high-frequency patterns>
Next, a specific example of the high frequency pattern will be described. FIG. 4 is a diagram illustrating an example of a high-frequency pattern. In this embodiment, for example, invisible information including a plurality of preset high-frequency patterns (patterns) is embedded in the background image. That is, in this embodiment, for example, at least one of the patterns shown in FIGS. 4A to 4E is embedded at a predetermined position on the background. 4A is a high frequency pattern of horizontal stripes, FIG. 4B is a high frequency pattern of vertical and horizontal stripes, FIG. 4C is a high frequency pattern of right lower diagonal stripes, and FIG. ) Is the high-frequency pattern of the lower left diagonal stripe, and FIG. 4E is a checkered high-frequency pattern. These high-frequency patterns are set based on, for example, the background color and background content (for example, patterns, landscapes, buildings, people, etc.), the content operated by the object, and the like.

  Therefore, in this embodiment, when extracting an object, the object can be extracted based on the high-frequency pattern, and can be displayed with a predetermined action on the object.

  In the present embodiment, the high-frequency pattern is embedded in the background image. However, the present invention is not limited to this. For example, the above-described high-frequency pattern may be embedded only in the object region. In this case, object extraction can be realized by extracting a region having a high-frequency pattern. However, it is not preferable that a high-frequency pattern is embedded in the object area although it is invisible information. Therefore, in such a case, it is preferable to display the object after extracting the object and then performing a process of deleting the high frequency pattern in the object area.

<About filtering processing>
Next, the filtering process in the process of S02 described above will be specifically described. The filtering processing in the present embodiment applies, for example, all the low-pass filters such as a preset vertical sobel filter, horizontal sobel filter, checkered pattern detection filter, and the number of feature points obtained as a result (for example, The high frequency pattern is determined from the area and position of the high frequency portion.

  Here, FIG. 5 is a diagram for explaining the contents of filtering. 5A shows an example of filter application using a vertical stripe image, FIG. 5B shows an example of filter application using a horizontal stripe image, and FIG. 5C shows an example of filter application using a lower left diagonal stripe image. Show.

  Further, in FIGS. 5A to 5C, the upper row shows an example of each stripe image embedded in the background, the middle row shows an example of an image after applying the lateral sobel filter as an example, and the lower row shows an example. The example of an image after applying a vertical direction Sobel filter is shown.

  In the filtering process according to the present embodiment, when filtering is performed with the same stripe, a whitish image in which the stripe is not conspicuous is obtained. Therefore, in the example of FIG. 5, the high frequency pattern disappears when the vertical direction Sobel filter is applied to the vertical stripe image (the lower part of FIG. 5A), and the high frequency pattern is applied to the horizontal stripe image when the horizontal direction Sobel filter is applied. It disappears (middle of FIG. 5B). Further, when a high frequency pattern remains even when the vertical direction sobel filter and the horizontal direction sobel filter are applied, as shown in FIG. 5C, it can be determined that the original image is a region of an oblique stripe image. .

  In the above example, the determination example of each of the vertical, horizontal, and lower left diagonal stripes is shown. However, the present invention is not limited to this, and includes, for example, the lower right diagonal stripes and the checkered pattern. In this case, by using a Sobel filter matched to the pattern, as described above, it is possible to perform pattern determination with high accuracy by acquiring an image in which the high-frequency pattern has disappeared. Note that the content of the filtering process is not limited to this.

  Here, in the present embodiment, for example, when a high-frequency image having a vertical stripe in the upper half and a horizontal stripe in the lower half is used, the following processing is performed.

  First, for example, all three types of filters shown in FIGS. 5A to 5C are applied to a target image (captured image) from which an object is extracted. Of these three trials, many feature points are obtained in the upper half of the image only when the vertical sobel filter is applied. The filter application result is obtained as a black and white image, and only the feature point portion is expressed in white and the other portions are expressed in black. Therefore, in this case, an image in which the upper half of the image is white and the lower half is black is obtained.

  Thus, in the present embodiment, it is possible to recognize where and what high-frequency pattern is embedded based on the areas (regions) of the white portion and the black portion. Further, in this case, since many feature points are obtained in the upper half of the result image using the vertical direction Sobel filter, it can be recognized that the upper half vertical stripe of the captured image is embedded. Similarly, when a feature point is obtained in the lower half of the image only when the horizontal direction Sobel filter is applied, it can be recognized that a horizontal stripe is embedded in the upper half of the image.

<Determination of high frequency pattern>
For the determination of the high-frequency pattern in the above-described processing of S04, for example, the combination of position information such as which stripe direction (vertical, horizontal, or diagonal) is in which region and how many types of high-frequency patterns are combined. For example, even on the same high frequency pattern, the illustration on the left side can be operated faster, and the illustration on the right side can be operated more slowly using the information. .

<Object Extraction Result When Conventional Method and This Embodiment are Applied>
Next, object extraction results when the conventional method and the present embodiment are applied will be described with reference to the drawings. FIG. 6 is a diagram illustrating an example of an object extraction result when the conventional method and the present embodiment are applied. 6A shows an example in which an object (USB memory in the example of FIG. 6) is extracted by a conventional technique, and FIG. 6B shows an example of object extraction when the present embodiment is applied. ing.

  The example illustrated in FIG. 6A illustrates an example in which the object 31-1 included in the image 30-1 is extracted using only the blue background. In this case, when the object has the same color as the blue background, the error rate is 74.4%. The error rate was defined as “the number of pixels that were not cut × 100 / the number of pixels in the target object to be cut”.

  However, in the present embodiment shown in FIG. 6B, since a blue background in which a high-frequency pattern is embedded in the background image 30-2 is used, the object 31-2 extracted by the above-described method has an error rate of 3. It was possible to extract at a very low value of .8%.

<Example of image and its operation>
Here, an example of an image applied to this embodiment and its operation will be described. FIG. 7 is a diagram for explaining an example of an image and an example of the operation of an object in the present embodiment. Note that the example of FIG. 7 shows an operation example of an object included in an image displayed on a screen such as a PC.

  In the image 40 shown in FIG. 7A, as an example, the upper half background image 41-1 shows the sky (light blue), and the lower half background image 41-2 shows the ground (green). Also, vertical stripes are embedded in the upper half light blue portion of the image 40, and horizontal stripes are embedded in the lower half green portion.

  In such an image 40, it is assumed that the user has drawn a white cloud object 42-1 in the light blue portion of the image 40 and a yellow bird object 42-2 in the green portion of the image 40, for example. In this case, for example, the cloud object 42-1 is repeatedly enlarged and reduced at a predetermined magnification, and the bird object 42-2 goes straight (moves) in a predetermined direction by the object extraction process and the invisible information handling process described above. ) Can be performed.

  Specifically, in the image 40 on the screen, as shown in FIG. 7B, the cloud object 42-1 is displayed at the same position while being repeatedly enlarged and reduced at a predetermined magnification at predetermined time intervals. . The bird object 42-2 moves to the left by a predetermined distance at a preset speed. The moving speed may be constant or variable, or a combination thereof. Further, the movement distance is not limited to the movement of a predetermined distance, and for example, the bird object 42-2 may move such that it moves out of the frame of the image 40, and faces the opposite direction at the end of the image 40. Then, a process of moving to the right side may be performed. In the example of FIG. 7B, the afterimage of the object 42 is displayed on the background image 41 for convenience of describing the positional relationship of the object 42 before and after the movement. However, in the present embodiment, the present invention is not limited to this. Instead, for example, a background image without an afterimage interpolated using surrounding color information excluding the object 42 may be displayed.

  In the present embodiment, the number (type) of high-frequency patterns (non-visualization information) embedded in the background can be set in correspondence with the size (size) of paper or screen corresponding to the background, for example. For example, in the case of A4 size paper (sheet), a maximum of four patterns can be combined. This is because, in drawing software as an application, if the number of non-visualization information patterns is more than 4 patterns, the area of each non-visualization information with respect to the entire background becomes narrow, and each picture (object) is narrower. This is because it has to be written on. In the present embodiment, the present invention is not limited to this, and the user may arbitrarily set the number of patterns (type).

  Further, although it is preferable to draw the object so as not to straddle different high-frequency patterns in the background, the present invention is not limited to this. For example, when a plurality of high-frequency patterns are straddled, the operation processing for the object is not performed, the operation set for each high-frequency pattern is performed, or a new operation combining the high-frequency patterns can be performed. . In the present embodiment, objects other than the high-frequency pattern are used as objects even if they extend over a plurality of high-frequency patterns, so that objects can be extracted.

  Such a plurality of high-frequency patterns are embedded at the time of creating a background sheet, for example. For example, when the creator of the background sheet wants to change the operation between the left half illustration (object) and the right half illustration (object), different high frequency patterns are embedded in the left half and the right half. This work may be performed manually by the sheet creator, or embedding can be automated by a program. In this case, for example, by selecting a high frequency pattern embedding region and an operation pattern on a PC, the corresponding high frequency pattern is automatically embedded in the selected region.

  In addition, regarding the operation after object extraction in the present embodiment, the operation content can be changed depending on the shape, size, color, etc. of birds and clouds even with the same high-frequency pattern (for example, stripes). Further, even if the pattern such as a stripe is the same, if the background color changes, different operations can be performed depending on the relationship between the stripe and the background color. For example, blue vertical stripes can be processed such as enlargement / reduction, and red vertical stripes can be processed such as rotation. In this case, a determination target for giving an action to a picture (object) drawn by the user includes not only stripes but also colors.

  In this embodiment, for example, an image including an object drawn by a user (for example, FIG. 7A) is printed on a paper medium or the like, and the printed paper medium is photographed by a camera (imaging means) or the like. The object motion as shown in FIG. 7B can also be performed on the obtained image. That is, in the present embodiment, an object can be appropriately extracted even for an image to which no meta information about the object is added, such as an image printed on a print medium such as paper. In the present embodiment, a predetermined operation can be performed on the extracted object or the like.

  As described above, according to the present invention, an object can be extracted with high accuracy. In addition, it is possible to efficiently obtain information using non-visualized information and provide a highly accurate image with excellent added value.

  The preferred embodiments of the present invention have been described in detail above, but the present invention is not limited to such specific embodiments, and various modifications, within the scope of the gist of the present invention described in the claims, It can be changed.

DESCRIPTION OF SYMBOLS 10 Object extraction apparatus 11 Input means 12 Output means 13 Storage means 14 Image acquisition means 15 Image analysis means 16 Object extraction means 17 Invisible information corresponding processing means 18 Transmission / reception means 19 Control means 21 Input device 22 Output device 23 Drive device 24 Auxiliary storage Device 25 Memory device 26 CPU (Central Processing Unit)
27 Network connection device 28 Recording medium 30, 40 Image 31, 42 Object 41 Background image

Claims (11)

In an object extraction device that extracts an object included in an acquired image,
Image analysis means for analyzing the image and obtaining the position and content of the invisible information;
Object extraction means for extracting an object contained in the image based on the non-visualization information obtained by the image analysis means;
Non-visualization information corresponding processing means for performing predetermined processing on the object extracted by the object extraction means in correspondence with the pattern of the non-visualization information;
An object extraction apparatus comprising: output means for outputting an image obtained by the non-visualization information correspondence processing means.   The object extracting apparatus according to claim 1, wherein the invisible information is a high frequency pattern embedded in either a background image or the object. The image analysis means includes
The filtering of the image is performed using a plurality of preset filters, and the pattern of the invisible information embedded in the image is acquired from the relationship between the filtered images. The object extraction device described in 1. The invisible information corresponding processing means is
4. The object extraction device according to claim 1, wherein a predetermined operation is performed on an object surrounded by the non-visualization information based on the pattern of the non-visualization information. 5. The invisible information corresponding processing means is
5. The object extraction device according to claim 1, wherein one or a plurality of operations are performed on an object based on contents of a plurality of pieces of non-visualization information obtained by the image analysis unit. In an object extraction method for extracting an object included in an acquired image,
Image analysis step for analyzing the image and obtaining the position and content of the invisible information;
An object extraction step of extracting an object included in the image based on the non-visualization information obtained by the image analysis step;
A non-visualization information corresponding processing step for performing a predetermined process on the object extracted by the object extraction step corresponding to the pattern of the non-visualization information;
And an output step of outputting an image obtained by the non-visualization information corresponding processing step.   The object extraction method according to claim 6, wherein the non-visualization information is a high-frequency pattern embedded in either a background image or the object. The image analysis step includes
The filtering of the image is performed using a plurality of preset filters, and the pattern of the invisible information embedded in the image is acquired from the relationship between the filtered images. The object extraction method described in 1. The invisible information corresponding processing step includes:
The object extraction method according to claim 6, wherein a predetermined operation is performed on an object surrounded by the non-visualization information based on the pattern of the non-visualization information. The invisible information corresponding processing step includes:
10. The object extraction method according to claim 6, wherein one or a plurality of operations are performed on an object based on the contents of a plurality of pieces of non-visualization information obtained by the image analysis step.   An object extraction program for causing a computer to function as each unit included in the object extraction device according to any one of claims 1 to 5.