JP2012138892A - Content providing system using invisible information, invisible information embedding device, invisible information recognition device, invisible information embedding method, invisible information recognition method, invisible information embedding program, and invisible information recognition program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To speedily embed and extract invisible information and to provide an accurate content having excellent added value.SOLUTION: An embedding device which embeds the invisible information in a prescribed position of an acquired image has: image analysis means that acquires object and position information included in the image; embedding target object setting means that sets an object into which the image is to be embedded from the object acquired by the image analysis means; and invisible information embedding means which embeds the invisible information corresponding to the object into the surrounding of the object obtained by the embedding target object setting means.

Description

  The present invention relates to a content providing system using invisible information, an invisible information embedding device, a recognition device, an embedding method, a recognition method, an embedding program, and a recognition program, and in particular, embedding and extracting invisible information. Content providing system using non-visualized information, non-visualized information embedding device, recognizing device, embedding method, recognizing method, embedding, for providing high-accuracy content services with high added value. Related programs and recognition programs.

  Conventionally, with the widespread use of mobile terminals with cameras, various digital contents using still images, moving images, and the like have been provided. Conventionally, a two-dimensional code such as QR code (registered trademark) printed on a print medium such as paper is read using a camera such as a portable terminal, and a content site obtained from the read two-dimensional code is accessed. Or a message corresponding to the content can be displayed.

  However, the two-dimensional code does not look good, and a space for arranging the two-dimensional code is also required. Therefore, in recent years, a technique has been disclosed in which information is acquired from an image of content without using a two-dimensional code, and corresponding processing is performed based on the acquired information (for example, Patent Document 1 and Patent Document 2). reference).

  The technique disclosed in Patent Document 1 acquires instruction information regarding image attributes as a condition for extracting a predetermined area included in an image, and extracts an area from the image based on the acquired instruction information. An image processing apparatus including an area specifying unit for specifying the image is shown. That is, Patent Document 1 discloses a technique for performing separation from a background image using a feature amount in an image.

  In addition, the technique disclosed in Patent Document 2 includes an input unit that inputs image data, an encoding unit that encodes image data to generate encoded data, and extracts an object of arbitrary shape from the image data. There is shown an image processing apparatus including shape data generating means for generating the shape data and an embedding means for embedding the shape data into encoded data by digital watermarking. Japanese Patent Application Laid-Open No. 2004-228561 discloses a technique in which a digital watermark is embedded in an image and a boundary between a place where the digital watermark is embedded and a place where the digital watermark is not embedded is used as background separation information.

JP 2006-13722 A JP 2004-80096 A

  However, for example, in the technique disclosed in Patent Document 1, since the feature image content is used to separate the background image, the background image and the object color are the same or similar. Cannot separate an object at an accurate position from a feature amount in an image.

  In the technique disclosed in Patent Document 2, when embedding a two-dimensional code or the like in an image, the original image is converted into the frequency domain, the two-dimensional code is embedded, and then further converted into the spatial domain and embedded. Need to do it. Further, when reading a two-dimensional code, it is necessary to convert the input image to the frequency domain, read the code, and then convert to the spatial domain to extract the code. Therefore, when a code is embedded or read, a lot of image processing occurs, which is not suitable for processing in a low-spec device such as a portable terminal or real-time processing.

  The present invention has been made in view of the above-described problems, and uses non-visualized information for quickly embedding and extracting non-visualized information and providing a highly accurate content service with excellent added value. It is an object of the present invention to provide a content providing system, an invisible information embedding device, a recognition device, an embedding method, a recognition method, an embedding program, and a recognition program.

  In order to solve the above-described problems, the present invention employs means for solving the problems having the following characteristics.

  The present invention comprises a non-visualization information embedding device for embedding non-visualization information at a predetermined position of an acquired image, and a recognition device for recognizing an object and non-visualization information included in the image obtained by the embedding device. In the content providing system using non-visualization information, the embedding device includes an embedding target object setting unit that sets an object in which the non-visualization information is embedded from an object included in the acquired image, and the embedding target object setting unit. And a non-visualization information embedding unit that embeds the non-visualization information corresponding to the object around the object obtained by the step, and the recognition device extracts an object from an embedding region of the non-visualization information included in the image And an object extracting means for performing the object extraction by the object extracting means. When the image is extracted, a non-visualization information analysis unit that analyzes the processing content for the object is generated from the de-visualization information, and an object that is displayed on the screen in correspondence with the processing content obtained by the non-visualization information analysis unit And a display information generating means.

  According to the present invention, in an embedding device that embeds invisible information at a predetermined position of an acquired image, an image analysis unit that acquires an object and position information included in the image, and an object obtained by the image analysis unit An embedding target object setting means for setting an object to be embedded in the image; and an invisible information embedding means for embedding the invisible information corresponding to the object around the object obtained by the embedding target object setting means. It is characterized by having.

  Further, the present invention provides a recognition apparatus for recognizing an object and non-visualization information included in an acquired image, by an object extraction unit that extracts an object from an embedded region of the non-visualization information included in the image, and the object extraction unit. When the object is extracted, a non-visualization information analysis unit that analyzes the processing content of the object from the non-visualization information, and an object that is displayed on the screen in correspondence with the processing content obtained by the non-visualization information analysis unit. Display information generating means for generating.

  Further, the present invention is an embedding program for causing a computer to function as each unit included in the embedding device described above. Furthermore, the present invention is a recognition program for causing a computer to function as each unit included in the above-described recognition device.

  According to the present invention, it is possible to quickly embed and extract invisible information and provide a highly accurate content service excellent in added value.

It is a figure which shows an example of schematic structure of the content provision system in this embodiment. It is a figure which shows an example of a function structure of the embedding apparatus in this embodiment. It is a figure which shows an example of a function structure of the recognition apparatus in this embodiment. It is a figure which shows an example of the hardware constitutions which can implement | achieve the embedding process and recognition process of non-visualization information in this embodiment. It is a flowchart which shows an example of the embedding process procedure of the invisible information in this embodiment. It is a flowchart which shows an example of the recognition process procedure of the invisible information in this embodiment. It is a figure for demonstrating the example of embedding invisible information in this embodiment. It is a figure for demonstrating the extraction of the invisible information in this embodiment, and the recognition example of an object. It is a figure for demonstrating the example of object operation | movement in this embodiment. It is a figure for demonstrating the invisible information which has a high frequency part and a low frequency part. It is a figure which shows the relationship between embedding strength and the gray scale value of a background. It is a figure which shows the example of the non-visualization code | symbol for monochrome images. It is a figure which shows the relationship between the read recognition precision with respect to the brightness which changed the gray image from 0 to 255, and non-visualization. It is a figure for demonstrating evaluation of the read recognition accuracy of the invisible information for every image in this embodiment. It is a figure which shows the evaluation result corresponding to FIG. 1 is a diagram illustrating an example of an AR application system in Embodiment 1. FIG. It is a figure which shows the operation example of the windmill by the acquired trimming marker. It is a figure for demonstrating the relationship between the non-visualization information contained in an image, and operation | movement instruction information. It is a figure which shows an example of the AR application system in Example 2. FIG. FIG. 10 illustrates an example of an AR application system according to a third embodiment. FIG. 10 is a diagram illustrating an example of an AR application system according to a fourth embodiment.

<About the present invention>
In the present invention, for example, an image or video displayed on a screen or a photo, a postcard, a poster, a card, or the like made of various print media such as paper or film is made invisible information by processing that cannot be recognized with the naked eye ( Marker) is embedded around an object (object information) included in the image, and the image or video is captured by a photographing device such as a digital camera or a camera provided in a mobile terminal (mobile phone, smartphone (registered trademark), etc.) Recognize the embedded marker by capturing a part of a print medium, etc., and capturing the captured image or video into a tablet PC such as a personal computer, mobile terminal, or ipad (registered trademark). And extract objects.

  In addition, the present invention enables marker recognition that combines image processing with a marker embedding method that enables recognition of an embedded marker even in a model with limited capacity and performance such as a portable terminal. .

  Further, in the present invention, for example, embedded information for operating the object is acquired from the recognized marker information, and the extracted target object is processed based on the acquired embedded information. In addition, the present invention provides a content providing system that performs a series of operations from embedding and extracting non-visualized information to processing contents for an object in an image.

  The non-visualized information may be a state where the retina of the user's eyes cannot be sensed, or information where the retina of the user's eyes can be sensed but the user's brain cannot identify it as information. Etc.

  The non-visualization information may be a high-frequency part or a low-frequency part in which the brightness (brightness difference) of each pixel in the image is adjusted, or may be character information that can be directly recognized, It may be a symbol, number, mark, pattern, color, one-dimensional code, two-dimensional code, or the like, or a combination thereof. As the code information, for example, a two-dimensional barcode such as a QR code can be used. In the present embodiment, the code information is not limited to the QR code. For example, JAN code, ITF code, NW-7, CODE39, CODE128, UPC, PDF417, CODE49, Data Matrix, Maxi Code, etc. Bar code information or the like can also be used.

  FIG. 1 is a diagram showing a content providing system using non-visualization information, a non-visualization information embedding device, a recognition device, an embedding method, a recognition method, an embedding program, and a recognition program according to the present invention. Will be described. The “image” described in the present embodiment includes one image such as a photograph and an image of continuous frames in a video. The content providing system in the present embodiment is, for example, an AR (Augmented Reality) system, but is not limited to this in the present invention, and is applied to other content services in general. The

<Content provision system using visualization information>
First, an outline of a content providing system using visualization information in the present embodiment will be described with reference to the drawings. FIG. 1 is a diagram illustrating an example of a schematic configuration of a content providing system according to the present embodiment. The content providing system 10 shown in FIG. 1 includes an input first print medium 11, a first image information acquisition device 12, an embedding device 13, a second print medium 14, and second image information. An acquisition device 15 and a recognition device 16 are included.

  The first print medium 11 is printed with an image before the invisible information is embedded. In the content providing system 10, a plurality of first print media 11 (in the example of FIG. 1, the first print media 11-1 and 11-2) can be used. In the present embodiment, as an example, the second print medium 14 in which the invisible information is embedded is generated with respect to the first print medium 11 using the embedding device 13. In the present invention, the present invention is not limited to this. For example, the image is directly acquired, the invisible information is embedded in the acquired image, and the embedded image is printed to generate the second print medium 14. Also good.

  In the present embodiment, the image or the like printed on the first print medium 11 includes an object. In the present invention, an object means object information for operating based on information included in non-visualized information or displaying related information, for example, a human being or an animal (face, hand, foot, etc.) ), Insects, flowers, trees, vehicles, desks, chairs, hats, glasses, shoes, bags, etc., and other objects that can be separated from the background.

  The image information acquisition device 12 captures the first print medium 11 that does not include the invisible information, and acquires it as image information. As the first image information acquisition unit 12, for example, a camera (imaging unit) 12-1, a scanner (information reading device) 12-2, or the like can be used, but the present invention is not limited to this. For example, a general apparatus configuration having a function capable of acquiring image information such as a copying machine, a portable terminal with a camera function, a game machine, a tablet PC (Personal Computer), or the like can be used.

  The embedding device 13 selects an object to be subjected to processing such as trimming in the recognition device 16 for the image data input from the first print medium 11 or the like, and is set in advance around the object. Invisible information is embedded, and the embedded image is output.

  The embedding device 13 may output the image after embedding the invisible information as a print medium (second print medium 14), or may output the image as it is.

  As the embedding device 13, for example, a general-purpose PC 13-1, a portable terminal 13-2, or the like can be used. However, the present invention is not limited to this. Further, the portable terminal 13-2 is provided with a built-in camera 12-3 as the first image information acquisition unit 12, whereby the image data of the first print medium 11 can be directly acquired. A specific functional configuration of the embedding device 13 will be described later.

  The second print medium 14 is a print medium in which the invisible information in the present embodiment is embedded. The second print medium 14 is embedded with invisibility information for performing, for example, trimming or a predetermined operation on a predetermined object.

  The second image information acquisition unit 15 acquires the image data of the second print medium 14 in the same manner as the first image information acquisition unit 12 described above. As the second image information acquisition means 15, for example, a camera 15-1, a scanner 15-2 or the like can be used. However, the present invention is not limited to this, and for example, a copying machine In general, a device having a function capable of acquiring image information, such as a portable terminal with a camera function, a game machine, or a tablet PC, can be used.

  The recognition device 16 recognizes the non-visualization information included in the image using the image data obtained from the second image information acquisition unit 15, and trims a predetermined object included in the image data from the recognized non-visualization information. . Note that trimming refers to, for example, extracting only a target object (including a rough outer frame of an object) by removing unnecessary portions from the image.

  Further, the recognition unit 16 causes the extracted object to perform an operation corresponding to the instruction content set in advance or included in the non-visualization information. The processing contents for the object include, for example, moving or rotating the object on the screen of the recognition device 16, enlarging or reducing, replacing with another object, displaying other different images or characters, outputting sound, Related information is displayed, and combinations thereof are also performed.

  In addition, the recognition device 16 can execute processing on the target object based on secondary instruction information by a user or the like without immediately performing an operation on the extracted object. As secondary instruction information, for example, wind pressure (for example, blowing a voice on a voice input means such as a microphone or actually blowing a wind) or sound (voice, music, specific) Words (for example, spells and shouts)), light (for example, hitting the sun), device rotation (for example, rotating, shaking, keeping horizontal or vertical), pressure (Including touching and dragging the screen), and other information obtained by environmental conditions such as temperature, time, location, weather, etc., and some of these information may be combined. The final instruction information may be set according to the instruction order of a plurality of secondary instruction information (for example, a call after rotation), a priority set in advance for each instruction information, or the like.

  The recognition device 16 may be a general-purpose PC 16-1, a portable terminal 16-2, or the like, for example. However, the present invention is not limited to this. Further, the portable terminal 16-2 is provided with a built-in camera 15-3 as the second image information acquisition unit 15, whereby the image data of the second print medium 14 can be directly acquired. A specific functional configuration of the recognition device 16 will be described later.

<Embedding device 13: functional configuration example>
Here, a functional configuration example of the embedding device 13 in the above-described embodiment will be described with reference to the drawings. FIG. 2 is a diagram illustrating an example of a functional configuration of the embedding device according to the present embodiment.

  The embedding device 13 shown in FIG. 1 includes an input unit 21, an output unit 22, a storage unit 23, an image acquisition unit 24, an image analysis unit 25, an embedding target object setting unit 26, and an embedding information setting. Means 27, embedded information generating means 28, invisible information embedding means 29, transmission / reception means 30, and control means 31 are provided.

  The input means 21 includes various instructions such as an image acquisition instruction, an image analysis instruction, an embedding target object setting instruction, an embedding information setting instruction, an embedding information generation instruction, a non-visualization information embedding instruction, and a transmission / reception instruction from the user. Accepts input such as start / end of. The input means 21 is composed of a pointing device such as a keyboard and a mouse if it is a general-purpose computer such as a PC, and comprises a group of operation buttons if it is a portable terminal or the like.

  Further, the input unit 21 may have a function of inputting an image or video captured by an imaging unit such as a digital camera. In that case, the imaging means may be built in the embedding device 13 or may have an external functional configuration. Furthermore, the input means 21 may have a voice input function for inputting voice or the like.

  The output unit 22 outputs the content input by the input unit 21 and the content executed based on the input content. Specifically, the output unit 22 embeds (synthesizes) the acquired image, the image analysis result, the embedding target object setting result, the set invisibility information, the generated invisibility information, and the invisibility information. E) Screen display of the embedded image and the result of processing in each configuration, audio output, and the like. Note that the output means 22 includes a display, a speaker, and the like.

  Further, the output means 22 may have a printing function such as a printer, and prints the above-described output contents on various print media such as paper, postcards, posters, etc., and the above-described second print medium. 14 can be provided to a user or the like.

  The storage unit 23 stores various information required in the present embodiment and various data at the time of execution or after execution of the embedding process. Specifically, the storage unit 23 acquires one or a plurality of images or videos obtained by input or photographing acquired by the image acquisition unit 24 stored in advance. In addition, the storage unit 23 stores the result analyzed by the image analysis unit 25, the determination result by the embedding target object setting unit 26, the setting content by the embedding information setting unit 27, and the embedding information generation unit 28. Embedded information, images embedded by the non-visualization information embedding unit 29, and the like are stored. Further, the storage means 23 can read out various data stored as required.

  The image acquisition unit 24 acquires an image, a video, or the like including an object to be embedded with the invisible information. The images and videos may be images and videos obtained by imaging means such as a camera, for example, catalogs, brochures, posters, photos, cards, stickers, magazines and other books, and product packaging (including boxes). Or a target image applied to a manual or the like. The image acquisition unit 24 can also acquire information and images stored in a database or the like captured by an external device connected to the communication network via the transmission / reception unit 30, and input You may use the image which the user etc. actually image | photographed with the camera etc. via the means 21. FIG.

  The image analysis unit 25 analyzes the image acquired by the image acquisition unit 24 and analyzes the contents included in the image. Specifically, an object made up of object information, coordinates, etc., such as what part (position, region) in the image is projected, how the object is moving in the video, etc. Get location information.

  The image analyzing unit 25, when acquiring an object, for example, stores a plurality of object images set in advance in the storage unit 23, performs pattern matching using the object image, and extracts the object. May be. In addition, for example, when the object is a person, the image analysis unit 25 may perform face detection from the facial feature portion of the person, digitize the facial feature amount, and perform person identification based on the result. Also good.

  Furthermore, the image analysis means 25 can also extract an object using the outline (outer frame) information of the object designated by the user or the like using the input means 21 or the like.

  Based on the result analyzed by the image analysis unit 25, the embedding target object setting unit 26 includes an object when the image data includes an object or when the image data includes an object. Is a target for embedding preset invisible information, etc., and an embedding target object is set from the determination result. The embedding target object setting unit 26 sets at least one of the plurality of objects when there are a plurality of objects in the image data. In this case, for an object, for example, the priority is set in advance, the position of the display area with respect to the entire screen of the object, and if it is a video, it is arbitrarily set according to the time the object was displayed. The embedded detailed information is also stored in the storage means 23.

  Here, in the embedding target object setting unit 26, whether or not the object included in the image is a target for embedding the invisible information is determined based on whether the embedding determination information preset by the user or the like is stored in the accumulating unit 23 or the like. The embedded determination may be performed using the embedded determination information, or if the operation or additional information for the object is stored in the storage means 23 and the additional information is stored May be determined to be an object to be embedded.

  Therefore, for example, when the image is analyzed by the image analysis unit 25 and it is analyzed that an object such as a windmill, a panda, or a product is included in a part of the image, the embedding target object setting unit 26 It is determined whether or not additional information related to the object is stored in the storage means 23. If additional information related to the object exists, it is determined that the object is an object to be embedded. The contents to be embedded (additional information) can also be set, for example, in the embedded information setting means 27 or the like, and the set contents are stored in the storage means 23.

  In the above-described embodiment, the embedding target object is set based on the analysis result by the image analysis unit 25. However, the present invention is not limited to this, and for example, the output unit 22 or the like. The image data may be displayed on the screen using a user, and an embedding target object may be selected by using the input means 21 from one or a plurality of objects included in the displayed image data.

  The embedding information setting unit 27, what information is embedded as additional information in the object set by the embedding target object setting unit 26, the content of the specific information, and a display method (for example, screen Display (including detailed information such as display size and position), audio output, print output, etc.) is set. Specifically, for example, when the object is a person, the embedded information setting unit 27 performs any operation (rotation, movement, enlargement, reduction, replacement with another object, etc.) on the object. The operation instruction information to be performed is set, the name, age, gender, height, hobby, career, etc. of the person are set, and the display method of the set various information is set. If the object is a wallet, clothes, etc., the brand name, product name, price, address of a homepage, blog, etc. are set, and the display method of the various information set is set. If the object is a book, the title, author, publication date, price, information about the author, etc. of the book are set, and the display method for the various information set is set. Note that the additional information set by the embedded information setting unit 27 includes video, images, and the like.

  The embedded information setting means 27 sets in what form information is added. For example, whether the additional information is a specific encrypted character, a pattern, a symbol, code information, or the like is set. In the case of code information or the like, it is preferable that a correspondence database is provided on the recognition device 16 side so that information on the code can be acquired. As described above, since a plurality of forms can be set, it is possible to select and set appropriate embedding information according to the content of the image to be embedded.

  The embedding information generation unit 28 generates embedding information including processing contents such as what kind of motion is to be performed on the object to be embedded (operation instruction information). The embedded information generating means 28 may generate directly as character information or code information. Specifically, the embedding information generation unit 28 embeds the embedding information in the original image to be embedded in order to make the embedding information difficult to see against a print medium or an image actually provided to the user. For example, an image using a low-frequency part and a high-frequency part, or an image using only a low-frequency part or only a high-frequency part is generated on the basis of color information or the like.

  Here, the low-frequency part in the present embodiment refers to a part or region in which the lightness is lower than that of the original image in which the invisible information is embedded, and the high-frequency part refers to the above-described original image. This means a part or region having a higher brightness with respect to the brightness. Details of the invisible information in the embedded information generating unit 28 will be described later.

  In addition, since the invisible information in the present embodiment is used for extracting an object on the recognition device 16 side, when embedding the embedded information, the embedded information may be embedded in the entire invisible information. Alternatively, it may be embedded in a part of the invisible information (for example, a corner or the periphery in the invisible information area (trimming marker)).

  In addition, the embedding information generation unit 28 acquires, as coordinate information, in which position in the image the non-visualization information is embedded. In the present embodiment, for example, the non-visualization information is embedded in a predetermined area around an object to be operated. That is, in the present embodiment, the recognition device 16 grasps the position of the object based on this invisible information and trims (extracts) the object.

  Accordingly, the non-visualization information in the present embodiment includes, for example, the use of extracting the object to be trimmed on the recognition device 16 side, the operation instruction information for instructing what operation the object should perform as necessary, and the like. There are uses for acquiring additional information and the like.

  The non-visualization information embedding unit 29 is generated by the embedding information generation unit 28 based on the image obtained by the image analysis unit 25 and the object position information including the object information and coordinate information of the image. The image is synthesized by embedding non-visualization information at a predetermined position.

  In the present embodiment, when embedding embedded information, it is preferable that embedded information corresponding to the object is embedded, for example, on or around the position of the object displayed in the image. First, the embedded image is embedded in the target image based on the object position information obtained by the image analysis means 25.

  That is, according to the present embodiment, it is possible to embed a plurality of non-visualized information in an appropriate place, instead of assigning one embedding information to the entire image. The invisible information embedding unit 29 can also embed invisible information for operating a plurality of or all objects included in the original image.

  Further, when the image to be embedded is a video, the invisible information embedding unit 29 moves following the movement of the object in the video being reproduced so that the invisible information is embedded on the object. can do. That is, the non-visualization information embedding unit 29 performs an embedding process on the captured image every time an embedding target image is input, and sequentially displays the embedded image or outputs it as the second print medium 14. be able to.

  The transmission / reception means 30 executes a desired external image (captured image, embedded image, etc.) from an external device connectable using a communication network or the like, an execution program for realizing the invisible information embedding process in the present invention, and the like. It is an interface for obtaining. The transmission / reception means 30 can transmit various information generated in the embedding device 13 to an external device.

  The control means 31 controls the entire components of the embedding device 13. Specifically, the control unit 31 acquires an image, analyzes an image, determines whether the object is an embedding target object, sets invisible information, and the like based on an instruction from the input unit 21 by a user or the like, for example. Control in each process such as embedding is performed. Note that the invisible information in the embedded information setting unit 27 and the invisible information in the embedded information generating unit 28 may be set and generated in advance and stored in the storage unit 23.

<Recognition device 16: functional configuration example>
Next, a functional configuration example of the recognition device 16 in the above-described embodiment will be described with reference to the drawings. FIG. 3 is a diagram illustrating an example of a functional configuration of the recognition apparatus according to the present embodiment.

  2 includes an input unit 41, an output unit 42, a storage unit 43, an embedded image acquisition unit 44, an object extraction unit 45, a non-visualization information analysis unit 46, and a display information generation unit. 47, a transmission / reception means 48, and a control means 49.

  The input unit 41 accepts input such as start / end of various instructions such as an embedded image acquisition instruction, a non-visualization information extraction instruction, a non-visualization information analysis instruction, a display information generation instruction, and a transmission / reception instruction from a user or the like. The input means 41 is composed of a pointing device such as a keyboard and a mouse if it is a general-purpose computer such as a PC, and a group of operation buttons if it is a portable terminal or the like. The input unit 41 also has a function of inputting an image or video taken by an imaging unit such as a digital camera. Note that the above-described imaging unit may be built in the recognition device 16 or may have an external functional configuration. Further, the input means 41 may have a voice input function for inputting voice or the like.

  The input unit 41 can acquire an embedded image from the above-described second print medium 14, that is, a print medium such as paper, a postcard, a poster, a photograph, or a card. In this case, an image capturing unit such as a camera and a function of reading data using a scanner function or the like are provided.

  The output means 42 outputs the contents input by the input means 41 and the contents executed based on the input contents. Specifically, the output unit 42 outputs additional information for an object displayed on the image or video obtained by the display information generation unit 47. Note that the output means 42 includes a display, a speaker, and the like.

  Further, the output means 42 may have a printing function such as a printer, and prints each output content such as the actual operation content for the above-mentioned object on various print media such as paper and provides it to the user or the like. You can also

  The accumulating unit 43 accumulates various information necessary in the present embodiment and various data at the time of execution of the invisible information recognition process or after execution. Specifically, the storage unit 43 includes the embedded image acquired by the embedded image acquisition unit 44, the non-visualization information (marker) acquired by the object extraction unit 45, and the non-visualization information analyzed by the non-visualization information analysis unit 46. , Embedded information, display contents generated by the display information generating means 47, and the like are stored.

  Further, the storage unit 43 can store related information for the data analyzed by the non-visualized information analysis unit 46. For example, when there is code information (including a character code, a two-dimensional code, etc.) that has non-visualization information, various data corresponding to the code information (for example, detailed information (characters) for an object corresponding to the code information , Video, image, audio, etc.), the size, color, time, position, operation content, etc. when data is displayed on the screen are stored in the storage means 43. Further, the storage means 43 can read various data stored when a code or the like is acquired, or when necessary.

  The embedded image acquisition unit 44 acquires image data corresponding to the second print medium 14 from the second image information acquisition device 15. The embedded image acquisition unit 44 may store the image data to be processed in the storage unit 43 in advance, and may store the image data from the storage unit 43 and is connected to the communication network via the transmission / reception unit 48. The embedded image may be acquired from an external device. The embedded image includes a video.

  The object extraction unit 45 extracts an object included in the embedded image. Specifically, for example, the object extraction unit 45 performs filtering at a predetermined frequency on the input embedded image, and acquires the invisible information embedded in the image. By performing filtering at a predetermined frequency, it is possible to easily extract the regions of the high frequency part and the low frequency part. Note that the present invention is not limited to this, and other methods may be used. Further, when there are a plurality of invisible information in the image, all the invisible information is extracted.

  Next, the object extracting means 45 extracts the outer frame (edge) of the object from the embedding position of the non-visualization information included in the image, and associates it with the extracted outer frame (edge) of the object. The object in is separated from the background and extracted.

  For example, the object extraction unit 45 extracts position information around the object by a combination of a low frequency part and a high frequency part set as the invisible information, and extracts the object based on the extracted position information. In the present embodiment, the object is not limited to the above-described low-frequency part and high-frequency part. For example, an object may be extracted based on a color difference, a brightness difference, or the like.

  The object extracting means 45 may extract an object from all the invisible information when there are a plurality of invisible information in the embedded image. At least one invisible information (predetermined information ( For example, invisible information at the leftmost of the embedded image, invisible information having the maximum size, etc.) may be extracted.

  In addition, the object extraction unit 45 also acquires the invisible information extraction position information indicating from which position the invisible information is extracted. The object extraction unit 45 causes the accumulation unit 43 to accumulate various information acquired as described above.

  Furthermore, in this embodiment, by extracting the invisible information, the direction (angle), inclination, etc. of the object can be acquired. Therefore, the object extraction unit 45 can correct the angle and inclination of the target object from information such as the direction and inclination obtained from the invisible information.

  The invisible information analyzing means 46 analyzes whether or not the invisible information obtained by the object extracting means 45 includes embedded information. If the embedded information is included, the specific invisible information is analyzed. Get the contents of the included information.

  For example, when a character (for example, rotation, movement to the right, etc.) is included as the embedded information in the invisible information, the character is acquired as additional information. Further, when a code is included as embedded information, the code information is acquired, and the acquired content (for example, code ID) is used as a key via the storage means 43 and the transmission / reception means 48. Search for external devices such as pre-set servers and databases connected to the communication network. As a result of the search, there are various processing contents (additional information) such as operation instruction information and display method for the object corresponding to the key. If so, get that information.

  Note that the invisible information analysis means 46 in this embodiment may have a reading function such as a barcode reader for reading a barcode. In that case, for example, when the invisible information is a two-dimensional barcode, additional information is acquired from the two-dimensional barcode using a barcode reader.

  The display information generation unit 47 generates the operation content for the object displayed on the screen based on the additional information obtained by the non-visualization information analysis unit 46, the display content based on the display method, and the like. Note that the display on the screen may be performed by providing another frame (another window) on the screen, or may be displayed on the position where the corresponding object is displayed, and output by voice. Also good.

  As the processing in the display information generation means 47, for example, as described above, the processing for erasing the object from the background is performed, or the object is displayed at the erased position, and enlarged, reduced, rotated, moved, and other objects are displayed. Substitution etc. can be performed. Therefore, it is possible to display the predetermined object included in the image by performing individual operations. Furthermore, the display information generation unit 47 can superimpose the displayed object on the background image, and can perform angle and inclination correction processing by space recognition.

  Here, in the present embodiment, what kind of operation is performed on which object can be acquired from, for example, non-visualization information around the object. Therefore, in the present embodiment, only by embedding non-visualization information around the object to be operated in the image, unique processing can be set on the embedding device 13 side for each object.

  Further, the display information generating unit 47 may visualize and display the acquired invisible information as it is, acquire additional information corresponding to the invisible information from the storage unit 43 or an external device, and the acquired additional information. Information may be displayed.

  Furthermore, when the result obtained by the non-visualized information analyzing unit 46 is a code, the display information generating unit 47 is a screen set for each additional information acquired from the accumulating unit 43 based on the code ID described above. Display information to be displayed is generated based on the size, color, time, position, operation content, and the like when displayed. When the target object is moving as a video, it may be displayed following the position of the object, or may be displayed fixed at the position initially displayed on the screen.

  The transmission / reception means 48 is an interface for acquiring a desired external image (captured image or the like) from an external device connectable using a communication network or the like, an execution program for realizing the invisible information recognition processing in the present invention, and the like. It is. The transmission / reception unit 48 can transmit various information generated in the recognition device 16 to an external device.

  The control means 49 controls the entire components of the recognition device 16. Specifically, for example, the control unit 49 obtains embedded images, extracts non-visualized information, analyzes non-visualized information, generates display information, and the like based on an instruction from the input unit 41 by a user or the like. Each control of processing is performed.

  With the above-described apparatus configuration, it is possible to provide a highly accurate image with excellent added value by efficiently acquiring information.

<Embedding device 13, recognition device 16: hardware configuration>
Here, in the embedding device 13 and the recognition device 16 described above, an execution program (embedding program, recognition program) capable of causing a computer to execute each function is generated, and the program is transmitted to, for example, a general-purpose personal computer or server. By installing the execution program, the invisible information embedding process and the recognition process in the present invention can be realized.

  Here, a hardware configuration example of a computer capable of realizing the invisible information embedding process and the recognition process in the present embodiment will be described with reference to the drawings. FIG. 4 is a diagram illustrating an example of a hardware configuration capable of realizing the invisible information embedding process and the recognition process in the present embodiment.

  4 includes an input device 51, an output device 52, a drive device 53, an auxiliary storage device 54, a memory device 55, a CPU (Central Processing Unit) 56 that performs various controls, and a network connection device. 57 are connected to each other by a system bus B.

  The input device 51 has a pointing device such as a keyboard and a mouse operated by a user or the like, and inputs various operation signals such as execution of a program from the user or the like. The input device 51 has an image input unit for inputting an image taken from an imaging means such as a camera.

  The output device 52 has a display for displaying various windows and data necessary for operating the computer main body for performing processing according to the present invention, and displays the program execution progress, results, and the like by the control program of the CPU 56. can do.

  Here, the execution program installed in the computer main body in the present invention is provided by a portable recording medium 58 such as a USB (Universal Serial Bus) memory or a CD-ROM, for example. The recording medium 58 on which the program is recorded can be set in the drive device 53, and the execution program included in the recording medium 58 is installed in the auxiliary storage device 54 from the recording medium 58 via the drive device 53.

  The auxiliary storage device 54 is a storage means such as a hard disk, and can store an execution program according to the present invention, a control program provided in a computer, and the like, and can perform input / output as necessary.

  The memory device 55 stores an execution program or the like read from the auxiliary storage device 54 by the CPU 56. The memory device 55 includes a ROM (Read Only Memory), a RAM (Random Access Memory), and the like.

  The CPU 56 controls processing of the entire computer, such as various operations and input / output of data with each hardware component, based on a control program such as an OS (Operating System) and an execution program stored in the memory device 55. Thus, each process in the embedding process and the recognition process can be realized. Various information necessary during the execution of the program can be acquired from the auxiliary storage device 54, and the execution result and the like can also be stored.

  The network connection device 57 obtains an execution program from another terminal connected to the communication network by connecting to a communication network or the like, or an execution result obtained by executing the program or an execution in the present invention The program itself can be provided to other terminals.

  With the hardware configuration as described above, the invisible information embedding process and the recognition process in the present invention can be executed. Further, by installing the program, the embedding process and the recognition process in the present invention can be easily realized by a general-purpose personal computer or the like.

  Next, the invisible information embedding process in the above-described embedding program and the invisible information recognizing process in the recognition program will be specifically described.

<Procedure for embedding non-visualized information>
First, a procedure for embedding non-visualized information in the present embodiment will be described. FIG. 5 is a flowchart illustrating an example of the embedding process procedure of the invisible information in the present embodiment.

  In the embedding process shown in FIG. 5, first, an image taken by an imaging means such as a camera is acquired (S01), the image is analyzed (S02), and an object included in the image, object position information, and the like are acquired. .

  Next, the embedding target object is set based on the information obtained in the process of S02 (S03), and it is determined whether or not the invisible information (marker) is embedded in the object (S04). Here, when the invisible information is embedded (YES in S04), the embedded information indicating the processing contents such as what kind of operation instruction information is embedded in the object is set (S05), and the process of S05 is performed. The embedded information set by is generated (S06).

  Also, embedding is performed at a predetermined position around a predetermined object in the image including the embedding information generated in the process of S06 (S07), and the combined image after the embedding is displayed or output as data by an output means such as a display. (S08). In the process of S08, the image obtained in S07 can be output to a print medium.

  When the invisible information is not embedded after the processing of S08 or in the processing of S04 (NO in S04), it is determined whether or not the invisible information is embedded in another image (S09). When the invisible information is embedded in another image (YES in S09), the process returns to S01 and the subsequent processing is repeatedly executed. Further, in the process of S09, when the invisible information is not embedded in another image (NO in S09), the invisible information embedding process is terminated.

  In the present embodiment, the user or the like may set at least one object from the image data displayed on the screen without performing the process of S02 described above.

<Recognition processing procedure for invisible information>
Next, the recognition process procedure of the invisible information in this embodiment will be described. FIG. 6 is a flowchart showing an example of the recognition processing procedure of the invisible information in the present embodiment.

  In the recognition process shown in FIG. 6, first, an embedded image in which the invisible information is embedded by the above-described embedding process is acquired (S11). In the process of S11, any means may be used as long as an image can be acquired. For example, as described above, a print medium may be photographed to acquire the image data, and the image data exists in advance. The embedded image to be acquired may be acquired from an external device or the like. Next, an object is extracted from the non-visualization information included in the acquired image (S12).

  Next, in the process of S12, it is determined whether or not embedded information including various processing contents such as operation instruction contents and display methods for the object is extracted from the invisible information (S13), and the embedded information is extracted. If so (YES in S13), the embedded information is analyzed (S14). Also, display information to be displayed on the screen or the like is generated from the additional information obtained from the analysis result of the embedded information (S15), and the generated content is displayed (S16).

  Here, it is determined whether or not the invisible information is recognized from another image (S17). When the invisible information is recognized from another image (YES in S17), the process returns to S11 and the subsequent processing is repeatedly executed. To do. Further, in the process of S17, when the invisible information is not recognized from another image (NO in S17), the invisible information recognition process is terminated.

  Through the processing described above, it is possible to quickly embed and extract invisible information and provide a highly accurate content service with excellent added value. In addition, by installing the program, the non-visualized information embedding process and the recognition process in the present invention can be easily realized by a general-purpose personal computer or the like.

<Example of embedding non-visualized information>
Next, an example of embedding invisible information in the present embodiment will be described with reference to the drawings. FIG. 7 is a diagram for explaining an example of embedding non-visualization information in the present embodiment. When embedding processing is performed, embedding is performed in the order of FIGS. 7A to 7D.

  First, in the case where the invisible information is inserted into a predetermined object 62 included in the image data 61 that is a photographed image input from the first print medium 11 or the like described above, as shown in FIG. As described above, an outer frame 63 of a cord having a predetermined shape is arranged around the object 62 as shown by an arrow in FIG.

  The outer frame shape is preferably a square in order to improve the reading accuracy in the recognition device 16, but is not limited to this in the present invention. The shape may be a square, a circle, an ellipse, an enlarged shape similar to the shape of an object, a shape free-ridden by an input means such as a mouse by a user or the like.

  Next, as shown in FIG. 7B, a rough outline 64 for the object 62 is designated. Note that the contour 64 can be arbitrarily set by a user or the like using input means such as a mouse or a touch panel. The outline 64 can also be roughly extracted based on a difference in luminance value between the object 62 and its background. The outline 64 may be a rough one, and may be an area including the entire object 62, for example.

  Next, as shown in FIG. 7C, a code inner frame 65 is set in correspondence with the shape of the contour 64. The code inner frame 65 does not exceed the outer frame 63, and an area outside the circumference of the outline 64 (for a predetermined number of pixels) is set, for example. Next, as shown in FIG. 7D, an encoding region of the high frequency unit 66 is added between the code inner frame 65 and the outer frame 63, and a low frequency unit 67 is added between the contour and the inner frame. Thereby, the invisible information for recognizing the position of the object 62 on the recognition device 16 side can be embedded.

  Further, in the present embodiment, a predetermined operation is performed for each predetermined object in the image by inserting each processing content such as operation instruction information and display method for the object by some character, symbol, code, or the like in the invisible information. Etc. can be performed. Thereby, a content service with added value can be provided.

<Extraction of non-visualized information and object recognition example>
Next, extraction of non-visualized information and object recognition examples in the recognition apparatus 16 according to the present embodiment will be described with reference to the drawings. FIG. 8 is a diagram for explaining an example of extracting the invisible information and recognizing an object in the present embodiment.

  In addition, when extracting the invisible information and recognizing the object according to the recognition processing of the present embodiment, the embedding is performed in the order of FIG. 8A to FIG. First, when inserting invisible information into a predetermined object 62 included in the image data 61 input from the first print medium 11 or the like described above, first, as shown in FIG. The code information included in the image 61 is acquired. Specifically, for example, a region where the outer frame 63, the high frequency unit 66, and the low frequency unit 67 exist is extracted.

  Next, as shown in FIG. 8 (b), the inner high frequency part 66 is made transparent from the outer frame 63, and further, as shown in FIG. 8 (c), the low frequency part is directed inward from the edge of the high frequency part 66. Make 67 transparent. Note that the low frequency unit 67 can be grasped based on the luminance information of the surrounding pixels, but is not limited to this in the present invention. For example, the inside of the high frequency unit by a predetermined number of bits. Only a predetermined number of bits may be made transparent, such as making it transparent, or the above-described transparency processing may be performed using a color difference, a light / dark difference, or the like.

  Thereafter, as shown in FIG. 8D, a trimmed image 71 composed only of objects can be obtained. The area around the object shown in FIG. 8D is given a pattern indicating that it is transparent for convenience.

<Operation example for object>
Next, in this embodiment, an operation example for an object included in provided content will be described with reference to the drawings. FIG. 9 is a diagram for explaining an object operation example in the present embodiment.

  When moving the object, for example, as shown in FIG. 9A, the trimming image 71 is superimposed on the captured image 61, and then, as shown in FIG. delete. Specifically, for example, the area of the object 62a is overcoated using pixels of the background image around the object 62a. In the present invention, the present invention is not limited to this. For example, the color may be painted using a preset color or the like.

  Next, the object 62b included in the trimmed image 71 is operated on the captured image 61 from which the object 62a is deleted as a background image. 9C shows the process of moving the object 62b to the right at a predetermined timing. However, the present invention is not limited to this, and the object 62b is enlarged. Or can be reduced, moved in a predetermined direction, or replaced with another object.

  Further, as a variation of the input and the corresponding operation, in this embodiment, for example, when a user blows on the voice input device or the like of the recognition device 16 (microphone input), a part or all of the object rotates, When tilted, the object rolls, when the screen of the recognition device 16 is touched, the object takes a reaction, and when the user goes to a specific place, the object can perform an action specific to the place.

  Furthermore, in this embodiment, the movement of the object can change depending on the direction in which it is facing, the object can sleep or wake up depending on the time, the movement of the object can change depending on the weather, etc. It is possible to perform an operation of deforming the object in accordance with the vibration, the moving speed, the moving distance, etc. associated with the operation.

<About low frequency part and high frequency part>
Here, the low frequency part and the high frequency part in the present embodiment will be described. Normally, there are frequencies for time (temporal frequency) and frequencies for space positions (spatial frequency). In the present embodiment, the frequency refers to the spatial frequency unless otherwise specified. Spatial frequency is defined as “the reciprocal of the period of the pixel value relative to the unit length”.

  The frequency in the present embodiment is not particularly limited. For example, the frequency may be set in the range of 0.2 to 2 [cycle / pixel] in the high frequency part and 0 to 1 [cycle / pixel] in the low frequency part. Specifically, it is sufficient that the high frequency portion has a higher frequency than the low frequency portion.

In addition, a grid composed of predetermined pixel regions (for example, 4 × 4 px (pixels) or the like) formed in a high-frequency part is sufficient if a bright part and a dark part are periodically repeated. For example, vertical stripes, horizontal stripes, grids State.
For example,
Bright light dark dark dark bright light bright dark dark dark or
Light / dark / dark / dark / light / dark / dark / light / dark.

  In addition, the brightness difference between the bright part and the dark part at that time may be 10 or more, preferably 50 or more, and more preferably 100 or more. As described above, the brightness difference in the present embodiment first generates a bright part and a dark part on the basis of the brightness of an image that is normally displayed, and uses the brightness difference between the generated bright part and the dark part. However, the present invention is not limited to this. For example, the brightness difference between the brightness of a normal image and the brightness at the low frequency part or the high frequency part may be used.

  In this case, for example, if the brightness difference considered to be a high-frequency part is about 15 or more with reference to an adjacent element in gray scale, it can be regarded as a high-frequency part. Specifically, a brightness difference of about 15 to 35 is an area that can be mainly used as a high-frequency part. In addition, an element is comprised by the pixel of 1 px or more in length and width, and in this embodiment, 1 element can be 2x2 px, for example.

  Here, for example, when the minimum value of the brightness difference that can be read by an imaging device such as a camera is 15, if the brightness difference exceeds 35, it can be easily recognized by human eyes. On the other hand, if the portion of the image to be embedded is extremely dark or bright, it is necessary to add a brightness difference of, for example, 35 or more in order to improve the reading accuracy by a camera or the like. Therefore, in the present embodiment, the code of the invisible information is generated by arbitrarily changing the brightness difference according to the brightness and brightness of the image (background) at the embedded position, the performance of the camera to be photographed, and the like.

  In addition, the usable pixel size in the additional information described above is not particularly limited because it varies depending on the distance between the image and the person viewing the image, for example. About 0.05 to 2 mm is preferable, and if the distance is about 10 m, about 0.5 mm to 20 mm is preferable. Even when used from a further distance, it is preferable to maintain the same pixel size / distance ratio.

  That is, in this embodiment, a non-visualization code that uses a high-frequency block made up of small bright and dark squares in a grid pattern (checkered pattern) is used as the non-visualization information in this embodiment. Thereby, it can embed in any place and can read from arbitrary angles.

  Here, FIG. 10 is a diagram for explaining the invisible information having the high-frequency part and the low-frequency part. In the structural example of the invisible image shown in FIG. 10A, the position detection pattern 81 has a rotation detection pattern 82. This makes it possible to accurately recognize the position where the code is and the direction of the code.

  Further, the position detection pattern 81 has a high frequency block 83 as a high frequency portion embedded in the image, and has an edge 84 for storing an object at the center. As shown in FIG. 10A, in the present embodiment, the position and direction of the image are detected by a detection pattern of surrounding positions and directions. These patterns are also composed of the high frequency block 83. In the example of FIG. 10A, the outer frame of the position detection pattern 81 is surrounded by a dotted line guideline, and the outer frame of the rotation detection pattern 82 is surrounded by a solid line guideline. do not do.

  Each block shown in FIG. 10 (b) is composed of an N × N grid arrangement in which bright and dark elements are alternated so that it is difficult for humans to visually recognize and reduce the field of view. As described above, the bright element (element) and the dark element are generated by increasing or decreasing the luminance with respect to the luminance (brightness) of the background.

  Note that the change in brightness is adjusted by the embedding strength (which is a ± α change in brightness). When the embedding strength is constant, the read recognition accuracy is not good when the image is very bright or very dark. Therefore, as shown in FIG. 10C, the read recognition accuracy is improved by changing the embedding strength of the background color. Specifically, when each RGB value of the background can be changed independently to change the brightness of the element, the color tone changes by raising the field of view of the image. Therefore, in this embodiment, in order to reduce the field of view and improve the accuracy of reading recognition, the original color image is converted to a gray level, and the embedding strength is set based on the value of the background gray scale.

Here, FIG. 11 is a diagram showing the relationship between the embedding strength and the gray scale value of the background. As shown in FIG. 11, the human eye is very sensitive to changes in white brightness and insensitive to changes in black. Therefore, the change in the embedding strength is given by the following equation using an equation in which the minimum strength is 20 as a quadratic function with respect to the gray scale value.
Gray level value α = 1/2167 × (αr−255) ² +20
<Experimental result>
Here, experimental results to which the present invention is applied will be described with reference to the drawings. The read recognition accuracy in this embodiment is evaluated by what code is embedded (code style).

  FIG. 12 is a diagram illustrating an example of a non-visualization code for a monochrome image. The code shown in FIG. 12 is about 3 cm square and can hold 100 bits of data. Each side of the high frequency block has four long elements. The element size is about 2 × 2 pixels. Further, as shown in FIG. 12, the high-frequency unit block has a checkerboard (checkered pattern) block for a monochrome image.

  FIG. 13 is a diagram illustrating a relationship between reading recognition accuracy and invisibility with respect to brightness obtained by changing a gray image from 0 to 255. In the present embodiment, an evaluation result is shown in which a difference between an image with a cord and an image without a cord is viewed from a point 50 cm away. As a result, when the brightness range of the background image is in the gray level of 40 to 180, the read recognition accuracy is 90% or more, and the case where it is determined that the background image is visualized is less than 10%.

  Furthermore, in order to confirm the effectiveness of the non-visualization code in the present embodiment, the read recognition accuracy of the non-visualization code embedded in three common types of images was evaluated as an example.

  FIG. 14 is a diagram for describing the evaluation of the read recognition accuracy of the invisible information for each image in the present embodiment. 14A is inserted as a test image shown in FIG. 14B into a black-and-white (monochrome) image, a female image, and an animal (manto baboon) image. Recognition accuracy was evaluated.

  Here, FIG. 15 is a figure which shows the evaluation result corresponding to FIG. 15A is a graph showing the recognition accuracy with respect to the distance from the camera to the code, and FIG. 15B is a graph showing the recognition accuracy with respect to the shooting angle of the camera.

  As shown in FIG. 15 (a), regarding the accuracy regarding the distance from the camera to the code, if the distance is about 7 to 11 cm, the recognition accuracy is 100% for all images. It has been shown.

  Moreover, as shown in FIG.15 (b), it was shown that the recognition accuracy is 100% with respect to all the images at an angle of about 0 to 40 with respect to the camera angle.

<Example of content application to which this embodiment is applied>
Next, a content application example to which the above-described embodiment is applied will be described with reference to the drawings. In the following example, an AR application system including a trimming marker as non-visualization information will be described.

<Example 1>
FIG. 16 is a diagram illustrating an example of the AR application system according to the first embodiment. FIG. 16A shows an example of an image using a trimming marker as non-visualization information. The trimming marker is embedded around the image for the windmill shown in FIG. In the first embodiment, it may be the entire periphery or a part of the periphery.

  For example, a user acquires an image of a windmill by taking a picture of the image with a camera provided in a mobile terminal such as a smartphone. When the user selects at least one of the objects included in the photographed image using the portable terminal, the AR marker in the first embodiment causes a trimming marker to be embedded in the object. In the example of FIG. 16, the trimming marker is embedded in the windmill, and the embedded image (windmill) is printed. In the first embodiment, the trimming marker may be performed only on a predetermined windmill (trimming object) to be trimmed on the recognition side among the plurality of windmills (objects) illustrated in FIG.

  Here, FIG. 16B shows the structure of the invisible trimming marker. The trimming marker includes a high-frequency part and a low-frequency part, and is composed of an object to be extracted (trimming object) at the center of the trimming marker. The position of the trimming marker is acquired by a position detection pattern existing around the object. Further, the direction (direction) of the object can be acquired by, for example, embedding the rotation detection pattern described above in a trimming marker. Note that the above-described processing is performed by a mobile terminal. Therefore, the portable terminal has functions as the first image information acquisition device 12 and the embedding device 13 described above, and these functions are realized by the AR application in the above-described embedding program.

  FIG. 16C is a diagram illustrating an example of extracting an object using a trimming marker. A user photographs a printed image including a trimming marker with a camera provided in a mobile terminal such as a smartphone. As described above, the mobile terminal (recognition device 16) performs the transparency processing in the order of the high-frequency part and the low-frequency part from the embedding position of the trimming marker, and acquires the trimming object in a state separated from the background. In addition, the mobile terminal generates an image obtained by virtually synthesizing the acquired trimming object with the background image, and displays the image on the screen.

  Next, how the object (windmill) extracted by the acquired trimming marker operates will be specifically described with reference to the drawings. FIG. 17 is a diagram illustrating an operation example of the windmill by the acquired trimming marker. Specifically, the operation transitions in the order of FIG. 17 (a) to FIG. 17 (d).

  For example, when a user takes a picture of a windmill image using a mobile terminal such as a smartphone (FIG. 17A), the windmill image (trimming object) that is a target image is obtained according to the above-described embodiment. It is separated from the background image by the technique (FIG. 17B). Thereafter, the windmill and the background image are displayed on the display screen of the portable terminal (FIG. 17C).

  Here, as the secondary instruction information described above, the user blows or breathes a voice input device such as a microphone provided on the smartphone or squeezes something, so that the volume (volume) or sound And an image of rotating the windmill according to the amount of change is created (FIG. 17D). It should be noted that what operation is performed for what secondary instruction information may be set in advance or may be extracted from the information of the trimming marker.

  Note that the above-described processing is performed by a mobile terminal. Therefore, the portable terminal has functions as the second image information acquisition device 15 and the recognition device 16 described above, and these functions are realized by the AR application in the recognition program described above.

  In the first embodiment, when an operation is instructed using the above-described trimming marker, if there are a plurality of trimming markers on the screen, the content of the embedded information included in the trimming marker corresponding to each object (windmill) May be operated for each object, or the operation content may be instructed to a plurality of extracted objects by instruction information included in one trimming marker. As a result, for example, when the same action is performed on a plurality of objects, it is not necessary to set embedding information including action instruction information etc. for the trimming marker corresponding to each object. Since a plurality of objects can be operated by information, an object operation can be realized efficiently and quickly.

  Here, FIG. 18 is a diagram for explaining the relationship between the non-visualization information included in the image and the operation instruction information. 18A shows an example of an embedded image, and FIG. 18B is an enlarged view of an object in which the invisible information and the operation instruction information (embedding information) are embedded (with the operation instruction information guidelines). FIG. 18 (c) shows a diagram without the guideline corresponding to FIG. 18 (b).

  Next, in the embedded image 91 shown in FIG. 18A, the non-visualization information 92-1 and 92-2 are embedded in two objects among the three windmill objects in the image. Therefore, in the first embodiment described above, the objects recognized and extracted by the recognition device 16 are only two windmills surrounded by the non-visualization information 92-1 and 92-2. In the examples of FIGS. 18A to 18C, the outer frames of the non-visualized information 92-1 and 92-2 are visualized for convenience of explanation.

  Next, in the example of FIG. 18B, operation instruction information (embedding information) 93-1 to 93-4 is embedded at the four corners of the non-visualization information 92-2. In the example of FIG. 18B, a guideline (solid line) is shown in the embedding area of the operation instruction information 93-1 to 93-4. However, in an actual image, as shown in FIG. Become.

  Here, where the operation instruction information is embedded in the non-visualized information is not limited to this in the present invention. For example, it may be at least one corner of the non-visualized information. It may be on the outer frame. Further, the operation instruction information 93-1 to 93-4 can be used as the rotation detection pattern described above.

  In the operation instruction information 93-1 to 93-4 shown in FIG. 18B, for example, 4-bit information is embedded in each corner. Accordingly, the operation instruction information 93-1 to 93-4 can generate one piece of information by connecting the information at the four corners in a predetermined order to make a total of 16 bits. Since 65536 code patterns can be acquired by using 16 bits, various operation contents and display methods can be set for an object by using information obtained from the code as an ID or an operation switch. can do.

  Furthermore, in the first embodiment, the operation corresponding to the above-described four corner embedding information may be executed in a predetermined order and timing, and the contents of one embedding information may be applied to all the extracted objects. May be executed.

  The above-described embedding information is embedded in a previously specified range. When the above-described invisible information analyzing means 46 analyzes the embedding information, it can be immediately embedded by referring to the specified range. Information can be acquired.

<Example 2>
FIG. 19 is a diagram illustrating an example of an AR application system according to the second embodiment. Note that the AR application system in FIG. 19 can also be performed using a portable terminal, as in the first embodiment. Note that the mobile terminal used here has functions as, for example, the second image information acquisition device 15 and the recognition device 16 described above.

  In the second embodiment, for example, a trimming object 101 (red dragonfly in the example of FIG. 19) that is a target for performing a predetermined operation is drawn in a paper medium 100 such as a picture book. Further, the trimming marker 102 described above is embedded around the trimming object 101. In FIG. 19, for convenience of explanation, the outer frame of the trimming marker 102 is indicated by a dotted line, but such a dotted line does not exist in the actual paper medium 100.

  In the trimming marker 102, as shown in FIG. 19, a low frequency part 103 is embedded around the outer shape of the trimming object 101, and a high frequency part 104 is embedded outside the low frequency part 103. In the trimming marker 102, operation instruction information 105-1 to 105-4 for causing the trimming object 101 to perform a predetermined operation is embedded.

  In the second embodiment, the paper medium 100 is photographed by a camera (image information acquisition device 15) provided in a portable terminal or the like. The mobile terminal (recognition device 16) acquires the trimming marker 102 from the captured image. Also, operation instruction information is embedded in the four corners of the trimming marker, and the operation of the trimming object 101 is changed according to the value of the operation instruction information.

  Here, in the operation instruction information 105-1 to 105-4, for example, information is embedded by 1 bit. Therefore, in order to obtain the operation instruction information from the operation instruction information 105-1 to 105-4, for example, information on the four corners is read in a specific order (for example, clockwise), and is set in advance according to the read result, for example. The trimming object 101 which is separated from the background image and is virtually combined is executed. Specifically, for example, when the operation instruction information (4 bits) is “1100”, the trimming object 101 is moved to the right. When the operation instruction information is “1010”, the trimming object 101 can be moved to the left, and when it is “1111”, the trimming object 101 can be rotated on the spot. The operation content in the second embodiment is not limited to this, and for example, the direction in which the red dragonfly, which is the trimming object 101, can be changed. Further, the posture (for example, position (coordinates) and direction) of the trimming object, the moving speed, and the like can be acquired from the operation instruction information.

  In addition, the execution timing of the above-described operation is determined by, for example, a predetermined operation by tapping the trimming object 101 displayed on the screen of the mobile terminal on the screen (including widely meanings such as hitting, touching, rubbing, etc.). However, the present invention is not limited to this. For example, when the in-camera (user-side camera) of the mobile terminal is used to recognize a change in brightness in real time, and the user tries to touch the trimming object 101 when, for example, the average brightness is lower than a predetermined value, Judgment may be made so as to cause the red dragonfly, which is the trimming object 101, to fly away.

  Note that the operation content set in accordance with the operation instruction information may be stored in advance in a storage unit or the like, or connected to a preset external device or the like via a communication network or the like, and connected to the external device The operation content of the target may be acquired from. As a result, for example, the same operation instruction content can be used, and different operations can be performed depending on the operation content stored in the storage unit or the like and the operation content acquired from the external device. Further, by acquiring the operation content from an external device or the like, it is possible to freely change the operation by managing the external device.

<Example 3>
FIG. 20 is a diagram illustrating an example of an AR application system according to the third embodiment. Note that the AR application system in FIG. 20 can also be performed using a portable terminal, as in the second embodiment.

  In the third embodiment, for example, a plurality of trimming objects 111-1 to 111-4 (in the example of FIG. 20) that are targets for performing a predetermined operation are drawn in a paper medium 110 such as a picture book. Further, the trimming markers 112-1 to 112-4 described above are embedded around the respective trimming objects 111-1 to 111-4 so as to correspond to the respective trimming objects.

  In the trimming markers 112-1 to 112-4, a low frequency part is embedded around the outer shape of each trimming object 111-1 to 111-4 as in FIG. 19, and a high frequency part is further outside the low frequency part. Is embedded. In the trimming markers 112-1 to 112-4, operation instruction information for causing the trimming objects 111-1 to 111-4 to perform a predetermined operation is embedded.

  In the third embodiment, the paper medium 110 is photographed by a camera (image information acquisition device 15) provided in a portable terminal or the like. The mobile terminal (recognition device 16) acquires the trimming markers 112-1 to 112-4 from the captured image. Further, the mobile terminal separates the trimmed trimming objects 111-1 to 111-4 from the trimming objects 111-1 to 111-4 and the background image based on the acquired trimming markers 112-1 to 112-4, and virtually combines these images. Is generated and displayed on the screen.

  As described above, the operation instruction information is embedded in the trimming markers 112-1 to 112-4, and each trimming object 111-1 is determined according to the value of the operation instruction information as in the above-described embodiment. The operation of ˜111-4 is changed. The operation instruction information may be embedded not in the four corners but around the trimming markers 112-1 to 112-4, for example, or at a predetermined position in the preset trimming marker. That's fine.

  For example, in the third embodiment, when a screen on which an image photographed by the user's hand 113 is displayed is tapped, a ripple spreads, and an operation is performed in which the wrinkles that are the trimming objects 111-1 to 111-4 escape radially. . Specifically, for example, when the screen is tapped with the fingertip of the user's hand 113, the coordinates where the mobile terminal (recognition device 16) is tapped and the coordinates of the eyelids that are the trimming objects 111-1 to 111-4 are acquired. Then, using each acquired coordinate, a direction vector in which each trimming object 111-1 to 111-4 proceeds is generated.

  In this case, angle information indicating which of the trimming objects 111-1 to 111-4 is facing is embedded in the operation instruction information. Further, the operation instruction information can be expressed by 0 to 360 degrees by embedding 8-bit information around each marker instead of the four corners of the trimming markers 112-1 to 112-4, for example.

  Furthermore, the mobile terminal adjusts the speed of the trimming objects 111-1 to 111-4 by normalizing the direction vector. By generating the vector in this way, the trimming objects 111-1 to 111-4 can be operated so as to escape in a radial pattern regardless of where the user taps on the screen on the mobile terminal.

<Example 4>
FIG. 21 is a diagram illustrating an example of the AR application system according to the fourth embodiment. Note that the AR application system in FIG. 21 can also be performed using a portable terminal, as in the second embodiment.

  In the fourth embodiment, for example, a plurality of trimming objects 121-1 and 121-2 (clothing in the example of FIG. 21) are drawn in a paper medium 120 such as a fashion magazine. Further, the trimming markers 122-1 and 122-2 are embedded around the trimming objects 121-1 and 121-2 so as to correspond to the respective trimming objects 121-1 and 121-2.

  In the fourth embodiment, the paper medium 120 is photographed by a camera (image information acquisition device 15) provided in a portable terminal or the like. In addition, the mobile terminal (recognition device 16) acquires the trimming markers 122-1 and 122-2 from the captured image. In addition, the mobile terminal separates the trimmed trimming objects 121-1 and 121-2 from the clothing and the background image based on the acquired trimming markers 122-1 and 122-2, and virtually combines these images. Is generated and displayed on the screen 130. In addition, the trimmed image can be enlarged / reduced, rotated and moved on the screen, and each time a product is photographed, the user adds an image of clothing etc. on the screen, so it can be coordinated as desired. Can enjoy.

  Therefore, in Example 4, it can provide as a fashion coordinate application, for example by performing the function mentioned above. Specifically, a trimming marker is embedded in a product such as a clothing object 131-1, a skirt object 131-2, or a shoe object 131-3 that is published in a fashion magazine, etc. When shooting at, each product is cut out on the screen 130 of the mobile terminal and displayed on the screen. The user can virtually coordinate on the screen 130 by selecting and moving his / her favorite items displayed on the screen 130 of the mobile terminal with the user's hand 132.

  In addition, in Example 4, it is not limited to clothing, For example, a hat, glasses, accessories, a bag, etc. are included, and a face image etc. can be taken in and it can coordinate with one's face.

  In the case of the above-described fashion coordinate application, for example, an image on the back side of a product (for example, a design on the back side of a jacket or a skirt), a price, color information, etc. are acquired from operation instruction information embedded in a trimming marker. can do. For example, when the data capacity is large, such as a design image on the back side, the image can be browsed by browser display on the Web or the like. That is, for example, when the information to be embedded is small (for example, about several bits), the operation instruction information is embedded at the four corners or the periphery of the marker, and when the information to be embedded is large, the information ID (identification information) In the process, trimming is performed, and at the same time, an ID is sent to a predetermined external device (server), and corresponding information can be acquired from the Web or the like.

  Here, in each of the embodiments described above, after recognizing the trimming marker, the operation associated with the operation instruction information may be performed immediately, or may be performed after a predetermined time has elapsed since the recognition, and after further recognition, You may perform when predetermined operation | movement and operation are performed with respect to the portable terminal (recognition apparatus).

  In addition, when performing an action on a plurality of objects, each object may be set with action instruction information, and complex processing for a plurality of objects from each action instruction information may be performed. Can also be performed. Note that the composite operation is an operation in which, for example, when there are two human objects, the two people cross shoulders, shake hands, or carry a shoulder wheel. However, the present invention is not limited to this.

  Furthermore, in the present embodiment, the motion instruction information can include motions for other objects. This makes it possible to perform a unified operation for the entire object. The unified operation is an operation in which all of a plurality of human objects shake hands, for example. Further, as the operation instruction information, for example, when the object is a person, additional information such as information on the person (name, age, gender, etc.), an address of a homepage, a blog, etc. can be embedded. Further, as additional information, information for displaying another object different from the trimming object on the screen can be added.

  Further, as another embodiment, for example, the above-described trimming marker is embedded in a product or food packaging, box, container, or manual, and the image is recognized by photographing it with a camera. Instructions, usage instructions, cooking methods, etc. can be displayed on the screen. For example, when there is a cosmetic product (for example, lipstick, eye shadow, mascara, etc.) as a trimming object, the color information of the cosmetic product is acquired by the trimming marker, and the color information is used for the face image of oneself or another person. You can also make up.

  Note that the paper media 100, 110, and 120 shown in the second to fourth embodiments may be images displayed on a screen. The image also includes a video.

  As described above, according to the present invention, it is possible to quickly embed and extract non-visualized information and provide highly accurate content with excellent added value.

  The preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited to the specific embodiments, and various modifications can be made within the scope of the gist of the present invention described in the claims. Can be changed.

DESCRIPTION OF SYMBOLS 10 Content provision system 11 1st print medium 12 1st image information acquisition apparatus 13 Embedding apparatus 14 2nd print medium 15 2nd image information acquisition apparatus 16 Recognition apparatus 21,41 Input means 22,42 Output means 23 , 43 Storage means 24 Image acquisition means 25 Image analysis means 26 Embedding object setting means 27 Embedding information setting means 28 Embedding information generating means 29 Non-visualized information embedding means 30, 48 Transmission / reception means 31, 49 Control means 44 Embedding Embedded image acquisition means 45 object extraction means 46 non-visualization information analysis means 47 display information generation means 51 input device 52 output device 53 drive device 54 auxiliary storage device 55 memory device 56 CPU
57 Network connection device 58 Recording medium 61 Image data (captured image)
62, 131 Object 63 Outer frame 64 Contour 65 Code inner frame 66, 104 High frequency section 67, 103 Low frequency section 71 Trimmed image 81 Position detection pattern 82 Rotation detection pattern 84 Edge 91 Embedded image 92 Non-visualized information 93 Operation instruction information ( Embedded information)
100, 110, 120 Paper medium 101, 111, 121 Trimming object 102, 112, 122 Trimming marker 105 Operation instruction information 113, 132 Hand 130 Screen

The present invention comprises a non-visualization information embedding device for embedding non-visualization information at a predetermined position of an acquired image, and a recognition device for recognizing an object and non-visualization information included in the image obtained by the embedding device. In the content providing system using non-visualization information, the embedding device includes an embedding target object setting unit that sets an object in which the non-visualization information is embedded from an object included in the acquired image, and the embedding target object setting unit. A non-visualization information embedding unit that embeds the non-visualization information corresponding to the object around the object obtained by the step, and the recognition device is surrounded by an embedding region of the non-visualization information included in the image An object extracting means for extracting an object, and the object extracting means by the object extracting means; When the object is extracted, a non-visualization information analysis unit that analyzes the processing content of the object is generated from the de-visualization information, and an object that is displayed on the screen corresponding to the processing content obtained by the non-visualization information analysis unit it has a display information generating means for you characterized.

Further, the present invention provides an invisible information embedding apparatus for embedding invisible information at a predetermined position of an acquired image, an image analysis means for acquiring an object included in the image and position information of the object, and the image analysis The object obtained by the means is obtained by the image analysis means for the embedding target object setting means for setting the object to be embedded of the invisible information and the object obtained by the embedding target object setting means. based on the position information of the object, around the object, and having a non-visible information embedding means for embedding an invisible information for object extraction corresponding to the object.

In the recognition apparatus for recognizing an object and non-visualization information included in an acquired image, the object extraction means for extracting an object surrounded by an embedding area of the non-visualization information included in the image, and the object When the object is extracted by the extracting means, the non-visualized information analyzing means for analyzing the processing content for the object from the non-visualizing information and the processing content obtained by the non-visualizing information analyzing means are displayed on the screen. Display information generating means for generating an object to be generated.

Claims (15)

Non-visualization information comprising an invisible information embedding device for embedding non-visualization information at a predetermined position of an acquired image, and a recognition device for recognizing the object and non-visualization information included in the image obtained by the embedding device. In the content provision system used,
The implanting device comprises:
An embedding target object setting means for setting an object for embedding non-visualization information from an object included in the acquired image;
A non-visualization information embedding unit for embedding the non-visualization information corresponding to the object around the object obtained by the embedding target object setting unit,
The recognition device is
An object extracting means for extracting an object from an embedding area of non-visualization information included in the image;
When the object is extracted by the object extraction means, from the non-visualization information, non-visualization information analysis means for analyzing the processing content for the object;
A content providing system comprising: display information generating means for generating an object to be displayed on the screen in correspondence with the processing content obtained by the invisible information analyzing means. In the non-visualization information embedding device for embedding the non-visualization information at a predetermined position of the acquired image,
Image analysis means for acquiring object and position information included in the image;
An embedding target object setting unit for setting an object to be embedded by the image from the object obtained by the image analysis unit;
An embedding apparatus comprising: non-visualization information embedding means for embedding the non-visualization information corresponding to the object around the object obtained by the embedding target object setting means. Embedded information setting means for setting the embedded contents of the invisible information embedded by the invisible information embedding means;
3. The embedding apparatus according to claim 2, further comprising embedding information generation means for generating invisible embedding information from the embedding information set by the embedding information setting means. The embedded information setting means includes:
4. The embedding apparatus according to claim 3, wherein the form of the invisible information is a two-dimensional code, and the code part of the two-dimensional code is composed of a low-frequency part and / or a high-frequency part for the brightness of the original image. . The invisible information embedding means is
5. The embedding device according to claim 2, wherein the invisible information corresponding to the object is synthesized based on the position information of the object acquired by the image analysis unit. In a recognition device for recognizing objects and non-visualization information included in an acquired image,
An object extracting means for extracting an object from an embedding area of non-visualization information included in the image;
When the object is extracted by the object extraction means, from the non-visualization information, non-visualization information analysis means for analyzing the processing content for the object;
A recognition apparatus comprising: display information generating means for generating an object to be displayed on the screen in correspondence with the processing content obtained by the invisible information analyzing means. The object extraction means includes
The recognition apparatus according to claim 6, wherein the image is subjected to a filtering process using a predetermined frequency, and non-visualized information is extracted from a region corresponding to the acquired frequency. In the non-visualization information embedding method of embedding the non-visualization information at a predetermined position of the acquired image,
An image analysis step of acquiring object and position information included in the image;
An embedding target object setting step in which the image is set as an embedding target object from the object obtained by the image analysis step;
A non-visualization information embedding step of embedding the non-visualization information corresponding to the object around the object obtained by the embedding target object setting step. An embedding information setting step for setting an embedding content of the invisibility information embedded in the invisibility information embedding step;
9. The embedding method according to claim 8, further comprising a non-visualized information generating step of generating invisible embedded information from the embedded information set by the embedded information setting step. The embedded information setting step includes:
The embedding method according to claim 9, wherein the form of the invisible information is a two-dimensional code, and the code part of the two-dimensional code is composed of a low frequency part and / or a high frequency part for the brightness of the original image. . The invisible information embedding step includes:
The embedding method according to any one of claims 8 to 10, wherein non-visualization information corresponding to the object is synthesized based on the position information of the object acquired by the image analysis step. In a recognition method for recognizing an object and non-visualization information included in an acquired image,
An object extraction step of extracting an object from an embedded region of the invisible information included in the image;
When the object is extracted by the object extraction step, from the non-visualization information, a non-visualization information analysis step of analyzing a processing content for the object;
And a display information generating step for generating an object to be displayed on the screen in correspondence with the processing content obtained by the invisible information analyzing step. The object extraction step includes:
The recognition according to claim 12, wherein the image is filtered using a predetermined frequency, non-visualization information is extracted from a region corresponding to the acquired frequency, and the extracted region is a predetermined pixel. Method.   The embedding program for functioning a computer as each means which the embedding apparatus of any one of Claim 2 thru | or 5 has.   The recognition program for functioning a computer as each means which the recognition apparatus of Claim 6 or 7 has.