JP2000057327A - Image related information supply system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image related information supply system which needs no intervention of a special data base system and also never disturbs the appreciation of art for a user. SOLUTION: This system includes a CPU 1, an image recorder 2 comprising a hard disk device, a CD-ROM device, a DVD digital video disk), etc., and records the image of an object of art, an input part 3 where a user inputs a prescribed instruction or the data to the CPU 1, a display 4 consisting of a liquid display device, etc., which is controlled by the CPU 1 and shows a prescribed image and its related information recorded by the recorder 2, and a speaker 5 which is controlled by the CPU 1 and outputs the prescribed voice information recorded by the recorder 2 in voices. Then the recorder 2 records the image information and also its related information that is embedded into the image information as an 'electronic watermark' and explains the image information.

Description

DETAILED DESCRIPTION OF THE INVENTION

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method of displaying related information such as characters or sounds (speech or sound) while displaying an image.
The present invention relates to an image-related information providing system that can output an image.

[0002]

2. Description of the Related Art Hitherto, a database system has been put to practical use which outputs related information in characters while displaying an image. In this system, certain image data is associated with character data as commentary data, recorded and recorded as a database. That is,
When a certain image is displayed and the display of the related information is desired, the related data is called from the database and is displayed in characters or the like.

Recently, accompanying information, detailed information,
It has been proposed to embed (add) relevant information (for example, the place name of the place reflected in the background of the live-action video and detailed information such as the store occupying each floor of the building reflected in the background). (This proposal was made on May 19, 1998.
The article is introduced in the article of the Nikkei Sangyo Shimbun dated, "The University of Tokyo Institute of Industrial Science Live-action Screen, Used to Compare Locations and Provide Information Related to Identification" (This article begins with a statement that "Masao Sakauchi of the Institute of Industrial Science, The University of Tokyo has developed a basic system that can guess the location and the name of a building from a building that is shown on a live-action screen such as a landscape.) .

[0004]

In the above-mentioned method of recording image data and character data of related information in the database in association with each other, when outputting certain related information, the related data must be output via the database system without fail. Information must be called. In other words, not only "image data" but also a "database system (a database system having a certain scale) including the image data, related character data, and a computer program for database" is required. Become.

[0005] The system introduced in the above article of the Nikkei Sangyo Shimbun dated May 19, 1998, “Tokyo University of Agriculture and Technology's live-action screen, location comparison and use for providing relevant information, etc.” The content is to add (embed) data or place name data.
The meaning of that "add (embed)" is not clear from this article. However, the photo attached to this article ("Nogizaka"
It is expected that, when the image is displayed, a place name or the like is displayed in the display image. In other words, the system introduced in this article simply consists of simply adding related information characters and the like to the image, and whenever the image is displayed, the related information etc. characters are automatically added together It will be displayed. However, it is not a problem that the related information such as the place name and the building name is always displayed when the image is displayed as long as the image is displayed for a practical purpose such as a map for navigation. When displaying images for the purpose of appreciating a video of a work of art, if related information such as text that is not a work of art is automatically displayed, it will hinder the user's appreciation. If it is displayed on the screen, it will detract from the interest of art appreciation).

The present invention has been made in view of the above-described problems of the prior art, and does not require the intervention of a database system including a special computer program (software) for a database, and can be implemented by a user. It is an object to provide an image-related information providing system that does not hinder viewing.

[0007]

[Means for Solving the Problems] Image information recording means for recording image information, and, in the image information, related information related to the image (eg, commentary on the image);
A related information recording means for embedding as a “digital watermark” so as not to be normally output,
Image display means for displaying the image information, and when the image information is displayed, based on a user's selection / instruction (for example, based on a user's specific instruction), And a related information output means for outputting the related information as characters or sounds (by reading the information). 2. Image information recording means for recording image information, and a specific person in a single image information, a specific part image showing a specific part such as flora and fauna, natural objects such as mountains and rivers, buildings, or machinery and equipment A specific partial image identification recording means for recording so as to be distinguishable from an image of another part, and a specific partial image recorded by the specific partial image identification recording means, wherein a relation related to the specific partial image is included. Related information recording means for embedding information (such as a commentary text of the specific partial image) as an “electronic watermark” so as not to be normally output, and an image display for displaying the image information Means, when the image information is displayed, based on a user's selection / instruction (for example, based on a user's designation of a specific partial image and a specific instruction);
And a related information output means for reading out the related information of the specified specific partial image in text or sound (by reading the "digital watermark" information). 3. Image information recording means for recording image information, and in the image information, related music information related to the image (for example, background music suitable for the image) can not be normally output Related music information recording means for embedding as an “electronic watermark”, image display means for displaying the image information, and when the image information is displayed, the Related music information output means for outputting the related music information as a sound (by reading the “digital watermark” information) based on (for example, based on a user's specific instruction). Image related information providing system. The “related music information” of the present invention includes:
For example, “song information” suitable for BGM (background music) when viewing the image (for example, when the image is a painting with a calm and calm feeling, classical music with a calm and calm feeling) Etc.). Further, the "related music information" of the present invention is, for example, music information such as a Christian chant related to the content of the religious painting when the image is a religious painting depicting Christian Jesus Christ. is there. Further, the “related name music information” of the present invention is, for example, music information of a Christmas song when the image is a painting related to Christmas, and a Hawaiian song when the image is a painting depicting the Hawaiian Islands. Information on folk music. 4. Sender image information recording means for recording "sender image" information such as a live-action video or a character indicating the sender or its alter ego to be attached to an e-mail or chat conversation, and the sender image information In order to embed the relevant information about the sender, such as the sender's profile, background, self-introduction, or his / her recent status report, as a "digital watermark" so that it can not be output normally Sender-related information recording means, image display means for displaying the image information, and when the image information is displayed, based on selection / instruction of a recipient of the e-mail or chat conversation ( For example, based on a user's specific instruction, the sender-related information for outputting the sender-related information in text or sound (by reading the “digital watermark” information). Image-related information providing system characterized in that it comprises an output unit. That is, in the present invention, the “sender image” is a “mail sender or speaker” attached to a conversation of e-mail or chat (real-time chat developed in a virtual space on a computer communication network). Images such as photographs, face photographs, and alter ego characters (avatars). " In the present invention, "data for self-introduction and status report" is embedded as an electronic watermark in "images of photographs and avatars" attached to such e-mails and chats. This image can be transmitted on the Internet or the like together with an e-mail message. Then, the recipient who receives this e-mail, etc., reads the "self-introduction and recent status report data" embedded in the attached "image" and outputs it by character display or audio output etc. Can be.

In the above-described embodiments of the present invention, the term "image" includes not only real images such as photographs, but also virtual character images, animation images, illustrations, and the like. The “image” includes not only a still image but also a moving image. In addition, the term "related information" refers to commentary information describing a specific object such as a person, animal, plant, building, or machine in the image, detailed information indicating a profile of the person, etc. It is used to include information. That is, “related information” in the present invention includes, for example, (1)
If the image is a live-action photo, include the date and time when the image was actually taken, the location, the photographer, a commentary on the location, and the like.
If the image is a virtual image such as an animation or computer graphic (CG), include a description of the scene of the image (such as a game scene), an author name,
(3) If the image is a live-action photograph of a work of art, such as a painting or sculpture, the name of the creator, the name of the work, and a commentary on the creator (what age and who belong to any art school) And the description of the work (such as when and under what circumstances and circumstances the work was produced).

[0009]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiment 1 FIG. FIG. 1 is a schematic block diagram showing Embodiment 1 of the present invention. In FIG. 1, 1
Is a CPU (central processing unit), 2 is an image recording device for recording an image of a work of art (for example, a hard disk device, a CD-ROM device, a DVD (digital video disk) device, etc.), 3 is User is CPU
An input unit 4 for inputting predetermined instructions and data to 1 is controlled by the CPU 1 to display a predetermined image recorded on the image recording device 2 and related information (for example, a liquid crystal display device or the like). 5) CPU
1 is a speaker for outputting predetermined related information recorded in the image recording apparatus 2 under the control of the information recording apparatus 1 by voice or the like. The display 4 has a relatively large main screen 4a and a relatively small child screen 4b inside the main screen 4a.
It has a picture-in-picture function that can display both. In the first embodiment, the image recording device 2
Contains image information and “digital watermark” in this image information.
And related information for explaining the image information is recorded. Here, the information embedded as “digital watermark” is information that can be normally prevented from being output when an image is displayed, for example, characters on the screen only when a certain special process is performed. The information is output by display or sound output from a speaker (or an earphone or a bedphone).

Here, "digital watermark" will be described. “Digital watermarking” is currently (1) copyright protection (by embedding the author's ID information and the author's digital signature in images and music data, facilitating discovery of unauthorized copying and suppressing unauthorized copying) (2) Encrypted communication (embedding other information to be concealed in images and sounds to use as a kind of encrypted communication), (3) Prevention of data tampering (embed an electronic watermark in the image, It is used for four purposes and purposes: (4) it is possible to judge the presence or absence of falsification by removing the watermark) and (4) content identification.

There are various techniques for "digital watermarking" used for such various purposes from a technical point of view. That is, the “digital watermark” is “a general term for a technique for embedding some information in content”, and is technically an aggregate of various techniques. Among the various methods, the main ones at present are methods for adjusting screen brightness in still images and moving images,
There are two methods of using "frequency conversion" in music and the like.
Hereinafter, these two methods will be described. The above sentence and the following sentence are published in "Nikkei Multimedia 1998"
It is a quotation from May 81, p.81. "Embedding a digital watermark by changing the brightness of the screen is the most common method. A place to embed a digital watermark in an image is determined, and a change is made to a bit string representing the luminance of the place (pixel).
For example, there is a method of embedding a digital watermark in the least significant bit of a bit string (see FIG. 2. FIG. 2 is a citation from the above-mentioned “Nikkei Multimedia, May 1998, p. 80. 2 shows a method for embedding a digital watermark by changing the luminance of pixels of an image.As shown in FIG. 2, the luminance of each pixel is expressed by the number of bits. A bit string representing information is embedded as a “watermark.” In practice, more complicated processing is performed, such as expressing one bit by combining a plurality of pixels. If this process is performed at eight places on the screen, 8-bit information can be embedded. Moreover, there is almost no change in appearance, so that the presence of the watermark does not need to be noticed. Of course, the actual process is not so simple. This is because digital watermarks can easily be erased during digital compression. For this reason, when embedding 1-bit data, a slightly more complicated process such as expressing the 1-bit by combining luminance changes of a plurality of pixels is required. Use "frequency conversion" for music content. Frequency conversion is a method of decomposing a complex waveform such as sound into a plurality of simple waveforms called frequency components, and is also used as an element technology of digital compression. The digital watermark embeds information by modifying this frequency component. If the waveform is returned to the original waveform thereafter, the watermark information becomes negligible noise and will not be noticed by the viewer. In the case of an image as well, a change in screen luminance can be regarded as a waveform and subjected to frequency conversion.
For this reason, it is possible to embed a digital watermark in an image by frequency conversion, and NEC technology adopts this method. On the other hand, a digital watermarking technique that focuses on embedded information, such as cryptographic communication, uses a completely different technique. For example, Kowa's SteganoFAX98 digital watermarking technology
Relies on facsimile data compression techniques. The facsimile applies data compression by replacing the image data read by a scanning line with a format such as "how many black pixels from this point, and how many white pixels from the next." The Stegano FAX 98 embeds watermark information by adjusting the number of black and white pixels. In this way, digital watermarking technology varies widely and seems to have to be used for each application. (The above is quoted from Nikkei BP, Nikkei Multimedia, May 1998, p. 81)

The image recording apparatus 2 shown in FIG. 1 uses the above-mentioned “digital watermarking” technique to store information of each image in the information.
Relevant information for explaining the image is embedded. When the user instructs to display a real image of a certain painting in the image recording device 2 (when the instruction information is input from the input unit 3), the CPU 1 displays the corresponding painting image on the main screen 4a of the display 4. (In the present system, the related information is recorded as “digital watermark”, so that it is set in advance so that it is not normally displayed). Also, when this painting image is displayed, when the user inputs, from the input unit 3, instruction information indicating that the user wants the related information of this painting image (related information about the author and the work of this painting), The CPU 1 reads out the relevant information embedded as a “digital watermark” in the corresponding picture image in the image recording device 2 and displays the relevant information on the child screen 4 b of the display 4 by characters (see FIG. 1). ("XXX ..." in the child screen 4b indicates a character.)
Alternatively, sound is output from the speaker 5. The user can select whether to display the related information in characters or to output the sound. Further, the related information may be output as characters and as audio. When the related information is displayed in characters, the parent screen 4a is not displayed in the child screen 4b.
May be displayed in the.

Embodiment 2 FIG. FIG. 3 is a block diagram showing Embodiment 2 of the present invention. In FIG. 3, parts that are substantially the same as those in FIG. 1 are denoted by the same reference numerals. In FIG.
CPU 1, input unit 2, display 4, and speaker 5
Is a component of a personal computer (personal computer) used by a user. In FIG. 3, 11
Is the Internet (information communication network open to the public), 1
Reference numeral 2 denotes a transmission / reception unit (for example, composed of a modem or a terminal adapter) for connecting the CPU (personal computer) 1 to the Internet 11;
A server (network management computer) 14 connected to 1 is a large-capacity image recording device connected to the server 13. This image recording device 14 includes
Similarly to the image recording device 2, a large amount of image information and related information to be embedded as “digital watermark” in the image information are recorded.

In the second embodiment, the user accesses the server 13 and the image recording device 14 via the Internet 11 without accessing the recording device connected to the personal computer at hand. Then, the “image information of the painting and the related information embedded as a digital watermark in the painting image” recorded in the image recording device 14 are transmitted and downloaded via a network by a method such as download. The user first displays the painting image on the parent screen 4a of the display 4 from the obtained "painting image information and related information" (in the present system, the related information is referred to as "digital watermark"). Since it is recorded, it is set in advance so that it is not normally displayed). Next, when the user desires to know the related information of the displayed painting image, the user inputs instruction information to that effect. Then, based on the instruction information, the CPU 1 converts the related information embedded in the painting image into
Output by character display or voice output method.

Embodiment 3 Next, a third embodiment of the present invention will be described with reference to FIG. In FIG. 4, parts that are substantially the same as those in FIG. 1 are denoted by the same reference numerals. The main difference between the third embodiment and the first embodiment is that the recorded contents of the image recording device 22 are slightly different. That is, in the third embodiment, the image recording device 22 records “one image” and “related information (electronic watermark information) corresponding to the one image” displayed on the main screen 4 a of the display 4. In addition, the "image of a specific portion in one image" and "related information (electronic watermark information) corresponding to the image of the specific portion" are also recorded. That is,
Referring to FIG. 4 as an example, the image recording device 22 determines in advance a certain “image of a specific portion” (an image displayed on the entire main screen 4a) of “one image” (image displayed on the entire screen). The image 4c) of a certain person in the image displayed on the entire main screen 4a and the “image of another part” (the image of the background part and the part of the other person 4d) are separated from each other as data. (Extract only the specific partial image 4c from other parts and extract it), and in the separated (extracted) specific partial image, embedding and recording related information of the specific partial image by electronic watermarking. Like that. Therefore, when the user views the painting image recorded in the image recording device 22, the user first displays the desired painting image on the main screen 4a of the display 4 (in the present system, the related information is “electronic Since it is recorded as a "watermark", it is set in advance so that it is not normally displayed.) Then, when the user selects a certain person (for example, FIG. 4) in one image displayed on the parent screen 4a.
This person 4 wants to know
When the user wants to output the related information of c, the following is performed. That is, the user designates the specific partial image (image of the specific person) 4c in one image to be displayed (for example, as shown in FIG. 4, the arrow 4e of the cursor is moved to the position of the specific person 4c). And clicks with a pointing device such as a mouse), and performs an operation for outputting relevant information of the specific partial image (image of the specific person). Then, the CPU 1 displays the relevant information of the specific partial image 4c in characters on the child screen 4b or the like, or outputs the information by voice through the speaker 5.

Here, a technique for separating and extracting only a specific object such as a specific person, animal, plant, building, machine, or the like from a certain image will be described. Such a technique is already known. For example, 199
The Nikkei Sangyo Shimbun, dated December 7, 2007, introduced the following technologies in an article entitled "Automatic Separation of Still Image Processing from Subjects and Backgrounds" I have. In other words, "The Ministry of Posts and Telecommunications Research Institute and IBM Japan Tokyo Research Laboratories have developed technology to extract subjects such as people and objects in still images and automatically separate the background. It can be used as a part and used to search for images based on it, and can also be used to create content (content of information).
The image captured by the computer is divided into small blocks of about 6 pixels, similar ones are gathered little by little and reconstructed to extract the contours, and cut out the hair, clothes, objects, etc. of the person. The Nikkei Sangyo Shimbun, dated December 22, 1997, states, "Automatic classification of subjects by color and shape, such as NTT, for image data retrieval." Pictures "and" actions are used as keywords. "
In the article titled, the following same and similar technologies are introduced. The Ministry of Posts and Telecommunications Research Institute and IBM Japan have also developed technologies for realizing similar image retrieval. A subject such as a person or object in a still image is cut out and separated from the background. Specifically, it divides the image into small blocks, extracts contours based on color, brightness, texture, etc., and extracts the head, clothes, buildings, etc. of the person. "

[0017] Also, "Nikkei Electronics (issued from 1997 to 1998)" issued by Nikkei BP Inc. is titled "Matsushita Electric Industrial Developing Video Imaging Technology for Extracting Objects from Background (05/05/1998)". The following technology is introduced in the article (note that the following sentence is actually a Nikkei E
lectonics Online Service
This is a citation of the information obtained in). That is, "Matsushita Electric Industries, Ltd. announced on May 28, 1998 that it had developed a video imaging technology" dual modulation type finder technology "capable of cutting out only an image of a subject from the background. ISDB (Int
Egrated Services Digital B
In next-generation digital broadcasting such as roadcasting, an application based on an image divided for each subject is considered. The technology is geared toward this move.
The technology employs a technique that uses an infrared camera to capture depth information about the subject. First, a subject is irradiated with an infrared laser subjected to light intensity modulation. The illumination pattern is 2 types, one with strong light intensity on the right side of the subject and one with strong left side.
type. A subject is alternately irradiated with the two irradiation patterns, and an image is taken with an infrared camera combined with a video camera. The depth of the subject can be calculated from two types of images having different irradiation patterns. Based on this depth information, the subject is cut out from the background. (Omitted) Copyright (C)
1997 Nikkei Business Publi
sites, Inc. All rights res
erved "

As described above, only a specific object such as a specific person in “one still image” (theoretically, “moving image” can be applied) is separated from other parts such as the background. It is possible to extract. Then, it is possible to embed “related information describing the image of the specific portion” as digital watermark information in the “image of the specific portion” thus separated and extracted. Then, while displaying the “one still image” (the related information is recorded as “digital watermark”, the user can set in advance so that it is not normally displayed). To specify a specific partial image therein (for example, the arrow 4 of the cursor on the screen of the display 4).
e is moved to the position of the specific person image 4c desired by the user and a mouse or the like is clicked to specify the specific person image 4c), and the related information (“digital watermark”) corresponding to the specific partial image ") Is displayed in characters (displayed on the sub-screen 4b) or output as sound (output from the speaker 5)
Can be done.

[0019]

As described above, according to the present invention, a user can display a certain "one image" while displaying a certain "one image" without using a conventional database system using a database program (software). Can be output in text or voice. Moreover, since the related information can be set so as not to be output unless the user desires, it is possible to prevent the user from obstructing the art appreciation (as in the prior art, In the method of simply adding related information, the related information is always displayed together with the image when displayed, so that the display of the related information hinders the user's appreciation of the art.) Further, only the data of “image information (the relevant information is written as a digital watermark)” in which the relevant information is embedded as a digital watermark can be directly distributed (network distribution) through the network. Distribution becomes very simple. (Conventionally, if image information with related information was to be sold, it would be necessary to sell not only the image information but also the corresponding system such as a database program together. T). In the present invention, the program for outputting the related information as the digital watermark embedded in the image information (the program for reading the “digital watermark”) may be separately loaded into the user terminal. Therefore, the above-mentioned “image information (related information written as a digital watermark)”
Can be managed and distributed separately from the “program for outputting related information embedded as a digital watermark”. Therefore, the above-mentioned "image information (data in which related information is written as a digital watermark; data having such a" data structure ")" (which has very small capacity because it does not include a program) is distributed on a network. By doing so, the distribution of "image information with related information" on the network can be made very efficiently and easily. In particular, conventionally, “image information” and “related information related to the image” are “recorded” as separate files, and then the two files are associated with each other and “managed”. Or "send." On the other hand, in the present invention, as described above, the “related information relating to the image” is “embedded” as a digital watermark in the “image information”, so that the “image information” and “the The “related information related to the image” can be collectively “recorded” as “one file”. Therefore, in the present invention, “image information” and “related information related to the image” are referred to as “one file”.
Can be “managed” or “transmitted” (distributed on a network). Therefore, "image information"
And the management of "related information related to the image" and network distribution are greatly simplified and facilitated.
This applies not only to the invention of claim 1 but also to the inventions of claims 2 to 4). Further, according to the present invention, it is possible to output related information corresponding only to an image of a specific portion in the “one image” while displaying a “one image”. Therefore, for example, when the user wants to display related information of a certain person among a plurality of persons displayed in the image while displaying a certain painting image or a photographed image, for example, By simply designating the person and requesting the related information, the related information is output in characters or voice, which is very convenient. In the present invention, by embedding related music information in the image information, the user can output the related music information as a sound while displaying the image according to his / her desire (selection). Thus, for example, when an image is displayed for art appreciation, a BG suitable for the image is displayed.
Since M (background music) or the like can be played as related music information, the effect of art appreciation can be enhanced. Further, according to the present invention, the sender can embed relevant information such as self-introduction and status report in the “sender image” attached to an e-mail or chat conversation, and attach it to an e-mail or the like and send it. . Then, the recipient can output related information such as self-introduction of the sender and a status report from the sender image attached to the received e-mail or the like in text or voice according to his / her request (selection). it can. Therefore, according to the present invention, communication between individuals or companies, such as e-mail and chat, can be enhanced.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a first embodiment of the present invention.

FIG. 2 is a diagram for explaining a technique of “digital watermarking”.

FIG. 3 is a block diagram showing a second embodiment of the present invention.

FIG. 4 is a block diagram showing a third embodiment of the present invention.

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) // G06F 3/16 330 H04N 7/08 Z

Claims (4)

[Claims] 1. An image information recording means for recording image information, and outputs "related information related to the image" such as a commentary of the image in the image information when the user does not designate the image information. Related information recording means for embedding as a `` digital watermark '' so as not to display, image display means for displaying the image information, and when the image information is displayed, based on a user instruction, And a related information output means for outputting the related information as characters or sounds. 2. An image information recording means for recording image information, and a specific part such as a specific person, a flora and fauna, a natural object such as a mountain or a river, a building, or a machine tool in one image information. The specific partial image shown, a specific partial image identification recording means for recording so as to be distinguishable from the image of other parts,
In the specific partial image recorded by the specific partial image identification recording unit, `` related information related to the specific partial image '' such as a commentary of the specific partial image is not output unless the user gives an instruction. Related information recording means for embedding the image information as an “electronic watermark”, image display means for displaying the image information, and when the image information is displayed, the designation based on a user instruction. And a related information output means for outputting the related information of the specified partial image as characters or sounds. 3. A user designates image information recording means for recording image information, and "related music information related to the image" in the image information, such as background music suitable for the image. Related music information recording means for embedding as an “electronic watermark” so as not to be output when not performed, image display means for displaying the image information, and a user instruction when the image information is displayed. And a related music information output unit for outputting the related music information as a sound based on the related information. 4. Sender image information recording means for recording "sender image" information such as a photographed image or a character indicating a sender or its alter ego to be attached to an e-mail or chat conversation, and the transmission When the recipient of the e-mail or chat conversation does not specify `` relevant information about the sender '' in the sender image information, such as the sender's profile, career, self-introduction, or self-status report. Sender-related information recording means for embedding as an “electronic watermark” so as not to be output, image display means for displaying the image information, and when the image information is displayed, the e-mail or chat And a sender-related information output means for outputting the sender-related information in text or sound based on an instruction of a recipient of the conversation. Image related information providing system. [0001]