JP2004133721A - System and device for creating haiku (seventeen-syllabled poem) - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide new system and device for creating Haiku, which create Haiku on the basis of images. <P>SOLUTION: A Haiku creating system 10 includes a camera 12, and the camera 12 is connected to a Haiku creating device 30 through a network 16. The camera 12 not only photographs a desired image in accordance with user's operation but also acquires photographic information like a photographing place of the image. The image photographed in accordance with user's operation and photographic information are transmitted to the Haiku creating device 30. In response to reception of the image and photographic information, the Haiku creating device 30 analyzes the image to extract features (luminance information and color information) of the image and selects a season word and words (sensitivity words or independent words) for use in a Haiku from features of the image and photographic information and takes out phrases (Haiku materials) including the selected season word and words from a Haiku DB 40 and combined taken-out Haiku materials to create a Haiku so as to meet restrictions on Haiku composition. Thus the Haiku is created on the basis of the image. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
[Industrial applications]
The present invention relates to a haiku generation system and a haiku generation device, and more particularly to, for example, a haiku generation system and a haiku generation device in which a camera having a communication function and a haiku generation device are connected via a communication network.
[0002]
[Prior art]
An example of this type of prior art is disclosed in Patent Document 1. In this prior art sentence generation device, it is determined whether or not an input natural language sentence matches a selected sentence rhythm pattern, and if not, a natural language sentence is matched so as to match the sentence rhythm pattern. It is output with some parts replaced.
[0003]
[Patent Document 1]
JP-A-7-160709 (pages 10 and 11, FIG. 14)
[0004]
[Problems to be solved by the invention]
However, in this conventional technique, the input is a natural language sentence, that is, a text, and a sentence cannot be generated from data (multimedia data) other than text data. On the other hand, studies are being conducted to generate text from multimedia information, such as expressing stock price fluctuations in the stock market in text or generating weather summary sentences from weather maps, but other methods are used to express text. There was no system or device that simply converted the content into text and generated text with different new content from multimedia information such as images (video).
[0005]
Therefore, a main object of the present invention is to provide a new haiku generation system and a new haiku generation device capable of generating a haiku based on an image.
[0006]
[Means for Solving the Problems]
A first invention is a haiku generating system in which a camera having a communication function and a haiku generating device are connected via a communication network, wherein the camera specifies a photographing means for photographing a desired image, at least a photographing place of the image. Photographing information acquiring means for acquiring photographing information including position information to be transmitted, and transmitting means for transmitting the image and the photographing information to the haiku generating device via a communication network, wherein the haiku generating device transmits the image and the image transmitted by the transmitting means. Receiving means for receiving the photographing information, analyzing means for analyzing the image received by the receiving means, extracting means for extracting image features from the analysis result of the analyzing means, image features extracted by the extracting means and receiving by the receiving means Extracting means for extracting haiku material from the haiku material database based on the obtained photographing information, and haiku material extracted by the extracting means Using comprises generating means for generating a haiku with a haiku generation system.
[0007]
A second invention is a haiku generation device that generates a haiku based on an image captured by a camera, an input unit that inputs an image and photographing information, an analysis unit that analyzes the image input by the input unit, and an analysis unit. Extracting means for extracting the features of the image from the analysis result of the image, extracting means for extracting the haiku material from the haiku material database based on the image characteristics and the photographing information extracted by the extracting means, and haiku material extracted by the extracting means. A haiku generating apparatus comprising a generating means for generating a haiku.
[0008]
[Action]
In the haiku generation system of the first invention, a camera having a communication function and a haiku generation device are connected via a communication network. The camera can take a desired image by the photographing means. At this time, shooting information including at least position information for specifying the shooting location is obtained. Such images and photographing information are transmitted to the haiku generator. Upon receiving the image and the photographing information, the haiku generation device analyzes the image and extracts features of the image. The extracting means extracts a haiku material from the haiku material database based on the extracted image features and shooting information. Then, the generating means generates a haiku using the taken haiku material.
[0009]
For example, the camera acquires shooting direction information at the time of shooting an image, and the haiku generation device estimates a shooting target based on the position information and shooting direction information included in the shooting information. Further, the photographing target is specified from the features of the image extracted by the extracting means. Then, the imaging target estimated by the estimation unit and the imaging target specified by the identification unit are integrated. That is, the photographing target is verified from different aspects, and the matching of the photographing target is achieved.
[0010]
Further, the haiku generating apparatus is provided with an image database for recording a plurality of still images to which additional information including luminance information, color information, composition information, and object information are added, respectively. The information and color information are analyzed, and the extraction unit determines the similarity between the luminance information and color information obtained from the analysis result of the analysis unit and the luminance information and color information included in the additional information of the still image recorded in the image database. Then, the composition information and the object information included in the additional information having the maximum similarity are extracted. Therefore, the specifying unit can specify the shooting target of the image based on at least one of the composition information and the object information extracted by the extracting unit.
[0011]
Furthermore, since the camera is provided with a clock circuit, it is possible to acquire information on the shooting date and time of the image (date and time information). This date and time information is included in the shooting information. In the haiku generating device, the first determining means determines a seasonal word corresponding to at least the season specified from the date and time information. In addition, the second determining means determines a phrase to be used in a haiku based on at least one of image characteristics (luminance information and color information) and photographing information (such as photographing place). The selecting means selects, from the haiku material database, a phrase including at least one of a seasonal word determined by the first determining means and a phrase determined by the second determining means, that is, a haiku material. In other words, you can select a haiku material based on the image.
[0012]
For example, the haiku generation device includes a user information database that records user information including at least information indicating the user's preference, so that the first determination unit can narrow down seasonal words determined based on at least date and time information according to the user information. it can. That is, the seasonal word that the user likes can be selected.
[0013]
The combination means combines the haiku materials extracted by the extraction means so as to satisfy the rules of the haiku composition to generate a haiku. In other words, the haiku is generated so as to follow the pattern of 5.7.5, and include the seasonal word in any one of the phrases.
[0014]
Furthermore, if the haiku generated by the generating means matches an existing haiku, the generated haiku is deleted, so that only a new haiku can be generated.
[0015]
A haiku generation device according to a second aspect of the present invention is configured such that an image or photographing information once photographed by a camera is input online or offline without connecting to a camera having a communication function via a communication network. The same configuration as the above-described haiku generation device can be used.
[0016]
However, in the haiku generation device of the second invention, if the user inputs the photographing information, it is not necessary to provide the camera with photographing information acquisition means. In such a case, it is possible to make the overall configuration cheaper than the haiku generation system described above.
[0017]
【The invention's effect】
According to the present invention, a haiku is generated by extracting a haiku material acquired based on the characteristics of the image and the photographing information, so that the haiku can be generated based on the image. That is, text expressing different contents can be generated from multimedia information such as an image.
[0018]
The above and other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.
[0019]
【Example】
Referring to FIG. 1, the haiku generation system (hereinafter, simply referred to as “system”) of this embodiment includes a camera 12, and a communication device 14 is connected to the camera 12. The camera 12 is connected to a communication device 18 via a communication device 14 and a communication network (network) 16 such as the Internet or a telephone network (including a mobile telephone network). A computer 32 provided in the haiku generator 30 is connected to the communication device 18. That is, the camera 12 having the communication function and the haiku generation device 30 are communicably connected via the network 16.
[0020]
Note that each of the camera 12 and the computer 32 may have a communication function therein.
[0021]
The haiku generator 30 includes a computer 32 such as a personal computer or a workstation, to which various databases (DB) are connected. Specifically, the user information DB 34, the image DB 36, the knowledge DB 38, the haiku DB 40, and the map DB 42 are connected to the computer 32.
[0022]
Note that these DBs 34 to 42 need not be directly connected to the computer 32 but may be connected via the network 16 as described above.
[0023]
As shown in FIG. 2A, information (user information; profile information and preference information) about a user (user) who generates a haiku using the haiku generating device 30 is stored in the user information DB 34 for each user. Be recorded. For example, the profile information includes information such as a user ID (identification information), name, gender, and age. Also, the preference information includes information such as a haiku type, a seasonal word, and a cut character that the user prefers.
[0024]
Such user information is information input at the time of user registration, and although not shown, is input using an input device such as a keyboard or a computer mouse connected to the computer 32, or is input via the network 16. It can be input using another connected terminal (including the camera 12).
[0025]
In the image DB 36, a plurality of image data are recorded, and each image data is classified into categories. Specifically, the categories are classified based on the brightness (luminance), hue (saturation), composition (natural landscape, person + natural landscape (background), person, artifact) of the image data. In the embodiment, labels (additional information) of luminance information, color information, composition information, and object information are added to each image data. That is, as shown in FIG. 2B, a plurality of image data files (image files) to which labels have been added are recorded in the image DB 36.
[0026]
For example, image data can extract image features such as luminance distribution and saturation distribution from pixel information obtained by mapping luminance values and color information into a feature space. As a method of extracting such a feature of an image, for example, an information processing society “humanities and computers” study group, '94 Vol. 24, pp. 9-16, "Extraction of Kansei Information in Painting-Extraction of Background Color and Main Color-" and IPSJ Transactions '99 Vol. 40, no. 03, pp. The method disclosed in “Image Search Using Kansei Words and Evaluation of Its Accuracy” at 886-898 can be applied.
[0027]
The composition information and the object information are information added by a creator or a developer of the image DB 36. In other words, the creator and the like describe the composition, that is, the natural landscape, the person + the natural landscape, the person, and the artificial object, and also describe the object such as the natural landscape and the artificial object included in the composition. In this embodiment, the objects correspond to, for example, “mountain”, “river”, “sea”, “town”, “temple” and the like, and are relatively roughly classified.
[0028]
History information and sightseeing information are registered in the knowledge DB 38. The history information and the sightseeing information are names of representative things such as place names, scenery, temples and shrines, and can be specified in relation to seasonal words. In other words, as shown in FIG. 3A, the names of representative things and seasonal words (seasonal data) recorded in a haiku DB 40 described later are described in the knowledge DB 38 in association with each other.
[0029]
Further, in the knowledge DB 38, data of a plurality of phrases corresponding to an independent word and a feeling word (impression word) are recorded. Here, the independent word means a word having a content (meaning) such as a noun, a verb, or an adjective, and excludes a word such as a conjunction, a particle, or an auxiliary verb. For example, words such as ““ temple ”,“ ○ grassland ”,“ thunder ”,“ sky ”,“ listening ”,“ sound ”,“ bright ”, and“ dark ”correspond to the independent words. This independent word includes the name of an object as described above. In addition, the kansei word refers to a word that is called up by a sense and represents the content of human experience dominated by the sense, and for example, a word corresponding to an adjective or an adverb corresponds thereto. For example, sentiment words (impression words) include words such as “wide”, “narrow”, “large”, “large”, “small”, “small”, “pretty”, “pretty”, and “ugly”. Applicable.
[0030]
As shown in FIG. 3 (A), each of the self-sustained word and the kansei word has the above-described image features (luminance information and color information) and a shooting environment (a shooting target, a shooting location, a shooting target, The label of “season and weather at the time of shooting” is added, and can be specified based on the characteristics of the image and the shooting environment.
[0031]
That is, the knowledge DB 38 can be said to be a dictionary DB for acquiring phrases (including seasonal words) used in haiku.
[0032]
As shown in FIG. 3B, the haiku DB 40 records a plurality of text data of existing haikus, and each text data is divided according to a pattern of 5.7.5. Characters), information indicating the middle phrase (next 7 characters) and the lower phrase (last 5 characters) are added. The reason why the pattern is divided according to the pattern is to use it as a material for generating a haiku (haiku material). Such a haiku material is added or changed by the creator or designer of the haiku DB 40.
[0033]
The haiku DB 40 records a plurality of text data (seasonal data) corresponding to typical seasonal words. More specifically, as shown in FIG. 3B, a plurality of seasonal data is described for each season, divided into seasons (spring, summer, autumn, winter). That is, a seasonal word can be selected based on a season (month and day). However, the seasons are divided into details such as Miharu (Early Spring / Middle Spring / Late Spring), Mitsatsu (Early Summer / Midsummer / Late Summer), Miaki (Early Fall / Middle Autumn / Late Fall), and Miwate (Early Winter / Middle Winter / Late Winter). May be described.
[0034]
In the map DB 42, map data for a specific area (in this embodiment, all over Japan) is recorded, and a place (position) can be specified based on latitude and longitude or an address. In addition, a shooting target (an object on a map) can be estimated from the specified position (the shooting location in this embodiment) and the direction of the camera 12 at the time of shooting (azimuth information and tilt information described later). .
[0035]
FIG. 4 is a block diagram showing an electrical configuration of the camera 12. Referring to FIG. 4, the camera 12 includes a lens 50, and a light image of a subject incident from the lens 50 is electrically converted by a CCD imager 52. Converted to a signal. The CCD imager 52 has, for example, color filters in a primary color Bayer array, and outputs an electric signal (camera signal) of each pixel according to a progressive scan (pixel sequential scan) under the instruction of the CPU 70.
[0036]
The camera signal output from the CCD imager 52 is supplied to a CDS / AGC circuit 54. The CDS / AGC circuit 54 performs well-known noise removal and level adjustment on the camera signal from the CCD imager 52. The camera signal processed by the CDS / AGC circuit 54 is converted by an A / D converter 56 into digital data (camera data). The camera data output from the A / D converter 56 is supplied to a signal processing circuit 58.
[0037]
The signal processing circuit 58 performs processing such as color separation, white balance adjustment, and YUV conversion on the camera data supplied from the A / D converter 56 in accordance with a command signal from the CPU 70, and converts YUV data corresponding to the light image of the subject into image data. Generate. The generated YUV data is written by the memory control circuit 60 to a RAM 62 such as an SDRAM. On the other hand, the video encoder 64 requests the memory control circuit 60 to read the YUV data in response to the processing instruction from the CPU 70, and converts the YUV data read by the memory control circuit 60 into an NTSC composite image signal. . The converted composite image signal is provided to the monitor 66, and as a result, a real-time moving image (through image) of the subject is displayed on the monitor 66.
[0038]
In the photographing mode, when the user operates (turns on) a shutter button (not shown) provided on the operation panel 86 at a desired timing, a photographing instruction is given to the CPU 70 from a system controller (hereinafter simply referred to as “syscon”) 78. Can be Then, the CPU 70 executes a photographing process.
[0039]
The CPU 70 instructs a timing generator (TG) (not shown) to photograph a subject at an optimal shutter speed and to read out all photoelectrically converted pixel signals. Thus, a high-resolution camera signal corresponding to the still image at the time when the shutter button is turned on is output from the CCD imager 52. The output camera signal is subjected to the same processing as described above, and high-resolution YUV data is temporarily stored in the RAM 62.
[0040]
Subsequently, the CPU 70 gives a compression instruction to a compression / expansion circuit 72 such as a JPEG codec. The compression / expansion circuit 72 requests the memory control circuit 60 to read the YUV data in response to the compression instruction. The memory control circuit 60 reads out the YUV data stored in the RAM 62, that is, the still image data for one screen, and supplies it to the compression / expansion circuit 72. The compression / expansion circuit 72 compresses the still image data according to the JPEG format, and writes the compressed image data (compressed image data) to the flash memory 76 under the instruction of the CPU 70.
[0041]
In this embodiment, the compressed image data is recorded in the flash memory 76. However, the compressed image data may be recorded in a removable recording medium such as a memory card or an optical (magnetic) disk.
[0042]
Further, when an imaging instruction is given from the system controller 78, the CPU 70 instructs the system controller 78 to acquire imaging information. Here, the photographing information includes positional information (latitude and longitude in this embodiment) of the camera 12 at the time of photographing, information on the left-right (turning) direction of the camera 12 (lens 50) (azimuth information), and information on the up-down direction of the camera 12. The information includes information on the direction (tilt) (tilt information), information on the temperature at the time of shooting (temperature information), information on the humidity at the time of shooting (humidity information), and date and time information indicating the date and time of shooting.
[0043]
That is, in the shooting mode, the CPU 70 executes the above-described shooting process in response to the shooting instruction, and instructs the system controller 78 to obtain shooting information. In response, the system controller 78 detects the outputs of the GPS 80 (including the antenna), the azimuth sensor 82, the tilt sensor 84, the temperature sensor 88, the humidity sensor 90, and the clock circuit 92. Data (photographing information data) is obtained and given to the CPU 70.
[0044]
In this embodiment, the photographing information data provided to the CPU 70 is text data.
[0045]
The CPU 70 records the photographing information data given from the system controller 78 in the EEPROM 68 in association with the compressed image data recorded in the flash memory 76. In other words, the compressed image data and the associated shooting information data can be managed with the same file name and different extensions. For example, the compressed image data can be managed by giving a file name such as “FILE01.jpg” and the associated shooting information data with a file name such as “FILE01.txt”.
[0046]
Furthermore, information on the weather at the time of shooting at the shooting location (weather information) can be added as the shooting information. For example, the user can input (add) weather information by operating the operation panel 86.
[0047]
In the playback mode, a thumbnail image or a file name of the compressed image data is displayed on the monitor 66, and the user operates a cursor key (not shown) provided on the operation panel 86 to select a desired image. Can be. When a desired image (thumbnail image or file name) is selected and a play button (not shown) provided on the operation panel 86 is operated, a reproduction instruction for the desired image is given from the system controller 78 to the CPU 70, In response, the CPU 70 reads out the compressed image data corresponding to the selected image from the flash memory 76.
[0048]
Subsequently, the CPU 70 gives a decompression instruction to the compression / decompression circuit 72 and supplies the read compressed image data to the compression / decompression circuit 72. The compression / decompression circuit 72 performs a decompression process on the compressed image data according to the JPEG format, and gives a write instruction to the memory control circuit 60. Then, the image data subjected to the decompression processing is stored in the RAM 62 by the memory control circuit 60.
[0049]
Then, the CPU 70 gives a processing instruction to the video encoder 64. In response, the video encoder 64 requests the memory control circuit 60 to read the YUV data, and converts the YUV data read by the memory control circuit 60 into an NTSC composite image signal. The converted composite image signal is provided to the monitor 66, and the selected desired photographed image (still image) is displayed on the monitor 66.
[0050]
Further, in the transfer mode, the thumbnail image or the file name of the compressed image data is displayed on the monitor 66 as in the case of the reproduction, and a desired image can be selected. Further, the user can operate the operation panel 86 to input personal information. In this embodiment, for example, a user ID included in the user information as described above is input.
[0051]
Subsequently, when a transfer button (not shown) provided on the operation panel 86 is operated, the system controller 78 inputs a transfer instruction to the CPU 70. In response to this, the CPU 70 sets the previously input user ID, the compressed image data corresponding to the selected image, and the shooting information data related to the compressed image data as a set of transfer data via the interface 74. To transfer. The transfer data is transferred to the haiku generator 30 (computer 32) via the communication device 14, the network 16, and the communication device 18, as shown in FIG.
[0052]
Each of the shooting mode, the reproduction mode, and the transfer mode is set by operating a mode selection button (not shown) provided on the operation panel 86.
[0053]
Specifically, the CPU 70 of the camera 12 performs processing according to the flowchart shown in FIG. As shown in FIG. 5, when the main power supply of the camera 12 is turned on, the CPU 70 starts processing, and determines whether or not the camera is in a shooting mode in step S1. If “NO” in the step S1, that is, if the mode is not the shooting mode, the process directly proceeds to a step S11. On the other hand, if “YES” in the step S1, that is, if it is the shooting mode, it is determined whether or not the shutter button is turned on in a step S3.
[0054]
If “NO” in the step S3, that is, if the shutter button is not turned on, the process returns to the same step S3 as it is. On the other hand, if “YES” in the step S3, that is, if the shutter button is turned on, the photographing process as described above is executed in a step S5 in response to this.
[0055]
Next, in step S7, photographing information data is obtained. That is, it instructs the system controller 78 to acquire the photographing information. Then, as described above, the system controller 78 detects the outputs of the GPS 80, the azimuth sensor 82, the inclination sensor 84, the temperature sensor 88, the humidity sensor 90, and the clock circuit 92, and obtains position information (longitude, latitude), The azimuth information of the camera 12 (lens 50), the inclination information of the camera 12 (lens 50), the temperature, the humidity, and the date and time are acquired. Then, in step S9, the acquired photographing information data is recorded in the EEPROM 68 in association with the compressed image data of the photographed image, and the process ends.
[0056]
In step S11, it is determined whether the mode is the transfer mode. If “NO” in the step S11, that is, if the mode is not the transfer mode, the process directly proceeds to a step S21. On the other hand, if “YES” in the step S11, that is, if it is the transfer mode, it is determined whether or not an image is selected in a step S13. If “NO” in the step S13, that is, if the image to be transferred is not selected, the process returns to the same step S13 as it is.
[0057]
On the other hand, if “YES” in the step S13, that is, if an image to be transferred is selected, it is determined whether or not a user ID (user information) is input in a step S15. If “NO” in the step S15, that is, if the user ID is not input, the process returns to the same step S15 as it is. On the other hand, if “YES” in the step S15, that is, if the user ID is input, it is determined whether or not the transfer button is turned on in a step S17.
[0058]
If “NO” in the step S17, that is, if the transfer button is not turned on, the process returns to the step S17 and waits until the transfer button is turned on. On the other hand, if “YES” in the step S17, that is, if the transfer button is turned on, in a step S19, the compressed image data corresponding to the selected image is read out from the flash memory 76 and associated with the compressed image data. The photographing information data to be recorded is read out from the EEPROM 68, and transferred to the haiku generator 30 via the network 16 together with the previously input user ID, and the process is terminated. That is, the transfer data is transferred to the computer 32.
[0059]
In step S21, it is determined whether the mode is the reproduction mode. If “NO” in the step S21, that is, if it is not the reproduction mode, the process returns to the step S1 as it is. On the other hand, if “YES” in the step S21, that is, if it is the reproduction mode, it is determined whether or not an image is selected in a step S23. If “NO” in the step S23, that is, if the image to be reproduced is not selected, the process returns to the same step S23 as it is. On the other hand, if “YES” in the step S23, that is, if an image to be reproduced is selected, in the step S25, the above-described reproducing process is performed on the selected image, and the process ends.
[0060]
On the other hand, the computer 32 included in the haiku generating device 30 executes the haiku generating process according to the flowcharts shown in FIGS. As shown in FIG. 6, when starting the haiku generation process, the computer 32 determines whether or not data, that is, transfer data (user ID, compressed image data, and photographing information data) has been received in step S31. If “NO” in the step S31, that is, if the transfer data is not received, the process returns to the same step S31 as it is.
[0061]
On the other hand, if “YES” in the step S31, that is, if the transfer data is received, the user information is read in a step S33. Specifically, the user information recorded in the user information DB 34 is read with reference to the user ID included in the transfer data. In a succeeding step S35, the received image data is analyzed. That is, the received compressed image data is expanded according to the JPEG format, and the luminance distribution and the saturation distribution of the image (captured image) are analyzed.
[0062]
Then, in step S37, the features of the image are extracted. That is, the similarity to the image data recorded in the image DB 36 is calculated from the analysis result (luminance distribution, saturation distribution) of the captured image, and the matching or the closest image data is specified from the calculation result, and the specified image data is specified. The composition information and the object information described in the label of the image data are acquired.
[0063]
Subsequently, in step S39, a shooting environment is created from the shooting information data. Here, the shooting environment refers to a shooting location, a shooting target, a season at the time of shooting, and weather (including temperature and humidity). Specifically, first, the position information (latitude and longitude) included in the shooting information data is input to the map DB 42, and the position of the camera 12 (shooting location) is specified. Next, when the azimuth information and the tilt information included in the shooting information data are input to the map DB 42, an object on the map (a shooting target) is estimated based on the shooting location, the direction information, and the tilt information specified earlier. . Next, the season is estimated from the date and time information included in the shooting information data and the specified shooting location. Further, the season is determined from the temperature information and the humidity information included in the photographing information data. At this time, a seasonal word corresponding to the determined season is acquired from the haiku DB 40.
[0064]
As described above, when the weather information is included in the shooting information, the weather at the time of shooting can be further added to the shooting environment information. Alternatively, it is also possible to acquire weather information provided on the Internet based on a shooting location and a shooting date and time.
[0065]
In the next step S41, the image capturing environment is integrated by matching the image data with the image capturing information data. Specifically, the imaging target specified by the composition information and the object information acquired in step S37 is integrated with the imaging target specified by the imaging location, azimuth information, and tilt information included in the imaging information data. For example, when the shooting target is specified as “mountain” from the composition information and the object information based on the image data, the shooting target estimated (specified) from the shooting location, the azimuth information, and the inclination information included in the shooting information data. In the case of “temple”, the shooting target specified from the image data does not match the shooting target specified from the shooting information data, so the shooting target is excluded from the shooting environment. Further, when the shooting target estimated from the shooting information data is “lakeside”, the shooting target specified from the image data does not match the shooting target specified from the shooting information data. In addition, the shooting target of the shooting environment can be determined as “mountain” and “lakeside”. In other words, when the shooting target is verified from different aspects of the image data and the shooting information data, the mismatching information is deleted or complemented with each other to achieve matching (optimization) of the shooting environment. It is.
[0066]
In such optimization, information that does not match is deleted or complemented with each other, but this is performed according to a rule determined in advance by a developer or a designer.
[0067]
In a succeeding step S43, an independent word and a feeling word are selected based on the user information, the feature of the image, and the photographing environment. That is, an independent word and a kansei word are selected from the knowledge DB 38 based on the characteristics of the image and the shooting environment, and the user's gender, age, favorite haiku type, etc. are selected from the selected independent word and the kansei word based on the user information. Independent words and sentiment words that match are selected.
[0068]
At this time, the names (independent words) of the objects included in the tourist information and the history information are also acquired based on the seasonal words acquired from the shooting environment.
[0069]
Then, in step S45, a haiku material is obtained. That is, a haiku material that includes a seasonal word acquired in step S39 or an independent word and a kansei word selected (selected) in step S43, or includes a word that is similar to (or similar to) the independent word or kansei word, that is, an upper phrase (5 characters) , The middle phrase (7 characters) and the lower phrase (5 characters) are acquired from the haiku DB 40.
[0070]
For example, in this embodiment, when the haiku material (phrase) is acquired based on the selected independent word or kansei word, the haiku material or kansei word is included in all the phrases recorded in the haiku DB 40. A score of the degree of similarity with the phrase is calculated, and a phrase including the phrase having a certain score or more is obtained. Here, the similarity refers to the degree to which the meanings of words and phrases are similar.
[0071]
Subsequently, in step S47, a haiku text that satisfies the restrictions on the haiku composition is generated according to a procedure described later in detail. In brief, the haiku text data (haiku text) is created by combining the phrases acquired in step S45 so as to follow the patterns 5, 7, and 5 and to include the seasonal word in any one of the phrases.
[0072]
In step S49 shown in FIG. 7, the generated first haiku is compared with the existing haiku. That is, one haiku is selected from the generated haikus and compared with the haiku recorded in the haiku DB 40. In a succeeding step S51, it is determined whether or not both are the same (match).
[0073]
If “NO” in the step S51, that is, if they do not match, the process directly proceeds to a step S55. On the other hand, if “YES” in the step S51, that is, if they match, in a step S53, the generated haiku is deleted, and the process proceeds to the step S55.
[0074]
In the step S55, it is determined whether or not the next haiku exists. If “YES” in the step S55, that is, if there is a next haiku, the next haiku is compared with the existing haiku in a step S57, and the process returns to the step S51. On the other hand, if “NO” in the step S55, that is, if there is no next haiku, the haiku is presented in the order of the score in a step S59, and the process is ended. That is, the generated haiku text data is transmitted to the camera 12 via the network 16.
[0075]
However, the generated haikus may be transmitted one by one in the score order, or all the haikus generated by attaching the numbers according to the score order to each haiku may be transmitted together.
[0076]
The text data of the haiku sent to the camera 12 is given to the CPU 70 via the interface 74, and is recorded in the EEPROM 68 from the CPU 70. Thereafter, when the user operates the operation panel 86 to input a haiku output instruction, the system controller 78 gives the haiku output instruction to the CPU 70. In response, the CPU 70 reads the text data of the haiku recorded in the EEPROM 68 and gives a processing instruction to the video encoder 64. Then, for example, a screen in which the haiku text is described on the blue-back screen is displayed on the monitor 66.
[0077]
However, instead of the blue background screen, the image from which the haiku was generated may be displayed on the monitor 66, and the image of the haiku may be overwritten on the image.
[0078]
When the process of generating a haiku text is started in step S47 in FIG. 6, as shown in FIG. 8, the computer 32 extracts a phrase candidate that can be an upper phrase from the haiku material extracted from the haiku DB 40 in step S71. In a succeeding step S73, the extracted upper phrase candidates are arranged in the order of score. That is, they are arranged in the order of the score of the degree of similarity to the independent word or the kansei word.
[0079]
Subsequently, in step S75, middle phrase candidates are extracted in descending order of similarity score. Then, in a step S77, a combination of the upper phrase and the middle phrase is generated. For example, in this embodiment, the upper phrase and the middle phrase are respectively combined in descending order of the similarity score. That is, the upper phrase candidate and the middle phrase candidate are combined like the first upper phrase and the first middle phrase, the second upper phrase and the second middle phrase, and so on. At this time, the upper and middle phrases are combined in this order.
[0080]
Then, in a step S79, the combinations of the upper phrase and the middle phrase are arranged in the order of score. However, here, a score (first score) obtained by adding the score of the similarity for each of the upper and middle phrases and a score (second connectivity) of the combination of the upper and middle phrases (good connectivity) The score (third score) is calculated by adding the upper phrase and the middle phrase in descending order of the third score.
[0081]
The second score is determined based on, for example, goodness of the connection between the meaning of the upper phrase and the middle phrase and the connection of the context, or whether or not the character includes a favorite character described in the user information (preference information). And is calculated according to a rule determined in advance by a developer or a designer.
[0082]
Next, in step S81, lower phrase candidates are extracted in descending order of the similarity score. Then, in step S83, a combination of an upper phrase, a middle phrase, and a lower phrase is generated. For example, in this embodiment, similar to the case where the upper phrase and the middle phrase are combined, the combination of the upper phrase and the middle phrase arranged in the descending order of the third score is displayed in the descending order of the similarity score. Concatenate the following phrases. At this time, the lower phrase is connected after the upper phrase / medium phrase.
[0083]
Then, in step S85, the combinations of the upper phrase, the middle phrase, and the lower phrase are arranged in the order of score. Here, a score (fourth score) obtained by adding the score of the similarity for each of the upper phrase, the middle phrase, and the lower phrase, and a score (second score) expressed by good connectivity between the upper phrase and the middle phrase, and the like. The score (sixth score) is calculated by adding the score (score) and the score (fifth score) represented by the good connectivity between the middle phrase and the lower phrase, and the upper phrase is determined in the descending order of the sixth score. A combination of a middle phrase and a lower phrase, that is, a plurality of haiku texts are arranged.
[0084]
When calculating the fifth score, as in the case of the second score, it is considered whether or not to include a favorite cut character described in the preference information.
[0085]
Finally, in step S87, the haiku text including two or more seasonal words is deleted from the generated haiku text, and the process of generating the haiku text is returned. Therefore, only the haiku text that satisfies the above-described haiku composition restrictions is generated.
[0086]
In this way, a plurality of haiku texts are generated. The combination of upper and middle phrases is reduced based on whether the third score is equal to or higher than a certain score, or based on whether the sixth score is equal to or higher than a certain score. It is also possible to reduce the generated haiku text. In addition, after a plurality of haiku texts are generated, the number of haiku texts may be adjusted by reducing the haiku texts having a low sixth score.
[0087]
According to this embodiment, a seasonal word or phrase used in a haiku is selected based on the characteristics of the image and the photographing information, and a haiku is created using a phrase including the seasonal word or phrase. Can be generated. That is, it is possible to generate a text having a content different from the content indicated by the information from multimedia information such as an image.
[0088]
In this embodiment, the haiku generation process is executed with the input of the transfer data as a trigger. However, the seasonal word determined (specified) after the acquisition of the transfer data is presented to the user, and the user is allowed to select the seasonal word. Then, the operation of the selection may be triggered by the operation of the haiku generation process.
[0089]
Further, in this embodiment, in order to simplify the operation of the user, the user information DB is provided, and the user information is read from the user information DB. Can be entered each time. In such a case, a haiku according to the user's preference at that time can be generated.
[0090]
Furthermore, in this embodiment, the camera 12 and the haiku generator 30 (computer 32) are connected via the network 16, but may be connected using a cable such as a USB cable. In this case, the haiku can be easily enjoyed not only when it is located in a remote place but also when it is located near the haiku generator.
[0091]
A haiku generator 30 according to another embodiment of the present invention is the same as the above-mentioned embodiment except that the transfer data (user ID, compressed image data, and photographing information data) are directly input to the computer 32. Therefore, duplicate description will be omitted.
[0092]
In the haiku generation device 30 of this other embodiment, the camera 12 and the computer 32 are connected by, for example, a USB cable (not shown), and the compressed image data is input (transferred) from the camera 12 to the computer 32. Shooting information is separately input. That is, the camera 12 shown in FIGS. 1 and 4 only needs to have at least the photographing function, and a general-purpose digital camera can be used.
[0093]
When the user inputs the compressed image data to the computer 32, a shooting information input screen as shown in FIG. 9 is displayed on a monitor (not shown) connected to the computer 32. On the shooting information input screen, an image based on the compressed image data input to the computer 32 by the user is displayed, and a column for inputting each of the shooting information is displayed. In addition, an "OK" button and a "Cancel" button are displayed below the photographing information input field.
[0094]
Note that, for convenience of the drawing, the photographing information is described so that only the positional information (photographing place), the photographing date and time, and the weather can be input. However, all the photographing information shown in the above-described embodiment can be input. it can.
[0095]
Further, on this shooting information input screen, a user ID can be input in other fields.
[0096]
The user can input shooting information and a user ID of an image displayed on the shooting information input screen using the input device. When the photographing information and the user ID are input on this photographing information input screen and the "OK" button is turned on, the computer 32 executes the haiku generation processing as described in the above embodiment. That is, in the above-described embodiment, the haiku generation process is executed by receiving the transfer data as a trigger, but in other embodiments, the haiku generation process is executed by turning on the “OK” button as a trigger. However, in order to stop the haiku generation (input), the user may turn on the “cancel” button.
[0097]
According to the other embodiment, since the transfer data is directly input, there is no need to provide a plurality of sensors or the like in the camera, and a system that is less expensive than the above embodiment can be provided.
[0098]
Note that user information such as profile information can also be input by providing an input screen similar to that of shooting information.
[Brief description of the drawings]
FIG. 1 is an illustrative view showing one example of a configuration of a haiku generation system of the present invention;
FIG. 2 is an illustrative view showing contents of a user information DB and an image DB shown in FIG. 1 embodiment;
FIG. 3 is an illustrative view showing contents of a knowledge DB and a haiku DB shown in the embodiment in FIG. 1;
FIG. 4 is a block diagram showing a specific configuration of the camera shown in FIG. 1 embodiment.
FIG. 5 is a flowchart showing the overall processing of the CPU of the camera shown in FIG. 1 embodiment.
FIG. 6 is a flowchart showing a part of a haiku generation process of a computer of the haiku generation device shown in FIG. 1 embodiment.
FIG. 7 is a flowchart showing another part of the haiku generation process of the computer of the haiku generation device shown in FIG. 1 embodiment.
FIG. 8 is a flowchart showing a haiku text generation process of a computer of the haiku generation device shown in FIG. 1 embodiment.
FIG. 9 is an illustrative view showing one example of a photographing information input screen displayed on a screen in another embodiment of the present invention;
[Explanation of symbols]
10 Haiku generation system
12 Camera
14, 18 ... communication device
16… Network
30… Haiku generator
32… Computer
34, 36, 38, 40, 42 ... DB
52… CCD imager
58… Signal processing circuit
64… Video encoder
66… Monitor
70 ... CPU
74… Interface
78… System controller
80 ... GPS
82… bearing sensor
84… Tilt sensor
86… Operation panel
88… Temperature sensor
90 ... humidity sensor
92 Clock circuit

Claims (14)

A haiku generation system in which a camera having a communication function and a haiku generation device are connected via a communication network,
The camera is
Photographing means for photographing a desired image,
A photographing information acquiring unit for acquiring photographing information including position information specifying at least a photographing place of the image, and a transmitting unit for transmitting the image and the photographing information to the haiku generating device via the communication network,
The haiku generator,
Receiving means for receiving the image and the photographing information transmitted by the transmitting means,
Analyzing means for analyzing the image received by the receiving means,
Extracting means for extracting features of the image from the analysis result of the analyzing means,
Using the haiku material extracted from the haiku material database based on the characteristics of the image extracted by the extracting device and the photographing information received by the receiving device, and the haiku material extracted by the extracting device A haiku generation system comprising a generation unit for generating a haiku. The shooting information further includes shooting direction information at the time of shooting the image,
The camera further includes a direction obtaining unit that obtains shooting direction information at the time of shooting the image,
The haiku generation device includes: an estimating unit that estimates an imaging target based on position information and imaging direction information included in the imaging information; an identification unit that identifies an imaging target from characteristics of the image; and an imaging unit that is estimated by the estimation unit. The haiku generation system according to claim 1, further comprising an integrating unit configured to integrate the target and the imaging target specified by the specifying unit. The haiku generating device further includes an image database that records a plurality of still images to which additional information including luminance information, color information, composition information, and object information are added,
The analysis means analyzes luminance information and color information of the image,
The extracting means determines the similarity between the luminance information and the color information obtained from the analysis result of the analyzing means and the luminance information and the color information included in the additional information, and is included in the additional information having the maximum similarity. Extract composition information and object information,
The haiku generation system according to claim 2, wherein the specifying unit specifies a shooting target of the image based on at least one of the composition information and the object information extracted by the extracting unit. The shooting information further includes date and time information indicating a shooting date and time,
The camera further includes a clock circuit for acquiring the shooting date and time of the image,
The haiku generating device is a first determining unit that determines a seasonal word corresponding to a season specified from at least the date and time information, and a haiku generating unit that determines a phrase to be used in a haiku based on at least one of the characteristics of the image and the shooting information. 2. A haiku material including at least one of a seasonal word determined by the first determining device and a phrase determined by the second determining device, and a selecting device for selecting a haiku material from the haiku material database. 4. The haiku generation system according to any one of claims 1 to 3. The haiku generation device further includes a user information database that records user information including at least information indicating user preference,
The haiku generation system according to claim 4, wherein the first determining means determines the seasonal word from the date and time information and the user information. The haiku generation system according to any one of claims 1 to 5, wherein the generation unit includes a combination unit that combines the haiku materials extracted by the extraction unit while satisfying a rule of a haiku composition. 7. The method according to claim 1, further comprising determining means for determining whether the haiku generated by said generating means matches an existing haiku, and deleting means for deleting a haiku whose determination result indicates a match. Haiku generation system described in any of them. A haiku generator that generates a haiku based on an image taken by a camera,
Input means for inputting the image and shooting information,
Analysis means for analyzing the image input by the input means,
Extracting means for extracting features of the image from the analysis result of the analyzing means,
Extracting means for extracting a haiku material from a haiku material database based on the characteristics of the image and the photographing information extracted by the extracting means, and generating means for generating a haiku using the haiku material extracted by the extracting means. Equipped with a haiku generator. The shooting information further includes shooting direction information at the time of shooting the image,
Estimating means for estimating a shooting target based on position information and shooting direction information included in the shooting information, specifying means for specifying a shooting target from the characteristics of the image, and a shooting target estimated by the estimating means and the specifying means 9. The haiku generation device according to claim 8, further comprising an integration unit configured to integrate the specified shooting target. Further comprising an image database for recording a plurality of still images to which additional information including luminance information, color information, composition information and object information are added,
The analysis means analyzes luminance information and color information of the image,
The extracting means determines the similarity between the luminance information and the color information obtained from the analysis result of the analyzing means and the luminance information and the color information included in the additional information, and is included in the additional information having the maximum similarity. Extract composition information and object information,
10. The haiku generation device according to claim 9, wherein the specifying unit specifies a shooting target of the image based on at least one of the composition information and the object information extracted by the extracting unit. The shooting information further includes date and time information indicating a shooting date and time,
First determining means for determining a seasonal word corresponding to a season specified from at least the date and time information, second determining means for determining a phrase used in a haiku based on at least one of the characteristics of the image and the photographing information, and 11. The haiku material database according to claim 8, further comprising a selecting device for selecting a haiku material including at least one of a seasonal word determined by the first determining device and a phrase determined by the second determining device from the haiku material database. Haiku generator described. Further comprising a user information database that records user information including at least information indicating user preference,
The haiku generating device according to claim 11, wherein the first determining means determines the seasonal word from the date and time information and the user information. 13. The haiku generation device according to claim 8, wherein the generation unit includes a combination unit that combines the haiku materials extracted by the extraction unit while satisfying rules of a haiku composition. 14. The method according to claim 8, further comprising determining means for determining whether the haiku generated by the generating means matches an existing haiku, and deleting means for deleting a haiku whose determination result indicates a match. A haiku generator according to any of the above.

Images