JPH05189531A - Image editing device - Google Patents

Abstract

PURPOSE:To enable everybody to easily and automatically perform all operations to edit a desired image by inputting the contents of this image in a natural language. CONSTITUTION:A language dictionary 101 is provided together with an image data storage control part 105 consisting of a header word dictionary 102, a class data control part 103 and an image data base 104, a language input part 106, a language analyzing part 107, an image retrieving part 108, an image processing part 109, an image synthesizing part 110, an image display part 111, and an image editing data storage pert 112. In such a constitution, the input sentences of a natural language are analyzed for editing and display of the images showing the contents of these sentences and the image parts are retrieved, processed and synthesized based on the information stored in both dictionaries 101 and 105.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a device for editing images by searching, processing, and synthesizing a plurality of image parts, and more particularly, for drawing in a word processor or editing a background natural image in a studio or the like. The present invention relates to an image editing device for performing.

[0002]

2. Description of the Related Art In recent years, the widespread use of word processors has made it possible for anyone to create a document similar to type printing. Along with this, there is an increasing demand for a function that can easily create illustrations and natural images and incorporate them into documents.

On the other hand, conventionally, only a function for drawing a basic figure such as a straight line, a curve, a circle, etc. and a simple processing for the basic figure is prepared, and an illustration or a drawing is created by properly using these functions. Was being done. Furthermore, the illustrations, figures, and images created by the above functions are made into parts and accumulated, and the reuse of these parts makes the work more efficient, which is also applied to image editing. .. That is, a plurality of image parts that have been made into parts are prepared in advance, and the images necessary for editing are searched one by one,
The image is edited by a procedure of processing all the images and finally synthesizing all the images. As an image editing apparatus using such a method, for example, reference [Inoue, Shibata, Nakasu, "Parts Image File System for Image Synthesis", IEICE Transactions, V
ol. J72-D-II, No11, pp. 1824-
1832, November 1989].

[0004]

However, in the conventional image editing apparatus, all the processes of searching, processing and synthesizing image parts are performed manually, and editing an image is a very laborious task. there were. Also, in operating the equipment, in addition to the keyboard, complicated operations of external input devices such as a mouse are required, and it takes a considerable period of time and training to become an expert of the equipment.

On the other hand, in terms of searching for image parts, a method of using a keyword and a method of narrowing down a target image by sequentially tracing a representative image are used in the image editing apparatus. When it becomes huge, matching of keywords takes a very long time, and in the latter, the performance of the image database is greatly influenced by the creator. Therefore, these methods make it easy to search for the target image. Will be difficult.

[0006]

In order to solve the above problems, an image editing apparatus of the present invention provides a language dictionary for storing information necessary for understanding a language sentence, an image component and the image component. A language sentence is analyzed using an image data storage management unit that stores and manages a procedure for performing processing, a language input unit that inputs a language sentence, and the language dictionary and information stored in the image data storage management unit. The image data storage management unit searches for the image component necessary for expressing the content of the language sentence and the procedure for processing the image component from the analysis result of the language sentence and the analysis result of the language sentence. And an image display unit for displaying the image component, which has an image editing unit for processing and synthesizing the image component according to the retrieved procedure.

[0007]

When the contents of the image to be created are input in natural language, the language analysis unit can use the language dictionary and the information stored in the image data storage management unit to express an image from the input sentence. A noun representing a target, information indicating an attribute of the target, and a relationship between the targets are extracted. next,
Using the result extracted by the language analysis unit, the image synthesis unit searches for an image component for the target and a procedure for performing processing / synthesis processing on the image component. Finally, using the procedure described above, the image parts searched by the image processing / compositing unit are searched / processed / combined to automatically combine the images representing the contents of the input sentence.

[0008]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of an image editing apparatus according to the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of an embodiment of an image editing apparatus according to the present invention. An image editing apparatus according to the present invention includes a language dictionary 101, a headword dictionary 102, a class data management unit 103, an image data storage management unit 105 including an image database 104, a language input unit 106,
Language analysis unit 107, image search unit 108, image processing unit 10
9, an image composition unit 110, an image display unit 111, and an image edit data storage unit 112.

2 to 4 show examples of description of the entry word dictionary 102, class data, and image parts which constitute the image data storage management unit 105 according to the present invention.

FIG. 2 shows an entry word dictionary 102 according to the present invention.
Is described in the form of a table. Headword data 2
Reference numeral 03 denotes an entry word description portion 201 and a pointer description portion 202 that points to class data representing the concept of the entry word.
In this embodiment, the class data name is used as a pointer that points to the class data representing the concept of entry word.

If all nouns that can be expressed in an image are registered as headword data 203 in the headword dictionary 102, whether a headword matching the noun exists in the headword dictionary 102 for a certain noun. It is possible to determine whether or not there is an image part representing the noun by simply checking. For example, in FIG. 2, since the nouns "table", "table", "dining table", and "table" are all registered in the headword dictionary 102 as headwords, it may be determined that image parts representing these nouns exist. it can. That is, in the search for image parts representing "table", "table", "table", "dining table", "tab"
This means that four words “le” can be used. A method of searching for the image component representing the noun will be described later in the description of the image search unit 108.

3 to 6 show the class data according to the present invention in the form of a list.

The class data is the description part 30 of the concept name.
1, a pointer description unit 302 that points to class data that represents a higher-level concept, a pointer description unit 303 that points to class data that represents a lower-level concept, and a description of a pointer that points to an image component that represents a concrete entity of the concept. Part 304
And a procedure description part 306 for processing the image parts.
And a description section 305 of variables required for the procedure. The contents of each description part will be described below using the example of the class data of the "table" shown in FIG. 3 and the "room" shown in FIG.

SuperClass and SubClass are description portions (302, 30) of pointers that point to class data representing a higher-level concept and class data representing a lower-level concept according to the present invention.
In the present embodiment, the name of the class data is used as a pointer that points to upper and lower class data. If the corresponding class data does not exist, it is represented by nil. In the class data “table” of FIG. 3, “furniture” is specified as the upper class data SuperClass, whereas “table” is specified.
Is conceptually more difficult to subdivide, so
The subordinate class data SubClass is nil. In the class data “room” of FIG. 4, “building” is specified as the upper class data SuperClass, and “kitchen” “living room” “bedroom” is specified as the lower class data SubClass.

Image represents an embodiment of a description section 304 of a pointer that points to an image component representing the concept of class data according to the present invention, and the name of the file storing the data of the image component is used for the pointer. .. In the class data “table” of FIG. 3, (Image IM # table1 IM # table2 IM # table3) is specified. This is after the Image, up to the parenthesis that corresponds to the parentheses immediately before the Image, and all strings starting with IM # are
It indicates that it is a file that contains image parts that represent the concept of class data. IM # table1, IM # table2, IM # table are files that contain image parts that represent "tables".
Indicates that there are 3 etc.

Method represents an embodiment of the procedure description part 306 in the class data according to the present invention, and is described in the format of (procedure name argument declaration procedure content). In the class data “table” of FIG. 3, Co
mposeParts is the name of the procedure, followed by parentheses
Relation and ClassName are the arguments of the procedure ComposeParts, and up to the parenthesis corresponding to the parentheses immediately before ComposeParts below it are the description of the processing contents of the procedure. ComposeP
arts is an embodiment of a procedure for automatically synthesizing images according to the present invention, and the processing content will be described later in the description of the image synthesizing unit 110.

Slot represents an embodiment of the variable description part 305 in the class data according to the present invention, and is described in the format of (variable type variable name to variable name). If the variables need to be initialized, the variable name portion is set to (variable name initial value). In the class data "table" of Fig. 3, three types Position, DefaultBG, P
The ositioningVerb variable is declared.

Variables declared in the Positon type are used in the procedure ComposeParts, and six variables P_up P_down P_right P_right end P_left P_left end are declared. Since these are not initialized, it is necessary to specify specific values in the image parts belonging to the class data “table”.

Variable pos declared with PositioningVerb type
The itioning_verb (310) is an embodiment of a variable that describes, as an initial value, a triplet of the relationship representing the operation according to the present invention, the class data representing the target object of the operation, and the positional relationship represented by the operation. The variable can be used to change the relationship representing the operation according to the present invention into a positional relationship. This will be described later in the description of the processing of the language analysis unit 107.

Variable declared with type DefaultBG default_b
g (311) is an embodiment of a variable for identifying whether to represent a background or an object according to the present invention. Depending on whether or not a DefaultBG type variable is declared in the class data, the class data is classified into the background or It is possible to identify which of the objects is represented. That is, Defa
The class data in which the ultBG type variable 311 is declared represents the object, and the class data in which it is not declared represents the background. In the class data “table” of FIG. 3, De
The variable default_bg (311) declared in faultBG type is initialized as DefaultBG (default_bg (in C # room)). This means that the “table” is the object and is usually placed in the “middle” of the “room”. Further, since the class data “room” in FIG. 4 is a concept representing the background, this variable is not declared. "Apple" in Fig. 5 and Fig. 6
If you can't specify what kind of background the class data is in, like "Mikan" in DefaultBG, use DefaultBG (default_bg nil) etc.
It suffices to state that the type variable 311 is declared. This nil is used when the background of the class data cannot be specified. When class data is to be finely classified by attributes such as shape and color, a lower class is newly created in the pointer description unit 303 that points to class data representing a lower concept of the class data, and the class is subdivided by this. Instead of declaring a variable for distinguishing the attribute in the slot description part 305 of the class data, and describing the specific value of the variable in each image component belonging to the class data, classification is performed according to this. Should be done. For example, if you want to further classify the class data "table" by attributes such as shape and color, in the slot description section 305 of "table", to distinguish shapes and colors like (Shape shape) (Color color) Variables of Slot are described in Slot 3
In each image component that is declared in 05 and belongs to the class data “table”, the specific value of the variable may be described as (shape square) (color brown).

FIGS. 7 to 10 are conceptual diagrams showing image parts used in the present invention. The image parts are composed of an attribute value description section 401 and pixel data 402. Hereinafter, the content of the attribute value and the pixel data will be described using the example of the image component representing the “table” shown in FIG. 7.

The attribute value description unit 401 gives specific values to variables that have not been initialized among the variables declared in the class data to which the image part belongs. Here, concrete values are given to six Position-type variables that are not initialized in the class data “table” in FIG. Further, in addition to this, specific values are given to the two variables data_size and actual_size in the image component representing the “table” (410, 411). This is an embodiment of the variable 410 (data_size) representing the size of the pixel data of the image component and the variable 411 (actual_size) representing the size of the object represented by the image component according to the present invention. Not declared in "table". This is a special variable that all image parts have uniquely, and is declared in the class data located at the top of the associated class data in the upper and lower levels. Inventive variables can be considered omitted by inheritance. These will be described later in the description of the image processing unit 109 together with the procedure CalcZoomRatio described in the highest class data.

On the other hand, since the default_bg and positioning_verb of the class data “table” in FIG. 3 have already been initialized in the class data, image parts belonging to the class data “table” are used unless they are special cases. It is not necessary to newly specify these concrete values.
If you want to specify it specially, write it as (default_bg (c # in the garden)) (positioning_verb (V # default left edge)).

As described above, the image data storage management unit according to the present invention
This completes the description of the 105 embodiment. Next, the processing of each part of the image editing apparatus according to the present invention and the flow of the image editing processing will be described with reference to FIG. 1 by using "1.

First, the example sentence (1) is input by the language input unit 106 and passed to the language analysis unit 107. Language input section 10
As 6, an input device such as a keyboard is used.

The language analysis unit 107 retrieves and generates information for automatically synthesizing an image from an input sentence and stores it in the image edit data storage unit 112. FIG. 11 shows the processing in the language analysis unit 107 in the form of a flowchart. Hereinafter, the processing in the language analysis unit 107 will be described with reference to FIG.
Follow the instructions below.

First, a morphological analysis 501 and a syntactic analysis 502 of an input sentence are performed using the language dictionary 101 to extract an object capable of image expression, information indicating the attribute of the object, and a relationship between the objects (503). ). Regarding the morphological analysis 501, for example, the document [Hisami, Nitta, "On Proposal of Morphological Analysis by Minimum Connection Cost Method and Evaluation of Computational Complexity", Technical Report of IEICE, NCL90-8, 1990.
Year], and also for the syntactic analysis 502, eg, Definit Close Gr
Ammar (definit close glamor) may be used.

Whether or not the noun extracted by the morphological analysis according to the present invention can be expressed as an image is determined by whether or not the headword data having the headword that matches the noun is registered in the headword dictionary 102. Is possible. That is, if the entry word data 203 having the entry word that matches the extracted noun is registered in the entry word dictionary 102, the noun can be expressed as an image. Although various relationships may be considered among the objects extracted by the language analysis unit 107, this embodiment shows a case where the relationship between the positional relationship and the motion is handled. The positional relationship has a basic data structure of a triple of (object positional relationship object), and indicates that the object of the third term is at the position of the second term with respect to the object of the first term. Similar to the positional relationship, the relationship of motion is based on the basic data structure of the triple of (target motion relationship target), and the target of the first term is the second target
It indicates that the behavior of the term is behaving. Hereinafter, this data structure will be referred to as a basic relation list. The basic relation list extracted for the example sentence (1) is as follows.

((C # table in the C # room) (C # apple on the C # table)) That is, this means that the position of the "table" with respect to the "room" is "medium" and the position of the "apple" with respect to the "table". Indicates "upper". If attribute information for modifying the target is given in the input sentence, it is also extracted in this process.
For example, "There is a blue apple on the round table in the room."
The extraction result of the basic relation list for is as follows.

[(C # room (C # table (shape circl
e))) (C # table (C # apple (color green)))) In this way, in the basic relation list, the attribute information is represented as a pair with the target. Note that the C # table in the second basic relationship list is the same as the first one, so attribute information is omitted. If they are different objects, they can be distinguished by newly providing attribute information. For example, the extraction result of the basic relation list for "a blue apple is at the right end of a round table and a red apple is at the left end" is as follows.

[((C # table (shape circle)) right edge (C #
Apple (color green))) (C # table left edge (C # apple (color red)))) Next, from the basic relation list extracted in the above processing, a tree structure (representing this as a relation tree Call) is generated (504). The relational tree represents an operation for each extracted basic relation list by a node and a relation by an arc (this is referred to as a basic relation tree), and one tree is used to combine nodes representing the same target into one tree. It is generated by repeating until. If there is attribute information for modifying the target, it is also described in the node representing the target. The basic relation list extracted for the example sentence (1) is first converted into a basic relation tree as shown in FIG. In these basic relation trees, both are "tables".
Since it contains a node that represents, this is combined into one. At this point in time, the number of trees is one, so this is regarded as a relational tree (FIG. 13), and the relational tree generation processing ends.

Finally, the relational tree is stored in the image editing data storage unit 1.
12 (505), the processing in the language analysis unit 107 is ended, and the processing is continued by returning to FIG.

The image retrieving unit 108 retrieves class data in which data on the object is described and image parts representing the object. Here, using the name of the target of each node of the relation tree stored in the image editing data storage unit 112 as a keyword, the class data describing the information about the target is searched. The image data representing the object and the procedure for performing the processing on the image data are described in the class data. The search can be performed by searching the entry word dictionary 102 for entry word data having an entry word that matches the keyword and tracing a pointer pointed to by the entry word data. In the relational tree of FIG. 13, “room”, “table”, and “apple” are included as nodes, and the class data may be traced from the entry word dictionary 102 using these as keywords.
All the searched class data are transferred to the image edit data storage unit 112. If the attribute information for modifying the target is described in each node, the likelihood of how much they match with the input attribute is attached to the image parts representing the concept of the searched class data. Can be done. In this regard, for example, the method described in Japanese Patent Application No. 3-255025 may be used.

With the above, the processing in the image retrieval section 108 is completed, and the processing in the image processing section 109 will be described by returning to FIG.

In the image processing unit 109, the image search unit 108
The image parts searched for are processed. FIG. 14 shows the flow of processing in the image processing unit 109 in the form of a flowchart. Here, a method of calculating the enlargement / reduction ratio of each image component using the procedure CalcZoomRatio declared in the highest-level class data and the attribute values data_size and actual_size of each image component, based on FIG. explain. In this description, the relation tree of FIG. 13 generated for the example sentence (1) used in the description of the language analysis unit 107 is used.

First, one image component representing the concept of the root node of the relation tree is selected, and the data_size of the image component is set to X.
Let _s , Y _s , and actual_size be S (801). Here, the root node of the relation tree is “room” from FIG. 13, and if the image parts belonging to it are those of FIG. 8, X _s = 640 Y _s = 400 S = 4.0 is obtained. Further, the number of nodes other than the root node of the relation tree is n, and each node is given a number from 1 to n (802). In FIG. 13, the number of nodes other than the root node of the relation tree is 2, so n = 2, and the node “table” is set.
To the node "apple" and the number 2 to the node "apple".

Next, i = 1 is set (803), one image component representing the concept of the i-th node is selected, and the image component of the image component is selected.
Let data_size be X _d , Y _d , and actual_size be D (80
4). Since i = 1 here, if the first node, that is, the image component showing the concept of the “table” in FIG. 7 is used, X _d = 256 Y _d = 128 D = 1.0 is obtained. And in the procedure CalcZoomRatio,
For the image component representing the concept of the root node of the relation tree, the scaling factor of the other image component representing the concept of the i-th node is calculated using, for example, the formula shown in FIG. 15 (8
05). However, in this equation, r _max and r _min are
Threshold value for correcting the value of r _cal when it is too large or too small, r _max when the value of r _cal is larger than r _max , and r _min when it is smaller than r _min To replace. Therefore, r _{m ax} and r _min are given appropriate values in advance. Since i = 1 now,
Enlargement / reduction ratio of "table" to "room" r ₁
When is calculated, r ₁ = 0.7.

Furthermore, the data_size of the image component representing the concept of the i-th node and the attribute value of the Position type variable are r _i
The pixel data is also multiplied by r _{i in} both the vertical and horizontal directions (8
06). Here, the data is rewritten for the image parts that express the concept of "table" as follows (data_size 256 * 0.7 128 * 0.7) (P_up x1 * 0.7 y1 * 0.7) (P_down x2 * 0.7 y2 * 0.7) (P_right x3 * 0.7 y3 * 0.7) (P_right edge x4 * 0.7 y4 * 0.7) (P_left x5 * 0.7 y5 * 0.7) (P_left edge x6 * 0.7 y6 * 0.7) here Then, since i <n (807), i is incremented by 1 (808) (that is, i = 2), and the process returns to 804, that is, the same process is performed, that is, enlargement of the "apple" image component to the "room" image component・ Calculate the reduction ratio r ₂ . When it is finished, if i <n is judged (807),
Since it is not satisfied, the processing in the image processing unit 109 ends.

Returning to FIG. 1, the process of the image synthesizing unit 110 will be described. The image combining unit 110 combines the image parts processed by the image processing unit 109. FIG. 16 shows the flow of processing in the image synthesizing section 110 in the form of a flowchart. Here, the procedure ComposeParts declared in each class data and the Position of each image part
A method of calculating the enlargement / reduction ratio of each image component using the type attribute value will be described with reference to FIG. In the description,
The relation tree of FIG. 13 generated for the example sentence (1) used in the description of the language analysis unit 107 is used.

First, the number of arcs included in the relation tree is n, and the arcs are given numbers 1 to n (1001). In FIG. 13, since the number of arcs included in the relation tree is 2, n = 2 is set, the arc connecting the node “room” and the node “table” is connected to the first arc, and the node “table” and the node “apple” are connected. Let the arc be the second arc.

Next, i = 1 is set (1002), and the procedure ComposeParts described in the class data of the node above the i-th arc is called (1003). Here, the class data of the node above the first arc, that is, the procedure ComposeParts described in "Room"
Call. Then, the same relation as the relation given to the i-th arc is set to the Relatio of the procedure ComposeParts.
Select from n (1004). In FIG. 13, the relation tree 1
Since the relation given to the th arc is "medium", the same relation as this is used in R of the ComposeParts procedure of "room".
Select from elation. Now, check whether the condition of Class is described in Relation (100
5), if specified, the node below the i-th arc, or one that matches the superordinate concept of the node
Select from s (1006) and proceed to 1007. Otherwise, select Class default and proceed to 1007. "room"
In ComposeParts' Relation Medium, Classes that match the concept of table or its superclass
When checking whether or not it is described in s, it is described in the third item, so select it.

Finally, with respect to the image component of the upper node of the i-th arc, the lower image component is placed in the location designated by the Class selected in 1006 (1007, 1008). In the Class default selected earlier, the location of the "table" with respect to the "room" is specified to be in P_, so the image part of the "table" is placed in the P_ of the image part of the "room". Place it in the position.

Since i <n (1009), i is incremented by 1 (1010) (that is, i = 2), 1
Returning to 003, the same processing is performed, that is, the image component of “apple” is arranged with respect to the image component of “table”. When i <n is judged at the time when it is finished (1009), since it is not satisfied, the process in the image synthesizing unit 110 is finished.

Finally, in the image display section 111, the image composition section 1
The composited image is displayed at 10, and all the processes of the image editing apparatus according to the present invention are completed.

The data stored in the image edit data storage unit 112, not the image data storage management unit 105, is used to refer to the data necessary for the processing of the image search / modification / synthesis unit of the image editing apparatus. .. This is to enable high-speed data reference even when the amount of image components accumulated in the image data accumulation management unit becomes enormous.

Now, in the image editing apparatus according to the present invention, in the input sentence, when the word indicating the positional relationship between the objects is omitted by the verb, and when the object indicating the background is omitted, the language analysis unit 107 It is possible to replenish these. FIG. 17 shows, in the form of a flow chart, the above-mentioned two processes added to the process in the language analysis unit 107 shown in FIG. The following describes the process of performing the above two supplements with reference to FIG. 17 by using “2. There are apples and mandarins in the table.” (2) as an example sentence.

First, in the language analysis unit 107, the example sentence (2)
The following two basic relation lists are extracted from

(Table V # Aru Apple) (Table V # Aru Mikan) This means that the relation between "apple" and "mandarin orange" with respect to "table" is represented by the verb "aru". In order to arrange the image parts in the synthesizing unit 110, it is necessary to change the relationship between the “table” and the “apple” and the “table” and the “mandarin orange” represented by the verb “arru” to the positional relationship. In the image editing apparatus according to the present invention, Positioning in which a set of a relationship indicating an action in class data and a position indicated by the action is described as an initial value
The above change is possible by using the Verb type positioning_verb (310). That is, it is checked whether or not the extracted relation is a relation representing a motion (1104), and the basic relation list including the relation representing a motion is given by the positioning_verb of the class data of the first term of the basic relation list. First of the initial values
A term whose term is the same as the action relation extracted by the language analysis unit 107 is selected, and the action relation is replaced with the positional relation described in the second term of the selected initial value (110).
5). For example, in the “table” of FIG. 3, it is described as (PositioningVerb (positioning_verb (V # is above))), which means that if there is any object “present” in the “table” The position of the object with respect to the "table" is "up". Therefore, the extracted basic relation list is changed to (table apples) (table oranges).

When the replenishment of the word representing the positional relationship between the objects omitted by the verb is completed, the language analysis unit 107 then generates a basic relation tree from the changed basic relation list (FIG. 18). , A relation tree is generated (Fig. 1
9).

Since the image component representing the root node of the created relational tree serves as a reference for determining the position of the image component of another node in the image synthesizing unit 110, the target of the root node represents the background. There must be. In the image editing apparatus of the present invention, the defaultBG type variable default_bg (31) for identifying whether the concept represented by the class data in the class data is the background or the object.
1) is described, and by using this, it is possible to determine whether the target of the root node is the background and to add a node representing the background to the relation tree if it is not the background.

The details of the method will be described below. First, it is checked whether or not a DefaultBG type variable is declared in the class data corresponding to the root node of the relational tree generated by the language analysis unit 107 (1107). If not declared, the root node represents the background, and the relation tree is transferred to the image edit data storage unit 112 as it is. If declared, the root node represents an object, and it is necessary to add a node representing the background to the relation tree as a root node. The node representing the background is added with reference to the initial value given to the DefaultBG type variable default_bg (311) of the class data corresponding to the root node of the relation tree. That is, default_
The first term of bg is the upper node of the root node, and the second term is the relation given to the arc connecting the two nodes (1108).

The above processing is performed on the relation tree of FIG. 19 created from the example sentence (2) as follows.

First, when the object of the root node of the relational tree is examined, it is a "table". In the class data “table” of FIG. 3, (DefaultBG (default_bg (C # room))) is described, and it is clear that the “table” represents the target object.
A background node must be added to the relation tree of FIG. In class data "table", the background of "table" and its location is "room" with default_bg
Since "middle" is designated, "room" is added as an upper node of the root node of the relation tree of FIG. 19, and the positional relation "middle" is given to the arc connecting the two nodes (FIG. 20). ..

In this way, the input sentence with the background omitted can be supplemented.

[0056]

According to the present invention, if the contents of an image to be created are input in natural language, all the operations up to the editing of the target image can be performed automatically, so that it is very troublesome in the past. It is possible to edit an image, which was a work, in a short time. Further, since the input means is only natural language, it is not necessary to operate an external input device other than a keyboard such as a mouse when operating the device, and anyone can easily edit the image. Further, when a natural language is input and post-editing is performed on an automatically combined image, all image parts related to image editing are searched and stored in advance, so that the combined image is formed. It is possible to replace the object with other image parts at high speed.

[Brief description of drawings]

FIG. 1 is a block diagram showing the configuration of an embodiment of an image editing apparatus according to the present invention.

FIG. 2 is a diagram showing an entry word dictionary according to the present invention in a table format.

FIG. 3 is a diagram showing class data according to the present invention using a list format, showing class data “table”.

FIG. 4 is a diagram showing class data according to the present invention using a list format, showing class data “room”.

FIG. 5 is a diagram showing class data according to the present invention using a list format, showing class data “apple”.

FIG. 6 is a diagram showing class data according to the present invention using a list format, showing class data “Mikan”.

FIG. 7 is a conceptual diagram showing image parts according to the present invention, showing a “table”.

FIG. 8 is a conceptual diagram showing an image part according to the present invention, showing “room”.

FIG. 9 is a conceptual diagram showing an image part according to the present invention, showing “apples”.

FIG. 10 is a conceptual diagram showing an image part according to the present invention,
What was shown about "Mikan"

FIG. 11 is a diagram showing the flow of processing in the language analysis unit in the form of a flowchart.

FIG. 12 is a conceptual diagram showing a basic relation tree in the embodiment of the image editing apparatus of the invention.

FIG. 13 is a conceptual diagram showing a relation tree in the embodiment of the image editing apparatus of the present invention.

FIG. 14 is a diagram showing the flow of processing in the image processing unit in the form of a flowchart.

FIG. 15 is a diagram showing an expression for calculating a scaling factor of an image component as processing of an image processing unit.

FIG. 16 is a diagram showing the flow of processing in the image composition unit in the form of a flowchart.

FIG. 17 is a diagram showing, in the form of a flow chart, the flow of processing when supplementary omission is further performed in the language analysis unit.

FIG. 18 is a conceptual diagram showing a basic relation tree in the embodiment of the image editing apparatus of the invention.

FIG. 19 is a conceptual diagram showing a relation tree in the embodiment of the image editing apparatus of the invention.

FIG. 20 is a conceptual diagram showing a relation tree in the embodiment of the image editing apparatus of the present invention.

[Explanation of symbols]

 101 language dictionary 102 headword dictionary 103 class data management unit 104 image database 105 image data storage management unit 106 language input unit 107 language analysis unit 108 image search unit 109 image processing unit 110 image synthesis unit 111 image display unit 112 image editing data storage Department

Claims (15)

[Claims] 1. A language dictionary for storing information necessary for understanding a language sentence, an image data storage management unit for storing and managing an image component and a procedure for processing the image component, and a language sentence storage unit. A language input unit for inputting, a language analysis unit for analyzing a language sentence using the language dictionary and information stored in the image data storage management unit, and a content of the language sentence from the analysis result of the language sentence. An image editing unit that searches the image data storage management unit for the image component necessary for expressing and the procedure for processing the image component, and performs the processing and composition of the searched image component by the searched procedure. , An image display unit for displaying image parts, inputting a natural language sentence, analyzing the input sentence necessary for editing and displaying the image showing the contents of the input sentence, and searching / processing the image component / Synthesis process Image editing apparatus comprising means for performing automatically by using the information stored with the language dictionary in the image data storage management unit. 2. The image editing apparatus according to claim 1, wherein the image component is composed of an attribute value description section and pixel data, and the procedure in the image data storage management section uses the attribute value as a parameter. 3. The image data storage management unit includes a description part of a name of a concept, a pointer pointing to an image part representing a concrete entity of the concept, a procedure for processing the image part, and a variable necessary for the procedure. A class data management unit that collects at least the class data that includes it and an image database that collects the image parts are included, and the procedure for performing processing on the image parts pointed by the pointer of the class data is described in the class data. The image editing apparatus according to claim 2, characterized in that: 4. The image data storage management unit further comprises a headword dictionary that collects headword data composed of a description portion of headwords and pointers pointing to class data representing the concept of the headwords. 4. The image editing apparatus according to claim 3, wherein the pointers of all the headword data representing "" indicate one class data representing the concept. 5. The class data has a description section of a pointer that points to class data representing a higher-level concept and class data representing a lower-level concept, and the procedure and variables described in the class data are stored in the pointer. 4. The method according to claim 3, wherein the lower class data pointed to by the lower class data is inherited, and a pointer to the upper class data is traced when necessary to refer to the procedure and variable of the upper class data. Image editing device. 6. A language analysis unit extracts an object capable of image expression, information indicating an attribute of the object, and a relationship between the objects from an analysis result of a language sentence, and the extracted image expression is possible. The object is represented by a node, the relationship between the objects is represented by an arc, and in a node representing a certain object, another object having some relation to the object is a node lower than the node, and the both nodes are represented by the relationship. 4. The image editing apparatus according to claim 3, further comprising means for generating a tree structure configured by connecting the arcs. 7. The image part describes a specific value for a variable declared in the class data in a description part of an attribute value of the image part designated by a pointer pointing to the image part. The image editing apparatus according to claim 6. 8. The class data variables are variable types, names,
An image pointed by the pointer of the class data when the initial value is given to the variable described in the class data that has a pointer that points to the image part The image editing apparatus according to claim 7, wherein a description of a specific value corresponding to the variable is omitted in the component unless necessary. 9. The class data includes at least a variable for identifying whether to represent a background or an object, and the linguistic analysis unit further includes the variable to represent the concept of the background or the object. 9. The image editing apparatus according to claim 8, further comprising means for identifying a watermark. 10. The class data representing an object is a variable for distinguishing whether the object represents a background or an object, and other class data representing a background in which the object exists, the background, and the object. The relationship is described as an initial value, and the language analysis unit further uses the initial value when the target of the root node of the tree structure generated by inputting the language sentence is identified as the object. 10. The image editing apparatus according to claim 9, further comprising means for generating a tree structure by adding a node representing a background and an arc connecting the node and a root node of the tree structure to the tree structure. 11. The class data includes at least a variable describing as an initial value a set of a relationship indicating a motion and a positional relationship indicated by the motion, and the language analysis unit further generates a tree generated by inputting a language sentence. 9. The image editing apparatus according to claim 8, further comprising means for changing a relationship indicating a motion into a positional relationship by using the variable in the arc of the structure. 12. The image editing unit stores an image component representing the target and a procedure for processing the image component by using a target capable of image representation and information indicating an attribute of the target. An image search unit having means for searching from the management unit, an image processing unit that processes image components according to the procedure, and an image composition unit that performs image component composition processing according to the procedure. The image editing device according to claim 7. 13. The class data is a variable that describes a size of pixel data of an image part pointed by a pointer of the class data and a size of an object represented by the image part, and processing of the image part using the variable. The image processing unit includes at least a procedure for performing processing, and has means for performing processing of an image component representing a target of each node of a tree generated by the language analysis unit by inputting a language sentence by the procedure. The image editing apparatus according to claim 12, wherein the image editing apparatus is an image editing apparatus. 14. The image synthesizing unit includes at least a variable that describes a positional relationship pointed to an object representing the class data, and a procedure for performing an image component synthesizing process using the variable. Has means for synthesizing an image component representing a target of each node of the tree generated by the language analysis unit by inputting a language sentence by the relation between the arcs of the tree and the procedure. Item 1
2. The image editing device described in 2. 15. An image edit data storage unit for temporarily storing the information analyzed by the language analysis unit and the information searched by the image search unit, and the processing in the image edit unit is stored in the image edit data storage unit. The image editing apparatus according to claim 1, wherein the image editing apparatus is performed using stored information.