JP5843544B2 - Information processing apparatus and control method thereof - Google Patents

Description

  The present invention relates to a technique for improving operability for a display object.

A system that supports organizing objects by drawing electronic objects on a table-type display or a wall-type display and enabling the displayed objects to be operated has attracted attention. Devices such as a touch panel and a digitizer are incorporated in the display of such a system, and an object displayed on the display can be touched. In general, an operation using a touch panel is easily used by a user who is unfamiliar with the device, and is used in many devices such as a mobile terminal and a copy machine. With such a system, it is possible to visually organize and classify the objects displayed on the display by performing intuitive operations (movement, etc.).
In Patent Document 1, when an operation for moving a photographic object displayed on a screen in any one of four surrounding directions is performed, the photos are sorted into groups corresponding to the operation direction. The user can call up a photograph for each group that has been assigned, and perform operations such as printing. Here, as a result of moving a plurality of objects in four directions according to the user's intention, they are only grouped into four types indicating the results. That is, the user movement work is a work for classification.

JP 2010-134859 A

There are various uses of the above-described system, and each object is not always moved with the main purpose of classifying the objects being displayed as in Patent Document 1. However, even if the user is not working for the main purpose of classification, if the objects are arranged and moved in a meeting or the like, there is a high possibility that the positional relationship has some meaning.
The present invention has been made in consideration of the above-mentioned problems. When a plurality of objects are arranged and moved for any purpose, metadata for facilitating understanding of the relationship between the objects for these objects. An object is to provide a method for providing the above.

In order to solve the above-described problem, the present invention provides an information processing apparatus that displays a plurality of objects on a display unit, the acquisition unit acquiring a layout of the plurality of objects displayed on the display unit, and the acquisition in the acquired layout by means and identifying means for identifying a visual relationship on the display among the plurality of objects, the data based on the identified visual relationships, meta to each of the plurality of objects Classifying a key object in an assigning means for giving as data, an operating means for executing an operation for selecting at least one of the displayed objects, and at least one object selected by the operating means and a classifying means for, said obtaining means is selected by said operation means When an operation for further moving at least one of the objects is performed, a layout changed in accordance with the movement is acquired, and the specifying unit is configured to acquire the key object and each of the other objects displayed. The visual relationship on the display is specified, and the assigning means assigns metadata based on the result specified by the specifying means to each of the plurality of objects, the visual relationship, characterized by relationship der Rukoto based on any one or combination of positions where the distance or size object is displayed or object, is placed between the object.

  According to the present invention, when a plurality of objects are arranged and moved for some purpose, metadata for facilitating understanding of the relationship between the objects can be given to the objects.

System overview of information processing equipment Hardware and functional configuration diagram of information processing apparatus Explanatory drawing of metadata grant processing Processing flow chart showing processing of information processing apparatus The figure which shows the table which is an example of the metadata provided Processing flow chart showing processing of information processing apparatus Explanatory drawing of metadata grant processing Processing flow chart showing processing of information processing apparatus The figure which shows the table which is an example of the metadata provided

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In addition, embodiment described below shows an example at the time of implementing this invention concretely, and is not restricted to this.

[First Embodiment]
In the first embodiment, in the arrangement in which a plurality of objects displayed on the display unit are operated and moved, a value (frequency) indicating that the degree of association between the objects is higher as the distance between the objects is smaller. An example of giving to each object as metadata will be described.

  FIG. 1 is a schematic diagram showing an example of a system to which this embodiment can be applied. FIG. 1A is a conference system for interactively generating ideas and discussions in a conference using a table type display 101 and a wall surface type display 102. Participants A and B perform various operations such as selection and movement of objects by touching or gesturing the objects 103 and 104 displayed on the displays 101 and 102, which are display units, using fingers or hands. Can be executed. FIG. 1A shows a system that projects an object on both the display 101 and the wall surface display 102 using a projector 105 that can project in both the downward and lateral directions. In addition, a touch sensor and a distance image sensor are used around each display in order to specify an operation by touch or gesture on the object. The touch sensor is incorporated in the display 101 and is used for touch detection when touching the object 103 displayed at a position where the hands of the participants A and B can reach. The distance image sensor is installed on the wall surface display 102 and is used for gesture detection when the object 104 displayed at a position out of the reach of the participants A and B is operated. Each of these devices is connected to the information processing apparatus 100 via a network or USB, and functions in cooperation. Note that although FIG. 1A illustrates an example in which two display units are used and a distance image sensor and a touch sensor are mounted on each of the two display units, the number of display units and the sensor method are not limited thereto. For example, the number of display units may be one. In addition, a large screen display such as a liquid crystal display can be used on a table or wall instead of the projection display unit such as the projector 105. In FIG. 1A, the information processing apparatus 100 is installed on the ceiling, but the installation position may be another place.

  FIG. 1B illustrates an example in which the information processing apparatus 100 is a tablet terminal. In this case, the user can perform various operations such as selection and movement of an object by touching the object 107 displayed on the touch screen display 106, which is a display unit including a touch sensor, using a finger or the like. .

  As described above, the present invention is applied to an information processing apparatus that can operate an object displayed on a display unit by using a touch operation or a gesture operation. Hereinafter, in the present embodiment, an example in which operation is mainly performed using the table type display 101 in the system of FIG.

  FIG. 2A is a hardware configuration diagram of the information processing apparatus 100 in the present embodiment. The information processing apparatus 100 includes a CPU (Central Processing Unit) 200, a RAM (Random Access Memory) 201, a ROM (Read Only Memory) 202, and a bus 203. Further, the display I / F 204, the storage I / F 205, the storage 206, and the input I / F 207 are included. However, an interface such as a network I / F or a memory card I / F is further included, and an information processing apparatus can be realized using the interface. Hereinafter, details of the hardware configuration of the information processing apparatus 100 will be described.

  The CPU 200 controls the entire information processing apparatus 100 using the RAM 201, the ROM 202, and the bus 203. In addition to the operating system (OS), each processing program, device driver, and the like according to the present invention are stored in the ROM 202, temporarily stored in the RAM 201, and appropriately executed by the CPU 200. Here, the OS, each processing program, and the like may be stored in the storage 206, in which case they are appropriately read into the RAM 201 when the power is turned on, and are activated by the CPU 200.

  The display I / F 204 converts a display image created in the information processing apparatus 100 into a signal that can be processed by the display 101 (or the projector 105 when the projection display unit is used). The storage I / F 205 mutually converts a data format that can be used inside the information processing apparatus 100 and a data format that is stored in the storage 206. The input I / F 207 receives distance information from the distance image sensor 105 and position information from the touch sensor as input signals, and converts them into information that can be processed by the information processing apparatus 100.

  The storage 206 is a large-capacity information storage device represented by a hard disk drive device, and the storage 206 stores a large amount of object data to be displayed on the information processing apparatus 100. The storage 206 itself may be mounted inside the information processing apparatus 100, or may be connected via various interfaces such as Ethernet (registered trademark), USB, and memory card I / F. At that time, a plurality of storages 206 can be connected.

  Hereinafter, the same reference numerals are given to the elements described with reference to the already described drawings, and the description thereof is omitted.

  FIG. 2B is a functional block diagram of the information processing apparatus 100 in the present embodiment. The function of the information processing apparatus 100 includes an object reading unit 300, a display control unit 301, an operation unit 302, a layout acquisition unit 303, a detection unit 304, a specification unit 305, an assignment unit 306, and a classification unit 307. Each of these functional blocks can be realized in various forms by a combination of hardware and software. In the present embodiment, each functional block corresponds to the functional unit of the CPU 200 described above.

  Data stored in the storage 206 is data (hereinafter, objects) digitized in various formats such as voice, handwritten characters, text, images, and moving images. In the initial state, it is assumed that information about the creation date and time is given to these objects, but information indicating the relationship between the objects is not given. In addition, the storage 206 may store not only objects that have been conceived during use of the system but also objects that have been created in the past. In this embodiment, an example in which a still image such as a photograph is handled as an object will be described.

  The object reading unit 300 reads an object to be displayed from the storage 206 and holds it in the RAM 201. In this embodiment, an example of reading an object from the locally connected storage 206 is shown for the sake of simplicity. However, it is also possible to read a storage existing on the network or an object displayed on the Internet via the network. It is. The read object is held in the RAM 201. At this time, if the read data is too large for the RAM 201, the read data may be subdivided and processed sequentially. Alternatively, it may be saved in the storage 206 as swap data that can be read immediately.

  The display control unit 301 generates a display image from the object held in the RAM 201 and outputs it to the display 101. An example of a display image created by the display control unit 301 is shown at 400 in FIG. In FIG. 3A, 401 a to 401 e indicate objects displayed on the displays 101 and 102 by being projected on the projector 105. Information indicating the position and size of each object on the display image is called layout information, and changing the position and size of each object by moving the object is called “layout. "Change."

  The operation unit 302 performs an operation on the selected object. In the present embodiment, the operations performed on the object by the operation unit 302 are object selection and movement operations. Further, in the system of the present embodiment, the operation on the information processing apparatus 100 is performed by performing a touch operation or a gesture operation using an object such as a hand or a finger. Here, a description will be given based on an example in which a user performs a touch operation with a hand on a table-type display 101 equipped with a touch sensor. First, the operation unit 302 acquires, from the detection unit 304, position information instructed by the user through a touch operation. Then, the display position information of the object 401 is acquired from the display control unit 301. If the touched position and the position where the object is displayed overlap, the object displayed at the position is selected. Further, the operation unit 302 determines a moving direction and a moving distance from the user's touch position information acquired from the detecting unit 304, and executes an operation of moving the object to that position. At this time, the display control unit 301 obtains information regarding movement from the operation unit 302, generates an image to be displayed in accordance with the movement operation, and outputs the image to the display 101. Here, the detection unit 304 detects a touch operation by the user based on a signal output by the touch sensor detecting and outputting a touch on the display 101. In the case of an object displayed on the wall surface display 102, the detection unit 304 detects position information instructed by a gesture operation based on a signal output from the distance image sensor. The operation unit 302 acquires the position information and executes selection and movement operations.

  The layout acquisition unit 303 acquires the layout information of the initial image from the display control unit 301, and further acquires the layout information changed in accordance with the movement of at least one object by the operation unit 302. For example, FIG. 3C shows a layout that is changed as the object 401c and the object 401d are selected from the state of FIG. 3A and the object 401d is moved.

  The specifying unit 305 obtains information on the layout currently displayed on the display unit from the layout acquisition unit 303, and specifies the visual relationship on the display of a plurality of objects. Here, the visual relationship on display is, for example, the relationship between the distances between the objects, the relationship between the displayed sizes, or the positional relationship in which directions are displayed in the vertical and horizontal directions. . The assigning unit 306 assigns metadata to each object based on information on the visual relationship between the objects acquired by the specifying unit 305. The metadata to be assigned here is a value (frequency) indicating the degree (level) in a certain index of the object to be assigned. For example, a frequency indicating that the degree of association between objects is higher as the distance between the objects is smaller is assigned to both the corresponding objects as metadata. The metadata assigned to the object is stored in the metadata table and stored in the storage. In this embodiment, the information indicated by the metadata provided by the assigning unit 306 is output on the display unit (display 101) by the display control unit 301 so that the information is displayed together with the object. For example, the arc 403a in FIG. 3C is displayed to indicate that the metadata indicating the degree of association has been given to both objects in response to an operation to reduce the distance between the object 401c and the object 401d. Has been.

  The classification unit 307 classifies the object selected by the operation unit 302 into a key object and a sub object. The classification of key object and sub-object is used when the identifying unit 305 limits the object to be noted, or when the information indicated by the metadata provided by the assigning unit 306 is displayed on the display 101. In this embodiment, an object that has not moved is classified as a key object, and an object that has moved is classified as a sub-object. For example, in the example of FIG. 3C, the moved object 401d is a sub-object, and an arc 403a centered on the key object 401c is displayed so as to indicate the degree of association of the sub-object 401d with the key object 401c. It is what. Processing for displaying the arc 403a will be described later.

  In the present embodiment, a layout of a plurality of objects displayed on the display unit is acquired, and a value (frequency) indicating that the degree of association between the objects is higher as the distance between the objects is smaller as metadata. Give to each object. FIG. 3 is an explanatory diagram of the metadata providing process in the present embodiment. FIG. 4 is a processing flowchart showing data display processing of the information processing apparatus 100 in the present embodiment. Details of the data display process and the metadata providing process of this embodiment will be described with reference to FIGS. 3 and 4.

First, it is assumed that the object displayed in the present embodiment is a photograph of a person. Less than,
An example in which the intimacy of Mr. A, Mr. B, Mr. C, Mr. D, and Mr. E reflected in the object of the photograph is assigned as metadata by the metadata adding process shown in FIGS. When metadata indicating the closeness of a person who is a subject is assigned to a photograph, the information processing apparatus 100 can use the metadata. For example, it is also possible to reproduce the display image at the time of adding metadata by associating the closeness with the magnitude relationship of the distance between objects. In addition, based on the given metadata, it is possible to extract only photographs showing a person who has a close relationship with a specific person to a certain extent and create an album.

  Hereinafter, in this embodiment, the description will be made assuming that the user performs a touch operation with the finger on the table type display 101 in the system of the information processing apparatus 100 in FIG.

  In the present embodiment, the data display process is started as the information processing apparatus 100 is turned on. When starting the data display process, the information processing apparatus 100 executes Step S10. In step S <b> 10, the object reading unit 300 reads an object to be displayed from the storage 206 and holds it in the RAM 201. Next, in step S11, the display control unit 301 creates a display image by randomly arranging the objects held in the RAM 201, and outputs the created screen to the display 101 as an initial image in step S12. An example of the initial image displayed on the display 101 at this time is shown in FIG. It is assumed that the photograph object displayed here includes five persons A, B, C, D, and E, one by one. In FIG. 3A, a total of five objects are displayed in a layout in which the same size is randomly arranged. However, the initial image may be created by arranging the objects in the order of the names from the upper left of the screen.

  In step S13, the information processing apparatus 100 determines whether an operation for selecting two or more objects among the plurality of displayed objects has been performed. That is, the detection unit 304 detects a touch operation at two or more places on the display, and the operation unit 302 determines whether the position is a position where an object is displayed. When the user touches an object that the user wants to select in the state where the initial image of FIG. 3A is displayed, the touch sensor embedded in the display 101 detects the touch and is converted into position information by the detection unit 304. The data is output to the operation unit 302. The operation unit 302 uses the position information output from the detection unit 304 and the layout information of the initial image acquired from the display control unit 301. If the object is displayed at the touched position, the operation unit 302 selects the object. Perform the operation. When two or more objects are selected according to the touch operation (YES in S13), the process proceeds to step S14. In the present embodiment, as shown in FIG. 3B, it is assumed that two objects, an object 401c and an object 401d, are selected according to a user's touch operation. If two or more objects are not selected by the touch operation (NO in S13), the process proceeds to step S17.

  In step S14, the operation unit 302 determines whether an operation for moving the selected object has been detected. That is, the operation unit 302 confirms whether the movement of the touched position has been detected following the touch for selecting the displayed object, based on the position information output from the detection unit 304. When the touch sensor detects that the user moves a finger touching the position where the object is displayed on the display 101, the detection unit 304 outputs the moving touch position to the operation unit 302 as position information. In step S14, the operation unit 302 determines that an operation for moving the object has been detected if information indicating that at least one of the touch positions detected in step S13 has moved is acquired from the detection unit 304. The example of FIG. 3B shows a display image in a state where two objects are selected as described above. The selected objects are an object 401c in which Mr. C is photographed and an object 401d in which Mr. D is photographed. The selected object is highlighted so that the user can identify that it has been selected. In FIG. 4B, for highlighting, the color is changed for easy understanding, but a method of displaying a frame surrounding the object only for the selected object or adding a check mark may be used. When the movement of the selected object is not detected (NO in step S14), the process proceeds to step S17. When the movement of the selected object is detected (YES in step S14), the metadata providing process in step S15 is started.

  FIG. 4B is a process flowchart showing the process of adding metadata (step S15) according to the first embodiment. When the metadata providing process is started, it is determined in step S20 whether only a part of the selected objects is moved (only one of the two objects). If there is an object that does not move among the selected objects, that is, one of the selected objects is moved and one of them is not moved (YES in step S20), the object classification processing by the classification unit 307 is performed in subsequent step S21. If all of the selected objects are moved, that is, if both objects are moved (YES in step S20), the process proceeds to step S24. In the example of the present embodiment, as shown in FIG. 3C, since only the object 401d is moved among the selected two objects, the process proceeds to step S21.

  In step S21, the classification unit 307 classifies the two objects selected by the operation unit 302 into a key object and a sub object. In the present embodiment, among the selected objects, an object that is not moved by a touch operation is classified as a key object, and an object that is moved is classified as a sub-object. Therefore, in the example of FIG. 3, the object 401c that has not been moved is classified as a key object, and the moved object 401d is classified as a sub-object. Mr. C is shown in the key object 401c, and Mr. D is shown in the sub-object 401d. Further, the key object may be highlighted so that the user can specify the key object and the sub-object. In FIG. 3C, the background of the key object is shaded to express a difference from the object 401d. When the classification process in the classification unit 307 is completed, the assigning unit 306 refers to the metadata table stored in the storage 206 attached to the object and stores it in the RAM 201 in step S22. When the object does not have a metadata table, an empty table is generated and referred to and stored in the RAM 201. The metadata table is a table that holds metadata to be given to the display object, and has a structure as shown in FIG. 5, and no metadata is given in the initial state (FIG. 5A). The metadata given in the present embodiment is a frequency indicating the familiarity of how close each of Mr. A, Mr. B, Mr. C, Mr. D, and Mr. E is reflected in the object. Assume that the two people are intimate. However, the adding unit 306 may refer to the metadata table read together with the object by the object reading unit 300 in step S10 and held in the RAM 201 in step S22.

  In step S <b> 23, the operation unit 302 acquires information on the moved position of the moved object, that is, the sub-object, from the detection unit 304, and holds this in the RAM 201. In step S <b> 24, the display control unit 301 uses the metadata information and the sub-object position information held in the RAM 201 to output a display image that has changed due to the movement operation to the display 101 to update the display image. To do. FIG. 3C shows how the position where the object 401d is displayed changes following the operation of the hand that moves the object 401d. In step S25, the operation unit 302 determines whether the movement of the sub object is completed. That is, the operation unit 302 confirms whether or not the movement of the touch position acquired from the detection unit 304 is continued. If the movement continues (NO in step S25), the movement position of the sub object is set again in step S23. The display image is updated in step S24. When the user releases the finger touching the display 101 for the operation of selecting and moving the sub-object, that is, when the detection unit 304 detects the release of the touch, the movement of the sub-object is completed (step S25). YES), the process proceeds to step S26.

In step S26, the layout acquisition unit 303 first acquires the layout of the object in the currently displayed display image. In this embodiment, the layout acquisition unit 305 considers the entire display image displayed on the display 101 as a coordinate plane, sets the origin, sets the center coordinates (x1, y1) of the key object, and the center coordinates (x2) of the sub object. , Y2). In addition, the layout acquisition unit 305 acquires layout information such as the shape, size, and color of the object in the display image as necessary. Next, the specifying unit 305 specifies the visual relationship on the display of each object based on the acquired layout information. Such a visual relationship is one of the elements that are most easily recognized when the user operates a plurality of objects displayed on the information processing apparatus 100. In the present embodiment, the visual relationship on display specified here is the magnitude relationship of the distance ΔL between the key object selected by the operation unit 302 and the sub-object. The desired distance between objects ΔL is:
D = √ {(x1-x2) * (x1-x2) + (y1-y2) * (y1-y2)}
Can be obtained. Further, the specifying unit 305 holds the obtained distance ΔL in the RAM 201. In the example of FIG. 3C, the first operation immediately after starting the information processing apparatus is described, and therefore there is only one information on the distance ΔL held in the RAM 201. Accordingly, the magnitude relationship is not determined, but in the second and subsequent processes, the magnitude relationship with the distance ΔL regarding other objects already held in the RAM 201 is specified. The second and subsequent processes will be described later.

  In step S27, the specifying unit 305 determines whether the metadata of each object needs to be updated using the metadata held in the RAM 201 and the distance ΔL between the objects. Specifically, the identifying unit 305 defines metadata defined from the calculated distance ΔL, and does not need to be updated if the metadata is the same as compared with the metadata set in the metadata table. If so, it is determined that it needs to be updated. In the example of this embodiment, the state of the association (metadata table) in the state of FIG. 3C is FIG. 5A, and no metadata is defined. Therefore, it is necessary to newly set metadata indicating the intimacy between the object 401c in which Mr. C is reflected and the object 401d in which Mr. D is reflected. Thus, the metadata is always updated at the first use. In the second and subsequent processes, if the magnitude relationship of the distance between the objects does not change even if the objects are moved, NO is determined. When updating the metadata table (YES in step S27), in step S28, the identifying unit 305 establishes a parent of the metadata table of Mr. C and the metadata table of Mr. D as shown in FIG. It is decided to give metadata of density 1. If no metadata update is necessary (NO in step S27), the metadata providing process is terminated and the process returns to the caller. In step S29, the assigning unit 306 displays “intimacy 1” shown in FIG. 5B in the table of the object 401c in which Mr. C is stored in the RAM 201 based on the determined metadata. , Metadata “intimate person D” is attached. Similarly, the assigning unit 306 adds metadata such as “intimacy 1” and “intimate person C” illustrated in FIG. 5B to the table of the object 401d in which Mr. D is stored in the RAM 201. Update by granting.

  Thus, the metadata providing process (step S15) is completed, and the result is returned to the caller process. When the metadata providing process (step S15) ends, the display image is updated in step S16. In the present embodiment, in the display image updated here, an arc 403a shown in FIG. 3C is displayed together with a plurality of objects in order to express the given metadata. The arc 403a is centered on the object 401c in which Mr. C is shown, the distance ΔL obtained in step S26 is the radius, and the object displayed at the position indicated on the arc is the object person with respect to the object 401c. The intimacy of is 1. This arc is used as a means for visually expressing the degree of the strength of the relationship between objects (in this embodiment, the degree of closeness) visually. Hereinafter, such a display means is used as a relevance scale. It is defined as

  When the update of the display image (step S16) ends, the information processing apparatus 100 determines whether to end the data display. For example, the data display is ended when an end instruction is issued from the application, when the detection unit 304 detects a predetermined end operation by the user, or when no operation is detected for a certain period of time or more (step S30). YES in S17). In step S 18, the layout information and the metadata table stored on the RAM 201 are stored in the storage 206. If the data display is not terminated (NO in step S17), the process proceeds to S13 again, which is realized by object selection (step S13), movement process (step S14, step S15), and display image update (step S16). The metadata adding process is repeated. FIGS. 3D to 3J show a state where the object to be selected with the finger 402a remains 401c from the state of FIG. 3C and the metadata adding process is performed while changing the objects to be selected with 402b one after another. ing. In the present embodiment, the user operates based on the fact that Mr. C is more intimate in the order of Mr. A, Mr. E, and Mr. D, and Mr. B is as close to Mr. C as Mr. D. As an example, the description will be continued while appropriately omitting the detailed contents of the overlapping processing steps.

  FIG. 3D shows a state where the object 401c in which Mr. C is reflected and the object 401a in which Mr. A is selected are selected (step S13). Since Mr. A is more intimate with Mr. C than Mr. D, the user moves the object 401a in which Mr. A is reflected to the inside of the arc 403a indicating the familiarity 1, as shown in FIG. (Step S14). In accordance with the operation, the information processing apparatus 100 executes a metadata providing process (step S15). As a result of the moving operation, the specifying unit 305 determines that the distance ΔL (C, A) between the object 401c in which Mr. C is reflected and the object 401a in which Mr. A is reflected is the relative distance ΔL (C , D) is specified (step S26). That is, it can be determined that Mr. A's familiarity with Mr. C is greater than Mr. D's familiarity 1 with Mr. C. In addition, since the metadata indicating the closeness of Mr. C and Mr. A is not set in the metadata table referred to at this time (step S22), it is determined that the metadata table needs to be updated. (Step S27). In order to set the familiarity greater than the familiarity 1, the specifying unit 305 needs to set the familiarity 2 for Mr. A in the metadata table of Mr. C shown in FIG. Similarly, it is determined that it is necessary to generate intimacy 1 for Mr. C from Mr. A's intimacy table shown in FIG. 5B (S28). Therefore, as shown in FIG. 5C, the granting unit 306 stores metadata of “intimacy 2” and “intimate person A” in the table of C, and “intimacy 1” in the table of A. Metadata “intimate person C” is assigned (step S29). Then, the display control unit 301 displays the arc 403b indicating the familiarity 2 (step S16).

  FIG. 4F shows a state where an object 401c in which Mr. C is reflected and an object 401e in which Mr. E is selected are selected according to the user's operation (step S13). Mr. E is more intimate with Mr. C than Mr. D and less intimate with Mr. A. Accordingly, as shown in FIG. 4G, the user moves the object 401e in which Mr. E appears to a position inside the arc 403a indicating the familiarity 1 and outside the arc 403b indicating the familiarity 2. (S14). As a result, the relative distance ΔL (C, E) between the object figure 4 (G) 401 c where Mr. C is reflected and the object figure (G) 401 e where Mr. E is reflected is the relative distance ΔL ( C, D). Similarly, it is specified that it is larger than the relative distance ΔL (C, A) between Mr. C and Mr. A (S26). Then, since there is a difference from the contents of the currently referenced metadata table, it can be determined that updating is necessary (step S27). The intimacy of Mr. E is greater than the intimacy 1 of Mr. D for Mr. C, less than the intimacy 2 of Mr. A, greater than intimacy 1 and smaller than intimacy 2 Because it is necessary. Therefore, as illustrated in FIG. 5D, the specifying unit 305 determines that the familiarity 3 is assigned to the metadata table of Mr. C and the familiarity 1 is assigned to the metadata table of Mr. E. Is added to the metadata table. At this time, the metadata of A for C is updated from “intimacy 2” and “intimate person A” to “intimacy 3” and “intimacy person A”. Further, the display control unit 301 updates and displays the arc 403b indicating the familiarity 3 and the arc 403c indicating the familiarity 2. In this way, by dynamically determining a value indicating intimacy and updating (correcting) the metadata table, it is possible to flexibly add metadata to the object. Further, when the value indicating the intimacy is dynamically determined, if the interval of the relevance scale (the interval between the arc 403a and the arc 403b in FIG. 3F) is narrow as shown in FIG. Hard to produce degrees. Therefore, in the present embodiment, the display control unit 301 automatically adjusts the gap between the relevance arcs by automatically moving adjacent relevance arcs outward or inward. In the example of FIG. 3G, the relevance arc 403a is automatically moved outward in order to newly display the arc 403c. Along with the movement, the object 401d in which Mr. D is reflected is also moved outward. This function makes it possible to maintain a display state that is always easy to see.

  FIG. 4H shows a state where the object 401c in which Mr. C is reflected and the object 401b in which Mr. B is selected are selected (S13). Since Mr. B is as close to Mr. C as Mr. D, the user places the object 401b in which Mr. B appears in the same position as the arc 403a indicating the familiarity 1, as shown in FIG. Move to become. As a result of the operation, the relative distance ΔL (C, B) between the object 401c where Mr. C is reflected and the object 401b where Mr. B is reflected is comparable to the relative distance ΔL (C, D) between Mr. C and Mr. D. It becomes. The specifying unit 305 determines that the area where the arc 403a is displayed (a range in consideration of the thickness of the arc) overlaps with the center coordinates of the object 401b without newly obtaining ΔL (C, D). From the acquisition result 303, the magnitude relationship of the distance ΔL can be specified. That is, in Mr. C's table shown in FIG. 5D, it is not necessary to change the value indicating the closeness with other objects set so far, but a new object is displayed in the Closeness 1 column. It is necessary to add a value based on the relationship with 401b. On the other hand, it is necessary to newly set the familiarity level 1 metadata for the table of Mr. B shown in FIG. 5D (YES in S27). For this purpose, the specifying unit 305 generates an item of closeness 1 in the table of Mr. B as shown in FIG. 5E (S28). Then, the assigning unit 306 additionally gives “intimacy 1” and “intimate person B” shown in FIG. 7E to the object 401c in which Mr. C is shown based on the generated relevance. . Similarly, the assigning unit 306 assigns “intimacy 1” and “intimate person C” shown in FIG. 5E to the object 401b in which Mr. B is reflected (S29). FIG. 4J shows a display image in which the user has completed the work of organizing and classifying human photograph objects based on the intimacy with respect to Mr. C.

  According to this embodiment, if metadata can be given even once, the information processing apparatus 100 can easily restore the familiarity classification state shown in FIG. Alternatively, this metadata can be used for other purposes. For example, from a large number of photos taken at events such as weddings and graduations, only the photos of Mr. C and photos of a person who has a relationship with Mr. C more than 2 moved are extracted and printed, It can also be used to create albums. In that case, using a well-known face identification technique that can identify a photo to which the same person has moved, metadata that indicates the person who has moved to the photo is also present in the photo object other than the object used in this sorting and classification work Is given. If it is used, it is possible to select photos of Mr. C and Mr. A and Mr. E who have a closeness of 2 or more. In the description of the present embodiment, since the object 401c to which Mr. C is moving is used as a key object, the frequency indicating the closeness of other persons with respect to Mr. C is given to each object as metadata, and the work is completed. However, a value indicating the closeness between the other objects can be set in each metadata table by continuing the operation using another object as a key object. It is also possible to extract and print photographs to be distributed to each person individually corresponding to intimate persons, which increases convenience. When the operation is continued by changing the key object, the metadata set so far is displayed as a relevance scale at the stage of updating the display image in step S24. By doing so, the user does not have to perform an unnecessary operation for giving duplicate contents.

  In the present embodiment, metadata of each object is given by storing a metadata table corresponding to each object in the storage separately from the object. In this way, metadata can be handled without having to change the file of the object itself or increase the file size. However, the method of assigning metadata to an object is not limited to this, and for example, metadata can be embedded in the object file itself. In this case, there is an advantage that when the file storage location is moved or read by another application, the object file is only the object itself.

  In the present embodiment, when the metadata is changed by an operation for moving the movement of the object, the metadata table is updated by storing new metadata. However, when the metadata is changed, the number of items in the metadata table may be increased, and metadata including the history information may be assigned to each object.

  So far, in the present embodiment, the case where two objects are selected has been described in order to simplify the description. When an operator moves a displayed object arbitrarily on a table-type display in a conference room system, the operation of touching two locations simultaneously using both arms is fundamental. It is. However, it is also possible to select and operate two or more of the displayed objects using a finger or the like to add metadata. In this case, in the object classification process in step S21, the moving object may be classified as a sub-object as in the present embodiment. Alternatively, the first selected object may be classified as a key object, and then the object selected until the touch on the key object is released may be classified as a sub-object.

  Further, in the present embodiment, only when the operation of moving the selected object is performed (YES in step S14), the metadata is added. As a modified example, metadata can be once added to the object at the stage where the object is selected (step S13) in the initial image. In this case, when the selected object is determined, the object classification process by the classification unit 307 and the layout acquisition process in the initial image by the layout acquisition unit 303 are executed. Based on the result, the specifying unit 305 determines metadata to be assigned to each object based on the magnitude relationship between the objects, and the adding unit 306 adds the determined metadata. Thereafter, when the object is moved, the metadata set from the layout of the initial image is changed as necessary.

  In this embodiment, an arc is displayed as the relevance scale, but arrows, rectangles, characters, and the like may be displayed as the relevance scale. Further, transparency (α value) can be used as the relevance scale, and the object can be displayed with higher transparency as the relevance of the object increases from the object with the higher relevance. Furthermore, when the display object is displayed in 3D, it is possible to use the depth direction (z-axis direction) as the relevance scale, and control so that an object with low relevance is displayed at a deep depth. .

  As described above, according to the present embodiment, useful metadata can be assigned to each object in accordance with an operation such as moving the displayed object. Specifically, a frequency indicating that the degree of association between objects is higher as the distance between the objects is smaller can be given to both the corresponding objects as metadata. The visual relationship between displayed objects, such as the distance between objects, is information that is most easily recognized by a user operating the information processing apparatus 100. Furthermore, according to an intuitive operation such as a touch operation, an operation for changing the visual relationship between such objects can be easily performed. Therefore, in this embodiment, by intuitively manipulating the visual relationship between objects, metadata that makes it easy to understand the relationship between objects can be given to the object. Can be provided. Note that the visual relationship between the objects used to determine the metadata takes into account not only the distance between the objects, but also the size relationship of the display size of the object and the positional relationship of the object placement position. Also good.

[Second Embodiment]
Next, a second embodiment of the present invention will be described with reference to the drawings. The system outline diagram, hardware configuration diagram, and functional block diagram are the same as those in the first embodiment described with reference to FIGS. 1 and 2. The same reference numerals are given to those already described, and description thereof is omitted.

  In the first embodiment, two or more objects are selected from a plurality of displayed objects, and metadata based on the magnitude relation of the distance between them is given only to the selected objects. In contrast, in the second embodiment, at least one object is selected from a plurality of displayed objects, and metadata is assigned to all objects based on the distance between the selected object and all other objects. To do.

  FIG. 6 is a process flowchart for explaining the data display process in the second embodiment. The processing steps from the start of the data display processing to step S12 are executed in the same manner as in the first embodiment. In the subsequent step S13A, the operation unit 302 determines whether at least one object is selected from the plurality of displayed objects. That is, if the position information of at least one place touched by the user on the display 101 acquired from the detection unit 304 indicates the inside of the displayed area of the object acquired from the display control unit 301, the process proceeds to step S14. . In step S14, as in the first embodiment, it is determined whether to move the object selected in step S13A. If it is determined that the object is to be moved, the process proceeds to the metadata providing process in step S15A.

  In the metadata providing process (step S15A) in the second embodiment, an object classification process is performed. Accordingly, step S22 is first executed, and the assigning unit 306 refers to the metadata tables of all objects and holds the current metadata in the RAM 201. In the subsequent processing, as in the first embodiment, the movement position of the moved object is acquired (step S23), the display image is updated accordingly (step S24), and repeated until the movement is completed (step S23). NO in S25). When the operation unit 302 determines that the movement of the object has been completed (YES in step S25), the layout acquisition unit 305 acquires the changed layout, and the visual on the display of all objects in the layout in which the specific unit is displayed. A specific relationship is specified (step S26). Similar to the first embodiment, also in the second embodiment, the visual relationship on the display specified in step S26 is the magnitude relationship of the distance ΔL between objects. Accordingly, the specifying unit 305 obtains the distance ΔL between all the displayed objects and the selected and moved objects, and specifies the magnitude relationship. In the next step S27, the specifying unit 305 confirms the difference from the current metadata held in the RAM 201, and determines whether the metadata needs to be updated. At the first use, a determination of YES is always made. Further, if the magnitude relationship of the distance between the objects does not change even if the object is moved, NO is determined. When updating the metadata (YES in step S27), the process proceeds to step S28A. If updating of the metadata is not necessary (NO in step S27), the metadata providing process is terminated and the process returns to the caller.

  In step S <b> 28, the specifying unit 305 assigns all displayed objects to the displayed metadata based on the magnitude relationship of the distance ΔL with the selected object identified in step S <b> 26. Determine the metadata. That is, a value (frequency) indicating that the degree of association between objects is higher as the distance ΔL between objects is smaller is determined as metadata to be newly added to the metadata table. In step S29, the assigning unit 306 assigns the determined metadata to all the objects being displayed. That is, information indicating the determined value (frequency) and the corresponding partner is stored in the metadata table of each object. When the metadata providing process (step S15A) is completed, the result is returned to the caller process, and the display image is updated in step S16 as in the first embodiment. In the first embodiment, the arc of the relevance scale is displayed in the display update process in step S16A in order to improve the operability when the user actively sets the relevance. In the present embodiment, such display is not performed, and the display for highlighting the selected object is canceled. In step S17, it is confirmed whether or not to end the data display process. As in the first embodiment, for example, when there is an end instruction from an application, when the detection unit 304 detects a predetermined end operation by the user, or when no operation is detected for a certain period of time or the like. The data display is terminated (YES in step S17). In step S 18, the layout information and the metadata table stored on the RAM 201 are stored in the storage 206. If not completed (NO in step S17), the process returns to step S13A and the process is repeated.

  In the first embodiment, each time an operation for selecting and moving an object is performed, the selected object is classified into a key object and a sub-object. Then, the strength of the degree of relevance of the sub-object to the key object is given as metadata only to the selected object. On the other hand, in the second embodiment, the process of classifying the selected object is not performed, and at least one selected object is always handled in the same manner as the key object in the first embodiment. That is, each time the selected object is moved, the strength of the degree of association with the selected object is assigned as metadata to all the other objects being displayed. Further, when the user changes the distance between the objects, a process of automatically assigning predetermined metadata in the background is performed. In other words, metadata is automatically assigned by simply moving the displayed objects, organizing and classifying them, rather than performing the operation with the user trying to give the object a degree of association. An interface with a low workload can be provided. In order not to make the user aware of the process of giving metadata, the relevance scale is not displayed in step S16A. However, also in the second embodiment, if the relevance scale is displayed after the metadata is added, the user can visually confirm whether or not the relevance degree as intended by the operation is correctly given. Therefore, a relevance scale may be displayed as a method for confirming whether accurate metadata is given.

  In this embodiment, when at least one selected object is moved, metadata is assigned to all display objects, but this is not a limitation. As a modification, when moving the selected object with the operation unit 302, if the moving direction is acquired in addition to the moving position, metadata is given only to one or more objects existing ahead of the moving direction. It is also possible to do. As a result, even if a plurality of objects are not selected at the same time as in the first embodiment, it is possible to perform control such that metadata is limited to an arbitrary range of objects.

  In the present embodiment, similarly to the first embodiment, metadata of each object is given by storing a metadata table corresponding to each object in the storage separately from the object. However, for example, metadata may be embedded in the object file itself. In the present embodiment, when the metadata is changed by an operation for moving the movement of the object, the metadata table is updated by storing new metadata. However, when the metadata is changed, the number of items in the metadata table may be increased, and metadata including the history information may be assigned to each object.

  As described above, by intuitively manipulating the visual relationship between objects, metadata that makes it easy to understand the relationship between objects can be given to the object, so this embodiment is highly convenient for the user. An interface can be provided.

[Third Embodiment]
Next, a third embodiment of the present invention will be described with reference to the drawings. The system outline diagram, hardware configuration diagram, and functional block diagram are the same as those in the first embodiment described with reference to FIGS. 1 and 2. In the third embodiment as well, elements that have already been described are assigned the same reference numerals, and descriptions thereof are omitted.

  In the first and second embodiments, a value (frequency) indicating the degree of relevance between objects is assigned to each object as metadata based on the magnitude relationship of the distance between the displayed objects. In the third embodiment, a value indicating the importance level of an object (based on a size relationship between displayed sizes (areas) of the object and a positional relationship indicating whether the objects exist in the vertical, horizontal, or horizontal direction of each other ( Frequency) is assigned to each object as metadata. In the third embodiment, similarly to the first and second embodiments, the table type display 101 of the conference system in FIG. 1A is used to operate a plurality of objects displayed on the display 101. More specifically, an example in which metadata is given to an operated object when an operation for determining the layout of a photo album, the layout of a project book, the layout of a magazine page, etc. by one or more people is performed. explain.

  FIG. 7 is an explanatory diagram of the metadata providing process in the present embodiment. 8A and 8B are processing flowcharts showing data display processing of the information processing apparatus 100 in this embodiment. FIG. 9 is a metadata table of each object created at this time. In addition, the highlighted part in each table of FIG. 9 shall show the location updated by each operation. Details of the data display process and the metadata providing process of this embodiment will be described with reference to FIGS. 7, 8, and 9.

  When starting the data display process, the information processing apparatus 100 executes step S30. Also in the present embodiment, the data display process is started as the image processing apparatus 100 is powered on. In step S <b> 30, the object reading unit 300 acquires an object to be displayed from the storage 206 and holds it in the RAM 201. In step S <b> 31, the display control unit 301 creates an initial display image by randomly arranging objects held in the RAM 201, and outputs the initial display image to the display 101. An example of the initial image displayed on the display 101 at this time is shown in FIG. A plurality of objects including objects with object IDs 1 to 5 are displayed at random. An area 702 in FIG. 7 is a layout frame, and objects are sequentially selected from randomly displayed objects and moved into the layout frame to determine the layout. The save button 703 is a button for saving the progress of layout adjustment.

  When the initial image display (step S31) ends, in step S32, the information processing apparatus 100 detects a touch operation on the display 101 and determines whether an object is selected. That is, the operation unit 302 determines whether the touch position information acquired from the detection unit 304 overlaps the display area of the object acquired from the display control unit 301, and selects the corresponding object if they overlap. If no object is selected (NO in step S32), the process proceeds to step S38. If an object has been selected (YES in step S32), the process of step S33 is subsequently executed. In step S <b> 33, the information processing apparatus 100 determines whether the selected object moves using the touch position information detected by the detection unit 304. That is, when the detection unit 304 detects that the position of the touch on the display 101 is moved without being released, an operation for moving the corresponding object by the operation unit 302 is executed. If the object is moved (YES in step S33), the layout editing process (step S34) is started. If the object is not moved (NO in step S33), the process proceeds to step S38.

  FIG. 8B is a process flowchart showing the layout editing process in step S34. When the layout editing process (step S34) is started, the information processing apparatus 100 first determines in step S40 whether or not the moved object exists inside the layout frame. If the object is not inside the layout frame (NO in step S40), the layout process ends as it is and returns to the caller. If the moved object is inside the layout frame (YES in step S40), the process of step S41 is subsequently executed. In step S <b> 41, the assigning unit 306 refers to the metadata table stored in the storage 206 attached to the object, and stores it in the RAM 201. When the object does not have a metadata table, an empty table is generated and referenced as shown in FIG.

  Next, in step S <b> 42, the operation unit 302 acquires the position where the object has been moved from the position information obtained from the detection unit 304. Further, when an operation for changing the size for displaying the object is performed, the changed size is acquired from the detection unit 304. In step S43, the display image is updated so that the user's operation is reflected. Next, in step S44, it is determined whether a series of operations for changing the layout in the layout frame has been completed by moving the object and changing the size.

  In FIG. 7B, the user selects an object 701c having an object ID of 3 from the object group displayed in the initial image by a touch operation, moves the object 701c within the layout frame, and arranges it by adjusting it to a desired size. Indicates the state. Here, the operation for adjusting the size of the object is an operation called pinch in which the user touches the object with two fingers (704a and 704b) and widens the interval between the fingers while maintaining the touch. To do. FIG. 7C shows a state in which the object with the object ID 4 is moved from the object group into the layout frame and arranged at the lower left of the object 701c in the second layout editing process. When the layout change is completed (YES in step S44), the process proceeds to step S45. If the layout change is not completed and the operation is continued (NO in step S44), the process returns to step S41.

  In step S45, the specifying unit 305 determines whether a plurality of objects exist in the layout frame in the currently displayed layout based on the acquired result. If there are a plurality of objects (YES in step S45), the process proceeds to step S46. If a plurality of objects do not exist (NO in step S45), the process proceeds to step S48. In FIG. 7B, since only the object 701c exists in the layout frame, the process proceeds to step S48. The processing after step S48 will be described later. On the other hand, in FIG. 7C, since there are two objects, the object 701c and the object 701d, in the layout frame, the processing step proceeds to step S46.

  In step S <b> 46, the layout acquisition unit 303 acquires the position information and the size of the object on the display image as layout information related to a plurality of objects existing in the layout frame, and stores them in the RAM 201. Here, the position information of the object is the center coordinates of the object when the layout frame is considered as a coordinate plane, and the size corresponds to the area of the object. Next, in step S47, the specifying unit 305 specifies the visual relationship on the display of a plurality of objects existing inside the layout frame based on the layout information acquired by the layout acquiring unit 303, and stores the information in the RAM 201. Hold. In the third embodiment, the visual relationship on the display is first the size relationship between the objects and the positional relationship between the objects in the vertical and horizontal directions. In FIG. 7C, the object “3” 701c is displayed in a size (area) four times larger than the object “4” 701d. Therefore, the specifying unit 305 specifies the magnitude relationship and stores it on the RAM 201.

  Next, in step S48, the specifying unit 305 determines metadata to be given to each object in the layout frame. In the present embodiment, first, based on the size relationship of the size of the object in the layout frame specified in step S47, a value (frequency) indicating the importance of the object (favorite for the user) is set to each object as metadata. To do. That is, the importance of the object is higher as the size of the object is larger, and the importance of the object is smaller as the object is smaller. Further, for objects of the same size, the importance is set based on the positional relationship of the objects within the layout frame, assuming that the object displayed at the upper left of the layout frame is an object having a higher importance. In accordance with the importance determination method, metadata (degree of importance) to be given to all objects existing in the layout frame is determined and stored in the RAM 201. However, as shown in FIG. 7B, when there is only one object in the layout frame, the metadata determination method based on the relationship between a plurality of objects is not applied and importance 1 is set. As shown in FIG. 9B, the importance of the object “3” 701c is 1. In the example of FIG. 7C, which is the second processing, since the object “3” 701c is large and the object “4” 701d is small, the importance of the object “3” starts from 1 as shown in FIG. 2 and the importance of the object “4” is set to 1. Further, in step S48, it is determined to give each object's ID as metadata indicating a related object to adjacent objects in the layout frame. Accordingly, the object “4” and the object “3” are respectively added to the related object column of the object “3” and the object “4” in the metadata table as shown in FIG. 9C.

  In step S49, the assigning unit 306 assigns the metadata determined in step S48 and stored on the RAM 201 to each object. That is, the table is updated by storing the determined metadata in the table of each object held on the RAM 201. With the above processing, the layout editing processing in step S34 is completed, and the result is returned to the caller.

  When the layout editing process is completed, the display control unit 301 updates the display image in step S35. When the update of the display image ends, in step S36, the information processing apparatus 100 determines whether or not the save button has been pressed. That is, it is determined whether a touch operation for pressing the save button has been performed based on whether the touch position information obtained from the detection unit 304 by the operation unit 302 is within the area where the save button is displayed. If the save button has not been pressed (NO in step S36), the process proceeds to step S38. When the save button has been pressed (YES in step S36), step S37 is executed, and the information processing apparatus 100 saves the layout information and metadata table in the layout frame held on the RAM 201 in the storage 206. . In step S38, it is determined whether or not to end the data display. For example, the data display is ended when an end instruction is issued from the application, when the detection unit 304 detects a predetermined end operation by the user, or when no operation is detected for a certain period of time or more (step S30). YES in S38). If the data display is not finished (NO in step S38), the process proceeds to S32 again, and the layout editing process in the present embodiment is repeated.

  An example of repeating the layout editing process will be described with reference to FIG. In addition, the detail about the process step which overlaps in the following description is abbreviate | omitted. FIG. 7D shows a display image updated by the operation of selecting the object with the object ID “1” from the object group and placing it in the layout frame following the state of FIG. S43). In this case, the object “1” is smaller than the object “3”, but is the same size as the object “4”. Furthermore, the object “4” is closer to the upper left than the object “1”. In accordance with such a visual relationship on the display of the object, the metadata table is updated as shown in FIG. 9D (steps S46 to S49). That is, the object “3” has an importance level 3, the object “4” has an importance level 2, and the object “1” has an importance level 1. The related object information also includes the objects “1” and “4” for the object “3”, the objects “1” and “3” for the object “4”, and the object “3” for the object “1”. , “4” are set. As can be seen from FIG. 7D, the objects adjacent to the object “3” are the object “1” and the object “4”. The same applies to the object “1” and the object “4”. FIG. 7E shows a display image updated by an operation of selecting an object with the object ID “2” from the object group and placing it in the layout frame (step S43). The size of the object “2” is the same as the objects “1” and “4”, and is farther from the upper left position of the layout frame than the objects “1” and “4”. As a result, the metadata table is updated as indicated by the highlighted portion in FIG. 9E (steps S46 to S49). FIG. 7F shows a display image updated by an operation in which an object with an object ID of 5 is selected from the object group and placed in the layout frame (step S43). The size of the object 5 is larger after the object 3 and larger than the objects “1”, “2”, and “4”. Therefore, the importance is the next higher than that of the object 3. As a result, the relevance level table is updated to the contents as indicated by the highlighted portion in FIG. 9F (steps S46 to S49).

  After laying out the objects “1” to “5” as shown in FIG. 7G, when the user presses the save button in step S36, the meta data shown in FIG. 9F is displayed together with the current layout information. A data table is also held in the storage 206.

  As described above, as described in the present embodiment, it is possible to automatically add metadata between objects and favorite (importance) information by a simple operation of laying out the objects displayed on the display. . In the present embodiment, an example in which a photographic object is laid out in a conference system having a table-type display has been described. However, the present invention is not limited to the conference system, and can be used for, for example, a tablet-type terminal as shown in FIG. 1B, a printing apparatus, a photo printing dedicated terminal installed in a store, or an album creating apparatus. By using such a system, the user can easily create a photo album layout, a project book layout, a magazine page layout, and the like by an intuitive operation. In addition, by using the automatically assigned metadata, when creating another layout candidate, use related objects as recommended object information, or use it to narrow down the search by extracting highly important objects. I can do it.

  In the present embodiment, the size of the object is used as an index of the importance of the object. In this case, for example, a threshold value such as “set different importance when there is a difference of 20% or more in the size (area) of the object” may be provided. By doing so, it is possible to avoid making a difference in importance unintentionally even when the user wants to change the size slightly for convenience of layout balance or the like. In addition to the size of the object, for example, the number of times the object is edited in the layout frame may be used as an index. In addition, a menu for setting the display time of images and effects for display is provided separately, and when creating the layout and configuration of image slideshows with this system, the display time of the photo objects to be displayed as slideshows is used as an index of importance. You can also That is, the longer the displayed object, the higher the importance, and the higher importance object can be extracted and used when searching. In the present embodiment, the object displayed at the upper left of the layout frame is set as an important object. For this reason, when the positional relationship between the objects is specified in step S47 and the information is stored in the RAM 201, the upper left position of the layout frame is used as an index representing the importance of the object. However, it is not limited to this. For example, the center of the layout frame may be the position with the highest importance. In addition, the user who performs the layout may be allowed to define which position is to be a position having a high degree of importance before the layout is performed.

  In the present embodiment, similarly to the first embodiment, metadata of each object is given by storing a metadata table corresponding to each object in the storage separately from the object. However, for example, metadata may be embedded in the object file itself. In the present embodiment, when the metadata is changed by an operation for moving the movement of the object, the metadata table is updated by storing new metadata. However, when the metadata is changed, the number of items in the metadata table may be increased, and metadata including the history information may be assigned to each object.

  As described above, by intuitively manipulating the visual relationship between objects, metadata that makes it easy to understand the relationship between objects can be given to the object, so this embodiment is highly convenient for the user. An interface can be provided.

(Other embodiments)
In the present embodiment described so far, a photograph of a person is used as an object to which metadata is added. However, as described above, the information processing apparatus 100 can handle not only a still image but also data digitized in various formats such as voice, handwritten characters, text, images, and moving images.

  The visual relationship of objects specified from the layout is, for example, color, in addition to the relationship between the distances between the objects, the size relationship at which the objects are displayed, or the positional relationship at which the objects are arranged. The relationship may be based. For example, it is possible to set a visual relationship by color to each object by a user operation, and thereby give metadata indicating the degree of association or priority to the object. As an example, when the operation unit 302 and the detection unit 304 determine that the selected object has been tapped (by tapping the display), a color frame or check mark corresponding to the number of taps is displayed on the object. Attached. Then, based on the color, meta data indicating indicators such as “an object with a frame of a color close to red has a higher degree of urgency” and “an object with a frame of a distant color set has a lower degree of association”. Data can also be added.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

Claims (9)

An information processing apparatus that displays a plurality of objects on a display unit,
Obtaining means for obtaining a layout of a plurality of objects displayed on the display unit;
A specifying means for specifying a visual relationship on display between the plurality of objects in the layout acquired by the acquiring means;
The data based on the identified visual relationships, and adding means for adding the metadata for each of the plurality of objects,
An operation means for performing an operation of selecting at least one object among the plurality of displayed objects;
Classifying means for classifying key objects in at least one object selected by the operation means ;
The acquisition unit acquires a layout changed along with the movement when an operation for further moving at least one of the objects selected by the operation unit is performed;
The specifying means specifies the visual relationship on the display between the key object and each of the other displayed objects,
The assigning means assigns metadata based on a result specified by the specifying means to each of the plurality of objects,
The visual relationship between the plurality of objects is a relationship based on one or a combination of a distance between the objects, a size at which the object is displayed, or a position at which the object is arranged. An information processing apparatus. The information processing apparatus according to claim 1 , wherein the classification unit classifies an object selected first among two or more objects selected by the operation unit as a key object. The information processing apparatus according to claim 1 , wherein the classifying unit classifies an object that is not moved among two or more objects selected by the operation unit as a key object. The operation means selects two or more objects from the plurality of objects displayed on the display unit,
The assigning means assigns metadata based on a result specified by the specifying means only to the two or more objects selected by the operating means among the plurality of objects being displayed. The information processing apparatus according to any one of claims 1 to 3 . The classification means classifies other objects not classified as key objects among two or more objects selected by the operation means as sub-objects,
The specifying means specifies the visual relationship on the display between the key object and the sub-object,
The assigning means is a result specified by the specifying means only for the two or more objects selected by the operating means among the plurality of objects being displayed based on the result specified by the specifying means. the information processing apparatus according to any one of claims 1 to 4, characterized in that to attach metadata based on. The display control unit according to any one of claims 1 to 5 , further comprising a display control unit that visually displays information indicated by the metadata on the display unit together with the object to which the metadata is added by the adding unit. The information processing apparatus according to item. The metadata, a plurality of objects of each information processing apparatus according to any one of claims 1 to 6, characterized in that a value indicating the degree at a certain index in the display. A program that causes a computer to function as the information processing apparatus according to any one of claims 1 to 7 by being read and executed by a computer. A control method for an information processing apparatus for displaying a plurality of objects on a display unit,
An acquisition step of acquiring a layout of the plurality of objects to be displayed by an acquisition unit;
A specifying step of specifying a visual relationship on display of the plurality of objects in the layout acquired by the acquiring unit by the specifying unit;
A granting step of granting metadata to each of the plurality of objects based on a result specified by the specifying unit;
A selection step of receiving and executing an operation of selecting at least one object among the plurality of displayed objects by an operation means;
The classifying means, at least one object selected by the selecting step, have a classification step of classifying the key object,
In the acquisition step, when an operation for further moving at least one of the objects selected in the selection step is accepted, a layout changed with the movement is acquired,
In the specifying step, the visual relationship on the display between the key object classified in the classification step and each of the other objects to be displayed is specified,
In the assigning step, metadata based on the result specified in the specifying step is assigned to each of the plurality of objects,
The visual relationship between the plurality of objects is a relationship based on one or a combination of a distance between the objects, a size at which the object is displayed, or a position at which the object is arranged. An information processing apparatus.

Images