JPH0981306A - Information processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To process and display the information required by a user via a simple operation and with high efficiency by observing the movements of the user's eyeballs to sequentially generate the velocity components of the eyeball movements and then calculating the activity of the eyeball movements to estimate the importance of the information. SOLUTION: The line of sight of a user 101 are measured by a line-of-sight detector 102 and sent to a visual point arithmetic part 103 where the voltage value is converted into a position included in a screen. Then an eyeball movement type decision part 104 decides the types of momentary movements of eyeballs, based on the output data of the part 103. An information acquisition rate calculation part 105 calculates an information acquisition rate from the time transition of the eyeball movement type. A visual activity standard calculation part 106 decides whether the eyeball movements are momentarily active or not based on the hitherto calculated information acquisition rates. An importance information generation part 107 operates the information acquisition rate in every time unit and outputs the operation result and the time unit as the importance information. Then the time unit is decided at an information unit detection part 109 and sent to the part 107.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a multimedia information processing apparatus for displaying / recording / reproducing / editing video / audio / documents in an integrated manner.

[0002]

2. Description of the Related Art It is well known that the performance of a microprocessor installed in a personal computer or the like has been remarkably improved, and an environment in which a wide variety of personal computers can process a large amount of information at high speed is being prepared. With the recent development of optical communication technology, communication control technology, magneto-optical disk, and memory device by high-density integrated circuit, on the one hand, the information transmission path and on the other hand, the fact that the information storage device has a large capacity also acts. A world is being realized in which a large amount of information can be electronically obtained with inexpensive devices even at home and office. Also, since information can be transmitted with a simple operation, the privilege of information transmission, which was previously limited to broadcasting stations and newspaper publishers, has been extended to general users, and more and more information is being exchanged. is there. This can be seen from the current spread of personal computer communications, electronic mail, electronic news and so on. Further, not only a personal computer but also a terminal capable of exchanging information in this way has spread in the form of various devices such as an electronic notebook, a mobile phone, and a facsimile. The types of information that can be handled are expanded from conventional character information (text data) to voice / music information, still image information, and moving image information, and the quality of each is dramatically improved. There is.

However, in many cases, the conventional method is adopted as the information classification method. Taking the world of electronic news as an example, there are hundreds of newsgroups (= “kukuri” for each topic), and dozens of posts are posted to each newsgroup per day. The user must spend a considerable amount of time searching for the news that he / she needs / interests in. This is each news
"News group → News", a very simple
And it is because the users are usually dealt with in a layered structure that cannot be defined by themselves. Even for music tapes and video tapes, the start position of each song / program can be determined by detecting the silent part or the recording start index signal, but the content of the song / program, or the scene or part of particular interest, can be determined. Since the pointing information cannot be recorded, the user has no choice but to write on their labels etc. themselves.

[0006] The development for automating the work of classifying the information or extracting the characteristic portion from the information is still actively carried out. However, many of them are aimed at substituting human judgment mechanisms, require advanced artificial intelligence, and are said to be costly at the moment, both in terms of development time and cost. I have no choice. In addition, such an automatic classification system requires the user to indicate what he / she wants, and is difficult to handle for users who have unclear vision of what they want to see and what kind of information they want to obtain. It can be assumed that such uninformed users will increase dramatically in an era when information is overflowing.

Rather, what is needed is information indicating that "the place where I feel is important is here", and it is not always necessary to describe the contents.
For example, when an attractive actor appears in only one scene in a movie, it is always necessary to know who the actor is, a man or a woman, and whether the object (subject) is a human. However, if the device just points to the scene, the requirements are often satisfied.
However, in the current flow of information classification, "the device examines the information → extracts characteristic parts as candidates → collates with the request information input by the user → performs classification / presentation"
Because of this procedure, it is necessary to carefully analyze the information content once.

On the other hand, now that general users are becoming sources of information, the demand for information processing tools is increasing. Conventional processing tools include word processors for document information and CAD (Computer Aided D) for graphic information.
esign) and drawing software. However, there are very few general-purpose tools that are easy to use, especially for video and audio. 2 users
We perform "cutting" editing by connecting two decks, searching for the desired location on one deck and playing it back, and recording it on the other deck. "Cutting"
Since at least two decks are required for editing and the operation is complicated, general users tend to avoid editing. Currently, some of the proposed editing methods temporarily store video / audio information in a storage medium such as a computer memory or hard disk and process the information in a computer editing environment. It requires a large-capacity storage medium and is expensive, and since the editing is basically "cutting", no particular improvement in operability can be expected.

The lack of a simple and efficient information automatic classification mechanism and information processing means as described above may greatly hinder the spread of the multimedia integrated environment in the future.

[0008]

As described above, in the conventional information processing apparatus, various types of various information can be efficiently sorted across various types of information, and the user's intention can be reflected and sorted. There was no way to do. For this reason, the user has to spend time and labor for processing the information, and even when this is automatically performed by the device, there is a drawback that it cannot be processed by adapting to the individual user. In addition, conventionally, the information processing method is also complicated, and there is a drawback that the cost tends to be high to realize it.

The present invention has been made to solve the above problems, and provides an information processing apparatus capable of efficiently processing and presenting information required by a user from a large amount of various kinds of information with a simple operation. To do.

[0010]

In order to solve the problem of estimating the temporal change of the importance level without discriminating the subject boundary from the image among the above problems, the information processing apparatus according to the present invention is In an apparatus for presenting image and image and audio information to a user, a means for observing the eye movement of a user who is observing or viewing the information, and a velocity component of the eye movement of the user observed by this means Means for sequentially generating, the means for obtaining the degree of eye movement activity from the velocity component generated by this means, and the degree of importance of the information based on the degree of eye movement activity obtained by this means And means for estimating.

[0011]

In the information processing apparatus according to the present invention, how much the user recognizes the presented information as important can be expressed by utilizing the line of sight. Since part of the user's intention display is obtained from the natural movement of the line of sight, the user is less proficient in operation, and the user can input the intention without being aware that it is being measured. Be seen. In this way, since the part of the original information that the user recognizes as important and the degree of that can be obtained by a simple input method, it is possible to perform processing such as presentation and rearrangement biased to "important part". It is possible, and we can hope to improve the efficiency of information access.

[0012]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A first embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing the configuration of an information processing apparatus according to an embodiment of the present invention. 12 to 14 are flowcharts for explaining the flow of the following processing.

The multimedia information stored in the recording medium 116 is read by the read control unit 114, and passes through the presentation form creating unit 113 to the image information presenting unit 110,
It is presented to the user 101 by the voice information presentation unit 111, the music information presentation unit 112, and the like. Presentation form creation unit 113
Is a video RAM, video processor, or sound manager, speaker driver, or MI.
Refers to the DI interface. In the above process, the television and the video tape recorder (VT
There is no difference from the conventional method of presenting information in multimedia devices such as a combination of R) and a personal computer equipped with a CD-ROM drive.

On the other hand, the line-of-sight of the user 101 viewing the information presented in this way is measured by the line-of-sight detector 102 and sent to the viewpoint calculation unit 103. Any type of line-of-sight detector may be used, but the following description will be made assuming a line-of-sight detector that outputs voltage values of two channels corresponding to the position on the two-dimensional coordinates of the viewpoint. To do. The data sent from the line-of-sight detector 102 to the viewpoint calculation unit 103 is, for example, as shown in FIG. 4a (in FIG. 12, the data as shown in FIG. 4a flows at the location of a. The same applies hereinafter). . The viewpoint calculation unit 103 converts this voltage value into a position on the screen and stores it in the temporary storage unit 108 (see FIG. 1).
2 S1201). The data output by the viewpoint calculation unit 103 is as shown in FIG. 4b. Based on this coordinate data,
The eye movement type determination unit 104 determines the type of eye movement at the time and time, and stores it again in the temporary storage unit 108 (S1203 to S1209 in FIG. 12).

Human eye movements are roughly divided into involuntary eye movements and tracking movements as shown in FIG. 2, and jumping movements can be distinguished from involuntary eye movements and follower movements by looking at the velocity component. is there. FIG. 3 schematically shows an actual line of sight when viewing a moving image. Person 301 in the screen
And two main objects (subjects), that is, the television stand 302 are shown. Circles (◯) in FIG. 3 are viewpoints recorded every 1/60 second. As shown here, slow motion continues while looking at the object (303),
It can be seen that when moving between objects, high speed motion is being performed (304).

In FIG. 5, similarly, the actually recorded viewpoint data are shown with the horizontal axis representing time and the vertical axis representing coordinate values. In the figure, the solid line and the broken line represent the X coordinate and the Y coordinate, respectively. Although the figure shows 3 seconds, it can be seen that the coordinates change abruptly 5 times within an extremely short time (50
1). This corresponds to a jumping movement. On the other hand, a pulsed motion that is much larger than the jumping motion is also seen three times (50
2) but this is a blink. Although such movement occurs due to the characteristics of the line-of-sight detector, it can be distinguished from the jumping movement because the coordinate values hardly change before and after the pulse-like movement.

As described above, the eye movement type determination unit 104 of FIG. 1 distinguishes the eye movement into three types of "blinking", "jumping", and other "gaze".
The data determined by the eye movement type determination unit 104 has a shape as shown in FIG. 4c. It should be noted that moving the viewpoint during blinking should be regarded as "jumping" because it involves movement of the object, and the eye movement type determination unit 104 also makes such a determination. The eye movement type determination unit 104 stores the eye movement type at the time and time again in the temporary storage unit 108.

With respect to human visual characteristics, it is known that the resolution of an image that is visible is reduced within about 200 milliseconds after a jumping motion. This is described, for example, in "Experimental Psychology of Eye Movements" (Nagoya University Press, 1993). Therefore, it can be said that the object can be actually gazed at the normal resolution after 200 ms after the jumping motion. Therefore, the time spent looking at each object is as shown by 503 in FIG. 504 is the time during which the resolution is degraded due to the jumping motion. The "time to look at each object" per unit time in the past, that is, the time excluding the jumping motion and 200 milliseconds thereafter is defined as the "information acquisition rate" (S13 in FIG. 13).
02-S1305). The jumping motion mentioned here includes a case where an object is moved during blinking. On the other hand, when the viewpoint before and after blinking has not changed significantly (502 in FIG. 5), the object is not moved, and therefore, in the following process, it is handled in the same manner as the gaze. In this way, the information acquisition rate calculation unit 105 calculates the information acquisition rate from the time transition of the eye movement type and stores it again in the temporary storage unit 108. This data takes the form of, for example, FIG. 4d.

When the information acquisition rate decreases, it can be said that the visual system tries to see many objects even at the cost of a reduction in resolution due to a jumping motion.
On the contrary, when there are few objects to be viewed on the screen, it is considered that the time when the information acquisition rate is high is long. Therefore, it is considered that the visual system is active in collecting information during the time when the information acquisition rate is decreasing, and the time function for evaluating the activity is defined as "visual activity level". The visual activity level is the time during which the information acquisition rate was decreasing in the past unit time. The time that “the information acquisition rate is going down” is
It is assumed that the minimum value of the information acquisition rate is included in a time section surrounded by two consecutive times at the moment when the second derivative of the information acquisition rate with time becomes zero. That is, it is the time interval 601 in FIG. In FIG. 6, 602 is the information acquisition rate, 603 is the time when the second derivative of the information acquisition rate 602 becomes zero, and 604 is the minimum value of the information acquisition rate 602. Since the actual calculation inside the computer takes discrete values in the time direction, the above "differential" may be replaced with "difference" to perform the calculation.
The visual activity level calculation unit 106 performs such a calculation. The visual activity level calculation unit 106 determines whether the eye movement is “active” or “inactive” at the time and time based on the information acquisition rate calculated up to now and through the above calculation process, and outputs it. This output is again stored in the temporary storage 108 (S1306 to S1308). The output data has a form as shown in FIG. 4e, for example.

The time resolution of the above process is defined by the capability of the processing system of the present invention, and it is divided into time units that are easy for the user to understand. The time resolution defined by the system is, for example, 1/30
Seconds, 1 / 60th of a second, 1 / 100th of a second, etc., and a time unit that is easy for the user to understand is, for example, "cut".
Or "scene" or 30 seconds, 3 minutes. The importance degree information generation unit 107 calculates the information acquisition rate in such a time unit that the user can easily understand, and outputs the calculation result and the time unit as importance degree information. In other words, "cut number 14 is from 3 minutes 23 seconds 15 fields to 3 minutes 29 seconds 27 fields. Its importance is 0.7.
Data such as "3". When a time unit such as the cut here is recorded together with the original information in advance, the information on the start time and the end time can be omitted. In addition, the user may be allowed to select from among the above-mentioned “cut”, “scene”, and how many seconds as to what time unit the importance level information is given.

This time unit is judged by the information unit detecting section 109 and sent to the importance degree information generating section 107. In the case where no break of information is recorded in the original information, the information unit detection unit 109 detects a scene change or the like by detecting the moment when the color tone or the brightness changes abruptly in the image information. Any method may be used to define these information units. Further, when the user particularly wants to correct the information division automatically detected by the information unit detection unit 109, the information division newly defined by the user may be used as the time unit for the importance information. Furthermore, if such a time unit is recorded in advance in the original information,
The information unit detection unit 109 may use the recorded time unit.

Of the importance information within the time unit, the method of calculating the importance is as follows, for example. As described above, "active" and "inactive" of the information acquisition rate are known for each time, so the ratio of the time of "active" to the time unit is adopted as the degree of importance (FIG. 14). Length 1
If a total of 4 seconds was "active" for a 0 second cut, the importance was 0.40, and a total of 2 minutes 6 seconds was "active" for a 3 minute long scene. Then 0.70,
If a movie that is 2 hours in length is "active" for a total of 40 minutes, it will be 0.33. The importance information is again stored in the temporary storage unit 108 and is recorded on the recording medium in association with the original information at an appropriate time. The appropriate time depends on the operating environment of the system. When the recording medium 116 and the mechanism around it are capable of recording at the same time as the information reproduction, the above process is performed simultaneously with the information presentation, and the importance information is recorded in parallel with the reproduction / presentation. Good. In addition, the temporary storage unit 108 is used until the information presentation is completed.
It is also possible to leave the data stored in the storage device and store the importance information in batch after the presentation. On the other hand, in FIG. 1, an optical disc is assumed as the recording medium 116, but the form of the recording medium 116 is not particularly limited to this, and the importance information is recorded on a medium different from the original information. May be. As the recording medium 116 for recording the original information and the importance information, an optical disk or a magneto-optical disk (M
O), a magnetic tape such as a video tape, a magnetic disk such as a floppy disk or a hard disk, an IC memory, or a remote computer through a communication cable may be used.

The importance recorded in this way is effective for comparison with other importance information whose time unit is set by the same method. For example, when the importance is recorded by the above means in a video of a family trip taken with a home video camera, the playback device cuts the importance 0.35 and the importance 0.64. Then, the system presumes that the latter is more likely to be retrieved after a lapse of time, and stores the image / audio information of the first few seconds of the cut in advance in the memory element. Since information stored in advance in the memory device can be accessed at high speed, it can be expected that the user can more smoothly reach the target image. Alternatively, when there are optical discs of a movie with a degree of importance of 0.20 and a movie of a degree of importance of 0.77, the disc changer (playback device capable of storing a plurality of discs) stores the latter in a position where it is easy to take out. The effect of can be expected.

Next, FIG. 7 shows the results of an experiment in which the degree of importance is given by the above method and the agreement with the degree of importance actually considered by the user is compared. In Fig. 7, a series of moving images consisting of five scenes is presented to the subject, and the scene is presented to the subject as "interesting /
I thought it was important. "
6 is a graph showing how important the system using the method of the present invention gives the images of the third place ... The image is a movie, a commercial, a landscape, a music video, and soccer, each of which has a duration of 30 seconds and a continuous 15-second plain screen. If the method of the present invention is valid, the graph should show a monotonically decreasing trend. Looking at FIG. 7, subject KNT
In Fig. 6, it can be seen that the order of importance declared by the subjects and the order of importance given by this system match. Also, in the subject NZS, the order is the same except for the image of the third place. Furthermore, even with data such as subject KTS, 1, 2
It is possible to distinguish between 3rd and 5th place.

In an image generated by CG (computer graphics) or the like, it may be known which position on the screen should belong to which object.
In such a case, there is a possibility that the accuracy of the above method can be improved by utilizing the relationship between the viewpoint position and the object, so that method will be described.

When there is an image as shown in FIG. 8a, it is assumed that the object area is known like the person 801 and the paper 802 in FIG. 8b, and the respective boundary lines are defined. At that time, if the viewpoint is “◯” or “x” in FIG. 8B, the “x” of those viewpoints is in a place that does not belong to either of the two objects. Among the jumping movements shown in FIG. 8b, in the procedure of importance estimation described above, movement within a defined object (803) or movement between defined objects (804) and movement to a non-object area are performed. Move (805), Move in non-object area (806)
Were treated exactly the same. For this reason, if there are many jumping movements in spite of movements related to the non-object area, it is regarded as “active”, and a high degree of importance may be given. When the object area is defined in advance, the jumping movement (movement between objects) is limited only when the end of the movement is a defined object.
Therefore, the following information acquisition rate, visual activity level, and importance may be calculated. In this case, the object definition information recorded on the recording medium 116 is read out by the control unit 11
It is read out in step 4 and passed as eye object information to the eye movement type determination unit 104, for example. Further, even if the object area is not defined in advance, in a system having a slight image analysis function, the system itself may analyze the image to define the object area and use the same. For this analysis, any method may be used as long as the object can be cut out from the image.

On the other hand, when the object information further includes information about the movement of the object, it is possible to determine "active" or "inactive" based on the similarity between the movement of the object and the movement of the viewpoint. It is assumed that there is a moving image as shown in FIG. 9A and the soccer ball 901 is flying toward the player 902. At this time, if the motion vector of the soccer ball is 903, a jumping motion 904 may occur due to following the ball. When the inner product of the vector 903 and the vector 904 is calculated, a large numerical value is obtained when both vectors are “similar”. On the other hand, when the viewpoint moves like the vector 905, this is irrelevant to the motion 903 of the object, and the vector 905 and the vector 903
The dot product of becomes a small (or large negative) number. If motion is defined in addition to area information for all objects in the screen, and if there is no object near the viewpoint that has a large inner product with the viewpoint vector, this is You can think of it as "movement." In such a case, similar to the above case, it is not regarded as a jumping motion for calculating the importance.

Next, a second embodiment of the present invention will be described with reference to the drawings. FIG. 10 is a block diagram showing the configuration of the information processing apparatus according to the embodiment of the present invention. In addition, FIG. 15 is a flowchart illustrating the flow of the following processing.

The process until the image recorded on the recording medium 1013 is presented by the image information presenting unit 1009 is the same as that described at the beginning of the first embodiment, and is omitted here. User 10 listening to images
The line-of-sight 01 is also detected by the line-of-sight detector 1002, and is sent to the temporary storage unit 1006 as a position on the screen by the viewpoint calculation unit 1003 as in the first embodiment (S1501 in FIG. 15). On the other hand, the motion detection unit 1008 uses the recording medium 1
From the image information recorded in 013, the estimation of the object area and its moving direction are detected by using the brightness, color, etc. as clues (S1502). This motion detecting means may be any method including the method currently used. The motion detection unit 1008 sends the object area and its motion at each time to the vector comparison unit 1004. The vector comparison unit 1004 compares this movement with the movement of the line of sight stored in the temporary storage unit 1006 (S1503). FIG. 11 schematically shows a state of calculation performed by the vector comparison unit 1004. The image shows a, b, c in FIG. 11 as time passes.
, The motion detection unit determines that the object region 1101 and the motion vector 110 of the object.
2 is output (a corresponds to a, b corresponds to e, c corresponds to f in the figure). On the other hand, the state of the line of sight at that time is shown in FIGS.
, The line movement tendency 1103 can be obtained by observing the average of the past several data from the respective moments d to f. Now, when the inner product of the motion vector 1102 of the object and the motion vector 1103 of the line of sight is taken, a large numerical value is obtained when moving in the same direction. The viewpoint is on or near the object,
Moreover, when the inner product of the viewpoint and the motion vector of the object is a large number (about 70 to 80% or more of the product of the lengths of the two vectors), the line of sight follows the object, and the object is of interest to the user. It can be said that it was something with. Thus, for one object, if the viewpoint is on or near it,
The inner product of the viewpoint motion and the object motion vector (or the inner product divided by the length of both vectors may be used.
At this time, the numerical value of the result of the division is the cosine of the angle formed by both vectors. The vector comparison unit 1004 stores the result of this inner product in the temporary storage unit 1006. In the importance information generation unit 1005, the inner product thus accumulated and the motion detection unit 100
The position and movement of the object area defined by 8 of the information delimiter defined by the information unit detection unit 1007 (may be determined by the user's operation. This is described in the first embodiment). The temporary storage unit 1006 prepares a format for writing back to the recording medium together with the result calculated in units.
To (S1504). This importance information is recorded on the recording medium 1013 via the write controller 1012 at an appropriate time. The “appropriate timing” has been described in the first embodiment. Further, the original information and the importance information may be different recording media, and since this is also described in the first embodiment, they are omitted here. Furthermore, when the object area and information about its movement are recorded in advance in the original information, the movement detection unit 1008 does not have to function.

As described above, when the importance of an object is recorded based on the movement, the following utilization method is expected. For example, in FIG. 11, it is assumed that an object other than the person 1104 exists and its importance is lower than that of the person 1104. Then, since it is determined that the most main object in this scene (or cut) is the person 1104, the image in which the main object is at the center, that is, FIG. 11b is selected as the key image for scene search.

[0031]

According to the present invention, it is possible to generate additional information of importance to multimedia information, and by using the importance information, the user can easily find the information necessary for himself. You can expect to reach it quickly. Further, by using the line of sight, the input operation of the importance information can be performed without being particularly conscious of the operation. This provides an information retrieval / access environment that is easy to use even for users who are unfamiliar with the operation in an environment where a large amount of multimedia information is available to anyone.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of an information processing apparatus according to an embodiment of the present invention.

FIG. 2 is a diagram showing classification of eye movements according to velocity according to an embodiment of the present invention.

FIG. 3 is a diagram showing an example of actual movement of a human eye according to an embodiment of the present invention.

FIG. 4 is a diagram showing a calculation process of the information processing apparatus according to the embodiment of the present invention.

FIG. 5 is a diagram showing an example of actual human eye movement according to an embodiment of the present invention.

FIG. 6 is a graph showing a processing method of the information processing apparatus according to the embodiment of the present invention.

FIG. 7 is a graph showing a result of an experiment performed by the method shown in the example of the present invention.

FIG. 8 is a conceptual diagram showing movement of a human viewpoint according to an embodiment of the present invention.

FIG. 9 is a conceptual diagram showing movement of a human viewpoint according to an embodiment of the present invention.

FIG. 10 is a block diagram showing a configuration of an information processing apparatus according to an embodiment of the present invention.

FIG. 11 is a conceptual diagram showing a processing method of the information processing apparatus according to the embodiment of the present invention.

FIG. 12 is a flowchart showing a processing method of the information processing apparatus according to the embodiment of the present invention.

FIG. 13 is a flowchart showing a processing method of the information processing apparatus according to the embodiment of the present invention.

FIG. 14 is a flowchart showing a processing method of the information processing apparatus according to the embodiment of the present invention.

FIG. 15 is a flowchart showing a processing method of the information processing apparatus according to the embodiment of the present invention.

[Explanation of Codes] 303 ... Viewpoint when gazing 304 ... Viewpoint movement when eyeball jumps 501 ... Change of viewpoint coordinates when eyeball jumps 502 ... Data when blinking Disturbance 504 ... Time when resolution of visual system is reduced 601 ... Time when eye movement is active 603 ... Time when second difference of information acquisition rate becomes 0 604 ... Minimum point of information acquisition rate 803 ... In object Jumping movement 804 ... Jumping movement between objects 805 ... Jumping movement outside the object 806 ... Jumping movement outside the object 903 ... Ball motion vector 904 ... Motion vector of eye movement following ball 905 ... Irrelevant to ball movement Eye movement vector 1102 ... Object movement vector 1103 ... Viewpoint movement vector

Claims (2)

[Claims] 1. A device for presenting an image and image and audio information to a user, a means for observing the eye movement of a user who is observing or viewing the information, and a device for observing the eye movement of the user. A means for sequentially generating a velocity component of the eye movement, a means for obtaining the degree of activity of the eye movement from the velocity component generated by this means, and a means for obtaining the degree of activity of the eye movement obtained by the means for obtaining An information processing apparatus comprising: means for estimating the importance of information. 2. A device for presenting an image and image and audio information to a user, means for observing the eye movement of the user who is observing or viewing the information, and an object in the image of the information. Means for obtaining the object movement information which is the movement of the object, means for comparing the object movement information obtained by this means with the eye movement observed by the observing means, and this means compares the eye movement and the object movement information. An information processing apparatus, comprising means for estimating an object in an image that is important to a user based on the result.