JP2004213486A - Image processor and processing method, storage medium, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily select one piece of information from a plurality of pieces of information. <P>SOLUTION: When a plurality of slave pictures are displayed on a display part 25, a visual line detection part 24 detects a user's visual line. An image specification part 22 determines which of the slave pictures displayed on the display part 25 is observed from the information related to the user's visual line supplied from the visual line detection part 24. A display control part 23 performs a control so as to display the observed slave picture in a large size. This invention is applicable to a television receiver displaying a plurality of programs simultaneously as slave pictures. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image processing apparatus and method, a recording medium, and a program, and more particularly, to an image processing apparatus that detects a user's line of sight and performs processing according to information of a portion located in the direction of the detected line of sight. And a method, a recording medium, and a program.
[0002]
[Prior art]
In recent years, television broadcasting has entered a multi-channel era. In response to the trend of the era, some television receivers use a single tuner, receive and store multiple programs in a time-division manner, or receive multiple programs using multiple tuners, Some so-called multi-screens generated by dividing one screen into a plurality of small screens can simultaneously display a plurality of programs.
[0003]
In addition, a DVD (Digital Versatile Disc) player and DVD software that can be reproduced by the DVD player are becoming widespread, and a combination of such a player and software realizes a function called multi-angle. The multi-angle is a function that allows an image obtained by capturing one scene from a plurality of angles (angles) to be switched according to a user instruction. It is considered that such a thing can be implemented even in digital television broadcasting which is becoming full-scale.
[0004]
Further, with the spread of personal computers and the enhancement of the performance of the personal computers, it has become possible to perform a plurality of operations in parallel with one personal computer. When performing such a plurality of tasks in parallel, a plurality of windows (images) are displayed on a display connected to the personal computer. In a state where a plurality of windows are displayed, it may be troublesome for the user to select a desired window, but in order to avoid such inconvenience, the user is required to select one of the displayed windows. Also, a method has been proposed in which a desired window can be easily selected. (For example, see Patent Document 1)
[0005]
[Patent Document 1]
JP-A-5-150929
[0006]
[Problems to be solved by the invention]
In the case where the television receiver is displaying images of a plurality of programs at the same time, one image is selected from among a plurality of displayed images at the time of selecting one program or reproduction by a DVD player or the like. When selecting (angle), the user operates a predetermined button provided on a remote controller or the like. However, since many functions have been added to recent television receivers, DVD players, and the like, the number of buttons provided on the remote controller has also increased, and it has been difficult to determine which button should be operated to perform desired processing. There was a problem that it was difficult to understand.
[0007]
The present invention has been made in view of such a situation, and detects a user's line of sight, detects a part that the user is gazing at for a predetermined time or more, and performs a process corresponding to information located in the part. To make it possible.
[0008]
[Means for Solving the Problems]
An image processing apparatus according to an aspect of the invention includes a detecting unit configured to detect a line of sight of a user, a display control unit configured to display a plurality of images on a display unit configured to display an image, and a user configured to display a plurality of images displayed by the display unit. It is characterized by comprising a determining means for determining the image being watched from the detection result by the detecting means, and a processing control means for controlling the processing relating to the image determined to be watched by the user by the determining means. I do.
[0009]
The processing control unit controls a process of enlarging only the image determined to be watched by the user on the display unit and displaying the image on the display unit, or a process of displaying the image on the display unit in a size larger than other images. You can make it.
[0010]
The processing control means may control a process of acquiring information linked to an image determined to be gazed by a user.
[0011]
The display control means may control such that a plurality of images displayed by the display means are changed in size to be displayed and a displayed position is changed over time.
[0012]
The display control means may control the display positions of the plurality of images displayed by the display means so as to be determined based on position information attached to each image.
[0013]
According to the image processing method of the present invention, a detection step of detecting a user's line of sight and an image being watched by the user among a plurality of images displayed by the display unit are determined from a detection result obtained by the processing of the detection step. It is characterized by including a determining step and a processing control step of controlling processing relating to an image determined to be gazing by the user in the processing of the determining step.
[0014]
The recording medium program according to the present invention includes a detecting step of detecting a user's line of sight, and determining, from a plurality of images displayed by the display unit, an image that the user is gazing at from a detection result obtained by the processing of the detecting step. And a process control step of controlling a process related to an image determined to be gazing by the user in the process of the determining step.
[0015]
The program according to the present invention includes a detecting step of detecting a user's line of sight, and a determining step of determining, from a plurality of images displayed by the display unit, an image that the user is gazing at from a detection result of the processing of the detecting step. And a processing control step of controlling processing relating to the image determined to be gazing by the user in the processing of the determining step.
[0016]
In the present invention, a process related to an image of a part that the user is watching is executed.
[0017]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a diagram showing a configuration of an embodiment of an image processing apparatus to which the present invention is applied. The image processing device 10 includes an image supply unit 21. The image supply unit 21 supplies image data to the image specifying unit 22 and the display control unit 23. The gaze detection information from the gaze detection unit 24 is also input to the image specifying unit 22.
[0018]
When an image based on the image data supplied from the image supply unit 21 includes a plurality of different images, one image can be selected from the plurality of different images, as described later. The image specifying unit 22 is provided as a determination unit for determining and specifying the selected image. Gaze detection information from the gaze detection unit 24 is used as information for specifying the selected image.
[0019]
The image specifying information from the image specifying unit 22 is supplied to the display control unit 23. The display control unit 23 extracts, from the image data supplied from the image supply unit 21, image data of an image determined to be selected by the user based on the image specifying information, and displays the image data based on the extracted image data. It is provided as display control means for controlling display of an image in the unit 25. The storage unit 26 appropriately stores data necessary for the image specifying unit 22 to specify an image, and supplies the stored data to the image specifying unit 22 as necessary.
[0020]
The image supply unit 21 includes, for example, an antenna for receiving a television broadcast, a tuner for extracting data of a predetermined channel from the received television broadcast, and the like. Alternatively, a device having a function of reading image data and audio data stored in a predetermined recording medium, such as a DVD player, may be used. The image supply unit 21 only needs to have a function of supplying an image displayed on the display unit 25.
[0021]
The line-of-sight detection unit 24 is provided as a detection unit for detecting the line of sight of the user, but any detection method may be used. For example, as shown in FIG. 2, a function for detecting the line of sight may be provided to the glasses 41 worn by the user. It is considered that the spectacles 41 are preferably in the form of spectacles. However, as long as the spectacles can be worn by the user, the spectacles 41 may be positioned at a portion where the user's line of sight can be monitored. Shape may be sufficient.
[0022]
When such glasses 41 are used, it is possible to detect the user's line of sight by detecting a portion of the user's eyeball called the iris and monitoring the movement of the detected iris. The method of detecting the iris of the user can be realized by using a camera having a function of distinguishing white (the iris) and black (the iris, a color other than white).
[0023]
When the spectacles 41 and the like are used, only the line of sight of the user wearing the spectacles 41 is a target for the line of sight detecting unit 24 to detect the line of sight. Therefore, for example, even in a situation where a plurality of users are simultaneously viewing a plurality of images displayed on the display unit 25, a problem such as which user's line of sight should be monitored may occur. Can be prevented.
[0024]
However, if the user needs to wear the glasses 41 and the like, there is a problem that the user's line of sight cannot be monitored. That is, if the user does not wear such glasses 41, an image cannot be selected as described later. Of course, it is one advantage that the user can determine whether or not to select an image based on whether or not the spectacles 41 are worn, but this is one advantage, but usually, for a user who does not wear spectacles, It is troublesome to view images while wearing the glasses 41. Also, it is troublesome for a user who usually wears glasses to wear the glasses 41 instead of or on the glasses.
[0025]
In consideration of such a situation, it is preferable to be able to detect the line of sight without performing the operation of wearing by the user. For example, an imaging device may be provided at a position where an image of the user's face can be captured, and the gaze of the user may be detected based on the image captured by the imaging device. As a position where the imaging device is attached, for example, a position as shown in FIG. 3 can be considered.
[0026]
In this case, when the user selects one image from the plurality of images displayed on the display unit 25, the purpose is to detect the user's line of sight. It suffices that the user's line of sight is detected while facing the display unit 25 and viewing the display unit 25 only. In consideration of such circumstances, the imaging device 61 is provided, for example, on the upper right side of the display unit 25. By installing the imaging device 61 at such a position, it is possible for the user to image the face of the user while viewing the image displayed on the display unit 25.
[0027]
Of the user's face imaged by the imaging device 61, for example, the positions of the eyes and the mouth are detected. The eyes are commonly composed of white and black eyes. The iris may be different in color depending on the user, but the iris is usually white. Therefore, for example, a portion determined to be white is extracted from the captured image, and it is determined whether another color such as black exists in the portion determined to be white. This makes it possible to detect the eye part.
[0028]
In addition, since the mouth has a color such as red that is different from other parts of the face (skin color part), it is determined that the mouth is a mouth from an image captured by the imaging device 61 by using its characteristic. It is possible to extract the part to be determined. It is possible to detect the direction in which the user is facing, using a triangle composed of the line connecting the eye and the mouth extracted in this way. In other words, in the case where the user's face is imaged from a fixed point, when the user changes the direction of the face, the shape of the triangle formed by the line connecting the eyes and the mouth changes according to the change, By monitoring the change, the direction in which the user is facing can be detected.
[0029]
Alternatively, if the eye portion is detected from the image captured by the imaging device 61 and the direction of the black eye portion is monitored among the detected eye portions, the direction of the user's line of sight is detected. It is possible.
[0030]
The gaze detection unit 24 has a function of detecting the gaze of the user, although any method may be used. The line-of-sight detection unit 24 constitutes a part of the image processing apparatus 10, but does not necessarily need to be integrated with other parts constituting the image processing apparatus 10, for example, like the glasses 41 shown in FIG. 2. Alternatively, the display unit 25 may be configured separately.
[0031]
The display unit 25 is configured by a television receiver or a combination of a projector and a screen.
[0032]
Next, the operation of the image processing apparatus 10 shown in FIG. 1 will be described with reference to the flowchart in FIG. The process based on the flowchart of FIG. 4 is performed on the assumption that the power of the image processing apparatus 10 is turned on by a user and the image is displayed on the display unit 25. In step S11, detection of the line of sight of the user (viewer) by the line of sight detection unit 24 is started. The gaze detection information on the gaze of the user detected by the gaze detecting unit 24 is output to the image specifying unit 22.
[0033]
In step S12, the display of the multi-screen is performed on the display unit 25 under the control of the display control unit 23. The multi-screen may be displayed as an initial screen when the power of the image processing apparatus 10 is turned on, or may be displayed by other processing. Such setting may be performed by the user, or may be performed using information (metadata) related to the image supplied by the image supply unit 21 as a determination material.
[0034]
Here, a multi-screen displayed on the display unit 25 will be described. FIG. 5 is an example of a multi-screen displayed on the display unit 25, and shows an example in which a multi-screen composed of a plurality of screens is displayed based on image data of a plurality of channels transmitted as a television broadcast. Is shown. The display example of FIG. 5 illustrates an example in which regions for displaying images of seven channels (CH) 1 to 7 are provided on the display screen of the display unit 25, and the corresponding regions are provided. 1 displays one image of a program broadcast on each channel.
[0035]
When displaying such a multi-screen on the display unit 25, the image processing apparatus 10 sequentially receives signals of predetermined seven channels and converts the image data of each channel into a normal screen size (for example, , The size is reduced to 1/7 of the size of the displayable area of the display unit 25 (however, when a predetermined gap is provided between the areas as shown in FIG. When displaying buttons and the like, the size is reduced to 1/7 or less in consideration of the space of the buttons and the like), arranged at a predetermined position, a multi-screen is generated, and displayed on the display unit 25. Let it. Such display control of the image is performed by the display control unit 23 using the image data from the image supply unit 21.
[0036]
In the display example illustrated in FIG. 5, the button 101 of “Previous” is a button that is operated (glanced) when displaying an image of a channel with a smaller number than the displayed channel, The button 102 of “next” is a button operated (glanced) when an image of a channel having a higher number than that of the displayed channel is displayed.
[0037]
Next, another display example on the display unit 25 will be described. FIG. 6 shows a display example when information such as a television broadcast or the Internet is displayed on the display unit 25. In the display example shown in FIG. 6, a world map is displayed on the display unit 25. The corresponding country name is displayed on the world map. In this case, the country name is selected. For example, each country name may be linked to a homepage from which information on the country can be obtained, or may be linked to a channel broadcasting a program in that country.
[0038]
FIG. 7 is a diagram illustrating another display example displayed on the display unit 25. The display example shown in FIG. 7 shows a state in which a soccer match is broadcast as a program of a predetermined channel, and an image of the soccer match is displayed. In the display example shown in FIG. 7, a picture imagining a soccer stadium is displayed as a background, and an image of a match that would be seen if the user was sitting at the stadium and watching a game is displayed on the picture. The image is displayed in an area provided at a position corresponding to the picture (that is, a position where the camera that is capturing the image is installed).
[0039]
In the example shown in FIG. 7, for example, an image when watching a game from the main stand is displayed in the area 131, and an image when watching a game from the stand on the home team side is displayed in the area 133 in the area 133. Shows an image when watching from the stand on the back stand side, and an image when watching from the stand on the away team side is displayed in the area 134, respectively.
[0040]
The display example shown in FIG. 7 shows a state in which imaging cameras are installed at four positions of the soccer stadium. In order to display such an image on the display unit 25, the image specifying unit 22 or the display control unit 23 analyzes the image data supplied from the image supply unit 21, and should display the image in any area of the display unit 25. You need to judge.
[0041]
For this purpose, for example, image data supplied by the image supply unit 21, in other words, in this case, image data of a program broadcast as a television broadcast, and image data received by the image supply unit 21 include: Contains information on where to display it. For example, as shown in FIG. 8, the transmitted / received data includes position information 131 ′ and image data 131 ″ as data for the region 131, and position data 132 ′ and image data 131 ′ as data for the region 132. 132 "is included, the position information 133 'and the image data 133" are included as the data for the area 133, and the position information 134' and the image data 134 "are included as the data for the area 134.
[0042]
The operation of the image processing apparatus 10 when such data is received will be described with reference to the flowchart in FIG. The processing of the flowchart shown in FIG. 9 is performed as the processing of step S12. That is, this is a process for displaying a multi-screen by a multi-angle in a television broadcast, and when other data is received, a process corresponding to the data is performed as a process related to the display of the multi-screen. In step S31, the image supply unit 21 receives the data as illustrated in FIG. 8 and supplies the data to the image specification unit 22 and the display control unit 23, respectively.
[0043]
When data supply to the image specifying unit 22 is started in step S31, the image specifying unit 22 separates position information and image data from the supplied data in step S32. When the data as shown in FIG. 8 is supplied, the position information 131 ′ to 134 ′ and the image data 131 ″ to 134 ″ are separated, respectively.
[0044]
When the position information and the image data are separated in the image specifying unit 22 by the processing in step S32, the image specifying unit 22 uses the separated position information, in this case, the position information 131 ′ to 134 ′. The process of determining the position (area) to be displayed in step S33 is executed.
[0045]
As the separated position information, for example, a camera that captures a game is provided with a GPS (Global Positioning System) device, and position information obtained from the GPS can be considered. If the position information obtained from the GPS can be transmitted to the image processing device 10 on the user side together with the image data output from the camera, the position information can be transmitted to the image processing device 10 as described above. It can be separated from image data and used as information for position determination.
[0046]
For example, when information obtained from the GPS is used as the position information, the image specifying unit 22 compares the latitude and longitude indicated by the position information of each of the separated position information 131 ′ to 134 ′. For example, of the latitudes indicated by the position information 131 ′ to 134 ′, it is determined which position information has the highest north latitude. Then, as a result of the determination, an image based on the image data (in this case, the image data 131 ″) corresponding to the position information (for example, the position information 131 ′) determined to have the highest north latitude is placed in the region 131 (FIG. What is necessary is just to perform a process so that it may be displayed on 7).
[0047]
Data indicating the result of such determination is supplied from the image specifying unit 22 to the display control unit 23. Note that information obtained by GPS may be used as position information, or other information may be used. As other information, simple information indicating which area of the display unit 25 is to be displayed, for example, information itself such as displaying in an upper area or being displayed in a lower area may be used.
[0048]
As an example of other information of the position information, an attribute of an image may be used. For example, four attributes of emotional, active, warm color and cool color are provided as image attributes. The display position may be specified by this attribute. The four attributes are provided to define the vertical direction and the horizontal direction. That is, for example, the attribute of emotional and active is used to determine the display position in the horizontal direction, and the attribute of warm and cool colors is used to determine the display position in the vertical direction.
[0049]
As for these attributes, one attribute is not necessarily assigned to one image data, and a ratio of each attribute may be assigned.
[0050]
Even in the case of relaying a soccer game as shown in FIG. 7, data output from a camera capturing an image in which players are actively moving includes information of the attribute of active, and vice versa. Then, data of an attribute of emotion is included in data output from a camera which captures an image in which players are not actively moving. In this way, it is possible to always display an image, for example, on the upper side by the data of the attribute of active, and further, the image displayed on the upper side is displayed in a large size. With such settings, the user can always browse the images in which the players are actively moving, watching the game, and deemed to be the most interesting.
[0051]
In addition, when the position display is controlled by such an attribute, one child image is displayed at various positions according to the attribute, and the child image can be dynamically displayed. .
[0052]
Based on any of the information, in step S33, the image specifying unit 22 determines an area for displaying an image based on the supplied image data, and supplies data indicating the determined content to the display control unit 23. In step S34, the display control unit 23 controls the display unit 25 so that an image based on the image data corresponding to the determination is arranged and displayed in the area determined by the image specifying unit 22.
[0053]
That is, the display control unit 23 is supplied with the data including the position information and the image data as shown in FIG. 8 from the image supply unit 21. The display control unit 23 processes the supplied image data. Then, the display unit 25 is controlled so that an image based on the processed image data is displayed in a predetermined area of the display unit 25 indicated by the data from the image specifying unit 22.
[0054]
By executing such processing, a screen as shown in FIG. 7 is displayed on the display unit 25.
[0055]
Here, the examples shown in FIGS. 5 to 7 have been described as display examples of the multi-screen, but these displays are merely examples and do not limit the present invention.
[0056]
In the above-described embodiment, the size and position of the image area (hereinafter, appropriately referred to as a sub-screen) constituting the multi-screen have been described as being fixed. Instead, when focusing on one child screen, the child screen may be displayed as if it moves around in the display area of the display unit 25, or may be displayed such that the displayed size changes.
[0057]
For example, when a relay program of a soccer match is displayed on the display unit 25 as in the display example shown in FIG. 7, a child screen which is considered to be the best according to the movement of the ball is displayed. You may make it display larger than a child screen. Further, in order to give a sense of realism, a display such as moving a child screen may be performed. In such a case, information such as which child screen is to be displayed larger or moved may be included in the data to be transmitted on the broadcast side of the program. Further, a display such as moving the child screen itself in accordance with the movement of the ball may be performed.
[0058]
Returning to the description with reference to the flowchart shown in FIG. 4, in step S12, when the multi-screen is displayed on the display unit 25, in step S13, it is determined whether the same image (region) has been watched for a predetermined time or more. Is determined. This determination is performed by the image specifying unit 22 determining whether the user's line of sight detected by the line of sight detection unit 24 is poured into one area in the multi-screen without moving for five seconds, for example. Is performed by
[0059]
The process of step S13 is repeated until it is determined that the user has been watching the same area for a predetermined time or more. When the user gazes at the same area for a predetermined time or more, it can be determined that the user is interested in the image (program) displayed in the area.
[0060]
If it is determined in step S13 that the same area has been watched for a predetermined time or more, the process proceeds to step S14, and an image (screen) corresponding to the area (child screen) determined to be watched is enlarged and displayed. In the multi-screen, each displayed screen is displayed in a size smaller than the maximum size that the display unit 25 can display. Such a small screen is displayed on the display unit 25 as a screen having a size larger than that of the child screen.
[0061]
For example, when it is determined that the user gazes at the image displayed in the area 131 in the state of the multi-screen illustrated in FIG. 7, the size is increased as compared with the case of the multi-screen in the area 131. The screen is as shown in FIG. In the display example shown in FIG. 10, the region 131 determined to be watched by the user is displayed in a larger size than the other regions 132 to 134, while the regions 132 to 134 are displayed in a smaller size. has been edited.
[0062]
In the display example shown in FIG. 10, the regions 132 to 134 are also displayed, but only the region 131 may be displayed. In such a case, the display area 25 is displayed in the same size or almost the same size as the displayable area. Such a display in which only the selected area is displayed is, for example, from a multi-screen as shown in FIGS. 5 and 6 to a display when one screen (program) is selected. Are suitable.
[0063]
The multi-screen shown in FIG. 5 displays programs broadcast on different channels in different areas at the same time. When one program is selected from such a multi-screen, the selection is made. It is preferable to display only the programs that were played. Conversely, if the program of the channel other than the selected program is kept displayed even after the selection, the user cannot concentrate on the selected program and the screen becomes difficult to see.
[0064]
The multi-screen shown in FIG. 6 is a screen in which a world map and corresponding country names are displayed on the map. When one country name is selected from such a screen, the multi-screen is displayed. The display is switched to a homepage or a program from which information on the country name (country) can be obtained. In this case, it is sufficient that only the homepage or the program related to the selected country is displayed, and it is not necessary to maintain the display of the country name of the non-selected country from the multi-screen state. Conversely, if the display is maintained, it will be difficult to see.
[0065]
In contrast to such a multi-screen, in the case of a multi-screen such as a soccer relay shown in FIG. 7, as shown in FIG. 10, an area other than the selected area (the area 131 in this case) (in this case, , Regions 132 to 134) should be displayed together. By maintaining the display of the regions 132 to 134, the user can browse images of other regions together, and can switch the display to an image of another region by a process described later. As shown in FIG. 10, a plurality of regions may be displayed.
[0066]
In such a case, for one piece of information (in this case, a soccer game), information from a plurality of viewpoints (in this case, images captured from a plurality of angles) is provided to the user at the same time. A lot of information. Also, even if such a large amount of information is provided to the user at the same time, it is difficult to grasp what kind of information each information is because it is information from a different viewpoint with respect to the same information. Such a situation is unlikely to occur. Therefore, in such a case, it is considered that providing a plurality of pieces of information at the same time is more preferable for the user than providing only one piece of information.
[0067]
As described above, when one screen is selected from the multi-screen, whether to display only the selected screen or to display the selected screen and the non-selected screen together is displayed. What is necessary is just to be able to switch according to the information (contents) to be performed. The switching may be performed by the user or a mechanism may be provided on the image processing apparatus 10 side, for example, by referring to data included in the received data.
[0068]
Further, in the case of a program that is considered to have a request such as repeating a goal scene or looking from another angle, as in the relay of a soccer game, as shown in FIG. Behind, a part of the scene at a time earlier in time is displayed. For example, the area 131-1 is a real-time image at that time, and the area 131-2 behind the area 131-1 is for a predetermined time (several tens of seconds or minutes). The image at the previous time, and an area 131-3 behind the area 131-2 is an image at a time earlier by a predetermined time.
[0069]
Similarly, for the other areas, the image at the previous time is displayed behind the real-time image. As described above, even in a display in which images at a plurality of times can be selected for one angle, the user gazes at an image in a desired area together with an image displayed behind. It is only possible to choose.
[0070]
By displaying the area displayed behind as a selectable image in this way, for example, it is possible to see a goal scene that has been missed, and to enjoy the same goal scene from a plurality of angles. Will be able to Note that the number of regions displayed behind the same angle (region) may be any number.
[0071]
Regardless of which display is performed, in this way, one program (region) is selected, and the user wants to display the multi-screen again while the image in the region is normally reproduced. When thinking, the multi-screen button 150 provided as a button for displaying the multi-screen shown in FIG. 10 is operated (gaze).
[0072]
In step S15 (FIG. 4), it is determined whether or not the multi-screen button 150 shown in FIG. 10 (FIG. 11) has been watched. If it is determined in step S15 that the multi-screen button 150 has not been watched, the display of the enlarged image is continued. On the other hand, if it is determined in step S15 that the multi-screen button 150 has been watched, the process returns to step S12, and the subsequent processing is repeated.
[0073]
The multi-screen button 150 as shown in FIG. 10 (FIG. 11) may be displayed to determine whether or not the button has been watched. However, without displaying such a button, the multi-screen button 150 may be displayed. Switching to the screen may be performed. For example, when switching to the multi-screen is instructed, it is considered that the user is tired of the displayed program or the like and desires to browse another program or the like. In such a case, it is considered that the user removes his / her line of sight from the enlarged screen.
[0074]
Therefore, as the process of step S15, it is determined whether or not the user's line of sight has been removed from the enlarged display screen (area) for a predetermined period of time to determine whether there has been an instruction to switch to the multi-screen. May be determined.
[0075]
By returning the process to step S12, for example, when a program that is broadcasting a soccer match as shown in FIG. 11 is being viewed, the display unit 25 is displayed on the multi-screen as shown in FIG. Is switched.
[0076]
In this way, by allowing the image watched by the user from the multi-screen to be normally reproduced (displayed differently from images displayed in other areas), the user can perform a desired operation without performing a button operation. Programs can be selected quickly and easily.
[0077]
In the above-described embodiment, it is assumed that a program is selected from the multi-screen as shown in FIG. 5, and a country name (information of the country) is selected from the multi-screen as shown in FIG. It has been described that an image captured from a predetermined angle is selected from the multi-screen as shown in FIG. As described above, in the present embodiment, what kind of multi-screen is displayed and what is selected can be changed as appropriate, and is not limited to the above-described embodiment.
[0078]
For example, although not shown, the song names of the songs recorded on the compact disc are displayed in a list as a multi-screen, and when the user gazes at the song desired to be reproduced from the song names displayed in the list, the song is displayed. Reproduction is also possible by applying the present invention.
[0079]
In the above-described embodiment, for example, in the case of the multi-screen shown in FIG. 5, that is, a multi-screen in which child screens corresponding to a plurality of programs are displayed, if the multi-channeling progresses, all the users There is a problem that it takes a long time to browse the child screen of the program. Therefore, a function of selecting a program to be displayed on the multi-screen on the image processing apparatus 10 side in advance according to the user's preference may be provided.
[0080]
Information on the user's preference is stored in the storage unit 26 (FIG. 1) every time the user selects a program from the multi-screen shown in FIG. As information to be stored, each time a program is selected, the genre to which the program belongs is determined, and the number of times the program of the genre is selected is accumulated, thereby determining the genre that the user prefers. I do. The information of the genre to which the program belongs can be included in the data of the received program, and the genre information can be obtained by extracting the data included in such a manner. .
[0081]
A program of a genre with a small accumulated number of times is not displayed as a multi-screen child screen, or even if it is displayed, it is displayed later in time. In other words, the programs of the genre with the largest number of times of accumulation are displayed as multi-screen child screens in order.
[0082]
Such information on the user's preference can be used even in a multi-screen or in a state where the selected image is displayed in an enlarged manner. For example, in a state where a multi-screen or an enlarged image is displayed, it is conceivable that two different areas 171 and 172 may overlap as shown in FIG. In the example shown in FIG. 12, since the region 172 is located behind the region 172, the user cannot see the image displayed in the region 172.
[0083]
Therefore, the user cannot recognize that the area 172 exists behind the area 171. This may occur, for example, when a new angle image is broadcasted in the program displayed in the area 171 and the area 172 is newly provided. is there.
[0084]
In such a case, using the information on the user's preference stored in the storage unit 26, the area displaying the image (information, program, etc.) that the user likes is determined, and based on the determination, the area 171 or the area 171 is determined. One of the areas 172 is displayed on the front. By providing such a function, it is possible to provide the user with information displayed at a position that is difficult to recognize.
[0085]
Of course, even when one area is hidden by the other area or one area overlaps with the other area, the image processing apparatus 10 can generate a user's favorite image (information or program). It is also possible to provide a mechanism for making a judgment and providing the information by applying the mechanism as described above.
[0086]
The above-described series of processing, for example, processing such as detection of a line of sight and switching of images based on the detection can be executed by hardware having respective functions, but can also be executed by software. When a series of processing is executed by software, it is possible to execute various functions by installing a computer in which the programs constituting the software are embedded in dedicated hardware, or by installing various programs For example, it is installed from a recording medium to a general-purpose personal computer or the like.
[0087]
Before describing the recording medium, a personal computer that reads or writes data from the recording medium will be briefly described. FIG. 13 is a diagram illustrating an internal configuration example of a general-purpose personal computer. A CPU (Central Processing Unit) 301 of a personal computer executes various processes according to a program stored in a ROM (Read Only Memory) 302. A RAM (Random Access Memory) 303 appropriately stores data and programs necessary for the CPU 301 to execute various processes. The input / output interface 305 is connected to an input unit 306 including a keyboard and a mouse, and outputs a signal input to the input unit 306 to the CPU 301. The input / output interface 305 is also connected to the output unit 7 including a display, a speaker, and the like.
[0088]
Further, the input / output interface 305 is also connected to a storage unit 308 composed of a hard disk or the like, and a communication unit 309 for exchanging data with another device via a network such as the Internet. The drive 310 is used to read data from or write data to a recording medium such as a magnetic disk 321, an optical disk 322, a magneto-optical disk 323, and a semiconductor memory 324.
[0089]
As shown in FIG. 13, the recording medium is a magnetic disk 321 (including a flexible disk) on which the program is recorded, which is distributed in order to provide the program to the user separately from a personal computer, and an optical disk 322 (CD-ROM). It is configured by a package medium including a ROM (Compact Disc-Read Only Memory), a DVD (including a Digital Versatile Disc), a magneto-optical disk 323 (including an MD (Mini-Disc) (registered trademark)), a semiconductor memory 324, or the like. Not only that, but also a hard disk including a ROM 302 and a storage unit 308 in which a program is stored, which is provided to a user in a state of being incorporated in a computer in advance.
[0090]
In this specification, the steps of describing a program provided by a medium include, in the order described, not only processing performed in chronological order but also processing in chronological order, or in parallel or individually. This includes the processing to be executed.
[0091]
Also, in this specification, a system refers to an entire device including a plurality of devices.
[0092]
【The invention's effect】
According to the present invention, a multi-screen can be displayed.
[0093]
ADVANTAGE OF THE INVENTION According to this invention, it becomes possible to easily select a desired screen from a multi-screen by gaze.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating a configuration of an embodiment of an image processing apparatus to which the present invention has been applied.
FIG. 2 is a diagram for describing a line-of-sight detection unit.
FIG. 3 is a diagram for describing a line-of-sight detection unit.
FIG. 4 is a flowchart illustrating an operation of the image processing apparatus.
FIG. 5 is a diagram for explaining a multi-screen.
FIG. 6 is a diagram for explaining a multi-screen.
FIG. 7 is a diagram for explaining a multi-screen.
FIG. 8 is a diagram for describing data received by the image processing apparatus.
FIG. 9 is a flowchart illustrating a process related to multi-screen display.
FIG. 10 is a diagram illustrating a display example including an enlarged screen.
FIG. 11 is a diagram illustrating a display example including an enlarged screen.
FIG. 12 is a diagram for describing a case where areas overlap.
FIG. 13 is a diagram illustrating a medium.
[Explanation of symbols]
Reference Signs List 10 image processing device, 21 image supply unit, 22 image specification unit, 23 display control unit, 24 gaze detection unit, 25 display unit, 26 storage unit, 41 glasses, 61 imaging device

Claims (8)

Detecting means for detecting the user's line of sight;
Display control means for displaying a plurality of images on a display means for displaying images,
Among the plurality of images displayed by the display unit, a determination unit that determines an image that the user is gazing from a detection result by the detection unit,
An image processing apparatus, comprising: a processing control unit configured to control a process related to the image determined to be gazing by the user by the determination unit. The processing control unit is configured to enlarge the image determined to be closely watched by the user and display the image on the display unit, or display the image on the display unit in a size larger than other images. The image processing apparatus according to claim 1, wherein the image processing apparatus controls processing. The image processing apparatus according to claim 1, wherein the processing control unit controls a process of acquiring information linked to the image determined to be gazing at the user. The display control means controls the plurality of images displayed on the display means so that the displayed size is changed and the displayed position is changed over time. 2. The image processing device according to 1. The display control means controls the display positions of the plurality of images displayed on the display means so as to be determined based on position information attached to each image. Item 2. The image processing device according to item 1. In an image processing method of an image processing apparatus including a display control unit that displays a plurality of images on a display unit that displays images,
A detecting step of detecting a line of sight of the user;
A judging step of judging, from among the plurality of images displayed by the display means, an image that the user is gazing at from a detection result obtained by the processing of the detecting step;
A process control step of controlling a process relating to the image determined to be watched by the user in the process of the determining step. A program for an image processing apparatus including a display control unit configured to display a plurality of images on a display unit configured to display an image,
A detecting step of detecting a line of sight of the user;
A judging step of judging, from among the plurality of images displayed by the display means, an image that the user is gazing at from a detection result obtained by the processing of the detecting step;
A computer-readable program recorded thereon, comprising: a processing control step of controlling processing relating to the image determined to be gazing by the user in the processing of the determining step. In a program executed by a computer that controls an image processing apparatus including a display control unit that displays a plurality of images on a display unit that displays images,
A detecting step of detecting a line of sight of the user;
A judging step of judging, from among the plurality of images displayed by the display means, an image that the user is gazing at from a detection result obtained by the processing of the detecting step;
A program controlling step of controlling a process relating to the image determined to be gazing by the user in the process of the determining step.

Images