JP4815949B2 - Multi-display device and display device - Google Patents

Description

  The present invention relates to a multi-display device suitable for use when providing important information to a user, and a display device used for the multi-display device.

  Currently, television broadcasting is becoming increasingly multi-channel, and the user needs to select one channel from among more channels, so among the vast amount of information provided by television broadcasting. It is becoming difficult to obtain important information.

  The amount of information that a user can obtain by simultaneously displaying a plurality of videos on a multi-display is greatly increased compared to the case where a single video is viewed using a single display. The displayed images cannot be viewed at the same time.

  For this reason, some multi-displays have a technique for determining which display screen the user is gazing at among the plurality of displays and displaying only the video corresponding to the screen (see, for example, Patent Document 1). ).

Japanese Patent Laid-Open No. 2000-138872

  However, even if such a technique is used, the user may miss important information provided by the video displayed on the display screen other than the display screen on which the user is gazing. I cannot say that.

  The present invention has been proposed in view of such a conventional situation, and provides a multi-display device that provides optimal information to a user using a plurality of displays and a display device used for the multi-display device. With the goal.

In order to achieve the above-described object, the multi-display apparatus according to the present invention displays a plurality of videos on a plurality of display screens, and supports a sound corresponding to each of the plurality of videos on each of the plurality of display screens. Display means for outputting from a speaker, gaze detection means for detecting a display screen being watched by a user among the plurality of display screens, emergency video detection means for detecting a video of information with a high degree of urgency, and a plurality of Video selection means for selecting one video signal from the video signals, control means for controlling the display means and the video selection section based on data of the display screen detected by the line-of-sight detection means, and the video an important image extracting means for extracting an image containing significant information in the video signal from the signal, the video selecting means, on the display unit, among the plurality of display screens, A video based on one video signal selected by the video selection unit is displayed on the display screen being watched by the user detected by the visual line detection unit, and corresponds to one video signal selected by the video selection unit The important information extracted by the important image extraction means is included in a display screen other than the display screen detected by the line-of-sight detection means among the plurality of display screens. When an image is displayed and a sound output of a speaker corresponding to a display screen other than the display screen detected by the line-of-sight detection unit is turned off, and a video with high urgency information is detected by the emergency video detection unit Displays an image of information with a high degree of urgency on the display screen detected by the line-of-sight detection means .

  In addition, the image processing device further includes emergency video detection means for detecting a video of information with a high degree of urgency, and the video selection means is detected by the urgency level detection means on the display screen detected by the line-of-sight detection means. Display the video.

  Further, the emergency video detection means sets a video including an event photographed by the surveillance camera as a video of information having a high degree of emergency.

In order to achieve the above-described object, a display device according to the present invention is a display device used in a multi-display device that displays a plurality of videos, and displays a plurality of videos on a plurality of display screens. Display means for outputting sound corresponding to each of a plurality of videos from a speaker corresponding to each of the plurality of display screens, and line-of-sight detection means for detecting a display screen being watched by a user among the plurality of display screens Video selection means for selecting one video signal from a plurality of video signals, and control means for controlling the display means and the video selection section based on data of the display screen detected by the line-of-sight detection means; , and a significant image extracting means for extracting an image containing significant information in the video signal from said video signal, said video selecting means, on the display means, said plurality of tables Of the screens, a video based on one video signal selected by the video selection unit is displayed on a display screen that is being watched by the user detected by the line-of-sight detection unit, and one selected by the video selection unit is displayed. the sound corresponding to the video signal is outputted to the speaker of the display means, of the display screen of the plurality of the key extracted by the key image extracting means on the display screen other than the detected display screen in visual axis detecting means From an emergency video detection means for displaying an image including various information, turning off the sound output of a speaker corresponding to a display screen other than the display screen detected by the line-of-sight detection means, and detecting a video with high urgency information When a video with high urgency information is input, the video with high urgency information is displayed on the display screen detected by the line-of-sight detection means .

  In addition, the video selection unit receives a video from an emergency video detection unit that detects a video with high urgency information, and displays the urgency level detection unit on the display screen detected by the line-of-sight detection unit. Display the video detected in.

  Further, the emergency video detection means sets a video including an event photographed by the surveillance camera as a video of information having a high degree of emergency.

  According to the multi-display device and the display device according to the present invention, it is possible to provide important information according to the user's line-of-sight information, and the efficiency with which the user acquires information is increased.

  Hereinafter, specific embodiments to which the present invention is applied will be described in detail with reference to the drawings.

  As shown in FIG. 1, the multi-display device according to the embodiment to which the present invention is applied includes a plurality of display devices 110 (display devices 110A, 110B,..., 110I). An image is displayed on the display screen 111 (display screens 111A, 111B,..., 111I). For example, the information of the video of the television program displayed on the display screen 111A that the user 14 is not watching is more than the information of the video of the television program displayed on the display screen 111E that the user 14 is watching. When the importance is high, the amount of information obtained by the user 14 can be improved by interrupting or switching the video displayed on the display screen 111A on the display screen 111E. Further, as shown in FIG. 2, when the user 14 is gazing at the display screen 111E of the display device 110E among the display devices 110 (display devices 110A, 110B,..., 110I), a sound is displayed. By automatically switching to the audio of the video displayed on the screen 111E, the user 14 can obtain information more reliably.

  Further, as shown in FIG. 3, the multi-display device according to the embodiment to which the present invention is applied includes, for example, the display device 110E among the display devices 110 (display devices 110A, 110B,..., 110I). When the user is gazing at the display screen 111E, the image of important information such as an image including a telop is detected on the display screens 111A, 111B, 111C, 111D, 111F, 111G, 111H, and 111I that the user 14 is not gazing at. The display screen is updated for each image, and the amount of information that the user 14 can obtain is maximized.

  The configuration of the multi-display device in the embodiment to which the present invention is applied will be described below.

  First, the configuration of the multi-display device in the first embodiment will be described.

  FIG. 4 is a schematic diagram of the multi-display device in the first embodiment.

  The display devices 110 (display devices 110A, 110B,..., 110I) display images on the respective display screens 111 (display screens 111A, 111B,..., 111I). The line-of-sight detection unit 12 includes a user photographing camera 121. As will be described later, the user 14 captures the face of the user 14 with the user photographing camera 121 and then performs image processing on the image of the eye so that the user 14 can select any display screen. The number of the display screen that the user 14 is viewing is notified. The remote controller 13 includes a power key of the display device 10, a channel selection key, a volume control key, and the like, and the user 14 controls the display device 110 using the remote controller 13.

  The internal configuration of the multi-display device 10 according to the first embodiment will be described.

  As shown in the block diagram of FIG. 5, the multi-display device 10 includes a switching unit 2, an importance detection unit 3, a display unit 4 (display units 4A, 4B, 4C,..., 4H, 4I), A control unit 5 and a line-of-sight detection unit 12 having a user monitoring camera 121 are provided.

  The switching unit 2 inputs a plurality of video signals V (video signals Va, Vb,..., Vi) and supplies the video signal V to the display unit 4 according to the detection result of the importance level detection unit 3. The importance detection unit 3 detects the importance of information in the video of the plurality of video signals V, and outputs importance data indicating the importance to the control unit 5 described later. The display unit 4 (display units 4A, 4B,..., 4I) includes a display screen 111 (display screens 111A, 111B,..., 111I) and speakers 112 (speakers 112A, 112B,..., 112I). And outputting video and audio based on the supplied video signal V. The control unit 5 controls the switching unit 2 based on the importance level data supplied from the importance level detection unit 3 and the visual line detection result data supplied from the visual line detection unit 12.

  Here, various display methods are assumed as video display methods with high importance of information. For example, each of a plurality of display screens is a video of a soccer game, a video of a news program, a video of a professional baseball game, etc. As an example of a method for displaying a video with high importance when the user 14 is viewing a display screen on which a plurality of different videos such as videos are displayed and the user 14 is viewing a video of a soccer game, When an emergency news video is displayed on a display screen that is not being watched, a method for displaying the emergency news video on the display screen on which the user 14 is watching, or professional baseball during a half-time of a soccer game There is a method of displaying an image of a scene of scoring chance of the game on a display screen on which the user 14 is gazing. Thus, according to the importance of the information detected by the importance detection unit 3, the switching unit 2 outputs the video signal V of the video including the information with the highest importance to the display unit 4. The display unit 4 displays an image including this most important information on the display screen that the user 14 is gazing at.

  The line-of-sight detection unit 12 includes a user photographing camera 121 that photographs the face of the user 14 and detects the line of sight of the user 14 according to the processing procedure shown in the flowchart of FIG.

  When the image data of the user 14 captured by the user imaging camera 121 is supplied in step S11, the line-of-sight detection unit 12 in the subsequent step S12, the color information (user 14) of the image of the user 14 in the supplied image data. The contours of both eyes are detected using the skin color, white eye color, black eye color, etc., and the positions of both ends of both eyes are detected.

  In step S <b> 13, the line-of-sight detection unit 12 limits a predetermined search range based on the positions of both eyes detected in step S <b> 12, and detects the nostril position using the luminance information of the image.

  In step S <b> 14, the line-of-sight detection unit 12 estimates the position and radius of the eyeball center 42 in the eyeball 41 based on the geometric positional relationship between the eye ends and the nostril positions, as shown in FIG. 7.

  In step S <b> 15, the line-of-sight detection unit 12 detects the position of the pupil center 44 in the pupil 43 using the luminance information in the eye outline.

  In step S <b> 16, the line-of-sight detection unit 12 detects the line of sight of the user 14 by calculating a vector connecting the eyeball center 42 detected in step S <b> 14 and the pupil center 44 detected in step S <b> 45 as the line-of-sight vector 45. The number of the display screen viewed by 14 is supplied to the control unit 5 as the line-of-sight detection result data. Thereafter, the line-of-sight detection unit 12 ends a series of processing operations.

  As described above, the multi-display device 10 according to the first embodiment detects the importance of information in a video when displaying video based on a plurality of video signals V on a plurality of display screens. As described above, a video that has a high urgency to be transmitted to the user 14 and has the highest importance of information, such as a video of the suspicious person taken by the surveillance camera, is displayed on the display screen detected by the line-of-sight detection unit 12. As a result, the amount of information obtained when the user 14 views a plurality of videos using the multi-display is significantly improved.

  The multi-display apparatus according to the first embodiment described above has a configuration in which the switching unit 2, the importance level detection unit 3, and the control unit 5 are provided outside the display unit 4, but the switching unit 2, the level of importance detection is performed. You may provide the structure which provided one part or all part of the part 3 and the control part 5 in each display part.

  Next, the configuration of the multi-display device in the second embodiment will be described. In addition, about the structure similar to the multi-display apparatus 10, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  FIG. 8 is a schematic diagram of the multi-display device in the second embodiment.

  The multi display device 20 includes a plurality of display devices 210 (display devices 210A, 210B,..., 210I), a line-of-sight detection unit 12, an emergency video detection unit 23, and a remote controller 13.

  The display devices 210 (display devices 210A, 210B,..., 210I) display images on the display screens 111 (display screens 111A, 111B,. The emergency video detection unit 23 has a monitoring camera 231, and when an event such as a suspicious person is detected in a video captured by the monitoring camera 231, the display device 210 (display devices 210A, 210B,..., 210I). Notify that.

  The internal configuration of the display device 210 (display devices 210A, 210B,..., 210I) used in the multi-display device 20 in the second embodiment will be described.

  As shown in FIG. 9, the display device 210 (display devices 210A, 210B,..., 210I) includes a control unit 52, an important image extraction unit 53, a video selection unit 54, and a display screen 111 (display screen). 111A) and the display unit 4 having speakers 212 (speakers 112A, 112B,..., 112I).

  When the interrupt event notification data and the line-of-sight detection result data are supplied from the emergency video detection unit 23 and the line-of-sight detection unit 12, respectively, the control unit 52 supplies the audio control data to the speaker 112 and outputs the sound output from the speaker 112. Is controlled by ON / OFF switching, and video selection data is supplied to the video selection unit 54.

  The important image extraction unit 53 extracts a video signal of an image including important information such as a telop image from the video signal V1 of the television program, and supplies it to the video selection unit 54.

  In the important image extraction unit 53, as shown in the block diagram of FIG. 10, the important image detection unit 531 detects a telop image from the video signal V 1 of the television program, and supplies this telop image to the memory 532. Further, the update flag signal is supplied to the memory 532 to update the memory 532, and the memory 532 continues to read the latest image.

  The video selection unit 54 selects and displays a specific video signal from the video signal V2 of the event detected by the emergency video detection unit 23, the video signal of the television program, and the video signal of the telop image of the television program. 4 is supplied.

  Here, processing operations performed by the display device 210 (display devices 210A, 210B,..., 210I) will be described with reference to the flowchart of FIG.

  When the display device 210 (display devices 210A, 210B,..., 210I) starts the processing operation, the user 14 displays the display screen 111 (111A, 111B,..., 111I) from the line-of-sight detection unit 12 in step S21. Line-of-sight detection result data indicating which display screen is being watched is supplied.

  In step S <b> 22, each display device 210 determines whether its own display screen is being watched by the user 14. In step S22, the display device that has determined that its display screen is being watched by the user 14 from the display device 210 moves from step S22 to step S23, and counts up the watch time. On the other hand, the display device that has determined that its own display screen is not being watched by the user 14 moves from step S22 to step S29, and counts up the non-gaze time.

  Of the display devices 210, the display device that has been determined that its own display screen is being watched by the user 14 in step S22 is referred to as a display device 210m. When the display device 210m proceeds from step S23 to step S24, the display device 210m determines whether or not event notification data is supplied from the emergency video detection unit 23.

  When the display device 210m determines in step S24 that event notification data has been supplied from the emergency video detection unit 23, the display device 210m proceeds from step S24 to step S25, and outputs and selects the video captured by the emergency video detection unit 23. To do. Thereafter, the display devices 210A, 210B,..., 210I end a series of processing operations.

  On the other hand, if the display device 210m determines in step S24 that the event notification data is not supplied from the emergency video detector 23, the display device 210m moves from step S24 to step S26, and the video of the television program that is a normal input video. Is selected and displayed on its own display screen.

  The display device 210m moves from step S26 to step S27, and determines whether or not the time during which the display screen of the display device 210 is being watched by the user 14 is longer than a certain time. In step S27, when the display device 210m determines that the time being watched by the user 14 is equal to or longer than a predetermined time, the display device 210m moves from step S27 to step S28, and outputs sound to a speaker built in the display device 210m. To ON. Thereafter, the display device 210 ends a series of processing operations. On the other hand, when the display device 210m determines in this step S27 that the time being watched by the user 14 is not longer than a predetermined time, the display device 210 ends a series of processing operations.

  Among the display devices 210, a display device that is determined not to be watched by the user 14 in step S22 described above is referred to as a display device 210n. When the display device 210n counts up the non-gaze time in step S29 and then moves from step S29 to step S30, the display device 210n determines whether or not the time not being watched by the user 14 is equal to or longer than a certain time. In step S30, when the display device 210n determines that the time not being watched by the user 14 is equal to or longer than a predetermined time, the display device 210n moves from step S30 to step S31 and outputs audio to the speaker built in the display device 210n. Is switched to OFF, and the process proceeds from step S31 to step S32. On the other hand, when the display device 210n determines in this step S30 that the time not being watched by the user 14 is not a certain time or more, the process proceeds from step S30 to step S32.

  In step S32, the display device 210n selects and outputs the telop image extracted by the important image extraction unit 53. Thereafter, the display device 210 ends a series of processing operations.

  As shown in the block diagram of FIG. 12, the emergency video detection unit 23 detects an event from the difference between the input video frame from the monitoring camera 121 and the past video frame stored in the memory 233 by the difference circuit 232. Then, the interrupt event notification data is supplied to the control unit 21 to notify the detection of the event.

  The multi-display device 20 has a configuration in which the display unit 4, the control unit 52, the important image extraction unit 53, and the video selection unit 54 are provided in the display device 210. However, the display unit 4, the control unit 52, The important image extraction unit 53 and the video selection unit 54 may be provided outside the display device 210.

  Here, as an example of event detection, the processing operation when the emergency video detection unit 23 detects a suspicious person will be described with reference to the flowchart of FIG.

  When the emergency video detection unit 23 starts the processing operation, the emergency video detection unit 23 proceeds to step S41. In step S41, the difference circuit 232 compares the frame of the input video from the monitoring camera 231 with the frame of the past video stored in the memory 233, and calculates the difference.

  The difference circuit 232 moves to step S42 and determines whether or not the difference is equal to or greater than a predetermined threshold value. If it is determined in step S42 that the difference is equal to or greater than the predetermined threshold, the difference circuit 232 proceeds to step S43. On the other hand, if it is determined in step S42 that the difference is not equal to or greater than the predetermined threshold value, the emergency video detection unit 23 ends the series of processing operations.

  In step S43, the difference circuit 232 determines whether or not the shape of the object in the input video is a humanoid. If it is determined in step S43 that the shape of the object in the input video is humanoid, the difference circuit 232 proceeds to step S44. On the other hand, if it is determined in step S43 that the shape of the object in the input video is not humanoid, the emergency video detection unit 23 ends the series of processing operations.

  In step S44, the difference circuit 232 determines whether or not the time during which an object having a humanoid shape appears in the input video is equal to or longer than a predetermined time. In step S44, when it is determined that the time when the object having a human shape appears in the input video is equal to or longer than a predetermined time, the difference circuit 232 proceeds to step S45. On the other hand, in this step S44, when it is determined that the time when the object having the shape of a human figure appears in the input video is not longer than the predetermined time, the emergency video detection unit 23 ends the series of processing operations.

  In step S <b> 45, the difference circuit 232 regards an object having a human shape as a suspicious person, and supplies interrupt event notification data for notifying that the suspicious person has been detected as an event to the control unit 52. Thereafter, the emergency video detection unit 23 ends a series of processing operations.

  When the emergency video detection unit 23 detects an event when the user 14 views the video of the television program, the multi-display device 20 described above displays the video of the event on the display screen on which the user 14 is gazing. To do. In addition, the telop image in the video of the television program is displayed on the display screen that is not watched by the user 14. As a result, when the user 14 views a video using a multi-display, it is possible to acquire a lot of important information in real time from both the watched screen and the screen not watched.

  It should be noted that the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present invention.

  In the embodiment described above, the number of displays constituting the display device 110 and the display device 210 is set to nine, but the number of displays is not limited to this.

  In the above-described embodiment, the line of sight of the user 14 is detected by performing image processing on the image of the eye of the user 14, but as a method of detecting the line of sight of the user 14, for example, infrared rays are applied to the face of the user 14 Any method may be used, such as a method using an infrared image reflected.

  In the above-described embodiment, the telop image is displayed on the display screen that the user 14 is not gazing at. However, the display screen is not limited to the case where the telop image is displayed. It is possible to provide an opportunity for the user 14 to select an image by displaying the image.

  In the above-described embodiment, the series of processing programs are preinstalled in the system, but instead of provision by such preinstallation, they may be provided by downloading via a network such as the Internet. Or you may provide via various recording media. As such a recording medium, various recording media such as a magnetic disk such as an optical disk such as a CD-ROM and a DVD, a removable hard disk device integrated with a drive mechanism, and a memory card can be applied.

It is a figure which shows a mode that the image | video of important information is interrupted and displayed on the display screen which the user is gazing at. It is a figure which shows a mode that the audio | voice of the image | video of the display screen which the user is gazing is output. It is a figure which shows a mode that an important image is detected and displayed on the display screen which the user is not gazing at. It is the schematic of the multi-display apparatus in 1st Embodiment. It is a block diagram which shows the structure of the multi-display apparatus in 1st Embodiment. It is a flowchart which shows the processing operation which a gaze detection part performs. It is a figure which shows the position and radius of the eyeball center in a user's eyeball which the gaze detection part estimated. It is the schematic of the multi-display apparatus in 2nd Embodiment. It is a block diagram which shows the structure of the display apparatus in 2nd Embodiment. It is a block diagram which shows the structure of an important image extraction part. It is a flowchart which shows the processing operation which each display in 2nd Embodiment performs. It is a block diagram which shows the structure of an event detection part. It is a flowchart which shows the processing operation at the time of an event detection part detecting a suspicious person.

Explanation of symbols

  2 switching unit, 3 importance detection unit, 4 display unit, 5 control unit, 10 multi-display device, 12 gaze detection unit, 13 remote control, 20 multi-display device, 23 emergency video detection unit, 52 control unit, 53 important image extraction , 54 video selection unit, 110 display device, 210 display device, 231 surveillance camera, 232 difference circuit, 233 memory, 531 important image detection unit, 532 memory

Claims (4)

Display means for displaying a plurality of videos on a plurality of display screens and outputting sound corresponding to each of the plurality of videos from a speaker corresponding to each of the plurality of display screens;
A line-of-sight detecting means for detecting a display screen being watched by the user among the plurality of display screens;
Emergency video detection means for detecting video of high urgency information,
Video selection means for selecting one video signal from a plurality of video signals;
Control means for controlling the display means and the video selection unit based on the data of the display screen detected by the line-of-sight detection means;
An important image extracting means for extracting an image including important information in the video signal from the video signal ,
The video selection means is based on one video signal selected by the video selection means on the display screen detected by the line-of-sight detection means among the plurality of display screens. Other than the display screen detected by the line-of-sight detection means among the plurality of display screens, the video corresponding to one video signal selected by the video selection means is output to the speaker of the display means. Displaying the image including the important information extracted by the important image extraction means on the display screen of the display and turning off the sound output of the speaker corresponding to the display screen other than the display screen detected by the line-of-sight detection means , when the image of the urgency is high information is detected by the emergency image detection means, multi which the urgency is to display a picture of high information detected display screen by said visual axis detecting means Isupurei apparatus. The emergency image detection means, multi-display apparatus of claim 1, wherein the video shall be the image of the urgency is high information including the event that the monitoring camera is taken. A display device used in a multi-display device that displays a plurality of images,
Display means for displaying a plurality of videos on a plurality of display screens and outputting sound corresponding to each of the plurality of videos from a speaker corresponding to each of the plurality of display screens;
A line-of-sight detecting means for detecting a display screen being watched by the user among the plurality of display screens;
Video selection means for selecting one video signal from a plurality of video signals;
Control means for controlling the display means and the video selection unit based on the data of the display screen detected by the line-of-sight detection means;
An important image extracting means for extracting an image including important information in the video signal from the video signal ,
The video selection means is based on one video signal selected by the video selection means on the display screen detected by the line-of-sight detection means among the plurality of display screens. Other than the display screen detected by the line-of-sight detection means among the plurality of display screens, the video corresponding to one video signal selected by the video selection means is output to the speaker of the display means. Displaying the image including the important information extracted by the important image extraction means on the display screen of the display and turning off the sound output of the speaker corresponding to the display screen other than the display screen detected by the line-of-sight detection means , When a video of information with a high degree of urgency is input from an emergency video detection means for detecting a video with a high degree of urgency, the information with a high degree of urgency is displayed on the display screen detected by the line-of-sight detection means Display device to display the image. The emergency image detection means, the display device according to claim 3, wherein you the video of the urgency is high information images including the event that the monitoring camera is taken.

Images