JP2013073222A - Image processing apparatus, image processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To avoid contact or collision of a viewer with a display device to prevent an unexpected situation.SOLUTION: A contact/collision possibility detection unit 11 detects a viewer who is possible to contact/collide with a display device for displaying an image. If the contact/collision possibility detection unit 11 detects a viewer who is possible to contact/collide with the display device, a warning display superposition unit 12 superposes a warning display on an image displayed on the display device.

Description

  The present invention relates to a technique for preventing a viewer from touching or colliding with a display device.

  2. Description of the Related Art In recent years, devices that provide various information using a large-sized video display device are widely used regardless of whether they are outdoors or indoors. For example, there are many outdoor-installed video display devices that provide various advertisements in front of a station or in commercial areas. Also, in the buildings of stations and airports, video display devices that display various information such as departure time and arrival time and operation status and advertisements are used.

  Since the display elements constituting the video display device have been increased in size and accuracy, it is expected that devices that provide such various information will increase in size in the future. In addition, by increasing the size and accuracy of the display element, it is possible to display an image that is mistaken for reality. Since video that is misunderstood in reality has a great appealing effect to viewers, it is expected that video display devices that can display high-definition video in addition to large size will be widely used in information provision devices such as electronic advertisements in the future. Is done.

  However, since the display video provided by such a video display device has a sense of reality, the viewer may not recognize that the video is a video and may misidentify that the entity exists. And if a viewer does not recognize that it is a video and misidentifies that it exists, an unexpected situation may occur. Therefore, techniques for avoiding viewer contact and collision are used in video display devices that display large and high-definition video, or information providing devices that use such video display devices.

  As a method for avoiding viewer contact and collision, there is a method of physically isolating the display surface with a pylon, a fence or a rope. According to this method, due to the presence of the pylon, the fence, or the rope, the viewer does not approach the display surface until it touches or collides, and the viewer can avoid contact or collision. However, in this method, there is a problem that the existence of a pylon, a fence, or a fence for physically isolating the display surface kills the appeal effect to the viewer.

  As another method, when there is a possibility of contact or collision, there is a method of issuing a warning to the viewer with sound or light to avoid the viewer's contact or collision. According to this method, when it is determined that there is a possibility of contact or collision, a warning sound is emitted from the speaker, or a warning light is flashed or rotated to alert the viewer, thereby avoiding the viewer's contact or collision. However, in this method, there is a problem that a warning is reached even for a viewer who does not have a possibility of contact or collision, and the viewer's emotion is killed. The warning sound from the speaker and the warning light from the warning light reach not only a viewer who has a risk of contact or collision but also a viewer who has no risk of contact or collision. Therefore, both viewers who have a risk of contact and collision and viewers who have no risk of contact and collision will be killed by the warning sound from the speaker or the warning light from the warning light. Furthermore, not only is the impression lost, but the information displayed by the video display device, for example, when the video display device is an electronic advertising device, the viewer has a bad impression on the target product. The fear cannot be denied.

  Patent Document 1 discloses a technique for measuring a distance between a viewer and a display surface and displaying a warning on the display surface when the distance is not appropriate. Patent Document 2 discloses a technique for transmitting a warning and generating a warning sound to a portable terminal of a viewer who has a risk of contact or collision.

Japanese Patent Laid-Open No. 3-200995 JP 2006-331154 A

  However, in the technique disclosed in Patent Document 1, since the warning condition is only the distance between the viewer and the display surface, there is no risk of contact or collision, but the viewer is not at an appropriate distance to the display surface. There is also a problem of issuing a warning. Examples of viewers who do not have a risk of contact or collision, but whose distance to the display surface is not appropriate include viewers stopped in front of the display surface and viewers moving in parallel with the display surface. It is done.

  Moreover, the technique disclosed in Patent Document 2 has a problem that a warning cannot be received unless a portable terminal capable of receiving an alarm is possessed. In public places such as stations, an unspecified number of people come and go, so not all viewers have portable terminals that can receive alarms. Therefore, the contact and collision avoidance according to the prior art is limited.

  Therefore, an object of the present invention is to prevent the viewer from touching or colliding with the display device and preventing the occurrence of an unexpected situation.

  The image processing apparatus according to the present invention has a detection unit that detects a viewer who may contact or collide with a display device that displays an image, and the detection unit may cause a contact or collision with the display device. And a superimposing unit that superimposes a warning display on an image displayed on the display device when the viewer is detected.

  ADVANTAGE OF THE INVENTION According to this invention, it can avoid that a viewer contacts or collides with a display apparatus, and it becomes possible to prevent generation | occurrence | production of an unexpected situation.

It is a figure which shows the structure of the video display apparatus which concerns on embodiment of this invention. It is a figure which shows the coordinate system used in embodiment of this invention. It is a figure which shows the structure of the concern detection part of a contact and a collision in 1st Embodiment. It is a figure for demonstrating the viewer information produced | generated by the viewer information collection part. It is a flowchart which shows the process which the possibility estimation part of a contact and a collision estimates the possibility of a contact and a collision. It is a flowchart which shows the process of the warning display superimposition part in 1st Embodiment. It is a figure which shows the example of a warning display area. It is a figure which shows the example which displayed the image | video which superimposed the warning display on the display surface. It is a figure which shows the structure of the possibility detection part of a contact and a collision in 2nd Embodiment. It is a flowchart which shows the process which the possibility estimation part of a contact and a collision in 2nd Embodiment estimates the possibility of a contact and a collision. It is a flowchart which shows the process of the warning display superimposition part in 4th Embodiment. It is a flowchart which shows the process of the warning display superimposition part in 5th Embodiment. It is a flowchart which shows the process of the warning display superimposition part in 6th Embodiment.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments to which the invention is applied will be described in detail with reference to the accompanying drawings. However, the present invention is not limited to the embodiments described below, and can be appropriately changed without departing from the gist of the present invention.

  First, in describing an embodiment of the present invention, a coordinate system to be used will be described. FIG. 2 is a diagram showing a coordinate system used in the present embodiment. The coordinate system used in this embodiment is a right-handed system. That is, the point at which the lower left end of the display surface 31 intersects the horizontal plane is taken as the origin. The x axis is determined perpendicular to the display surface 31 at the origin. The x-axis direction is a direction away from the display surface 31. Then, the y-axis and the z-axis are determined so as to be a right-handed coordinate system. The shape of the display surface 31 is not limited to a flat surface, and the display surface 31 may be a curved surface. Even if the display surface 31 is a curved surface, the coordinate system is the same.

  Further, in describing the embodiment of the present invention, when it is necessary to clearly distinguish the viewers based on the possibility of contact or collision, the viewers who are likely to contact or collide are referred to as viewers 35. A viewer who does not have a risk of collision is defined as a viewer 36. In FIG. 2, the viewer 35 who has a risk of contact or collision is moving toward the display surface 31, and there is a risk of contact or collision. On the other hand, the viewer 36 who has no risk of contact or collision moves in parallel with the display surface 31 and has no risk of contact or collision.

  FIG. 1 is a diagram showing a configuration of a video display apparatus according to the first embodiment of the present invention. As shown in FIG. 1, the video display device according to the first embodiment includes a contact / collision detection unit 11 and a warning display superimposing unit 12. The contact or collision risk detection unit 11 detects a viewer who is predicted to contact or collide with the display surface 31 within a predetermined period Pt0. Then, the contact / collision detection unit 11 collects the detected information for each viewer, generates the target information 101 that may be detected, and outputs the target information 101 to the warning display superimposing unit 12. The warning display superimposing unit 12 superimposes a signal, that is, a warning display, that is sufficient for a viewer who may be in contact with or collides to recognize the presence of the video display device, on the video signal, based on the target information 101 of the target. Note that the video display device according to the present embodiment has a configuration as an application example of the image processing device.

  Here, the contact or collision fear detection unit 11 will be described in detail. FIG. 3 is a diagram illustrating a configuration of the contact or collision fear detection unit 11. As illustrated in FIG. 3, the contact or collision risk detection unit 11 includes a viewer information collection unit 21 and a contact or collision risk estimation unit 22.

FIG. 4 is a diagram for explaining the viewer information (111 in FIG. 3) generated by the viewer information collection unit 21. As shown in FIG. 4, the viewer information 111 includes the coordinates Pm, the velocity vector Vm, and the face direction vector Am of the viewer 41. Here, the subscript m is an identifier for identifying the viewer 41. The coordinates Pm of the viewer 41 are the coordinates of a point representing the viewer 41. For example, the coordinate value at the center of the top of the viewer 41 is set as the coordinate Pm. Of course, the coordinate Pm of the viewer 41 is not limited to the coordinate value at the center of the top of the viewer 41. The coordinates Pm, the velocity vector Vm, and the face direction vector Am are represented by component display as follows.
Pm = (Pxm, Pym, Pzm)
Vm = (Vxm, Vym, Vzm)
Am = (Axm, Aym, Azm)
The viewer information 111 is not limited to the above three items. That is, it does not prevent the viewer information 111 from including more items.

  Next, the operation of the viewer information collection unit 21 will be described. Many methods have been proposed for acquiring position information of an object such as a position and speed, and the present invention does not depend on a position information acquisition method. Therefore, as long as at least the coordinates Pm, the velocity vector Vm, and the face direction vector Am of the viewer 41 can be acquired, the viewer information collecting unit 21 may be configured using any known technique. For example, Susumu Kubota, Masayuki Maruyama, Tomonori Ikumi, Masami Takahata: Human Traffic Line Measurement System Using Multiple Omnidirectional Cameras, Toshiba Review, Vol. 63, no. 10 (2008) discloses a technique for detecting a flow line of a person using an omnidirectional camera. By applying this known technique, it is possible to detect the coordinate Pm, the velocity vector Vm, and the face direction vector Am of the viewer 41.

  Further, the means for detecting the coordinate Pm and the velocity vector Vm of the viewer 41 may be different from the means for detecting the face orientation vector Am of the viewer 41. In this case, after detecting the coordinate Pm and the velocity vector Vm of the viewer 41 and the face direction vector Am of the viewer 41 by different means, the viewer information collection unit 21 combines them to view the viewer information. 111 is formed. For example, the detection means for detecting the coordinates Pm and the velocity vector Vm of the viewer 41 using an ultrasonic sensor and the means for detecting the face orientation vector Am by face recognition from video captured by a camera are separate detection means. It is an example using. Of course, the combination of the detection means is not limited to the combination of the detection means for the coordinates Pm and the velocity vector Vm by the ultrasonic sensor and the detection means for the face orientation vector Am by face recognition.

  Next, the operation of the contact or collision fear estimation unit 22 will be described. Based on the viewer information 111, the contact or collision risk estimation unit 22 estimates (estimates) a value indicating the risk of contact or collision with the display surface 31 for each viewer 41 within a predetermined period Pt0. ) Then, the contact / collision estimation unit 22 extracts a viewer 35 who has a risk of contact or collision based on the value indicating the possibility of contact or collision. Thereafter, the contact or collision risk estimation unit 22 outputs the viewer information 111 of the viewer 35 who may have contact or collision as the feared target person information 101. Here, there is a possibility that the target information 101 is information including the coordinates Pm, the velocity vector Vm, the face direction vector Am, and a value indicating the possibility of contact or collision of the viewer 35 who may have contact or collision.

  In the present invention, means for estimating the risk of contact or collision is not limited. Therefore, any means may be used as long as it can be estimated that there is a possibility of contact or collision with the display surface 31 within the period Pt0. Therefore, an example using linear prediction will be described as an example of the contact or collision risk estimation unit 22. Prior to explaining the operation of the contact / collision estimation unit 22, the buffer distance Bl and the vector equation Ds representing the range of the display surface 31 will be described.

  First, the buffer distance Bl will be described. In the present embodiment, the buffer distance B1 is defined as follows. That is, the buffer distance B1 is a distance where the viewer 41 does not touch the display surface 31 even if the viewer 41 extends his / her hand if the viewer 41 is separated from the display surface 31 by the buffer distance B1 or more. Therefore, the buffer distance B1 has a value larger than the length of the viewer's 41 hand. The value of the buffer distance B1 may be determined in advance, or may be dynamically calculated according to the shape of the display surface 31, the position of the viewer 41, or the height of the viewer 41.

  Next, the vector equation Ds representing the range of the display surface 31 will be described. In the present embodiment, the display surface 31 is a flat surface. The display surface 31 is assumed to have a size of dy in the horizontal direction and dz in the vertical direction, including the size of members such as a frame. Assuming that the element of the vector equation Ds representing the range of the display surface 31 is Ds = (Dsx, Dsy, Dsz), the vector equation Ds representing the range of the display surface 31 is expressed by the following Equation 1. In this embodiment, since the origin is a point where the lower left end of the display surface 31 intersects the horizontal plane and the x axis is perpendicular to the display surface 31, the x component of the vector equation Ds representing the range of the display surface 31 is used. The value of Dsx is always 0. Of course, the shape of the display surface 31 is not limited to a plane. Even if the display surface 31 is a curved surface, it can be similarly implemented if the vector equation Ds is defined.

  It is assumed that the area where the pixels exist on the display surface 31 has a size of diy in the horizontal direction and diz in the vertical direction. Hereinafter, the region where the pixels of the display surface 31 are present is referred to as “pixel region of the display surface 31”. Here, the lower left coordinate toward the pixel area of the display surface 31 is (0, dioy, dioz). diy, diz, dioy and dioz, and dy and dz satisfy the relational expression shown in the following Expression 2.

In the present embodiment, the value indicating the possibility of contact or collision assumes three values of 0, 1, and 2. And each value shows the following states.
Value indicating the possibility of contact or collision State 0 There is no possibility of contact or collision with the display surface 31 within the period Pt0 1 There is a possibility of contact or collision with the display surface 31 within the period Pt0 2 The viewer 35 within the buffer distance Bl Exist
That is, if the viewer 35 reaches out, there is a risk of contact or collision.

  Accordingly, in the present embodiment, the contact or collision risk estimation unit 22 is concerned with the viewer 41 having a value of 1 or 2 indicating the risk of contact or collision as the viewer 35 who may have contact or collision. Person information 101 is generated. And about the viewer 41 whose value which shows the possibility of a contact and a collision is 0, the possibility estimation part 22 of a contact and a collision does not produce | generate the target information 101 with a possibility.

  Next, referring to the flowchart shown in FIG. 5, a process in which the contact or collision risk estimation unit 22 estimates the risk of contact or collision will be described. The contact or collision risk estimation unit 22 repeats the process shown in FIG. 5 for each viewer 41 every moment to estimate the risk of contact or collision.

  In step S101, the contact or collision risk estimation unit 22 compares the distance L between the viewer 41 and the display surface 31 with the buffer distance Bl, and the position of the viewer 41 is separated from the display surface 31 by the buffer distance Bl. It is determined whether or not. The distance L between the viewer 41 and the display surface 31 may be obtained by calculating the length of a perpendicular line drawn from the coordinate Pm of the viewer 41 to the vector equation Ds representing the range of the display surface 31. In the present embodiment, since the display surface 31 is a flat surface, the distance L between the viewer 41 and the display surface 31 is the x-direction component Pxm of the coordinates Pm of the viewer 41. Therefore, if Pxm> B1 and the position of the viewer 41 is separated from the display surface 31 by the buffer distance B1, the process proceeds to step S102. On the other hand, if Pxm ≦ B1 and the distance L between the viewer 41 and the display surface 31 is equal to or less than the buffer distance B1, the process proceeds to step S107.

  In step S <b> 102, the contact / collision estimation unit 22 predicts the future position Fm of the viewer 41 linearly. Here, the future position is a position where the viewer 41 is predicted to arrive within the period Pt0. When t is a parameter, the future position Fm of the viewer 41 is expressed by the following formula 3.

  In step S <b> 103, the contact or collision risk estimation unit 22 calculates an intersection Gm between the vector equation Ds representing the range of the display surface 31 and the future position Fm of the viewer 41. At the intersection Gm, Ds = Fm. When this relationship is expressed by component display, it is as shown in Equation 4. The result of solving Equation 4 for each variable is Equation 5. When the solutions of t, Dsy, and Dsz shown in Expression 5 are within the range of possible values, the intersection point Gm exists. That is, when all the relational expressions shown in Expression 6 are satisfied, the intersection point Gm exists. On the other hand, if there is any relational expression that does not hold among the relational expressions representing Expression 6, the intersection point Gm does not exist.

  In step S104, the contact or collision risk estimation unit 22 determines whether or not the intersection point Gm exists. If the intersection Gm does not exist, the process proceeds to step S105. On the other hand, when the intersection point Gm exists, the process proceeds to step S106. When the intersection point Gm does not exist, there is no possibility that the viewer 41 touches or collides with the display surface 31 within the period Pto. Therefore, in step S105, the contact or collision risk estimation unit 22 sets the value indicating the risk of contact or collision to 0. On the other hand, when the intersection point Gm exists, the contact or collision risk estimation unit 22 estimates that the viewer 41 may contact or collide with the display surface 31 within the period Pt0. Therefore, in step S106, the contact or collision risk estimation unit 22 sets the value indicating the risk of contact or collision to 1. In addition, when the distance between the viewer 41 and the display surface 31 is equal to or less than the buffer distance Bl, there is a possibility that the display surface 31 may come into contact with or collide if the hand is reached at this time. Therefore, in step S107, the contact or collision risk estimation unit 22 sets the value indicating the risk of contact or collision to 2.

  Next, the operation of the warning display superimposing unit 12 will be described with reference to FIG. FIG. 6 is a flowchart showing the processing of the warning display superimposing unit 12. The warning display superimposing unit 12 outputs the video 103 on which the warning display is superimposed by repeating the process shown in FIG. 6 for the video input 102 every moment for each viewer 35 who has a possibility of contact or collision.

  In step S <b> 111, the warning display superimposing unit 12 calculates an intersection Sm between a line segment passing through the coordinate Pm and parallel to the face orientation vector Am and the vector equation Ds representing the range of the display surface 31. The component display of the line segment that passes through the coordinate Pm and is parallel to the face orientation vector Am is expressed by the following Expression 7 using u as a parameter. Since the intersection Sm is a point on the vector equation Ds, Expression 8 is established. When Equation 8 is solved for Sym and Szm, the value of each component at the intersection Sm is Equation 9.

  In step S112, the warning display superimposing unit 12 sets the warning display area Ws around the intersection Sm. The warning display area Ws may be rectangular or elliptical. FIG. 7 is a diagram showing a case where the warning display area Ws has a rectangular shape and a case where the warning display area Ws has an elliptical shape.

  FIG. 7A shows a rectangular warning display area Ws51 having a size of 2 Wsy in the y-axis direction and 2 Wsz in the z-axis direction. FIG. 7B shows an elliptical warning display area Ws51 having a major axis of 2 Wsy and a minor axis of 2 Wsz. Of course, when the shape of the warning display area Ws51 is an ellipse, the major axis is not limited to the y-axis direction.

  Here, Wsy and Wsz indicating the size of the warning display area Ws51 are determined depending on a value indicating a possibility of contact or collision. Wsy and Wsz used when the value indicating the possibility of contact or collision is 1, respectively, Wsy1 and Wsz1, and Wsy and Wsz used when the value indicating the possibility of contact or collision are 2 are respectively Wsy2. And Wsz2. The values of Wsy1, Wsy2, Wsz1, and Wsz2 satisfy the following expression 10. In step S112, when the value indicating the possibility of contact or collision is 1, the warning display superimposing unit 12 selects Wsy1 and Wsz1 and sets the warning display region Ws51. On the other hand, when the value indicating the possibility of contact or collision is 2 in step S112, the warning display superimposing unit 12 selects Wsy2 and Wsz2 and sets the warning display area Ws51.

  Wsy1, Wsy2, Wsz1, and Wsz2 may be determined in advance, or may be determined from the distance L between the viewer 35 and the display surface 31 that may be contacted or collided, and a value that indicates the possibility of contact or collision. It may be calculated each time step S112 is executed. In step S <b> 113, the warning display superimposing unit 12 determines whether or not there is an intersection between the warning display area Ws <b> 51 and the pixel area of the display surface 31. When there is an intersection between the warning display area Ws51 and the pixel area of the display surface 31, the following Expression 11 is established. Therefore, when Formula 11 is true, the process proceeds to step S114. On the other hand, when Expression 11 is false, there is no intersection between the warning display area Ws51 and the pixel area of the display surface 31, and thus the process for the viewer 35 who has a risk of contact or collision at the current time is terminated.

  In step S114, the warning display superimposing unit 12 calculates the average hue value AH of the warning display area Ws51 and compares it with the threshold value AHth0. When the average hue value AH is larger than the threshold value AHth0, the process proceeds to step S115. On the other hand, when the average value AH of the hue is equal to or less than the threshold value AHth0, the process proceeds to step S116.

  Here, the calculation method of the average value of the hue in step S114 is not limited. Therefore, the average value AH of hues may be obtained using a simple average or a weighted average. The threshold value AHth0 may be calculated in advance based on the characteristics of the display surface 31 and the status of the installation location, or there is a change in the characteristics of the display surface 31 and the status of the installation location each time step S114 is executed. It may be calculated every time.

  In step S115, the warning display superimposing unit 12 decreases the hue of the warning display area Ws51 by Hth 0%. In step S116, the warning display superimposing unit 12 increases the hue of the warning display area Ws51 by Hth1%.

  Here, the decrease rate Hth0 and the increase rate Hth1 are determined depending on values indicating the possibility of contact or collision. The values of Hth0 and Hth1 used when the value indicating the possibility of contact or collision is 1 are Hth01 and Hth11, respectively. Further, the values of Hth0 and Hth1 used when the value indicating the possibility of contact or collision is 2 are Hth02 and Hth12, respectively. Hth01, Hth02, Hth11, and Hth12 satisfy the following expression 12. In step S115, when the value indicating the possibility of contact or collision is 1, the warning display superimposing unit 12 selects Hth01 and decreases the hue of the warning display area Ws by Hth01%. On the other hand, when the value indicating the possibility of contact or collision is 2, the warning display superimposing unit 12 selects Hth02 and decreases the hue of the warning display area Ws51 by Hth02%. On the other hand, if the value indicating the possibility of contact or collision is 1 in step S116, the warning display superimposing unit 12 selects Hth11 and increases the hue of the warning display area Ws51 by Hth11%. On the other hand, when the value indicating the possibility of contact or collision S is 2, the warning display superimposing unit 12 selects Hth12 and decreases the hue of the warning display area Ws51 by Hth12%.

  Hth01, Hth02, Hth11, and Hth12 may be determined in advance based on the characteristics of the display surface 31 and the installation location. Alternatively, each time step S115 or S116 is executed, the calculation may be performed based on the image in the warning display area Ws51, the characteristics of the display surface 31, and the installation location. As described above, in the present embodiment, the hue change is used for the warning display, but of course, the use of the hue change for the warning display is not limited.

  FIG. 8 is a diagram illustrating an example in which a video with a warning display superimposed thereon is displayed on the display surface 31. 8 (a) and 8 (b), the hue is in the direction in which the face of the viewer 35 who is likely to contact or collide, that is, in the center of the visual field of the viewer 35 who may be contacted or collided. A changed warning display area Ws51 appears. In the example illustrated in FIG. 8, the warning display area Ws51 appears on the display object 32. By the appearance of the warning display area Ws51 having a hue different from that of the surroundings, the viewer 35 who has a risk of contact or collision recognizes the presence of the display surface 31. On the other hand, when the distance from the display surface 31 is long as shown in FIG. 8A, or when the face direction is different from the viewer 35 who may be in contact with or collided as shown in FIG. 8B. The viewer 36 who is not likely to collide is less likely to recognize the warning display area Ws51.

  Next, a second embodiment of the present invention will be described. Note that the configuration of the video display apparatus according to the second embodiment is the same as the configuration shown in FIG. FIG. 9 is a diagram illustrating the configuration of the contact or collision fear detection unit 11 according to the second embodiment. The contact / collision risk detection unit 11 illustrated in FIG. 9 includes a viewer information collection unit 21, a contact / collision risk estimation unit 24, and an existence recognition estimation unit 23.

  The viewer information collection unit 21 in the second embodiment shown in FIG. 9 is the same as the viewer information collection unit 21 in the first embodiment shown in FIG. Accordingly, the viewer information 111 output from the viewer information collection unit 21 in the second embodiment includes the coordinates Pm, the velocity vector Vm, and the face direction vector Am of the viewer 41. The viewer information 111 is not limited to the above three items. That is, it does not prevent more items from being included in the viewer information 111.

Next, the process of the presence recognition estimation part 23 is demonstrated. The presence recognition estimation unit 23 estimates whether the viewer 41 recognizes the presence of the display surface 31 and outputs the result as a presence recognition estimated value 112. In the present embodiment, the existence recognition estimated value 112 takes binary values of 0 and 1, and each value indicates the following state.
Presence recognition estimated value State 0 It is estimated that the viewer 41 does not recognize the presence of the display surface 31 1 It is estimated that the viewer 41 recognizes the presence of the display surface 31

  When the viewer 41 approaching the display surface 31 recognizes the presence of the display surface 31 and determines that the contact or collision occurs, the viewer 41 decreases the speed or moves away from the display surface 31 in order to avoid contact or collision. Change the direction of movement. Therefore, the presence recognition estimation unit 23 obtains a component in the direction perpendicular to the display surface 31 of the velocity vector Vm, and estimates whether the viewer 41 recognizes the presence of the display surface 31 based on the value. . In the present embodiment, since the display surface 31 is a flat surface, the component of the velocity vector Vm in the direction perpendicular to the display surface 31 is Vxm. Therefore, Vxm is classified as shown in the following table, and it is estimated whether or not the viewer 41 recognizes the presence of the display surface 31.

  In the above table, in category 1, Vxm is greater than 0, and the viewer 41 is moving away from the display surface 31. Therefore, the presence recognition estimation unit 23 estimates that the viewer 41 is exercising to avoid contact or collision with the display surface 31 and sets the value of the presence recognition estimation value 112 to 1. In category 2, Vxm is equal to 0, and the distance between the viewer 41 and the display surface 31 does not change. In this case, the presence recognition estimation unit 23 estimates that the viewer 41 recognizes the presence of the display surface 31 and sets the value of the presence recognition estimation value 112 to 1. In classification 3, Vxm is 0 or less, and the x-axis is negative in the direction approaching the display surface 31. Therefore, the viewer 41 is moving in a direction approaching the display surface 31. Furthermore, since the value obtained by differentiating Vxm with respect to time is 0 or less, the viewer 41 approaches the display surface 31 without changing the speed or increasing the speed. In this case, the presence recognition estimation unit 23 estimates that the viewer 41 does not recognize the presence of the display surface 31, and sets the value of the presence recognition estimation value 112 to 0. In category 4, Vxm is 0, and the value obtained by differentiating Vxm with respect to time is greater than 0. In this case, the viewer 41 is approaching the display surface 31, but the approaching speed is decreasing. Therefore, the presence recognition estimation unit 23 estimates that the viewer 41 recognizes the presence of the display surface 31 and sets the value of the presence recognition estimation value 112 to 1.

  Next, processing of the contact and collision risk estimation unit 24 will be described. The contact / collision possibility estimation unit 24 occasionally sets a value indicating the possibility of contact or collision with the display surface 31 within a predetermined period Pt0 for each viewer 41 based on the viewer information 111 and the presence recognition estimated value. Guess every moment. Then, the contact / collision risk estimation unit 24 extracts viewers 35 who have a risk of contact or collision based on the estimated contact or collision value. Thereafter, the presence recognition estimation unit 23 outputs the viewer information 111 of the viewer 35 who is likely to be contacted or collided as the target information 101 that is likely to exist. Here, there is a possibility that the target information 101 is information including the coordinates Pm, the velocity vector Vm, the face direction vector Am, and a value indicating the possibility of contact or collision of the viewer 35 who may be contacted or collided. is there.

In the second embodiment, as in the first embodiment, the values indicating the possibility of contact and collision estimated by the contact and collision risk estimation unit 24 take three values of 0, 1, and 2, respectively. The values shall indicate the following states:
Value indicating the possibility of contact or collision State 0 The viewer 41 exists outside the buffer distance Bl,
Is there a possibility of contact or collision with the display surface 31 within the period Pt0?
Or, the viewer 41 recognizes the presence of the display surface 31. 1 The viewer 41 exists outside the buffer distance Bl,
There is a risk of contact or collision with the display surface 31 within the period Pt0.
And the viewer 41 does not recognize the presence of the display surface 31 2 The viewer 41 exists within the buffer distance Bl.

  The contact or collision risk estimation unit 24 generates the target information 101 with the possibility of the viewer 41 having a value of 1 or 2 indicating the possibility of contact or collision as the viewer 35 with the risk of contact or collision. On the other hand, for the viewer 41 whose value indicating the risk of contact or collision is 0, the contact or collision risk estimation unit 24 may not generate the target information 101.

  FIG. 10 is a flowchart illustrating a process in which the contact / collision estimation unit 24 estimates the risk of contact / collision in the second embodiment. The contact or collision risk estimation unit 24 repeats the process shown in FIG. 10 for each viewer 41 every moment to estimate the risk of contact or collision. Of the processes shown in FIG. 10, steps S101, S102, S103, S105, S106 and S107 are the same as the processes of the same reference numerals in the first embodiment shown in FIG. In the following, the difference from the process shown in FIG. 5 in the first embodiment will be described, and the description of the same process as in the first embodiment will be omitted.

  In step S124, the contact or collision risk estimation unit 24 determines whether or not the intersection point Gm exists. If the intersection Gm does not exist, the process proceeds to step S105. On the other hand, when the intersection point Gm exists, the process proceeds to step S125. In step S <b> 125, the contact or collision risk estimation unit 24 evaluates the existence recognition estimated value 112. Then, when the existence recognition estimated value 112 is 1, that is, when it is estimated that the viewer 41 recognizes the presence of the display surface 31, the process proceeds to step S105. On the other hand, when the existence recognition estimated value 112 is 0, that is, when it is estimated that the viewer 41 does not recognize the presence of the display surface 31, the process proceeds to step S106.

  Next, a third embodiment of the present invention will be described. The configuration of the video display apparatus according to the third embodiment is the same as the configuration shown in FIG. Further, the configuration of the contact or collision risk detection unit 11 in the third embodiment is the same as the configuration shown in FIG. 3, but in this embodiment, fuzzy control is applied to the implementation of the contact or collision risk estimation unit 22. Applied. Note that fuzzy control is a well-known technique, and for example, in a factory, an automatic guided vehicle is used for control to avoid an obstacle.

  The contact / collision estimation unit 22 in the third embodiment is based on fuzzy theory, and the degree of fitness that the future position of the viewer 41 intersects with the display surface 31 from the coordinates Pm and the velocity vector Vm of the viewer 41. A multidimensional membership function μ (Pm, Vm) is set for calculating. Then, the contact or collision risk estimation unit 22 uses this multidimensional membership function μ (Pm, Vm) to estimate the risk of contact or collision.

  Here, the multidimensional membership function μ (Pm, Vm) depends on the shape of the display surface 31 and the situation of the place where the display surface 31 is installed. In fuzzy control, the membership function μ is generally determined from the experimental result by the maximum likelihood method. Since the present invention does not depend on the method of generating the multidimensional membership function μ (Pm, Vm), the multidimensional membership function μ (Pm, Vm) may be determined by the maximum likelihood method from the experimental results. Of course, the generation of the multidimensional membership function μ (Pm, Vm) is not limited to the maximum likelihood method.

  The value of the multidimensional membership function μ (Pm, Vm) takes a value between 0 and 1. This value may be included in the target person information 101 as it is and may be used by the warning display superimposing unit 12. Or you may make it take the three values of 0, 1, and 2 similarly to the value which shows the possibility of contact and a collision shown in 1st Embodiment. In this case, the value of the multidimensional membership function μ (Pm, Vm) is compared with the threshold value Fth0. The value is 1. On the other hand, when the value of the multidimensional membership function μ (Pm, Vm) is less than or equal to the threshold value Fth0, a value indicating the possibility of contact or collision is set to 0. If the viewer 41 is present within the buffer distance Bl, a value indicating the possibility of contact or collision is set to 2 regardless of the value of the multidimensional membership function μ (Pm, Vm).

  Next, a fourth embodiment of the present invention will be described. The configuration of the video display apparatus according to the fourth embodiment is the same as the configuration shown in FIG. FIG. 11 is a flowchart showing the processing of the warning display superimposing unit 12 in the fourth embodiment. Hereinafter, the processing of the warning display superimposing unit 12 in the fourth embodiment will be described with reference to FIG. Of the processes shown in FIG. 11, steps S111, S112, and S113 are the same as the processes of the same reference numerals in the first embodiment shown in FIG. In the following, the difference from the process shown in FIG. 6 in the first embodiment will be described, and the description of the same process as in the first embodiment will be omitted.

  When there is an intersection between the warning display area Ws51 and the pixel area of the display surface 31, in step S134, the warning display superimposing unit 12 moves from the rear of the display surface 31 to the front in the vicinity of the intersection Sm52 in the video input 102. Evaluate the presence of subjects who are in motion. For example, if the video input 102 is an image in which many balls are jumping, there are balls that fly forward from the rear of the display surface 31 in the balls. If there is an object moving in the vicinity of the intersection Sm52 from the rear to the front of the display surface 31, the process proceeds to step S135. On the other hand, when there is no target moving in the vicinity of the intersection Sm52 from the rear to the front of the display surface 31, the process proceeds to step S136.

  In step S135, the warning display superimposing unit 12 performs affine transformation on the video input 102 so that a target moving from the rear to the front of the display surface 31 exists in the warning display area Ws51, and displays a warning. Is generated. At this time, when there are a plurality of objects moving from the rear to the front of the display surface 31, affine transformation is performed so that the object closest to the intersection Sm52 exists in the warning display area Ws51.

  If there is no target moving in the vicinity of the intersection Sm52 from the rear of the display surface 31 to the front, the warning display superimposing unit 12 moves from the rear of the display surface 31 to the front in step S136. The target is inserted into the warning display area Ws51, and the video 103 on which the warning display is superimposed is generated. Alternatively, the method of changing the hue of the warning display area Ws51 described in the first embodiment may be used in step S136.

  Next, a fifth embodiment of the present invention will be described. The configuration of the video display apparatus according to the fifth embodiment is the same as the configuration shown in FIG. FIG. 12 is a flowchart showing the processing of the warning display superimposing unit 12 in the fifth embodiment. Hereinafter, the processing of the warning display superimposing unit 12 in the fifth embodiment will be described with reference to FIG. Of the processes shown in FIG. 12, steps S111, S112, and S113 are the same as the processes of the same reference numerals in the first embodiment shown in FIG. In the following, the difference from the process shown in FIG. 6 in the first embodiment will be described, and the description of the same process as in the first embodiment will be omitted.

  If there is an intersection between the warning display area Ws51 and the pixel area of the display surface 31, in step S137, the warning display superimposing unit 12 captures the viewer 35 who may be in contact with or colliding. Then, the warning display superimposing unit 12 corrects the lighting reflection, hue, saturation, or brightness so that the captured video looks like a video reflected in a mirror. Then, the warning display superimposing unit 12 superimposes the video that reflects the viewer with the mirror effect obtained in this manner on the warning display area Ws51 around the intersection Sm52, and superimposes the video 103 on which the warning display is superimposed. create. Here, a device such as a camera that captures the viewer 35 who may be in contact with or colliding may use the viewer information collection unit 21 or may be separately installed and used. .

  Next, a sixth embodiment of the present invention will be described. The configuration of the video display apparatus according to the sixth embodiment is the same as the configuration shown in FIG. FIG. 13 is a flowchart showing the processing of the warning display superimposing unit 12 in the sixth embodiment. Hereinafter, the processing of the warning display superimposing unit 12 in the sixth embodiment will be described with reference to FIG. Of the processes shown in FIG. 13, steps S111, S112, and S113 are the same as the processes of the same reference numerals in the first embodiment shown in FIG. In the following, the difference from the process shown in FIG. 6 in the first embodiment will be described, and the description of the same process as in the first embodiment will be omitted.

  If there is an intersection between the warning display area Ws51 and the pixel area of the display surface 31, in step S133, the warning display superimposing unit 12 determines the distance L between the viewer 35 and the display surface 31 that may be in contact with or colliding with each other. The buffer distance B1 is compared. If the distance L between the viewer 35 who may be in contact with or colliding with the display surface 31 is less than the buffer distance Bl, the process proceeds to step S137. On the other hand, if the distance L between the viewer 35 who may be in contact with or colliding with the display surface 31 is greater than or equal to the buffer distance Bl, the process proceeds to step S134.

  Steps S134, S135, and S136 in FIG. 13 are the same as the processes of the same reference numerals in the fourth embodiment shown in FIG. Further, step S137 in FIG. 13 is the same as the processing of the same reference numerals in the fifth embodiment shown in FIG. That is, when the distance between the viewer 35 who may be in contact with or colliding with the display surface 31 is short and less than the buffer distance B1, the pseudo mirror described in the fifth embodiment is present in step S137. A warning with a simple video is carried out. On the other hand, when the viewer 35 who is likely to contact or collide is far away from the display surface 31 and the distance L between the viewer 35 who may possibly contact or collide and the display surface 31 is greater than or equal to the buffer distance Bl. In steps S135 and S136, the video effect warning described in the fourth embodiment is performed. In step S136, the method of changing the hue of the warning display area Ws51 described in the first embodiment may be used.

  In the above-described embodiment, a warning can be issued only to a viewer who is likely to contact or collide. In addition, contact and collision can be achieved without using devices or instruments that would kill the appealing effects of video display devices that provide realistic images, or even if viewers do not possess special devices or devices. A warning can be issued to a viewer who has a concern. It is also possible to issue a warning that does not kill the viewers' inspiration.

  Therefore, according to the above-described embodiment, a viewer who has a possibility of contact or collision can avoid contact with or collision with the video display device, and can prevent an unexpected situation. In addition, according to the above-described embodiment, viewers who do not have a risk of contact or collision are not bothered by a warning, and viewers who have a risk of contact or collision may be killed even if they receive a warning. Therefore, it is possible to maintain a good impression on the information provided by the video display device. Furthermore, according to the above-described embodiment, it is possible to avoid contact and collision even if the viewer does not have a special device or instrument, so that the installation location of the video display device is not limited. .

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

  11: Contact / collision detection unit, 12: Warning display superimposing unit, 21: Viewer information collection unit, 22, 24: Contact / collision estimation unit, 23: Presence recognition estimation unit

Claims (10)

Detecting means for detecting a viewer who may be in contact with or colliding with a display device for displaying an image;
And a superimposing unit that superimposes a warning display on an image displayed on the display device when the viewer who is likely to contact or collide with the display device is detected by the detection unit. Image processing device. The detection means includes
Obtaining means for obtaining at least one of the position of the viewer and the velocity vector of the viewer;
First estimation means for estimating a possibility that the viewer will contact or collide with the display device based on the information acquired by the acquisition means;
The image processing apparatus according to claim 1, wherein the viewer who is likely to contact or collide with the display device is detected based on an estimation result by the first estimation unit. The detection means includes
A second estimating means for estimating whether or not the viewer recognizes the presence of the display device;
The first estimation means detects the viewer who may contact or collide with the display device based on the information acquired by the acquisition means and the estimation result by the second estimation means. The image processing apparatus according to claim 2, wherein:   The image processing apparatus according to claim 1, wherein the superimposing unit selects the warning display to be superimposed on an image displayed on the display device from a plurality of warning displays.   The said superimposing means selects the said warning display superimposed on the image displayed on the said display apparatus from several warning displays according to the estimation result by the said 1st estimation means. 4. The image processing device according to any one of items 3.   The warning display includes an image in which the hue of an area corresponding to the warning display is changed, an image including an object that moves forward from the rear of the image processing apparatus, and an image that reflects the viewer with a mirror effect. The image processing apparatus according to claim 1, wherein the image processing apparatus includes at least one of the images. The first estimating means includes
Predicting means for predicting the future position of the viewer based on the position of the viewer and the velocity vector of the viewer;
The said viewer estimates that there exists a possibility of a contact or a collision with respect to the said display apparatus, when the position estimated by the said prediction means and the said display apparatus have an intersection. Image processing device.   The first estimating means is based on fuzzy theory for calculating a degree of fitness in which the future position of the viewer has an intersection with the display device based on the position of the viewer and the velocity vector of the viewer. The image processing apparatus according to claim 2, wherein the viewer estimates the possibility of contact or collision with the display device using a multidimensional membership function. An image processing method executed by an image processing apparatus,
A detection step of detecting a viewer who may be in contact with or colliding with a display device for displaying an image;
And a superimposing step of superimposing a warning display on an image displayed on the display device when the viewer who is likely to contact or collide with the display device is detected by the detecting step. Image processing method. A detection step of detecting a viewer who may be in contact with or colliding with a display device for displaying an image;
And causing the computer to execute a superimposing step of superimposing a warning display on an image displayed on the display device when the viewer who is likely to contact or collide with the display device is detected by the detection step. program.

Images