JP2012103746A - Method for display control for fascinating viewer, digital signage system, and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a novel display technique letting a person nearby a display screen of digital signage approach the screen on getting interested in display while involuntarily thinking of "what is it?".SOLUTION: (1) An image conversion process sets a view point in front of a foreground plane, and generates projection image data obtained by seeing through a background plane behind the foreground plate from the viewpoint. (2) The projection image data includes an image of the background plane after hidden surface removal with the image of the foreground plane. (3) An image display process sequentially displays the projection image data sequentially generated by the image conversion process on the display screen. (4) A face recognition process photographs the face of a person who views the display screen by a camera, and generates face position data corresponding to the interval distance between the display screen and face. (5) The image conversion process sets the view point based upon the face position data when the face recognition process generates the face position data, and generates the projection image data based upon the view point.

Description

  The present invention relates to a display control technique in the field of digital signage (electronic advertising system), and more particularly to a display technique that attracts the interest of a person who views a display screen.

  For digital signage, the first mission is to have the people in the store or public places see the advertising work to display with interest, and the advertising work that can be accepted with a good impression by the viewer It is desirable that

  In the invention of Japanese Patent Laid-Open No. 2009-128498 “Electronic Advertising System”, based on an advertisement summary downloaded by a mobile terminal of a person around the display screen, advertisement details corresponding to the summary are displayed. In the invention of Japanese Patent Application Laid-Open No. 2010-176648 “Information Processing System and Method by Cooperation of Mobile Terminal and Signage”, a signal from a mobile terminal of a person watching the display screen is received to change display contents.

  In the invention of “Digital Signage System” in Japanese Patent Application Laid-Open No. 2010-218489, a touch panel is attached to a display screen, and an advertisement is displayed, and a questionnaire or the like is performed by a touch input of a person who views the advertisement. In the invention of Japanese Patent Application Laid-Open No. 2010-111361, “Behavior Analysis Device, Behavior Analysis Method, and Program”, a person viewing an advertisement work is photographed with a camera attached to a display screen, and the characteristics of the person and the advertisement work viewed are linked. And record.

  In the invention of Japanese Patent Application Laid-Open No. 2010-111363 “Facial Image Extraction Device / Method and Electronic Advertising Device and Program”, people who faced the screen with a camera attached to the display screen were photographed, and they were there by chance. Extraction is performed by distinguishing people from those who are interested in viewing the displayed content.

  For example, in the case of a digital signage with a display screen of about 40 to 80 inches in an environment such as a station or a shopping mall, many people walk too much in front of the screen and some people are standing close to the screen. Also many. The distance between people and the display screen is as close as several meters or less, and if people are interested in the display content, they will approach the screen and read the detailed display content.

  This digital signage of the installation environment as described above is a so-called eye catcher in which a person around the display screen is unintentionally attracted to the display and approaches the screen. It was created for the purpose of providing a new display technique that is excellent in ability.

The core of the present invention is a display control method using a computer specified by the following items (1) to (6).
(1) The original image data includes at least a foreground plane and a foreground plane arranged at a predetermined interval in the three-dimensional coordinate system. (2) The image conversion process sets a viewpoint in front of the foreground plane. (3) The projection image data includes an image of the foreground plane from which the hidden surface is erased by the image of the foreground plane. (4) The image display process sequentially displays the projection image data sequentially generated by the image conversion process on the display screen. (5) The face recognition process captures the face of the person viewing the display screen with a camera. Creating face position data corresponding to the distance between the display screen and the face. (6) The image conversion process is a face recognition process. When you create a face position data, and set the viewpoint based on the face position data, to create a projection image data based on the viewpoint

1 shows an overview of a computer system that implements the method of the present invention. The structure of the original image data and the processing outline of the image conversion process in the method of the present invention will be shown. The example of a projection image display in the situation where the face position of the person who looks at a display screen is about 2 meters away from the screen. A display example of a projected image when the face position is about 1 meter from the screen. Display example of projected image when face position is close to 50 centimeters from the screen A display example in a situation where a person who has approached the screen up to 50 centimeters changes the position of his face and shakes his gaze to the right.

  An outline of a computer system for implementing the method of the present invention is shown in FIG. The basic configuration of a digital signage as a computer system is not significantly different from a known existing system. The display 1 has a screen size of about 40 to 80 inches, and is installed so that people in the vicinity can easily see the screen in an environment such as a station yard or a shopping mall.

  The display 1 is connected to the computer 2 and an image supplied from the computer 2 is displayed on the display 1. A camera 3 is attached to the display 1, and an arrangement relationship is set so that the camera 3 captures the face of a person viewing the screen from the front.

  The video output of the camera 3 is taken into the computer 2 and processed by a known algorithm by a face recognition process. The face recognition process recognizes and extracts a face image of a person watching the screen from the video output of the camera 1 and calculates face position data corresponding to the position of the person's face with respect to the display screen.

  The face position data includes Z data corresponding to the distance between the display screen and the face, and XY data corresponding to the two-dimensional position of the face in a plane parallel to the display screen. When the camera 3 is not a two-lens camera (two cameras) capable of stereoscopic viewing but a single camera, the distance between the display screen and the face is determined from the distance between the eyes in the recognized face image or the face image. Can be calculated.

  The system management process in the computer 1 extracts image data to be displayed from various image data stored in the image database based on a preprogrammed display schedule, and directly or appropriately selects the image data. The image is edited and processed and displayed on the display 1 through an image display process. The method of the present invention is implemented as one form in which an image is displayed on the display 1 by the computer 2.

  In carrying out the method of the present invention, the computer 2 extracts original image data to be displayed from the image database, performs perspective projection processing on the original image data by an image conversion process, and generates projection image data generated by the processing. To the image display process.

  FIG. 2 shows the configuration of the original image data and the processing outline of the image conversion process in the method of the present invention. The original image data includes at least a foreground plane and a background plane arranged at a predetermined interval in the three-dimensional coordinate system. The image conversion process sets a viewpoint in front of the foreground plane, creates projection image data in which the foreground plane and the background plane behind it are projected from the viewpoint on the projection plane by using a known algorithm. Data is displayed on the display 1 through an image display process. The projected image data to be displayed includes an image of the background plane that has been hidden by the foreground plane image.

  In the face recognition process described above, when a face image of a person viewing the display screen is extracted from the video output of the camera 3, the face position data is calculated and this data is passed to the image conversion process. However, during the period when the face image of the person who looks at the display screen is not detected, the face recognition process notifies the image conversion process accordingly.

  During the period when the face recognition process does not detect the face image of the person who looks at the display screen, the image conversion process creates projection image data with the initial position preset in accordance with the original image data as a viewpoint, and displays it on the display 1. Display. When this initial position is converted into, for example, face position data of a person viewing the display screen, the distance from the screen (Z data) is 3 meters, and the two-dimensional position of the face (XY data) in a plane parallel to the display screen Is the position corresponding to the center of the screen.

  When the projected image data calculated with the viewpoint as the initial position is displayed, when the face recognition process detects the face image of the person viewing the display screen and outputs the face position data, the image conversion process Projection image data is created by resetting the face position data as a viewpoint, and is displayed on the display 1. When a person who looks at the display screen further approaches the screen or moves his / her body left / right / up / down, the face position data changes, and the image conversion process recreates projection image data using the successively changing face position data as a viewpoint. As a result, the image displayed on the display 1 changes.

  In the case where the method of the present invention is implemented, a display example of a projected image in a situation where the face position of the person viewing the display screen is separated from the screen by, for example, about 2 meters is shown in FIG. FIG. 4 shows a display example of a projected image when the face is close to about a meter, and FIG. 5 shows a display example of a projected image when the face position is close to about 50 centimeters from the screen. FIG. 6 shows a display example in a situation where a person approaching the position changes the face position and swings his / her line of sight to the left. In this example, the foreground plane is a plain board with a round hole, and a projection image corresponding to viewing a picture of a dog on the background plane is displayed from the round hole.

  When a person who has seen the image displayed on the display 1 by the method of the present invention approaches the display screen and looks at the screen from the front, the projection image created with the viewpoint as the initial position approaches. Changes to a projected image created with the face position data of the person as the viewpoint. In other words, in the initial state, the foreground plane image is displayed far away and small, and most of the foreground plane image is hidden by the foreground plane image. Many parts of the scene plane image become larger and attract more viewer interest. As the person gets closer to the screen, more parts of the background plane image appear larger.

  If a person who is approaching the screen and sees the display wants to see the backplane image hidden in the foreground plane image and moves his / her face slightly to change his / her line of sight up / down / left / right, the background plane image is hidden. You can see the part you had. In this way, a new display technique that excels at the ability of the so-called eye catcher, in which people around the display screen are attracted to the display and approach the screen unintentionally. The object of the invention to provide the above can be achieved.

In carrying out the method of the present invention, the following modes can be adopted.
(1) The initial position as the viewpoint is not fixed in a period in which the face image of the person who looks at the display screen cannot be detected, but is changed regularly or irregularly.
(2) The foreground plane image in the original image data is a moving image (animation, etc.).
(3) The image of the background plane in the original image data is a moving image (animation, etc.).

Claims (5)

A display control method using a computer specified by the following items (1) to (6).
(1) The original image data includes at least a foreground plane and a foreground plane arranged at a predetermined interval in the three-dimensional coordinate system. (2) The image conversion process sets a viewpoint in front of the foreground plane. (3) The projection image data includes an image of the foreground plane from which the hidden surface is erased by the image of the foreground plane. (4) The image display process sequentially displays the projection image data sequentially generated by the image conversion process on the display screen. (5) The face recognition process captures the face of the person viewing the display screen with a camera. Creating face position data corresponding to the distance between the display screen and the face. (6) The image conversion process is a face recognition process. When you create a face position data, and set the viewpoint based on the face position data, to create a projection image data based on the viewpoint The face recognition process creates face position data including the distance between the display screen and the face and the two-dimensional position information of the face in a plane parallel to the display screen.
The method according to claim 1, wherein the viewpoint is set based on the interval distance and the two-dimensional position information in the created face position data. The method according to claim 1, wherein the image conversion process creates projection image data based on a predetermined viewpoint set in advance during a period in which the face recognition process does not create face position data.   The digital signage system which performs the method in any one of Claims 1-3 with the computer provided with the camera which image | photographs the person who watches a display screen and the said screen.   The computer program which performs the method in any one of Claims 1-3 with the computer provided with the camera which image | photographs the person who watches a display screen and the said screen.