JP2013118484A - Display direction control apparatus, display direction control method, display direction control program and video display device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a display direction control apparatus capable of reducing a throughput for detecting the position of a person from a captured image.SOLUTION: A display direction control apparatus comprises: an image capturing section 41 which captures an image obtained by imaging the front of a display screen S; a region determining section 42 which determines a region subjected to detection of the position of a person and a region not subjected to the detection regarding the captured image; a person detecting section 43 which detects the position of the person from the region subjected to the detection in the captured image; a person estimating section 44 which estimates the position of the person based on past detection results of the person detection section 43 regarding the region not subjected to the detection in the captured image; and a control section 45 which controls a direction of the display screen S based on the detected position of the person and the estimated position of the person.

Description

  The present invention relates to a display direction control device, a display direction control method, a display direction control program, and a video display device.

  As a conventional television receiver, an imaging camera or the like is used to capture an image on the front side of a video display unit that displays video, identify a facial image included in the acquired image, and identify the identified facial image. Some video display units are rotated by a rotating stand with a motor (a swivel stand) so that the screen direction faces the position (see, for example, Patent Document 1).

  In Patent Document 1, a television receiver periodically captures an image with an imaging camera, identifies a face image in the image, and sets a rotating stand so that the position of the identified face image is in the center area of the image. Control. When a plurality of face images are identified, the television receiver determines, for example, the center position of the area where the plurality of face images exist as the face image position, and the face image position is the center area of the image. Control the rotating stand to come to.

JP 2009-296521 A US Patent Application Publication No. 2002/0102024

  However, the technique described in Patent Document 1 requires a large amount of processing to identify a face image in an image.

  An object of the present invention is to provide a display direction control device, a display direction control method, a display direction control program, and a video display device that can reduce the amount of processing for detecting the position of a person from a captured image.

The display direction control device according to the present invention is:
Image acquisition means for acquiring an image obtained by imaging the front of the display screen;
A region determining means for determining a region to be detected for the position of a person and a region not to be detected for the acquired image;
Person detecting means for detecting the position of a person from the area to be detected in the acquired image;
A person estimation unit that estimates a position of a person based on a past detection result of the person detection unit for a region that is not the detection target in the acquired image;
Control means for controlling the orientation of the display screen based on the detected position of the person and the estimated position of the person;
It is characterized by providing.

Moreover, the video display device according to the present invention includes:
A display unit having a display screen for displaying video;
An imaging device that images the front of the display screen to acquire an image;
A rotating stand that rotatably supports the display unit;
A region determining means for determining a region to be detected for the position of a person and a region not to be detected for the acquired image;
Person detecting means for detecting the position of a person from the area to be detected in the acquired image;
A person estimation unit that estimates a position of a person based on a past detection result of the person detection unit for a region that is not the detection target in the acquired image;
Control means for controlling the orientation of the display screen by controlling the rotation of the display unit by the rotary stand based on the detected position of the person and the estimated position of the person;
It is characterized by providing.

Further, the display direction control method according to the present invention includes:
An image acquisition step of acquiring an image in which the front of the display screen is captured;
An area determining step for determining an area to be detected for the position of a person and an area not to be detected for the acquired image;
A person detecting step of detecting a position of a person from the area to be detected in the acquired image;
A person estimation step for estimating a position of a person based on a past detection result of the person detection step for a region not to be detected in the acquired image;
A control step for controlling the orientation of the display screen based on the detected position of the person and the estimated position of the person;
It is characterized by having.

Further, the display direction control program according to the present invention includes:
An image acquisition step of acquiring an image in which the front of the display screen is captured;
An area determining step for determining an area to be detected for the position of a person and an area not to be detected for the acquired image;
A person detecting step of detecting a position of a person from the area to be detected in the acquired image;
A person estimation step for estimating a position of a person based on a past detection result of the person detection step for a region not to be detected in the acquired image;
A control step for controlling the orientation of the display screen based on the detected position of the person and the estimated position of the person;
Is executed by a computer.

  According to the present invention, the processing amount for detecting the position of a person from a captured image can be reduced.

1 is a block diagram showing a configuration of a video display device according to Embodiment 1. FIG. 1 is a schematic front view showing a configuration of a video display device according to Embodiment 1. FIG. It is a figure for demonstrating rotation of a display screen. 1 is a block diagram illustrating a configuration of a display direction control device according to Embodiment 1. FIG. It is a figure for demonstrating determination of the rotation direction of a display screen. It is a figure for demonstrating the angle position of a display screen, and the angle position of a person. It is a figure for demonstrating the rotation angle of a display screen. It is a figure which shows the imaging range of an imaging device when a display screen exists in an initial position. It is a figure which shows the image imaged in the state of FIG. It is a figure which shows the imaging range of an imaging device after a display screen rotated only angle (theta) from the state of FIG. It is a figure which shows the image imaged in the state of FIG. 4 is a flowchart illustrating a processing procedure of the display direction control device according to the first embodiment. It is a figure for demonstrating determination of the target angular position of a display screen. It is a figure for demonstrating determination of the target angular position of a display screen. It is a figure for demonstrating determination of the target angular position of a display screen. It is a figure for demonstrating the process at the time of estimating the position of a person from the detection result of the past several times. It is a figure for demonstrating calculation of the angle position of a person.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
Embodiment 1 FIG.
First, the configuration of the video display device according to the first embodiment will be described. FIG. 1 is a block diagram illustrating a configuration of a video display apparatus 100 according to the first embodiment. FIG. 2 is a schematic front view showing the configuration of the video display apparatus 100 according to the first embodiment. The video display device 100 is a device that displays video, for example, a television receiver. The video display device 100 detects the position of a person from an image obtained by capturing the front of the display screen, and controls the orientation of the display screen based on the position of the person.

  In FIG. 1, the video display device 100 includes a display unit 10, a rotary stand 20, an imaging device 30, and a display direction control device (hereinafter referred to as “control device”) 40.

The display unit 10 includes a display screen S that displays video, and is a liquid crystal display, for example.
The rotary stand 20 is a stand that rotatably supports the display unit 10. Specifically, the rotary stand 20 supports the display screen S so as to be rotatable in a predetermined angular range in the horizontal direction. For example, as shown in FIG. 3, when the display unit 10 is viewed from above, the display screen S is rotated counterclockwise (counterclockwise, arrow X1 direction) from the initial position indicated by the solid line to a maximum rotation angle (for example, 30 °). ) And can be rotated clockwise (clockwise, arrow X2 direction) by a maximum rotation angle (for example, 30 °). In FIG. 3, the arrow direction D0 indicates the front direction of the display screen S at the initial position or the imaging direction of the imaging device 30 when the display screen S is at the initial position.

  The rotary stand 20 includes a drive unit that rotates the display screen S around a substantially vertical axis within a predetermined angle range. The drive unit is configured by, for example, a motor or a gear. The rotation of the display screen S by the drive unit (for example, the start or stop of rotation or the rotation direction) is controlled by the control device 40. The rotary stand 20 may include a rotary encoder or the like that acquires a direction in which the display screen S is facing, an angular position of the display screen S, or a rotation angle of the display screen S.

  The imaging device 30 captures the front of the display screen S to acquire an image, and outputs the acquired image (specifically, digital image data) to the control device 40. Specifically, the imaging device 30 is fixed to the display unit 10 so as to face the front of the display screen S. For example, the imaging device 30 is installed at the center in the left-right direction of the display unit 10. Acquisition of an image by the imaging device 30 and output of the captured image to the control device 40 are repeatedly performed at regular intervals, for example, 30 times per second.

  The imaging device 30 includes a CCD (Charge Coupled Device) element, a lens, and the like, and acquires, for example, a grayscale image having a horizontal resolution of 640 pixels and a vertical resolution of 480 pixels. However, the imaging device 30 may use an imaging element other than a CCD element, such as a CMOS (Complementary Metal Oxide Semiconductor) element. Further, the imaging device 30 may acquire a color image instead of a grayscale image. Further, the resolution of the image is not limited to 640 × 480 pixels, and may be higher or lower as long as a person can be detected. Note that, depending on the angle of view of the imaging device 30, generally, the higher the resolution, the farther a person can be detected.

  The control device 40 detects the position of the person from the image captured by the imaging apparatus 30 and controls the rotation of the display unit 10 by the rotary stand 20 based on the position of the person, thereby changing the orientation of the display screen S. Control. Specifically, the control device 40 detects the position of the viewer viewing the video displayed on the display screen S based on the image acquired by the imaging device 30, and displays the display screen from the position of the viewer. The display screen S is rotated by controlling the rotary stand 20 so that S faces in an easy-to-see direction. In this example, the control device 40 determines the rotation direction for rotating the display screen S according to the position of the viewer who is viewing the display screen S, and the display screen S is rotated in the determined rotation direction. The rotary stand 20 is driven.

  Next, the configuration of the display direction control device 40 of Embodiment 1 will be described. FIG. 4 is a block diagram illustrating a configuration of the display direction control device 40 according to the first embodiment.

  In FIG. 4, the control device 40 includes an image acquisition unit 41, a region determination unit 42, a person detection unit 43, a person estimation unit 44, a control unit 45, and a rotation amount determination unit 46.

  The image acquisition unit 41 acquires an image in which the front of the display screen S is captured. Specifically, the image acquisition unit 41 acquires an image captured by the imaging device 30. And the image acquisition part 41 performs a low-pass filter process in a time direction or a space direction according to the quality of the imaged image based on the resolution of the imaging device 30 and the environment in which the video display device 100 is used, Remove noise components from the image. Then, the image acquisition unit 41 outputs an image from which noise has been removed (hereinafter referred to as “captured image”) to the region determination unit 42.

  The area determination unit 42 includes, for the captured image acquired by the image acquisition unit 41, an area that is a detection target of a person position (hereinafter referred to as “detection target area”) and an area that is not a detection target of a person position ( Hereinafter, it is referred to as “non-detection target region”). Specifically, the region determination unit 42 selects a region in which the position of the person can be estimated from the past detection result of the person detection unit 43 in the captured image, or a region in which person detection has been performed in the past, as a non-detection target region. The other area is determined as the detection target area. In this example, the region determination unit 42 determines a region included in the previously acquired image (that is, the region captured last time) as a non-detection target region among the captured images, and the other region (that is, the previous image). The newly imaged area) is determined as the detection target area. Specifically, the area determination unit 42 captures a captured image acquired this time from the previous captured image capture (hereinafter referred to as “previous capture”) (hereinafter referred to as “current capture”). The area included in the previously acquired image is specified based on the rotation angle (or the rotation amount) of the display screen S until “)”, the area is determined as a non-detection target area, and the other areas are determined. Determine the detection target area. The rotation angle of the display screen S is supplied from the rotation amount determination unit 46 described later to the region determination unit 42. For example, the region determination unit 42 outputs an image of the detection target region in the captured image to the person detection unit 43 as a detection target image. Alternatively, the region determination unit 42 outputs the captured image and the determination result to the person detection unit 43.

  The person detection unit 43 detects the position of the person from the region determined as the detection target region by the region determination unit 42 in the captured image acquired by the image acquisition unit 41. For example, the person detection unit 43 receives the detection target image from the region determination unit 42 and detects the position of the person from the detection target image. Alternatively, the region determination unit 42 receives the captured image and the determination result from the region determination unit 42 and detects the position of the person from the detection target region indicated by the determination result in the captured image.

  A method for detecting a person in the person detection unit 43 is not particularly limited, and for example, a method called a viola-Jones method disclosed in Patent Document 2 or the like can be used. In summary, the Viola-Jones method sets a small area in an image while changing the position and size of the small area in the image, extracts a feature amount from the small area, and uses the discriminator to extract the small area. It is determined whether or not the area is a face. According to this Viola-Jones method, for example, the center position of a human face in a captured image can be detected, and at the same time, the size of the face in the captured image and the approximate orientation of the face can be calculated.

  In this example, the person detection unit 43 detects information such as the position of the person, the size of the face, and the orientation of the face in the captured image. In addition, when a plurality of persons are detected, the person detection unit 43 detects the position of the person, the size of the face, and the face direction in the captured image for each of the detected plurality of persons. Then, based on the detected information, the person detection unit 43 specifies the position of the person viewing the display screen S (that is, the viewer). Specifically, for each detected person, the person detection unit 43 obtains attribute information by weighting the size and orientation of the person's face, and by performing threshold processing on the attribute information, It is determined whether or not the display screen S is viewed. For example, the person detection unit 43 calculates the attribute value by weighting and adding the face size and orientation of the person with a predetermined weight, and if the attribute value is equal to or greater than a predetermined threshold, the person is a viewer. Is determined. Then, the person detection unit 43 outputs position information and attribute information indicating the position of the person to all the persons determined to be viewers to the control unit 45. Here, the position information of the person is, for example, the coordinates of the position of the person in the coordinate system of the captured image (for example, the center coordinates of the face).

  In addition, the person detection unit 43 estimates the region determined as the non-detection target region by the region determination unit 42 among the captured images acquired by the image acquisition unit 41 from the past detection results of the person detection unit 43. The position of the person to be used is used. Specifically, the person detection unit 43 receives the position information of the person estimated based on the past detection results and the attribute information of the person from the person estimation unit 44 described later, and sends them to the control unit 45. Output. The person detection unit 43 combines the detection result of the person detection unit 43 and the estimation result of the person estimation unit 44 to generate information including the position information and attribute information of the person corresponding to the captured image acquired this time. The information may be output to the control unit 45.

  In addition, the person detection unit 43 may include other information such as the person's position information and face size information, such as the vertical length of the face, the shoulder width, the distance between both eyes, the position of each eye, and the coordinates of the head vertex. The information may be detected.

  The person estimation unit 44 determines the position of the person based on the past detection result of the person detection unit 43 for the region determined as the non-detection target region by the region determination unit 42 among the captured images acquired by the image acquisition unit 41. Is estimated. Specifically, the person estimation unit 44 stores the detection result of the person detection unit 43 and estimates the position of the person based on the past detection result of the person detection unit 43 and the rotation history of the display screen S. To do. More specifically, the person estimation unit 44 captures the current captured image based on the past detection result of the person detection unit 43 and the rotation angle of the display screen S from the past capturing time to the current capturing time. The position of the person detected in the past in is estimated. In this example, the person estimation unit 44 acquires the rotation angle of the display screen S from the previous imaging time to the current imaging time from the rotation amount determination unit 46 described later, and acquires the position of the person detected last time and the acquisition. Based on the rotation angle thus determined, the position of the person detected last time in the current captured image is estimated. That is, the person estimation unit 44 estimates where the position of the person detected last time has moved in the current captured image. The person estimation unit 44 then outputs the estimated position information of the person and the attribute information of the person to the person detection unit 43. As the person attribute information, for example, attribute information included in past detection results is used.

  The control unit 45 controls the orientation of the display screen S based on the position of the person detected by the person detection unit 43 and the position of the person estimated by the person estimation unit 44. Specifically, the control unit 45 controls the orientation of the display screen S by controlling the rotation of the display unit 10 by the rotary stand 20 based on the position of the person. In the example of FIG. 4, the control unit 45 includes a direction determination unit 45a and a drive control unit 45b.

  The direction determination unit 45 a determines the rotation direction of the display screen S based on the position information and attribute information of the person from the person detection unit 43. Specifically, the direction determination unit 45a determines the rotation direction to the left if the position of the person is on the left side of the front of the display screen S, determines the right direction if the person is on the right, and does not rotate if the position is the front. And decide. For example, as shown in FIG. 5, the area of the captured image is divided into three areas A1, A2, and A3, and the direction determining unit 45a determines whether the position of the person exists in any of the areas A1 to A3. Determine the direction of rotation. Specifically, the direction determination unit 45a determines the left direction if the position of the person is included in the area A1, determines the right direction if it is included in the area A3, and determines not to rotate if the position is included in the area A2. .

  In addition, when the position determination unit 45a receives the position information and attribute information of a plurality of persons, that is, when a plurality of persons are detected or estimated as viewers, A position near the middle of the person is obtained, and the rotation direction is determined based on the position. Specifically, the direction determination unit 45a weights and averages the position of each person using the attribute information of each person as a weight, and determines the rotation direction of the display screen S based on the obtained position.

  The drive control unit 45b drives and controls the rotation of the rotary stand 20 so that the display screen S rotates in the rotation direction determined by the direction determination unit 45a. For example, in the top view of the display unit 10 in FIG. 3, the drive control unit 45b rotates the display screen S counterclockwise (in the direction of the arrow X1 in FIG. 3) when the direction determination unit 45a determines the left direction. If the direction determining unit 45a determines the right direction, the display screen S is rotated clockwise (in the direction of the arrow X2 in FIG. 3).

  In one aspect, the drive control unit 45b rotates the display screen S in the determined rotation direction by a predetermined angle (for example, a minute angle that is a few tenths of the maximum rotation angle of the display screen S). . In this aspect, for example, the process of rotating the display screen S by a predetermined angle is repeated until the position of the person enters the area A2 in FIG. 5, thereby rotating the display screen S to face the person. However, when the drive control unit 45 receives an instruction for the rotation direction from the direction determination unit 45a, the drive control unit 45 may continue to rotate the display screen S in the specified rotation direction until an instruction for the next rotation direction is received.

  When the imaging device 30 captures an image, or when the image acquisition unit 41 acquires an image, the rotation amount determination unit 46 rotates the display screen S from the previous imaging time to the current imaging time ( Or the rotation amount) is obtained and output to the region determination unit 42 and the person estimation unit 44.

  Specifically, every time imaging is performed by the imaging device 30, the drive control unit 45 b outputs the angular position (or rotational position) of the display screen S at the time of imaging to the rotation amount determination unit 46 to determine the rotation amount. The unit 46 calculates the rotation angle of the display screen S by subtracting the angular position of the display screen S at the previous imaging from the angular position of the display screen S at the current imaging.

  Referring to FIG. 6, the angular position of the display screen S at a certain point in time depends on the angle θs of the front direction Ds of the display screen S at that point in time with respect to a predetermined direction (for example, the front direction of the display screen S at the initial position). expressed. Then, as shown in FIG. 7, the rotation amount determination unit 46 subtracts the angular position θs1 of the display screen S at the previous imaging from the angular position θs2 of the display screen S at the time of the current imaging, thereby The rotation angle θ (= θs2−θs1) is calculated. The angular position of the display screen S may be measured by a rotary encoder of the rotary stand 20 and sent to the drive control unit 45b, or calculated by the drive control unit 45b based on a drive control signal to the rotary stand 20. May be. In FIG. 7, an arrow Ds (current) indicates the front direction of the display screen S at the time of current imaging, and an arrow Ds (previous) indicates the front direction of the display screen S at the time of previous imaging.

  Note that the angular position θs and the rotation angle θ of the display screen S are positive in the counterclockwise direction and negative in the clockwise direction. The same applies to other angles thereafter.

Next, an example of the operation of the video display apparatus 100 according to Embodiment 1 will be described.
FIG. 8 is a diagram illustrating an imaging range of the imaging device 30 when the display screen S is in the initial position when the video display device 100 is viewed from above. FIG. 8 shows an imaging surface 31, an angle of view α, a focal length f, a right end direction A, and a left end direction B of the image pickup apparatus 30. In FIG. 8, a person P exists on the left side in the imaging range.

  Assume that the video display apparatus 100 is activated in the state shown in FIG. First, the video display apparatus 100 captures an area between the directions A and B by the imaging apparatus 30, and performs noise reduction processing on the captured image by the image acquisition unit 41 to obtain, for example, a captured image as illustrated in FIG. Here, the region determination unit 42 determines all regions of the captured image as detection target regions because it is in the initial state. The person detection unit 43 detects a face image for all areas of the captured image, weights the face size and orientation information of the person P to obtain attribute information, and the person P views the attribute P from the attribute information. The position information and attribute information of the person P are output to the control unit 45.

  The direction determination unit 45a determines the rotation direction of the rotary stand 20 from the position information of the person P. Here, the direction determination unit 45a determines that the position of the person P is included in the area A1 in FIG. 5, determines the rotation direction to the left (direction of arrow X1 in FIG. 3), and drives the rotation in the left direction. The control unit 45b is instructed.

  The drive control unit 45b drives the rotary stand 20 according to an instruction from the direction determination unit 45a to rotate the display screen S leftward by a predetermined angle θ. Thereby, in the top view of the video display device 100, the imaging range of the imaging device 30 is as shown in FIG. FIG. 10 shows the imaging surface 31 of the imaging device 30, the right end direction A ′ of the imaging range, the left end direction B ′, and the points a ′ and b ′ on the imaging surface 31 corresponding to the directions A ′ and B ′. Has been. Furthermore, directions A and B at the time of previous imaging are indicated by broken lines, and a point b on the imaging surface 31 corresponding to direction B is indicated. In FIG. 10, the person P exists substantially in front of the imaging range, and the person Q exists on the left side of the imaging range. The position of the person P in FIG. 10 is the same as the position of the person P in FIG. That is, the person P is not moving.

  In the state shown in FIG. 10, the next process is started with a certain time elapsed since the previous imaging. First, an area between the directions A ′ and B ′ is imaged by the imaging device 30, and noise reduction processing is performed on the captured image to obtain a captured image as illustrated in FIG. The area determination unit 42 classifies the captured image into a detection target area and a non-detection target area. Specifically, the region determining unit 42 receives the rotation angle θ of the display screen S from the previous imaging time to the current imaging time from the rotation amount determination unit 46, and based on the rotation angle θ, the current captured image The position at the left end (position BI in FIG. 11) of the previous imaging range is calculated, and the detection target area and the non-detection target area are classified based on the position.

Specifically, in FIG. 10, the length of the line segment a′b ′ is 2 · f · tan (α / 2), and the length of the line segment a′b is f · tan (α / 2). ) + F · tan (α / 2−θ). Therefore, the ratio (a′b / a′b ′) between the length of the line segment a′b and the length of the line segment a′b ′ is expressed by the following formula (1).
a′b / a′b ′ = {tan (α / 2) + tan (α / 2−θ)} / {2 · tan (α / 2)} (1)
Using the above equation (1), the region determination unit 42 can obtain the position of the point b on the imaging surface 31 corresponding to the left end direction B at the previous imaging, and further, the point b in the current captured image (Ie, the leftmost position of the previous imaging range) BI can be obtained.

  For example, if the horizontal width of the captured image is w, in the captured image of FIG. 11, the position BI corresponding to the point b is the distance w · {tan (α) from the position AI ′ corresponding to the point a ′ in the horizontal direction. / 2) + tan (α / 2−θ)} / {2 · tan (α / 2)}.

  In the captured image shown in FIG. 11, a rectangular area R1 whose horizontal position is from position AI 'to position BI is an area within the imaging range in the previous process, and the previous person detection process has been performed. Therefore, in the current process, the region determination unit 42 classifies the rectangular region R1 as a non-detection target region. On the other hand, the rectangular area R2 from the position BI 'to the position BI whose horizontal position corresponds to the point b' is an area outside the imaging range in the previous process, and the previous person detection process has not been performed. Therefore, in the current process, the area determination unit 42 classifies the rectangular area R2 as a detection target area.

  The person estimation unit 44 performs geometric correction on the position information of the person P in the previous detection result according to the rotation angle θ of the display screen S, and obtains the position information of the person P in the current captured image. Thereby, the position of a person is estimated about rectangular area R1 (non-detection object area | region) among the captured images of FIG. The person estimation unit 44 outputs the position information of the person P obtained this time and the attribute information of the person P included in the previous detection result to the person detection unit 43.

  The person detection unit 43 performs face image detection processing on the rectangular area R2 (detection target area) in the captured image of FIG. 11 to detect the position, size, and orientation of the face of the person Q, and the face of the person Q Attribute information is obtained by weighting the size and orientation information, and it is determined from the attribute information that the person Q is a viewer, and the position information and attribute information of the person Q are obtained. Then, the person detection unit 43 outputs the position information and attribute information of the person Q and the position information and attribute information of the person P from the person estimation unit 44 to the direction determination unit 45a.

  The direction determining unit 45a weights and averages the position of the person P and the position of the person Q using the attribute information of the person P and the attribute information of the person Q as weights, and the rotation direction of the display screen S based on the obtained position. To decide. Here, a position near the middle between the person P and the person Q is obtained by the weighted average, and the direction determining unit 45a determines that the position of the person is included in the area A1 in FIG. 5 and sets the rotation direction to the left (FIG. 3). And instruct the drive control unit 45b to rotate leftward. In response to this instruction, the drive control unit 45b controls the rotary stand 20 so that the display screen S rotates leftward.

  Next, an example of a processing procedure of the display direction control device 40 will be shown. FIG. 12 is a flowchart illustrating a processing procedure of the display direction control device 40 in the video display device 100 according to the first embodiment. For example, the processing is started when the video display device 100 is activated (or powered on), and thereafter repeatedly executed at regular intervals, and is ended when the video display device 100 is powered off. Alternatively, the processing is triggered by the occurrence of a specific event such as activation of the video display device 100 or detection of an operation on the video display device 100 (for example, operation of a remote control key). This process is repeatedly executed at regular intervals until the viewer faces the viewer (specifically, until the viewer position enters the area A2 in FIG. 5).

The control device 40 controls the imaging device 30 to acquire a captured image (S11).
Next, the control device 40 determines a detection target region and a non-detection target region in the captured image based on the previous rotation result of the display screen S (S12). Specifically, the control device 40, based on the rotation angle of the display screen S in the previous process, of the current captured image, the area captured last time, and the area newly captured this time relative to the previous time Is determined, the previously imaged area is determined as the non-detection target area, and the newly imaged area is determined as the detection target area.

  Next, the control device 40 corrects the position of the person in the previous person detection result based on the rotation angle of the display screen S in the previous process for the non-detection target region in the captured image, and the current captured image The position of the person detected last time is calculated to obtain position information of the person (S13).

  Further, the control device 40 detects the position of the person from the detection target area in the captured image (S14). Specifically, the control device 40 detects the position, size, and orientation of the person's face, calculates the attribute information of the person from these, determines whether the viewer is a viewer, and is determined to be a viewer Obtain person position information and attribute information.

  Next, the control device 40 combines the estimation result of step S13 and the detection result of step S14 to generate information including the position information and attribute information of the person corresponding to the current captured image (S15).

  Next, the control device 40 determines a rotation direction for rotating the display screen S based on the position information and attribute information of the person included in the information generated in step S15 (S16).

  Next, the control device 40 determines whether or not it is determined not to rotate in step S16 (S17). If it is determined that it is determined not to rotate (S17: YES), the control device 40 controls the rotating stand 20 so as not to rotate the display screen S (S18), and ends the process. On the other hand, when it is determined that the rotation is not determined (S17: NO), the control device 40 determines whether the right direction is determined in step S16 (S19). If it is determined that the right direction is determined (S19: YES), the control device 40 controls the rotary stand 20 to rotate the display screen S in the right direction (S20), and ends the process. On the other hand, when it is determined that the right direction has not been determined (S19: NO), the control device 40 controls the rotary stand 20 to rotate the display screen S in the left direction (S21), and ends the process. .

According to the first embodiment described above, the following effects (1) to (5) can be obtained.
(1) In the present embodiment, the display direction control device determines a detection target region and a non-detection target region for a captured image obtained by capturing the front of the display screen, and detects a person's position in the detection target region. The position of the person is estimated based on the past detection result, and the orientation of the display screen is controlled based on the detected person position and the estimated person position. That is, in a configuration in which a person is detected from a captured image obtained by capturing the front of the display screen and the orientation of the display screen is controlled based on the position of the person, a detection target region and a non-detection target region are determined from the captured image, In the captured image, the position of the person is detected for the detection target area, and the position of the person is estimated based on the past detection result for the non-detection target area. For this reason, according to this Embodiment, compared with the case where the position of a person is detected with respect to the whole area | region of a captured image, the processing amount for detecting the position of a person can be reduced.

  For example, it is possible to reduce the amount of calculation required for person detection per captured image, and to shorten the time required for person detection. As a result, it is possible to shorten the time required from when the image is taken to when the display screen S is rotated, and to realize an agile operation. Specifically, in a configuration in which the detection process takes a long time, the delay time from imaging to the start of rotation control of the rotary stand is large, especially when processing is performed while the display screen is rotating, following the rotation control of the display screen. There is a problem that the display screen is not accurately directed to the position of the person or the position of the display screen is not immediately stabilized. On the other hand, according to the present embodiment, it is possible to reduce the detection processing time, thereby reducing the delay time from imaging to the start of display screen rotation control. It is possible to improve the followability, stability, or accuracy of rotation control when processing is performed.

  In addition, the number of processes per unit time can be increased, and the accuracy of person position detection can be improved. In addition, power consumption for one process can be reduced. Furthermore, it becomes possible to configure the display direction control device using an inexpensive CPU having a low processing capacity, and the cost can be reduced.

  (2) The display direction control device determines an area in which the position of the person can be estimated from a past detection result of the person detection unit as a non-detection target area from the acquired captured image, and detects other areas as detection targets Decide on an area. According to this aspect, the position of a person can be obtained by detecting or estimating the entire captured image.

  (3) The display direction control device determines an area included in the previously acquired captured image (that is, the previously captured area) as a non-detection target area from the acquired captured images, and detects other areas. Determine the target area. According to this aspect, it is possible to easily determine the region.

  (4) The display direction control device determines the detection target region and the non-detection target region based on the rotation angle of the display screen from when the captured image acquired last time is captured to when the captured image acquired this time is captured. To do. According to this aspect, the region can be easily determined using the rotation angle. Specifically, based on the rotation angle of the display screen from the previous imaging to the current imaging, an area included in the previously acquired captured image among the captured images acquired this time (that is, the previously captured area) Can be easily identified, and the region can be easily determined.

  (5) The display direction control device estimates the position of the person based on the past detection result of the person detection unit and the rotation history of the display screen. For example, the position of the person is estimated using the previous detection result of the person detection unit and the angle at which the display screen is actually rotated (specifically, the rotation angle from the previous imaging to the current imaging). According to this aspect, the position of the person can be easily estimated.

  In addition, the display direction control apparatus according to Embodiment 1 may be configured as shown in the following (a) to (g).

  (A) In the above description, the configuration in which the rotation direction of the display screen S is determined from the position of the person is exemplified, but the control unit 45 may determine the target direction that the display screen S should face from the position of the person. . Specifically, the direction determination unit 45a may determine the target direction of the display screen S based on the position information and attribute information of the person from the person detection unit 43.

  In this case, for example, as shown in FIG. 13, the direction determining unit 45a obtains a horizontal distance Lv between the horizontal center position Pc and the person position Pv in the captured image, and calculates an angle θt corresponding to the distance Lv. As shown in FIG. 14, the target angle position θst (= θs + θt) of the display screen S is determined by adding the angle θt to the angle position θs of the display screen S at the time of the current imaging. In FIG. 14, an arrow Dv indicates the direction of the person, and an arrow Ds (current time) indicates the front direction of the display screen S at the time of current imaging.

  Alternatively, the direction determining unit 25 identifies which region the person's position belongs to among the M (A is an integer of 2 or more) regions A1 to AM obtained by dividing the captured image, as shown in FIG. A target angular position θt is determined based on rotation angles θt1 to θtM preset for the specified region. For example, when the position of the person exists in the area Am (1 ≦ m ≦ M), the rotation angle θtm set in the area Am is added to the angle position θs of the display screen S, and the target angle position θst (= θs + θtm) is determined.

  The drive control unit 45b controls the rotation of the rotary stand 20 so that the display screen S faces the target direction determined by the direction determination unit 45a. Specifically, the drive control unit 45b controls the rotation of the rotary stand 20 so that the angular position of the display screen S matches the target angular position θst.

  In the above configuration, the direction determination unit 45a may determine the angle θt as the target direction of the display screen S instead of the angular position θst (= θs + θt) and output the angle θt to the drive control unit 45b. In this case, the drive control unit 45b controls the rotation of the rotary stand 20 so that the display screen S is rotated by the rotation angle θt from the current imaging position.

  Further, in the above configuration, the processing of the control device 40 may be executed at regular intervals, or may be executed every time a specific event occurs, such as detection of an operation of a remote control key. During the rotation of the display screen S, the start of the next process may be allowed or prohibited. That is, the control device 40 may update the target direction by performing the next imaging while the display screen S is rotating before the display screen S faces the target direction, or until the display screen S faces the target direction. May start the next process after the display screen S faces the target direction without starting the next process.

  (B) In the above description, the configuration in which the position of the person is estimated from the previous detection result is exemplified, but the person estimation unit 44 may estimate the position of the person using the detection result before the previous time. . For example, the person estimation unit 44 may estimate the position of the person from the detection results in the period from the present to a predetermined time, or estimate the position of the person from the latest N detection results (N ≧ 2). May be.

  With reference to FIG. 16, the process at the time of estimating a person's position from the past detection results will be described. FIG. 16 shows a case where three times of imaging are sequentially performed, and the first, second, and third imaging ranges PA1, PA2, and PA3 are shown. In the current (third time) imaging range PA3, the person a is detected this time for the area a3 that has not been detected in the past, and the previous time (second time) for the area a2 that was detected last time. The position of the person is estimated based on the detection result and the rotation angle θ2 of the display screen S from the previous time to the current time, and the detection result of the previous time and the previous time for the area a1 detected in the previous time (first time). To the current position based on the rotation angle (θ1 + θ2) of the display screen S.

  (C) The person estimation unit 44 may accumulate past detection results of the person's position, and may estimate the person's position using the accumulated past detection results. In this case, the person estimation unit 44 may clear the accumulated detection results at a predetermined timing (for example, periodically). Thereby, accumulation of errors in estimating the position of the person can be reset or avoided, and the accuracy of estimating the position of the person can be kept high. The clearing interval is performed, for example, every 1 to several seconds, but may be more or less than that.

  (D) The region determination unit 42 may determine the entire region of the captured image as a detection target region at a predetermined timing (for example, periodically). According to this aspect, it is possible to reset or avoid accumulation of errors in estimating the position of a person, and it is possible to maintain high accuracy in estimating the position of a person. The interval of execution of the processing is performed, for example, every 1 to several seconds, but may be longer or shorter than that.

  (E) In the above description, the configuration using the coordinates of the person in the coordinate system of the captured image as the position of the person is illustrated. However, as shown in FIG. 6, the front of the display screen S at the time of imaging is used as the position of the person. The person's angular position θvr with respect to the direction Ds may be used. This angular position θvr can be said to be a relative angle position of a person based on the orientation of the display screen S at the time of imaging. In FIG. 6, an arrow Dv indicates the direction of the person.

  In this aspect, the person detection unit 43 detects the position of the person in the captured image, obtains a horizontal shift amount of the person position from the image center, and converts the shift amount into the angular position θvr.

  The direction determination unit 45a determines the rotation direction of the display screen S based on the detected angular position θvr of the person. Alternatively, the direction determination unit 45a determines the detected angle position θvr of the person as the target rotation angle θt of the display screen S as it is.

  As shown in FIG. 17, the person estimation unit 44 subtracts the rotation angle θ of the display screen S from the previous imaging time to the current imaging time from the previously detected person's angular position θvr1, thereby The angle position θvr2 (= θvr1-θ) of the person with respect to the front direction of the display screen S at the time of imaging is calculated. In FIG. 17, the arrow Dv indicates the direction of the person, the arrow Ds (current) indicates the front direction of the display screen S at the time of current imaging, and the arrow Ds (previous) indicates the direction of the display screen S at the previous imaging. The front direction is shown.

  (F) Also, as shown in FIG. 6, as the position of the person, the angle position θv of the person with respect to a predetermined direction (for example, the front direction of the display screen S at the initial position) D0 may be used. This angular position θv can be said to be an absolute person's angular position independent of the rotation of the display screen S.

  In this aspect, the person detection unit 43 calculates the angle position θvr of the person in the same manner as in the above (e), and adds the angle position θvr to the angle position θs of the display screen S at the time of imaging to obtain the angle position of the person. θv (= θs + θvr) is obtained.

  The direction determination unit 45a determines the rotation direction of the display screen S by comparing the angle position θv of the person detected by the person detection unit 43 with the angle position θs of the display screen S at the time of the current imaging. Alternatively, the direction determination unit 45a determines the detected angular position θv of the person as the target angular position θst of the display screen S as it is.

  The person estimation unit 44 estimates the angle position θv of the person detected in the past as it is as the angle position of the current person.

  (G) In the above description, the configuration for determining the non-detection target region and the detection target region based on the rotation angle of the display screen S from the previous imaging to the current imaging is illustrated. Is based on a predetermined direction (for example, the front direction of the display screen S at the initial position) D0 as a reference, and the angle range of the past captured image corresponding to the past detection result used for the estimation of the person position (that is, the person position is (An estimated angle range) may be stored, and a region corresponding to the angle range in the current captured image may be determined as a non-detection target region.

Embodiment 2. FIG.
Hereinafter, the video display apparatus according to the second embodiment will be described. The video display device according to the second embodiment differs from the video display device according to the first embodiment in the estimation of a person, and the other portions are substantially the same. In the following description, the description of the same parts as those in the first embodiment is omitted or simplified, and the same or corresponding elements as those in the first embodiment are denoted by the same reference numerals.

  In the present embodiment, when estimating the position of the person based on the past detection result for the non-detection target region, the person estimation unit 44 and the previously acquired image corresponding to the past detection result and the currently acquired imaging The movement of the position of the person in the past detection result with respect to the image is detected, and the position of the person is estimated based on the result. That is, the person estimation unit 44 performs a motion detection process on the position of the person for the non-detection target region in the captured image, and estimates the person position based on the result. The motion detection process is executed by, for example, a matching operation between a block image and a captured image.

  Specifically, when estimating the position of the person based on the past detection result for the non-detection target region, the person estimation unit 44 captures a past captured image corresponding to the past detection result from the non-detection target region. Among the above detection results, a part that matches or is similar to a part including the position of the person (or a part in the vicinity of the position) in the past detection result is detected, and the position of the detected part is estimated as the position of the person.

  More specifically, the person estimation unit 44 stores the entire past captured image corresponding to the past detection result in addition to the past person position detection result. Then, when estimating the position of the person from the past detection result, the person estimation unit 44, based on the past captured image corresponding to the detection result, the block including the position of the person in the detection result (or the vicinity of the position) Block), a block having the highest similarity to the block is searched from the non-detection target area of the captured image acquired this time, and the position of the searched block is estimated as the position of the person.

For example, the person estimation unit 44 cuts out a block image of a predetermined size (here, 128 × 128 pixels) centering on the position of the person detected last time from the previous captured image. In addition, as in the first embodiment, the person estimation unit 44 adds conversion based on the rotation angle of the previous display screen S to the previous detection result, and detects the person detected last time in the current captured image. Estimate the position. Then, the person estimation unit 44 cuts out the block image of the predetermined size (here, 128 × 128 pixels) centered on the estimated person position from the current captured image, and is cut out from the previous captured image. and calculates the average value E / 128 ² of the error of the sum E of the pixel values of each pixel of the block image. Next, the person estimation unit 44 is separated from the estimated person position by m pixels in the horizontal direction and n pixels in the vertical direction (m and n are integers satisfying −64 ≦ m, n ≦ 64, for example). The same calculation is performed on the block image of the predetermined size (here, 128 × 128 pixels) centered on the position, and among the obtained average value of the sum of the (m + 1) × (n + 1) errors, The values (M, N) of m and n corresponding to the smallest one are obtained. Then, the person estimation unit 44 sets a position shifted by M pixels in the horizontal direction and N pixels in the vertical direction from the estimated person position as a new person estimated position.

  At this time, the person estimation unit 44 may determine that the target person does not already exist in the captured image when the minimum value of the average value of the sum of errors is larger than a predetermined threshold.

  As for the attribute information, the person estimation unit 44 uses, for example, the attribute information obtained last time without change.

  Note that the block size and the search range (m and n value ranges) described above are examples, and may be set to appropriate values according to, for example, the resolution of the imaging device 30 and the imaging interval.

  A part of the processing by the person estimation unit 44 may be executed by the person detection unit 43.

  In this case, for example, the person estimation unit 44 estimates the position of the person in the current captured image based on the position of the person detected last time and the rotation angle of the display screen S, as in the first embodiment. The estimated person position is output to the person detection unit 43. Furthermore, the person estimation unit 44 stores the entire previous captured image, cuts out a 128 × 128 pixel block centered on the position of the person detected last time from the previous captured image, and extracts the person detection unit. Output to 43.

The person detection unit 43 receives, from the region determination unit 42, captured images of the entire region including not only the detection target region but also the non-detection target region. Then, the person detection unit 43 detects the position of the person in the captured image from the region determination unit 42 as in the first embodiment for the detection target region, and performs motion detection processing for the non-detection target region. And the position of the person is estimated based on the result. Specifically, the person detection unit 43 cuts out a block of 128 × 128 pixels centered on the position of the person estimated by the person estimation unit 44 from the current captured image, and extracts the block from the person estimation unit 44. It calculates an average value E / 128 ² of the error of the sum E of the pixel values for each pixel. Next, the person detection unit 43 also applies a 128 × 128 pixel block centered at a position separated by m pixels in the horizontal direction and n pixels in the vertical direction from the position of the person estimated by the person estimation unit 44. The same calculation is performed, and the values (M, N) of m and n corresponding to the smallest one among the average values of the total sum of the (m + 1) × (n + 1) errors obtained are obtained. The person detection unit 43 sets a position shifted by M pixels in the horizontal direction and N pixels in the vertical direction from the position of the person estimated by the person estimation unit 44 as a new person estimated position.

According to the second embodiment described above, the following effect (6) can be obtained in addition to the effects (1) to (5).
(6) When estimating the position of the person based on the past detection result for the non-detection target region, the display direction control device determines whether the captured image acquired between the past and the current captured image corresponding to the past detection result. The movement of the position of the person in the past detection result is detected, and the position of the person is estimated based on the result. For this reason, according to the present embodiment, the position of the person can be estimated in consideration of the movement of the person detected in the past, and the accuracy of the estimation of the person position can be improved.

  In the first and second embodiments described above, the function of the display direction control device or the video display device may be realized only by hardware resources such as an electronic circuit, or by cooperation of hardware resources and software. May be. When realized by the cooperation of hardware resources and software, the functions of the display direction control device or the video display device are realized by, for example, a computer program being executed by a computer. More specifically, the function of the display direction control device or the video display device is as follows. A computer program recorded in a recording medium such as a ROM (Read Only Memory) is read out to a main storage device and a central processing unit (CPU: Central). It is realized by being executed by Processing Unit). The computer program may be provided by being recorded on a computer-readable recording medium such as an optical disk, or may be provided via a communication line such as the Internet.

  In addition, this invention is not limited to the said embodiment, In the range which does not deviate from the summary of this invention, it can implement with a various aspect.

  DESCRIPTION OF SYMBOLS 10 Display part, 20 rotation stand, 30 imaging device, 31 imaging surface, 40 display direction control apparatus, 41 image acquisition part, 42 area | region determination part, 43 person detection part, 44 person estimation part, 45 control part, 45a direction determination part 45b drive control unit, 46 rotation amount determination unit, 100 video display device, S display screen.

Claims (12)

Image acquisition means for acquiring an image obtained by imaging the front of the display screen;
A region determining means for determining a region to be detected for the position of a person and a region not to be detected for the acquired image;
Person detecting means for detecting the position of a person from the area to be detected in the acquired image;
A person estimation unit that estimates a position of a person based on a past detection result of the person detection unit for a region that is not the detection target in the acquired image;
Control means for controlling the orientation of the display screen based on the detected position of the person and the estimated position of the person;
A display direction control device comprising: The area determination means determines an area in which the position of a person can be estimated from the past detection results of the person detection means as an area not to be detected from the acquired image, and detects the other areas. Decide on the target area,
The display direction control apparatus according to claim 1. The region determining means determines, in the acquired image, a region included in the previously acquired image as a region that is not the detection target, and determines other regions as the detection target region,
The display direction control device according to claim 1, wherein the display direction control device is a display direction control device. The region determination means determines the region to be detected and the region not to be detected based on the rotation angle of the display screen from the time of capturing the previously acquired image to the time of capturing the image acquired this time. decide,
The display direction control device according to claim 1, wherein the display direction control device is a display direction control device. The person estimation means estimates the position of the person based on a past detection result of the person detection means and a history of rotation of the display screen;
The display direction control apparatus according to claim 1, wherein the display direction control apparatus is a display direction control apparatus. The person estimation means accumulates past detection results of the person detection means, estimates the position of the person based on the accumulated detection results, and clears the accumulated detection results at a predetermined timing. ,
The display direction control apparatus according to claim 1, wherein the display direction control apparatus is a display direction control apparatus. The region determination means determines all regions of the acquired image as the detection target region at a predetermined timing.
The display direction control apparatus according to claim 1, wherein the display direction control apparatus is a display direction control apparatus. When estimating the position of the person based on the past detection result for the region that is not the detection target, the person estimation unit between the past image corresponding to the past detection result and the image acquired this time Detecting the movement of the position of the person in the past detection result, and estimating the position of the person based on the result,
The display direction control apparatus according to claim 1, wherein the display direction control apparatus is a display direction control apparatus. When estimating the position of the person based on the past detection result for the area that is not the detection target, the person estimation unit includes a past image corresponding to the past detection result from the area that is not the detection target. Detecting a portion that matches or is similar to the portion including the position of the person in the past detection result, and estimates the position of the detected portion as the position of the person;
The display direction control apparatus according to claim 1, wherein the display direction control apparatus is a display direction control apparatus. A display unit having a display screen for displaying video;
An imaging device that images the front of the display screen to acquire an image;
A rotating stand that rotatably supports the display unit;
A region determining means for determining a region to be detected for the position of a person and a region not to be detected for the acquired image;
Person detecting means for detecting the position of a person from the area to be detected in the acquired image;
A person estimation unit that estimates a position of a person based on a past detection result of the person detection unit for a region that is not the detection target in the acquired image;
Control means for controlling the orientation of the display screen by controlling the rotation of the display unit by the rotary stand based on the detected position of the person and the estimated position of the person;
A video display device comprising: An image acquisition step of acquiring an image in which the front of the display screen is captured;
An area determining step for determining an area to be detected for the position of a person and an area not to be detected for the acquired image;
A person detecting step of detecting a position of a person from the area to be detected in the acquired image;
A person estimation step for estimating a position of a person based on a past detection result of the person detection step for a region not to be detected in the acquired image;
A control step for controlling the orientation of the display screen based on the detected position of the person and the estimated position of the person;
A display direction control method characterized by comprising: An image acquisition step of acquiring an image in which the front of the display screen is captured;
An area determining step for determining an area to be detected for the position of a person and an area not to be detected for the acquired image;
A person detecting step of detecting a position of a person from the area to be detected in the acquired image;
A person estimation step for estimating a position of a person based on a past detection result of the person detection step for a region not to be detected in the acquired image;
A control step for controlling the orientation of the display screen based on the detected position of the person and the estimated position of the person;
A display direction control program for causing a computer to execute the above.

Images