JP2011033993A - Information presenting apparatus and method for presenting information - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an information presenting apparatus presenting information to a large indefinite number of users by using an information presenting unit that can present information only to the finite number of users at a time, wherein the apparatus can present information to more users. <P>SOLUTION: The information presenting apparatus presents information to the large indefinite number of users by using an information presenting unit that can present information only to the finite number of users at a time, wherein the apparatus includes: a user condition-detecting unit that detects a user's condition; and a user selecting unit that sequentially changes users as objects to which the information presenting unit presents the information on the basis of the detection result of the user condition-detecting unit. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an information presentation device and an information presentation method.

  So-called digital signage, which presents information using an electronic display, has attracted attention as an advertising means in streets and stores. It is important for advertisements to attract many people's eyes and leave an impression. One way to do this is to combine video presentation with a sense of depth and depth with digital signage.

  There are several specific methods for presenting a video with a sense of depth and depth, and one of them is a method for presenting a video with a sense of depth and depth by reproducing motion parallax. Motion parallax is one of the clues of depth perception for a human to recognize a three-dimensional space from a two-dimensional retina image. When a human changes the position and orientation of the head, the images that are visible according to the change are different. For example, FIG. 44 shows a state where the user 1001 is viewing a situation in which a cube 1002 is placed on the lower side in front and a sphere 1003 is placed on the upper side in the back. When the user 1001 is at the center position, the user 1001 seems to have a sphere 1003 at the center upper part of the field of view and a cube 1002 at the center lower part as shown in FIG. At this time, the cube 1002 has its front and top surfaces visible.

  Next, in FIG. 44, when the user 1001 translates from the center to the position A, both the sphere 1003 and the cube 1002 appear to move to the right of the field of view as shown in FIG. 1002 has a larger movement amount. In addition to the front and top surfaces of the cube 1002, the left side surface can be newly seen. On the other hand, when the user 1001 is translated from the center to the position B in FIG. 44, both the sphere 1003 and the cube 1002 appear to move to the left of the field of view as shown in FIG. 47, and the cube 1002 moves. The amount is large. In addition to the front and top surfaces of the cube 1002, the right side surface can be newly seen.

  From here, the user 1001 can recognize that the cube 1002 having a larger amount of movement in the visual field is in front of the sphere 1003, and recognize the depth structure of the cube 1002 from the change in the appearance of the side of the cube 1002. can do. In this way, humans can judge by integrating information on changes in the position and orientation of the head due to their own movement and the difference in appearance at that time, and can perceive a three-dimensional spatial structure, that is, a three-dimensional feeling and a sense of depth. it can. By using this perceptual mechanism, the image output to the display is appropriately changed according to the change in the position of the human head, so that it looks as if it is a three-dimensional image while it is a two-dimensional image. It is possible to perceive.

  There are several methods for reproducing motion parallax and presenting images. One of them is a method of measuring a human viewpoint position using some sensor, and generating and presenting an image so as to reproduce an image seen from the viewpoint position. For example, in Patent Literature 1, a human head position is detected from an image acquired by a camera installed on the upper part of a display device, and an image corresponding to the relative relationship between the display device and the head position is output, so that motion parallax is reduced. The reproduced video is presented.

  Digital signage is often installed in places where many people come and go, such as stations, busy streets, department stores, and storefronts. In such a place where people are moving in many cases, that is, where it is considered that there is a lot of human head movement, presentation of images with a three-dimensional feeling and depth feeling by reproducing such motion parallax is more eye-catching. , And has the advantage of becoming a more impressive advertisement. In the method for reproducing motion parallax, as described above, a portion that cannot be seen from the front can be seen by moving the head. Therefore, for example, when this method is used for advertising a certain product, it is possible to see the appearance of the product from various directions, and there is an advantage that the product image can be conveyed to a person more specifically.

  Patent Document 2 discloses a plurality of sensors for detecting a user, a modulation unit that modulates an audio signal output from an audio reproduction unit and generates a signal emitted as an ultrasonic wave from a superdirectional speaker, A support base that changes the direction of the superdirective speaker, and a control unit that outputs the audio signal from the sound reproduction unit according to the detection signal of the sensor and controls the support base to change the direction of the superdirective speaker. Thus, a voice loudspeaker is described that can detect a user who has moved to follow the loudspeak range of the voice and can be easily installed.

  Further, in Patent Document 3, when there are a plurality of users who view a display main body, a camera and a microphone provided on the front surface of the display main body, which can be rotated up and down, left and right, and a display video on the display, Describes a television device having a controller that specifies any one user as a target user and changes the posture of the display main body to follow the movement of the target user when the target user moves. Patent Document 3 also describes a target changing unit that changes the target user specified by the controller.

JP 2008-146221 A JP 2006-333122 A JP 2006-324952 A

  However, in an apparatus that presents information to an unspecified number of users using an information presentation unit that can present information only to a limited number of users at the same time, such as Patent Document 1 and Patent Document 2 described above, As in Reference 3, there is a problem that even if the target user is determined, the number of users who can present information is limited.

  The present invention has been made in view of such circumstances, and its purpose is to present information to an unspecified number of users by using an information presenting unit that can simultaneously present information to only a finite number of users. An information presentation apparatus and an information presentation method that provide an information presentation apparatus and an information presentation method capable of presenting information to more users.

(1) The present invention has been made to solve the above-described problems, and the information presenting apparatus of the present invention uses an information presenting unit that can present information to only a finite number of users at the same time. An information presentation device for presenting information to a user, a user state detection unit that detects the state of the user, and a user to whom the information presentation unit presents information based on a detection result of the user state detection unit And a user selection unit for sequentially changing the.

(2) Moreover, the information presentation apparatus of this invention is the above-mentioned information presentation apparatus, Comprising: The said user selection part repeats selection and release | release of the said target user based on the detection result of the said user state detection part. Thus, the target users are sequentially changed.

(3) Moreover, the information presentation device of the present invention is any one of the information presentation devices described above, wherein the user selection unit includes information indicating a degree of interest in the information presented by the information presentation unit, and the information Detection of at least one of the information indicating the ease of transmission of the information presented by the presenting unit and the information indicating the number of times the information is presented by the information presenting unit for each user by the user state detecting unit It is generated based on the result, and the target user is selected based on the generated information.

(4) Moreover, the information presentation apparatus of this invention is the above-mentioned information presentation apparatus, Comprising: The said user selection part is the degree of interest with respect to the information which shows the information which the selected time shows, and the said information presentation part A user who is selecting at least one of information indicating information, information indicating ease of transmission of information presented by the information presenting unit, and information indicating the number of times information is presented by the information presenting unit Each is generated based on the detection result of the user state detection unit, and the user to be released is determined based on the generated information.

(5) Moreover, the information presentation apparatus according to the present invention is any one of the information presentation apparatuses described above, wherein the information indicating the degree of interest or the information indicating the ease of transmission of the information is a position where the user is present. It is characterized by including the information which shows.

(6) Moreover, the information presentation apparatus of the present invention is any one of the information presentation apparatuses described above, wherein the information indicating the degree of interest or the information indicating the ease of transmission of the information indicates the orientation of the user. It is characterized by including information to show.

(7) Moreover, the information presentation apparatus of the present invention is the information presentation apparatus described above, wherein the information indicating the degree of interest or the information indicating the ease of transmission of the information includes the user, the information presentation unit, and the like. It includes the information indicating the distance.

(8) Moreover, the information presentation apparatus of the present invention is the information presentation apparatus described above, wherein the information indicating the degree of interest or the information indicating the ease of transmission of the information is information indicating the degree of stillness of the user. It is characterized by including.

(9) Moreover, the information presentation apparatus according to the present invention is any one of the information presentation apparatuses described above, wherein the information indicating the degree of interest includes information indicating the movement of the user's head. It is characterized by.

(10) Moreover, the information presentation apparatus of the present invention is the information presentation apparatus described above, wherein the information indicating the degree of interest includes information indicating a waiting time during which the user is not selected. .

(11) Moreover, the information presentation apparatus according to the present invention is the information presentation apparatus described above, and the information indicating the degree of interest indicates a change in distance from the information presentation unit since the user is selected. It is characterized by including information.

(12) Moreover, the information presentation apparatus according to the present invention is the information presentation apparatus described above, and the user selection unit releases the selected user when a preset time has elapsed since the selection. It is characterized by.

(13) Further, the information presentation method of the present invention is an information presentation method in an information presentation apparatus that presents information to an unspecified number of users using an information presentation unit that can simultaneously present information to only a finite number of users. The information presentation device includes a first process for detecting the state of the user, and the information presentation unit provides information to be presented by the information presentation unit based on a detection result of the first process. And a second step of sequentially changing the users.

  According to the present invention, the information presenting unit sequentially changes the users to whom information is presented based on the state of the user, so that information can be presented to more users.

It is a perspective view which shows the hardware constitutions of the motion parallax reproduction image presentation system in the 1st Embodiment of this invention. It is a top view which shows the hardware constitutions of the motion parallax reproduction video presentation system in the embodiment. It is a schematic block diagram which shows the structure of the motion parallax reproduction image generation apparatus 10 in the embodiment. It is a figure which shows the hierarchical structure of the data recorded on the user information recording part 13 in the embodiment, and the detail. It is a figure which shows the list | wrist of the data recorded on the system information recording part 14 in the embodiment. It is a perspective view explaining the information which the data which the user information recording part 13 and the system information recording part 14 in the embodiment represent memorize | stores. It is a top view explaining the information which the data which the user information recording part 13 in the same embodiment and the system information recording part 14 store represents. It is a flowchart explaining operation | movement of the user selection / release process part 15 in the embodiment. It is a flowchart explaining the selected user addition process (S2) in the embodiment. It is a flowchart explaining the selected user release process (S3) in the embodiment. It is a figure which shows the list of the intermediate variables used for calculation of the fitness F_n (t) and the selected time index | exponent R_n (t) in the embodiment. It is a top view explaining the calculation method of user confrontation degree C_n (t) in the embodiment. It is explanatory drawing (top view) regarding the calculation method of service area adaptation A_n (t) in the embodiment. It is explanatory drawing (top view) regarding another calculation method of service area suitability A_n (t) in the embodiment. It is a schematic block diagram which shows the structure of the user information detection part 12 in the embodiment. It is a flowchart explaining operation | movement of the user position and direction calculation part 19 in the embodiment. It is a flowchart explaining operation | movement of the user three-dimensional information generation part 20 in the embodiment. The distance measurement camera 91, the distance measurement camera 92, and the distance measurement camera 93 installed on the ceiling in the same embodiment are viewed from the side, taking a scene where two candidate users are standing on the ground. is there. It is a figure explaining the silhouette information by the distance data image | photographed with the distance measurement camera 91 in the embodiment. It is a figure explaining the silhouette information by the distance data image | photographed with the distance measurement camera 92 in the embodiment. It is a figure explaining the silhouette information by the distance data image | photographed with the distance measurement camera 93 in the embodiment. It is a figure which shows a mode that three silhouette information in the same embodiment was integrated and the common area | region was calculated. It is a figure which shows a mode that the silhouette of the area | region below 50 cm on a floor was deleted from the integrated silhouette in the same embodiment. It is a figure which shows a mode that the integrated silhouette in the same embodiment was divided | segmented into the block of a predetermined size. It is a figure which shows a mode that the block which divided | segmented the silhouette in the same embodiment was classified into two clusters by clustering. It is a figure which shows a mode that the silhouette below 24 cm from the upper part was deleted for every cluster in the same embodiment. It is a flowchart explaining operation | movement of the user information collation part 21 in the embodiment. It is a flowchart explaining step S51 (current information integration process) of FIG. 27 in the embodiment. It is a flowchart explaining step S52 (integration processing with past information) of FIG. 27 in the embodiment. It is a flowchart explaining step S72 (information update process) of FIG. 29 in the embodiment. It is a figure explaining the example which displays the motion parallax reproduction image in the embodiment only on a part of screen. It is a figure explaining the example which shows a motion parallax reproduction image to two selected users in the embodiment. It is the figure which showed the system which can divide | segment a screen into two area | regions, and three viewpoint positions of A, B, and C in the embodiment. It is the top view which showed the system which can divide | segment the surface into two area | regions, and three viewpoint positions of A, B, and C in the embodiment. It is a schematic block diagram which shows the structure of the motion parallax reproduction image generation device 10 in the 2nd Embodiment of this invention. It is a flowchart explaining operation | movement of the user selection process part 26 in the embodiment. It is a flowchart explaining step S91 (selection user selection process) of FIG. 36 in the embodiment. It is a perspective view which shows the hardware constitutions of the advertisement display system in the 3rd Embodiment of this invention. It is a schematic block diagram which shows the structure of the advertisement video reproduction apparatus 27 in the same embodiment. It is a perspective view which shows the hardware constitutions of the audio | voice guidance system in the 4th Embodiment of this invention. It is a schematic block diagram which shows the structure of the audio | voice reproduction apparatus 31 in the embodiment. It is the figure which showed three viewpoint positions of the audio | voice guide system and A, B, and C which can present audio | voice information to two persons in the embodiment. It is the figure (top view) which showed three viewpoint positions of the audio | voice guide system and A, B, and C which can present audio | voice information to two persons in the embodiment. It is the figure which showed a mode that the user 1001 was looking at the condition where the cube 1002 has been placed in the lower front and the ball 1003 in the upper rear. It is a figure which shows the mode of the cube 1002 and the ball | bowl 1003 seen from the user 1001 when the user 1001 exists in the center position. It is a figure which shows the mode of the cube 1002 and the sphere 1003 seen from the user 1001 when the user 1001 is moved to the position A from the center. It is a figure which shows the mode of the cube 1002 and the sphere 1003 seen from the user 1001 when the user 1001 is moved from the center to the position B.

[First Embodiment]
Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. Hereinafter, a user selected in the motion parallax reproduction video presentation system that implements the present invention is referred to as a selected user, and a user who is around the system and may be selected by the system is a candidate user. Call it. 1 and 2 are diagrams illustrating a hardware configuration of the motion parallax reproduction video presentation system according to the first embodiment of the present invention. The motion parallax reproduction video presentation system in this embodiment includes a display 6, a camera 7, a plurality of cameras 8, a plurality of distance measurement cameras 9, and a motion parallax reproduction video generation device 10. 1 and 2 show a state in which one of the two candidate users 5 on the front face of the display 6 is viewing the motion parallax reproduction image displayed on the display 6. 1 is a perspective view of the motion parallax reproduction video presentation system as viewed from obliquely above, and FIG. 2 is a top view.

The camera 7 is installed on the upper part of the display 6 and photographs candidate users in the vicinity of the front of the display 6. In the present embodiment, only one unit is installed, but a plurality of units may be provided.
The camera 8 is an area including the vicinity of the display 6 and is installed so as to photograph candidate users in an area wider than the photographing area of the camera 7 from various directions. In FIG. 1 and FIG. 2, four cameras 8 are installed so as to cover the area in front of the display 6 from four directions, but the number is not limited to this, and a larger or smaller number of cameras 8 is provided. May be installed. Note that each camera 8 is connected to the motion parallax reproduction video generation device 10 and outputs an imaging result. However, in order to avoid complication of the drawings, lines indicating connection are not shown in FIGS. 1 and 2. Moreover, although the camera 7 is installed in the upper part of the display 6 and the some camera 8 is installed in the ceiling part, these may be provided with the same function.

  The distance measuring camera 9 is installed downward from the ceiling. A normal camera acquires information on luminance and color of a subject for each pixel and outputs it as image data. On the other hand, the distance measuring camera 9 is a camera that acquires distance data to a subject for each pixel together with normal image data or instead of normal image data and outputs it as distance image data. Such a camera has already been put into practical use and commercially available. For example, a product “SwissRanger SR3000” manufactured by MESA can acquire a monochrome normal image having a resolution of 176 × 144 pixels and a distance image having the same resolution. These distance measuring cameras 9 are installed so as to photograph candidate users in a wider area including the vicinity of the display 6. In FIG. 1 and FIG. 2, nine distance measuring cameras 9 are installed. However, the number is not limited to this number, and a larger or smaller number of distance measuring cameras 9 may be installed. Note that each distance measuring camera 9 is connected to the motion parallax reproduction video generation device 10, but a line indicating the connection is not shown in order to avoid complexity of the drawing.

  The motion parallax reproduction video generation device 10 performs processing for detecting the positions and orientations of a plurality of candidate users from image information and distance information captured by the camera 7, the plurality of cameras 8, and the plurality of distance measurement cameras 9. A process of selecting and releasing one or several persons from among them, a process of generating an image according to the viewpoint position of the selected candidate user, and outputting the image to the display 6 are performed. Details of these processes will be described in order below. In addition, when acquiring candidate user information in a wider range, a larger number of distance measurement cameras 9 are installed, or a larger number of cameras 8 are installed in various locations in various directions.

  FIG. 3 is a schematic block diagram illustrating a configuration of the motion parallax reproduction video generation device 10. In the present embodiment, in order to simplify the description, a case where a motion parallax reproduction image based on a three-dimensional CG model is generated will be described as an example. However, a motion parallax reproduction image may be generated by other methods. The motion parallax reproduction video generation device 10 (information presentation device) receives information from the camera 7, the plurality of cameras 8, and the plurality of distance measurement cameras 9, and outputs the generated video to the display 6 (information presentation unit). The dynamic parallax reproduction video generation device 10 includes a camera related information storage unit 11, a user information detection unit 12, a user information recording unit 13, a system information recording unit 14, a user selection / release processing unit 15, a video generation unit 16, and display related information. A storage unit 17 and a three-dimensional CG model data storage unit 18 are provided.

  The camera-related information storage unit 11 includes an angle of view of the camera 7, information on the attachment position and orientation of the camera 7 with respect to the display 6, an angle of view of the camera 8, information on the attachment position and orientation of the camera 8 with respect to the display 6, and a distance measurement camera. 9, and information on the mounting position and orientation of the distance measuring camera 9 with respect to the display 6 are held. The user information detection unit 12 (user state detection unit) detects the status of the candidate user. That is, in this embodiment, the user information detection unit 12 includes image information and distance information acquired by the camera 7, the camera 8, and the distance measurement camera 9, information stored in the camera related information storage unit 11, and user information recording. From the information recorded in the unit 13, the states (head positions and orientations) of a plurality of candidate users are calculated. Further, the user information detection unit 12 compares the information on the past head position and orientation of each candidate user with the current information, specifies the information of the same candidate user, and records it in the user information recording unit 13.

  The user information recording unit 13 stores current and past information of each candidate user. The system information recording unit 14 stores information related to the motion parallax reproduction video presentation system. The user selection / release processing unit 15 (user selection unit) sequentially changes the user to whom the display 6 presents information based on the detection result of the user information detection unit 12. That is, in the present embodiment, the user selection / release processing unit 15 selects an appropriate candidate user from each candidate user based on the detection result of the user information detection unit 12 and determines the selected user. A process of releasing a selected user who meets a predetermined condition from the selected user is repeatedly performed. At this time, the user selection / release processing unit 15 records information on user selection / release in the user information recording unit 13 and the system information recording unit 14. Further, the user selection / release processing unit 15 sends information regarding the currently selected user to the video generation unit 16.

  The display related information storage unit 17 holds information on the size and resolution of the display portion of the display 6. The three-dimensional CG model data storage unit 18 stores information related to a model for displaying a three-dimensional CG. For example, when displaying an image in which a sphere and a cube are arranged, model data of the sphere and the cube and information on the virtual arrangement of those models are stored in advance. The video generation unit 16 includes information on the selected user sent from the user selection / release processing unit 15, information stored in the display related information storage unit 17, and information stored in the 3D CG model data storage unit 18. Is used to synthesize an image viewed from the viewpoint position of the selected user and output it to the display 6.

  Next, details of the processing of each block will be described. First, data stored in the user information recording unit 13 and the system information recording unit 14 will be described. FIG. 4 is a diagram for explaining the hierarchical structure of data stored in the user information recording unit 13 and its details. The user information recording unit 13 stores a list of candidate users and information (candidate user status information) regarding each candidate user recorded in the list. Each candidate user is given an ID number and stored in the candidate user list.

  At the top of the hierarchical structure is an ID list of candidate users, 1, 2,. . . n,. . . As such, the ID numbers of candidate users are stored in order. For each candidate user ID number, the current selected time T_n, the current waiting time W_n, the number N_n of past selections, and the time list are stored. The current selected time T_n is information indicating the time since becoming the selected user when the candidate user n is the selected user, and is “0” when the candidate user n is not the currently selected user. is there. The current waiting time W_n is information indicating the time after the candidate user n is recognized by the system, or the time since the candidate user n is removed from the selected user, while the candidate user n is the selected user. Is reset to “0”.

  The number N_n of past selections is information representing the number of times that the candidate user n has been selected as a selected user so far. In addition, the time list is t1, t2,. . . t,. . . As described above, the time value at which the data is recorded is stored in order. For each time, information on the candidate user at that time (position information P_n (t), head orientation D_n (t)) is stored as lower information. The position information P_n (t) is information representing the center position of the head of the candidate user n at time t. The head direction D_n (t) is information representing the value of the unit vector in the front direction of the face at time t.

  FIG. 5 is a diagram for explaining the hierarchical structure of data stored in the system information recording unit 14 and its details. The system information recording unit 14 stores information related to the motion parallax reproduction video presentation system. The maximum selectable number of persons Max, the currently selected number of persons U, the position Ps, and the direction Ds are stored as information related to the motion parallax reproduction video presentation system. The maximum selectable number Max is the number of users that the motion parallax reproduction video presentation system can handle at the maximum. For example, when the motion parallax reproduction image is provided to one user using the entire screen of the display 6, Max = 1. Further, for example, when the screen of the display 6 is divided into two areas on the left and right sides, and each person provides motion parallax reproduction images to a total of two users, Max = 2. Furthermore, when the screen of the display 6 is divided into three regions and motion parallax reproduction images are provided to a total of three users, one each, Max = 3.

  The current number of selected people U is a value indicating how many users the motion parallax reproduction video presentation system currently provides the motion parallax reproduction video. For example, if no motion parallax reproduction video is currently provided to anyone, U = 0. Further, for example, in a system that can divide the screen of the display 6 into three regions and provide motion parallax reproduction images for each of three users, that is, a system with Max = 3, the divided screens. U = 1 when a motion parallax reproduction image is provided to one user using one of the regions, and no motion parallax reproduction image is provided to anyone in the remaining two regions. The position Ps is a coordinate at the center of the screen of the display 6 of the motion parallax reproduction video presentation system. The direction Ds is a vector that represents the direction in which the screen of the display 6 of the motion parallax reproduction video presentation system is facing, and is a unit normal vector value in which the video output side of the display 6 is positive.

  FIG. 6 and FIG. 7 illustrate some of the data stored in the user information recording unit 13 and the system information recording unit 14, and their relationship will be described. FIG. 6 is a view of a scene in which there are two candidate users in front of the display 6 as viewed from diagonally above. In such a case, the position P_n (t) of the candidate user n, the direction D_n (t) of the candidate user n, the system position Ps, and the system direction Ds are information on the position or orientation shown in FIG. FIG. 7 shows the same scene viewed from directly above.

  FIG. 8 is a flowchart for explaining the processing of the user selection / release processing unit 15. In the following description, it is assumed that each process of the motion parallax reproduction video presentation system is executed at regular time intervals, and the current time is t, and the user information recording is performed by the process at the previous time. The time of the data stored in the unit 13 is time t ′, and the difference is dt = t−t ′. That is, dt is a time interval between one process and the next process (hereinafter referred to as a process unit time), and 1 / dt is the number of processes per second, in other words, the process time. Refresh rate.

  As described above, the user selection / release processing unit 15 selects an appropriate candidate user from each candidate user to determine a selected user, and selects a user who meets a predetermined condition among the selected users. A process of releasing from the selected user is performed, and the information is recorded in the user information recording unit 13 and the system information recording unit 14. Further, the user selection / release processing unit 15 sends information regarding the currently selected user to the video generation unit 16.

As shown in FIG. 8, the user selection / release processing unit 15 includes a selected user information update process (S1), a selected user addition process (S2), a selected user release process (S3), and a selected user. The user information transmission process (S4) is sequentially repeated.
First, details of the information update process (S1) of the selected user will be described. In the information on candidate users stored in the user information recording unit 13, a user who has T_n ≠ 0 is a selected user, and a user who has W_n ≠ 0 is a candidate user other than the selected user. Therefore, the user selection / release processing unit 15 performs a process of T_n = T_n + (t−t ′) for all users where T_n ≠ 0, that is, all selected users, and increments the selected time T_n. .

  Next, details of the selected user addition process (S2) are shown in FIG. First, the user selection / release processing unit 15 extracts candidate users other than the selected user (S11). As described above, in the candidate user information recorded in the user information recording unit 13, a user who has T_n ≠ 0 is a selected user, and a user who has W_n ≠ 0 is a candidate user other than the selected user. . Therefore, the user selection / release processing unit 15 extracts users for which W_n ≠ 0. The extracted user is hereinafter referred to as “additional process target user”. Next, the user selection / release processing unit 15 calculates the fitness F_n (t) for each additional processing target user (S12). Here, the fitness F_n (t) is a value for measuring the suitability / fitness of the candidate user n as a selected user of the system. The higher the fitness F_n (t), the more suitable the user as the selected user. The specific calculation method will be described later.

  Next, the user selection / release processing unit 15 excludes additional processing target users that satisfy F_n (t) <FConst (S13). This avoids setting the user to be selected as an additional processing target user who is not suitable as a selected user even if the current selected number U is equal to or less than the maximum selectable number Max. This is a process intended to prevent unnecessary services from being performed. Let k be the number of additional processing target users remaining in step S13.

  Next, the user selection / release processing unit 15 determines whether or not k> 0 (S14). When the result of this determination is No (when k> 0 is not satisfied), the user selection / release processing unit 15 ends the processing as it is. When the determination result in step S14 is Yes (when k> 0), the user selection / release processing unit 15 determines whether or not k ≧ (Max−U) (S15). When the result of this determination is Yes (when k ≧ (Max-U)), the user selection / release processing unit 15 adds (Max-U) names in descending order of the fitness F_n (t). A processing target user is selected and added to the selected user (S16).

  More specifically, the user selection / release processing unit 15 selects (Max-U) names in descending order of the fitness F_n (t) among the candidate user information recorded in the user information recording unit 13. For the information regarding the additional processing target user, processing is performed such that the current waiting time W_n = 0, the current selected time T_n = t−t ′, and the number N_n = N_n + 1 selected in the past. In this case, the current waiting time W_n is set to 0 since the selection wait state is canceled, and the selected time T_n is incremented because the selected state is entered. Since the selection is made, “1” is added to N_n. It means that.

  Subsequently, the user selection / release processing unit 15 updates the current selection number U = Max among the information on the motion parallax reproduction video presentation system stored in the system information recording unit 14 (S17). This means that since the (Max-U) name was added to the selected user this time, the selected user originally changed from the U name to U + (Max-U) = Max name. The selected user addition process is thus completed.

  On the other hand, if the determination in step S15 is No (when k ≧ (Max-U) is not satisfied), the user selection / release processing unit 15 adds all the remaining k users to be added to the selected user. (S18). More specifically, similar to the processing in step S16, the current waiting time W_n = with respect to the information regarding the remaining additional processing target user k name among the information on the candidate users recorded in the user information recording unit 13 = 0, the current selected time T_n = t−t ′, and the number of past selections N_n = N_n + 1 are performed. These meanings are also the same as in step S16. Subsequently, the user selection / release processing unit 15 updates the current selection number U = U + k among the information on the motion parallax reproduction video presentation system stored in the system information recording unit 14 (S19). The meaning of this process is the same as in step S17. The selected user addition process is thus completed.

  Next, details of the selected user release process (S3) are shown in FIG. First, the user selection / release processing unit 15 determines whether or not the current selection number U> 0 (S21). In the case of No (when U> 0 is not satisfied), that is, since the current selection number U = 0 and there is no selected user, the user selection / release processing unit 15 performs the selected user release processing as it is. finish. On the other hand, when the determination result in step S21 is Yes (when U> 0), the user selection / release processing unit 15 calculates a selected time index R_n (t) for each selected user (S22). . Here, the selected time index R_n (t) is a value indicating a service providing time that is a time for providing service to each selected user. If this value is large, a longer time service is provided. If it is small, it will provide a shorter time service. The specific calculation method will be described later.

  Next, the user selection / release processing unit 15 releases the selected user satisfying R_n (t) <T_n from the selected user (S23). More specifically, the current waiting time W_n = t−t ′ with respect to the information of the selected user satisfying R_n (t) <T_n among the candidate user information stored in the user information recording unit 13. Then, the current selection time T_n = 0 is performed. Each of these is released from the selected state and returned to the selection waiting state, so that the current waiting time W_n is incremented. Since the selected state is released, the value of the selected time T_n is reset to “0”. It means that.

  Since the value of the current selected time T_n for the selected user increases monotonously as time passes, even if the value of the selected time index R_n (t) is a constant value, R_n (t) < The condition T_n is satisfied. In other words, basically, a process of dividing a service providing time into a certain period is performed. Actually, the time for providing the service varies depending on the calculation method of the selected time index R_n (t). This will be described in detail later.

  The number of selected users released at this time is m. Subsequently, the user selection / release processing unit 15 updates the current selection number U = U−m among the information on the motion parallax reproduction video presentation system stored in the system information recording unit 14 (S24). This means that since m names were released from the selected user this time, what was originally the U name for the selected user has been reduced to the U-m name. The selected user release process is thus completed.

  Finally, details of the selected user information transmission process (S4) will be described. The user selection / release processing unit 15 selects all of the selected user information stored in the user information recording unit 13 for the current selected time T_n ≠ 0, that is, all the selected users. The user ID number, the position information P_n (t), and the current number U of selected users are sent to the video generation unit 16.

  Next, a method for calculating the fitness F_n (t) by the user selection / release processing unit 15 will be described. FIG. 11 is a diagram showing a list of intermediate variables used for calculating the fitness F_n (t) and the selected time index R_n (t). The user selection / release processing unit 15 calculates these intermediate variables for each candidate user n and uses them to calculate the fitness F_n (t) and the selected time index R_n (t). The outline is as follows.

The relative distance Dist_n (t) is a relative distance between the candidate user n and the motion parallax reproduction video presentation system. The user stillness S_n (t) is a variable obtained by quantifying the degree to which the candidate user n stops at one place and does not move. The user operation degree M_n (t) is a variable obtained by quantifying how much the candidate user n is moving or moving. The user facing degree C_n (t) is a variable obtained by quantifying how much the candidate user n is facing the display. The service area suitability A_n (t) is a variable obtained by quantifying the degree to which the candidate user n is properly in the service area of the system.
Next, a method for calculating the fitness F_n (t) in the present embodiment will be described. The user selection / release processing unit 15 calculates the fitness F_n (t) using Expression (1).

Here, Dist_const, Sconst, Mconst, Wconst, and Nconst are the relative distance Dist_n (t), user staticity S_n (t), user behavior M_n (t), current waiting time W_n, and past number of times N_n, respectively. It is a preset constant for adjusting the contribution ratio of each intermediate variable or data to the fitness F_n (t) for each intermediate variable or data.

Subsequently, while explaining calculation examples of each intermediate variable, the role and effect of each intermediate variable or data in the fitness F_n (t) will be described.
The user selection / release processing unit 15 calculates the user confrontation degree C_n (t) according to Expression (2).

Here, (Ps−P_n (t)) / || Ps−P_n (t) || is a unit vector from the position P_n (t) of the candidate user n to the position Ps of the motion parallax reproduction video presentation system. . The user's confrontation degree C_n (t) is obtained by calculating the inner product of the unit vector and the head direction D_n (t) of the candidate user n. FIG. 12 shows the relationship between this unit vector and D_n (t). FIG. 12 is a view from above. The more the candidate user n is looking straight at the center of the display, the smaller the angle formed by (Ps−P_n (t)) / || Ps−P_n (t) || and D_n (t). Therefore, the inner product approaches 1. Conversely, when candidate user n turns away from the display, the angle between (Ps−P_n (t)) / || Ps−P_n (t) || and D_n (t) increases, and the inner product thereof The value of becomes smaller.

Therefore, the user confrontation degree C_n (t), which is the inner product value of (Ps−P_n (t)) / || Ps−P_n (t) || and D_n (t), is the direction of the candidate user n, that is, It can be used as an index of how straight the candidate user n is looking at the display. The more the candidate user n is looking straight at the display, the closer the user confrontation C_n (t) is to 1. If the angle formed by (Ps−P_n (t)) / || Ps−P_n (t) || and D_n (t) exceeds 90 °, the inner product value becomes negative. To avoid this, C_n (t) = 0 when the inner product value is negative as shown in equation (2).
The user selection / release processing unit 15 calculates the service area suitability A_n (t) by Expression (3).

(P_n (t) -Ps) / || P_n (t) -Ps || is a unit vector from the position Ps of the motion parallax reproduction video presentation system to the position P_n (t) of the candidate user n. The service area suitability A_n (t) is obtained by calculating the inner product of this unit vector and the direction Ds of the motion parallax reproduction video presentation system. FIG. 13 shows the relationship between the unit vector and the direction Ds. FIG. 13 is a view from above. The closer the position of candidate user n is to the front of the display, the smaller the angle formed by (P_n (t) −Ps) / || P_n (t) −Ps || and Ds. Therefore, the inner product approaches 1.

  Conversely, when the candidate user n is located in the horizontal direction away from the front of the display, the angle formed by (P_n (t) −Ps) / || P_n (t) −Ps || and Ds is 90 °. The inner product value becomes smaller. Furthermore, if the candidate user n turns to the back of the display, the angle exceeds 90 °, so the value of the inner product becomes negative. Therefore, the service area suitability A_n (t), which is the inner product value of (P_n (t) −Ps) / || P_n (t) −Ps || and Ds, indicates how much the candidate user n is in the front direction of the display. It can be used as an indicator of whether or not In other words, it indicates that the front direction of the display is the best service area, and the service area becomes worse as the distance from the display area increases. The closer the candidate user n is to the front direction of the display, the closer the service area suitability A_n (t) is to 1. As described above, when the angle formed by (P_n (t) −Ps) / || P_n (t) −Ps || and Ds exceeds 90 °, the inner product value becomes negative. In order to avoid this, as shown in Expression (3), when the inner product value is negative, the service area suitability A_n (t) = 0 is set.

  In Formula (1) for calculating the fitness F_n (t), if the value of the user confrontation C_n (t) or the service area fitness A_n (t) increases, the fitness F_n (t) also increases. User n is likely to be selected as a selected user. The fact that the user facing degree C_n (t) is large indicates that the candidate user n is facing the display direction, and there is a possibility that he / she is interested in the motion parallax reproduction video presentation system. Also, even if you are not interested in the motion parallax reproduction video presentation system at present, the video displayed on the display is in the eye and you can expect to be interested. That is, the user confrontation degree C_n (t) is information indicating the degree of interest or information indicating ease of transmission of information regarding the candidate user n.

  Further, the large value of the service area suitability A_n (t) indicates that the candidate user n is located in a direction close to the front of the display, and is interested in the motion parallax reproduction video presentation system. There is a possibility that. Even if you are not interested in the motion parallax reproduction video presentation system at present, depending on the orientation of the head, you can expect to see the video displayed on the display because it is in a position where you can see it well. That is, the service area suitability A_n (t) is information indicating the degree of interest or information indicating the ease of transmission of information regarding the candidate user n.

  Since the term of user confrontation C_n (t) or service area suitability A_n (t) in the suitability F_n (t) is oriented in the direction of the display 6 in this manner, the motion parallax reproduction video presentation system Candidate users who are likely to be interested, or candidates who can be expected to be interested because the video displayed on the display is in their eyes, or depending on the orientation of the head By selecting as a selected user with priority given to candidate users who can expect to see the video displayed on the display because they are in a position where they can see well, services can be provided more effectively, and advertising information The effect that the possibility of being transmitted can be increased can be realized.

  In Equation (1), the user confrontation degree C_n (t) and the service area suitability degree A_n (t) are multiplied by all other variables. That is, if at least one of the user confrontation degree C_n (t) and the service area conformity degree A_n (t) is 0, the conformity degree F_n (t) is set regardless of other intermediate variables and data values. 0. Thus, the degree of user confrontation C_n (t) and the service area suitability A_n (t) have a larger influence on the calculated value of the suitability F_n (t) than the values of other intermediate variables and data.

This is because, with respect to the degree of user confrontation C_n (t), if the candidate user n is not facing the direction in which the display 6 is in sight, the candidate user n appears on the display 6 regardless of other conditions. This is because the output video cannot be seen and advertisement information is not transmitted to the candidate user n. In addition, regarding the service area suitability A_n (t), if the candidate user n is not in a position where the display 6 can be seen, the video of the candidate user n output to the display 6 regardless of other conditions. This is because information cannot be transmitted to the candidate user n. For this reason, as shown in Expression (1), the user facing degree C_n (t) and the service area suitability A_n (t) are multiplied by all other variables, so that candidate users to whom no advertising information is transmitted are selected users. You can avoid that.
Formula (4) shows an example of a calculation method different from formula (3) for service area suitability A_n (t).

Here, the range of the position of the candidate user where the display 6 is easy to see, that is, the service area is defined by an angle θ from the front direction of the display to the left, right, up and down as shown in FIG. Note that FIG. 14 is a top view, and only the angle θ to the left and right is shown. In Expression (4), A_n (t) = 1 when the candidate user is within this range, and A_n (t) = 0 when the candidate user is outside the range.

  When Expression (3) is used, as the candidate user n is closer to the front direction of the display 6 as described above, A_n (t) is calculated to be closer to 1, that is, the fitness is higher. It becomes easy to be chosen by the selection user. For this reason, there is an advantage that priority can be given to the candidate user who is in the position where the display 6 is most easily seen, but only the candidate user in the vicinity of the front is selected, and the candidate user who is in a position shifted from the front is difficult to be selected. is there. On the other hand, in the equation (4), as long as the user is within the range of the angle θ, the service area suitability A_n (t) of the candidate user is equal to 1. Therefore, as long as the user is within the range of the angle θ, the user is selected by the selected user. There is no difference in ease. Although the candidate user who is in the position where the display 6 is most easily viewed is not necessarily selected, there is a characteristic that the candidate users around the motion parallax reproduction video presentation system can be selected uniformly regardless of the position.

  For example, when the display 6 is an LCD (Liquid Crystal Display), the value of the angle θ is determined according to the viewing angle characteristics of the LCD. In this case, a range in which the video displayed on the display 6 can be seen with an acceptable image quality is considered as a service area as it is. Further, for example, a value of an angle narrower than the viewing angle characteristic may be used as the angle θ, such as 45 ° or 30 °.

The relative distance Dist_n (t) is the relative distance between the candidate user n and the motion parallax reproduction video presentation system as described above, and the user selection / release processing unit 15 uses the Dist_n (t) = || Ps−P_n (t) ||
In the equation (1) for calculating the fitness F_n (t), the relative distance Dist_n (t) is inserted as a term Dist_const / Dist_n (t) in parentheses. This term increases as the relative distance Dist_n (t) decreases. That is, the closer the candidate user n is to the display, the greater the value of the fitness F_n (t), and the candidate user n is likely to be selected as the selected user.

Therefore, a candidate user who is expected to have come close to the display 6 with interest is given priority as a selected user, or because the user is near the display 6, there is a high possibility of seeing an advertisement. A candidate user who seems to be a priority user is selected as a selected user. That is, the information indicating the distance between the candidate user and the information presenting unit is information indicating the degree of interest or information indicating how easily the information is transmitted. As a result, it is possible to effectively provide the service, and to realize the effect that the possibility that the advertisement information is transmitted can be increased. Note that the influence of the relative distance Dist_n (t) on the fitness F_n (t) can be adjusted by changing the value of Dist_const. Increasing Dist_const increases the contribution of the relative distance Dist_n (t).
The user selection / release processing unit 15 calculates the user stillness S_n (t) by Expression (5).

Further, Psma_n (t) in the equation (5) is calculated by the equation (6).

This Psma_n (t) is obtained by calculating a simple moving average of l pieces of data nearest to the position P_n (t) of the candidate user n, and is hereinafter referred to as an average position of the candidate user n. For example, when the refresh rate of the process is 30 times / sec, l is a value obtained by averaging the position data of 1 second of the candidate user n if l = 30. The term || Psma_n (t) −Psma_n (t ′) || in the denominator of the equation (5) is the average position of the candidate user n at time t and the time t ′ of the processing at the previous time point. The distance between the average positions of candidate users n. That is, an operation for obtaining an average moving distance of the candidate user n during the processing unit time.

  By taking the reciprocal of this value, it is used as a variable indicating how much the candidate user is stationary, that is, the user stationary degree. The reason why the denominator is incremented by 1 is to prevent the denominator from becoming zero. The user stillness S_n (t) obtained in this way takes a maximum value of 1 when the candidate user n remains at the same position, and takes a smaller value when the candidate user n is moving. . That is, it can be seen that the larger the value of the user stationary degree S_n (t), the higher the possibility that the candidate user n is stationary and stopped.

  In the equation (1) for calculating the fitness F_n (t), the user stillness S_n (t) is inserted as a term “Sconst × S_n (t)” in parentheses. This term increases as the user stillness S_n (t) increases. That is, as the possibility that the candidate user n is stationary and stopped is higher, the value of the fitness F_n (t) increases, and the candidate user is easily selected as the selected user. There are various reasons why the candidate user is stopped, but there is a possibility that the candidate user is stopped because he / she is interested in the motion parallax reproduction video presentation system. Even if this is not the case, the candidate user who stops for whatever reason sees the advertisement displayed on the display rather than the candidate user who moves near the motion parallax reproduction video presentation system. The possibility is considered high.

Such a candidate user can be preferentially selected as a selected user, so that the service can be provided more effectively and the possibility that advertisement information can be transmitted can be realized. That is, the user stillness S_n (t) is information indicating the degree of interest or information indicating ease of transmission of information regarding the candidate user n.
Note that by changing the value of Sconst, it is possible to adjust the influence of the user stillness S_n (t) on the fitness F_n (t). Increasing Sconst also increases the contribution of user staticity S_n (t).
The user selection / release processing unit 15 calculates the user operation degree M_n (t) by Expression (7).

The first term in the equation (7) is a movement distance || P_n (t + (1-i) · dt) −P_n (t−i · dt) || Are calculated at the latest m times and added to each other, and the cumulative movement distance for m unit times is obtained. For example, when the refresh rate of the process is 30 times / sec and m = 60, m is a value obtained by accumulating the moving distance of candidate user n for 2 seconds.

  The second term is to obtain the distance between the current head position of the candidate user n and the head position at a time m units before the processing, and between m unit times The straight line distance moved to is obtained. As described above, the user operation degree M_n (t) is a value obtained by subtracting the linear moving distance calculated in the second term from the cumulative moving distance of the candidate user n calculated in the first term. This value means the following:

  For example, when candidate user n swings his / her head from side to side, the position of candidate user n is almost stationary when viewed on average, but the moving distance of the head increases. Accordingly, the cumulative movement distance of the first term is increased, the linear movement distance of the second term is decreased, and as a result, the value of the user operation degree M_n (t) is increased. On the other hand, for example, when the candidate user n passes in front of the motion parallax reproduction video presentation system, the cumulative movement distance of the first term and the linear movement distance of the second term are substantially equal, and as a result, the user The value of the operation degree M_n (t) becomes small. From this, the user operation level M_n (t) is a value indicating how much the candidate user n is moving the head, and the movement of the head is merely passed in front of the motion parallax reproduction video presentation system. Cases that are large can be excluded. Note that (cumulative movement distance of the first term) ≧ (linear movement distance of the second term) always holds, and therefore the user operation degree M_n (t) ≧ 0 always holds.

  In the equation (1) for calculating the fitness F_n (t), the user behavior M_n (t) is inserted as a term Mconst × M_n (t) in parentheses. This term increases as the user operation level M_n (t) increases. That is, the greater the amount that candidate user n moves his / her head, the greater the value of fitness F_n (t), making it easier for the candidate user to be selected as the selected user.

  The motion parallax reproduction video presentation system dealt with by the present invention generates an image to be appropriately output to the display 6 in accordance with the head position, so that the user who sees it can see how the head looks when moving the head. It is envisaged to take action such as enjoying the change of the camera or moving the head to see the displayed object from various directions. From here, even if a certain candidate user n has not yet been selected as a selected user, the candidate user n has taken action that another user who has already been selected as the selected user and is watching an advertisement is moving his head. You can think of the situation as if you look at it and move your head. This indicates that the candidate user n is interested in the motion parallax reproduction video presentation system. That is, the user operation level M_n (t) is information indicating information indicating the degree of interest regarding the candidate user n.

  In terms of the fitness F_n (t), the term relating to the user behavior M_n (t) is selected as a selected user with priority given to the candidate user n who seems to be interested in the motion parallax reproduction video presentation system. By doing so, it is possible to provide an effect that the service can be provided more effectively and the transmission efficiency of the advertisement information can be increased. Note that by changing the value of Mconst, it is possible to adjust the influence of the user operation level M_n (t) on the fitness level F_n (t). Increasing Mconst also increases the contribution of user operability M_n (t).

In Expression (1) for calculating the fitness F_n (t), the current waiting time W_n of the candidate user n is inserted as a term Wconst × W_n in parentheses. This term increases as the current waiting time W_n increases. That is, as the candidate user n has a larger current waiting time W_n, the value of the fitness F_n (t) increases, and the candidate user is easily selected as the selected user. When the current waiting time W_n is large, it means that the candidate user n stays in the vicinity of the motion parallax reproduction video presentation system for a long time. This may be an interest in the motion parallax reproduction video presentation system. That is, the current waiting time W_n is information indicating the degree of interest regarding the candidate user n.
Thus, the term regarding the current waiting time W_n in the fitness F_n (t) preferentially selects the candidate user n who seems to be interested in the motion parallax reproduction video presentation system as the selected user. Thus, it is possible to provide an effect that the service can be provided more effectively and the possibility that the advertisement information is transmitted can be increased.

  Furthermore, since priority is given to candidate users who have been waiting for a longer time, it is possible to avoid biasing selected users, provide services fairly, and to transmit advertising information to more users Can also be realized. Note that by changing the value of Wconst, the influence of the current waiting time W_n on the fitness F_n (t) can be adjusted. Increasing Wconst also increases the contribution of the current waiting time W_n.

  In the formula (1) for calculating the fitness F_n (t), the number N_n of past selections of the candidate user n is inserted as a term Nconst × 1 / (N_n + 1) in parentheses. This term becomes smaller as the number of times N_n selected in the past is larger. That is, the candidate user who has become the selected user many times has a smaller value of the fitness F_n (t), and the candidate user is less likely to be selected as the selected user. In order to provide a service to a larger number of users using the motion parallax reproduction video presentation system, it is avoided to select the same candidate user as a selected user many times and provide the service, and a candidate that has never been selected. It is better to give priority to users. Also, even when selecting candidate users that have been selected in the past again, it is better to select candidate users with a smaller number of selections in the past in order to provide services to more users. That is, the service can be provided more fairly.

  As described above, it is difficult to reselect candidate users who have been selected in the past according to the term related to the number N_n selected in the past in the fitness F_n (t), and the candidate users who have been selected in the past more frequently. It becomes difficult to select again, the service can be provided more evenly, and the effect that advertisement information can be transmitted to more users can be realized. By changing the value of Nconst, it is possible to adjust the influence of the number N_n selected in the past on the fitness F_n (t). If Nconst is increased, the contribution of the number N_n selected in the past also increases.

  Next, the first to third examples of the calculation method of the selected time index R_n (t) by the user selection / release processing unit 15 will be described. In the first example, the user selection / release processing unit 15 calculates the selected time index R_n (t) using Expression (8).

This is the simplest example, and uses a predetermined constant Rconst preset as the selected time index R_n (t). Therefore, in the process of step S23 of FIG. 10, if the current selected time T_n exceeds R_n (t) = Rconst, the candidate user n is released from the selected state. That is, when the predetermined time Rconst is exceeded after the candidate user n is selected by the selected user, the selected user n is released from the selected state. Thereby, it is possible to provide the service to any selected user fairly for a certain period of time, and to realize the effect that the advertising information can be transmitted to more users. In addition, as Rconst is larger, the candidate user n is less likely to be removed from the selected user, that is, the service providing time is longer.
In the second example, the user selection / release processing unit 15 calculates the selected time index R_n (t) using Expression (9).

The first term of Equation (9) is the same predetermined constant Rconst as in Equation (8). The effect realized by this constant is the same as described above.
In the second term of Equation (9), t−T_n is the time when candidate user n becomes the selected user, and Dist_n (t−T_n) is the time when candidate user n becomes the selected user. It is a relative distance from the display 6. Therefore, (Dist_n (t) −Dist_n (t−T_n)) is a distance indicating how close the candidate user n is to the display 6 from the time when the candidate user n becomes the selected user to the current time. Show.

  If the candidate user is close to the display 6, the value of the second term is increased, and the value of the selected time index R_n (t) is increased, and the candidate user n is not easily released from the selected user. On the contrary, if the candidate user is away from the display 6, the value of the second term becomes negative, and consequently the value of the selected time index R_n (t) becomes small, so that the candidate user n is easily removed from the selected user. Become.

  After the candidate user becomes the selected user, if he / she is interested in the video on the display 6, he / she will continue to watch the video without leaving the display 6 at least any more, or he / she will approach the display 6 if he / she is more interested. Conceivable. Conversely, if there is no interest, it is considered that the user will move away from the display 6. That is, the information (second term) indicating the change in distance from the display 6 since the selected user is selected is information indicating the degree of interest. The second term takes into account such behavior of the selected user and increases or decreases the time for providing the service.

  In other words, a longer time service is provided to the selected user who seems to be more interested in the video on the display 6, and a shorter time service is provided to the selected user who is considered less interested in the video on the display 6. By doing so, it is possible to provide an effect that the service can be provided more effectively and the possibility that the advertisement information is transmitted can be increased. Dist_const_r is a predetermined constant, and by changing this value, the influence of the second term on the selected time index R_n (t) can be adjusted. Increasing Dist_const_r also increases the contribution of the second term.

  The third term of Equation (9) is a value obtained by multiplying the user operation degree M_n (t) by a predetermined constant Mconst_r. This term increases as the user operation level M_n (t) increases. That is, the larger the amount that the candidate user n moves his / her head, the greater the value of the selected time index R_n (t), and the more difficult the candidate user n is removed from the selected user. As described above, since the motion parallax reproduction video presentation system handled by the present invention generates an image to be output to the display 6 appropriately according to the head position, the user viewing this moved the head. Imagine taking the action of enjoying the change in the way you look. It can be considered that the moving amount of the head of the candidate user n who has become the selected user is actively watching the video with interest in the motion parallax reproduction video presentation system.

  In the selected time index R_n (t), the term related to the user activity M_n (t) is longer for the candidate user n who seems to be interested in the motion parallax reproduction video presentation system. By providing the service, it is possible to provide an effect that the service can be provided more effectively and the possibility that the advertisement information is transmitted can be increased. Note that by changing the value of Mconst_r, it is possible to adjust the influence of the user operation degree M_n (t) on the selected time index R_n (t). Increasing Mconst also increases the contribution of user operability M_n (t).

  The fourth term of Equation (9) is a value obtained by multiplying 1 / (N_n + 1) by a predetermined constant Nconst_r for the number N_n of past selections of candidate user n. This term becomes smaller as the number of times N_n selected in the past is larger. That is, as the candidate user who has become the selected user many times in the past, the value of the selected time index R_n (t) becomes smaller, and the candidate user n is more easily released from the selected user. The second time and the third time to be selected as the selected user. . . For the candidate users who will be, the service provision time will be shortened from the first time, and as a result, the opportunity and time to provide the service to other candidate users will be extended to provide the services to many users uniformly. It is desirable from the viewpoint.

  As described above, in the selected time index R_n (t), the term related to the number N_n selected in the past is such that the service providing time is averaged by shortening the service providing time for candidate users with a large number of past selections. Can be achieved and the service can be provided more evenly. Furthermore, as a result, it is possible to increase the opportunities for providing services to other candidate users, and to realize the effect that the advertising information can be uniformly transmitted to more users. Note that by changing the value of Nconst_r, the influence of the number N_n selected in the past on the selected time index R_n (t) can be adjusted. Increasing Nconst_r also increases the contribution of the number N_n selected in the past.

  In the third example, the user selection / release processing unit 15 calculates the selected time index R_n (t) using Expression (10).

This equation (10) is obtained by multiplying equation (9) by the user confrontation degree C_n (t) and the service area suitability A_n (t). The values of the user facing degree C_n (t) and the service area suitability A_n (t) are 1 at the maximum and 0 at the minimum according to the expression (2) or the expression (4). Therefore, when the user confrontation degree C_n (t) and the service area conformity degree A_n (t) are the maximum values, that is, the selected user is facing straight toward the display 6 and is located in front of the display 6. If so, the value of equation (10) is equal to equation (9). On the other hand, when the value of the user confrontation degree C_n (t) or the service area suitability A_n (t) is small, for example, when the selected user turns to a direction other than the display 6 or when the selected user displays If the user has moved to a position where 6 cannot be seen, the value of the selected time index R_n (t) becomes small, that is, the time for providing the service becomes short.

In equation (10), the selected time index R_n (t) can be less than or equal to a predetermined constant Rconst and can be zero. This is because the candidate user n outputs to the display 6 regardless of the other conditions for the user facing degree C_n (t) unless the candidate user n faces the eyes of the display 6. If the candidate user n is not in a position where the display 6 can be seen, the candidate user n can be viewed regardless of other conditions. Since the video output to the display 6 cannot be viewed, it is useless to provide any service. In addition, the fact that the non-selected user has moved to such a position may be a case where the user loses interest in the video on the display 6. For the above reasons, there is no problem even if service provision is stopped in a short time. As a result, it is possible to increase the service opportunities for other candidate users, that is, to provide services to more users and to transmit advertising information to more users. it can.
In the calculation of the fitness F_n (t) and the selected time index R_n (t), the formula (2) is given as an example of calculating the user confrontation degree C_n (t), but other examples are also conceivable. As an example, the following formula (11) is given.

This is a value obtained by averaging the values of C_n (t) calculated by the equation (2) for a certain time k. This value C′_n (t) can be used in place of C_n (t) in the formula of the fitness F_n (t) and the selected time index R_n (t). It is considered that the operation of changing the direction of the human head is performed relatively frequently. Therefore, instead of using only the head orientation information at a certain time, a phenomenon that the candidate user was not selected as the selected user due to a short-term observation by using the average value over a certain period of time Can be prevented.

  As described above, the user selection / release processing unit 15 according to the present embodiment sequentially changes the selected user by repeating selection and release of the user based on the user state detected by the user information detection unit 12. For this reason, an information presentation device (motion parallax reproduction video) that presents information to an unspecified number of users using an information presentation unit that can simultaneously present information to only a finite number of users, such as the display 6 in the present embodiment. Since the generation apparatus 10) includes the user selection / release processing unit 15, an effect that information can be presented to more users can be obtained. Furthermore, when the user is selected, the fitness F_n (t) shown in Expression (1) is used, and when the user is released, the selected time index R_n shown in Expression (8) to Expression (10) is used. By using (t), it is possible to present information to more users and to increase the possibility that information can be transmitted.

  In the present embodiment, as parameters for calculating the fitness F_n (t), the user confrontation C_n (t), the service area fitness A_n (t), the relative distance Dist_n (t), and the user stillness S_n (T), user operation level M_n (t), current waiting time W_n, and the number N_n selected in the past have been used. For example, only the service area suitability level is used, and some of them are used. You may make it calculate.

  In the present embodiment, as parameters for calculating the selected time index R_n (t), in equation (10), the user confrontation degree C_n (t), the service area suitability A_n (t), and the candidate user n The distance (Dist_n (t) −Dist_n (t−T_n)), the user operation degree M_n (t), and the number of past selections from the time when the user becomes the selected user to the current time Although it has been described that N_n is used, for example, as shown in Expression (8) or Expression (9), calculation may be performed using a part of these.

  The description of other blocks in the motion parallax reproduction video generation device 10 shown in FIG. 3 is continued. As described above, the user information detection unit 12 includes the image information and distance information acquired by the camera 7, the camera 8, and the distance measurement camera 9, the information stored in the camera related information storage unit 11, and the user information recording unit 13. From the recorded information, a process for calculating the positions and orientations of the heads of a plurality of candidate users is performed, and the information on past head positions and orientations of each candidate user is compared with the current information, and the same information is obtained. Candidate user information is identified and recorded in the user information recording unit 13.

  FIG. 15 is a schematic block diagram illustrating the configuration of the user information detection unit 12. The user information detection unit 12 includes a user position / orientation calculation unit 19, a user three-dimensional information generation unit 20, and a user information collation unit 21. The user position / orientation calculation unit 19 includes the angle of view of the camera 7 held in the camera related information storage unit 11, information on the mounting position and orientation of the camera 7 with respect to the display 6, the angle of view of the camera 8, and the camera 8 with respect to the display 6. Information on the orientation and position of each candidate user's head is calculated using information on the attachment position and orientation and video information from the cameras 7 and 8.

  The user three-dimensional information generation unit 20 captures information on the angle of view of the distance measurement camera 9, information on the attachment position and orientation of the distance measurement camera 9 with respect to the display 6, and the distance measurement camera 9, which are held in the camera related information storage unit 11. From the obtained distance image, three-dimensional information of a space including the candidate user is generated, and one or more regions where the candidate user's head position is estimated are identified therefrom, and the head of each candidate user is determined therefrom. Calculate the position. The user information matching unit 21 integrates the head position information of each candidate user obtained by the user three-dimensional information generation unit 20 and the head direction and position information obtained by the user position / orientation calculation unit 19. The duplication is deleted, the head direction and position information of the integrated candidate user and the past information recorded in the user information recording unit 13 are collated, and the head of the integrated candidate user is verified. Which candidate user information in the past record is specified as information on the direction and position of the part, and the current information is integrated into the past information and recorded in the user information recording unit 13.

  FIG. 16 is a flowchart for explaining the operation of the user position / orientation calculation unit 19. First, the user position / orientation calculation unit 19 extracts a skin color region from an image input from the camera 7 or 8 in order to narrow down a region where a face may be captured (S31). Next, the user position / orientation calculation unit 19 performs face area extraction processing on the extracted skin color area (S32). Template matching can be used for this. Next, the user position / orientation calculation unit 19 estimates the orientation of the face (S33). For example, the eye region can be further specified from the face region, and the orientation of the face can be detected from the difference between the eye position in the face region and the size of the left and right eyes.

  Finally, the user position / orientation calculation unit 19 estimates the coordinates of the user's face position relative to the display 6 (S34). This is because the face becomes smaller as the face is farther from the camera, so the position and size of the face area in the image, the angle of view of the camera 7 recorded in the camera-related information storage unit 11, and the position and orientation of the camera 7 attached to the display 6. Information, the angle of view of the camera 8, and information on the mounting position and orientation of the camera 8 with respect to the display 6 can be calculated. At this time, if there are a plurality of candidate user faces in the image, face direction estimation and face position estimation are performed for all of them. Further, these processes are performed on all the video information by the camera 7 and the camera 8.

  In this process, it is necessary to determine the position of the face to be specifically determined as the face position estimation value. For example, the center position of the left and right eyes may be determined. Further, by the above processing, a face of a candidate user is photographed by a plurality of cameras 7 or 8, and face direction estimation and face position estimation are performed separately, that is, a plurality of face direction estimations for a certain candidate user. Values and face position estimates may exist. About this, the process which eliminates duplication is performed in the process of the user information collation part 21 demonstrated later.

  FIG. 17 is a flowchart for explaining the operation of the user three-dimensional information generation unit 20. First, the user three-dimensional information generation unit 20 backprojects the distance information obtained by each distance measuring camera 9 into a space, and creates silhouette information for each (S41). Next, the user three-dimensional information generation unit 20 integrates the silhouette information and creates one silhouette information (S42). Next, the user three-dimensional information generation unit 20 deletes information on an area 50 cm or less above the floor from the silhouette information (S43). Next, the user three-dimensional information generation unit 20 performs clustering by dividing the silhouette information into blocks of a predetermined size (S44). At this time, the clustering is terminated when the minimum value in the Euclidean square distance between the clusters exceeds a predetermined threshold. By this processing, the blocks are clustered into several regions. Next, the user three-dimensional information generation unit 20 deletes the silhouette below 24 cm from the top for each cluster (S45). Finally, the user three-dimensional information generation unit 20 calculates the center of gravity position of each cluster (S46), and sets this as the head center position of each candidate user.

  A more detailed description of the operation of the user three-dimensional information generation unit 20 will be given with reference to FIGS. FIG. 18 is a side view of a state in which the distance measuring camera 91, the distance measuring camera 92, and the distance measuring camera 93 installed on the ceiling are shooting a scene in which two candidate users are standing on the ground. is there. Hereinafter, in order to simplify the description, the information in the depth direction in the figure is omitted, and the description will proceed in the two-dimensional space of xy as shown in the lower left in the figure. The distance measuring cameras 91, 92, and 93 are part of the distance measuring camera 9 in FIG.

  First, the user three-dimensional information generation unit 20 performs a distance image backprojection process (silhouette creation) (S41). FIG. 19 is a diagram for explaining distance data photographed by the distance measuring camera 91. Since the distance information to the object located closest to the distance measurement camera 91 can be acquired, the user three-dimensional information generation unit 20 projects the information on the space, as shown in FIG. Get silhouette information. A space indicated by diagonal lines is an area where some object may exist. In this way, detailed information on the shadowed area is unknown. Similarly, the user three-dimensional information generation unit 20 projects distance data photographed by the distance measurement camera 92 on the space, and obtains silhouette information as shown in FIG. Furthermore, the user three-dimensional information generation unit 20 projects distance data photographed by the distance measurement camera 93 on the space, and obtains silhouette information as shown in FIG. Thus, three different pieces of silhouette information can be obtained by the three distance measuring cameras.

  Next, the user three-dimensional information generation unit 20 performs silhouette integration (S42). That is, the user three-dimensional information generation unit 20 extracts a common area of the three silhouette information obtained in step S41, thereby obtaining an integrated silhouette information as shown in FIG. Compared with the original scene in FIG. 18, it can be seen that although there are differences in details, the area where the two candidate users are present is generally correctly obtained as a silhouette. That is, the regions shown in FIGS. 18 to 20 that are not shadowed by the silhouette information by the individual distance measuring cameras complement each other by integrating them, and a more accurate silhouette as shown in FIG. Information is obtained. In addition, since the case where three distance measuring cameras 9 are installed is taken as an example here, the three silhouette information obtained from the three distance measuring cameras 91, 92, 93 are integrated, If there are a large number of units, it is sufficient to create silhouette information for the number of units and integrate them all into one.

  Next, as shown in FIG. 23, the user three-dimensional information generation unit 20 deletes the silhouette of the area 50 cm or less above the floor from the integrated silhouette (S43). This is because the distance information of the floor itself and the influence when luggage or the like is placed near the floor are excluded, and only the region of the human body is extracted and processed. Here, 50 cm is used as a preset value. However, the value is not limited to this, and may be another value.

  Next, the user three-dimensional information generation unit 20 performs clustering (S44) processing. That is, the user three-dimensional information generation unit 20 first divides the space into blocks having a predetermined size. For example, it is divided into cubes with a side of 5 cm. This is shown in FIG. Next, the user three-dimensional information generation unit 20 performs clustering on all blocks including silhouettes using the coordinates of the center of the blocks. Clustering or cluster analysis is one of data analysis methods and is generally used as a method for automatically classifying given data. When this method is applied to the block group as described above, it is possible to connect blocks having a short distance and classify them into several clusters, that is, to divide silhouettes for each individual.

  Clustering is broadly divided into a hierarchical method in which data is classified hierarchically and a non-hierarchical method in which data is classified into a specific number of clusters. In this case, since it is unclear how many candidate users are included in the silhouette, it is difficult to apply the non-hierarchical method of classifying into a specific number of clusters. Therefore, it is easier to use the former hierarchical method. Among hierarchical clustering methods, there are several methods in detail. For example, methods such as a centroid method, a median method, and a Ward method can be applied.

  Also, assuming that the human body is usually in a posture that is almost vertical, the y value in the height direction of the coordinates of each block does not contribute to the block classification, in other words, a certain block has y It can be seen that the cluster to which the value belongs does not change due to the difference in value. Therefore, clustering may be performed using only the coordinate values in other directions without using the y value.

  In addition, the person and the person are located at a certain distance, that is, the person's head center positions are rarely close to, for example, 40 cm or less. Therefore, in the clustering operation, the Euclidean square distance between the centroids of the clusters is calculated for all combinations of the clusters, and the clustering is terminated when the minimum value among them exceeds a predetermined threshold (for example, 40 cm). The number of clusters remaining at this time is the current number of candidate users, and each cluster is a silhouette for each candidate user. FIG. 25 is a schematic diagram showing the situation. The silhouette is divided into two clusters, a vertically striped region P1 and a horizontally striped region P2. Therefore, the number of candidate users is two, and the two clusters P1 and P2 represent the silhouettes of the candidate users.

  Next, the user three-dimensional information generation unit 20 deletes silhouettes below 24 cm from the top for each cluster (S45). That is, as shown in FIG. 26, the blocks corresponding to the silhouettes P1a and P2a of the region from the top to 24 cm are left for each of the clusters P1 and P2 in FIG. 25, and the blocks below it are deleted. This is because only the head region is extracted in each cluster. Since the length of the human head in the vertical direction (total head height) is about 24 cm on average, this length is used. By this process, only the head region of each candidate user is extracted and remains in each cluster.

  Finally, the user three-dimensional information generation unit 20 performs the process of calculating the center of gravity of each cluster (S46). That is, in each cluster P1a, P2a of FIG. 26, the barycentric positions G1, G2 of all the blocks belonging to the cluster are obtained. Thereby, the center coordinate of the head of each candidate user is calculated. In FIG. 26, black circles G1 and G2 indicate the center positions of the heads of the candidate users. The center position of each candidate user's head can be obtained by using information on the angle of view of the distance measurement camera 9 stored in the camera-related information storage unit 11 and information on the mounting position and orientation of the distance measurement camera 9 with respect to the display 6. , As a value relative to the display 6.

  In the above, in order to simplify the explanation of the processing procedure in the user three-dimensional information generation unit 20, the information in the depth direction in the figure is omitted, and the explanation is given in the two-dimensional space of xy as shown in the lower left of FIG. went. In practice, it goes without saying that the same processing is performed in the three-dimensional space of xyz. At this time, in the description of the clustering (S44) process, it is described that the clustering may be performed using only the value of x without using the value of y. Assuming that the direction is the height direction, the value of y is not used, and clustering may be performed using only the values of x and z.

Next, the process in the user information collation part 21 is demonstrated. As shown in FIG. 27, the user information collation unit 21 performs two processes, which are roughly divided into a current information integration process (S51) and a past information integration process (S52).
First, details of the current information integration process (S51) will be described. Here, the total number of cameras 7 and 8 is l, and the number of face direction estimated values and face position estimated values of the candidate user detected by the user position / orientation calculating unit 19 for the i-th camera is m. Let Pij be the estimated face position value of the jth candidate user for that person. Note that Pij is a three-dimensional coordinate value. On the other hand, the number of candidate users detected by the user three-dimensional information generation unit 20 is n, and the center position of the head of the kth candidate user is Ck. Ck is also a three-dimensional coordinate value.

  In step S51, for the kth candidate user detected by the user three-dimensional information generation unit 20, the user information collation unit 21 performs the integration process illustrated in FIG. This is because the candidate user's face position estimated value Pij and the candidate user's head center position obtained by two different methods, the method in the user position / orientation calculation unit 19 and the method in the user three-dimensional information generation unit 20 This is a process for searching for a value related to the same candidate user with the value of Ck.

First, the user information collating unit 21 sets || Pij−Ck ||, that is, the face position estimated value Pij of the j-th candidate user detected by the i-th camera and the k-th in all combinations of i and j. A distance from the center position Ck of the candidate user's head is obtained (S61).
Next, the user information collating unit 21 collects a set of i and j that satisfies || Pij−Ck || <T (S62). Here, T is a predetermined threshold value, for example, a value of 12 cm is used. As described above, specifically, for example, the center position of the left and right eyes is obtained as the face position estimation value. In this case, the head center position and the face position estimation value are shifted by about the radius of the head. Since the radius of the human head on average is about 8 cm, the i-th camera close to the center position Ck of the head of the k-th candidate user can be determined with a threshold of about 12 cm in consideration of detection errors. It is expected that the face position estimated value Pij of the j-th candidate user detected in step 1 can be selected correctly.

  Next, the user information collation unit 21 determines whether two or more pairs of i and j are collected (S63). This is because there is a possibility that the user position / orientation calculation unit 19 captures the k-th candidate user detected by the user three-dimensional information generation unit 20 and obtains the face position estimation value Pij, respectively. As a result, a plurality of sets of i and j may be collected.

When two or more pairs of i and j are collected in step S63, the user information collating unit 21 sets a unit vector representing the direction of the camera i and a candidate user for each collected group of i and j. An inner product with a unit vector representing the orientation of the face of j is calculated (S64). This is to select data estimated in a situation where the candidate user is facing the camera as much as possible in order to select a pair that is expected to have the highest reliability from a plurality of sets of i and j collected. It is. This is because the estimation accuracy is expected to be higher when the candidate user faces the camera as much as possible because the camera can shoot the face from the front without distortion. How much the candidate user is facing the camera can be determined from the inner product of the unit vector representing the direction of the camera i and the unit vector representing the direction of the face of the candidate user j. When facing completely, the unit vector representing the direction of the camera i and the unit vector representing the direction of the face of the candidate user j are opposite directions, and the inner product thereof is -1. If the face direction is deviated from that, the inner product is larger than -1.
Therefore, the user information collation unit 21 determines the set of i and j whose inner product is closest to −1 as the set corresponding to the kth candidate user (S65).

On the other hand, when two or more pairs of i and j are not classified in step S63, the user information collating unit 21 further determines whether one pair of i and j is classified (S66).
If the result of this determination is that one i, j pair has been classified, the user information collating unit 21 determines that the one i, j pair is a group corresponding to the kth candidate user (S67). ).
On the other hand, as a result of the determination in step S66, if no combination of i and j is classified, the user information matching unit 21 determines that there is no combination corresponding to the kth candidate user (S68). This is because the k-th candidate user detected by the user three-dimensional information generation unit 20 was not in a position where the user position / orientation calculation unit 19 photographed from either the camera 7 or the camera 8. The case is considered.
For the candidate user k for which the corresponding i, j pair is determined, the head position and head direction of the j-th candidate user estimated by the i-th camera are represented by the head position of the candidate user k and Match with head orientation.
The above process is executed for all candidate users detected by the user three-dimensional information generation unit 20.

  Next, details of integration processing (S52) with past information by the user information collating unit 21 will be described. FIG. 29 shows a flow of integration processing by the user information collation unit 21. First, the user information matching unit 21 determines whether there is past information in the user information recording unit 13 (S71). If there is past information as a result of this determination, the user information collating unit 21 uses the current information created in the current information integration process (S51) and the past recorded in the user information recording unit 13. Integration with information and processing for updating information are performed (S72). More detailed processing will be described later. On the other hand, as a result of the determination in step S71, if there is no past information in the user information recording unit 13, the user information collating unit 21 uses the current information created in the current information integration process (S51) as new information. Is recorded as (S73). Usually, there is no past information only once just after the system starts to move.

  Further, the information update process (S72) will be described in more detail. Since the candidate user ID in the current information integration process (S51) is not integrated with past information recorded in the user information recording unit 13, it is referred to as a provisional candidate user in this description. The head position of the provisional candidate user n is Pn, and the head direction is Dn. In step S72, the current information, that is, the provisional candidate user n information at time t created in the current information integration process (S51) and the information of candidate user m at time t ′ recorded in the user information recording unit 13 are processed. Then, the user information collation unit 21 performs the information update process shown in FIG. This is because the candidate user may move and have different positions at the time t ′ and the time t, so that the correspondence between the candidate user at the time t ′ and the temporary candidate user at the time t is calculated while considering the difference. It is.

  First, the user information collation unit 21 calculates || P_m (t ′) − Pn || for all combinations of m and n (S81). Next, the user information collation unit 21 searches for n that satisfies || P_m (t ′) − Pn || <T2 for each m (S82). Here, T2 is a predetermined threshold value. This value can be determined based on the normally assumed movement speed of the human head and how many times the system acquires candidate user information per second (processing refresh rate).

  For example, it is assumed that the moving speed of the human head is a maximum of 120 cm / sec and the processing refresh rate is 30 times / sec. At this time, the unit time of the process is t−t ′ = 1/30 = 33.33 msec, and the maximum movement amount of the human head per unit time of the process is 120/30 = 4 cm. Therefore, for example, if T2 = 4 cm, P_m (t ′) and Pn satisfying || P_m (t ′) − Pn || <T2 are highly likely to be data relating to the same candidate user. If T2 = 4 cm, the value is sufficiently smaller than the head diameter of about 16 cm, and the possibility of erroneously recognizing data of another candidate user in the vicinity is low.

  In other words, this means that there is a possibility that the data may be mistaken as data of another candidate user in the vicinity unless the refresh rate is fast to some extent. For example, when the processing refresh rate is 5 times / sec, the maximum movement amount of the human head per unit time of processing is 120/5 = 24 cm. This is a value larger than the diameter of the human head of about 16 cm, and it can be seen that there is a high possibility that an error will occur.

  Next, the user information collation unit 21 determines whether or not n that matches the condition exists for each m in the process of step S82 (S83). As a result of this determination, if it exists, the user information matching unit 21 associates the temporary candidate user n with the candidate user m as the same person for the set of m and n (S84). Subsequently, the user information matching unit 21 records P_m (t) = Pn and D_m (t) = Dn in the user information recording unit 13 (S85). That is, data whose association is found is associated with past data and recorded as the latest data. Furthermore, the user information collation unit 21 updates T_m and W_m (S86). Specifically, for T_m, nothing is performed when T_m = 0, and T_m + (t−t ′) is overwritten on T_m when T_m ≠ 0. Similarly, for W_m, nothing is performed when W_m = 0, and W_m + (t−t ′) is overwritten on W_m when W_m ≠ 0.

  On the other hand, as a result of the determination in step S83, if n that matches the condition does not exist in each m, the user information matching unit 21 deletes information from all the candidate users m from the user information recording unit 13. (S87). This occurs when a candidate user goes out of the detection range of the system and information cannot be obtained. After the process of step S86 or step S87 is completed, it is determined whether there is an object that is not associated with any data at time t 'in the data at time t (S88). In other words, it is determined whether or not there is a provisional candidate user n that is not associated with any of the candidate users m among the provisional candidate users n.

  If there is a provisional candidate user n that satisfies the condition of step S88, that is, it indicates a new candidate user that does not exist in the past data, the user information matching unit 21 determines that the provisional candidate user n Information is added and recorded in the user information recording unit 13 as a new candidate user (S89). Specifically, in the data recorded in the user information recording unit 13 shown in FIG. 4, a new ID number is added to the candidate user ID list, and information (T_n, W_n, N_n, time list and Record the subordinate information). At this time, T_n = 0, W_n = t−t ′, N_n = 0, and initial values are substituted. When there are a plurality of provisional candidate users that satisfy this condition, all of them are added and recorded as new candidate users.

  Returning to FIG. 3, the video generation unit 16 receives the information on the currently selected user sent from the user selection / release processing unit 15 and the size of the display portion of the display 6 stored in the display related information storage unit 17. The viewpoint position of the selected user using the resolution information, the model data for displaying the 3D CG stored in the 3D CG model data storage unit 18 and the virtual arrangement information of those models. Are combined and output to the display 6.

  The video output to the display 6 may present the motion parallax reproduction video on the entire screen, or, as shown in FIG. 31, the motion parallax reproduction video region 22 which is a part of the screen displays the motion parallax reproduction video. In the other normal video area 23, the normal video, that is, the video that does not depend on the head position of the selected user may be presented. In the latter case, the video displayed in the motion parallax reproduction video area 22 can be viewed appropriately only by the selected user and is difficult for other users to see, but the video displayed in the normal video area 23 is open to anyone. Looks normal. Thus, for example, if an article to be advertised is presented in the motion parallax reproduction video area 22 using motion parallax video processing, and an explanatory text of the article is displayed in the normal video area 23, at least the explanatory text of the article is Anyone can read it, and the selected user can see the article from various directions.

  In addition, an example of display when the screen is divided into two left and right regions and motion parallax reproduction images are provided to a total of two users, one each, will be described with reference to FIG. FIG. 32 shows a situation where there are two selected users of the selected user 51 and the selected user 52 in front of the display 6. At this time, the screen of the display 6 is divided into two areas, which are a partial display area 24 and a partial display area 25, respectively. A video generated according to the head position of the selected user 51 is presented in the partial display area 24, and a video corresponding to the head position of the selected user 52 is presented in the partial display area 25. By doing so, it is possible to simultaneously show an advertisement using motion parallax reproduction video to two selected users. Furthermore, it is possible to increase the number of divisions, such as 3 divisions, 4 divisions,..., And to show advertisements using motion parallax reproduction images to more selected users at the same time. Since the display area becomes small and difficult to see, it is necessary to determine the number of divisions appropriately in consideration of the size of the display 6 and the display contents.

  Motion parallax reproduction video presentation system capable of providing motion parallax reproduction video to two users at maximum by dividing the screen into two areas in the first embodiment, that is, maximum selection in the first embodiment A specific example of how display is performed when the selected user changes with the passage of time will be described by taking as an example a motion parallax reproduction video presentation system in which the possible number Max = 2. FIG. 33 shows a display that can divide the screen shown in FIG. 32 into two areas, and three viewpoint positions A, B, and C, and FIG. 34 is a view from above. . On the other hand, it is assumed that the selected user at certain times t1, t2, t3, t4, and t5 (where t1 <t2 <t3 <t4 <t5) changes as follows.

(1) Time t1: There is no selected user at any of positions A, B, and C.
(2) Time t2: Selected user U1 comes to position B.
(3) Time t3: The selected user U1 remains at the position B. The selected user U2 has newly arrived at the position A.
(4) Time t4: The selected user U1 remains at the position B. The selected user U2 who was at position A has left.
(5) Time t5: The selected user U1 remains at the position B. The selected user U3 has newly arrived at position C.

In this case, the display at each time is performed as follows, for example.
(1) Time t1: The number U of selected users is 0. Since no user is selected, the motion parallax reproduction video is not presented. For example, a normal advertisement video that does not reproduce motion parallax is played.
(2) Time t2: The number U of selected users is 1. Therefore, the screen is not divided, and the motion parallax reproduction video directed to the selected user U1 at the position B is presented on the entire screen.
(3) Time t3: The number of selected users U is 2. Therefore, the screen is divided into two areas, and the video shown to the selected user U1 at position B at time t2 is reduced or trimmed and moved to the partial display area 25. In the partial display area 24, presentation of the motion parallax reproduction video toward the selected user U2 at the position A is started. In FIG. 34, since the selected user U2 is located at the position A and the selected user U1 is located at the position B from the left side in FIG. 34, the screen is also displayed in the partial display area 24 from the left side to the selected user U2. This is because it is easier for both the selected users to view the directed video in the partial area 25 than to display the video directed to the selected user U1.

(4) Time t4: The number U of selected users is 1. Since the selected user U2 at the position A has left, the display of the motion parallax reproduction video presented in the partial display area 24 is stopped. The motion parallax reproduction video presented in the partial display area 25 toward the selected user U1 at the position B at time t3 is displayed with the display area expanded over the entire screen.
(5) Time t5: The number of selected users U is 2. Therefore, the screen is divided again into two areas, and the video shown to the selected user U1 at position B using the entire screen at time t4 is reduced or trimmed and moved to the partial display area 24. In the partial display area 25, the presentation of the motion parallax reproduction image for the selected user U3 newly coming to the position C is started. In FIG. 34, since the selected user U1 is located at the position B and the selected user U3 is located at the position C in order from the left side, the screen is also displayed in the partial display area 24 in the partial display area 24 from the left side. This is because it is easier for both the selected users to view the directed video in the partial area 25 than to display the video directed to the selected user U3.

  In this way, the number and position of displaying the motion parallax reproduction video are changed and displayed according to the number and position of the selected users at each time. In the above example, the selected user U1 is always at the position B at any time, but depending on the presence and position of other selected users, the entire screen, the partial display area 25, the entire screen, and the partial area at each time It will change to 24.

  In the above embodiment, for the sake of simplicity, the video displayed on the display is generated using the three-dimensional CG model, but it can also be applied to a live-action video. For example, a large number of videos may be captured and stored by a camera array in which a large number of cameras are arranged in advance, and an appropriate video may be selected according to the viewpoint position of the selected user and displayed. Alternatively, three-dimensional image data may be generated from a multi-viewpoint image and a multi-viewpoint distance image, and a video viewed from an arbitrary viewpoint position may be generated using the data.

[Second Embodiment]
The second embodiment of the present invention is a motion parallax reproduction video presentation system similar to the first embodiment, and the hardware configuration is also shown in FIGS. 1 and 2 as in the first embodiment. . The difference is the processing in the motion parallax reproduction video generation device 10. The user selection / release processing unit 15 in the first embodiment shown in FIG. 3 is the same as the user in the second embodiment as shown in FIG. The selection processing unit 26 (user selection unit) is changed. That is, the user selection processing unit 26 does not perform the release process that was in the user selection / release processing unit 15. 35, the same reference numerals (6 to 14, 16 to 18) are assigned to portions corresponding to the respective portions in FIG. 3, and the description thereof is omitted.

  Details of processing in the user selection processing unit 26 are shown in FIG. This corresponds to the processing (FIG. 8) in the user selection / release processing unit 15 in the first embodiment. First, the user selection processing unit 26 performs a selection process for a selected user (S91). Details will be described later. Next, the user selection processing unit 26 determines whether or not a predetermined threshold T3 seconds has elapsed after selection (S92). When the time has elapsed, the process returns to the top of the process, and the user selection processing unit 26 executes the selection process of the selected user (S91) again. That is, a process of reselecting the selected user is performed every predetermined time T3 seconds set in advance. When the time of T3 seconds has not yet elapsed in step S92, the user selection processing unit 26 performs a selected user position information transmission process (S93). This is because the position information P_n (t) is sent to the video generation unit 16 for all users who have T_n ≠ 0 in the information on the candidate users recorded in the user information recording unit 13, that is, the selected users, This is processing for sending the number U of selected users to the video generation unit 16.

  Details of the selection process (S91) of the selected user by the user selection processing unit 26 are shown in FIG. This corresponds to the details (FIG. 9) of the selected user addition processing (S2) in the first embodiment, and the processing is similar. Therefore, for ease of explanation, in FIG. 37, the reference numeral of the process corresponding to FIG. 9 is the same as that of FIG. 9, and the reference numeral is used when there is no corresponding process of FIG. Not. First, the user selection processing unit 26 calculates the fitness F_n (t) for all candidate users (S102). Here, the fitness F_n (t) is the same as that described in the first embodiment, and the calculation method thereof is also the same.

  Next, the user selection processing unit 26 excludes candidate users satisfying F_n (t) <FConst (S103). This is a process aimed at avoiding unnecessary services from avoiding a candidate user whose fitness is not more than a predetermined threshold FConst set in advance, that is, a candidate user not suitable as a selected user. Let k be the number of candidate users remaining in the process of step S103. Next, the user selection processing unit 26 determines whether or not k> 0 (S104). As a result of this determination, if k> 0 is not true, that is, if No, the user selection processing unit 26 ends the process as it is.

  Further, as a result of the determination in step S104, if k> 0, that is, if Yes, the user selection processing unit 26 determines whether or not k ≧ Max (S105). As a result of this determination, if k ≧ Max, that is, if Yes, the user selection processing unit 26 selects a Max name candidate user in the descending order of the fitness F_n (t) and sets it as a selected user. Perform (S106). More specifically, the user selection processing unit 26 applies the data regarding the candidate users with the Max name in the descending order of the fitness F_n (t) in the candidate user information recorded in the user information recording unit 13. , W_n = 0, N_n = N_n + 1. This means that the current waiting time W_n is set to 0 because it has left the selection waiting state, and 1 is added to N_n because it has been selected. Thus, the user selection processing unit 26 ends the process.

  On the other hand, if k ≧ Max is not satisfied as a result of the determination in step S105, that is, if No, the user selection processing unit 26 sets all remaining k candidate user names as selected users (S108). More specifically, as in the processing in step S106, the user selection processing unit 26 applies the remaining data to the additional processing target user k in the candidate user information recorded in the user information recording unit 13. , W_n = 0, N_n = N_n + 1. The meaning is the same as that of step S106. Thus, the user selection processing unit 26 ends the process.

  As described above, every time the predetermined time T3 seconds elapses, the process of reselecting the selected user is performed, so that it is not specified using the information presentation system that can present information only to a finite number of users at the same time. In the case of presenting information to a large number of users, an effect that information can be presented to a larger number of users can be obtained. Furthermore, when the selected user is reselected, by using the fitness F_n (t) as an index, it is possible to increase the probability that information is transmitted.

[Third Embodiment]
In the first embodiment and the second embodiment described above, a motion parallax reproduction video presentation system has been described as an example of an information presentation system that can present information to only a limited number of users at the same time. In the embodiment, an advertisement display system that repeatedly reproduces an advertisement video (moving image) for a certain length of time as an advertisement on the street will be described as an example.

  In such an advertisement display system, a person who came to the place during the playback of the advertisement video and watched it saw the advertisement video from the beginning again after the video playback was finished, and watched until he started watching Or it will be seen again to the end. In this case, the former will see the first half after seeing the second half of the advertising video, and it is difficult to understand the contents of the advertising video because the video is not viewed in the correct order. After viewing the video, the entire advertisement video is viewed again, and the content of the advertisement video is easy to understand, but a longer time is required to view the entire video. Therefore, in this embodiment, by recognizing a person who seems to be interested in the video advertisement, and starting playback of the video toward that person, any person can view the video from the beginning. It can be so.

FIG. 38 is a diagram illustrating a hardware configuration of the advertisement display system according to the present embodiment. 1 is substantially the same as the hardware configuration of the first embodiment of the present invention shown in FIG. 1, but the advertisement display system in FIG. 38 is configured such that the motion parallax reproduction video generation device 10 in FIG. The difference from the motion parallax reproduction video presentation system of FIG. 1 is that the playback device 27 (information presentation device) is changed.
FIG. 39 is a schematic block diagram showing the configuration of the advertisement video reproduction device 27. As shown in FIG. The advertisement video reproduction device 27 corresponds to the motion parallax reproduction video generation device 10 (FIG. 3) in the first embodiment. 39, the advertisement video reproduction device 27 includes a change in the video generation unit 16 to a video reproduction unit 28, a change in the 3D CG model data storage unit 18 to a video data storage unit 29, and a display related information storage unit 17. It is different from the reproduction motion parallax image generation device 10 in FIG. In FIG. 39, the same reference numerals (6 to 9, 11 to 15) are assigned to the portions corresponding to the respective portions in FIG.

  The video data storage unit 29 stores advertisement video data. When there is a user who is newly selected as the selected user using the information related to the selected user sent from the user selection / release processing unit 15, the video reproduction unit 28 stores the video data storage unit 29. The reproduction of the advertisement video data is started and displayed on the display 6. The information sent from the user selection / release processing unit 15 to the video data storage unit 29 is recorded in the user information recording unit 13 as described in the description of the selected user information transmission process (S4) in FIG. The ID of the selected user, the position information P_n (t) thereof, and the number U of the currently selected users regarding all the users who are T_n ≠ 0 in the candidate user information, that is, the selected users.

  In addition, since the present invention can also select a plurality of selected users, for example, a display presenting a video can be divided into a plurality of regions as appropriate so that video playback can be started at different timings. By using the present invention together with the system, it becomes possible to simultaneously present advertisement videos to a plurality of people at appropriate timings. For example, in the third embodiment, the screen of the display 6 is divided into two regions and a system that can provide advertisement video to a maximum of two users, that is, the maximum selectable number Max = 3 in the third embodiment. Taking the system 2 as an example, an example of how the display is specifically performed when the selected user changes as time passes will be described. In this description, FIG. 33 and FIG. 34 are used as in the same description in the first embodiment. On the other hand, it is assumed that the selected user at certain times t1, t2, t3, t4, and t5 (where t1 <t2 <t3 <t4 <t5) changes as follows.

(1) Time t1: There is no selected user at any of positions A, B, and C.
(2) Time t2: Selected user U1 comes to position B.
(3) Time t3: The selected user U1 remains at the position B. The selected user U2 has newly arrived at the position A. A candidate user also came at position C, but is not yet a selected user.
(4) Time t4: The selected user U1 remains at the position B. The selected user U2 who was at position A has left. Instead, the candidate user who was in the position C was selected as the selected user U3.
(5) Time t5: The reproduction of the advertisement video for the selected user U1 who is at the position B is finished, and the selected user U1 is released from the selected user. The selected user U3 remains at the position C.

In this case, the display at each time is performed as follows, for example.
(1) Time t1: The number U of selected users is 0. Since no user is selected, for example, a still image is displayed. Or, even if there is no viewer, the advertisement video is played.
(2) Time t2: The number U of selected users is 1. Therefore, the division of the screen is not performed, and the reproduction of the advertisement video is started toward the selected user U1 who is at the position B in the entire screen.
(3) Time t3: The number of selected users U is 2. Therefore, the screen is divided into two areas, and the advertisement video shown to the selected user U1 at position B at time t2 is reduced or trimmed and moved to the partial display area 25. At this time, the moving is performed while the reproduction of the advertisement video is continued. In the partial display area 24, the reproduction of the advertisement video is started toward the selected user U2 in the position A. In FIG. 34, since the selected user U2 is located at the position A and the selected user U1 is located at the position B from the left side in FIG. 34, the screen is also displayed in the partial display area 24 from the left side to the selected user U2. This is because it is easier for both the selected users to view the directed video in the partial area 25 than to display the video directed to the selected user U1.

(4) Time t4: The number of selected users U is 2. Since the selected user U2 at the position A has left, the advertisement video presented in the partial display area 24 stops playing even if it is in the middle of playing. The advertisement video presented in the partial display area 25 toward the selected user U1 at position B at time t3 is moved to the partial display area 24. At this time, the moving is performed while the reproduction of the advertisement video is continued. In the partial display area 25, the reproduction of the advertisement video is started toward the selected user U3 newly coming to the position C. In FIG. 34, since the selected user U1 is located at the position B and the selected user U3 is located at the position C in order from the left side, the screen is also displayed in the partial display area 24 in the partial display area 24 from the left side. This is because it is easier for both the selected users to view the directed video in the partial area 25 than to display the video directed to the selected user U3.
(5) Time t5: The number of selected users U is 1. The advertisement video presented in the partial display area 25 toward the selected user U3 at the position C at time t4 continues to be displayed with the display area expanded over the entire screen.

  In this way, the number and position of the advertisement video are changed and displayed according to the number and position of the selected users at each time. In the above example, it is realized that all of the selected users U1, U2, U3 see the advertisement video from the beginning. Further, since the selected user U1 was always there, the advertisement video could be viewed from the beginning to the end. On the other hand, regarding the selected user U2 who left in the middle of the reproduction of the advertisement video, the reproduction is immediately stopped, and instead, the reproduction of the advertisement video for the selected user U3 is immediately started. In other words, since the video can be presented to any selected user from the beginning of the advertising video and does not start to be seen from the middle, the content of the advertisement is easy to understand, and the information transmission efficiency is high. In addition, since the reproduction of the advertisement video to the selected user who leaves on the way stops immediately, and the information presentation system selects the other candidate user and starts the reproduction of the advertisement video, the information presenting system can transmit the total amount of information to the entire user. There is an effect that it is possible to increase the amount of information or to increase the information transmission efficiency.

  In addition, since this 3rd Embodiment is not a motion parallax reproduction image presentation system, the situation where a user enjoys the change of appearance by moving a head to the left and right does not generate | occur | produce. Therefore, in Equation 1 for calculating the fitness F_n (t), the term related to the user behavior M_n (t) may be deleted. Alternatively, since it is considered that the action of seeing the advertisement from the back of the fence is reflected, a term relating to the user action degree M_n (t) may be left.

[Fourth Embodiment]
In the fourth embodiment of the present invention, as an information presentation system capable of presenting information only to a finite number of users at the same time, an audio guide that repeatedly plays an audio guide for a certain length of time in an art museum or a museum. A system will be described as an example. If the voice guide is being played repeatedly, the visitor who came to the location in the middle of the voice guide will hear from the beginning again after it has finished, and will listen to the point where he / she started listening to the middle or to the end again become. In this case, since the former does not listen to the explanation in the correct order, there is a problem that it is difficult to understand. In the latter, a longer time is required to hear the whole explanation. Therefore, in this embodiment, by recognizing a visitor who seems to be close to the exhibition and interested in the tour, and starting playback of the voice guide for that visitor, every visitor starts the voice guide at the beginning. To be able to hear from.

  The voice guidance system according to the present embodiment is equipped with a super directional speaker that outputs a sound with very strong directivity using ultrasonic waves on a support base that can change the direction, and follows the movement of the user. To change the direction of loud sound. This voice guidance system can select a user from among a plurality of people and present the voice guide so that only the user can hear it. Furthermore, if multiple super directional speakers mounted on a support base that can change the orientation are used, voice guides can be presented to multiple visitors at the same time at appropriate timings. It becomes possible.

  Thus, the fourth embodiment of the present invention is an audio guide system that repeatedly reproduces audio information of a certain length of time as an audio guide in a museum or museum. FIG. 40 is a conceptual diagram showing a hardware configuration of the voice guidance system in the present embodiment. 1 is almost the same as the diagram illustrating the hardware configuration of the first embodiment of the present invention shown in FIG. 1, but the voice guidance system in FIG. 40 has the display 6 and the camera 7 in FIG. 1 is different from the motion parallax reproduction video system in FIG. 1 in that the parallax reproduction video generation device 10 is changed to an audio reproduction device 31 (information presentation device) and a speaker 30 with a support base (information presentation unit) is added. In addition, although it is not an element which comprises an audio | voice guidance system, the display object 32 used as the object of audio | voice guidance is also described in FIG. 40, the same reference numerals (5, 8, 9) are assigned to portions corresponding to the respective portions in FIG. 1, and description thereof is omitted.

Since the present embodiment is an audio guide system related to exhibits in art museums and museums, the display 6 is unnecessary. The camera 7 installed on the display 6 was also excluded. As described above, there is no significant functional difference between the camera 7 and the plurality of cameras 8, and the operation of this system is possible even without the camera 7.
The speaker 30 with a support base is a super directional speaker that outputs a sound having a very strong directivity using ultrasonic waves mounted on a support base capable of changing the direction, and is installed on the ceiling. The direction can be controlled from the outside (sound reproduction device 31). Although not filled in in order to avoid complication of the figure, the speaker 30 with a support base is connected to the sound reproducing device 31.

  In FIG. 40, two speakers 30 with support bases are shown, but the number is not limited to this number. A larger or smaller number of speakers 30 with support bases may be installed as necessary. When two speakers 30 with support bases are installed as shown in FIG. 40 and each presents a voice to one selected user, the voice information should be simultaneously presented to a maximum of two selected users. That is, it is a system with a maximum selectable number Max = 2. However, a single speaker 30 with a support base does not necessarily present voice to one selected user. For example, when presenting a voice to one selected user using two speakers 30 with support bases, or toward one selected user using a larger number of speakers 30 with support bases. In some cases, voice is presented. Since the position of the speaker 30 with the support base is fixed, it becomes difficult to present the sound appropriately when the selected user is far away. Therefore, it is possible to respond to any position of the selected user by presenting the voice to one selected user using the plurality of speakers 30 with the support bases at various positions.

The audio reproduction device 31 performs processing for detecting the positions and orientations of a plurality of candidate users from the image information and distance information captured by the plurality of cameras 8 and the plurality of distance measuring cameras 9, and one of the plurality of candidate users is selected. In addition, a process of selecting / releasing several names is performed, and voice data and direction control data are output to the speaker 30 with a support base so that voice is reproduced at the position of the selected user.
Note that the exhibit to which the audio guide adds explanation is shown in FIG. This is installed in place of the display 6 in FIG. 1 and corresponds to the display 6 from the viewpoint of an object to be viewed by the user.

  FIG. 41 is a schematic block diagram showing the configuration of the audio playback device 31. This corresponds to the motion parallax reproduction video generation device 10 (FIG. 3) in the first embodiment. The difference from the motion parallax video generation device 10 of FIG. 3 is that there is no input from the camera 7, the video generation unit 16 is in the audio reproduction unit 33, and the display related information storage unit 17 is in the speaker related information storage unit 34. That is, the three-dimensional CG model data storage unit 18 is changed to the voice data storage unit 35, and the output destination from the voice generation unit 33 is changed to the speaker 30 with a support base. The data recorded in the system information recording unit 14 is partly different from the description in the first embodiment. These will be described in detail below.

  As described with reference to FIG. 5 in the description of the first embodiment, the system information recording unit 14 indicates the maximum selectable number Max, the currently selected number U, the position Ps, and the direction Ds in the audio. Store as information on the guide system. Among these, the position Ps and the direction Ds are information on the position and the direction regarding the display 6 in the first embodiment, and more specifically, the position Ps is the display of the motion parallax reproduction video presentation system. 6 is the coordinate of the center of the screen, and the direction Ds is a vector representing the direction in which the screen of the display 6 of the motion parallax reproduction video presentation system is facing, and the unit in which the video output side of the display 6 is positive The normal vector value was recorded.

  In the fourth embodiment, instead of these, information on the position and orientation of the exhibit 32 is recorded as the position Ps and the orientation Ds. The method of obtaining the specific value varies depending on the type of the exhibit. For example, when the exhibit is a drawing, for example, the position Ps is a coordinate at the center of the drawing, and the direction Ds is a unit normal vector in which the direction in which the drawing of the drawing can be seen is positive. For example, when the exhibit is a fossil or the like in a glass case, for example, the position Ps is set to the coordinates of the center of the front surface of the glass case, and the direction Ds is directed to the outside of the front surface of the glass case. Let the unit normal vector be positive in the direction of.

The speaker related information storage unit 34 holds information on the position of the speaker 30 with a support base. When there are a plurality of speakers 30 with support bases, information is held for each of them.
The voice data storage unit 35 stores voice data of a voice guide related to the exhibit 32.
The sound reproducing unit 33 determines whether or not there is a user newly selected as the selected user by using the information regarding the selected user sent from the user selection / release processing unit 15. If the result of this determination is that there is a newly selected user, the voice playback unit 33 starts playback of the voice guide voice data stored in the voice data storage unit 35 and the sound The direction of the speaker 30 with a support base is controlled so that it can be heard by the selected user. When the selected user moves, the direction of the speaker 30 with the support base is controlled accordingly.

  The information sent from the user selection / release processing unit 15 to the audio reproduction unit 33 is the candidate stored in the user information recording unit 13 as described in the description of the selected user information transmission processing (S4) in FIG. These are the ID number of the selected user, the position information P_n (t), and the current number U of the selected users for all users whose information is T_n ≠ 0, that is, the selected users.

  Here, in the fourth embodiment, a voice guide system that includes two speakers 30 with support bases, each presenting a voice to one selected user, that is, the maximum selectable number of persons in the fourth embodiment. Taking a voice guidance system with Max = 2 as an example, an example of how voice presentation is specifically performed when the selected user changes as time passes will be described. FIG. 42 shows three viewpoint positions of two speakers 30 with a support base (a speaker 301 with a support base and a speaker 302 with a support base), an exhibit 32, and A, B, and C. 43 is a view from above. It is assumed that the two speakers with support bases can present audio to any of the three viewpoint positions A, B, and C. On the other hand, it is assumed that the selected user at certain times t1, t2, t3, t4, and t5 (where t1 <t2 <t3 <t4 <t5) changes as follows.

(1) Time t1: There is no selected user at any of positions A, B, and C.
(2) Time t2: Selected user U1 comes to position B.
(3) Time t3: The selected user U1 remains at the position B. The selected user U2 has newly arrived at the position A. A candidate user also came at position C, but is not yet a selected user.
(4) Time t4: The selected user U1 remains at the position B. The selected user U2 who was at position A has left. Instead, the candidate user who was in the position C was selected as the selected user U3.
(5) Time t5: The reproduction of the voice guide for the selected user U1 at the position B is finished, and the selected user U1 is released from the selected user. The selected user U3 remains at the position C.
In this case, presentation of the voice guide at each time is performed as follows, for example.

(1) Time t1: The number U of selected users is 0. Since no user is selected, the audio guide is not reproduced.
(2) Time t2: The number U of selected users is 1. Using the speaker 301 with a support base, the direction is controlled toward the selected user U1 at the position B, and the reproduction of the voice guide is started.
(3) Time t3: The number of selected users U is 2. At time t2, the voice guide that has been sent to the selected user U1 at position B using the speaker 301 with the support base continues to be reproduced as it is. In addition, using the speaker 302 with a support base, the direction is controlled toward the selected user U2 at the position A, and the reproduction of the voice guide is started.

(4) Time t4: The number of selected users U is 2. At time t1 and t2, the voice guide that has been sent to the selected user U1 at position B using the speaker 301 with the support base continues to be reproduced. Since the selected user U2 who has been at the position A has left, the voice guide that has been heard by the selected user U2 at the position A using the speaker 302 with the support base stops. Instead, the direction is controlled toward the selected user U3 at the position C by using the speaker 302 with the support base, and the reproduction of the voice guide is started.
(5) Time t5: The number of selected users U is 1. At time t4, the audio guide that has been sent to the selected user U3 at position C using the speaker 302 with the support base continues to be reproduced as it is.

  Thus, according to the number and position of the selected users at each time, the number and position of the voice guides are changed and presented. In the above example, it is realized that all of the selected users U1, U2, U3 listen to the voice guide from the beginning. Moreover, since the selected user U1 was always there, he was able to listen to the voice guide from the beginning to the end. On the other hand, for the selected user U2 who left in the middle of the reproduction of the voice guide, the reproduction is immediately stopped, and instead, the reproduction of the voice guide for the selected user U3 is immediately started. That is, since it can be presented to the selected user from the beginning of the voice guide and does not begin to be heard in the middle, it is easy to understand the contents of the voice guide, and thus the probability that information is transmitted can be increased. There is an effect. In addition, since the reproduction of the voice guide to the selected user who leaves in the middle stops immediately, and another candidate user is selected and the reproduction of the voice guide is started, information can be presented to more users. There is.

By the way, at time t3 described above, the speaker 301 with a supporting base presents a voice guide to the selected user U1 at the position B, and the speaker 302 with a supporting base speaks toward the selected user U2 at the position A. In the situation where the guide is presented, the relationship between the speaker with the support base and the selected user has crossed, but there is no problem in operation.
In addition, since this 4th Embodiment is also not a motion parallax reproduction image presentation system, the situation where a user enjoys the change of appearance by moving a head to the left and right does not generate | occur | produce. Therefore, in Equation 1 for calculating the fitness F_n (t), the term related to the user behavior M_n (t) may be deleted. Alternatively, since it is considered that the action of looking at the exhibit from the back of the fence is reflected, a term relating to the user action degree M_n (t) may be left.

  Also, the user information detection unit 12, the user selection / release processing unit 15, the video generation unit 16 in FIG. 3, and the user information detection unit 12, the user selection processing unit 26, the video generation unit 16, and the user in FIG. 39 in FIG. The information detection unit 12, the user selection / release processing unit 15, the video generation unit 28, and a program for realizing the functions of the user information detection unit 12, the user selection / release processing unit 15, and the audio reproduction unit 33 in FIG. The processing of each unit may be performed by recording the program on a readable recording medium, reading the program recorded on the recording medium into a computer system, and executing the program. Here, the “computer system” includes an OS and hardware such as peripheral devices.

  The “computer-readable recording medium” refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, and a CD-ROM, and a storage device such as a hard disk built in the computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory in a computer system serving as a server or a client in that case, and a program that holds a program for a certain period of time are also included. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes design changes and the like within a scope not departing from the gist of the present invention.

  The present invention relates to a motion parallax reproduction video presentation system that performs digital signage using motion parallax reproduction video, an advertisement display system that performs digital signage using a certain length of advertisement video, and a voice guide of a certain time length related to exhibits However, the present invention is not limited to this.

DESCRIPTION OF SYMBOLS 5 ... Candidate user 6 ... Display 7 ... Camera 8 ... Camera 9 ... Distance measuring camera 10 ... Motion parallax reproduction image generation device 11 ... Camera related information storage part 12 ... User information detection part 13 ... User information recording part 14 ... System information recording Unit 15 ... User selection / release processing unit 16 ... Video generation unit 17 ... Display related information storage unit 18 ... 3D CG model data storage unit 19 ... User position / orientation calculation unit 20 ... User 3D information generation unit 21 ... User information Verification unit 26 ... User selection processing unit 27 ... Advertising video playback device 28 ... Video playback unit 29 ... Video data storage unit 30 ... Speaker with support 31 ... Audio playback device 33 ... Audio playback unit 34 ... Speaker related information storage unit 35 ... Audio data storage

Claims (13)

An information presentation device that presents information to an unspecified number of users using an information presentation unit that can present information only to a finite number of users at the same time,
A user state detector for detecting the state of the user;
An information presentation apparatus comprising: a user selection unit that sequentially changes a user to whom information is presented based on a detection result of the user state detection unit.   2. The user selection unit according to claim 1, wherein the user selection unit sequentially changes the target user by repeatedly selecting and releasing the target user based on a detection result of the user state detection unit. Information presentation device. The user selection unit is presented with information by the information presenting unit, information indicating the degree of interest in the information presented by the information presenting unit, information indicating the ease of transmission of the information presented by the information presenting unit, and the information presenting unit. And generating at least one of the information indicating the number of times for each user based on the detection result of the user state detection unit, and selecting the target user based on the generated information. The information presentation device according to claim 1 or 2. The user selection unit includes information indicating a selected time, information indicating a degree of interest in information presented by the information presentation unit, and information indicating ease of transmission of information presented by the information presentation unit. Generating at least one of the information indicating the number of times the information is presented by the information presenting unit based on the detection result of the user state detecting unit for each selected user, and based on the generated information The information presentation device according to claim 2, wherein a user to be released is determined.   The information presentation apparatus according to claim 3 or 4, wherein the information indicating the degree of interest or the information indicating the ease of transmission of the information includes information indicating a position where the user is present.   The information presentation apparatus according to claim 3 or 4, wherein the information indicating the degree of interest or the information indicating the ease of transmission of the information includes information indicating the orientation of the user.   The information presentation apparatus according to claim 3, wherein the information indicating the degree of interest or the information indicating how easily the information is transmitted includes information indicating a distance between the user and the information presentation unit.   The information presentation apparatus according to claim 3, wherein the information indicating the degree of interest or the information indicating how easily the information is transmitted includes information indicating the degree of stillness of the user.   The information presentation apparatus according to claim 3, wherein the information indicating the degree of interest includes information indicating a movement of the head of the user.   The information presentation apparatus according to claim 3, wherein the information indicating the degree of interest includes information indicating a waiting time when the user is not selected.   The information indicating apparatus according to claim 4, wherein the information indicating the degree of interest includes information indicating a change in distance from the information presenting unit since the user is selected. The information presentation device according to claim 2, wherein the user selection unit releases the selected user when a preset time has elapsed since the selection. An information presentation method in an information presentation device that presents information to an unspecified number of users using an information presentation unit that can present information only to a finite number of users at the same time,
A first process in which the information presentation device detects the state of the user;
The information presentation method includes: a second process in which the information presentation unit sequentially changes a target user to present information based on a detection result of the first process.

Images