JP4110323B2 - Information output method and apparatus, program, and computer-readable storage medium storing information output program - Google Patents

Description

  The present invention relates to an information output method and apparatus, a program, and a computer-readable storage medium storing the information output program, and more particularly to a still image according to the operation of one or more persons on a large screen. The present invention relates to an information output method and apparatus for dynamically changing the output of moving images, character information, etc., a program, and a computer-readable storage medium storing an information output program.

  Due to advances in computer processing capacity and cheaper displays, information provision using plasma displays, projectors, and the like has begun in public spaces such as station premises and building entrances.

  On the other hand, attempts have been made to dynamically change information output in accordance with human movements due to the development of sensor technology (see, for example, Patent Documents 1 and 2 and Non-Patent Document 1). Such technologies include detecting the position and movement of a person using an image captured by a surveillance camera, and specifying an attribute (gender, age, etc.) associated with the person using a wireless tag that transmits an ID. There are those that acquire sound using a microphone, and are used mainly in entertainment facilities and pavilions.

  An object of the present invention is to provide optimal information (especially advertisement information) according to the movement of a person in a large-scale space such as a station premises or a building entrance. Specific examples are shown below.

・ Floating advertisement:
As shown in FIG. 11, the floating advertisement outputs one or a plurality of information display windows for displaying information such as advertisements on a large screen. The output is changed as follows according to the movement of the person.

  -Increase the size of the window when a person passes in front of the window.

  -Display detailed information when a person stops.

  -A map, coupon information, or the like is displayed when an operation such as stagnation for a long time, pointing, or making a voice is performed.

  -The information displayed in each window is selected based on the ID of the detected wireless tag. For example, when there is a person nearby who has a wireless tag associated with a man, advertisement information for men is displayed.

・ Map advertisement:
As shown in FIG. 12, the map advertisement displays a map on the background of the large screen in the initial state. The output is changed as follows according to the movement of the person.

  -Pop up an information display window when a person stops in front of a predetermined position on the map.

  -Display information about the store at the corresponding location.

  -Display detailed information when a person approaches the screen.

In these information providing methods, information suitable for a person can be provided in accordance with the natural operation of the person. For example, for those who are not interested, provide information that attracts interest, for those who are interested, detailed information, and for those who are interested, maps and coupons to guide you to the store Information can be provided as appropriate.
JP 2001-43384 A "Multimedia Information Space Input Method, Input Device, Input / Output Cooperation Method, Input / Output Cooperation Device, Input Program, and Recording Medium Recording Input / Output Cooperation Program" Japanese Patent Application Laid-Open No. 2001-216527 “Multimedia Information Space Input Device and Method, and Recording Medium Recording the Program” Tamio Kihara, Machiko Prairie, Hiroshi Yasuda “The Art of the Place and Network Art” Information Processing Society of Japan Vol.44 No2, 2003

  However, in order to realize the information providing method as described above, the conventional technique has the following problems.

  The first is the problem of application limitations.

  Many of the prior arts are mainly used in entertainment facilities and the like, so all functions are built for each application. For this reason, it is difficult to flexibly switch or combine the two information providing methods, and to set an output method in detail according to the action of a person. For this reason, there is a problem that it takes a very large cost to use a system developed for a specific purpose in order to provide different places and different types of information.

  Secondly, there is a problem of restriction of application place.

  In order to detect the operation, usually, one surveillance camera installed right above the detection area is used. However, since the angle of view of the surveillance camera is limited, it is difficult to detect the movement of a person in a wide area in a place where the ceiling is low. Since the provision of information as described above targets the existing space, it is necessary to flexibly deal with restrictions on the monitoring area of various sizes and shapes, the installation position of the camera, etc. difficult.

  The present invention has been made in view of the above points, solves the problems of application restrictions and application place restrictions, and various information according to the actions of persons such as floating advertisements and map advertisements in various public spaces. It is an object to provide an information output method and apparatus, a program, and a computer-readable storage medium storing the information output program.

  FIG. 1 is a diagram for explaining the principle of the present invention.

The present invention (Claim 1) is an information output method for dynamically changing the output of information to a display device in accordance with the operation of one or more persons.
A sensor process (step 1) in which the sensor means analyzes based on inputs from various detection targets;
Event generating means acquires the analysis result from the sensor means, when certain conditions are met where there is a result of the analysis, the Louis vent developmental process caused the event (step 2),
A script execution process (step 3) in which the script execution means executes a function defined according to the generated event;
A screen control process (step 4) in which the image control means executes switching of contents to be displayed or change of the output position or output size in accordance with the screen control command specified in the function. ,
In the sensor process,
Captured images are obtained from a plurality of surveillance cameras, the captured images are analyzed, and each pixel of the captured image is converted to binary representing white and black representing the difference between the background and a background that exceeds a certain threshold. Converting, obtaining a continuous region of black pixels as a closed region, detecting the position of the person from the captured image by setting the center of gravity of the closed region as the position of the person,
The distance between the centroid of the closed region and the black pixel of the closed region is calculated, the pixel having the maximum distance is obtained, and when the maximum distance exceeds a certain threshold, the pixel having the maximum distance from the centroid seek direction vector connecting to, this as a pointing direction of the person with respect to the closed area, maximum when the distance does not exceed a certain threshold of multiple images synchronization analysis to determine the pointing is not performed It performs excess degree,
In the event the development process,
Generate an event according to the value of the pointing direction,
In the script execution process,
Execute the specified function according to the event that occurred,
In the screen control process,
The content information to be displayed is switched in accordance with the condition for the time change of the position of the person specified in the executed function or the presence / absence of pointing .

The present invention (Claim 2) , in the multiple image synchronization analysis process,
A user interface capable of mapping one or a plurality of captured images manually adjusted at the four corners of the captured image of the monitoring camera on the content screen, and designating a monitoring area on the mapped screen; The captured image having the mapped shape is automatically distorted, and the motion of the person is detected only for the monitoring area designated via the user interface.

The present invention (Claim 3) , in the multiple image synchronization analysis process,
When the images of multiple surveillance cameras are mapped, if the shooting areas overlap each other, the pixels in the overlapped areas are ORed with the black pixels to create one combined image. Then, the position of the person is detected with respect to the synthesized image.

The present invention (Claim 4) , in the event generation process,
Obtain a list of events to be monitored from the script execution means, accumulate the analysis results from the sensor means in the storage means, and generate an event if the accumulated analysis result set satisfies the conditions for occurrence of each event .

  FIG. 2 is a principle configuration diagram of the present invention.

The present invention (Claim 5) is an information output device that dynamically changes the output of information to a display device in accordance with the action of one or more persons,
Sensor means 15 for performing analysis based on information acquired from each device including a monitoring camera, a wireless tag receiver, and a microphone;
An event generation means 40 for acquiring an analysis result from the sensor means 15 and generating an event when the analysis result satisfies a certain condition;
Script execution means 50 for executing a function defined according to the generated event;
Screen control means 60 for switching the content to be displayed or changing the output position or the output size in accordance with the screen control command specified in the function,
The sensor means 15
Analyzing the captured images of a plurality of monitoring cameras, a plurality image synchronization analysis means to detect the position and movement of a person,
Multiple image synchronization analysis means
Captured images are obtained from a plurality of surveillance cameras, the captured images are analyzed, and each pixel of the captured images is converted into binary values of white representing the background and black representing the difference between the background and a certain threshold. Means for converting, obtaining a continuous area of black pixels as a closed area, and detecting the position of the person from the captured image by setting the center of gravity of the closed area as the position of the person;
The distance between the centroid of the closed region and the black pixel of the closed region is calculated, the pixel having the maximum distance is obtained, and when the maximum distance exceeds a certain threshold, the pixel having the maximum distance from the centroid connecting it obtains a direction vector, which was used as a pointing direction of the person with respect to the closed region, if not exceeding a certain threshold maximum distance, means for determining the pointing is not performed, the Including
The event generation means 40
Including means for generating an event according to the value of the pointing direction;
The script execution means 50
Means for executing a function defined in accordance with an event generated by the event generation means 40;
The screen control means 60
It includes means for executing switching of content information to be displayed according to a condition with respect to time change of the position of the person designated in the function executed by the script execution means or presence / absence of pointing .

The present invention (Claim 6), in the plurality of images synchronized analysis means,
Means for mapping one or a plurality of photographed images manually adjusted at the four corners of the photographed image of the surveillance camera on the content screen;
A user interface that allows you to specify the monitoring area on the mapped screen;
Means for automatically distorting the captured image of the mapped shape and detecting the movement of the person only in the monitoring area designated via the user interface.

According to the present invention (Claim 7) , in the multiple image synchronization analysis means ,
When the images of a plurality of surveillance cameras are mapped, if the shooting areas are overlapped with each other, the pixels in the overlapped areas are ORed with the black pixels to create one synthesized image. Means,
Means for detecting the position of a person with respect to the synthesized image.

The present invention (Claim 8 ) provides an event generating means ,
Obtain a list of events to be monitored from the script execution means, accumulate the analysis results from the sensor means in the storage means, and generate an event if the accumulated analysis result set satisfies the conditions for occurrence of each event Including means.

The present invention (Claim 9) is an information output program for causing a computer to function as each means constituting the information output apparatus according to any one of Claims 5 to 7.

The present invention (Claim 10) is a computer-readable storage medium storing the output program according to Claim 9.

  In the present invention, the problem of application restriction and application place restriction can be solved, so it is possible to easily provide various information according to the actions of people such as floating advertisements and map advertisements in various public spaces. It is.

  FIG. 3 shows the configuration of the information output apparatus according to the embodiment of the present invention, and FIG. 4 is a sequence chart showing the operation of the information output apparatus according to the embodiment of the present invention.

  The information output apparatus 1 shown in FIG. 1 includes a multiple image synchronization analysis unit 10, a wireless tag management unit 20, a voice input unit 30, a sensor event generation unit 40, a script execution unit 50, and a screen control unit 60. The unit 10, the wireless tag management device 20, and the voice input device 30 are collectively referred to as a sensor device.

  The processing of each device is as follows.

  The multiple-image synchronization analysis apparatus 10 performs image analysis processing (step 10) on the captured image captured by one or a plurality of surveillance cameras, and transmits the image analysis result to the sensor event generation unit 40. . These processes are repeated until an end command is acquired. In the image analysis process (step 10), a plurality of photographed images are combined, and the position, speed, acceleration, and pointing direction of a person (hereinafter referred to as a shadow) in the monitoring area are detected and output as an image analysis result. To do. Details of the image analysis processing are shown in FIG. 5 and details will be described later.

  The wireless tag management device 20 executes wireless tag analysis processing (step 20) on the ID of the wireless tag received by one or a plurality of wireless tag receivers, and uses the wireless tag information analysis result as a sensor event generation device. 40. These processes are repeated until an end command is acquired. In the wireless tag analysis process (step 20), first, the ID of the wireless tag existing in the sensing area of each receiver is acquired from each wireless tag receiver, and the wireless tag information analysis result (time, wireless tag receiver ID) , Sensed wireless tag ID, position). This process uses existing technology.

  The voice input device 30 performs voice analysis processing (step 30) on the voice input from one or a plurality of microphones, and transmits the voice analysis result to the sensor event generation device 40. These processes are repeated until an end command is acquired. In the voice analysis process (step 30), voice is acquired from each microphone, the voice is analyzed, and a voice analysis result (time, microphone ID, volume, or voice recognition result) is output. This process uses existing technology.

  The sensor event generating device 40 acquires an analysis result from each sensor device (the multiple image synchronization analyzing unit 10, the wireless tag managing device 20, and the voice input device 30), and executes an event generating process (step 40). In the event generation process (step 40), each analysis result is accumulated, and an event is generated and transmitted to the script execution unit 50 when the set of accumulated analysis results satisfies a specific condition set in advance. Conditions for generating an event are set based on a monitoring event setting request. Here, the event generation conditions are not particularly limited, but examples include the following.

-Image analysis results: Shadow entry and exit to a specific position, stagnation for a certain period of time, speed exceeds a certain value, acceleration exceeds a certain value, and the number of shadows recognized and increased in the entire monitoring area Detection of movements such as pointing, etc .:
-RFID tag information analysis results: RFID tags entering and exiting specific areas of each RFID tag receiver, stagnation within a certain time in the sensing area, etc .:
-Voice analysis results: Volume exceeds a certain value, falls below a certain value, a specific word is detected by speech recognition, etc .:
Details of the above event generation processing are shown in FIG. 6, and details will be described later.

  Further, the sensor event generation unit 40 executes a sensor state acquisition process (step 50) in response to the sensor state acquisition request from the script execution unit 50, and transmits the sensor state. Here, although the sensor state is not particularly limited, for example, there are the following, which are extracted from the analysis result of each sensor.

Image analysis results: number of shadows in the monitoring area, position, speed, acceleration, determination of movement (pointing direction), shadow size, time when each shadow was first recognized, etc .:
Wireless tag information analysis result: number of wireless tags present in the sensing area of each wireless tag receiver, ID and position of each wireless tag, time when each ID was first recognized, etc .:
-Voice analysis result: current volume (dB), voice analysis result (text), etc .;
The script execution unit 50 acquires an action script from 70 as an action script, and executes a script execution process (step 60). In the script execution process (step 60), when an event is acquired from the sensor event generation unit 40, a function (a combination of a sensor state acquisition instruction, a screen state acquisition instruction, a screen control instruction, a command execution instruction, and a calculation / control instruction) The event handler is executed below. Details of the script execution processing are shown in FIG. 7, and details will be described later.

  The screen control unit 60 acquires a screen control request from the script execution unit 50 and executes a screen control process (step 8). In the screen control process (step 80), content (still image, moving image, text, etc.) is displayed in a window having a designated position and size. It is assumed that the content is stored in the content database 80 or distributed via the network 90.

  Further, in response to a screen state acquisition request from the script execution unit 50, a screen state acquisition process (step 70) is executed and the screen state is returned. Here, the screen state is not particularly limited, but includes, for example, the number of displayed windows, the position and size of each window, the displayed content name, the playback state of a moving image, and the like.

  Next, each of the above processes will be described in detail.

  First, the image analysis process (step 50) shown in FIG. 5 will be described.

  Before performing this processing, the plurality of image synchronization analysis units 10 manually map the captured image on the computer screen so as to match the actual imaging region of the surveillance camera. FIG. 8 shows an actual shooting area and a shot image when two monitoring cameras are used. As shown in FIG. 8, the two surveillance cameras are arranged so as to be able to capture an area where they overlap each other. Also, it is assumed that the surveillance camera cannot be installed directly above the shooting area and is shooting from below in the figure. For this reason, the actual imaging area of each surveillance camera is trapezoidal. Since the captured image obtained from the surveillance camera is rectangular as shown in FIG. 8, the actual trapezoidal imaging region is distorted into a rectangle.

  FIG. 9 shows a state where the administrator manually maps such a distorted captured image using a user interface on the computer screen. By manually performing the four corners of each captured image (FIG. 9a) on the user interface, mapping is performed in a form close to the actual captured image (FIG. 9b). Next, the monitoring area actually used for analysis is designated.

  Hereinafter, each step of the image analysis process (FIG. 5) will be described on the assumption that this mapping has already been performed.

  Step 101) Captured images are acquired from one or more monitoring cameras.

  Step 102) For each captured image, each pixel is converted into a binary value of white and black. Here, the background color is white, and the difference from the background color exceeds a certain threshold is “black”. That is, the pixel in the region where the person exists is “black”.

  Step 103) The binarized image is synthesized. Each step is as follows.

  (A) As shown in FIG. 9, the acquired image (rectangle) is distorted so as to match the shape (trapezoid) of the actual shooting area.

  (B) Cut out only the portion of the monitoring area.

  (C) Perform a logical OR with respect to overlapping pixels. That is, when either is “black”, the value of the pixel is black (FIG. 9c).

  Step 104) A closed region list is created from the frame image. Here, the closed region is a region in which black pixels are continuous and has an area (the number of black pixels) that exceeds a certain threshold. In FIG. 9c, two closed regions are extracted.

  Step 105) The current time t is acquired.

  Step 106) Determine whether the closed region list is empty. If it is empty, the process proceeds to step 115. If it is not empty, the process proceeds to step 107.

  Step 107) The closed region k at the head of the list is taken out. An appropriate ID is assigned to k.

  Step 108) Calculate the barycentric positions of all black pixels included in k.

  Step 109) The distance from each pixel included in k to the center of gravity is calculated, and the pixel p having the longest distance and the length thereof are specified. When this length exceeds a certain threshold value, the direction of the pixel connected from k to p is acquired, and this is set as the pointing direction. If it does not exceed, there is no pointing.

  Step 110) It is determined whether or not the current process is the second frame or later. If it is the second frame or later, the process proceeds to step 111, and if not, the process proceeds to step 114.

  Step 111) Find the shadow 1 whose center of gravity is closest to k among the previous analysis results saved in the internal storage area. The difference d between the centroid of k and the centroid of 1 is calculated. If d0 <d <d1, d is the speed of k. Here, d0 and d1 are constants, and respectively represent a lower limit indicating whether or not there is a movement and an upper limit whether or not the same shadow can be specified. When d <d1, ID 1 is substituted for ID k.

  Step 112) It is determined whether the current process is the third frame or later. If it is after the third frame, the process proceeds to step 113; otherwise, the process proceeds to step 114.

  Step 113) The difference between the speed of k and the speed of 1 is calculated, and this is set as the acceleration of k.

  Step 114) “Current time t, ID of closed region k, position of center of gravity, velocity, acceleration, pointing direction” is added to the shadow list.

  Step 115) The shadow list is transmitted to the sensor event generator as an image analysis result.

  Step 116) The shadow list is stored in the internal storage area as the previous analysis result.

  Next, event generation processing (step 40) in the sensor event generation unit 40 will be described with reference to the flowchart of FIG.

  Step 401) A list of monitoring events is acquired from the script execution unit 50. Substitute the number of monitoring events in num and 0 in i.

  Step 402) Acquire an image analysis result and store it in the internal memory. The image analysis result is a set of the following records, and is sorted and stored by time using “time + shadow ID” as a key.

Record: “Time, shadow ID, center of gravity, speed, acceleration, pointing direction”
Step 403) The wireless tag information analysis result is acquired and stored in the internal memory. The wireless tag information analysis result is a set of the following records, and is sorted and stored by time using “time + wireless tag receiver ID + wireless tag ID” as a key.

Record: “Time, RFID tag receiver ID, RFID tag ID, position”
Step 404) Acquire voice analysis results and store them in the internal memory. The voice analysis result is a set of the following records, and is sorted and stored by time using “time + microphone ID” as a key.

Record: “Time, microphone ID, volume or voice recognition result”
Step 405) It is determined whether the monitoring event list has been processed to the end. If i <num, go to Step 406; otherwise, go to Step 411.

  Step 406) The i-th monitoring event e in the list is acquired.

  Step 407) Substitute i + 1 for i.

  Step 408) It is determined whether or not the event occurrence condition of e is satisfied by examining the analysis result stored in the internal memory. Here, detailed processing for determining whether or not the generation condition is satisfied is not defined, but for example, when an event that the number of persons in the monitoring area increases, the following condition determination processing is performed.

  (A) The number m of records having the latest time of the image analysis result is acquired.

  (B) The number n of records having the previous time is acquired.

  (C) When mn is positive, an event of increasing the number of people is generated.

  Step 409) The following branch processing is performed depending on whether the event occurrence condition is satisfied. If yes, go to step 410; otherwise, go to step 405.

  Step 410) The event name satisfying the condition and the variable associated with the event are transmitted to the script execution unit 50. Here, the argument is, for example, the shadow of the person who has entered, as in the example of the action script of FIG. Details of FIG. 10 will be described later.

  Step 411) It is determined whether an end command has been executed by the administrator. If it has been executed, the process ends. If it has not been executed, the process proceeds to step 402.

  Next, the script execution process (step 60) in the script execution part 50 is demonstrated along the flowchart of FIG.

  The action script describes a set of functions (event handlers) that are called when an event occurs. It is assumed that the action script is created manually or automatically and stored in the action script database 70. In the event handler, a sensor status acquisition command, a screen status acquisition command, a screen control command, a command execution command, and a calculation / control command can be specified. An example of the action script is shown in FIG. Details will be described later in Examples. When the administrator selects and executes an action script, the following script execution process is executed.

  Step 601) An action script is acquired from the action script database 70.

  Step 602) The name of the event handler described in the action script is transmitted to the sensor event generating unit 40 as a monitoring event.

  Step 603) It is determined whether an end command has been executed by the administrator. If it has been executed, the process ends. If it has not been executed, the process proceeds to step 604.

  Step 604) An event is acquired from the sensor event generator 40.

  Step 605) The event handler corresponding to the event is analyzed, and an instruction sequence sorted in the execution order of the instructions is created. The event handler analysis method and instruction sequence creation method are based on existing technology.

  Step 606) It is determined whether the instruction sequence is empty. If it is empty, the process proceeds to step 603. If it is not empty, the process proceeds to step 607.

  Step 607) The first instruction e in the instruction sequence is acquired.

  Step 608) The following branch processing is performed. If e is a sensor state acquisition command, the process proceeds to step 609; otherwise, the process proceeds to step 610.

  Step 609) The sensor state acquisition request specified in e is transmitted to the sensor event generating unit 40, and the sensor state obtained as a result is stored in the internal memory.

  Step 610) The following branch processing is performed. If e is a screen state acquisition command, the process proceeds to step 611; otherwise, the process proceeds to step 612.

  Step 611) The screen state acquisition request designated in e is transmitted to the screen control unit 60, and the resulting screen state is stored in the internal memory.

  Step 612) The following branch processing is performed. If e is a screen control command, the process proceeds to step 613; otherwise, the process proceeds to step 614.

  Step 613) Using the sensor state and screen state information stored in the internal memory, the window size, position, content type, and the like are determined and transmitted to the screen control unit 60.

  Step 614) The following branch processing is performed. If e is a command execution instruction, the process proceeds to step 615; otherwise, the process proceeds to step 616.

  Step 615) Commands such as mail transmission and status output to the administrator screen are executed. Here, the sensor state and screen state information stored in the internal memory is used. In addition, usable commands are not particularly limited.

  Step 616) Using the information stored in the internal memory, arithmetic operations, comparison operations, variable settings, control instructions, etc. are executed, and the results are stored in the internal memory.

  Hereinafter, the floating advertisement in FIG. 11 is used to describe an actual operation example. FIG. 10 shows a part of a script for realizing a floating advertisement. The functions “InArea”, “OutArea”, and “RemainArea” are event handlers.

  In FIG. 10, “shadow” and “win” are an object representing each shadow of a person detected by the image analysis process and an object representing each information display window displayed on the screen. Each object has the following methods, which are called by “object name.method name”.

・ Shadow object ・ x (), y (): Position acquisition ・ HandUp (): Judgment whether or not pointing is being performed ・ waitedTime (): Stagnation time at the same location These methods are used to acquire the sensor status This is a sensor state acquisition command.

-Win object -setSize (width, height): Change the window size to width width and height height.

  SetContents (“character string”): The display information is changed to the content specified by the character string.

  Since these methods are instructions for controlling the screen, they are screen control instructions.

  The functions of the functions used in the event handler are as follows.

  Convert (x, y): The position (x, y) of the shadow detected by the image analysis process is converted into a position (wx, wy) on the screen and returned. This instruction is an arithmetic / control instruction.

  • select_window (wx, wy): Returns the window (win) that surrounds the position specified by wx, wy. This command is a screen state acquisition command.

  The event generation conditions and processing outline of each event handler are as follows.

・ InArea (shadow)
Event occurrence condition: Occurs when a shadow of a person enters a predetermined area.

  Process overview: The position of the shadow of the person who has entered is acquired, and the size of the information display window (win) on the screen corresponding to that position is increased.

・ OutArea (shadow)
Event occurrence condition: Occurs when a shadow of a person leaves a predetermined area.

  Process overview: Acquires the position where the shadow of the person who left was last, and specifies the information display window (win) on the screen corresponding to the position. The window size is set to the normal size, and the display content is for the initial screen.

・ RemainArea (shadow)
Event occurrence condition: Occurs when a shadow of a certain person stays in a predetermined area for a certain period of time.

  Process overview: The position of a shadow of a stagnant person is acquired, and the information display window (win) on the screen corresponding to the position is specified. If the stagnation time is longer than a predetermined time (t0) or if pointing is performed, a map screen is output, and if not, detailed information is output.

  The process that is actually called when a person stops for a certain time before an advertisement and moves is as follows.

  (1) A person passes in front of an information display window.

  (2) As a result of the image analysis process, a shadow representing the person is detected and enters a specified position.

  (3) An entry event occurs, and the event handler InArea is called with the entered shadow as an argument.

  (4) In Inrea, the process of FIG. 10 is performed, and the size of the information display window in front of you is enlarged.

  (5) When the stopped time exceeds a certain time, the event handler RemainArea is called with the stopped shadow as an argument.

  (6) In RemainArea, the processing of FIG. 10 is executed, and at this time, the stagnation time is shorter than t0, so the display information is changed to detailed information.

  (7) Furthermore, RemainArea is called several times in the same way over time. A map screen is displayed when the stagnation time exceeds t0. Further, the map screen is immediately displayed if the finger is pointing even if it does not exceed t0.

  (8) When the person moves and leaves the shadow from the predefined area, the event handler OutArea is called.

  (9) In OutArea, the processing of FIG. 10 is executed, the size of the information display window in front of the eyes is returned to the normal size, and the initial screen is displayed.

  In this way, by writing a simple script as shown in FIG. 10, it is possible to output information corresponding to the action of a person such as a floating advertisement. Similarly, by writing event handlers and sensor state acquisition commands for the wireless tag management unit 20 and the voice input device 30 in a script, information output using these sensor devices can be performed.

  The following shows how the above-mentioned problem of application limitation is solved.

  The script execution device can check the sensor state and the screen state at the timing when the event described in the script occurs, and can execute an appropriate action (screen control). For this reason, the screen control which combined various sensor information is easily realizable by describing a simple script. Furthermore, if a command execution command is used, it can be extended to actions using devices other than the screen control device (contact by e-mail when the number of people increases). In addition, since only events described in the script are monitored, high-speed processing is possible. As described above, the present invention can easily apply a system developed for a specific purpose to various applications for another purpose.

  Next, it will be shown how the above-mentioned problem of limitation of application place is solved.

The multiple image synchronization analysis unit 10 can specify the position of a person from an image obtained by combining images taken by n cameras. In addition, it is possible to perform correction when the installation position of the monitoring camera is not directly above the monitoring target area. Therefore, since the ceiling is low, even if the wide area that cannot be covered by the angle of view of one surveillance camera or the installation position of the surveillance camera is not directly above, the area can be covered by the image composition function and the correction function. . In addition, since the correspondence between the imaging area of the surveillance camera and the surveillance area can be easily set using the user interface, even if there are areas of any shape, for example, when there are corners, the surveillance area should be created all over. Can do.
Furthermore, since the analysis result can be seen from the event generation unit 40 as a single combined area, it is not necessary to handle each monitoring camera image individually when acquiring an event or state. This makes it easier to write scripts. As described above, the present invention can solve the problem of restriction of the application location.

  Further, each component of the output information device shown in FIG. 3 can be configured as an individual device to be configured as an output information system.

  In the present invention, the operations shown in FIGS. 4 to 7 are constructed as a program, installed in a computer used as an information output device, executed by a control means such as a CPU, or distributed through a network. It is possible to make it.

  It is also possible to store the built program in a portable storage medium such as a hard disk, a flexible disk, or a CD-ROM connected to a computer used as an information output device, and install it on the computer at the time of execution. is there.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made within the scope of the claims.

  The present invention can be applied to a technology that provides advertisement information according to the movement of a person in a large-scale space such as an entertainment facility, a pavilion, a station premises, or a building entrance.

It is a figure for demonstrating the principle of this invention. It is a principle block diagram of this invention. It is a block diagram of the information output device in one embodiment of this invention. It is a sequence diagram of the outline | summary operation | movement of the information output device in one embodiment of this invention. It is a flowchart of the image analysis process in one embodiment of the present invention. It is a flowchart of the event generation process in one embodiment of this invention. It is a flowchart of the script execution process in one embodiment of this invention. It is the imaging | photography area | region and picked-up image in one embodiment of this invention. 3 is a method for synthesizing captured images according to an embodiment of the present invention. It is an example of the action script in one embodiment of this invention. It is an example of a floating advertisement. It is an example of a map advertisement.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Information output device 10 Multiple image synchronous analysis part 15 Sensor means 20 Wireless tag management part 30 Audio | voice input part 40 Event generation means, sensor event generation part 50 Script execution means, Script execution part 60 Image control means, Image control part 70 Action script Database 80 Content database 90 Network

Claims (10)

In an information output method for dynamically changing the output of information to a display device according to the action of one or more persons,
A sensor process in which sensor means analyzes based on inputs from various detection targets;
If the event generating means acquires the analysis result from said sensor means, satisfies a specific condition where there is a result of the analysis, and event generation process of generating an event,
A script execution means for executing a function defined according to the generated event;
The image control means comprises a screen control process for switching the content to be displayed or changing the output position or the output size in accordance with the screen control command specified in the function,
In the sensor process,
Captured images are obtained from a plurality of surveillance cameras, the captured images are analyzed, and each pixel of the captured image is converted to binary representing white and black representing the difference between the background and a background that exceeds a certain threshold. Converting, obtaining a continuous region of black pixels as a closed region, detecting the position of the person from the captured image by setting the center of gravity of the closed region as the position of the person,
The distance between the centroid of the closed area and the black pixel of the closed area is calculated, and the pixel having the maximum distance is obtained. When the maximum distance exceeds a certain threshold, the maximum distance from the centroid is calculated. seek direction vector connecting the pixels with, this as a pointing direction of the person with respect to the closed region, if not exceeding a certain threshold maximum distance plurality determines that pointing is not performed image to synchronize analysis excessive degree,
In the previous Symbol event occurrence process,
Generate an event according to the value of the pointing direction,
In the script execution process,
Execute a function defined according to the generated event;
In the screen control process,
An information output method for executing switching of content information to be displayed according to a condition with respect to a temporal change in a position of a person designated in the executed function or presence / absence of pointing . In the multiple image synchronization analysis process,
It has a user interface capable of mapping one or a plurality of photographed images manually adjusted at the four corners of the photographed image of the surveillance camera on the content screen and designating a surveillance area on the mapped screen. The information output method according to claim 1, wherein the photographed image having the mapped shape is automatically distorted, and the motion of the person is detected only for the monitoring area designated via the user interface. In the multiple image synchronization analysis process,
When mapping an image of said plurality of surveillance cameras, when such imaging regions overlap each other, the pixel of the overlapping region, by taking a logical sum of black pixels, create one synthesized image and, the combined claims 1 or 2 information output method according to detect the position of the person on the image. In the event generation process,
From the script execution unit acquires an event list of monitored, the analysis result from the sensor means to accumulate in the storage means, when the accumulated analysis result set satisfies the generation condition for each event, the event The information output method according to claim 1 to be generated. An information output device that dynamically changes the output of information to a display device according to the action of one or more persons,
Sensor means for performing analysis based on information acquired from each device including a monitoring camera, a wireless tag receiver, and a microphone;
Event generation means for acquiring an analysis result from the sensor means and generating an event when the analysis result satisfies a certain condition;
Script execution means for executing a function defined according to the generated event;
Screen control means for switching the content to be displayed, or changing the output position or the output size in accordance with the screen control command specified in the function,
The sensor means includes
Analyzing the captured images of a plurality of monitoring cameras, a plurality image synchronization analysis means to detect the position and movement of a person,
The multiple image synchronization analysis means includes:
Captured images are obtained from a plurality of surveillance cameras, the captured images are analyzed, and each pixel of the captured images is converted into binary values of white representing the background and black representing the difference between the background and a certain threshold. Means for converting, obtaining a continuous area of black pixels as a closed area, and detecting the position of the person from the captured image by setting the center of gravity of the closed area as the position of the person;
The distance between the centroid of the closed area and the black pixel of the closed area is calculated, and the pixel having the maximum distance is obtained. When the maximum distance exceeds a certain threshold, the maximum distance from the centroid is calculated. seek direction vector connecting the pixels with, this as a pointing direction of the person with respect to the closed region, if not exceeding a certain threshold maximum distance, means for determining the pointing is not performed Including,
The event generating means includes
Means for generating an event in accordance with the value of the pointing direction;
The script execution means includes
Means for executing a function defined in accordance with the generated event by the event generating means;
The screen control means includes
Information including: means for executing switching of content information to be displayed according to a condition with respect to time change of a position of a person designated in the function executed by the script execution means or presence / absence of pointing Output device. The multiple image synchronization analysis means includes:
Means for mapping one or a plurality of photographed images obtained by manually adjusting the four corners of the photographed image of the monitoring camera on the content screen;
A user interface that allows you to specify the monitoring area on the mapped screen;
6. An information output apparatus according to claim 5 , further comprising means for automatically distorting the photographed image of the mapped shape and detecting a human motion only on a monitoring area designated via the user interface. The multiple image synchronization analysis means includes:
If the imaging areas overlap each other when the images of the plurality of surveillance cameras are mapped, the pixels in the overlapping areas are ORed with the black pixels to create one synthesized image. Means to
7. An information output apparatus according to claim 5 , further comprising means for detecting a position of a person with respect to the synthesized image. The event generating means includes
An event list to be monitored is acquired from the script execution means, the analysis results from the sensor means are accumulated in the storage means, and if the accumulated analysis result set satisfies the occurrence conditions of each event, the event is 6. The information output device according to claim 5 , further comprising means for generating. 8. An information output program for causing a computer to function as each means constituting the information output apparatus according to claim 5. A computer-readable storage medium storing the output program according to claim 9.

Images