JP2005244626A - Monitor unit - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce burden of work of a monitoring person and to precisely identify storage of a desired object of monitoring in a storage position. <P>SOLUTION: When a panoramic picture created by sequentially changing a photographing direction and image-picking up a photographing range is monitored, the photographing range is set so that the storage position where a moving object as the object of monitoring is stored is photographed. Differences in luminance levels are sequentially detected between one unit picture constituting the created panoramic picture and a unit picture in the same photographing direction, which is image-picked up earlier the unit picture. The detected difference of the luminance levels is compared with a threshold which is preset in accordance with the luminance level of the moving object. Prescribed information is displayed on a display face with the panoramic picture created based on a compared result. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a monitoring device and method, a program, a recording medium, and a monitoring system for monitoring a wide range of situations via a panoramic image obtained by sequentially changing shooting directions.

  In recent years, so-called cinema complexes with multiple screens installed in one building have opened in various places. The person in charge confirming the admission ticket brought by the audience in this compound movie theater is usually arranged near the entrance of the building, but is often not arranged in each projection room where each screen is installed. For this reason, it is allowed to allow free entry and exit between the projection rooms by the spectators who enter the hall, and it is extremely unreasonable particularly in a complex movie theater that employs a fee system that charges a separate entrance fee for each projection room.

  Therefore, each movie theater has attempted to increase the number of observers to monitor the audiences who freely enter and leave the projection room without paying an entrance fee, but the cost performance is poor.

  In each complex movie theater, attempts have been made to automatically determine the seating status through a switch installed in each seat. In this method, the switch is configured as a piezoelectric element that generates an electrical signal by pressing input, and the seating situation is automatically detected by detecting the electrical signal generated by pressing input corresponding to the seating of the audience by a subsequent system. To determine. However, since it is necessary to purchase a switch corresponding to the number of seats in each projection room and install it for each seat, it has not been possible to reduce costs and labor. In addition, since the reliability of the switches as piezoelectric elements varies, the seating situation cannot be accurately determined.

  For this reason, in order to realize monitoring with high reliability and excellent cost performance, a wide range of images can be obtained by shooting the shooting target while sequentially shifting the shooting direction of the camera and combining them to obtain a panoramic image. A monitoring system capable of monitoring the situation has been proposed (see, for example, Patent Document 1). In addition, a centralized monitoring and recording system has been proposed that enables monitoring images to be collected from a plurality of installed monitoring video cameras, recorded on a recording medium such as a video tape, and monitored (for example, Patent Document 2). reference.).

JP-A-10-108163 JP 2000-243062 A

  By the way, when using the image recorded on the recording medium as described above to detect any change in the shooting range, the monitoring person observes the situation of the monitoring image input every moment for every scan. Had to do. In particular, the monitor must always identify minute changes in the image, the appearance of minute objects, etc., requires excessive effort and time, and is a system that relies on the eyes of the monitor. Therefore, there was also a risk that this situation change would be overlooked.

  In addition, there are cases in which luggage or transported goods may be stacked on the seat in addition to the spectator as a monitoring target, so this is easily identified and focused only on the storage at the storage location in the monitoring target It was also necessary to realize the analysis. That is, it is necessary to avoid unnecessary monitoring of the moving subject and to improve the convenience of analysis by selecting a desired monitoring target by the monitoring person himself / herself.

  In addition, when affected by subtle fluctuations such as illuminance fluctuations in the light source and ambient brightness, changes can be detected by simply comparing the brightness levels even with the same moving subject. However, there was a problem that the effectiveness of monitoring could not be guaranteed.

  Therefore, the present invention has been devised in view of the above-described problems, and the object of the present invention is to reduce the burden on the labor of the observer and to accurately accommodate the desired monitoring object at the accommodation position. It is an object to provide a monitoring apparatus and method, a program, a recording medium, and a monitoring system that can be well identified.

  In order to solve the above-described problem, the present invention provides a storage position in which a moving subject as a monitoring target is stored when monitoring a panoramic image generated by sequentially changing the shooting direction and capturing a shooting range. Set the shooting range to capture the image, and sequentially detect the difference in luminance level between one unit image constituting the generated panoramic image and the unit image taken in the same shooting direction before the unit image Then, the detected luminance level difference is compared with a threshold value set in advance according to the luminance level of the moving subject, and predetermined information is displayed on the display surface together with the panoramic image generated based on the comparison result.

  That is, the monitoring device to which the present invention is applied is a monitoring device that monitors a panoramic image generated by capturing a shooting range by sequentially changing the shooting direction. Between the shooting setting means for setting the shooting range so as to be captured, one unit image constituting the generated panoramic image, and a unit image in the same shooting direction taken before the unit image. Difference detection means for sequentially detecting a difference in luminance level, comparison means for comparing the difference in luminance level detected by the difference detection means with a threshold value set in advance according to the luminance level of the moving subject, and the comparison Display control means for displaying predetermined information on the display surface together with the generated panoramic image based on the comparison result by the means.

  In addition, the monitoring device to which the present invention is applied is a monitoring device that monitors a panoramic image generated by capturing a shooting range by sequentially changing the shooting direction. It is obtained between the shooting setting means for setting the shooting range so as to capture, one unit image constituting the generated panoramic image, and a unit image taken in the same shooting direction before the unit image. Vector detection means for sequentially detecting vectors based on the normalized correlation value, comparison means for comparing the direction of the vector detected by the vector detection means with the previously detected vector, and comparison by the comparison means Display control means for displaying predetermined information on the display surface together with the generated panoramic image based on the result.

  In addition, the monitoring method to which the present invention is applied is a monitoring method for monitoring a panoramic image generated by capturing a shooting range by sequentially changing the shooting direction, and captures a storage position where a moving subject as a monitoring target is stored. As described above, the shooting range is set, and a difference in luminance level is sequentially detected between one unit image constituting the generated panoramic image and a unit image taken in the same shooting direction before the unit image. Then, the difference in the detected luminance level is compared with a threshold value set in advance according to the luminance level of the moving subject, and predetermined information is displayed on the display surface together with the generated panoramic image based on the comparison result. .

  The program to which the present invention is applied is a program for causing a computer to monitor a panoramic image generated by capturing a shooting range by sequentially changing shooting directions. The shooting range is set so as to capture the stored position, and the luminance between one unit image constituting the generated panoramic image and the unit image taken in the same shooting direction before the unit image is set. Differences in level are sequentially detected, the detected luminance level difference is compared with a threshold value set in advance according to the luminance level of the moving subject, and predetermined information is generated together with the generated panoramic image based on the comparison result. Is displayed on the display screen.

  The recording medium to which the present invention is applied is a recording medium on which a program for causing a computer to monitor a panoramic image generated by capturing a photographing range by sequentially changing the photographing direction is recorded. The photographing range is set so as to capture the accommodation position where the moving subject as a target is accommodated, and one unit image constituting the generated panoramic image and the same photographing direction taken before the unit image are taken. Differences in brightness level between unit images are sequentially detected, the detected brightness level difference is compared with a threshold value set in advance according to the brightness level of the moving subject, and the generation is performed based on the comparison result. The computer is caused to display predetermined information on the display surface together with the panoramic image.

  In addition, a monitoring system to which the present invention is applied captures an imaging device that generates a panoramic image by imaging a shooting range by sequentially changing the shooting direction, and a storage position where a moving subject as a monitoring target is stored. Set the shooting range, sequentially detect a difference in brightness level between one unit image constituting the generated panoramic image and a unit image taken in the same shooting direction before the unit image, A control device that compares the detected brightness level difference with a threshold value set in advance according to the brightness level of the moving subject, and transmits the comparison result through a connected communication network; and at least the control device And a terminal device for displaying the comparison result received via the communication network on a display surface.

  The present invention sets a shooting range so as to capture an accommodation position where a moving subject as a monitoring target is accommodated when monitoring a panoramic image generated by imaging the imaging range by sequentially changing the imaging direction, A difference in luminance level is sequentially detected between one unit image constituting the generated panoramic image and a unit image taken in the same shooting direction before the unit image, and the detected difference in luminance level is described above. Comparison is made with a threshold set in advance according to the luminance level of the moving subject, and predetermined information is displayed on the display surface together with the panoramic image generated based on the comparison result.

  Thereby, since it is sufficient to confirm the details for the first time when a warning is issued from the monitoring device, the monitoring effort can be greatly reduced. Further, it is possible to realize a situation analysis focusing only on the accommodation at the accommodation position in the monitoring target, and thus the convenience of the analysis can be improved.

  Hereinafter, the best mode for carrying out the present invention will be described in detail with reference to the drawings. A monitoring system 1 to which the present invention is applied includes a camera unit 2 that captures an image of a subject and generates an image signal, a monitoring device 5 that transmits the image signal, and a monitoring device 5 as shown in FIG. Display 6, a terminal device 9 for a plurality of users to execute applications, a terminal display 10 connected to the terminal device 9, a camera unit 2, a monitoring device 5, and a terminal device 9. And a network 8 for realization.

  As shown in FIG. 2, the camera unit 2 in the monitoring system 1 includes a pantilter unit 3 and a camera unit 4 integrally formed. The pantilter unit 3 is configured as a turntable for freely changing the shooting direction with respect to, for example, two axes of pan and tilt.

  The camera unit 4 is disposed on a turntable that constitutes the pantilter unit 3 and images a subject while adjusting the shooting direction in the horizontal direction or the vertical direction according to control by the monitoring device 5. In addition, the camera unit 4 captures an image of the subject by increasing or reducing the shooting magnification by sequentially changing the shooting angle of view in accordance with control by the monitoring device 5. By installing a plurality of the camera units 4 with respect to one monitoring device 5, it is possible to capture the same subject at different shooting angles and obtain multifaceted image information.

  As shown in FIG. 2, the monitoring device 5 is connected to the image input / output unit 13 for performing predetermined processing on the image signal transmitted from the camera unit 2, and a moving image based on the image signal. An arithmetic processing unit 15 that generates an image, a server 53 that is connected to the arithmetic processing unit 15 and records the image signal, and an operation unit 16 for the user to control the monitoring device 5 are provided.

  The monitoring device 5 is configured by an electronic device such as a personal computer (PC), for example, records an image signal transmitted from the camera unit 2, and displays the recorded image signal to a user via the display 6. In addition, when a desired image region or image position is designated by the user, the monitoring device 5 performs control so as to select an optimal one from the recorded image signals and display it. The monitoring device 5 also serves as a so-called central control device for controlling the entire network 8 and transmits an image in response to a request from another terminal device 9. Details of the configuration of the monitoring device 5 will be described later.

  The network 8 includes, for example, an Internet network connected to the monitoring device 5 via a telephone line, ISDN (Integrated Services Digital Network) / B (broadband) -ISDN connected to a TA / modem, and the like. It is a public communication network that enables bidirectional transmission and reception. Incidentally, when the monitoring system 1 is operated in a certain narrow area, the network 8 may be configured by a LAN (Local Area Network). Further, the network 8 may be capable of transmitting MPEG images in addition to still images. In such a case, MPEG data is continuously transmitted from one channel based on the Internet protocol (IP), and still image data is transmitted from another channel at regular intervals.

  The terminal device 9 is a PC for a user waiting in each home or company to acquire an image from the monitoring device 5 via the network 8 and execute a desired process. By connecting a plurality of terminal devices 9 to the network 8, it becomes possible to simultaneously provide the application of the monitoring system 1 to a plurality of users. The terminal device 9 displays the image acquired from the monitoring device 5 on the terminal display 10. In addition, the terminal device 9 generates a request signal in response to a designation operation by the user and transmits it to the monitoring device 5. In addition, about the block structure of the terminal device 9, the structure of the monitoring apparatus 5 mentioned later is quoted, and description is abbreviate | omitted.

  Next, the camera unit 2 and the monitoring device 5 in the monitoring system 1 to which the present invention is applied will be described in detail.

  FIG. 3 shows a detailed configuration of the camera unit 2 and the monitoring device 5. In FIG. 2, each component of the camera unit 2 and the monitoring device 5 is connected to a common controller bus 21.

  The pantilter unit 3 constituting the camera unit 2 includes a tilt unit 3a and a pan unit 3b that control a turntable for changing the imaging direction. The camera unit 4 constituting the camera unit 2 includes a lens control unit 23 mainly for changing the angle of view of the lens unit 22, and an imaging unit 24 disposed at a position orthogonal to the optical axis of the lens unit 22. An IEEE (Institute of Electrical and Electronics Engineers) 1394 interface 25 for transmitting the image signal generated by the imaging unit 24 to the image input / output unit 13 and a GPS (Global Positioning) for detecting the current position of the camera unit 2 System) receiving unit 28 and metadata generating unit 29 attached to GPS receiving unit 28. Incidentally, the IEEE 1394 interface 25 may be replaced with Ethernet (registered trademark).

  The image input / output unit 13 includes a buffer memory 51 connected to the IEEE 1394 interface 25 and an encoder 52 connected to the buffer memory 51.

  The arithmetic processing unit 15 is also connected to the image compression unit 54 for compressing the image read from the server 53, and the graphic controller 55 that is connected to the server 53 and the image compression unit 54 and generates an image to be displayed on the display 6. A CPU 56 for controlling each unit via the controller bus 21, a memory card 61 and a clock 62 connected to the I / O port 58, respectively.

  The operation unit 16 includes a keyboard 59 and a mouse 60 for the user to specify a desired image area and image position from the image displayed on the display 6.

  The Tilt unit 3a and the Pan unit 3b rotate a stepping motor configured as a drive source for the turntable based on a drive signal from the CPU 56. Thereby, the photographing direction of the camera unit 4 placed on the turntable can be changed to the horizontal direction or the vertical direction.

  The lens control unit 23 performs an automatic aperture control operation and an automatic focus control operation on the lens unit 22 based on the drive signal from the CPU 56. The lens control unit 23 changes the shooting angle of view for the subject based on the drive signal. As a result, the camera unit 4 can also adjust the shooting magnification sequentially to capture the subject.

  The imaging unit 24 is configured by a solid-state imaging device such as a CCD (Charge Coupled Device), for example, forms an object image incident through the lens unit 22 on the imaging surface, generates an image signal by photoelectric conversion, This is transmitted to the IEEE 1394 interface 25.

  The GPS receiver 28 detects the installation location and shooting direction of the camera unit 2 based on a signal transmitted by the GPS system. By providing the GPS receiving unit 28, particularly when a plurality of camera units 2 are installed, it is possible to control both shooting directions in conjunction with each other. The output signal from the GPS receiving unit 28 is supplied to the metadata generating unit 29, and metadata including position information such as latitude, longitude, azimuth, altitude, time, various parameters, etc. is generated based on the positioning result by GPS. Is done. The metadata generation unit 29 supplies the generated position information and metadata to the encoder 52. In the present invention, the configuration of the GPS receiving unit 28 and the metadata generating unit 29 may be omitted.

  The buffer memory 51 temporarily stores an image signal supplied from the IEEE 1394 interface 25 based on a control signal from the CPU 56. The image signal temporarily stored in the buffer memory 51 is supplied to the encoder 52, and is compressed and encoded based on a standard such as JPEG (Joint Photographic Experts Group). Incidentally, the encoder 52 may add position information and metadata supplied from the metadata generation unit 29 to an image signal to be compressed and encoded. The encoder 52 outputs the compressed and encoded image signal to the server 53 or the image compression unit 54. Note that the processing in the encoder 52 is omitted when compression encoding is not performed on the supplied image signal.

  The server 53 sequentially records the image signal output from the encoder 52 in association with the position information and metadata. Incidentally, the server 53 may be replaced with, for example, a hard disk or a removable disk-shaped recording medium. The image signal recorded in the server 53 is read out to the image compression unit 54 and the graphic controller 55 based on control by the CPU 56. By controlling the image signal recorded in the server 53 to be recorded in the memory card 61, the user can also transfer the captured image to another PC. Further, by controlling the image signal recorded in the server 53 to be recorded in the network server (not shown), the server 53 can be replaced with a network server (not shown).

  The image compression unit 54 generates a compressed image or a thumbnail image for each JPEG format image signal read from the server 53. In addition, the image compression unit 54 reads an image recorded in the server 53 and generates a moving image in accordance with control by the CPU 56. Incidentally, as a compression method for generating this moving image, it may be executed based on, for example, MPEG, Motion-JPEG, Motion-JPEG2000 or the like.

  The graphic controller 55 executes a painting process on the display 6 based on the image signal read from the server 53 or the image signal output from the image compression unit 54. The graphic controller 55 controls contrast and luminance in the display 6 based on control by the CPU 56.

  The CPU 56 controls a drive signal for driving the pantilter unit 3 and the lens control unit 23 and each unit in the monitoring device 5 when an image area and an image position are designated by the user via the keyboard 59 and the mouse 60. A control signal for transmitting is transmitted via the controller bus 21. The CPU 56 receives a predetermined request signal from the terminal device 9, selects an optimal still image, moving image, or various information recorded in the server 53, and transmits this to the terminal device 9. To control.

  Next, an imaging operation in the monitoring system 1 to which the present invention is applied will be described.

  FIG. 4 shows a case where the camera unit 2 captures an image within a shooting range indicated by a black frame with a shooting angle of view u. In order to capture the entire imaging range with the imaging angle of view u, it is necessary to shift the imaging direction sequentially in the horizontal or vertical direction. If the size of the shooting range can be expressed by i × j times the size of a frame (hereinafter referred to as a unit image) obtained by imaging at an arbitrary shooting angle of view u, at least i × j shooting directions are set. There is a need to. An entire image representing the entire imaging range can be synthesized by pasting i × j unit images captured at the imaging angle of view u.

  Here, the coordinates (M, N) of each unit image in the photographing range are 1, 2,..., M, i from the left end in the horizontal direction, and 1, 2,. .., J, the CPU 56 sends a drive signal to the Tilt unit 3a and the Pan unit 3b, so that the shooting direction of the camera unit 4 is first matched with the coordinates (1, 1) located at the upper left. Perform imaging. An image signal based on the unit image generated by imaging the coordinates (1, 1) is temporarily stored in the buffer memory 51, and is compressed and encoded by the encoder 52 based on the JPEG standard. This image signal is simultaneously recorded with position information and metadata indicating the shooting direction transmitted from the GPS 28 and recorded in the server 53.

  Similarly, the CPU 56 transmits a drive signal to the Tilt unit 3a and the Pan unit 3b, thereby shifting the shooting direction of the camera unit 4 to the right by one frame and taking an image in accordance with the coordinates (2, 1). Execute. An image signal generated by taking an image of the coordinates (2, 1) is also recorded in the server 53. Based on the control by the CPU 56, the camera unit 4 executes imaging by sequentially changing the imaging direction in the horizontal direction as coordinates (3, 1), (4, 1)... (I, 1).

  After the imaging of the first row is completed, the camera unit 4 adjusts the shooting direction to the coordinates (1, 2) of the second row based on the control by the CPU 56, executes the imaging, and then sequentially in the horizontal direction. Imaging is performed while shifting. When such an operation is repeated and imaging is finished up to the coordinates (i, j), the server 53 enters a state in which image signals based on i × j unit images taken for each coordinate are recorded.

  Incidentally, the image signal based on each unit image recorded in the server 53 is sequentially read out by the image compression unit 54 and reduced so as to fit the size of the display screen of the display 6. Each reduced unit image is displayed on the display 6 via the graphic controller 15. By displaying all the i × j unit images recorded in the server 53 on the display 6, one panoramic overall image (panoramic image) is synthesized. By executing the above imaging operation at regular intervals, it is possible to acquire an entire image showing the latest situation of the imaging range.

  FIG. 5 shows an example in which the entire image synthesized by pasting together the captured i × j unit images is displayed on the entire image display unit 70 of the display 6. The monitoring device 5 may cause the entire image display unit 70 to display a boundary between the unit images constituting the entire image, or may display only a seamless entire image. In addition, the monitoring device 5 may cause the entire image display unit 70 to display one entire image captured at an imaging angle of view that can capture the entire imaging range as an alternative to the panoramic entire image.

  Incidentally, the display screen 45 is further provided with an enlarged image display unit 71 for displaying an enlarged image obtained by enlarging the unit image. The enlarged image display unit 71 may enlarge and display one unit image designated by the user among unit images constituting the entire image displayed on the entire image display unit 70. A moving image may be sequentially displayed for each unit image shooting direction. Thereby, the user can also confirm in real time the situation of the shooting direction in the specified unit image.

  The user can designate a desired image region and image position on the entire image display unit 70 and the enlarged image display unit 71 using the keyboard 59 and the mouse 60. The display units 70 and 71 may further display a line of sight and a pointer for executing the above-described designation operation in conjunction with the movement of the mouse 60 or the like.

  Further, by executing an input operation via the mouse 60 on the display screen 45, the shooting magnification of the unit image displayed on the enlarged image display unit 71 is enlarged or reduced, and the shooting direction of the camera unit 4 is set to be horizontal or vertical. It is possible to perform shooting direction control for adjustment in the direction and setting of various modes. Incidentally, the history of the whole image displayed on the whole image display unit 70 is displayed via the history display unit 81, and the density of the whole image displayed on the whole image display unit 70 is shown via the shading display unit 82. Is displayed. The shooting magnification changed for each unit image displayed on the enlarged image display unit 71 is displayed via the magnification display unit 83.

  In addition, when the difference detection button 84 displayed on the display screen 45 is clicked via the mouse 60, one unit image constituting the generated whole image and the unit when the difference detection described later is performed. A difference in luminance level is detected from a unit image in the same shooting direction that is imaged before the image.

  This difference detection is executed by comparing the brightness level of the entire image composed of unit images recorded in the server 53 with the reference entire image. Here, the entire image for detecting a difference on the image is referred to as a comparison entire image, and the reference entire image is referred to as a reference entire image.

  As shown in FIG. 6A, in the case where the reference entire image is the entire image a1, the entire image a2 is formed when a difference occurring on the image is detected for the entire image a2 as the comparison entire image. The luminance level is compared between the unit image and the unit images constituting the entire image a1. As shown in FIG. 6A, the luminance levels are compared by unit images at the same coordinates (1, 1), (2, 1), (3, 1),..., (M, N). In other words, it is executed between unit images in the same shooting direction. Thereby, the change of the luminance level of the comparison whole image with respect to the reference whole image can be detected according to each photographing direction.

  When comparing the luminance level of the unit image at the coordinates (M, N), for example, between the comparison whole image (whole image a2) and the reference whole image (whole image a1), the CPU 56 firstly compares these two images. One unit image is read from the server 53. Next, for example, as shown in FIG. 6B, the CPU 56 compares the luminance level for each pixel (m, n) at the same position between the two read unit images. In the comparison of the luminance levels, the luminance level difference may be obtained for the pixel (m, n) at the same position. The CPU 56 can detect the difference in luminance level between the read unit images via the luminance level difference value obtained for each pixel. Incidentally, the comparison of the luminance levels between the pixels may be performed for all the pixels constituting the unit image, or may be performed for only some of the pixels constituting the unit image.

  Incidentally, in the present invention, the luminance levels may be compared as described above for only some of the unit images constituting the comparison overall image. At this time, the user can detect a difference in luminance level of only the designated unit image by designating a desired unit image using the mouse 60 or the like.

  When a unit image constituting the entire image a3 is recorded on the server 53 at the next timing, the entire image a3 becomes a comparison entire image, and the same shooting direction is set between the entire image a1 as the reference entire image. A difference is detected between the unit images at. Further, at the next timing, when a unit image constituting the entire image a4 is recorded in the server 53, the entire image a4 becomes a comparison entire image, and the same shooting direction is obtained between the entire image a1 as the reference entire image. A difference is detected between the unit images at.

  Next, a procedure for monitoring the seating situation by the monitoring system 1 to which the present invention is applied will be described with reference to FIG.

  First, after step S11, a so-called vacant seat whole image is acquired before the audience is seated. In step S <b> 12, the monitoring device 5 sets a shooting range based on a user instruction via the operation unit 16 so that all seats where the moving subject (audience) is seated can be captured, and the shooting angle of view u is set. decide.

  Next, the process proceeds to step S <b> 13, and the monitoring system 1 captures the entire image in the vacant seat state and records this in the server 53. The whole image recorded in the server 53 becomes the reference whole image. The reference whole image is sequentially read out by the image compression unit 54 and displayed on the whole image display unit 70 in the display 6.

  Next, the process proceeds to step S <b> 14, and the user (monitor) monitors an area for detecting a difference on the image via the operation unit 16 (hereinafter, referred to as “visual difference”) while viewing the reference entire image displayed on the entire image display unit 70. This is called the difference detection frame.) FIG. 8 shows an example of the difference detection frame set by the user. It can be seen that the difference detection frames (ID-1 to ID-8) are set for each seat position in the reference whole image in the vacant seat state.

  Incidentally, the difference detection frame can be arbitrarily set. For example, when the seating situation is monitored only for the designated seat, the difference detection frame may be set only for the designated seat. In this embodiment, the case where a rectangle is displayed on the bitmap of the acquired reference whole image as the difference detection frame to be set has been described. However, the present invention is not limited to such a case. A frame may be displayed. In addition, when an ID number (ID-n) is attached to each difference detection frame, it is possible to facilitate the collation on the database by uniquely associating it with the seat number. Further, the present invention is not limited to the case where the difference detection frame is provided, and for example, a difference in luminance level may be detected for all unit images constituting the entire image.

  Next, the process proceeds to step S15, and the entire image when the audience is seated is acquired. The entire image is captured within the imaging range set in step S11 described above and under the same imaging angle of view u. For example, if the photographing system 1 is installed in a movie theater, the timing for capturing the entire image in step S15 is during screening, but it may be arbitrarily determined according to the usage situation. In addition, when the spectator moves at short time intervals, it is not possible to effectively detect the difference only by acquiring the entire image only at a specific time point. Therefore, several entire images may be acquired continuously. .

  Next, the process proceeds to step S <b> 15, and the monitoring system 1 captures the whole image when seated and records this in the server 53. The whole image recorded in the server 53 becomes a comparison whole image. The comparison whole images are sequentially read out by the image compression unit 54 and displayed on the whole image display unit 70 in the display 6.

  Here, when the difference detection button 84 displayed on the display screen 45 is clicked via the mouse 60, the luminance between the unit images in the same shooting direction between the reference whole image and the comparison whole image. A level difference is detected. From the detected luminance level difference, the luminance level difference in the pixels constituting the difference detection frame is extracted. In this monitoring apparatus 5, when the difference detection button 84 is clicked with the mouse, the difference may be detected not only when the difference in luminance level is detected, but also at a predetermined interval, or the camera. You may make it detect a difference for every imaging in the part 4. FIG.

  At this time, the monitoring device 5 is not limited to the case of comparing the difference in luminance level between all the unit images constituting the reference whole image and the comparison whole image. For example, the unit image in which a difference detection frame is set. Difference detection may be performed by focusing only on the image. For example, as shown in FIG. 9, when the difference detection frame ID-n is set over the unit images (1, 1), (2, 1), (1, 2), (2, 2), For only the unit images (1, 1), (2, 1), (1, 2), (2, 2), a difference in luminance level is detected between unit images in the same shooting direction, and other unit images ( Such difference detection can be omitted for (3, 1), (1, 3), and (2, 3). Thereby, the processing speed can be increased, and highly efficient difference detection can be realized.

  After the luminance level difference is detected for each unit image including the difference detection frame, the process proceeds to step S17. In step S17, the brightness level difference detected in step S16 is compared with a preset threshold value.

  Here, when the difference in the luminance level of the difference detection frame is detected through the difference value of the luminance level obtained for each pixel, it is determined whether or not the difference value exceeds the threshold value. As a result, when the obtained difference value exceeds the threshold value, it is determined that some difference has occurred in the image region within the difference detection frame as compared with the reference whole image. On the other hand, when the obtained difference value is equal to or smaller than the threshold value, it is determined that no difference has occurred in the image area within the difference detection frame even when compared with the reference whole image.

  In addition, by setting the threshold value used for this determination according to the luminance level of the audience as the shooting target, the difference value obtained when the audience is seated in each seat where the difference detection frame is set is obtained. The threshold will be exceeded. When the seat is vacant, the obtained difference value is equal to or less than the threshold value.

  FIG. 10 shows a comparative overall image obtained by imaging the state in which the audience is seated in some of the seats in which the difference detection frames (ID-1 to ID-8) are set in step S14. As shown in FIG. 10, for the difference detection frames ID-3, ID-4, ID-6, and ID-7 in the seat where the spectator is seated, the luminance level of the spectator is superimposed as compared with the entire reference image. Therefore, the obtained difference value exceeds the threshold value. In contrast, for the difference detection frames ID-1, ID-2, ID-5, and ID-8 in the vacant seat, the luminance level does not change compared to the reference overall image, and the obtained difference value is Below threshold.

  The monitoring device 5 is displayed on the display 6 only when the calculated difference value exceeds the threshold value, in order to alert the monitoring staff about only the difference detection frame and notify the seating of the audience. Some information is displayed together with the entire image (step S18). As a result, it is sufficient for the monitoring staff to confirm the details only when the monitoring device 5 is alerted, and the monitoring effort can be greatly reduced. At this time, hatching or color may be applied to the difference detection frame displayed on the entire image display unit 70 as information to be displayed. Alternatively, identification information may be input in advance for each difference detection frame, and this may be displayed only when the obtained difference value exceeds the threshold value. As a result, it is possible to input the seat reservation status as the identification information in advance and display the seat reservation status as the identification information when the seating of the audience is detected.

  Further, the monitoring device 5 cumulatively stores various information such as the comparison result in this step S17, the audience, and the brightness level of the seat, and compares them with the comparison result newly acquired in step S17. May be displayed via the display 6. For example, as shown in FIG. 10, the monitoring device 5 is assigned identification information indicating reserved in the difference detection frames ID-3, ID-6, and ID-7, and is not reserved in the difference detection frame ID-4. May be displayed on the output screen 45 at the same time. At this time, the characters “empty” indicating that they are not seated may be displayed on the output screen 45 at the same time for the difference detection frame whose calculated difference value is equal to or less than the threshold value.

  By visually recognizing the color of the difference detection frame and the display of the identification information, the monitor can easily identify, for example, that the spectator is sitting in an unreserved seat. Even when the monitor does not visually recognize the display screen 45 on the display 6, this can be left as a history on the monitoring device 5 side. Thereby, the observer can analyze the seating situation of the audience after the fact.

  Further, in addition to displaying the comparison result in step S17 via the display 6, the monitoring device 5 may transmit it to another terminal device 9 via the network 8 based on the procedure shown in FIG. Good.

  First, in step S31, each user who operates the terminal device 9 accesses the monitoring device 5 connected to the network 8.

  Next, the process proceeds to step S <b> 32, and the monitoring device 5 creates a publicly available image list for all images recorded in its own server 53 that can be disclosed to the user, and sends this to the terminal device 9. Send. Incidentally, in this publicly available image list, not only the file name and file size of each whole image but also a reduced whole image may be pasted. This publicly available image list is displayed on the terminal display 10 via the network 8 and the terminal device 9.

  Next, the process proceeds to step S33, and the user selects a desired entire image from the publicly available image list. The terminal device 9 transmits an entire image transmission request C1 to the monitoring device 5 in response to a selection operation by the user.

  In step S <b> 34, the monitoring device 5 receives the entire image transmission request C <b> 1, reads the entire image selected by the user and the accumulated difference information from the server 53, and transmits this to the terminal device 9. The transmitted whole image is displayed on the terminal display 10 through the network 8 and the terminal device 9 in a superimposed manner with the accumulated difference information as described above (step S65). Thereby, monitoring from a remote location can also be realized.

  Such a monitoring system 1 can also be installed in each projection room of a complex movie theater called a so-called cinema complex. FIG. 12 (a) shows a result of obtaining a so-called empty reference overall image before the audience is seated. It can be seen that a difference detection frame can be set for each seat individually. FIG. 12B shows the case where the audience sits on one seat in the projection room where the difference detection frame is set. As shown in FIG. 12 (b), the difference detection frame set for each seat has a difference corresponding to the luminance level of the spectator sitting on the seat, and as a result that the obtained difference value exceeds the threshold, hatching or Colors are applied to alert the observer. The monitor can easily identify that the spectator has arrived at the unreserved seat only by visually recognizing the difference detection frame alerted on the output screen 45. In this way, it is possible to check the free entry and exit between the projection rooms by the spectators who enter the hall, particularly in a complex movie theater that adopts a fee structure that charges a separate entrance fee for each projection room.

  In other words, since the monitoring system 1 can monitor the seating situation of the audience through the luminance level of the pixel, it is extremely accurate as compared with the case where the switch incorporating the piezoelectric element is used, and even a slight change is observed. It can be accurately captured. In addition, by setting the threshold according to the audience brightness level more accurately, for example, even when luggage or other transported items are placed on the seat, this can be easily identified, and only as a monitoring target Monitoring that focuses on the audience can be realized. In other words, by optimizing the threshold value according to the desired monitoring target, the monitoring person can select the monitoring target himself, avoiding cumulative monitoring of unnecessary moving subjects, and improving the convenience of analysis. It can also be improved.

  In the monitoring system 1 to which the present invention is applied, even when the illumination in the projection room is turned off during the screening, it is possible to perform imaging based only on the screen illumination. As a result, the entire image that can be acquired is apparently dark, but a slight change in luminance level according to the seating of the audience can be detected. For this reason, the above threshold value is set in advance assuming that the image is picked up based only on the screen illumination, and it is not overlooked even by a slight change in luminance level by comparison with the set threshold value. It is possible to monitor the seating of the audience at any time. In other words, in the monitoring system 1 to which the present invention is applied, the convenience of analysis can be improved by further optimizing the threshold corresponding to the photographing environment in advance.

  In the monitoring system 1 to which the present invention is applied, when performing imaging in a dark place, imaging may be performed using an infrared cut filter in order to further improve detection accuracy. Hereinafter, when imaging is performed through the infrared cut filter, it is referred to as a Night mode. For an image captured in a normal bright place, difference detection is performed based on the data of the R, G, B, and Gray elements. In this Night mode, difference detection is performed based only on the data of the Gray element. This is because the Night mode image is mainly data of brightness only.

  FIG. 13 shows a difference detection procedure when such a Night mode is introduced. When the process proceeds to step S16 described above and difference detection is started within the difference detection frame (step S71), the process proceeds to step S72, and it is determined whether or not to shift to the Night mode from the obtained image. As a result, if it is determined to enter the Night mode, the process proceeds to step S74, and otherwise, the process proceeds to step S73.

  When the process proceeds to step S73, difference detection is performed based on the data of the R, G, B, and Gray elements for the captured image, and then the process proceeds to step S75. On the other hand, when the process proceeds to step S74, the difference detection is executed based on only the gray data for the captured image, and then the process proceeds to step S75.

  In step S75, the detected luminance level difference is compared with a preset threshold value. Since the subsequent processing is the same as that in step S16 described above, description thereof is omitted.

  Further, the monitoring system 1 optimizes the threshold according to the luminance level of the subject to be photographed and the photographing environment, so that the monitoring system 1 is not limited to the application in the movie room of the complex movie theater, but for example, as shown in FIG. It is possible to monitor the seating situation in the same manner. First, wide-angle imaging is performed to image a wide range of seats as shown in FIG. In this imaged seat, for example, when the inside of the dotted line frame shown in FIG. 14 (a) is a specially designated seat, the area is enlarged, and a difference detection frame is provided for each seat as shown in FIG. 14 (b). Set. Further, since the brightness varies depending on whether the stadium is indoor type or outdoor type, the threshold value may be optimized in advance according to the shooting environment.

  Moreover, in the monitoring system 1 to which the present invention is applied, by installing the camera unit 2 in, for example, an open space parking lot, it is possible to acquire the availability of each parking space without using a sensor. In such a case, the shooting direction and the shooting angle of view of the camera unit 4 are adjusted so as to include a parking lot for acquiring the availability, and a threshold value is set in advance according to the luminance level of the car to be monitored. A difference detection frame is set for each parking space. Thereby, the vacancy situation can be monitored for each parking space. In addition, in the present system 1 capable of monitoring effectively even under low illuminance, it is possible to monitor the vacant situation at any time, not limited to day and night.

  Moreover, in the monitoring system 1 to which the present invention is applied, the reserved boarding ticket can be confirmed by installing the camera unit 2 in, for example, a train. In such a case, the shooting direction and the shooting angle of view of the camera unit 4 are adjusted so as to include the reserved seat, and the confirmation of the reserved ticket is realized by monitoring the passenger sitting in the reserved seat. The threshold value is set in advance according to the luminance level. The difference detection frame is set for each seat. As a result, it is possible to save time for the conductor to go around the seats and check the tickets brought by the passengers one by one. In particular, by transmitting the comparison result of the luminance level in the difference detection frame to the external terminal device 9 via the network 8 as a wireless communication network, the seat availability is monitored centrally for all trains traveling on the route. You can also.

  That is, in the present invention, any moving subject such as a spectator, a car, or a passenger may be a monitoring target, and the shooting range is set so as to capture a storage position such as a seat or a parking space in which the monitoring target is stored. Once set, it can be applied to any situation monitoring.

  Further, in the monitoring system 1 to which the present invention is applied, the above-described comparison of luminance levels between unit images may be executed for each of R, G, and B primary color components. As described above, this luminance level comparison is performed between unit images at the same coordinate (M, N), in other words, between unit images in the same shooting direction. Thereby, the change of the luminance level in each primary color component of the comparison entire image with respect to the reference entire image, in other words, the difference value of the luminance level in each primary color component can be detected for each photographing direction.

  Further, in the monitoring system 1 to which the present invention is applied, a vector based on the normalized correlation value may be sequentially detected instead of comparing the luminance levels between the above-described unit images.

  In such a case, the arithmetic processing unit 15 calculates the normalized correlation value while shifting the other unit image by at least one pixel with respect to the reference pixel position in the difference detection frame of one unit image, thereby obtaining the maximum correlation. The obtained target position is specified. Next, a vector (hereinafter referred to as a luminance vector) obtained between the target position and the reference pixel position is extracted. The arithmetic processing unit 15 executes this operation for all the difference detection frames described above.

  Next, the arithmetic processing unit 15 sequentially identifies the direction of the obtained luminance vector. Then, it is determined whether or not the direction of the identified luminance vector matches the direction of the luminance vector of the same pixel position identified previously. As a result, if a change in luminance vector between the two is detected, it is determined that some difference has been detected for the pixel position. On the other hand, if no change in the luminance vector between the two is detected, it is determined that no difference has occurred for the pixel position.

  By the way, this normalized correlation value becomes high when the pixels to be compared together constitute a bright area, and when either one constitutes a bright area but the other constitutes a dark area. Lower. Therefore, by identifying the luminance vector based on these normalized correlation values, there is no difference because the direction of the luminance vector does not change for the image area that is only bright overall and has no change. It can be judged as. Further, an area that is dark overall and has a small difference in luminance level but has some change, can be determined as a difference portion because the direction of the luminance vector changes.

  Thereby, in the monitoring system 1 which can detect this brightness | luminance vector sequentially, the effectiveness of monitoring can be ensured, even when it receives influence by subtle changes, such as illumination intensity fluctuation | variation of a light source, and ambient brightness.

  Furthermore, it goes without saying that the present invention may be applied to a program for causing a computer to execute the processing described above, or a recording medium on which such a program is recorded.

It is a figure which shows the structure of the monitoring system to which this invention is applied. It is a figure for demonstrating further about the structure of the monitoring system to which this invention is applied. It is a block block diagram of a camera unit and a monitoring device. It is a figure for demonstrating the case where the camera unit images within the imaging | photography range shown with a black frame with the imaging | photography angle of view u. It is a figure which shows the structural example of the display screen on a display. It is a figure for demonstrating operation | movement of the monitoring system to which this invention is applied. It is a flowchart for demonstrating about the procedure which compares the difference of a luminance level. It is a figure which shows an example of the difference detection frame set by the user. It is a figure for demonstrating about the case where a difference detection is focused on only the unit image in which the difference detection frame was set. It is a figure for demonstrating about the case where the identification information which shows a reservation condition is shown on an output screen with a whole image. It is a flowchart which shows the procedure which transmits the comparison result in step S17 to a terminal device. It is a figure which shows the whole image acquired by the monitoring system which concerns on this invention installed in the compound movie theater. It is a flowchart which shows the difference detection procedure in the monitoring system which introduced Night mode. It is a figure for demonstrating about the case where the seat condition of a stadium is monitored.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Monitoring system, 2 Camera unit, 3 Pantilta part, 4 Camera part, 5 Monitoring apparatus, 6 Display, 8 Network, 9 Terminal apparatus, 10 Terminal display, 22 Lens part, 23 Lens control part, 24 Imaging part, 25 IEEE1394 interface , 28 GPS reception unit, 29 metadata generation unit, 51 buffer memory, 52 encoder, 53 server, 54 image compression unit, 55 graphic controller, 56 CPU, 58 I / O port, 59 keyboard, 60 mouse

Claims (16)

In a monitoring device that monitors a panoramic image generated by imaging a shooting range by sequentially changing shooting directions,
Shooting setting means for setting the shooting range so as to capture a storage position where a moving subject as a monitoring target is stored;
A difference detecting means for sequentially detecting a difference in luminance level between one unit image constituting the generated panoramic image and a unit image in the same photographing direction imaged before the unit image;
Comparing means for comparing the difference in brightness level detected by the difference detection means with a threshold value set in advance according to the brightness level of the moving subject;
A monitoring apparatus comprising: display control means for displaying predetermined information together with the generated panoramic image on a display surface based on a comparison result by the comparison means. The difference detection means sequentially detects a difference value of a luminance level between one unit image constituting the generated panoramic image and a unit image taken in the same shooting direction before the unit image. And
The comparison means compares the difference value of the luminance level obtained by the difference detection means with the set threshold value,
The monitoring apparatus according to claim 1, wherein the display control unit displays the predetermined information on the display surface when a difference value of the luminance level exceeds the set threshold value. The monitoring apparatus according to claim 1, wherein the difference detection unit detects a difference in luminance level between unit images including an image area constituting the accommodation position. The monitoring device according to claim 1, wherein the comparison unit compares the difference in luminance level detected by the difference detection unit with a threshold value set in advance in accordance with a photographing environment. The monitoring apparatus according to claim 1, further comprising selection means for selecting a unit image for detecting the difference in luminance level. The monitoring apparatus according to claim 1, further comprising adjusting means for adjusting the set threshold value. The monitoring apparatus according to claim 1, further comprising: a transmission unit that transmits a comparison result by the comparison unit to another electronic device via a connected communication network. The shooting setting means sets the shooting range so as to capture the seat as the accommodation position,
The monitoring device according to claim 1, wherein the comparison unit compares the difference in luminance level with a threshold value set in advance according to a luminance level of the human being as the moving subject. The monitoring apparatus according to claim 1, further comprising an imaging unit that sequentially images the unit images constituting the panoramic image in the imaging direction. The monitoring apparatus according to claim 1, further comprising a server that records the captured unit image on a recording medium. In a monitoring device that monitors a panoramic image generated by imaging a shooting range by sequentially changing shooting directions,
Shooting setting means for setting the shooting range so as to capture a storage position where a moving subject as a monitoring target is stored;
A vector for sequentially detecting a vector based on a normalized correlation value obtained between one unit image constituting the generated panoramic image and a unit image taken in the same shooting direction before the unit image. Detection means;
Comparing means for comparing the direction of the vector detected by the vector detecting means with the previously detected vector;
A monitoring apparatus comprising: display control means for displaying predetermined information together with the generated panoramic image on a display surface based on a comparison result by the comparison means. The monitoring apparatus according to claim 11, wherein the vector detection unit obtains the normalized correlation value between unit images including an image region constituting the accommodation position. In a monitoring method for monitoring a panoramic image generated by imaging a shooting range by sequentially changing shooting directions,
Set the shooting range to capture the storage position where the moving subject as the monitoring target is stored,
A difference in luminance level is sequentially detected between one unit image constituting the generated panoramic image and a unit image taken in the same shooting direction before the unit image,
The detected brightness level difference is compared with a threshold set in advance according to the brightness level of the moving subject,
A monitoring method, comprising: displaying predetermined information together with the generated panoramic image on a display surface based on the comparison result. In a program for causing a computer to monitor a panoramic image generated by sequentially changing a shooting direction and imaging a shooting range,
Set the shooting range to capture the storage position where the moving subject as the monitoring target is stored,
A difference in luminance level is sequentially detected between one unit image constituting the generated panoramic image and a unit image taken in the same shooting direction before the unit image,
The detected brightness level difference is compared with a threshold set in advance according to the brightness level of the moving subject,
A program for causing a computer to display predetermined information on a display surface together with the generated panoramic image based on the comparison result. In a recording medium recorded with a program for causing a computer to monitor a panoramic image generated by sequentially changing the shooting direction and imaging a shooting range,
Set the shooting range to capture the storage position where the moving subject as the monitoring target is stored,
A difference in luminance level is sequentially detected between one unit image constituting the generated panoramic image and a unit image taken in the same shooting direction before the unit image,
The detected brightness level difference is compared with a threshold set in advance according to the brightness level of the moving subject,
A recording medium on which is recorded a program for causing a computer to display predetermined information on a display surface together with the generated panoramic image based on the comparison result. An imaging device that generates a panoramic image by imaging a shooting range by sequentially changing shooting directions;
The photographing range is set so as to capture the accommodation position where the moving subject as the monitoring target is accommodated, and one unit image constituting the generated panoramic image and the same photographing direction captured before the unit image are captured. The difference in luminance level between the unit images is sequentially detected, the difference in the detected luminance level is compared with a threshold value set in advance according to the luminance level of the moving subject, and the comparison result is compared. A control device for transmitting via a communication network;
A monitoring system comprising: at least a terminal device that displays the comparison result received from the control device via a communication network on a display surface.

Images