JP6270111B2 - Mobile object display system and mobile object display program - Google Patents

Description

  The present invention relates to a mobile object display system and a mobile object display program.

  As a conventional example of a system for automatically tracking and monitoring a moving body, a system including a plurality of turning cameras can be cited. For example, Patent Document 1 includes a moving object detection camera that images a moving object over a wide imaging range and a monitoring camera that magnifies and captures the moving object, and turns both cameras according to the position of the moving object. A surveillance camera system is described. However, such a system that requires a plurality of turning cameras tends to be an expensive system because it also requires turning means for the cameras.

  On the other hand, a fish-eye lens camera (hereinafter referred to as an omnidirectional camera) that captures a 180-degree hemispherical field space around the camera as a fisheye image (hereinafter referred to as an all-round image) is known as a non-rotating camera. Patent Document 2 describes a technique for correcting an image taken by an omnidirectional camera. By using an omnidirectional camera, only one camera is required, and no camera turning drive means is required, so an inexpensive shim tem can be expected.

JP-A-11-103457 Japanese Patent No. 4048511

  Conventionally, when a monitoring target is displayed on a monitoring monitor in a system using an omnidirectional camera, only one of the all-round image and the gaze image is displayed. The “gaze image” here is an image that is corrected by giving a viewpoint parameter at an arbitrary angle to the all-round image, and is an enlarged image of an area where a gaze target, for example, a moving body is located. The assigning of the viewpoint parameter is performed when the supervisor designates an area where the gaze target exists in the all-round image displayed on the monitoring monitor by operating the mouse of the personal computer. FIG. 11A is an image diagram of the all-round image V1 displayed on the monitoring monitor. The area inside the circle C is a portion that is distorted and displayed in a fish-eye shape by an omnidirectional camera, and the area outside the circle C appears black as an outer frame obtained from the CCD element (in the figure, it is shaded for convenience) Show). Conventionally, when it is desired to watch the moving object (person) J shown in the all-round image V1, the supervisor designates the area A around the moving object J by operating the mouse. A viewpoint parameter is given by this mouse operation, and a gaze image V2 (FIG. 11B) which is a corrected enlarged image of the region A is displayed on the monitoring monitor instead of the all-round image V1.

  However, in the technique for displaying the gaze image V2 by operating the mouse, when the moving body J moves, as shown in FIGS. 11C and 11D, the moving body J is displayed on the edge of the gaze image V2, or the gaze image is displayed. May deviate from V2. In this case, the monitor returns to the all-round image V1 again, and the area A around the moving body J is redesignated by the mouse operation, which takes time to handle the gaze image V2.

  The present invention has been created to solve such problems, and provides a mobile object display system and a mobile object display program that are inexpensive and can display a gaze image without requiring troublesome operations. The purpose is that.

In order to solve the above-mentioned problem, the present invention includes an omnidirectional camera that captures a space in the hemispherical field of view of the camera as an all-round image, and displays a substantial imaging region inside the circle of the all-round image at the center of the monitor. In the moving object display system, a moving object extracting unit that detects and tracks a moving object from a captured image of the omnidirectional camera, and a gaze image that generates a gaze image by enlarging the periphery of the moving object tracked by the moving object extracting unit. comprising a generator, the gaze image generated by the gaze image generating unit generates a composite image of the structure that overlaps a portion of the entire periphery image and the composite image generating unit for displaying the composite image monitor, a, The composite image generation unit is characterized in that a gaze image of a new moving object is arranged with priority over a corner portion of an all-round image .

According to this mobile object display system, the following effects are produced.
(1) Since the all-around image and the gaze image are displayed at the same time, monitoring the all-around image covering a wide area makes it possible to easily grasp the general number of people and grasp the flow of the person, and The movement of the can be grasped in detail.
(2) Since only a single omnidirectional camera is required as a camera, it is less expensive than the prior art that requires two cameras, a moving object detection camera that covers a wide imaging range and a monitoring camera that enlarges and captures the moving object. Display system.
(3) The moving object tracking unit of the moving object extracting unit can always automatically display the moving object on the gaze image, and no human operation such as a mouse operation is required to display the gaze image.

  The outer frame portion other than the substantial imaging area inside the circle in the entire circumference image is a non-imaging area, and this non-imaging area is widely occupied by the corner portion. Accordingly, when the size of the rectangular image of the gaze image is large enough to overlap the substantial shooting area of the entire circumference image, the gaze image is preferentially arranged in the corner portion, so that the overlap amount of the gaze image with respect to the entire circumference image is increased. And the hindrance to the monitoring of the all-around image can be suppressed.

  Further, according to the present invention, the composite image generation unit arranges the gaze image of the new moving body at a corner portion farthest from the center of gravity coordinates of the moving body among the corner portions of the vacant all-round image. Features.

  According to this moving body display system, since the distance between the position of the moving body and the gaze image in the all-round image can be increased, the frequency at which the moving body moves and is hidden in the gaze image in the all-round image is probabilistically reduced. be able to.

  Further, according to the present invention, the all-round image is a horizontally long image, and the composite image generation unit displays a gaze image of a new moving object when the gaze image is already arranged at all corners. It arrange | positions in the edge center part far from the gravity center coordinate of the said mobile body within the edge center part.

  According to this mobile object display system, the gaze image is arranged at the center of the right and left edges that occupy the non-photographing area next to the corner portion, thereby suppressing the overlap margin of the gaze image with respect to the all-round image as much as possible. The obstruction of monitoring can be suppressed. Moreover, since the distance between the position of the moving body and the gaze image in the all-round image can be increased by arranging it at the center of the edge far from the center of gravity coordinates of the moving body, the moving body moves in the all-round image and gazes. The frequency of hiding in the image can be reduced.

  Further, according to the present invention, when the gaze image has already been arranged at the left and right edge center portions, the synthesized image generation unit displays the gaze image of the new moving body within the upper and lower edge center portions. It is arranged at the center of the edge far from the barycentric coordinates.

  According to this mobile object display system, the overlap of the gaze image with respect to the all-around image is suppressed as much as possible by arranging the gaze image at the middle of the upper and lower edges that occupy the non-photographing area next to the center of the left and right edges. It is possible to suppress the hindrance to monitoring the peripheral images. Moreover, since the distance between the position of the moving body and the gaze image in the all-round image can be increased by arranging it at the center of the edge far from the center of gravity coordinates of the moving body, the moving body moves in the all-round image and gazes. The frequency of hiding in the image can be reduced.

Further, the present invention, the mobile object extraction unit, the same pixel motion vector a macroblock and forward predictive coding of intra encoding a threshold value above T ₁ of macroblocks contained in a P-picture frame When a binary moving object map is created as a value, a motion region surrounding the aggregate of pixel values is created, and the number of intra-coded macroblocks in the motion region is greater than or equal to a threshold T ₂ The assembly is detected as a moving body.

  According to this moving object display system, a moving object can be accurately detected even in compressed video data at a low frame rate (for example, 8 fps).

  Further, according to the present invention, the moving body extraction unit obtains a pseudo inertia vector connecting the centroids of motion regions having the same ID between the immediately preceding P picture frame and the current P picture frame, and the current P is calculated from the pseudo inertia vector. It is characterized by predicting the position of a motion region of a picture frame.

  According to this mobile object display system, the tracking accuracy of the mobile object is improved even in compressed video data at a low frame rate (for example, 8 fps).

In addition, the present invention is tracked by a moving object extraction step of detecting and tracking a moving object from a captured image of an omnidirectional camera that captures the space of the hemispherical field of view of the camera as an all-round image on the computer, and the moving object extraction step. Generating a synthesized image having a configuration in which the gaze image generated in the gaze image generation step overlaps a part of the entire circumference image, and generates a synthesized image. a synthetic image generation step of displaying on the monitor, is the execution, the composite image generating step is characterized by placing preferentially gaze image of the new mobile to the corner portion of the entire periphery image.

According to this moving body display program, the following effects are produced.
(1) Since the all-around image and the gaze image are displayed at the same time, monitoring the all-around image covering a wide area makes it possible to easily grasp the general number of people and grasp the flow of the person, and The movement of the can be grasped in detail.
(2) Since this is a display program related to a captured image of a single omnidirectional camera, compared to the conventional technology requiring two cameras, a moving object detection camera that covers a wide imaging range and a monitoring camera that expands and captures the moving object. And a simple display program.
(3) The moving object tracking unit of the moving object extracting unit can always automatically display the moving object on the gaze image, and no human operation such as a mouse operation is required to display the gaze image.

  Advantageous Effects of Invention According to the present invention, a mobile object display system that is inexpensive and can display a gaze image without requiring a laborious operation.

1 is a configuration block diagram of a mobile display system of an embodiment. It is explanatory drawing which shows the display position of the gaze image in a synthesized image. It is explanatory drawing which shows an example of a synthesized image. (A), (b) is a block diagram of the configuration of the motion detection unit and the motion tracking unit, respectively. It is an image figure of a moving object map and a motion area. It is a processing flow figure of a motion detection part. It is a processing flow figure of a motion tracking part. It is a top view which shows the method of extracting a predetermined | prescribed rectangular area | region from a perimeter image. It is a perspective view which shows the method of calculating | requiring a viewpoint parameter by polar coordinate transformation from a distortion plane image. It is a flowchart which shows an example of the display position determination of a gaze image. It is explanatory drawing which shows the display method of the conventional gaze image.

  In FIG. 1, the mobile object display system 1 of the present embodiment captures a space around the camera (360 ° in all directions) and a field of view of a hemispherical lens, for example, approximately 180 ° as a full-circle image V1 (FIG. 2). An omnidirectional camera 2 is provided. The omnidirectional camera 2 is a camera that can simultaneously deliver captured image data as two types of data, compressed video data of the H.264 standard and JPEG data. In general, since the display unit of the monitor 6 has a horizontally long rectangular shape, the all-round image V1 displayed on the monitor 6 is a horizontally long rectangular image as shown in FIG. The all-round image V1 is a substantial photographing region in which the inside of the circle C is displayed in a fish-eye distorted manner, and the outer region of the circle C is a region that appears black as an outer frame obtained from the CCD element (in the figure, for convenience, a mesh). (Indicated by multiplication).

  In FIG. 1, a moving body display system 1 detects a moving body from a captured image of an omnidirectional camera 2 and a moving body extracting unit 3 that detects and tracks the moving body, and enlarges a gaze around the moving body tracked by the moving body extracting unit 3. A gaze image generation unit 4 generated as an image V2 (FIG. 2), and a composite image V3 having a configuration in which the gaze image V2 generated by the gaze image generation unit 4 overlaps a part of the entire circumference image V1 are generated. And a composite image generation unit 5 that displays V3 on the monitor 6.

"Moving object extraction unit 3"
The moving body extraction unit 3 according to the present embodiment includes the motion detection unit 11 and the motion tracking unit 21 illustrated in FIG.

"Motion detector 11"
In FIG. 4A, the motion detection unit 11 includes an encoding determination unit 12, a motion vector calculation unit 13, a binarization processing unit 14, a motion region creation unit 15, and a moving body identification unit 16. Prepare.

"Encoding determination unit 12"
The encoding determination unit 12 inputs compressed video data captured by the omnidirectional camera 2 and processed according to the H.264 standard via the H.264 capture unit 7 (FIG. 1). The compressed video data includes three frame coding modes (an intra-frame coding frame (I picture frame), a forward prediction frame (P picture frame), and a bidirectional prediction frame (B picture frame)). Further, the compressed video data includes intra coding using only information in a frame and forward predictive coding referring to a past frame as a block coding mode used for compression of a block constituting a macroblock. . The encoding determination unit 12 determines whether the encoding of individual macroblocks included in each P picture frame is intra encoding or forward predictive encoding.

"Motion vector calculation unit 13"
For the macroblock determined to be forward prediction encoding by the encoding determination unit 12, the motion vector calculation unit 13 acquires a motion vector from a motion vector detection unit (not shown) implemented in the H.264 standard, The motion vector of the entire macro block is calculated. As an example, a macroblock consisting of 16 × 16 pixels is divided into a plurality of partitions (for example, a block size of 8 × 8 pixels), and the motion vector of the entire macroblock is calculated from the motion vector of each partition.

“Binarization processing unit 14”
For a macroblock determined to be intra-coded by the encoding determination unit 12, the binarization processing unit 14 sets the pixel value corresponding to the macroblock to 1. Further, the encoding determining unit 12 for the forward predictive coding and the determined macroblocks, binarization processing section 14, the magnitude of the motion vector calculated by the motion vector calculation unit 13 is the threshold value above T ₁ when the pixel values corresponding to the macroblock is 1 and when it is less than the threshold value T _1, the pixel values corresponding to the macroblock to zero.

“Movement area creation unit 15”
In the binarization distribution map (moving object map 17 (FIG. 5)) created by the binarization processing unit 14, the motion region creation unit 15 is connected with a pixel value 1 in, for example, four directions, up, down, left, and right. A rectangular region surrounding a so-called “4-connected” component or an “8-connected” component (hereinafter referred to as an aggregate S) with four additional oblique directions is created. Hereinafter, this rectangular area is referred to as a motion area BBox. In the moving object map 17 of FIG. 5, the pixel value “1” is shown as white, the pixel value “0” is shown as black (shown in gray for convenience), and a case where three moving regions BBox are created is shown. Yes.

"Moving object identification unit 16"
Mobile identification unit 16, the motion region BBox created by the motion region creating section 15, if the number of macroblocks intra-coded contained in the motion area BBox is certain threshold T ₂ or more, The assembly S in the motion region BBox is regarded as a moving body, and the moving body is specified. For example, in FIG. 5, two motion regions BBox ₁ and motion region BBox ₂ are regarded as moving bodies when the number of intra-coded macroblocks is greater than or equal to a threshold T ₂ , and motion region BBox ₃ is number of macroblocks intra-coded is considered noise as was less than the threshold T _2.

FIG. 6 is a process flow diagram of the motion detector 11. The encoding determination unit 12 determines whether encoding of individual macroblocks included in each P picture frame is intra encoding or forward predictive encoding (step S101). For example, the binarization processing unit 14 sets the pixel value corresponding to the macroblock to 1 (step S102). If it is forward prediction encoding in step S101, the motion vector calculation part 13 will calculate the motion vector of the whole macroblock (step S103). Motion vector makes a determination of whether the threshold value above T ₁ (step S104), 2 binarization processing unit 14, if the threshold value above T ₁ and the pixel values corresponding to the macroblock 1 ( step S105), the pixel values corresponding to the macroblock to 0 if less than the threshold value _{T 1} (step S106).

The binarization processing unit 14 creates the moving object map 17 from the pixel values obtained in steps S102, S105, and S106 (step S107), and the motion region creation unit 15 moves the motion region that surrounds the set S of pixel values 1. A BBox is created (step S108). Mobile identification unit 16, the number of intra-macro blocks in the motion area BBox makes a determination of whether there are certain threshold _{T 2} or more (step S109), the moving area if the threshold value _{T 2} or more BBox identified that there is a moving body within (step S110), and identified as noise is less than the threshold _{T 2} (step S111).

"Motion tracking unit 21"
4B, the motion tracking unit 21 includes a motion region prediction unit 22, a motion region comparison determination unit 23, and an ID allocation unit 24.

“Motion Region Prediction Unit 22”
The motion region prediction unit 22 predicts the position of the motion region BBox in the current frame from the motion vector of the motion region BBox that is considered to have a moving body in the immediately preceding frame (this is referred to as a pseudo inertia vector). To do. The pseudo inertia vector is a vector (the start point is the center of the immediately preceding frame and the end point is the center of the current frame) that connects the centers of the motion regions BBox having the same ID as described later between adjacent P picture frames.

"Motion region comparison / determination unit 23"
Based on the position of the motion region BBox predicted by the motion region prediction unit 22, the motion region comparison / determination unit 23 compares the motion region BBox of the previous frame with the motion region BBox created by the motion region creation unit 15 in the current frame. (For example, the rectangular overlap rate and the approximate rate in the pseudo inertia vector direction) are calculated, and the motion region BBox is compared and determined between both frames.

"ID allocation unit 24"
Among the motion area BBox of the current frame compared with the moving area comparison section 23, a motion region having the greatest similarity among there is a motion area BBox with a predetermined threshold value T ₃ or more similarity previous frame The same ID as the ID of the motion area BBox of the immediately preceding frame is assigned to BBox. Motion area BBox similarity smaller than the threshold value _{T 3} is regarded as noise. When different mobile bodies cross each other, the motion areas BBox of the respective mobile bodies are overlapped to form one large motion area BBox, but each movement is performed by allocating and holding the pseudo inertia vector and each ID. Body tracking accuracy is improved.

FIG. 7 is a process flow diagram of the motion tracking unit 21. The motion region prediction unit 22 acquires a pseudo inertia vector of the motion region BBox (step S201), and predicts the position of the motion region BBox of the current frame (frame (t + 1)) from the pseudo inertia vector (step S202). The motion region comparison / determination unit 23 compares the motion region BBox between the immediately preceding frame (frame (t)) and the current frame (frame (t + 1)) (steps S203 and S204). If the motion region BBox degree of similarity is the threshold T ₃ or there are a plurality, if the maximum similarity among them (YES in step S205), identifies the same mobile, the current frame motion region The same ID as that of the immediately preceding motion region BBox is assigned to BBox (step S206). In the case of NO in step S205, it is specified as another moving body (step S207). Also identified as noise if less than the threshold value _{T 3} in step S204 (step S208).

"Gaze image generator 4"
In FIG. 1, JPEG data that is simultaneously distributed from the omnidirectional camera 2 with the compressed video data of the H.264 standard is passed through the JPEG capture unit 8 and the RGB decoding unit 9 to the gaze image generation unit 4 as an all-round image V1. Entered. In creating the gaze image V2 from the entire circumference image V1, viewpoint parameters and an enlargement ratio are required. In this embodiment, in the rectangle information extracted as shown in FIG. 8 (the motion region BBox created by the motion region creation unit 15), the viewpoint parameter is calculated from the center of gravity G of the rectangle, and the enlargement ratio is calculated from the size of the rectangle. By doing so, the gaze image V2 is created. Since the all-round image V1 can theoretically be represented by a hemispherical image, the viewpoint parameter corresponding to the center of gravity G is calculated by polar coordinate transformation as shown in FIG. 9, and the gaze image V2 is output together with the enlargement ratio.

From FIG. 8, the coordinates (x _G, y _G ) of the center of gravity G can be expressed as Equation (1). x _R and y _R are the coordinates of the rectangular reference corner portion R, w _R is the rectangular width, and h _R is the rectangular height.

From FIG. 9, since the viewpoint P is an intersection of a straight line passing through the center of gravity G and a straight line perpendicular to the xy plane and a hemisphere, the viewpoint parameters φ and θ satisfy Expression (2).
However, “0 ≦ φ ≦ π / 2” and “0 ≦ θ <2π”.

The enlargement ratio mp can be expressed by Expression (3) from the extracted rectangular size (w _R , h _R ) and the size (w _i , h _i ) of the all-round image V1. Where k is a positive constant.

  As described above, by using the viewpoint parameters φ and θ and the enlargement ratio mp, it is possible to create the gaze image V2 with the central portion of the rectangle as the viewpoint.

“Composite image generator 5”
In FIG. 1, JPEG data from the omnidirectional camera 2 is also input to the composite image generation unit 5 as the all-round image V 1 via the JPEG capture unit 8 and the RGB decoding unit 9. The composite image generation unit 5 generates a composite image V3 from the entire circumference image V1 and the gaze image V2 input from the gaze image generation unit 4. The synthesized image V3 has a screen configuration in which the gaze image V2 is displayed on the front side and a part of the entire circumference image V1 is hidden accordingly. A moving body tracked by the motion tracking unit 21 is always displayed at the center of the gaze image V2.

  In the present embodiment, the composite image generation unit 5 arranges the gaze image V2 of the new moving body with priority over the corner portions P1 to P4 of the all-round image V1. For example, as shown in FIG. 2, the number of display positions of the gaze image V2 in the composite image V3 is eight in total: four corner portions P1 to P4, left and right edge center portions P5 and P6, and upper and lower edge center portions P7 and P8. is there. A flow example of determining the display position of the gaze image V2 of the new moving body in this case is as follows. Referring to FIG. 10, it is determined whether or not there are vacancies in corner portions P1 to P4 (step S01). If there are vacancies, gaze at the corner portion farthest from the center of gravity coordinates of the moving object in all-round image V1. The image V2 is arranged (step S02). Of course, here, when there is one empty space, the corner portion is interpreted as the corner portion farthest from the center of gravity coordinates of the moving object.

  If there is no space in the corner portions P1 to P4 in step S01, it is determined whether or not there are spaces in the left and right edge center portions P5 and P6 (step S03). The gaze image V2 is arranged at the center of the far edge (step S04). Of course, when there is one vacant space, the center of the edge is interpreted as the center of the edge farthest from the barycentric coordinates of the moving object.

  If there is no vacancy in the left and right edge center portions P5 and P6 in step S03, it is determined whether or not there is a vacancy in the upper and lower edge center portions P7 and P8 (step S05). The gaze image V2 is placed at the center of the edge farthest from the coordinates (step S06). Of course, when there is one vacant space, the center of the edge is interpreted as the center of the edge farthest from the barycentric coordinates of the moving object. If there is no vacancy in the upper and lower edge center portions P7 and P8 in step S05, the process ends abnormally.

  In FIG. 3, the gaze image V2 of the moving body J1 is displayed at the corner portion P1, which is the corner portion farthest from the barycentric coordinates of the moving body J1 in the all-round image V1, and the gaze image V2 of the moving body J2 is the all-round image. A composite image V3 in a state displayed on the corner portion P3 that is the corner portion farthest from the barycentric coordinates of the moving object J2 in V1 is shown.

  Further, the display position of the gaze image V2 is fixed while a certain moving object is being tracked. For example, even if the moving body J1 moves to the upper left in FIG. 3 and the corner portion P1 is not the corner portion farthest from the barycentric coordinates of the moving body J2, the gaze image V2 of the moving body J1 is the corner as it is. It continues to be displayed on the part P1.

"effect"
As described above, the moving body display that includes the omnidirectional camera 2 that captures the space of the hemispherical field of view of the camera as the all-round image V1 and displays the substantial photographing region inside the circle in the all-round image V1 at the center of the monitor 6. In the system 1, a moving body extraction unit 3 that detects and tracks a moving body from a captured image of the omnidirectional camera 2, and a gaze image that is generated as a gaze image V2 by enlarging the periphery of the moving body tracked by the moving body extraction unit 3. The generating unit 4 generates a composite image V3 having a configuration in which the gaze image V2 generated by the gaze image generating unit 4 overlaps a part of the entire circumference image V1, and displays the composite image V3 on the monitor 6 5, the following effects can be obtained.

(1) Since the all-round image V1 and the gaze image V2 are displayed simultaneously, the gaze image can be easily obtained by monitoring the general circumference image V1 covering a wide area and easily grasping the flow line of the person. The movement of a certain person can be grasped in detail by V2.
(2) Since only a single omnidirectional camera 2 is required as a camera, as compared with the conventional technology that requires two cameras, a moving object detection camera that covers a wide imaging range and a monitoring camera that enlarges and captures the moving object. It becomes an inexpensive display system.
(3) The moving body tracking function of the moving body extraction unit 3 can always automatically display the moving body on the gaze image V2, and does not require a human operation such as a mouse operation to display the gaze image V2.

  In addition, a moving object is detected from an image captured by the omnidirectional camera 2 that captures a space around the camera with a hemispherical field of view of approximately 180 ° as an all-round image V1 in a computer (not shown) configured with a CPU, memory, and the like. And a moving body extraction step to be tracked, a gaze image generation step of generating a gaze image V2 by enlarging the periphery of the moving body tracked by the mobile body extraction step, and a gaze image V2 generated by the gaze image generation step According to the moving body display program for generating the composite image V3 having a configuration overlapping with a part of the image V1 and displaying the composite image V3 on the monitor 6, the following effects can be obtained. Is done.

According to this moving body display program, the following effects are produced.
(1) Since the all-around image and the gaze image are displayed at the same time, monitoring the all-around image covering a wide area makes it possible to easily grasp the general number of people and grasp the flow of the person, and The movement of the can be grasped in detail.
(2) Since this is a display program related to a captured image of a single omnidirectional camera, compared to the conventional technology requiring two cameras, a moving object detection camera that covers a wide imaging range and a monitoring camera that expands and captures the moving object. And a simple display program.
(3) The moving object tracking unit of the moving object extracting unit can always automatically display the moving object on the gaze image, and no human operation such as a mouse operation is required to display the gaze image.

  Further, in the rectangular all-round image V1, the outer frame portion other than the substantial photographing area inside the circle is a photographing area, and this photographing area is widely occupied by the corner portions P1 to P4. Accordingly, when the gaze image has a large rectangular size, the gaze image V2 of the new moving body is arranged with priority over the corner portions P1 to P4 of the all-round image V1, thereby overlapping the gaze image V2 with respect to the all-round image V1. And the hindrance to the monitoring of the all-round image V1 can be suppressed.

  Further, the composite image generation unit 5 arranges the gaze image V2 of the new moving body at the corner portion farthest from the barycentric coordinates of the moving body among the corner portions P1 to P4 of the vacant all-round image V1. Then, since the distance between the position of the moving body in the all-round image V1 and the gaze image V2 can be increased, the frequency at which the moving body moves and hides in the gaze image V2 in the all-round image V1 can be reduced. Thereby, the obstruction of the monitoring of the all-round image V1 can be suppressed.

  When the all-around image V1 is a horizontally long image, the composite image generating unit 5 displays the gaze image V2 of the new moving object on the left and right edges when the gaze image V2 is already arranged at all the corner portions P1 to P4. If the central portion P5, P6 is arranged at the center of the edge far from the center of gravity coordinates of the moving body, the image of the gaze at the left and right edge central portions P5, P6 that occupy the non-photographing area next to the corner portions P1-P4. By arranging V2, the overlap margin of the gaze image V2 with respect to the all-round image V1 can be suppressed as much as possible, and hindrance to monitoring of the all-round image V1 can be suppressed. Moreover, since the distance between the position of the moving body in the all-round image V1 and the gaze image V2 can be increased by disposing it at the center of the edge far from the center of gravity coordinates of the moving body, the moving body moves in the all-round image V1. Thus, the frequency of hiding in the gaze image V2 can be reduced.

  Furthermore, when the gaze image V2 is already arranged in the left and right edge center portions P5 and P6, the composite image generation unit 5 displays the gaze image V2 of the new moving body within the upper and lower edge center portions P7 and P8. If the structure is arranged at the center of the edge far from the center of gravity coordinates of the moving body, the gaze image V2 is arranged at the upper and lower edge centers P7 and P8 that occupy the non-photographing area next to the left and right edge centers P5 and P6. Thus, the overlap margin of the gaze image V2 with respect to the all-round image V1 can be suppressed as much as possible, and hindrance to monitoring of the all-round image V1 can be suppressed. Moreover, since the distance between the position of the moving body in the all-round image V1 and the gaze image V2 can be increased by disposing it at the center of the edge far from the center of gravity coordinates of the moving body, the moving body moves in the all-round image V1. Thus, the frequency of hiding in the gaze image V2 can be reduced.

The moving object extraction unit 3, the same pixel value motion vectors a macroblock and forward predictive coding of intra encoding a threshold value above T ₁ of macroblocks contained in a P-picture frame "1" And a motion area BBox surrounding the set S of pixel values “1” is created, and the number of intra-coded macroblocks in the motion area BBox is a threshold T. _{When the number is two} or more, if the aggregate S is detected as a moving object, the moving object can be detected with high accuracy even in compressed video data at a low frame rate (for example, 8 fps).

  Furthermore, the moving body extraction unit 3 obtains a pseudo inertia vector that connects the centroids of the motion regions BBox having the same ID between the immediately preceding P picture frame and the current P picture frame, and determines the current P picture frame from the pseudo inertia vector. If the configuration is such that the position of the motion region BBox is predicted, the following effects can be obtained. Conventionally, it is known to use an average of motion vectors as a tracking unit for a moving object. This is effective when the motion vector changes smoothly due to a high frame rate such as 30 fps, but in the case of compressed video data with a low frame rate that is often used in surveillance cameras, the motion vector changes between adjacent frames. May not be smooth and does not necessarily match the movement of the object. In contrast, in this embodiment, since the difference between the centroids of the motion region BBox is used as the motion vector of the motion region BBox, the tracking accuracy of the moving object is improved even with compressed video data at a low frame rate (for example, 8 fps). To do.

  The preferred embodiment of the present invention has been described above. Various functions can be added to the composite image V3. For example, the monitor can shift the gaze image V2 automatically arranged by the composite image generation unit 5 to an arbitrary position by dragging the mouse. Also, it can have a function of changing to an arbitrary size.

DESCRIPTION OF SYMBOLS 1 Mobile body display system 2 Omnidirectional camera 3 Mobile body extraction part 4 Gaze image generation part 5 Composite image generation part 6 Monitor 11 Motion detection part 21 Motion tracking part S Aggregation V1 Whole-round image V2 Gaze image V3 Composite image

Claims (7)

In a mobile display system comprising an omnidirectional camera that captures a hemispherical field of view of the camera as an all-round image, and displays a substantial imaging region inside the circle of the all-round image at the center of the monitor,
A moving body extraction unit for detecting and tracking the moving body from the captured image of the omnidirectional camera;
A gaze image generation unit that generates a gaze image by enlarging the periphery of the mobile body tracked by the mobile body extraction unit;
A composite image generation unit that generates a composite image having a configuration in which the gaze image generated by the gaze image generation unit overlaps a part of the entire circumference image, and displays the composite image on a monitor;
With
The composite image generation unit
A moving object display system characterized in that a gaze image of a new moving object is arranged in preference to a corner portion of an all-round image. The composite image generation unit
2. The moving object display system according to claim 1 , wherein the gaze image of the new moving object is arranged at a corner part farthest from the center of gravity coordinates of the moving object among corners of the vacant all-round image. . The all-round image is a horizontally long image,
The composite image generation unit
When a gaze image is already arranged at all corners, a gaze image of a new moving object is arranged at the center of the edge far from the barycentric coordinates of the moving object within the center of the left and right edges. The moving body display system according to claim 2 . The composite image generation unit
When a gaze image is already arranged at the center of the left and right edges, the gaze image of the new moving body is arranged at the center of the edge far from the center of gravity coordinates of the moving body within the upper and lower edges of the center. The moving body display system according to claim 3 . The moving body extraction unit includes:
A macro-block and forward prediction encoding intra-coded included in the P-picture frame motion vectors to create a binarization of the moving object map to the same pixel value and the threshold value above T ₁ macroblock ,
Creating a motion region surrounding the collection of pixel values;
When the number of macroblocks intra-coded in the motion area is the threshold T ₂ or more, claims 1 and detects the aggregates as mobile in any one of claims 4 The moving body display system described. The moving body extraction unit includes:
A pseudo inertia vector connecting the centroids of motion areas having the same ID between the immediately preceding P picture frame and the current P picture frame is obtained, and the position of the motion area of the current P picture frame is predicted from the pseudo inertia vector. The moving body display system according to claim 5 . On the computer,
A moving object extraction step for detecting and tracking a moving object from a captured image of an omnidirectional camera that captures a space in the hemispherical field of view of the camera as an all-round image;
A gaze image generation step of generating a gaze image by enlarging the periphery of the mobile body tracked by the mobile body extraction step;
The gaze gazing image generated by the image generating step generates a composite image of the structure that overlaps a portion of the entire periphery image and the synthetic image generating step of displaying the composite image monitor,
Was executed,
The composite image generation step includes
A moving object display program characterized in that a gaze image of a new moving object is arranged in preference to a corner portion of an all-round image .

Images