JP5349632B2 - Image processing method and image processing apparatus - Google Patents

Description

  The present invention relates to an image processing method and an image processing apparatus capable of efficiently retrieving image data of each moving body included in surveillance camera video data captured by a surveillance camera installed in an urban area or a structure.

  2. Description of the Related Art Conventionally, a technique for searching for a specific person from image data forming video data such as a TV program or a monitoring video is known. For example, Patent Document 1 discloses a person image search apparatus that can efficiently search for the same person who appears in different scenes by using different feature amounts for search processing and filtering processing.

  Specifically, in this patent document 1, a similar search technique for searching data similar to a search key designated by a user from video content and presenting the result, and similar face images and scenes in the video content are collectively displayed. A grouping technique is used in which an image group is created and only representative images of the image group are presented to the user so that the user can easily select a desired face image or scene.

  There is also known a technology that enables browsing / searching of surveillance video data recorded for a long time within a few tenths of the recording time. For example, Patent Document 2 discloses an image storage device configured to preset a change state or target on image data called a monitoring event and store a frame (image data) in which the change state has occurred. .

JP 2009-110460 A Japanese Patent No. 4678043

  However, the above-mentioned Patent Documents 1 and 2 are effective when extracting image data relating to a specific person, but roads imaged by monitoring cameras installed in various places of urban areas and structures. This is not always effective when analyzing surrounding surveillance camera video data. In particular, when analyzing surveillance camera video data captured by a large number of surveillance cameras, the conventional techniques represented by Patent Documents 1 and 2 require much time for video analysis.

  More specifically, the surveillance camera video data captured by the surveillance camera installed in an urban area or a structure includes various moving bodies such as a pedestrian walking on a road, a two-wheeled vehicle, and an automobile. For this reason, when analyzing such surveillance camera video data, it is necessary to analyze not only a specific person but also the behavior of an unspecified number of moving objects.

  In such a case, as described in Patent Documents 1 and 2, if the processing focused only on a specific person or a specific vehicle is repeated, the same number of moving bodies included in the monitoring camera video data is used. You will have to repeat the process. In particular, when the surveillance camera stores surveillance camera video data on a sequential access method recording medium such as a tape, it repeatedly reproduces the surveillance camera video data stored on the recording medium and extracts image data relating to each moving object. Processing time is delayed.

  In addition, when the above-mentioned Patent Documents 1 and 2 are used, even if video data consisting of a set of image data of each mobile object is extracted, it is only possible to reproduce the video data of each mobile object. Yes, it takes time to analyze the mutual relationship of mobile objects. For example, when analyzing the situation where the vehicle A collides with the vehicle B and the person X is hit by the vehicle B as a result, the video data of each moving object including the person X, the vehicle A, and the vehicle B is independently obtained. If you don't play it, you won't be able to grasp the situation and it's not efficient.

  The present invention has been made to solve the above-described problems of the prior art, and efficiently converts image data of each moving body included in surveillance camera video data captured by a surveillance camera installed in an urban area or a structure. An object of the present invention is to provide an image processing method and an image processing apparatus that can be searched automatically.

In order to solve the above-described problems and achieve the object, the present invention provides a movement classified into different categories from a plurality of time-series monitoring image data forming monitoring video data imaged by a predetermined monitoring imaging device. an image processing method in an image processing apparatus for extracting the moving object image data cut out a region including a body, an extraction processing step of extraction processes moving object image data from said plurality of monitor image data, the extraction process An identification processing step for identifying the type of the moving body based on the moving body image data extracted by the step, and a plurality of moving body image data extracted by the extraction processing step by the identification processing step. And a grouping process step of grouping each type of the moving body and storing it in a predetermined storage unit.
Further, according to the present invention, in the above invention, a reproduction display step for reproducing and displaying the monitoring video data acquired from the predetermined monitoring imaging device or the storage and reproduction device, and capturing of the monitoring video data displayed in the reproduction display step are performed. And a monitoring image storing step of storing in the storage unit as the monitoring image data.
Further, the present invention is the above invention, wherein in the extraction processing step, the monitoring video data acquired from the predetermined monitoring imaging device or the storage / reproduction device is reproduced and displayed, and the reproduced and displayed monitoring video data is captured. The mobile image data is extracted as the monitoring image data.

  Further, in the present invention, in the above invention, the extraction processing step extracts the moving body image data from each monitoring image data when the moving body having a predetermined size or more is included in the plurality of monitoring image data. It is characterized by that.

Further, in the present invention according to the above invention, in the grouping processing step, in addition to the type of the moving body included in the moving body image data being the same, the speed of the moving body is within a separately set range. Each of the movements as a condition belonging to the same group that satisfies any one or a plurality of combinations among the conditions that the moving body has the same color and the moving body has the same size It is characterized by grouping body image data.
In the present invention, the group includes a group of a person, a motorcycle, and an automobile.
In the present invention, the thumbnail image of one moving body image data among a plurality of moving body image data belonging to each group extracted from each monitoring image data by the extraction processing step is displayed. It includes a process.
Further, the present invention is characterized in that, in the above invention, the method further comprises a step of displaying mobile image data corresponding to the selected thumbnail image in response to selection of the thumbnail image from the user.
Further, the present invention is characterized in that, in the above invention, the method further comprises a step of reproducing and displaying the moving body image data corresponding to the selected thumbnail image as a moving image upon selection of the thumbnail image from the user. .

  In the above invention, the present invention further includes a monitoring video reproduction processing step of reproducing the monitoring video data captured by the monitoring imaging device, wherein the extraction processing by the extraction processing step and the identification processing by the identification processing step Is performed in parallel with the reproduction processing of the monitoring image data forming the frame of the monitoring video data by the monitoring video reproduction processing step, and the grouping processing by the grouping processing step is performed after the reproduction processing by the monitoring video reproduction processing step is completed. It is characterized by performing.

In the above invention, the present invention further includes a monitoring video reproduction processing step of reproducing the monitoring video data captured by the monitoring imaging device, wherein the extraction processing by the extraction processing step and the identification processing by the identification processing step Is performed in parallel with the reproduction processing of the monitoring image data forming a frame of the monitoring video data by the monitoring video reproduction processing step, and the grouping processing by the grouping processing step ends the reproduction processing by the monitoring video reproduction processing step. It is characterized by being performed without waiting .

  Also, in the present invention, in the above invention, the grouping process step stores a plurality of moving body image data relating to moving bodies belonging to the same group in the same folder of the storage unit, and a predetermined moving body is designated. And a display control step of controlling display of a plurality of moving body image data in a folder corresponding to the moving body as video data on a predetermined display unit.

  The present invention further includes a crossing relationship determination step of determining a crossing relationship between a plurality of moving bodies based on the moving body image data extracted by the extraction processing step in the above invention, wherein the display control step includes: When the moving body is designated, a plurality of moving body image data in a folder corresponding to the moving body and the moving body determined to be in a crossing relationship with the moving body in the crossing relationship determination step are used as video data. Display control is performed on a predetermined display unit.

The present invention also provides moving body image data obtained by cutting out a region including moving bodies classified into different categories from a plurality of time-series monitoring image data forming the monitoring video data captured by a predetermined monitoring imaging device. an image processing apparatus for extracting an extraction processing means for extraction processes moving object image data from the plurality of monitoring image data, on the basis of the moving object image data extracted by the extraction processing unit, the mobile object And a predetermined storage unit that groups the plurality of moving body image data extracted by the extraction processing unit for each type of moving body that has been subjected to the identification processing by the identification processing unit. And a grouping processing means for storing the data.

According to the present invention, and Extraction processes moving object image data from a plurality of monitor image data based on the extracted moving object image data, to identify processing the type of the moving body, extracting the treated plurality of mobile Since the image data is grouped according to the type of the moving body and stored in a predetermined storage unit, the moving body image data of a plurality of moving bodies included in the monitoring image data is grouped, so that the monitoring camera video It becomes possible to search efficiently the image data of each moving body included in the data.

FIG. 1 is an explanatory diagram for explaining the concept of the image processing apparatus according to the first embodiment. FIG. 2 is a block diagram illustrating an internal configuration of the image processing apparatus according to the first embodiment. FIG. 3 is an explanatory diagram for explaining a display screen for accepting designation of a group. FIG. 4 is an explanatory diagram for explaining the display after receiving the designation of the group. FIG. 5 is a flowchart illustrating a processing procedure of the image processing apparatus. FIG. 6 is an explanatory diagram for explaining the operation of the image processing apparatus. FIG. 7 is an explanatory diagram for explaining the determination of the possibility of contact. FIG. 8 is a block diagram illustrating an internal configuration of the image processing apparatus according to the second embodiment. FIG. 9 is a flowchart illustrating a processing procedure performed by the intersection determination unit. FIG. 10 is a flowchart showing a processing procedure by the mobile video reproduction processing unit.

  Exemplary embodiments of an image processing method and an image processing apparatus according to the present invention will be described below in detail with reference to the accompanying drawings. In the embodiment described below, it is assumed that a surveillance camera is fixed to a building and a mobile body (a person, a two-wheeled vehicle, and an automobile) located on the road is searched by the surveillance camera.

  First, the concept of the image processing apparatus according to the first embodiment will be described. FIG. 1 is an explanatory diagram for explaining the concept of the image processing apparatus according to the first embodiment. As shown in the figure, when the image processing apparatus receives sequential monitoring video data from the monitoring camera, the image processing apparatus reproduces the monitoring video data on the display unit (monitoring video data playback processing). Here, a situation in which the car A and the person B are shown in each of the frames 0 to 2 is shown. Instead of the monitoring video data received online from the monitoring camera, the separately recorded monitoring video data may be reproduced on the display unit.

  In parallel with the monitoring video data reproduction process, the image processing apparatus performs a moving body extraction process for extracting moving body image data including a moving body from the monitoring image data (frame) forming the monitoring video data. Here, not only the moving body image data of a specific moving body specified in advance is extracted, but in principle, moving body image data of all moving bodies included in each frame is extracted.

  Specifically, the initial background image learned in advance and the monitoring image data are compared, and the difference is extracted as moving body image data. With this process, the moving body image data including the car A and the moving body image data including the person B are cut out from each of the frames 0 to 2.

  When the extraction of the mobile object image data is completed, the image processing apparatus continues the identification process for each mobile object included in the mobile object image data. Specifically, it is identified by pattern matching or the like whether each moving body included in the moving body image data belongs to a person, a two-wheeled vehicle, or an automobile. With this processing, the type of the moving body image data including the car A becomes “car”, and the type of the moving body image data including the person B becomes “person”.

  The image processing apparatus performs a grouping process for grouping the moving object image data for each moving object after the reproduction process and the identification process of the monitoring video data. By this grouping process, when the same moving object is shown in a plurality of frames, moving object image data including the moving object can be collected as one group.

  Specifically, it is a condition belonging to the same group that the types are the same and the speed of the moving body is within a predetermined range set separately. The speed of the moving object can be determined by a change in the position of the moving object image data between frames and a change in the size of the moving object image data between frames.

  It is preferable to increase the predetermined range when the type is “car” and decrease the predetermined range when the type is “person”. Further, the distance from the monitoring camera to the moving body may be used for setting the predetermined range. When the monitoring camera is fixed, the position of the moving object image data in the frame corresponds to the distance from the monitoring camera to the moving object. When this distance is large, the predetermined range is set small, and when the distance is small, the predetermined range is set large.

  Moreover, you may use as a condition for belonging to the same group that the color of a moving body is the same. Specifically, a histogram is taken for the color of the moving object, and it is determined whether the color information matches with the color having the highest value as the color information of the moving object.

  By such grouping processing, the moving body image data of the car A and the moving body image data of the person B are grouped. If there are different moving bodies of the same type, such as when there are a plurality of cars in the frame, grouping is performed for each moving body.

  The image processing apparatus can perform a moving body video reproduction process using the moving body image data belonging to the group by receiving the designation of the group after the grouping process. By this moving body video reproduction processing, it is possible to individually reproduce each grouped moving body. In addition, it is possible to reproduce the image data of the entire frame including the moving body image data. Therefore, if the group of the car A is designated, the video of the car A is reproduced on the display unit, but the video of the person B is not reproduced at this time.

  In this way, moving body image data is extracted from the monitoring video data captured by the monitoring camera, the type of the moving body is identified, and the moving body image data is grouped for each moving body. Can be searched efficiently.

  Next, the internal configuration of the image processing apparatus according to the first embodiment will be described. FIG. 2 is a block diagram illustrating an internal configuration of the image processing apparatus according to the first embodiment. As shown in FIG. 2, the image processing apparatus 30 is connected to a plurality of monitoring cameras 10 a to 10 b via a network 20. A video recorder 11 for recording video data is connected to the monitoring camera 10b. Therefore, the monitoring video data captured by the monitoring cameras 10a to 10b can be output to the image processing device 30, and the monitoring video data recorded in the video recorder 11 according to the setting of the video recorder 11 is subjected to image processing. It can also be output to the device 30. In this embodiment, for convenience of explanation, a case where the monitoring camera 10a is directly connected to the image processing device 30 or a case where the monitoring camera 10a is connected to the image processing device 30 via the video recorder 11 will be described. Video data stored in a PDA or the like can also be input to the image processing apparatus 30.

  The image processing apparatus 30 includes an I / F unit 31, a display unit 32, an input unit 33, a storage unit 34, and a control unit 35. The I / F unit 31 is an interface unit for receiving the monitoring video data captured by the monitoring cameras 10 a to 10 b or the monitoring video data recorded in the video recorder 11 via the network 20. The display unit 32 is a liquid crystal panel, a display device, or the like, and the input unit 33 is a keyboard, a mouse, or the like.

  The storage unit 34 is a storage device including a flash memory or a hard disk device, and stores monitoring video data 34a and moving body image data 34b. The monitoring video data 34a is video data captured by the monitoring cameras 10a to 10b, and is individually stored corresponding to the captured monitoring camera. Further, the monitoring video data 34a may be stored separately by being separated by a predetermined time unit (sequence) such as several hours.

  The moving body image data 34b is image data of a moving body cut out from the monitoring video data 34a. The moving object image data 34b is stored in groups for each moving object based on the type of moving object as described above.

  The control unit 35 is a control unit that controls the entire image processing apparatus 30, and includes a monitoring video reproduction processing unit 35a, a moving body extraction processing unit 35b, an identification processing unit 35c, a grouping processing unit 35d, and a moving body video reproduction processing unit 35e. Have Actually, programs corresponding to these functional units are stored in a ROM or a non-volatile memory (not shown), and these programs are executed by a CPU (Central Processing Unit) so that the monitoring video reproduction processing unit 35a can be moved. The body extraction processing unit 35b, the identification processing unit 35c, the grouping processing unit 35d, and the moving body video reproduction processing unit 35e are caused to execute corresponding processes.

  When the surveillance video reproduction processing unit 35a receives sequential surveillance video data 34a from the surveillance cameras 10a to 10b or the video recorder 11, the surveillance video data 34a is stored in the storage unit 34 for each surveillance camera and is also displayed on the display unit. 32 is a processing unit that performs playback processing. Specifically, each time the monitoring video data 34a is sequentially received as a video stream, the video stream is displayed on the display unit 32, and on the other hand, an image is captured and stored as the monitoring video data 34a in the storage unit 34. In addition to storing and displaying the received monitoring video data 34a as a series of processes, when the monitoring video data 34a is received, the monitoring video data 34a is temporarily stored in the storage unit 34 and stored at another timing. The monitoring video data 34 a stored in the unit 34 can be read out and reproduced on the display unit 32.

  The moving body extraction processing unit 35b is a processing unit that receives monitoring video data (frame) from the monitoring video reproduction processing unit 35a and extracts and extracts moving body image data of one moving body.

  The object to be cut out is a moving body having a predetermined size or more. The size of the moving body is obtained from the number of pixels. Specifically, the frame includes a moving body having a size larger than the frame rate of the monitoring cameras 10a to 10b, the distance from the monitoring cameras 10a to 10b to the moving body, and the scene imaged by the monitoring cameras 10a to 10b. In this case, the moving object extraction processing unit 35b performs extraction processing of the moving object image data.

  The identification processing unit 35c is a processing unit that performs identification processing of a moving object from moving object image data. Specifically, it is identified by pattern matching or the like whether each moving body included in the moving body image data belongs to a person, a two-wheeled vehicle, or an automobile. Also, frame identification information that identifies a frame from which mobile body image data is extracted, position information of mobile body image data in the source frame, the number of pixels of mobile body image data, color information, and a monitoring video reproduction processing unit for the corresponding frame The elapsed time from the reproduction start point 35a is associated with the moving object image data. Then, the above extraction and identification processes are repeated for moving bodies of all frames.

  The grouping processing unit 35d is a processing unit that divides the moving body image data into groups for each moving body and stores the moving body image data as the moving body image data 34b to which the group ID is assigned in the storage unit 34. More specifically, moving body image data having the same type, moving body speed within a predetermined range, and matching color information are set to the same group. Further, the matching may be determined by calculating a predetermined feature amount. Arbitrary algorithms can be used for matching. This grouping process is completed in several seconds to several minutes. This grouping process can also be started in the middle of the extraction and identification process for the moving body of each frame described above.

  When the mobile video reproduction processing unit 35e receives a group designation from a user (operator), the mobile video reproduction processing unit 35e reads out the mobile body data 34b belonging to the designated group from the storage unit 34 and reproduces the data on the display unit 32. Part. This reproduction process can be executed at any time if there is a grouped process.

  When a group designation is accepted from the user, the group that can be designated is displayed in a list on the display unit 32 together with thumbnail images. FIG. 3 is an explanatory diagram for explaining a display screen for accepting designation of a group.

  FIG. 3A shows information on groups that can be specified in a table format. Specifically, this is a table in which “No.” indicating a number that uniquely identifies a group is associated with a frame number, a detected thumbnail, an elapsed time, a type, and the number of frames. The frame number indicates from which frame of the monitoring video data 34a the moving body image data 34b belonging to the group has appeared. The detected thumbnail is a thumbnail image of the first moving body image data 34b among the moving body image data 34b belonging to the group.

  The elapsed time indicates the elapsed time from the playback start point of the monitoring video playback processing unit 35a until the first moving body image data 34b appears. The type is the type of the moving object identified by the identification processing unit 35c, and the number of frames is the number of moving object image data 34b belonging to the group.

  FIG. It shows that one group is an image of a car that appears 15 frames from the 156th frame (32 minutes 30 seconds) of the monitoring video data 34a. No. The group 2 indicates that images of people appearing 53 frames from the 263rd frame (42 minutes 44 seconds) of the monitoring video data 34a. No. The group 3 indicates that the images of the car appearing 9 frames from the 1044th frame (1 hour 21 minutes 51 seconds) of the monitoring video data 34a. No. The group 4 indicates that the image is of the type “two-wheeled vehicle” that appears three frames from the 1552th frame (2 hours 7 minutes 29 seconds) of the monitoring video data 34a.

  FIG. 3B is a modification of the group display method that can be specified. In FIG. 3B, the detection thumbnails of each group are displayed side by side, and the number, frame number, elapsed time, type, and number of frames that uniquely identify the group are displayed below the detection thumbnail.

  When the designation of the group is accepted from any of the display screens shown in FIGS. 3A and 3B, the moving body video reproduction processing unit 35e displays the moving body image data belonging to the designated group on the display unit 32. Play it up. Moreover, you may make it display the mobile body image data which belongs to the designated group as a list. FIG. 4 is an explanatory diagram for explaining the display after receiving the designation of the group.

  FIG. 4A shows a display screen in the case of performing playback display. In the reproduction display shown in FIG. 4A, the monitoring image frame including the moving body image data 34b belonging to the designated group in the monitoring video data 34b is reproduced as a moving image. Accordingly, it is possible to confirm all the moving objects corresponding to the designated group and other moving objects imaged at the same time as the moving object. At this time, the moving body corresponding to the designated group may be surrounded by a rectangle or the like so that it can be distinguished from other moving bodies. For playback control, the "Reverse frame advance" button, the "Play / Pause" button, and the "Frame advance" button are displayed, and the "Print" button is displayed to accept the print operation of the display screen. Yes.

  FIG. 4B shows a display screen displaying a list of moving body image data belonging to the specified group. In this case, thumbnail images of all moving body image data belonging to the designated group are displayed, and the frame number and elapsed time are displayed below each thumbnail image. Also, if all thumbnail images could not be displayed in one screen, all thumbnail images can be confirmed by changing the screen by operating the “Previous screen” and “Next screen” buttons. To do. The display screen can be printed by operating the “print” button.

  If the user selects any thumbnail image with a mouse or the like, the moving object image data 34b corresponding to the selected thumbnail image is displayed. At this time, it is desirable that the display size of the mobile object image data 34b can be arbitrarily enlarged and reduced.

  In addition to simply displaying a list of mobile image data belonging to a specified group, the mobile image data may be reproduced and displayed as a moving image. By reproducing and displaying the moving body image data as a moving image, the movement of the moving body corresponding to the designated group can be selectively confirmed.

  Next, a processing procedure by the image processing apparatus 30 will be described. FIG. 5 is a flowchart showing a processing procedure of the image processing apparatus 30. As shown in FIG. 5, the image processing apparatus 30 first sets an initial background (step S101). As this initial background, an image acquired when the monitoring cameras 10a to 10b are installed may be set as the initial background, or the first frame of each sequence may be set as the initial background. .

  After setting the initial background (step S101), the image processing apparatus 30 accepts a user input and sets identification parameters (step S102). The identification parameters include the minimum size of the moving body to be cut out as moving body image data, pattern matching data for identifying the type, and the like.

  After setting the identification parameter (step S102), the image processing apparatus 30 receives input of monitoring video data captured by the monitoring cameras 10a to 10b (step S103). For the input of the monitoring video data, the data received from the monitoring cameras 10a to 10b may be used as input in real time, or the monitoring video data 34a designated by the user among the monitoring video data 34a stored in the storage unit 34 is used. It may be used as input.

  The image processing apparatus 30 performs a process of extracting the moving body image data from the input monitoring video data (step S104), and executes an identification process on the extracted moving body image data (step S105). By this identification processing, the type of the moving object is identified. The identified type, frame identification information for identifying the source frame, position information of the mobile image data in the source frame, an image thumbnail obtained by reducing the mobile image data, the number of pixels of the mobile image data, color information, The elapsed time from the start point of the monitoring video data is associated with the moving body image data and stored in the storage unit 34 together with the moving body image data.

  The image processing apparatus 30 determines whether or not one sequence is completed after the identification process (step S105) (step S106). If the sequence has not ended (step S106; No), the image processing apparatus 30 proceeds to the moving body extraction process (step S104).

  If the sequence is finished (step S106; Yes), the image processing apparatus 30 executes a grouping process for the moving object image data (step S107). By this grouping process, the moving object image data is grouped for each moving object and stored in a folder generated for each group.

  If the result of grouping performed by the image processing apparatus 30 is not appropriate (step S108; No), the grouping condition is changed and the grouping process is executed (step S107).

  If the result of grouping becomes appropriate (step S108; Yes), the image processing apparatus 30 ends the process. The moving object image data appropriately grouped in this way is useful when a user searches for an image of a desired moving object.

  Next, the operation of the image processing apparatus 30 will be further described. FIG. 6 is an explanatory diagram for explaining the operation of the image processing apparatus 30. As shown in FIG. 6, when the user performs an operation for inputting monitoring video data (step S201) and an operation for setting an identification parameter (step S202), the image processing apparatus 30 moves the moving body image data from the monitoring video data. Starts to be extracted and identified (step S301).

  The moving body image data extracted and identified from the start of the extraction and identification of the moving body image data (step S301) to the end of the extraction and identification of the moving body image data (step S302) is temporarily stored in the storage unit 34. Stored in

  The user performs an operation of browsing the mobile object image data extracted and identified from the input unit 33 (step S203), and the image processing apparatus 30 displays the mobile object image data (step S303). If it is determined that there is no need for grouping (step S204; No), the processing ends.

  On the other hand, if it is determined that grouping is necessary (step S204; Yes), the user performs a grouping condition setting operation (step S205). The grouping condition includes a speed range setting for classifying into the same group, a size of a moving object to be grouped, a feature amount calculation method and an algorithm used for determining coincidence. In addition, the moving body and surrounding conditions may be classified as scenes, and settings may be performed for each scene. The “scene” here means, for example, a sidewalk and a roadway.

  The image processing apparatus 30 groups the moving body image data based on the grouping condition set by the user (step S304), and stores the moving body image data in a folder for each group.

  The user browses the image data as a result of grouping (step S206), and if the group is not properly grouped (step S206; No), changes the grouping condition (step S205). And when it groups appropriately (step S206; Yes), a process is complete | finished.

  As described above, in the first embodiment, each of the plurality of moving body image data 34b included in the monitoring video data 34a captured by the monitoring cameras 10a to 10b is extracted, and the extracted moving body image data 34b is extracted. On the basis of this, the type of the moving body is identified, and the plurality of moving body image data 34b is grouped for each moving body type and stored in the storage unit 34. Therefore, the moving body image data is efficiently stored. It is possible to search.

  In the first embodiment, the moving body image data included in the monitoring video data is grouped for each moving body, and the individual display can be performed. However, in the second embodiment, the moving bodies that may be related are described. A configuration for determining and controlling display will be described.

  When a plurality of moving bodies such as a car and a car, a car and a person are shown in the same frame of the monitoring video data, there is a possibility that the moving bodies have caused a contact accident or the like. Therefore, in the image processing apparatus according to the second embodiment, the possibility of contact is determined for a plurality of moving objects shown in the same frame of the monitoring video data, and the possibility of contact is displayed when moving object image data is displayed. Display control is performed on the moving object image data of other moving objects having the same.

  FIG. 7 is an explanatory diagram for explaining the determination of the possibility of contact. As shown in FIG. 7A, even if the moving body image data of the car A and the moving body image data of the person B exist simultaneously in the same frame of the monitoring video data, the moving body image of the car A If the data and the moving body image data of the person B do not cross, that is, do not overlap, there is no possibility that the car A and the person B are in contact with each other.

  On the other hand, as shown in FIG. 7B, when the moving body image data of the car A and the moving body image data of the person B intersect, that is, when there are overlapping frames, the car A and the person B contact each other. There is a possibility.

  Therefore, the image processing apparatus 130 according to the second embodiment determines a crossing relationship between a plurality of moving body image data, and stores the presence / absence of crossing between moving bodies as crossing relationship data. FIG. 8 is a block diagram illustrating an internal configuration of the image processing apparatus 130 according to the second embodiment.

  As shown in FIG. 8, the image processing device 130 is different from the image processing device 30 shown in FIG. 2 in that the storage unit 34 further stores the cross relationship data 134 and the control unit 35 further includes a cross determination unit 131. . Further, the processing operation of the mobile video playback processing unit 132 is also different from that of the mobile video playback processing unit 35e shown in FIG. Since other configurations and operations are the same as those of the image processing apparatus 30 shown in FIG. 2, the same components are denoted by the same reference numerals and description thereof is omitted.

  The intersection determination unit 131 determines the intersection relationship using the grouped moving body image data. Specifically, it is determined whether or not each moving body image data in the group overlaps with moving body image data belonging to another group on the same frame. If there is a crossing group as a result of the determination, the frame number information crossed with the crossing group ID is stored in the storage unit 34 as crossing relationship data 134.

  When the mobile video playback processing unit 35e receives designation of a group from the user, the mobile video playback processing unit 132 refers to the cross relationship data 134 and there is a group in a cross relationship with the specified group. Then, a group that is in a cross relationship with the designated group is set as the target of the reproduction process.

  Next, a processing procedure performed by the intersection determination unit 131 will be described. FIG. 9 is a flowchart illustrating a processing procedure performed by the crossing determination unit 131. As shown in FIG. 9, the crossing determination unit 131 selects one group (step S401), and selects moving body image data in the selected group (step S402).

  The intersection determination unit 131 determines whether or not the selected moving body image data overlaps with moving body image data belonging to another group on the same frame (step S403). If it overlaps with the moving body image data belonging to another group on the same frame (step S403; Yes), the crossing determination unit 131 sets a crossing relationship between the selected group and the crossing group (step S404). ).

  After the setting of the crossing relationship (step S404) or when there is no overlap with the mobile body image data belonging to another group (step S403; No), the crossing determination unit 131 crosses all the mobile body image data of the selected group. It is determined whether the determination has been made (step S405).

  When the moving body image data that has not been subjected to the crossing determination remains (step S405; No), the crossing determination unit 131 selects the moving body image data that has not been subjected to the crossing determination (step S402).

  If all the moving body image data of the selected group has been selected (step S405; Yes), the crossing determination unit 131 determines whether or not all the groups have been selected (step S406).

  When an unselected group remains (step S406; No), the crossing determination unit 131 selects an unselected group (step S401). If all groups have been selected (step S406; Yes), the process is terminated.

  Next, a processing procedure performed by the mobile video reproduction processing unit 132 will be described. FIG. 10 is a flowchart showing a processing procedure performed by the mobile video reproduction processing unit 132. As illustrated in FIG. 10, when the mobile video reproduction processing unit 132 accepts designation of a group to be reproduced (step S501; Yes), there is a group that intersects with the group designated based on the cross relationship data 134. It is determined whether or not to perform (step S502).

  If there is a group that intersects with the designated group (step S502; Yes), the mobile image reproduction processing unit 132 reproduces both the group that intersects with the designated group (step S503), and performs processing. finish. On the other hand, if there is no group that intersects with the designated group (step S502; No), the mobile video reproduction processing unit 132 reproduces only the designated group (step S504) and ends the process.

  As described above, the image processing apparatus 130 according to the second embodiment determines a crossing relationship between a plurality of moving objects based on the moving object image data 134, and when the moving object is designated, Since the display control is performed on the moving body image data 34b of the group corresponding to the moving body that is in a crossing relationship with the moving body, a plurality of moving bodies that may be related can be collectively displayed.

  In the first and second embodiments, the case where the grouping process is performed after the end of one sequence is illustrated, but the grouping process can be executed even in the middle of the sequence. Therefore, the timing for executing the grouping process can be arbitrarily set, for example, the grouping process is executed every time the monitoring video data is reproduced for a predetermined time.

  Each configuration illustrated in the first and second embodiments is functionally schematic and does not necessarily need to be physically configured as illustrated. That is, the form of distribution / integration of each device is not limited to the one shown in the figure, and all or a part thereof may be functionally / physically distributed / integrated in arbitrary units according to various loads and usage conditions. Can be configured.

  For example, the monitoring video data captured by the monitoring cameras 10a to 10b may be stored in another server and then transmitted to the image processing apparatus by batch processing at predetermined time intervals.

  Further, the storage unit 34 may be configured to store an extraction feature amount used for extraction of a mobile body and a search feature amount for searching for a desired mobile body from grouped mobile body image data. Further, the extracted feature quantity and the search feature quantity may be stored in a storage unit different from the storage unit 34 or may be stored in an external device.

  As described above, the image processing method and the image processing apparatus according to the present invention efficiently search each moving body image data included in the monitoring camera video data captured by the monitoring camera installed in the city area or the structure. Useful in cases.

10a to 10b Monitoring camera 20 Network 30, 130 Image processing device 31 I / F unit 32 Display unit 33 Input unit 34 Storage unit 34a Monitoring image data 34b Moving object image data 35 Control unit 35a Monitoring image reproduction processing unit 35b Moving object extraction process Unit 35c identification processing unit 35d grouping processing unit 35e, 132 mobile image reproduction processing unit 131 crossing determination unit 134 crossing relationship data

Claims (14)

Image processing apparatus for extracting moving body image data obtained by cutting out areas including moving bodies classified into different categories from a plurality of time-series monitoring image data forming monitoring video data imaged by a predetermined monitoring imaging apparatus An image processing method in
An extraction processing step of Extraction processes moving object image data from said plurality of monitored image data,
An identification processing step for identifying the type of the moving body based on the moving body image data extracted by the extraction processing step;
A grouping processing step of grouping a plurality of moving body image data extracted by the extraction processing step for each type of moving body identified by the identification processing step and storing the grouped data in a predetermined storage unit. An image processing method.   A reproduction display step for reproducing and displaying the monitoring video data acquired from the predetermined monitoring imaging device or the storage / reproduction device, capturing the monitoring video data displayed in the reproduction display step, and storing the monitoring video data in the storage unit as the monitoring image data The image processing method according to claim 1, further comprising a monitoring image storing step of storing.   In the extraction processing step, the monitoring video data acquired from the predetermined monitoring imaging device or storage / reproduction device is reproduced and displayed, the reproduced and displayed monitoring video data is captured, and the moving body image data is used as the monitoring image data. The image processing method according to claim 1, wherein the extraction processing is performed. The extraction process is claimed in claim characterized in that the predetermined size or more mobile bodies when included in the plurality of monitoring image data, and extracts handle moving object image data from the monitor image data 1 and 2 Or the image processing method of 3 . In the grouping process, in addition to the same type of moving object included in the moving object image data, the speed of the moving object is within a separately set range, and the color of the moving object is the same. It is characterized in that each moving body image data is grouped on the condition that satisfying any one or a plurality of combinations among the conditions that the size of the moving body is the same belongs to the same group. The image processing method according to any one of claims 1 to 4 .   The image processing method according to claim 1, wherein the group includes a group of a person, a motorcycle, and an automobile.   The method includes a step of displaying thumbnail images of one moving body image data among a plurality of moving body image data belonging to each group extracted from each monitoring image data by the extraction processing step, respectively. Item 8. The image processing method according to Item 1.   8. The image processing method according to claim 7, further comprising a step of displaying moving body image data corresponding to the selected thumbnail image in response to selection of the thumbnail image from the user.   9. The image processing according to claim 7, further comprising a step of reproducing and displaying moving body image data corresponding to the selected thumbnail image as a moving image in response to selection of the thumbnail image from the user. Method. A monitoring video reproduction processing step of reproducing the monitoring video data imaged by the monitoring imaging device;
The extraction processing by the extraction processing step and the identification processing by the identification processing step are performed in parallel with the reproduction processing of the monitoring image data forming the frame of the monitoring video data by the monitoring video reproduction processing step, and the group by the grouping processing step The image processing method according to claim 1, wherein the conversion processing is performed after the reproduction processing by the monitoring video reproduction processing step is completed. A monitoring video reproduction processing step of reproducing the monitoring video data imaged by the monitoring imaging device;
The extraction processing by the extraction processing step and the identification processing by the identification processing step are performed in parallel with the reproduction processing of the monitoring image data forming the frame of the monitoring video data by the monitoring video reproduction processing step, and the group by the grouping processing step Processing is performed without waiting for the end of the playback process in the monitoring video playback process step.
The image processing method according to claim 1 . The grouping process step stores a plurality of moving body image data related to moving bodies belonging to the same group in the same folder of the storage unit,
And a display control step of controlling display of a plurality of moving body image data in a folder corresponding to the moving body as video data on a predetermined display when a predetermined moving body is designated. The image processing method according to claim 1. Based on the moving body image data extracted by the extraction processing step, further includes a crossing relationship determination step of determining a crossing relationship between a plurality of moving bodies,
The display control step includes
When the moving body is designated, a plurality of moving body image data in a folder corresponding to the moving body and the moving body determined to be in a crossing relationship with the moving body in the crossing relationship determination step are used as video data. The image processing method according to claim 11 , wherein display control is performed on a predetermined display unit. Image processing apparatus for extracting moving body image data obtained by cutting out areas including moving bodies classified into different categories from a plurality of time-series monitoring image data forming monitoring video data imaged by a predetermined monitoring imaging apparatus Because
An extraction processing means for Extraction processes moving object image data from said plurality of monitored image data,
Identification processing means for identifying the type of the moving object based on the moving object image data extracted by the extraction processing means;
Grouping processing means for grouping a plurality of moving body image data extracted by the extraction processing means for each type of moving body identified by the identification processing means and storing the grouped data in a predetermined storage unit. An image processing apparatus.

Images