JP2007134948A - Device for imaging person in parked vehicle - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a device for imaging a person in a parked vehicle which can efficiently be used without using a large-capacity recording device and can easily detect the person in the parked vehicle from the recorded video. <P>SOLUTION: When a no-movement still area in an image can be extracted from video data of a monitor area from a wide-angle camera 110, a feature quantity is extracted from the still area and pattern matching between the feature quantity and a vehicle template is performed to calculate position information and time information of a vehicle area when the there is the vehicle area. Then while it is decided how many minutes there is no movement in the vehicle area, a pan and tilt zooming camera 111 is controlled based upon the position information to obtain video data from the pan and tilt zooming camera 111, a movement area in the image is extracted from the video data, and further a face area is extracted; and then the best shot matching a criterion most among face images is decided and recorded in a recording section 150. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a parked vehicle occupant imaging device that monitors and records parked vehicles and passengers with a surveillance camera.

  2. Description of the Related Art Conventionally, a monitoring recording device that monitors and records parked vehicles and passengers with a monitoring camera has been developed (see, for example, Patent Document 1). The monitoring and recording apparatus disclosed in Patent Document 1 searches for a parked vehicle with a wide-angle camera and monitors a passenger with a pan / tilt / zoom camera.

  Since the parked vehicle does not know which part of the road is parked, the position information of the parked vehicle is first detected by the wide-angle camera, and the pan / tilt zoom camera is pan / tilt zoomed to the vicinity of the parked vehicle from the detected position information. When the occupant returns to the vehicle, the occupant can be captured more clearly by capturing the occupant with the pan / tilt / zoom camera.

JP 2004-7374 A

  However, in the conventional monitoring and recording device, since it is not known when the passenger of the parked vehicle returns to the vehicle, it is necessary to continuously take an image, and the resource of the recording device such as an HDD (Hard Disk Drive) is depleted. There is a problem that a remarkably large-capacity recording apparatus is required and the cost is increased accordingly. In addition, there is a problem that work efficiency is poor because it is necessary to find a passenger of the parked vehicle from the video recorded for a long time.

  The present invention has been made in view of such circumstances, and enables efficient use without using a large-capacity recording device, and easily finds a occupant of a parked vehicle from the recorded video. It is an object of the present invention to provide a parked vehicle occupant imaging device.

The above object is achieved by the following configuration.
(1) a first imaging means arranged toward the monitoring area, a second imaging means for changing the imaging range in accordance with an instruction from the outside, and monitoring area video information from the first imaging means, A static region extracting means for extracting a static region having no motion, a background change detecting means for detecting a background change from the static region of the monitoring region video information extracted by the static region extracting means, and a background change detecting means. When a change in the background is detected, a feature amount extraction unit that extracts a feature amount of the region where the background is changing, and a pattern matching between the feature amount extracted by the feature amount extraction unit and the car template To determine whether the feature quantity is a vehicle. If the feature amount is determined to be a vehicle, the vehicle pattern matching means for calculating the position information and time information of the vehicle is calculated by the vehicle pattern matching means. A camera control means for giving a command to the second imaging means based on the position information and enabling the second imaging means to take an image of a vehicle occupant; and from the second imaging means A moving area extracting means for extracting a moving area with motion from video information, a recording means for recording an image, and a face area extracted from the moving area extracted by the moving area extracting means to the recording means A face area extracting means for recording; a parking time determining means for calculating a time without movement from the vehicle position information and time information calculated by the vehicle pattern matching means; and a time calculated by the parking time determining means. When it is less than a preset parking time, an image erasing unit that deletes all face images captured by the second imaging unit and recorded in the recording unit, and a time calculated by the parking time determination unit Preset If it is longer than the parking time, the best shot that best matches the criteria is determined from all the face images captured by the second imaging means and recorded in the recording means. Best shot discriminating means for recording in the recording means.

(2) a first imaging means arranged toward the monitoring area, a second imaging means for changing the imaging range in accordance with an instruction from the outside, and the monitoring area video information from the first imaging means from within the image A static area extracting means for extracting a static area without movement, a background change detecting means for obtaining a background from a plurality of static areas of the monitoring area video extracted by the static area extracting means, and a background by the background change detecting means When a change is detected, a feature amount extraction unit that extracts a feature amount of the area where the background changes, and a pattern matching between the feature amount extracted by the feature amount extraction unit and the car template is performed. Determining whether the feature amount is a vehicle, and if it is determined to be a vehicle, vehicle pattern matching means for calculating position information and time information of the vehicle; A first moving area extracting means for extracting a moving area; and a person who extracts a human area from the moving area extracted by the first moving area extracting means and calculates position information and time information of the extracted human area. An area extracting means; a person feature amount extracting means for calculating a feature amount of the person area extracted by the person area extracting means; and a person having the feature amount calculated by the person feature amount extracting means And if there is a person tracking means for calculating position information, and giving a command to the second imaging means based on the position information calculated by the person tracking means. The camera control means for controlling the second imaging means so as to be able to take an image of the vicinity of the occupant of the vehicle, and a moving region having a motion in the image is extracted from the video information from the second imaging means. Second moving region extracting means and for recording an image Recording means, a face area extracting means for extracting a face area from the moving area extracted by the second moving area extracting means and recording it in the recording means, and the vehicle calculated by the vehicle pattern matching means Parking time determining means for calculating a time without movement from the position information and time information of the vehicle, and when the time calculated by the parking time determining means is less than a preset parking time, the second imaging means When the time calculated by the parking time determination means is equal to or longer than a preset parking time, the second image pickup means deletes the entire face image that has been captured and recorded in the recording means. Best shot discriminating means for discriminating the best shot that best matches the judgment criteria among all the face images that have been picked up and recorded in the recording means, and that records only the image discriminated as the best shot in the recording means , Comprising a.

(3) In the parked vehicle occupant imaging device described in (1) or (2) above, the parking area captured by the first imaging unit at the time when the best shot image is selected by the best shot discrimination unit Vehicle image control means for recording the image in the recording means.

(4) a first imaging means arranged toward the monitoring area, a second imaging means for changing the imaging range in accordance with an instruction from the outside, and monitoring area video information from the first imaging means, A static area extracting means for extracting a static area without movement, a background change detecting means for obtaining a background from a plurality of static areas of the monitoring area video information extracted by the static area extracting means, and the background change detecting means When a change in the background is detected, a feature amount extraction unit that extracts a feature amount of the region where the background changes, a recording unit for recording an image, and the feature amount extraction unit that extracts the feature amount Perform pattern matching between the feature quantity and the car template, and if the feature quantity is determined to be a vehicle, calculate the position information and time information of the vehicle and output a frame rate control command with a high frame rate, Vehicle pattern matching means for outputting a frame rate control command at a low frame rate when the vehicle cannot be determined after determining both, and the second imaging means based on the position information calculated by the vehicle pattern matching means The camera control means for controlling the second imaging means so that the face image of the passenger of the vehicle can be taken, and the second imaging means at a time interval according to a frame rate setting command from the vehicle pattern matching means. Frame rate control means for capturing video data from the image pickup means and recording it on the recording means, and parking time determination means for calculating time without motion from the vehicle position information and time information calculated by the vehicle pattern matching means And when the time calculated by the parking time determination means is less than a preset parking time, the second imaging hand Image erasing means for erasing all face images picked up in stages and recorded in the recording means.

(5) In the parked vehicle occupant imaging device described in any of (1) to (4) above, face data storage means for storing a plurality of face data, alarm output means for issuing an alarm, and the face Face search means for matching the face in the data storage means with the newly acquired face of the illegally parked passenger and operating the alarm output means when the face matches.

(6) In the parked vehicle occupant imaging device according to (5), the face search unit is configured to perform networking when a face in the face data storage unit and a newly acquired face of an illegally parked occupant are matched. To the centralized server.

  In the parked vehicle occupant imaging device described in (1) above, recording is performed only when a parked vehicle is recognized, and only the best shot image that best meets the criteria is recorded among the parked vehicle occupant's face. In addition, the amount of data for monitoring the occupants of the parked vehicle can be reduced, and the occupants can be easily identified.

  In the parked vehicle occupant imaging device described in (2) above, since the vehicle occupant is tracked and position information is acquired, the best shot face image can be acquired from the time of getting off and when the vehicle is boarded. The best shot image of the vehicle occupant can be recorded.

  In the parked vehicle occupant imaging device described in (3) above, the best shot image of the parked vehicle occupant and the parked vehicle can be recorded.

  In the parked vehicle occupant imaging device described in (4) above, when the parked vehicle is recognized, the moving image obtained from the second imaging means is recorded at a high frame rate, and if the parked vehicle cannot be recognized, the low frame rate Therefore, the amount of data for monitoring the occupants of the parked vehicle can be reduced, and the occupants can be easily identified.

  In the parked vehicle occupant imaging device described in (5) above, an alarm is issued immediately when a specific person recorded in the face data storage means such as an illegal parking addict is illegally parked. Can contribute to the clearance.

  In the parked vehicle occupant imaging device described in (6) above, when a specific person recorded in the face data storage means such as an illegal parking addict parks illegally, it can be immediately delivered to the centralized server via the network. it can.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments for carrying out the invention will be described in detail with reference to the drawings.

(Embodiment 1)
FIG. 1 is a block diagram showing a schematic configuration of a parked vehicle occupant imaging device according to Embodiment 1 of the present invention. In the figure, a parked vehicle occupant imaging device 100 according to the present embodiment includes a wide-angle camera 110, a pan / tilt / zoom camera 111, and a monitoring control recording device 112. The monitoring control recording device 112 includes a static area extraction unit 121, buffers A122 and B123, background selection units 124 and 125, a frame difference unit 126, a feature amount extraction unit 127, a car pattern matching unit 128, and a parking time. A determination unit 129, a camera control unit 130, a moving region extraction unit 131, a face region extraction unit 132, a best shot determination unit 133, an image erasure unit 134, and a recording unit 150 are provided.

  The wide-angle camera 110 corresponds to the first imaging unit. The pan / tilt / zoom camera 111 corresponds to a second imaging unit. The static area extraction unit 121 corresponds to a static area extraction unit. Further, the buffers A122 and B123, the background selection units 124 and 125, and the frame difference unit 126 constitute background change detection means. The feature quantity extraction unit 127 corresponds to feature quantity extraction means. The car pattern matching unit 128 corresponds to car pattern matching means. Moreover, the parking time determination part 129 respond | corresponds to a parking time determination means. The camera control unit 130 corresponds to camera control means. The moving area extracting unit 131 corresponds to moving area extracting means. The face area extracting unit 132 corresponds to a face area extracting unit. The best shot discrimination unit 133 corresponds to best shot discrimination means. The image erasing unit 134 corresponds to an image erasing unit. The recording unit 150 corresponds to a recording unit.

  The wide-angle camera 110 includes a wide-angle lens having a focal length of 50 mm or less in terms of 35 mm so that the entire monitoring target area can be imaged. The pan / tilt zoom camera 111 includes a mechanism for panning (horizontal direction) and tilting (vertical direction) the camera body, and an optical zoom by changing the focal length of the lens. FIG. 2 is a diagram illustrating an example of a positional relationship among the wide-angle camera 110, the pan / tilt / zoom camera 111, the monitoring control recording device 112, the vehicle 200, the driver 210 of the vehicle 200, and the roadway 220. As shown in the figure, the wide-angle camera 110 and the pan / tilt / zoom camera 111 are each oriented and arranged in the monitoring direction. At this time, the pan / tilt / zoom camera 111 can be arranged at a fixed distance so as not to collide with each other. Note that the pan / tilt / zoom camera 111 can arbitrarily set the shooting direction by pan and tilt operations. However, the pan / tilt / zoom camera 111 is set so that the center of the horizontal angle and the vertical angle is centered in the monitoring target area. Is desirable.

  In FIG. 1, a static area extraction unit 121 extracts a non-motion area (static area) in an image from video data (monitoring area video) from the wide-angle camera 110 using a difference method or a motion vector method. For example, when the difference method is used, a difference between the current frame image and the previous frame image is obtained to extract a motion region, and a region other than the motion region is extracted as a static region. In this way, a static area is extracted from the monitoring area. The buffers A122 and B123 temporarily store the static area extracted by the static area extraction unit 121 for each frame. The buffer A122 and the buffer B123 are different in data storage size. The reason for this will be described at the next frame difference unit 126.

  The background selection unit 124 selects a background image from the static region group stored in the buffer A122. Here, a foreground and a background exist in the video imaged by the wide-angle camera 110, and the foreground is always moving such as a person or a car, and the background is a thing without movement such as a wall or the ground. It can be assumed that the background has much more background than the foreground and information remains in the static area. From the static area group stored in the buffer A122, for example, the most frequent from the existing population. A background image is created using a mode value for each pixel as a background pixel by using a least squares median method that takes a value. The background selection unit 125 performs the same processing as the background selection unit 124 and selects a background image from the static region group stored in the buffer B123.

  The frame difference unit 126 is selected by the background selection unit 125 based on the background image P1 selected by the background selection unit 124 based on the static region group stored in the buffer A122 and the static region group stored in the buffer B123. The frame difference of the background image P2 is taken. Here, since the data storage sizes of the buffer A 122 and the buffer B 123 are different as described above, for example, the data capacity of the buffer A 122 is equivalent to information for 300 seconds, and the data capacity of the buffer B 123 is 10 It is assumed that the background image P1 is generated from 0 to 300 seconds of information in the buffer A122, and the background image P2 is generated from 290 to 300 seconds of information in the buffer B123. When the vehicle is stopped, the buffer A122 has a large amount of information for 288 seconds before the stop, so the vehicle information is not reflected in the background image P1, but the buffer B123 has only the information amount after the stop. The vehicle information is reflected in the image P2.

  When there is a static region from the result of the frame difference obtained by the frame difference unit 126, the feature amount extraction unit 127 retains the static region, and the feature amount for matching the aspect ratio, size, and shape from the static region. Calculate The car pattern matching unit 128 uses a matching algorithm such as a DP (Dynamic Programming) matching method to perform pattern matching between the car template and the feature quantity of the static area calculated by the feature quantity extraction unit 127, and there is a car area. Determine whether or not. If there is a vehicle area, its position information and time information are calculated and stored.

  The parking time determination unit 129 calculates how much the vehicle area has not changed from the position information and time information of the vehicle area held in the vehicle pattern matching unit 128. In this case, if the video data from the wide-angle camera 110 is, for example, NTSC (National Television Standards Committee) video data composed of 721 × 240 pixels, the X coordinate 0 to 719 and the Y coordinate 0 to 239 of the monitoring area are used. The coordinate data of the rectangular area is represented as position information. The time information is expressed in a format that can be calculated in seconds, such as YYYY / MM / DD HH: MM: SS.

  If the time calculated by the parking time discriminating unit 129 is less than the general parking time, the image erasing unit 134 determines that the parking vehicle is not a parked vehicle and is a parked vehicle from the position information and the time information. The entire face image that has been captured is deleted, and the position information and time information are deleted. Further, when the time calculated by the parking time determination unit 129 is equal to or longer than the general parking time, it is determined as a parked vehicle, and the best shot image is extracted from the face image stored in the recording unit 150 in the best shot determination unit 133. Give an instruction to select.

  The camera control unit 130 performs camera control such as panning, tilting, and zooming in order to pan-tilt-zoom the pan / tilt / zoom-equipped camera 111 in the vicinity of the parked vehicle based on the position information held in the car pattern matching unit 128. This camera control includes a command to start shooting.

  The moving area extraction unit 131 extracts a moving area having movement in the image from the video data from the camera 111 equipped with pan / tilt / zoom. For example, the difference between the current frame image and the previous frame image is obtained to extract a region with motion. The face area extracting unit 132 extracts a face area from the moving area extracted by the moving area extracting unit 131 using an algorithm for obtaining a face area such as an elliptical hough, and outputs the extracted face area. The best shot discriminating unit 133 records the face image extracted by the face area extracting unit 132 in the recording unit 150, and receives the best shot image selection command from the image erasing unit 134, thereby storing the face stored in the recording unit 150. The images are compared based on predetermined criteria (contrast is appropriate, face size is large, face direction is front, etc.), and the best shot image picked up most clearly is selected. Also, images other than the best shot image are deleted.

  Next, the operation of the parked vehicle occupant imaging device 100 configured as described above will be described using the flowcharts of FIGS. 3 and 4. First, the static region extraction unit 121 captures video data of a monitoring region from the wide-angle camera 110 (step S201), and uses a difference method, a motion vector method, or the like, from the video data, a region where there is no motion, that is, a static region. Is extracted (step S202). After detecting the static area from the monitoring area, the static area is stored in the buffer A 122 and the buffer B 123 (steps S203 and S204). In this case, if there is no empty memory that can store a static area in the buffer A 122 and the buffer B 123, the old information is overwritten sequentially. For example, it is assumed that one static area is extracted per second, and 30 static areas can be stored in the buffer A122. The static area at all times is stored in the buffer A122 from 0 to 30 seconds, but the static area extracted at the first second is overwritten at 31 seconds.

  Next, the background selection unit 124 creates a background image P1 from the static region group stored in the buffer A122 (step S205). Similarly, the background selection unit 125 creates a background image P2 from the static region group stored in the buffer B123 (step S206). Next, the frame difference unit 126 calculates the frame difference between the background image P1 and the background image P2 (step S207), and determines whether there is a stopped object from the result (step S208). If there is an object in the background, the feature quantity extraction unit 127 calculates a feature quantity for matching such as aspect ratio, size, and shape (step S209). Next, the vehicle pattern matching unit 128 uses the algorithm for extracting an arbitrary pattern based on the template information such as the aspect ratio, size, and shape such as the DP matching method, so that the vehicle is within the static region held in the vehicle pattern matching unit 128. Matching is performed to determine whether an area exists (step S210). Then, it is determined from the matching result whether there is a vehicle area (step S211). If there is a vehicle area (YES in step S211), position information / time information of the vehicle is calculated (step S212).

  Next, the camera control unit 130 gives a pan / tilt / zoom command to the pan / tilt / zoom camera 111 based on the position information calculated by the car pattern matching unit 128 (step S213). As a result, the pan / tilt / zoom camera 111 captures the vicinity of the vehicle area. Then, the moving region extraction unit 131 takes in the video data from the pan / tilt / zoom camera 111 and detects a moving region in motion using a difference method, a motion vector method, or the like (step S214). Next, the face area extraction unit 132 performs matching to determine whether a face area exists in the motion area using, for example, an elliptical Hough algorithm (step S215). If matched, the face image is recorded in the recording unit 150 (step S216).

  On the other hand, when the vehicle has started and moved out of the wide-angle camera 110 and the vehicle area cannot be matched, the determination in step S211 is NO, so the parking time determination unit 129 is acquired by the vehicle pattern matching unit 128. The time is calculated from how much position information and time information the car does not move. And it is determined whether it is a parked vehicle (step S217). If the calculated time is less than the general parking time, it is determined that the vehicle is not a parked vehicle but a parked vehicle, and the full-face image captured by the parked vehicle is deleted from the recording unit 150 (step S218). Further, the position information and time information are deleted (step S219).

  On the other hand, when the calculated time is equal to or longer than the general parking time, it is determined that the vehicle is a parked vehicle, and the best shot is compared from the face image stored in the recording unit 150 based on a predetermined comparison criterion. An image is selected (step S220). In this case, in the best shot comparison criterion, for example, when the three best shots of the contrast of the face image, the face image size, and the face direction information are used as the discrimination criterion, the extracted face image is scored for each element and scored. The total score of the face image is calculated by multiplying the object by a predetermined weighting factor. The face image extracted from the current frame and the face image extracted from the past frame are sequentially compared for each frame by the total score, and the one having the highest total score is determined as the best shot. Other images are deleted (step S221).

  As described above, according to the parked vehicle occupant imaging device 100 according to the present embodiment, if an area without motion, that is, a static area in the image can be extracted from the video data of the monitoring area from the wide-angle camera 110, the static area is extracted from the static area. The feature amount is calculated, pattern matching between the obtained feature amount and the car template is performed. If there is a car area, the position information and time information are calculated, and how much the car area does not move. On the other hand, on the basis of the position information, the pan / tilt / zoom camera 111 is controlled to obtain video data from the pan / tilt / zoom camera 111, a moving area in the image is extracted from the video data, and a face area is further extracted. Is extracted and recorded in the recording unit 150, and if the time when there is no movement in the vehicle area is less than the general parking time, the whole face image that is determined to be a stopped vehicle and recorded in the recording unit 150 If the time when there is no movement in the vehicle area is longer than the general parking time, the best shot image is selected from the face images recorded in the recording unit 150 as a parked vehicle, and the remaining face images are deleted. To do. Therefore, when it is determined that the vehicle is a parked vehicle, only the best shot image of the passenger of the vehicle is stored, so that the amount of data for monitoring the passenger of the parked vehicle can be reduced and the passenger can be easily identified. Can do.

  In the above embodiment, the static areas extracted by the static area extraction unit 121 are stored in the two buffers A122 and B123. However, the number of buffers is not limited to two, and two. There may be more than one. However, the background selection unit 125 needs to be increased accordingly.

(Embodiment 2)
FIG. 5 is a block diagram showing a schematic configuration of the parked vehicle occupant imaging device according to Embodiment 2 of the present invention. 5 that are the same as those in FIG. 1 are given the same reference numerals, and descriptions thereof are omitted.

  In FIG. 5, the parked vehicle occupant imaging device 300 according to the present embodiment includes a moving region extraction unit 131, a face region extraction unit 132, and a best shot discrimination of the parked vehicle occupant imaging device 100 according to the above-described first embodiment. A vehicle pattern matching unit 322 in which a frame rate control unit 321 is provided instead of the unit 133 and a new function is added to the vehicle pattern matching unit 128 of the parked vehicle occupant imaging device 100 according to the first embodiment. Is different. The frame rate control unit 321 changes the interval at which video data from the pan / tilt / zoom camera 111 is captured and outputs the video data to the recording unit 150. The car pattern matching unit 322 gives a frame rate control command to the frame rate control unit 321 so as to capture an image at an arbitrary frame rate. The frame rate control unit 321 takes in video data from the pan / tilt / zoom camera 111 at time intervals according to the frame rate control command. The frame rate control unit 321 corresponds to a frame rate control unit.

  Next, the operation of the parked vehicle occupant imaging device 300 will be described using the flowcharts of FIGS. 6 and 7. Here, steps S401 to S413 in FIG. 6 are the same as steps S201 to S213 in FIG.

  6 and 7, in step S412, the vehicle pattern matching unit 128 determines that the static region is a vehicle region, calculates position information and time information of the vehicle region, and gives a camera control command to the camera control unit 130. Thereafter, a frame rate control command is given to the frame rate control unit 321 to record a moving image on the recording unit 150 at a high frame rate (step S414). The frame rate control unit 321 records the video data from the pan / tilt / zoom camera 111 in the recording unit 150 at a high frame rate when a frame rate control command is given.

  On the other hand, when the vehicle has started and moved out of the wide-angle camera 110 and the vehicle area cannot be matched, the process moves from step S411 to step S415 to determine whether the vehicle is a parked vehicle. If the vehicle is a parked vehicle, the vehicle pattern matching unit 322 inputs a frame rate control command to the frame rate control unit 321 so as to record at a low frame rate (step S417). When this frame rate control command is given, the frame rate control unit 321 records video data from the pan / tilt / zoom camera 111 on the recording unit 150 at a low frame rate. If it is determined in step S415 that the vehicle is not a parked vehicle, that is, a stopped vehicle, the image erasing unit 134 is instructed to delete a moving image of the stopped vehicle from the position information and time information. As a result, the image erasing unit 134 deletes the moving image from the recording unit 150 (step S416).

  As described above, according to the parked vehicle occupant imaging device 300 according to the present embodiment, when there is a car area by performing pattern matching with the car template, the position information and time information are calculated, and the car area is While determining whether there is no movement, the video data from the camera 111 equipped with pan / tilt / zoom is recorded in the recording unit 150 at a high frame rate. If the time that has continued is longer than the general parking time, the video data from the camera 111 equipped with pan / tilt / zoom is recorded on the recording unit 150 at a low frame rate, and the time when the vehicle area has continued is generally If it is less than the parking time, the recorded data in the recording unit 150 is deleted. Therefore, it is possible to record at a high frame rate only when a parked vehicle is generated.

(Embodiment 3)
FIG. 8 is a block diagram showing a schematic configuration of a parked vehicle occupant imaging device according to Embodiment 3 of the present invention. 8 that are the same as those in FIG. 1 are assigned the same reference numerals, and descriptions thereof are omitted.

  In FIG. 8, the parked vehicle occupant imaging device 500 according to the present embodiment has the same configuration as that of the parked vehicle occupant imaging device 100 according to Embodiment 1 described above, and in addition, a vehicle image storage unit 521 and a vehicle image. The difference is that a control unit 522 is provided. The vehicle image storage unit 521 stores the vehicle image extracted by the vehicle pattern matching unit 128 for each frame. The vehicle image control unit 522 acquires the vehicle image stored in the vehicle image storage unit 521 at the time when the best shot image determined by the best shot determination unit 133 is captured, and records it in the recording unit 150. Then, the car image stored in the car image storage unit 521 is deleted. The vehicle image storage unit 521 and the vehicle image control unit 522 constitute a vehicle image control unit.

  The operation of the parked vehicle occupant imaging device 500 configured as described above will be described with reference to the flowchart of FIG. The parked vehicle occupant imaging device 500 according to the present embodiment has a configuration in which a vehicle image storage unit 521 and a vehicle image control unit 522 are added to the parked vehicle occupant imaging device 100 according to the first embodiment. The description of the configuration is omitted.

  The vehicle image storage unit 521 stores the extracted vehicle image for each frame (step S601). The vehicle image control unit 522 acquires the time when the best shot of the face image was captured (step S602), and then records the vehicle image at the time when the best shot of the facial image was captured in the recording unit 150 (step S603). . Next, the car image stored in the car image storage unit 521 is deleted (step S604).

  As described above, according to the parked vehicle occupant imaging device 500 according to the present embodiment, the vehicle image at the time when the best shot image was obtained is recorded in the recording unit 150, so the parked vehicle occupant best shot image and the parking Both vehicles can be recorded in the recording unit 150.

(Embodiment 4)
FIG. 10 is a block diagram showing a schematic configuration of a parked vehicle occupant imaging device according to Embodiment 4 of the present invention. 10 that are the same as those in FIG. 1 are assigned the same reference numerals, and descriptions thereof are omitted.

  In FIG. 10, a parked vehicle occupant imaging device 700 according to the present embodiment has the same configuration as that of the parked vehicle occupant imaging device 100 according to the first embodiment described above, and also includes a face database 721 and a face search unit. 722 and an alarm output unit 723 are different. The face database 721 stores a plurality of face images. The face search unit 722 searches for the face image stored in the face database 721 using the face image stored in the recording unit 150 as a key. The alarm output unit 723 outputs an alarm. The face database 721 corresponds to a face data storage unit. The alarm output unit 723 corresponds to an alarm output unit. The face search unit 722 corresponds to face search means.

  The operation of the parked vehicle occupant imaging device 700 configured as described above will be described with reference to the flowchart of FIG. The parked vehicle occupant imaging device 700 according to the present embodiment is configured by adding a face database 721, a face search unit 722, and an alarm output unit 723 to the parked vehicle occupant imaging device 100 according to the first embodiment. Therefore, the description of the configuration other than these is omitted.

  First, the face search unit 722 searches the face information existing in the face database 721 using the best shot image obtained just before as a key (step S801). The search algorithm may be a matching method using a face identification detector using AdaBoost learning, for example. The face database 721 may be a face database of an illegal parking addict, for example.

  Then, it is determined whether or not the acquired face image exists in the face database 721 (step S802). If it exists (YES in step S802), the face search unit 722 outputs an alarm to the alarm output unit 723. Give instructions. As a result, the alarm output unit 723 outputs an alarm (step S803). The alarm output unit 723 may sound a buzzer, or the parked vehicle occupant imaging device 700 may have an I / F (interface) connected to a general network such as an IP (Internet Protocol) network. Alternatively, a method may be used in which illegal parking information is transmitted to a central server or the nearest dispatching station.

  Thus, according to parked vehicle occupant imaging device 700 according to the present embodiment, the best shot image and face database 721 are searched for a face, and an alarm is output when the best shot image exists in face database 721. Therefore, when a specific person stored in the face database 721, such as an illegal parking addict, parks illegally, an alarm can be issued immediately.

  In addition, the parked vehicle occupant imaging device 300 according to the second embodiment and the parked vehicle occupant imaging device 500 according to the third embodiment are the same as the parked vehicle occupant imaging device 700 according to the present embodiment. Means (that is, means for matching the face in the face database registered in advance with the newly acquired face of the illegally parked passenger and issuing an alarm when the face matches) may be added. .

(Embodiment 5)
FIG. 12 is a block diagram showing a schematic configuration of a parked vehicle occupant imaging device according to Embodiment 5 of the present invention. 12 that are the same as those in FIG. 1 are assigned the same reference numerals, and descriptions thereof are omitted.

  In FIG. 12, the parked vehicle occupant imaging device 900 according to the present embodiment has the same configuration as that of the parked vehicle occupant imaging device 100 according to the above-described first embodiment. The difference is that an area extracting unit 922, a person feature amount extracting unit 923, and a person tracking unit 924 are provided.

  The moving area extraction unit 921 has the same function as the moving area extraction unit 131, and extracts a moving area with movement in the image from the video data from the wide-angle camera 110. For example, a moving region with motion is extracted by obtaining a difference between the image of the current frame and the image of the previous frame. The human region extraction unit 922 extracts a human region from the moving region extracted by the moving region extraction unit 921 by an algorithm that acquires a human region from template information such as an aspect ratio, size, and shape using a DP matching algorithm. Then, the position information and time information of the person at that time are extracted.

  The person feature amount extraction unit 923 calculates a feature amount that can be distinguished for each person such as height, body shape aspect ratio, and clothing color of the human region extracted by the human region extraction unit 922. The person tracking unit 924 determines whether or not a person having the feature amount calculated by the person feature amount extraction unit 923 exists in the image captured by the wide-angle camera 110, and if it exists, calculates position information. . The camera control unit 130 gives pan, tilt, and zoom commands to the pan / tilt / zoom camera 111 based on the position information calculated by the person tracking unit 924, and captures an image of the vicinity of the vehicle occupant on the pan / tilt / zoom camera 111. Control to a possible angle of view.

  The moving region extracting unit 131 corresponds to a second moving region extracting unit. The moving region extraction unit 921 corresponds to first moving region extraction means. The human area extracting unit 922 corresponds to a human area extracting unit. The person feature quantity extraction unit 923 corresponds to a person feature quantity extraction unit. The person tracking unit 924 corresponds to a person tracking unit.

  The parked vehicle occupant imaging device 900 configured as described above will be described with reference to the flowcharts of FIGS. First, as initialization, the number of passengers is set to “0” (step S1000). Next, the static area extraction unit 121 captures the video data of the monitoring area from the wide-angle camera 110 (step S1001), and uses the difference method, the motion vector method, or the like to extract a non-motion area in the image, that is, a static area from the video data. Extract (step S1002). After extracting the static area from the monitoring area, the static area is stored in the buffer A 122 and the buffer B 123 (steps S1003 and S1004). In this case, if there is no empty memory that can store a static area in the buffer A 122 and the buffer B 123, the old information is overwritten sequentially. For example, it is assumed that one static area is extracted per second, and 30 static areas can be stored in the buffer A122. The static area at all times is stored in the buffer A122 from 0 to 30 seconds, but the static area extracted at the first second is overwritten at 31 seconds.

  Next, the background selection unit 124 creates a background image P1 from the static region group stored in the buffer A122 (step S1005). Similarly, the background selection unit 125 creates a background image P2 from the static region group stored in the buffer B123 (step S1006). Next, the frame difference unit 126 calculates the frame difference between the background image P1 and the background image P2 (step S1007), and determines whether there is a stopped object from the result (step S1008). If an object is present in the background, the feature quantity extraction unit 127 calculates a feature quantity for matching such as aspect ratio, size, and shape (step S1009). Next, the vehicle pattern matching unit 128 uses an algorithm that extracts an arbitrary pattern based on template information such as aspect ratio, size, and shape such as the DP matching method to match whether the vehicle region exists in the static region. This is performed (step S1010). Then, it is determined from the matching result whether there is a vehicle area (step S1011), and if there is a vehicle area (in the case of YES), the position information and time information of the vehicle are calculated (step S1012).

  Next, the moving area extraction unit 921 takes in the video data of the monitoring area from the wide-angle camera 110, and detects a moving area in the image, that is, a moving area from the video data by using a difference method, a motion vector method, or the like ( Step S1013). Next, the human region extraction unit 922 uses an algorithm that extracts an arbitrary pattern based on template information such as aspect ratio, size, and shape, such as DP matching method, to match whether a human region exists in the moving region. (Step S1014), it is determined whether there is a human area (step S1015). If there is a human area, feature quantities that can be distinguished for each person, such as height, body shape aspect ratio, and clothing color, are calculated in all detected human areas (step S1016). Then, it is determined whether or not the calculated feature amount is a feature amount newly generated around the position information of the vehicle area (step S1017). If it is a newly generated feature quantity, it is determined that this person area is a person who gets off the car, and the number of passengers is increased by 1 and held as a passenger feature quantity (step S1018).

  Next, the person tracking unit 924 checks whether there is an equivalent passenger feature amount in the feature amount calculated in step S1016. If the equivalent value exists, the person tracking unit 924 determines that the equivalent feature amount is the passenger. Then, position information is calculated (step S1019). Next, the camera control unit 130 transmits a pan / tilt / zoom command to the position information calculated by the person tracking unit 924 to the pan / tilt / zoom camera 111 (step S1020), and an image of the vicinity of the passenger is captured from the pan / tilt / zoom camera 111. Is done. In addition, the parking time determination unit 129 calculates parking time information based on the vehicle area and position information acquired by the vehicle pattern matching unit 128 to determine how much the vehicle has changed.

  Next, the moving region extraction unit 131 takes in video data in the vicinity of the human region from the camera 111 equipped with pan / tilt / zoom, and detects a moving region in motion using a difference method, a motion vector method, or the like (step S1021). For example, when using the difference method, the moving region extraction unit 131 obtains a difference between the current frame image and the previous frame image, and detects a moving region from the monitoring region.

  Next, the face area extraction unit 132 performs matching to determine whether a face area exists in the motion area using, for example, an elliptical Hough algorithm (step S1022). If matching is performed, the face image is stored in the recording unit 150 (step S1023). Even if the person region having the passenger feature amount moves outside the angle of view of the wide-angle camera 111, matching and face image storage are performed until the parked vehicle moves.

  If the vehicle area cannot be matched because the vehicle has started and moved out of the angle of view of the wide-angle camera 110, it is determined in step S1011 that there is no vehicle area. Therefore, the parking time determination unit 129 includes the vehicle pattern matching unit 128. From the position information and time information acquired in step 1, the time is calculated for how many minutes the vehicle has not changed. When the time calculated by the parking time determination unit 129 is less than the general parking time, the parking time determination unit 129 determines that the parking vehicle is not a parked vehicle, and proceeds from step S1024 to step S1025. All face images captured by the stopped vehicle are deleted from the time information. After deleting all the face images, the position information and time information are deleted (step S1026).

  On the other hand, when the time calculated by the parking time determination unit 129 is equal to or longer than a general parking time, it is determined that the vehicle is a parked vehicle, so the best shot determination unit 133 is stored in the recording unit 150. The face image is compared based on a predetermined comparison criterion, and the best shot image is selected (step S1027). In this case, in the best shot comparison criterion, for example, when the three best shots of the contrast of the face image, the face image size, and the face direction information are used as the discrimination criterion, the extracted face image is scored for each element and scored. The total score of the face image is calculated by multiplying the object by a predetermined weighting factor. The face image extracted from the current frame and the face image extracted in the past frame are sequentially compared for each frame by the total score, and the one having the highest total score is determined as the best shot. Other images are deleted by the image erasure unit 134 (step S1028).

  As described above, according to the parked vehicle occupant imaging device 900 according to the present embodiment, the moving region extracting unit 921 and the human region extracting unit 922 extract the human region in the wide-angle camera 110, and the person feature amount extracting unit 923 Since the person tracking unit 924 tracks the passenger and acquires the position information, the best shot face image can be acquired from the time of getting off and at the time of getting on, and the clearer best shot image of the parked vehicle passenger can be recorded. it can.

  In the present embodiment, the same means as in the parked vehicle occupant imaging device 700 according to the above-described fourth embodiment, that is, the face in the face database registered in advance and the newly acquired face of the illegally parked occupant It is also possible to add a means for issuing an alarm when matching is performed.

  The present invention has an effect that enables efficient use without using a large-capacity recording device, and can easily find out the occupant of the parked vehicle from the recorded image. Thus, the present invention can be applied to a parked vehicle occupant imaging device that monitors and records parked vehicles and occupants.

The block diagram which shows schematic structure of the parked vehicle passenger imaging device which concerns on Embodiment 1 of this invention. The figure which shows an example of the positional relationship of the wide angle camera, the camera equipped with pan tilt zoom, the monitoring control recording device, the vehicle, and the driver of the vehicle in the parked vehicle occupant imaging device according to Embodiment 1 of the present invention. The flowchart for operation | movement description of the parked vehicle passenger imaging device which concerns on Embodiment 1 of this invention. The flowchart for operation | movement description of the parked vehicle passenger imaging device which concerns on Embodiment 1 of this invention. The block diagram which shows schematic structure of the parked vehicle passenger imaging device which concerns on Embodiment 2 of this invention. Flow chart for explaining the operation of the parked vehicle occupant imaging device according to the second embodiment of the present invention. Flow chart for explaining the operation of the parked vehicle occupant imaging device according to the second embodiment of the present invention. The block diagram which shows schematic structure of the parked vehicle passenger imaging device which concerns on Embodiment 3 of this invention. The flowchart for operation | movement description of the parked vehicle passenger imaging device which concerns on Embodiment 3 of this invention. The block diagram which shows schematic structure of the parked vehicle passenger imaging device which concerns on Embodiment 4 of this invention. The flowchart for operation | movement description of the parked vehicle passenger imaging device which concerns on Embodiment 4 of this invention. The block diagram which shows schematic structure of the parked vehicle passenger imaging device which concerns on Embodiment 5 of this invention. Flowchart for explaining operation of parked vehicle occupant imaging device according to embodiment 5 of the present invention Flowchart for explaining operation of parked vehicle occupant imaging device according to embodiment 5 of the present invention Flowchart for explaining operation of parked vehicle occupant imaging device according to embodiment 5 of the present invention

Explanation of symbols

100, 300, 500, 700, 900 Parked vehicle occupant imaging device 110 Wide-angle camera 111 Camera with pan / tilt zoom 112, 113, 114, 115, 116 Monitoring control recording device 121 Static region extraction unit 122 Buffer A
123 Buffer B
124, 125 Background selection unit 126 Frame difference unit 127 Feature amount extraction unit 128, 322 Car pattern matching unit 129 Parking time determination unit 130 Camera control unit 131 Moving region extraction unit 132 Face region extraction unit 133 Best shot determination unit 134 Image erasure unit DESCRIPTION OF SYMBOLS 150 Recording part 321 Frame rate control part 521 Car image storage part 522 Car image control part 721 Face database 722 Face search part 723 Alarm output part 921 Dynamic area extraction part 922 Human area extraction part 923 Person feature-value extraction part 924 Person tracking part

Claims (6)

First imaging means arranged toward the monitoring area;
A second imaging means for changing the imaging range in accordance with an external command;
A static area extracting means for extracting a static area having no motion in the image from the monitoring area video information from the first imaging means;
Background change detection means for detecting a change in background from the static area of the monitoring area video information extracted by the static area extraction means;
A feature amount extracting means for extracting a feature amount of a region in which the background is changed when a background change is detected by the background change detecting means;
Performs pattern matching between the feature quantity extracted by the feature quantity extraction means and a vehicle template, determines whether the feature quantity is a vehicle, and if it is a vehicle, calculates position information and time information of the vehicle Vehicle pattern matching means,
Camera control means for giving an instruction to the second imaging means based on the position information calculated by the vehicle pattern matching means, and enabling the second imaging means to image a passenger of the vehicle;
Moving area extracting means for extracting a moving area having movement from the video information from the second imaging means;
Recording means for recording an image;
A face area extracting means for extracting a face area from the moving area extracted by the moving area extracting means and recording it in the recording means;
A parking time determination means for calculating time without movement from the position information and time information of the vehicle calculated by the vehicle pattern matching means;
When the time calculated by the parking time determination unit is less than a preset parking time, an image erasing unit that deletes the entire face image captured by the second imaging unit and recorded in the recording unit;
When the time calculated by the parking time determination unit is equal to or more than a preset parking time, the best match among the all face images captured by the second imaging unit and recorded in the recording unit Best shot discriminating means for discriminating the best shot and recording only the image discriminated as the best shot in the recording means;
A parking vehicle occupant imaging device. First imaging means arranged toward the monitoring area;
A second imaging means for changing the imaging range in accordance with an external command;
A static area extracting means for extracting a static area having no motion in the image from the monitoring area video information from the first imaging means;
Background change detecting means for obtaining a background from a plurality of static areas of the monitoring area video extracted by the static area extracting means;
A feature amount extracting means for extracting a feature amount of a region in which the background is changed when a background change is detected by the background change detecting means;
Performs pattern matching between the feature quantity extracted by the feature quantity extraction means and a vehicle template, determines whether the feature quantity is a vehicle, and if it is a vehicle, calculates position information and time information of the vehicle Vehicle pattern matching means,
First moving area extracting means for extracting a moving area having movement in an image from the monitoring area video information;
A human area extracting means for extracting a human area from the moving area extracted by the first moving area extracting means and calculating position information and time information of the extracted human area;
Human feature quantity extraction means for calculating the feature quantity of the human area extracted by the human area extraction means;
A person tracking unit that determines whether or not a person having the feature amount calculated by the person feature amount extraction unit exists in the monitoring area video information, and if there is a person tracking unit that calculates position information;
Camera control for giving an instruction to the second imaging unit based on the position information calculated by the person tracking unit and controlling the second imaging unit so that an image of the vicinity of the occupant of the vehicle can be captured. Means,
Second moving area extracting means for extracting a moving area having movement in an image from video information from the second imaging means;
Recording means for recording an image;
A face area extracting means for extracting a face area from the moving area extracted by the second moving area extracting means and recording it in the recording means;
A parking time determination means for calculating time without movement from the position information and time information of the vehicle calculated by the vehicle pattern matching means;
When the time calculated by the parking time determination unit is less than a preset parking time, an image erasing unit that deletes the entire face image captured by the second imaging unit and recorded in the recording unit;
When the time calculated by the parking time determination unit is equal to or more than a preset parking time, the best match among the all face images captured by the second imaging unit and recorded in the recording unit Best shot discriminating means for discriminating the best shot and recording only the image discriminated as the best shot in the recording means;
A parking vehicle occupant imaging device.   The vehicle image control means which records the image of the parking area imaged by the said 1st imaging means in the said recording means at the time when the best shot image was selected by the said best shot discrimination means. The parked vehicle occupant imaging device described in 1. First imaging means arranged toward the monitoring area;
A second imaging means for changing the imaging range in accordance with an external command;
A static area extracting means for extracting a static area having no motion in the image from the monitoring area video information from the first imaging means;
A background change detecting means for obtaining a background from a plurality of static areas of the monitoring area video information extracted by the static area extracting means;
Feature amount extracting means for extracting a feature amount of a region where the background is changed when a background change is detected by the background change detecting means;
Recording means for recording an image;
Pattern matching between the feature quantity extracted by the feature quantity extraction means and a car template is performed, and when the feature quantity is determined to be a vehicle, position information and time information of the vehicle are calculated and a frame with a high frame rate is calculated. Vehicle pattern matching means for outputting a frame rate control command at a low frame rate when it is determined that the vehicle is not available after outputting a rate control command and being a vehicle;
Camera control for giving a command to the second imaging unit based on the position information calculated by the vehicle pattern matching unit and controlling the second imaging unit so that a face image of a passenger of the vehicle can be captured Means,
Frame rate control means for capturing video data from the second imaging means at a time interval according to a frame rate setting command from the vehicle pattern matching means and recording the video data on the recording means;
A parking time determination means for calculating time without movement from the position information and time information of the vehicle calculated by the vehicle pattern matching means;
When the time calculated by the parking time determination unit is less than a preset parking time, an image erasing unit that deletes the entire face image captured by the second imaging unit and recorded in the recording unit;
A parking vehicle occupant imaging device. Face data storage means for storing a plurality of face data;
An alarm output means for issuing an alarm;
A face search unit that performs matching between a face in the face data storage unit and a newly acquired illegally parked passenger's face, and activates the alarm output unit when it matches,
The parked vehicle occupant imaging device according to any one of claims 1 to 4.   6. The parked vehicle occupant according to claim 5, wherein when the face in the face data storage unit matches a newly acquired face of an illegally parked occupant, the face search unit distributes the face to the centralized server via a network. Imaging device.

Images