WO2013125301A1

WO2013125301A1 - Surveillance system

Info

Publication number: WO2013125301A1
Application number: PCT/JP2013/051753
Authority: WO
Inventors: 秋彦香西; 照久高野; 真史安原
Original assignee: 日産自動車株式会社
Priority date: 2012-02-24
Filing date: 2013-01-28
Publication date: 2013-08-29

Abstract

A surveillance system (1) is equipped with surveillance terminal devices (10) and a central surveillance device (20) capable of communicating via a telecommunication line network (30). The central surveillance device (20) acquires surveillance information that includes at least position information and is output from the surveillance terminal devices (10), sets a surveillance site where the subject of tracking that has been tracked by a surveillant is predicted to be, selects a surveillance terminal device (10) that monitors a mobile body approaching and/or moving away from the prescribed surveillance area in reference to the set surveillance site, and outputs image transmission commands for transmitting surveillance information, including image information, to the selected surveillance terminal device (10).

Description

Monitoring system

The present invention relates to a monitoring system.
This application claims the priority of Japanese Patent Application No. 2012-39328 filed on Feb. 24, 2012, and the above-mentioned designated countries are permitted to be incorporated by reference. The contents described in the application are incorporated into the present application by reference and are part of the description of the present application.

A security device that detects occurrences of abnormalities is known by installing a plurality of security camera devices in shopping streets, store entrances, home entrances and other streets, and monitoring surrounding images captured by the security camera devices. (Patent Document 1).

JP 2011-215767 A

However, since the imaging area of the fixed security camera device is fixed, when the movable tracking target such as a human or a vehicle moves and leaves the imaging area of the fixed security camera device, the tracking target is continued. Problem that it can not be monitored.

An object of the present invention is to provide a monitoring system capable of continuously monitoring a tracking object moving at random by using a camera mounted on a moving body.

The present invention selects a monitoring terminal device of a mobile capable of imaging a monitoring area based on a monitoring point where a tracking target is predicted to exist, and monitoring information including image information for the selected monitoring terminal device. The above object is achieved by outputting an image transmission command to be transmitted.

According to the present invention, it is possible to acquire an image of the tracking target from a monitoring terminal apparatus mounted on a mobile capable of imaging a monitoring area based on a monitoring point where a tracking target is predicted to be present. It can be monitored continuously. As a result, the central monitoring device can continuously monitor the randomly moving tracking target by using the monitoring terminal device mounted on the randomly moving mobile object.

It is a schematic diagram which shows the monitoring system which concerns on one embodiment of this invention. It is a block diagram which shows the monitoring system of FIG. It is a perspective view which shows arrangement | positioning of the vehicle-mounted camera in the monitoring system of FIG. 1, and its imaging range. It is a top view which shows arrangement | positioning of the vehicle-mounted camera in the monitoring system of FIG. 1, and its imaging range. It is a figure showing an example of a photography picture of a front in-vehicle camera. It is a figure which shows an example of the picked-up image of the vehicle-mounted camera of the right side. It is a figure showing an example of a photography picture of a rear in-vehicle camera. It is a figure which shows an example of the picked-up image of the vehicle-mounted camera of the left side. It is a figure which shows an example of the picked-up image of the vehicle-mounted camera of indoor. It is a figure showing an example of a surveillance picture generated based on a plurality of pictures. It is a figure for demonstrating distortion correction processing of a surveillance image. It is a schematic diagram which shows an example of a projection model. It is a cross-sectional schematic diagram in alignment with xy surface of the projection model shown in FIG. It is a figure which shows the example of an image displayed on the display of a central monitoring apparatus. It is a flowchart which shows the main control content by the side of the monitoring terminal device of the monitoring system of FIG. It is a flowchart (the 1) which shows the main control content by the side of the central monitoring apparatus of the monitoring system of FIG. It is a flowchart (the 2) which shows the main control content by the side of the central monitoring apparatus of the monitoring system of FIG. It is a figure which shows the example of information of a database.

One embodiment shown below embodies the surveillance system concerning the present invention to surveillance system 1 which carries out central surveillance of the security in a town with authorities, such as a police station and a fire department. That is, position information of each of a plurality of moving bodies, image information around the moving bodies, and time information are acquired at a predetermined timing, and the position information, the image information and the time information are transmitted via wireless communication. The information is transmitted to a central monitoring device installed at the authority, and the position information is displayed on the map information, and the image information and the time information are displayed on the display as needed. Therefore, as shown in FIG. 1, the monitoring system 1 of this example acquires and processes monitoring information via the telecommunication network 30 and the monitoring terminal device 10 for acquiring monitoring information such as position information and image information. A central monitoring device 20 is provided.

FIG. 2 is a block diagram showing a specific configuration of the monitoring terminal device 10 and the central monitoring device 20. As shown in FIG. The monitoring system of the present embodiment continuously acquires monitoring information regarding a movable specific tracking target.

The monitoring terminal device 10 is a terminal device mounted on a plurality of mobile units V, and is mounted on each of a plurality of mobile units and a position detection function for detecting position information of each of the plurality of mobile units V. An image generation function that captures the surroundings of the moving object with a camera to generate image information, a time detection function, an information acquisition control function that acquires position information, image information, and time information at a predetermined timing, and Monitoring information generation function for generating monitoring information including position information and / or image information, and communication function for outputting the position information, image information and time information to the central monitoring device 20 and acquiring a command from the central monitoring device 20 And the function of reporting the occurrence of an abnormality. The monitoring information generation function can generate monitoring information including sighting information such as having witnessed a person fleeing from an abnormal occurrence site at the time of abnormality, and having witnessed a tracking target indicated by the central monitoring device 20. This sighting information is used when setting the surveillance area.

Therefore, a plurality of in-vehicle cameras 11a to 11e, an image processing device 12, a communication device 13, an in-vehicle control device 14, a position detection device 15, and a notification button 16 are provided. In addition, the monitoring terminal device 10 according to the present embodiment can exchange information with the vehicle controller 17 that centrally controls the vehicle speed sensor 18, the navigation device 19, and other in-vehicle electronic devices. The monitoring terminal device 10 can transmit the vehicle speed information acquired via the vehicle speed sensor 18 and the vehicle controller 17 to the central monitoring device 20 as part of the monitoring information. The time information may be omitted because it is information mainly used for post hoc analysis of events.

The mobile V on which the monitoring terminal device 10 is mounted is not particularly limited as long as it travels a target monitoring area, and includes mobiles such as passenger cars, two-wheeled vehicles, industrial vehicles, and trams. Vehicle V1, private passenger car V2 and emergency passenger car V3 are included, but taxis and route buses V1 that run at random and always in a predetermined area are particularly preferable. FIG. 1 exemplifies a taxi V1, a private passenger car V2, an emergency passenger car V3 such as a police car, a fire engine or an ambulance, but when these are generically referred to as a mobile V or a passenger car V.

Each moving body V includes a plurality of on-vehicle cameras 11a to 11e (hereinafter collectively referred to as cameras 11), an image processing device 12, a communication device 13, an on-vehicle control device 14, a position detection device 15, and a notification button 16 Is mounted. The camera 11 is configured of a CCD camera or the like, captures an image of the periphery of the moving body V, and outputs an imaging signal to the image processing apparatus 12. The image processing device 12 reads an imaging signal from the camera 11 and performs image processing on image information. Details of this image processing will be described later.

The position detection device 15 includes a GPS device and its correction device, detects the current position of the mobile V, and outputs the current position to the on-vehicle control device 14. The notification button 16 is an input button installed in the vehicle compartment, and information (error information) for reporting an abnormality when a driver or a passenger discovers an incident (an incident related to security such as an accident, a fire, or a crime) It is a manual button to input). This information can include position information of the mobile V that has reported an abnormality.

The on-vehicle control device 14 includes a CPU, a ROM, and a RAM, and controls the image processing device 12, the communication device 13, and the position detection device 15 when the notification button 16 is pressed. The image information, the position information of the mobile V detected by the position detection device 15, and the time information from a clock built in the CPU are output to the central monitoring device 20 via the communication device 13 and the telecommunication network 30. . In addition, a command requesting information such as an image transmission command is acquired from the central monitoring device 20 received via the telecommunication network 30 and the communication device 13, and the image processing device 12, the communication device 13 and the position detection device 15 are obtained. The communication device 13 controls monitoring information including image information generated by the image processing device 12, position information of the moving object V detected by the position detection device 15, and time information from a clock built in the CPU. And, it outputs to the central monitoring device 20 via the telecommunication network 30 and the monitoring terminal device 10 mounted on another mobile unit V. The in-vehicle control device 14 can store monitoring information including image information, position information, time information and the like for at least a predetermined time.

Further, for the acquired image information, the in-vehicle control device 14 extracts the subject included in the image information, further extracts the feature of the subject, and stores the subject information and the feature information in association with each piece of image information. Can be When the on-vehicle control device 14 acquires a feature to be tracked from the central monitoring device 20, it can search the stored monitoring information for image information including a subject having the feature. Then, when the image information of the subject having the feature to be tracked is retrieved, the in-vehicle control device 14 can include report information indicating the presence of the trace object in the monitoring information and output it.

Furthermore, for the acquired image information, the in-vehicle control device 14 extracts the subject included in the image information, and further extracts the feature of the subject, and when the subject is a human, a human head, particularly a human The subject information and the feature information can be stored in association with each piece of image information by extracting features for measuring the face similarity. Whether or not the subject is a human can be determined by the size, shape, movement of the limbs, etc. The head can be determined by the position (upper part), color, shape, etc. of the subject. Among them, the face part can be extracted by the arrangement of the feature parts such as color, eyes, nose, mouth and eyebrows. In addition, as the feature of the face, it is possible to extract the size, positional relationship, and the like of each feature portion such as eyes, nose, mouth and eyebrows.

Then, when the in-vehicle control device 14 acquires the image information or face feature of the face to be tracked from the central monitoring device 20, the in-vehicle control device 14 stores the image information of which the similarity between the face to be tracked and the face is a predetermined value or more. It is possible to search from within the monitoring information. When image information including a face whose similarity is equal to or more than a predetermined value is searched, notification information indicating the presence of a tracking target can be included in monitoring information and output. Incidentally, the determination of the similarity of the face can be quantitatively performed based on the contour of the face and the positional relationship of the eyes, eyebrows, nose and mouth constituting the face, and the like. The determination method of the face similarity is not particularly limited, and techniques known at the time of filing can be applied. The feature of the face to be tracked may be determined on the side of the central monitoring device 20, or may be determined on the side of the monitoring terminal device 10. When the central monitoring device 20 side obtains face features, the obtained face features are included in an image transmission command and sent to the monitoring terminal device 10. On the other hand, when the feature of the face is obtained on the monitoring terminal device 10 side, the central monitoring device 20 sends image information of the face to the monitoring terminal device 10.

The communication device 13 is a communication means capable of wireless communication, and exchanges information with the communication device 23 of the central monitoring device 20 via the telecommunication network 30. When the telecommunication network 30 is a commercial telephone network, a mobile telephone communication device can be used for general purpose, and when the telecommunication network 30 is a dedicated telecommunication network of the monitoring system 1 of this example, it is

dedicated Communication devices

13 and 23 can be used. Note that, instead of the telecommunication network 30, a wireless LAN, WiFi (registered trademark), WiMAX (registered trademark), Bluetooth (registered trademark), a dedicated wireless channel, or the like can be used.

The central monitoring device 20 stores the information acquisition function of acquiring the position information and the image information output from the monitoring terminal device 10 described above, and at least temporarily stores the acquired monitoring information in the database 26 in association with the position information. It has a function and a display control function for displaying the map information from the map database, displaying and controlling the received position information on the map information, and displaying the received image information on the display 24. Further, the central monitoring device 20 refers to the monitoring information in the database 26 and specifies a tracking target to be tracked by the monitoring person, and sets a monitoring point setting function of setting a monitoring point where the tracking target is predicted to be present, A selection function of selecting the monitoring terminal device 10 of the passenger car V belonging to the monitoring area based on the selected monitoring point, and at least including image information for the selected monitoring terminal device 10, including image information as necessary It has an instruction output function of outputting an image transmission instruction for transmitting monitoring information.

The monitoring point setting function of the present embodiment can also set a monitoring area based on the monitoring information acquired from each passenger car V. In this case, the monitoring point setting function sets the monitoring area based on the position information of the monitoring information without performing processing of setting a predetermined monitoring point from the monitoring information. That is, the monitoring area may be defined in association with the position information of each monitoring information, or may be defined in association with the monitoring point obtained from each monitoring information. The monitoring point setting function of the present embodiment repeats setting of the monitoring point at a predetermined cycle, and sets a monitoring point or a monitoring area according to the moved position of the tracking target.

The selection function of this embodiment selects a passenger car V capable of imaging a monitoring area which is sequentially set with reference to new monitoring information or a monitoring point. In this example, a passenger car V belonging to (exists) in the surveillance area is selected. Moreover, the selection function of the present embodiment selects a passenger car V capable of imaging a monitoring point. For example, the selection function selects the passenger car V capable of imaging the monitoring point based on the positional relationship (distance / direction) between the monitoring point and the current position of the passenger car V and the traveling direction (imaging direction) of the passenger car V .
As a result, even if a tracking target such as a suspect of the incident escapes, each monitoring area is imaged by the monitoring terminal devices 10 mounted on a plurality of different passenger cars V capable of imaging, and the tracking targets are cooperated in cooperation Since the monitoring information on the object is collected, the movement of the tracking object can be captured continuously.

The central control unit 21 includes a CPU, a ROM, and a RAM, and controls the image processing unit 22, the communication unit 23, and the display 24 to receive position information, image information, and time information transmitted from the monitoring terminal 10. The image processing is performed as necessary, and the image is displayed on the display 24.

The image processing device 22 has a map database, displays the map information from the map database on the display 24, and superimposes the position information detected by the position detection device 15 of the monitoring terminal device 10 on the map information. Do. Further, image processing is performed to display the image information captured by the on-vehicle camera 11 of the monitoring terminal device 10 and processed by the image processing device 12 on the display 24.

The display 24 can be configured by, for example, a liquid crystal display device of a size capable of displaying two window screens on one screen or two liquid crystal display devices each of which displays two window screens. Then, on one of the window screens, a screen in which the position information of each moving object V is superimposed on the map information is displayed (see FIG. 1), and on the other window screen, the image captured by the on-vehicle camera 11 is displayed. Display the relevant image information.

The input device 25 includes a keyboard or a mouse, and is used to input an information acquisition command to be output to a desired moving body V, or to input a processing command to various information displayed on the display 24. . In the above-described selection process of the monitoring terminal device 10, the monitoring point as the reference of the monitoring area area can also be input by the monitoring person via the input device 25. Although not particularly limited, the observer may designate a monitoring point by clicking (selecting and inputting) an icon of each point superimposed and displayed on the map information, and may set a monitoring area based on this monitoring point. it can.

The monitoring point in the present embodiment is a point at which a tracking target is predicted to exist. The setting method of this monitoring point is not limited to the above-mentioned method, and based on the report from the monitoring terminal device 10, the monitoring point predicted to have a tracking target may be set. For example, the central control device 21 acquires monitoring information including sighting information for reporting that the tracking target is output from the monitoring terminal device 10 and based on the position information of the monitoring terminal device 10 for which the sighting information is output. Thus, it is possible to set a monitoring point where the existence of the tracking target is predicted. This is because the passenger car V on which a person who witnesses the tracking target is located near the monitoring point, and the monitoring terminal device 10 is likely to be able to image the monitoring area. According to this method, it is possible to automatically set the monitoring point accurately without delaying the transmission of tracking information to be tracked.

Further, the central control device 21 sequentially acquires monitoring information including image information in a predetermined cycle, and the tracking target specified by the observer based on the image information in which the situation around the moving object is captured in time series. The movement direction can be calculated, and the monitoring point can be set on the side of the calculated movement direction. According to the image information obtained by imaging the surroundings of the moving object in a predetermined cycle, it is possible to determine in which direction the tracking target has moved (run away). The point can be set accurately.

According to the same viewpoint, the central control device 21 acquires monitoring information including image information over time in a predetermined cycle, and based on the image information in which the situation around the moving object is captured in time series, The movement speed of the identified tracking target can be calculated, and the monitoring point can be set according to the calculated movement speed. According to the image information obtained by imaging the surroundings of the moving object in a predetermined cycle, it is possible to determine whether the movement speed of the tracking object, for example, whether the tracking object escapes on foot or escapes using a vehicle, etc. It is possible to accurately set a monitoring point where a tracking target is predicted to exist at the monitoring timing of.

In addition, the setting method of the monitoring point which considered the moving speed can be utilized with the setting method of the monitoring point which considered the moving direction mentioned above. The monitoring point can be set more accurately by considering the moving direction and the moving direction. In the case where the first set monitoring point is a moving road such as a highway, a bridge or a one-way road, and the moving direction is limited, only the moving speed is considered. It can also be set. The road information for determining whether or not the moving direction is limited may be input by the supervisor or road information included in advance may be read out from the map information.

In addition, when time information is included in the monitoring information, it is possible to set the next monitoring timing (the timing when the tracking target exists at the monitoring point) based on the time information. Although not particularly limited, the monitoring terminal device 10 sets the monitoring timing after a predetermined time has elapsed from the time when the monitoring information capturing the tracking target is generated or acquired, and sets the monitoring point where the tracking target exists at the monitoring timing. can do.

The central control unit 21 sets a predetermined monitoring area based on the set monitoring point, and selects the monitoring terminal device 10 of the mobile V capable of imaging the monitoring area. The setting method of the monitoring area is not particularly limited, and an area having a predetermined distance from the monitoring point can be set as the monitoring area. As described above, the monitoring point can be set in consideration of the speed of the tracking target, but the speed of the tracking target may be considered when setting the monitoring area. Specifically, when the moving speed of the tracking target is high, the monitoring area can be set wide, and when the moving speed of the tracking target is low, the monitoring area can be set narrow. At this time, in consideration of the vehicle speed detected by the vehicle speed sensor 18 of the passenger car V on which the monitoring terminal device 10 is mounted, the speed to be tracked may be set as the relative speed with respect to the passenger car V on which the monitoring terminal device 10 is mounted.

The central control device 21 outputs an image transmission instruction to transmit monitoring information including image information to the monitoring terminal device 10 of the selected mobile unit V. In response to this, the on-vehicle control device 14 of the monitoring terminal device 10 sends out monitoring information including image information of the monitoring area. As described above, in the fixed camera, since the imaging area is limited, the tracking target is out of the imaging area If it does, it can not be imaged. On the other hand, even in the case where a camera mounted on a moving object captures an image of a tracking target, there is a problem that both the camera and the tracking target move randomly, so that it is not possible to continuously monitor a specific tracking target. . On the other hand, since the monitoring system 1 according to the present embodiment sends monitoring information including image information to the monitoring terminal device 10 capable of imaging the monitoring area based on the monitoring point where the tracking target is predicted to exist, random It is possible to effectively collect image information of a monitoring area where it is predicted that there will be tracking objects moving to the. That is, regardless of where the tracking target exists, the movable monitoring terminal device 10 can capture the tracking target.

The central control unit 21 can also send out an image transmission command informing of the feature to be tracked. Specifically, the central control device 21 has a feature extraction function of extracting a feature to be tracked from image information acquired in the past. In the present embodiment, the features of the tracking target include the color of the tracking target, the size of the tracking target, the moving speed of the tracking target, the number of tracking targets, the posture of the tracking target, and the like. Specifically, when the object to be tracked is a human, the color of the clothes, height, running means such as running away or running away by car, number of people, hairstyle, gender, etc. can be the features to be tracked. . By including such a feature in the image transmission command, it is possible to obtain information on the tracking target from the image information captured by the monitoring terminal device 10 in the past. For example, the central control device 21 outputs, to the selected monitoring terminal device 10, an image transmission command for transmitting monitoring information on a subject having the extracted feature. As a result, the occupant of the mobile unit V and the monitoring terminal device 10 can recognize a specific tracking target, and the monitoring accuracy can be improved. For example, when the feature of “red clothes, 190 cm tall male” is sent to a mobile V capable of capturing an image of the monitoring area (for example, the mobile V present in the monitoring area), the mobile V It can be expected that the crew members pay more attention to their surroundings and draw attention. As a result, it is possible to collect tracking information of sighting targets and prevent the tracking targets from being lost.

The in-vehicle control device 14 of the monitoring terminal device 10 having received the image transmission command refers to the data subjected to the feature analysis on the image information captured in the past, and searches for the subject having the feature included in the image transmission command. The on-vehicle control device 14 of the monitoring terminal device 10 includes the notification information indicating the presence of the tracking target in the monitoring information and outputs it when the image information of the subject having the feature to be tracked is retrieved. It is possible to quickly identify the location of the tracking target moving to the site, and to effectively collect the image information. As a result, even if the subject of tracking is not lost or the subject of tracking is lost, the footprint of the subject of tracking can be reliably traced by searching for a subject whose feature is the same as the target of tracking.

Furthermore, when the tracking target is a human being, the central control unit 21 can also send out an image transmission command notifying the feature of the face. Specifically, the central control unit 21 extracts a feature of a tracking target from image information acquired in the past, and a feature evaluation function of determining whether the tracking target is a human based on the feature. Prepare. Whether or not a person is a human can be determined using a method known at the time of application based on features such as the size, shape, and movements of limbs of a subject included in image information. When it is determined that the tracking target is a human, the central control device 21 selects an image transmission command for transmitting monitoring information including the image information of the tracking target face acquired from the image information, as the selected mobile V Output to the monitoring terminal device 10 of FIG. Also, known feature extraction techniques can be used to identify human tracking target face portions.

The in-vehicle control device 14 of the monitoring terminal device 10 having received the image transmission command refers to the image transmission command with reference to the data subjected to feature analysis for the purpose of similarity determination with respect to the face image included in the image information captured in the past. Search for subjects having features in common with the face of the image information included in. The in-vehicle control device 14 of the monitoring terminal device 10, when the image information of the subject similar to the face of the tracking target (the similarity is equal to or more than the predetermined value) is searched, Since the information is included in the monitoring information and output, it is possible to quickly find out the position of the randomly moving tracking target and effectively collect the image information. This makes it possible to notify each monitoring terminal device of the face of the person who is the object of tracking to the occupant of another mobile V. Therefore, even if the object of tracking is not lost or the object of tracking is lost, the features of the object are common to the object of tracking By searching for the subject to be tracked, it is possible to accurately trace the footprint of the tracking target.

Furthermore, the monitoring terminal device 10 wirelessly communicates with the monitoring terminal device 10 mounted on another moving object V when the image information of the tracking target face and the image information having a degree of similarity equal to or higher than a predetermined value are searched. It is possible to output image information retrieved through inter-vehicle communication, that is, image information of a face to be tracked. As a result, since it is possible to share information on the tracking target persona between the monitoring terminal device 10 capable of imaging the monitoring area and the monitoring terminal device 10 located in the vicinity thereof, the search for the tracking object is strengthened. Can.

In addition to the monitoring terminal device 10 mounted on the mobile unit V, the monitoring terminal device 10 described above is fixed to be mounted at a predetermined position and have an image generation function for capturing an image and generating image information. A type of surveillance terminal 10 may be included. Thus, the existing fixed camera can be effectively used, and image information from different viewpoints can be acquired. The position information of the fixed monitoring terminal device 10 can be stored in advance.

The communication device 23 is a communication means capable of wireless communication, and exchanges information with the communication device 13 of the monitoring terminal device 10 via the telecommunication network 30. When the telecommunication network 30 is a commercial telephone network, a mobile telephone communication device can be used for general purpose, and when the telecommunication network 30 is a dedicated telecommunication network of the monitoring system 1 of this example, it is

dedicated Communication devices

13 and 23 can be used.

Next, mounting positions and imaging ranges of the on-vehicle cameras 11a to 11e will be described. Here, a passenger car V will be described as an example of the moving body V. The cameras 11a to 11e are configured using imaging devices such as CCDs, and the four on-vehicle cameras 11a to 11d are respectively installed at different positions outside the passenger car V, and respectively capture four directions around the vehicle. One in-vehicle camera 11e is installed in the passenger compartment of the passenger car V and captures an image of the passenger compartment. The camera 1 of the present embodiment is provided with a zoom-up function for imaging a subject in an enlarged manner, and can arbitrarily change the focal length in accordance with the control command or can arbitrarily change the imaging magnification in accordance with the control command.

For example, as shown in FIG. 3, the on-vehicle camera 11a installed at a predetermined position in front of the passenger car V such as the front grille is an object or road surface existing in the space SP1 in front of the passenger car V Shoot the front view). An on-vehicle camera 11b installed at a predetermined position on the left side of the passenger car V, such as the left side mirror portion, is an object or road surface existing in the space SP2 on the left side of the passenger car V and its surrounding (left side view) To shoot. An on-vehicle camera 11c installed at a predetermined position in the rear portion of the passenger car V, such as a rear finisher portion or a roof spoiler portion, is an object or road surface present in the space SP3 in the rear portion of the passenger car V To shoot. An on-vehicle camera 11d installed at a predetermined position on the right side of the passenger car V, such as the right side mirror portion, is an object or road surface existing in the space SP4 on the right side of the passenger car V and the surrounding area (right side view) To shoot. Although not shown in FIG. 3, one on-vehicle camera 11e is installed, for example, on the ceiling of the passenger compartment, and captures an image of the indoor area SP5 as shown in FIG. Etc. for crime prevention or crime notification.

FIG. 4 is a view of the arrangement of the on-vehicle cameras 11a to 11e viewed from above the passenger car V. As shown in FIG. As shown in the figure, four of the on-vehicle camera 11a for capturing the area SP1, the on-vehicle camera 11b for capturing the area SP2, the on-vehicle camera 11c for capturing the area SP3, and the on-vehicle camera 11d for capturing the area SP4 are the passenger cars V It is installed along a left turn (counterclockwise) or a right turn (clockwise) along the outer periphery VE of the body. That is, when viewed along the outer periphery VE of the body of the passenger car V counterclockwise (counterclockwise) shown by the arrow C in the figure, the on-vehicle camera 11b is installed on the left next to the on-vehicle camera 11a. The on-vehicle camera 11c is installed on the left side of the on-vehicle camera 11c, and the on-vehicle camera 11a is installed on the left side of the on-vehicle camera 11d. Conversely, when viewed along the outer periphery VE of the body of the passenger car V in the opposite direction (clockwise) to the direction of the arrow C shown in the figure, the on-vehicle camera 11d is installed on the right of the on-vehicle camera 11a. An on-vehicle camera 11c is installed on the right, an on-vehicle camera 11b is installed on the right of the on-vehicle camera 11c, and an on-vehicle camera 11a is installed on the right of the on-vehicle camera 11b.

5A shows an example of an image GSP1 obtained by imaging the area SP1 by the front on-vehicle camera 11a, FIG. 5B shows an example of an image GSP2 obtained by imaging the area SP2 by the left-side on-vehicle camera 11b, and FIG. 5D shows an example of an image GSP4 in which the in-vehicle camera 11d in the right side images an area SP4. FIG. 5E shows an in-vehicle camera 11e in the room. It is an image figure which shows an example of image GSP5 which imaged indoor area SP5. Incidentally, the size of each image is 480 pixels long by 640 pixels wide. The image size is not particularly limited, as long as it can be reproduced by a general terminal device.

The number and position of the on-vehicle cameras 11 can be appropriately determined according to the size, shape, detection area setting method, and the like of the passenger car V. The plurality of on-vehicle cameras 11 described above are provided with identifiers according to their positions, and the on-vehicle control device 14 can identify each of the on-vehicle cameras 11 based on the respective identifiers. Further, the on-vehicle control device 14 can transmit an imaging instruction and other instructions to the specific on-vehicle camera 11 by attaching an identifier to the instruction signal.

The in-vehicle control device 14 controls the image processing device 12 to acquire an imaging signal captured by the in-vehicle camera 11, and the image processing device 12 processes the imaging signal from each in-vehicle camera 11 to obtain FIG. Convert to image information shown in 5E. Then, the on-vehicle control device 14 generates a monitoring image based on the four pieces of image information shown in FIGS. 5A to 5D (image generation function), and projects this monitoring image on the side surface of the projection model of the cylinder. Mapping information for projection onto a surface is associated with a monitoring image (mapping information addition function), and is output to the central monitoring device 20. Hereinafter, the image generation function and the mapping information addition function will be described in detail.

The monitoring image is generated based on the four image information obtained by imaging the surroundings of the passenger car V, and the processing of associating the mapping information with this is executed by the monitoring terminal device 10 as in this example, and by the central monitoring device 20 It can also be performed. In this case, four pieces of image information obtained by imaging the periphery of the passenger car V are transmitted as they are from the monitoring terminal device 10 to the central monitoring device 20, and are monitored by the image processing device 22 and the central control device 21 of the central monitoring device 20. The image may be generated, the mapping information may be associated, and projection conversion may be performed.

First, the image generation function will be described. The on-vehicle control device 14 of the monitoring terminal device 10 according to the present embodiment controls the image processing device 12 to obtain imaging signals of the on-vehicle cameras 11a to 11e, respectively. One monitoring image is generated such that the image information of the on-vehicle cameras 11a to 11d installed in the counterclockwise direction is arranged in the installation order of the on-vehicle cameras 11a to 11d.

As described above, in the present embodiment, the four on-vehicle cameras 11a to 11d are installed in the order of the

cameras

11a, 11b, 11c, and 11d counterclockwise (counterclockwise) along the outer periphery VE of the body of the passenger car V Therefore, the on-vehicle control device 14 integrates the four images captured by the on-vehicle cameras 11 a to 11 d according to the order of installation of the on-vehicle cameras 11 a to 11 d (in-vehicle cameras 11 a → 11 b → 11 c → 11 d). Connect horizontally to generate a single surveillance image. In the monitoring image of the present embodiment, the images are arranged such that the ground contact surface (road surface) of the passenger car V is the lower side, and the images are connected to each other at the side in the height direction (vertical direction).

FIG. 6 is a diagram showing an example of the monitoring image K. As shown in the figure, the monitoring image K of this embodiment is a captured image GSP1 obtained by imaging the area SP1 by the front on-vehicle camera 11a along the direction P from the left to the right in the drawing, and the on-vehicle camera 11b on the left side. The captured image GSP2 captured the area SP2, the captured image GSP3 captured the area SP3 by the rear on-vehicle camera 11c, and the captured image GSP4 captured the area SP4 by the on-vehicle camera 11d on the right side are arranged in this order in this order. These four images are arranged as a series of images. By displaying the monitoring image K generated in this way, from the left end to the right side in order, with the image corresponding to the road surface (the ground contact surface of the vehicle) down, the observer turns counterclockwise around the vehicle V It can be viewed on the display 24 in the same way as looking around.

In addition, when generating one monitoring image K, four images acquired at substantially the same time as the imaging timings of the on-vehicle cameras 11a to 11d are used. Thus, the information included in the monitoring image K can be synchronized, so that the situation around the vehicle at a predetermined timing can be accurately represented.

Further, the monitoring image K generated from each captured image at which the imaging timing of the camera is substantially simultaneous is stored over time, and a monitoring image K of a moving image including a plurality of monitoring images K is generated per predetermined unit time. You may By generating the monitoring image K of the moving image on the basis of the images at the same imaging timing, it is possible to accurately express the change of the situation around the vehicle.

By the way, when the image of each imaging area is stored over time and the monitoring image K of the moving image generated for each imaging area is transmitted to the central monitoring device 20, depending on the function of the central monitoring device 20, a plurality of In some cases, you can not play the video simultaneously. In such a conventional central monitoring device 20, a plurality of moving pictures can not be reproduced and displayed at the same time. Therefore, when reproducing each moving picture, it is necessary to switch screens and reproduce moving pictures one by one. That is, in the conventional central monitoring device 20, it is not possible to simultaneously view videos (moving images) in a plurality of directions, and there is a disadvantage that it is not possible to monitor the entire periphery of the vehicle on one screen.

On the other hand, since the on-vehicle control device 14 of the present embodiment generates one monitoring image K from a plurality of images, it is possible to simultaneously reproduce moving images of different imaging directions regardless of the function of the central monitoring device 20. it can. That is, by continuously reproducing the monitoring image K (moving image reproduction), the four images included in the monitoring image K are simultaneously and continuously reproduced (moving image reproduction), and the state change of the area different in direction is displayed on one screen Can be monitored.

Further, the monitoring terminal device 10 of the present embodiment generates a monitoring image K by compressing the data amount of the image so that the number of pixels of the monitoring image K becomes substantially the same as the number of pixels of the images of the onboard cameras 11a to 11d. You can also While the size of each image shown in FIGS. 5A to 5E is 480 × 640 pixels, in the present embodiment, compression processing is performed so that the size of the monitoring image K is 1280 × 240 pixels as shown in FIG. Do. As a result, the size (1280 × 240 = 307,200 pixels) of the monitoring image K becomes equal to the size (480 × 640 × 4 sheets = 307,200 pixels) of each image, so that the central monitoring that received the monitoring image K Image processing and image reproduction can be performed regardless of the function of the apparatus 20 side.

Furthermore, the in-vehicle control device 14 according to the present embodiment can also add a line figure indicating the boundaries of the arranged images to the monitoring image K. Taking the monitoring image K shown in FIG. 6 as an example, the on-vehicle control device 14 takes a rectangular partition image Bb, Bc, Bd, Ba, Ba 'between the respective images as a line figure indicating the boundaries of the arranged images. Can be attached to the surveillance image K. As described above, by arranging the partition images at the boundaries of the four images, it is possible to separately recognize each of the images having different imaging directions in the series of monitoring images K. That is, the partition image functions as a frame of each captured image. In addition, since the distortion of the image is large near the boundary of each captured image, arranging the partition image at the boundary of the captured image can suggest that the image in the region with large distortion is hidden or that the distortion is large. .

The on-vehicle control device 14 according to the present embodiment can also generate the monitoring image K after correcting distortion when four images are projected on the projection plane set on the side surface of the projection model described later. . The peripheral region of the captured image is prone to distortion of the image, and in the case of the on-vehicle camera 11 using a wide-angle lens in particular, the distortion of the captured image tends to be large. It is desirable to correct the distortion of the captured image using the defined image conversion algorithm and the correction amount.

Although not particularly limited, as shown in FIG. 7, the on-vehicle control device 14 reads information of the same projection model as that of the projection model on which the monitoring image K is projected in the central monitoring device 20 from the ROM, and captures the image on the projection plane of this projection model. Images can also be projected and pre-corrected for distortions that occur in the projection plane. The image conversion algorithm and the correction amount can be appropriately defined according to the characteristics of the on-vehicle camera 11 and the shape of the projection model. As described above, by correcting distortion in the case where the image K is projected on the projection plane of the projection model in advance, it is possible to provide a highly visible monitoring image K with less distortion. In addition, by correcting distortion in advance, it is possible to reduce positional deviation between the images arranged side by side.

Next, the mapping information addition function will be described. In the monitoring terminal device 10 of the present embodiment, the on-vehicle control device 14 projects the generated monitoring image K on the projection plane set on the side surface of the projection model M of a column whose bottom surface is the ground plane of the passenger car V. A process of associating the mapping information for the image with the monitoring image K is performed. The mapping information is information for making the central monitoring device 20 that has received the monitoring image K easily recognize the projection reference position. FIG. 8 is a view showing an example of a projection model M according to this embodiment, and FIG. 9 is a schematic cross-sectional view along the xy plane of the projection model M shown in FIG.

As shown in FIGS. 8 and 9, the projection model M of the present embodiment is a regular octagonal prism whose base is a regular octagon and has a height along the vertical direction (z-axis direction in the figures). The shape of the projection model M is not particularly limited as long as it is a cylindrical body having side surfaces adjacent along the boundary of the bottom, and a cylindrical body, or a rectangular cylindrical body such as a triangular cylindrical body, a rectangular cylindrical body, or a hexagonal cylindrical body It can also be an anti-prism body whose bottom is a polygon and whose side is a triangle.

Moreover, as shown to the same figure, the bottom face of the projection model M of this embodiment is parallel to the ground plane of the passenger car V. In addition, on the inner side surface of the side surface of the projection model M, projection surfaces Sa, Sb, Sc, Sd (hereinafter collectively referred to as projection surface S) that project an image around the passenger car V grounded on the bottom surface of the projection model M. It is set. The projection surface S is a portion of the projection surface Sa and a portion of the projection surface Sb, a portion of the projection surface Sb and a portion of the projection surface Sc, a portion of the projection surface Sc and a portion of the projection surface Sd, and the projection surface Sd It can also be constituted by a part of and a part of projection plane Sa. The monitoring image K is projected on the projection surface S as a video of the passenger car V over the viewpoint R (R1 to R8, hereinafter collectively referred to as a viewpoint R) above the projection model M surrounding the passenger car V.

The in-vehicle control device 14 according to the present embodiment associates the reference coordinates of the captured image disposed at the right end or the left end with the monitoring image K as mapping information. Taking the monitoring image K shown in FIG. 6 as an example, the on-vehicle controller 14 is disposed at the right end as mapping information (reference coordinates) indicating the start position or the end position of the monitoring image K when projected onto the projection model M. Coordinates A (x, y) of the upper left vertex of the captured image GSP1 and coordinates B (x, y) of the upper right vertex of the captured image GSP2 arranged at the left end are attached to the monitoring image K. The reference coordinates of the captured image indicating the start position or the end position are not particularly limited, and may be the lower left vertex of the monitoring image K disposed at the left end or the lower right vertex of the monitoring image K disposed at the right end. The mapping information may be attached to each pixel of the image data of the monitoring image K, or may be managed as a separate file from the monitoring image K.

As described above, the central monitoring device 20 that has received the monitoring image K by associating the monitoring image K with the information indicating the start position or the end position of the monitoring image K, that is, the reference coordinates used as a reference in projection processing as mapping information. Since the reference position at the time of projection processing can be easily recognized, the monitoring images K arranged in the arrangement order of the on-vehicle cameras 11a to 11d are sequentially and easily projected on the projection plane S on the side surface of the projection model M be able to. That is, as shown in FIG. 9, the captured image GSP1 ahead of the vehicle is projected on the projection surface Sa located in the imaging direction of the on-vehicle camera 11a, and the captured image on the right side of the vehicle on the projection surface Sb located in the imaging direction GSP2 is projected, the captured image GSP3 behind the vehicle is projected on the projection surface Sc located in the imaging direction of the on-vehicle camera 11c, and the captured image GSP4 in the left side of the vehicle is projected on the projection surface Sd located in the imaging direction of the on-vehicle camera 11d can do.

Thereby, the surveillance image K projected on the projection model M can show an image that can be seen as if the surroundings of the passenger car V are looked around. That is, since the surveillance image K including four images arranged in a row in the horizontal direction according to the installation order of the on-vehicle cameras 11a to 11d is projected on the side in the same horizontal direction in the column of the projection model M, In the surveillance image K projected on the projection plane S of the projection model M of the column, the image around the passenger car V can be reproduced while maintaining its positional relationship.

Note that the on-vehicle control device 14 according to the present embodiment stores the correspondence relationship between each coordinate value of the monitoring image K and the coordinate value of each projection surface S of the projection model M as mapping information and adds it to the monitoring image K. Although it can be, it may be stored in advance in the central monitoring device 20.

Further, the positions of the viewpoint R and the projection plane S shown in FIGS. 8 and 9 are merely examples, and can be set arbitrarily. In particular, the viewpoint R can be changed by the operation of the operator. The relationship between the viewpoint R and the projection position of the surveillance image K is defined in advance, and when the position of the viewpoint R is changed, it is viewed from the newly set viewpoint R by executing predetermined coordinate conversion. The surveillance image K can be projected onto the projection plane S (Sa to Sd). A known method can be used for this viewpoint conversion process.

As described above, the on-vehicle control device 14 of the present embodiment generates the monitoring image K based on the image information captured at a predetermined timing, and the monitoring image K shows mapping information, reference coordinates, and a line figure The information of the partition image is associated and stored temporally according to the imaging timing. Although not particularly limited, the on-vehicle control device 14 may store the monitoring image K as one moving image file including a plurality of monitoring images K per predetermined unit time, or a mode in which transfer / reproduction can be performed by streaming method Monitoring image K may be stored.

The communication device 23 of the central monitoring device 20 receives the monitoring image K transmitted from the monitoring terminal device 10 and the mapping information associated with the monitoring image K. Further, the image information taken by the in-vehicle camera 11e in the room is separately received. In this surveillance image K, images of the four on-vehicle cameras 11 installed at different positions of the body of the passenger car V as described above are installed along the clockwise or counterclockwise direction along the outer periphery of the body of the passenger car V The in-vehicle cameras 11a to 11d are arranged according to the installation order (the clockwise or counterclockwise order along the outer periphery of the body of the vehicle V). Further, mapping information for causing the monitoring image K to be projected on the projection surface S of the octagonal prism projection model M is associated with the monitoring image K. The communication device 23 transmits the acquired monitoring image K and the mapping information to the image processing device 22.

The image processing device 22 reads out the projection model M stored in advance, and sets it on the side surface of the octagonal cylinder projection model M whose bottom surface is the contact surface of the passenger car V shown in FIGS. 8 and 9 based on the mapping information. A display image is generated by projecting the monitoring image K on the projection planes Sa to Sd. Specifically, each pixel of the received monitoring image K is projected on each pixel of the projection planes Sa to Sd according to the mapping information. Further, when projecting the surveillance image K onto the projection model M, the image processing device 22 recognizes the start point (right end or left end of the surveillance image K) of the surveillance image K based on the reference coordinates received together with the surveillance image K. Then, the projection processing is performed so that the start point coincides with the start point (the right end or the left end of the projection surface S) defined in advance on the projection model M. Further, when projecting the monitoring image K onto the projection model M, the image processing device 22 arranges a line figure (partition image) indicating the boundary of each image on the projection model M. The partition image may be attached to the projection model M in advance, or may be attached to the monitoring image K after the projection processing.

The display 24 displays the monitoring image K projected on the projection plane S of the projection model M. FIG. 10 shows an example of a display image of the monitoring image K. In addition, by using the input device 25 such as a mouse or a keyboard or the display 24 as the touch panel input device 25, the viewpoint can be freely set and changed by the operation of the supervisor. Since the correspondence between the viewpoint position and the projection surface S is previously defined in the image processing device 22 or the display 24 described above, the monitor image K corresponding to the changed viewpoint is displayed on the display 24 based on the correspondence. can do.

Next, the operation of the monitoring system 1 according to the present embodiment will be described. FIG. 11 is a flowchart showing the operation of the monitoring terminal device 10, FIGS. 12A and 12B are flowcharts showing the operation of the central monitoring device 20, and FIG. 13 is a diagram showing an example of database information.

As shown in FIG. 11, in the monitoring terminal device 10, a surrounding image and an indoor image are acquired from the in-vehicle camera 11 at predetermined time intervals (one routine shown in the same drawing), and the image processing device 12 converts it into image information. Convert (step ST1). Further, the current position information of the passenger car V on which the monitoring terminal device 10 is mounted is detected from the position detection device 15 provided with the GPS (step ST2). The position detection device 15 can also be configured as part of the navigation device 19.

In step ST3, it is determined whether or not the report button 16 for reporting an abnormality is pressed. If the report button 16 is pressed, the process proceeds to step ST4, and the image information obtained in step ST1 and the information obtained in step ST2 The position information and the time information of the CPU are associated, and these are transmitted as monitoring information to the central monitoring device 20 via the communication device 13 and the telecommunication network 30 together with the abnormal information indicating that an abnormality has occurred. As a result, since the occurrence of abnormalities related to security, such as accidents and crimes, is automatically transmitted to the central monitoring device 20 together with the position information of the passenger car V and the image information around the passenger car V, surveillance in the city is further strengthened. It will be done. In the present example, the image information and the position information are acquired in the first steps ST1 and ST2, but the image information and the position information may be acquired at the timing between the steps ST3 and ST4.

Returning to step ST3, when the notification button 16 is not pressed, the process proceeds to step ST5, and communicates with the central monitoring device 20 to acquire a control command.

Subsequently, in step ST6, the monitoring terminal device 10 determines whether or not an image transmission instruction has been acquired from the central monitoring device 20. If an image transmission instruction has been acquired, the process proceeds to step ST7 and image information and position information are obtained. , Monitoring information including time information to the central monitoring device 20. The monitoring terminal device 10 includes monitoring information including notification information notifying that the tracking target exists when the image transmission command includes the feature of the tracking target, and the image information of the subject having this feature is searched. Can be sent. Further, in the monitoring terminal device 10, when the image transmission command includes the image information of the face to be tracked, and the image information of the subject of the face similar to this face (the similarity is equal to or more than a predetermined value) is searched. , May transmit monitoring information including notification information notifying that a tracking target exists. In addition, when the storage instruction is included in the image transmission instruction, the image information, the position information and the time information are stored.

Returning to step ST6, even if the image transmission command is not acquired from the central monitoring device 20, if the passenger car V exists in the key monitoring area defined in advance in step ST8, the process proceeds to step ST10 and the image information Send including monitoring information. On the other hand, when the image transmission command is not obtained and it is not the key monitoring area, the process proceeds to step ST9, and monitoring information not including the image information, that is, time information and position information is transmitted to the central monitoring device 20.

Subsequently, the operation of the central monitoring device 20 will be described based on FIGS. 12A and 12B. In step ST11 of FIG. 12A, position information and time information are acquired from all the passenger cars V, and are stored at least temporarily in the database 26. FIG. 13 is a diagram showing an example of information stored in the database 26. As shown in FIG. As shown in FIG. 13, monitoring information including image information, position information, time information, tracking target features, and tracking target face features acquired from passenger car V (monitoring terminal device 10) is associated with the position information. Are stored. That is, when position information is specified, a series of monitoring information can be called. Further, this monitoring information can include a mobile unit ID (monitoring terminal device ID) for identifying the monitoring terminal device 10. The mobile unit ID may be the address of the communication device 13 of the monitoring terminal device 10.

In step ST12, the passenger car V is displayed on the map information of the map database displayed on the display 24 as shown in the upper left of FIG. 1 based on the position information acquired in step ST11. The position information of the passenger car V is acquired and transmitted at a predetermined timing for each routine in FIG. 11, so that the supervisor can grasp the current position of the passenger car V in a timely manner.

In step ST13, whether or not abnormality information notified from the monitoring terminal device 10 of the passenger car V, that is, a notification that an abnormality related to security such as an accident or a crime has occurred has been received, or notification from the monitoring terminal device 10 of the passenger car V It is determined whether the received sighting information, that is, the report that the sighting object has been witnessed has been received. The abnormality information or sighting information is output when the passenger of the passenger car V presses the notification button 16 of the monitoring terminal device 10.

If there is abnormality information or sighting information, the passenger car V for which the abnormality information is output is specified in step ST14, image information and time information are received from the monitoring terminal device 10 of the passenger car, and the image information is displayed on the display 24 Do. Further, as shown in the upper left of FIG. 1, highlighting is performed such as changing a color so that the passenger car displayed on the map information can be distinguished from other passenger cars. Thereby, while being able to visually recognize on a map information the position where abnormality generate | occur | produced or the report of a witness was visible, it is possible to grasp the abnormal content or the traced object witnessed on the display 24. The processes in steps ST13 to 20 are processes in the case where abnormality information or sighting information is reported as an example, and the existing position of the passenger car V who has reported the abnormality information or sighting information is selected as a monitoring point However, even if abnormal information or sighting information is not notified, and the observer arbitrarily designates a place to be monitored (monitoring point), the processing from step ST13 to step 20 can be executed similarly. In this case, the place designated by the supervisor becomes the surveillance point.

In step ST15, the central monitoring device 20 sets a monitoring point where it is predicted that there is a tracking target to be monitored paying attention to the position where the passenger car V which has output the abnormality information is present. There is a high possibility that there is a tracking target at the point where the sighting information was reported, and there is a high possibility that there is a person who caused the abnormality or a person involved in the abnormality at the point where the anomaly information was reported. It is. In this example, although it is a position of passenger car V which reported unusual information or sighting information, a supervisor can also set up arbitrarily.

Further, in step ST16, the central monitoring device 20 selects, based on the monitoring point, another vehicle capable of imaging the monitoring area set on the basis of the monitoring point, that is, the monitoring terminal device 10. Although it is an example, central monitoring device 20 chooses other vehicles, ie, monitoring terminal device 10, which are present in the monitoring area within a predetermined distance from the monitoring point. The central monitoring device 20 selects another vehicle capable of imaging the monitoring point, that is, the monitoring terminal device 10. Although it is one example, the central monitoring device 20 is another vehicle existing within a predetermined distance from the monitoring point, and another vehicle existing in a predetermined direction with respect to the monitoring point (other vehicles whose imaging direction is directed to the monitoring point) Choose
As described above, the monitoring area may be a circular area of the same distance from the monitoring point, or may be a band area of a predetermined distance along the road including the monitoring point, or when a right turn or left turn is considered at an intersection. A may be a fan-shaped area of a predetermined distance and a predetermined central angle. The monitoring area may be a circular area of the same distance from the position information included in the monitoring information, or may be a band area of a predetermined distance along the road including the monitoring point, or a right turn or left turn at an intersection etc. A may be a fan-shaped area of a predetermined distance and a predetermined central angle. The central monitoring device 20 can also select the monitoring terminal device 10 of another vehicle approaching the monitoring area that can enter the monitoring area within a predetermined time. This is because other vehicles approaching the monitored area can image the monitored area after a predetermined time has elapsed, even if not currently present in the monitored area.

In the present embodiment, not only the passenger car V approaching the monitoring area but also the monitoring area is separated in consideration of the possibility that the tracking target is lost and the selection of the monitoring point may be delayed with respect to the movement of the tracking target. The passenger car V is also targeted for selection. The passenger car V separated from the surveillance area may have imaged the surveillance area in the past. The central monitoring device 20 preferably transmits an image transmission command specifying the imaging time of the image information to the passenger car V separated from the monitoring area. Thereby, even if the monitoring area is set after the time has passed since the sighting information was reported or after the time has elapsed since the occurrence of the incident, the reporting or incident occurrence of the sighting information According to the image transmission command specifying the imaging time before and after the time, it is possible to retroactively collect the report of sighting information or the image information before and after the occurrence of the incident.

The method of selecting the passenger car V capable of imaging the surveillance area is not particularly limited. For example, first, the passenger car who reported abnormality, the passenger car who sent sighting information, or the current position of the passenger car arbitrarily selected by the supervisor The road where the point (Y) exists is identified, and the passenger car V traveling on the road is extracted with reference to the database 26. Then, the specified position (X) of the passenger car V, the moving speed (V), and the traveling direction are specified. The moving speed and the traveling direction of the passenger car V may be obtained based on the temporal change of the position information, or may be obtained based on the moving speed acquired by being included in the monitoring information. Subsequently, the central monitoring device 20 selects the passenger car V whose (Y−X) / V is less than a predetermined value as the traveling direction of the passenger car V approaches the monitoring point. If this (Y−X) / V is too small, the monitoring point is immediately passed, so a lower limit may be set. Then, in the same step, the central monitoring device 20 transmits an image transmission command to the selected monitoring terminal device 10. In addition, since the monitoring point and the monitoring area set based on this are updated at a predetermined cycle and change from moment to moment as the tracking target moves, a passenger car capable of imaging the monitoring area even if the tracking target moves randomly It can be selected.

The image transmission instruction can include information for specifying an imaging direction. The central monitoring device 20 calculates the imaging direction based on the positional relationship between the monitoring point and the monitoring area. The imaging direction may be expressed by an azimuth, or may be expressed by identification information of the on-vehicle camera 11 if the position of the on-vehicle camera 11 is known. Thereby, the video of the surveillance point can be reliably acquired by the camera 11 of the passenger car V in the surveillance area. Moreover, since only necessary image information can be transmitted, the amount of transmission data can be reduced.

In addition, when the monitoring terminal device 10 is provided with the navigation device, the monitoring including the image information is automatically performed at the timing when the own vehicle intrudes into the monitoring area from the monitoring point transmitted by the central monitoring device 20 and the current position. Information can also be sent to central monitoring device 20.

At step ST17, the position information of the passenger car V which has output the abnormality information is transmitted to an emergency passenger car such as a police car, an ambulance, a fire engine or the like. In this case, image information may be attached and transmitted in order to notify of abnormal content. As a result, it is possible to dispatch an emergency passenger car before a notification from the field comes in, and it is possible to promptly cope with an accident or a crime.

At step ST18, all position information, image information and time information received from the monitoring terminal device 10 are recorded on the recording medium. This record is used to resolve them after an accident or crime. If there is no abnormality information in step ST13, the process proceeds to step ST21 without performing the processing in steps ST14 to ST18.

In step ST19, it is determined whether or not the tracking monitoring state of the tracking target has been released, and if the tracking monitoring state has been released, the processing from step 21 is performed. On the other hand, if it has not been released, in order to continue monitoring the monitoring point, if the passenger car V selected in the previous step ST16 passes the monitoring area or the monitoring area can not be imaged, the step Returning to ST16, a passenger car V capable of imaging a monitoring area anew is selected.

Although a method for selecting a passenger car V for imaging a surveillance area is not particularly limited, the same method as the process of step ST16 can be used.

As described above, as long as the tracking target of the tracking target is continued, the monitoring passenger car V capable of imaging the monitoring area is successively selected, so that the camera 11 is mounted even if the tracking target moves randomly. Even if the passenger car V moves randomly, it is possible to continuously track and image a specific tracking target and obtain monitoring information on the tracking target.

After the tracking monitoring is canceled, step 21 and subsequent steps shown in FIG. 12B can be performed. In step ST21, it is determined whether there is an instruction to transmit image information from an emergency passenger car such as a police car, an ambulance or a fire engine. If an image transmission instruction is input, the process proceeds to step ST22. In step ST22, it is determined whether the passenger car V exists in the area specified by the transmission instruction of the image information. If the passenger car V exists, the process proceeds to step ST23. Then, in step ST23, a transmission instruction of the image information is output to the passenger car V present in the area specified by the transmission instruction of the image information. Thereby, the image information from the passenger car V can be acquired in step ST11 of FIG. 12A of the next routine, and this is transferred to the emergency passenger car or the meaning of the transmission command from the emergency passenger car is grasped. be able to. If the process does not correspond to steps ST21 and ST22, the process proceeds to step ST24 without performing the process of steps ST21 to ST23.

In step ST24, it is determined whether a passenger car V exists in the vicinity of a suspicious area such as a crime prone area set in advance, and if it exists, the process proceeds to step ST25 to transmit image information to the passenger car V Output a command. Suspicious places are streets and streets that are insecure. This makes it possible to strengthen surveillance of streets and streets that are suspicious places, and can be expected to prevent crime in advance. When the passenger car V does not exist in the vicinity area of the suspicious part, the process proceeds to step ST26 without performing the process of step ST22.

In step ST26, it is determined whether the passenger car V exists in the vicinity of the important point monitoring position where it is possible to image the important point monitoring target whose details should be monitored. If the passenger car V exists in the vicinity of the important point monitoring position Then, it outputs a priority monitoring command requesting transmission of image information to which the focus monitoring target is expanded for the passenger car V. As a result, it is possible to monitor the focus monitoring target in detail, to effectively detect a suspicious object that causes an event or an accident in the specified focus monitoring target, and it is possible to expect crime prevention. When the passenger car V does not exist in the vicinity of the important point monitoring position, the process proceeds to step ST28 without performing the process of step ST27.

In step ST28, the passenger car V does not travel within a predetermined time within a predetermined area (not limited to the suspicious spot and the key monitoring area) where monitoring is required based on the position information received from each passenger car V It is determined whether or not there is a route, and when there is such a route, it is monitored whether there is a passenger car V traveling on the route. Then, if there is a passenger car V traveling the route most recently, the process proceeds to step ST29, and a transmission instruction of image information is output to the passenger car V. As a result, it is possible to automatically acquire image information of a route other than a suspicious spot or a key monitoring area and in which the passenger car V has a small traffic volume. If there is no route that satisfies the condition of step ST28, the process returns to step ST11 of FIG. 12A without performing the process of step ST29.

As described above, the monitoring system of the present embodiment has the following effects.
(1) The monitoring system 1 of this example selects the monitoring terminal device 10 of a mobile capable of imaging a monitoring area based on the monitoring point where the tracking target is predicted to exist, and selects the selected monitoring terminal device 10 A monitoring terminal device 10 mounted on a mobile capable of imaging a monitoring area based on a monitoring point predicted to have a tracking target by outputting an image transmission command for transmitting monitoring information including image information. Since it is possible to obtain an image of the tracking target from, it is possible to continuously monitor the moving tracking target.
By selecting the monitoring terminal device 10 present in the monitoring area based on the monitoring point as the “monitoring terminal device 10 capable of capturing an image of the monitoring area”, it is possible to efficiently collect a captured image of the tracking target.
Since the imaging area of a general fixed security camera device is fixed, when the movable tracking target such as a human or a vehicle moves and leaves the imaging area of the fixed security camera device 10, the tracking target is continued And there is a problem that it can not be monitored. On the other hand, in the case of using a camera mounted on the moving body V, there is a problem that it is not possible to continuously monitor a specific tracking target because both the camera and the tracking target move randomly. On the other hand, according to the monitoring system 1 of the present embodiment, the monitoring terminal device 10 uses the monitoring terminal device 10 mounted on the moving object V which moves randomly while using the camera 11 mounted on the moving object V. Thus, it is possible to continuously monitor a tracking target which moves randomly.

(2) In the surveillance system 1 of this embodiment, the passenger car V on which the person witnessed the tracking object is located in the vicinity of the surveillance point, and there is a high possibility that the surveillance terminal 10 can image the surveillance area From the viewpoint, when the monitoring information including the sighting information notifying that the tracking target has been witnessed is acquired from the monitoring terminal device 10, the tracking target is obtained based on the position information of the monitoring terminal device 10 to which the sighting information is output. By setting a monitoring point whose presence is predicted, it is possible to automatically set a monitoring point and a monitoring area based thereon without delaying the transmission of tracking information to be tracked.

(3) In the monitoring system 1 of this example, the movement direction of the tracking target is calculated based on the image information acquired over time, and the monitoring point is set on the calculated movement direction side. It is possible to appropriately set a monitoring point where an object is predicted to exist and a monitoring area based thereon.

(4) In the monitoring system 1 of the present example, the moving speed of the tracking target is calculated based on the image information acquired over time, and the monitoring point according to the calculated moving speed is set. Even when the vehicle escapes or the vehicle escapes, the monitoring point and the monitoring area can be appropriately set.

(5) In the monitoring system 1 of this example, since the image transmission command notifying the feature of the tracking object is sent to the monitoring terminal device 10, the occupant of the mobile V and the monitoring terminal device 10 recognize the specific tracking object Monitoring accuracy can be improved.

(6) In the monitoring system 1 of this example, when the monitoring terminal device 1 can search for the image information of the subject having the feature of the tracking target, the monitoring information includes notification information notifying the existence of the tracking target. Since the output is performed, the position of the randomly moving tracking target can be quickly identified, and the image information can be effectively collected. As a result, even if the subject of tracking is not lost or the subject of tracking is lost, it is possible to accurately trace the footprint of the target of tracking by searching for a subject whose feature is the same as that of the target of tracking.

(7) In the monitoring system 1 of this example, when the tracking target is a human, an image transmission command notifying the feature of the face is sent out, so the occupant of the mobile V and the monitoring terminal device 10 perform specific tracking. The target face can be recognized, and the monitoring accuracy can be improved.

(8) In the monitoring system 1 of this example, when the image information of the subject similar to the face to be tracked (the similarity is equal to or more than the predetermined value) is retrieved from the image information captured in the past. Since the notification information notifying the existence of the tracking target is included in the monitoring information and output, the existence position of the tracking target moving randomly can be identified quickly, and the image information can be collected effectively. As a result, even if the subject of tracking is not lost or the subject of tracking is lost, it is possible to accurately trace the footprint of the target of tracking by searching for a subject whose feature is the same as that of the target of tracking.

(9) In the monitoring system 1 of this example, when the image information of the face to be tracked and the image information having the similarity equal to or more than the predetermined value are searched, the monitoring terminal device 10 mounted on another mobile V is Since the image information of the face to be tracked is output via wireless communication (inter-vehicle communication), the person to be tracked is between the monitoring terminal device 10 capable of imaging the monitoring area and the monitoring terminal device 10 located in the vicinity thereof. Since the phase information can be shared, it is possible to strengthen the search for the tracking target.
Ru.

(10) In the monitoring system 1 of the present embodiment, the monitoring terminal device 10 is attached to a predetermined position in addition to the monitoring terminal device 10 mounted on the mobile unit V, and images the surroundings to generate image information The existing fixed camera can be effectively used by including the fixed type monitoring terminal device 10 having the image generation function.

(11) The monitoring system 1 of this example selects the monitoring terminal device 10 of a mobile capable of imaging the monitoring point where the tracking target is predicted to be present, and includes the image information for the selected monitoring terminal device 10 By outputting an image transmission command for transmitting monitoring information, it is possible to acquire an image of the tracking object from the monitoring terminal device 10 mounted on a mobile body capable of imaging the monitoring point where the tracking object is predicted to be present. The subject can be monitored continuously.

(12) The monitoring system 1 of this example selects the monitoring terminal device 10 of a mobile capable of capturing an image of the monitoring area where the tracking target is expected to be present, and includes image information for the selected monitoring terminal device 10 By outputting an image transmission command for transmitting monitoring information, it is possible to acquire an image of the tracking target from the monitoring terminal device 10 mounted on a mobile body capable of imaging the monitoring area where the tracking target is predicted to be present. The subject can be monitored continuously.

(13) The monitoring method of this example has the same operation and effect as the monitoring system including the monitoring terminal device 10 and the central monitoring device 20.

In the embodiment described above, the position information of the passenger car V and the image information from the on-vehicle cameras 11a to 11e are acquired, but it is combined with the image information from the fixed camera 11f installed in the city shown in FIG. May be acquired. In addition, as for passenger car V which acquires position information and picture information, as shown in Figure 1, it is desirable to use taxi V1 and the bus which run the territory which is decided beforehand, but even if private passenger car V2 and emergency passenger car V3 are used Good.

In the above-described embodiment, five in-vehicle cameras are mounted on the passenger car V, and an image of 360 ° around is acquired as image information using four in-vehicle cameras 11a to 11d. It may be omitted. Further, particularly in an environment where image information can be acquired from many passenger cars V, such as an area where traffic is heavy, the four on-vehicle cameras 11a to 11d may be three or less.

The passenger car V corresponds to a moving body according to the present invention, the position detection device 15 corresponds to a position detection means according to the present invention, and the on-vehicle camera 11 and the image processing device 12 correspond to an image generation means according to the present invention The in-vehicle control device 14 corresponds to the image search means, the storage means, and the output means according to the present invention, the CPU of the in-vehicle control device 14 corresponds to the time detection means according to the present invention, and the notification button 16 corresponds to the present invention. The communication device 13 corresponds to command receiving means and information output means according to the present invention. The central control unit 21 corresponds to monitoring point setting means, selection means, feature extraction means, the database 26 corresponds to a database, and the communication device 23 and the input device 25 receive information acquisition means according to the present invention, abnormality information acceptance. The display 24 corresponds to display means according to the present invention.

DESCRIPTION OF SYMBOLS 1 ... Vehicle monitoring system 10 ... Monitoring

terminal device

11, 11a-11e ... Vehicle-mounted camera 11f ... Fixed camera in the town 12 ... Image processing device 13 ... Communication device 14 ... Vehicle-mounted control device 15 ... Position detection device 16 ... Report button 17 ... Vehicle controller 18 Vehicle speed sensor 19 Navigation device 20 Central monitoring device 21 Central control device 22 Image processing device 23 Communication device 24 Display 25 Input device 30 Telecommunication network V, V1, V2, V3 Mobile body M: Projection model S, Sa, Sb, Sc, Sd: Projection surface R1 to R8: Perspective

Claims

Monitoring comprising: position detecting means for detecting positional information of each of a plurality of moving bodies, and image generating means mounted on each of the plurality of moving bodies and imaging the periphery of the moving bodies to generate image information A monitoring system comprising a central monitoring device for obtaining monitoring information from a terminal device via wireless communication, comprising:
The central monitoring device
Information acquisition means for acquiring monitoring information including at least position information detected by the position detection means of the monitoring terminal device and time information detected by the time detection means;
Monitoring point setting means for setting a monitoring point where a tracking target specified by the observer is predicted to exist, based on the acquired monitoring information;
Selecting means for selecting a monitoring terminal apparatus of a mobile capable of imaging a predetermined monitoring area based on the set monitoring point;
And a command output unit configured to output an image transmission command for transmitting monitoring information including the image information to the monitoring terminal apparatus of the selected mobile object.
The monitoring information including sighting information notifying that the information processing means of the central monitoring device has witnessed the tracking target, which is inputted through the command input means of the monitoring terminal device and is outputted through the information output means Get
The monitoring according to claim 1, wherein the monitoring point setting means sets a monitoring point at which the presence of the tracking target is predicted, based on position information of the monitoring terminal device from which the sighting information is output. system.
The monitoring information acquired by the information acquisition unit includes the image information generated by the image generation unit,
The monitoring point setting means calculates the moving direction of the tracking target specified by the observer based on the image information acquired temporally from the monitoring terminal device, and sets the monitoring point on the calculated moving direction side. The monitoring system according to claim 1 or 2, characterized in that:
The monitoring information acquired by the information acquisition unit includes the image information generated by the image generation unit,
The monitoring point setting means calculates the moving speed of the tracking target specified by the monitor based on the image information acquired temporally from the monitoring terminal device, and sets the monitoring point based on the calculated moving speed. The monitoring system according to any one of claims 1 to 3, characterized in that:
The central monitoring device further comprises a feature extraction unit for extracting the feature of the tracking object from the image information acquired from the monitoring terminal device,
The command output unit of the central monitoring device monitors an image transmission command for transmitting monitoring information on a subject having the feature of the tracking target extracted by the feature extraction unit, the monitoring of the mobile object selected by the selection unit The monitoring system according to any one of claims 1 to 4, wherein the monitoring system outputs data to a terminal device.
The image generation means of the monitoring terminal device at least temporarily stores in the storage means monitoring information including the generated image information.
The monitoring terminal device
Image search means for searching for image information including a subject having the feature of the tracking object acquired from the central monitoring device from the monitoring information stored in the storage means;
And output means for outputting notification information indicating the presence of the tracking object in the monitoring information when the image information of the object having the feature of the tracking object is searched by the image search means. The monitoring system according to claim 5, comprising.
The central monitoring device further comprises a feature extraction unit for extracting the feature of the tracking object from the image information acquired from the monitoring terminal device,
And a feature evaluation unit that determines whether the tracking target is a human based on the features extracted by the feature extraction unit.
If the feature evaluation means determines that the tracking target is a human, the command output unit of the central monitoring device may use the face image information or the face feature of the tracking target acquired from the image information. The monitoring system according to any one of claims 1 to 4, characterized in that an image transmission instruction for transmitting the included monitoring information is output to the monitoring terminal device of the mobile unit selected by the selection means.
The image generation means of the monitoring terminal device at least temporarily stores in the storage means monitoring information including the generated image information.
The monitoring terminal device
The image information of the face of the tracking target included in the image transmission command acquired from the central monitoring device or the image of which the feature and the similarity with the feature of the face are a predetermined value or more are searched from the monitoring information stored in the storage unit Search means,
When the image search means searches for image information of a face to be tracked or image information having a similarity with a face feature and a feature having a predetermined value or more, the monitoring information includes notification information notifying the presence of the trackee. The monitoring system according to claim 7, further comprising: output means for outputting.
The output unit of the monitoring terminal device is mounted on another moving body when the search unit searches for image information of the face to be tracked or image information having a similarity to a feature of the face that is equal to or more than a predetermined value. The monitoring system according to claim 8, wherein the retrieved image information is output to the monitoring terminal device via wireless communication.
10. The monitoring terminal device according to any one of claims 1 to 9, wherein the monitoring terminal device includes a fixed monitoring terminal device mounted at a predetermined position and including an image generation unit configured to capture an image of the surroundings to generate image information. The surveillance system described in Section.
Monitoring comprising: position detecting means for detecting positional information of each of a plurality of moving bodies, and image generating means mounted on each of the plurality of moving bodies and imaging the periphery of the moving bodies to generate image information A monitoring system comprising a central monitoring device for obtaining monitoring information from a terminal device via wireless communication, comprising:
The central monitoring device
Information acquisition means for acquiring monitoring information including at least position information detected by the position detection means of the monitoring terminal device and time information detected by the time detection means;
Monitoring point setting means for setting a monitoring point where a tracking target is predicted to exist based on the acquired monitoring information;
Selecting means for selecting a monitoring terminal device of a mobile capable of imaging the monitoring point;
And a command output unit configured to output an image transmission command for transmitting monitoring information including the image information to the monitoring terminal apparatus of the selected mobile object.
Monitoring comprising: position detecting means for detecting positional information of each of a plurality of moving bodies, and image generating means mounted on each of the plurality of moving bodies and imaging the periphery of the moving bodies to generate image information A monitoring system comprising a central monitoring device for obtaining monitoring information from a terminal device via wireless communication, comprising:
The central monitoring device
Information acquisition means for acquiring monitoring information including at least position information detected by the position detection means of the monitoring terminal device and time information detected by the time detection means;
Monitoring point setting means for setting a monitoring area where a tracking target is predicted to exist, based on the acquired monitoring information;
Selection means for selecting a monitoring terminal device of a mobile capable of imaging the monitoring area;
And a command output unit configured to output an image transmission command for transmitting monitoring information including the image information to the monitoring terminal apparatus of the selected mobile object.
Acquiring monitoring information including at least position information and time information of each mobile unit from monitoring terminal devices mounted on a plurality of mobile units;
Setting a monitoring point where a tracking target specified by the observer is predicted to exist, based on the acquired monitoring information;
Selecting a monitoring terminal apparatus of a mobile capable of imaging a predetermined monitoring area based on the set monitoring point at the monitoring time;
Outputting an image transmission command for transmitting monitoring information including image information for imaging the periphery of the selected moving body to the selected moving body.