WO2013161345A1

WO2013161345A1 - Monitoring system and monitoring method

Info

Publication number: WO2013161345A1
Application number: PCT/JP2013/053277
Authority: WO
Inventors: 照久高野; 秋彦香西; 真史安原
Original assignee: 日産自動車株式会社
Priority date: 2012-04-24
Filing date: 2013-02-12
Publication date: 2013-10-31

Abstract

Provided is a monitoring system comprising a central monitoring device (20), a camera (11) that picks up images from different directions in the vicinity of each moving body (V), and monitoring terminal devices (10). The central monitoring device (20) comprises: a function of calculating the communication rate with the monitoring terminal devices (10); a function of generating a monitoring information transmission instruction; and a function of transmitting the monitoring information transmission instruction that has thus been generated to a selected monitoring terminal device. If the transmission rate is less than a prescribed threshold value, the function of generating a monitoring information transmission instruction generates a monitoring image in which a first image of a first resolution which is a relatively high resolution and a second image of lower resolution than the first resolution are mixed under prescribed composition conditions with the information that is transmitted, and transmits monitoring information containing this monitoring image to the central monitoring device (20). The monitoring terminal devices (10) are equipped with: a function of generating a monitoring image in accordance with prescribed composition conditions contained in the monitoring information transmission instruction, and a function of transmitting the monitoring information including monitoring images in accordance with the monitoring information transmission instructions to the central monitoring device (20).

Description

Monitoring system and monitoring method

The present invention relates to a monitoring system and a monitoring method.
This application claims priority based on Japanese Patent Application No. 2012-098545 filed on Apr. 24, 2012. For designated countries that are allowed to be incorporated by reference, The contents described in the application are incorporated into the present application by reference and made a part of the description of the present application.

A security device that detects the occurrence of abnormalities is known by installing multiple security camera devices in shopping streets, store entrances, home entrances, and other streets, and monitoring surrounding images captured by the security camera device (Patent Document 1).

JP 2011-215767 A

However, security camera devices installed in towns continue to send captured images with uniform resolution to the monitor even when the communication speed is low. There is a problem that transmission is delayed.

The present invention provides a monitoring system that reduces the amount of communication data and suppresses the delay of information transmission by adjusting the amount of information of a monitoring image to be transmitted when the communication speed is equal to or lower than a predetermined value. For the purpose.

According to the present invention, when the communication speed is equal to or lower than a predetermined value, a first image having a first resolution with a relatively high resolution and a second image having a second resolution lower than the first resolution are set to a predetermined composition condition. The above-mentioned object is achieved by causing the monitoring terminal device to execute a monitoring information transmission command for generating and transmitting the monitoring image mixed in the above.

According to the present invention, when the communication speed is low, a monitoring image formed by mixing information with high resolution and information with low resolution on a monitoring terminal device mounted on each mobile unit is displayed in the central monitoring device. Therefore, it is possible to suppress an increase in the amount of communication data while ensuring the transmission frequency of information. That is, the central monitoring device can acquire rough information with a small amount of communication data a plurality of times instead of acquiring detailed information with a large amount of communication data once when the communication speed is low. Since the monitoring terminal device is controlled at the same time, information can be collected in real time without encouraging the congestion of the communication line that causes a decrease in the communication speed.

1 is a schematic diagram showing a monitoring system according to an embodiment of the present invention. It is a block diagram which shows the monitoring system of FIG. It is a perspective view which shows the example of arrangement | positioning of the camera in the monitoring system of FIG. It is a top view which shows the imaging range of the camera in the monitoring system of FIG. It is a figure which shows an example of the picked-up image of a front camera. It is a figure which shows an example of the picked-up image of the right side camera. It is a figure which shows an example of the picked-up image of a rear camera. It is a figure which shows an example of the picked-up image of the camera of the left side. It is a figure which shows an example of the monitoring image produced | generated based on the some image. It is a figure for demonstrating the distortion correction process of the monitoring image. It is a schematic diagram which shows an example of a projection model. It is a cross-sectional schematic diagram along xy plane of the projection model shown in FIG. It is FIG. 1 which shows an example of the display image shown on the display of a central monitoring apparatus. It is FIG. 2 which shows an example of the display image shown on the display of a central monitoring apparatus. It is FIG. 3 which shows an example of the display image shown on the display of a central monitoring apparatus. It is FIG. 4 which shows an example of the display image shown on the display of a central monitoring apparatus. It is FIG. 5 which shows an example of the display image shown on the display of a central monitoring apparatus. It is FIG. 6 which shows an example of the display image shown on the display of a central monitoring apparatus. It is FIG. 7 which shows an example of the display image shown on the display of a central monitoring apparatus. It is FIG. 8 which shows an example of the display image shown on the display of a central monitoring apparatus. It is a flowchart which shows the main control content by the monitoring terminal device side of the monitoring system of FIG. It is a flowchart which shows the main control content by the central monitoring apparatus side of the monitoring system of FIG. It is a flowchart which shows the main control content by the central monitoring apparatus side of the monitoring system of FIG. It is a flowchart which shows the main control content by the central monitoring apparatus side of the monitoring system of FIG. It is a figure for demonstrating the transmission state of the monitoring image based on 1st composition conditions. It is a figure for demonstrating the transmission state of the monitoring image based on 2nd composition conditions. It is a figure for demonstrating the transmission state of the monitoring image based on 3rd organization conditions. It is a figure for demonstrating the transmission state of the monitoring image based on 4th composition conditions. It is a figure for demonstrating the aspect of the production | generation process of monitoring information.

In one embodiment shown below, a monitoring system according to the present invention is used in a city by a supervisor such as a police station or a fire department or a security consignment company using captured images of cameras mounted on a plurality of moving bodies. It is embodied in a monitoring system 1 that centrally monitors the security of the country.

Each of the monitoring terminal devices 10 mounted on the plurality of moving bodies acquires the position information, the monitoring image around the moving body, and the time information at a predetermined timing, and the position information, the monitoring image, and the time The monitoring information including the information is transmitted to the central monitoring device 20 installed on the monitor side via wireless communication. The central monitoring device 20 accumulates monitoring information including at least a monitoring image and position information acquired from the monitoring terminal device 10, and displays the position information of the moving body on the map information in a superimposed manner via a display or the like. A monitoring image and time information captured at each moving body or each position are displayed. Therefore, as shown in FIG. 1, the monitoring system 1 of this example is mounted on a moving body V and transmits monitoring information such as position information and monitoring images via a telecommunication network 30. A central monitoring device 20 that acquires and processes the monitoring information.

In particular, when the communication speed of information received from the monitoring terminal device 10 is equal to or lower than a predetermined threshold, the monitoring system 1 according to the present embodiment has a predetermined knitting condition defined from the viewpoint of reducing the amount of communication data. A monitoring information transmission command for generating a mixed monitoring image is transmitted via wireless communication. Then, on the mobile body V side, a monitoring image according to the monitoring information transmission command acquired via wireless communication is generated, and monitoring information including the monitoring image is transmitted to the supervisor side via wireless communication. As a result, the supervisor side adjusts the resolution of the image information according to the communication speed reduction state, and acquires the monitoring information including the monitoring image with the communication data amount reduced.

The monitor can select the moving object V for which a monitoring image is to be generated based on the position of the moving object superimposed on the map information, the movement schedule of the moving object V acquired in advance, and the like. The monitor can select the moving body V by pointing the target moving body V with a pointer such as a cursor or a touch pen, or by touching the touch panel display screen with a finger. Further, the monitor specifies the direction in which he / she wants to pay attention according to the location where the accident has occurred, the location designated by the notification, the location to be monitored, and other points of interest based on the position and traveling direction of the moving object V. The monitor inputs to the central monitoring device 20 information specifying the selected moving object V, information specifying one or more directions to be watched, and a change in the watching direction. The monitor designates the start point and end point or start point, start point, intermediate point, and end point of the range to be watched with a pointer such as a cursor or a touch pen, or touches the touch panel display screen with a finger. By doing so, the gaze direction can be input.

The mobile body V on which the monitoring terminal device 10 is mounted is not particularly limited as long as it travels in the target monitoring area, and includes mobile bodies V such as automobiles, motorcycles, industrial vehicles, and trams. As shown in FIG. 1, automobiles include private automobiles V2 and emergency automobiles V3, and in particular, taxis and route buses V1 that travel randomly and constantly in a predetermined area are preferably included. FIG. 1 illustrates an emergency vehicle V3 such as a taxi V1, a private vehicle V2, a police car, a fire engine or an ambulance, but these are collectively referred to as a moving body V or a passenger vehicle V.

FIG. 2 is a block diagram illustrating a specific configuration of the central monitoring device 20 and the monitoring terminal device 10.
The monitoring terminal device 10 and the central monitoring device 20 can communicate via the telecommunication network 30. The communication device 23 of the monitoring terminal device 10 is a communication means capable of wireless communication, and exchanges information with the communication device 23 of the central monitoring device 20 via the telecommunication network 30. When the telecommunications network 30 is a commercial telephone network, a mobile phone communication device can be used widely, and when the telecommunications network 30 is a dedicated telecommunications network for the monitoring system 1 of this example, it is dedicated to it. The

communication devices

23 and 13 can be used. Instead of the telecommunication network 30, a wireless LAN, WiFi (registered trademark), WiMAX (registered trademark), Bluetooth (registered trademark), a dedicated wireless line, or the like can be used.

The central monitoring device 20 inputs the positional information and the monitoring image transmitted from the monitoring terminal device 10 mounted on each of the moving bodies V to the database in order to monitor the city by the captured image of the camera of the moving body V. It has an input function and a display control function for displaying the received monitoring image on the display 24 while superimposing the received positional information on the display 24 on the map information read from the map database.

Furthermore, the central monitoring apparatus 20 of this embodiment selects the moving body V for which a monitoring image is to be acquired with reference to the position of the moving body V on the map information presented by the monitor, and monitors the selected moving body V. It has a function of generating a monitoring information transmission command for the terminal device 10. Specifically, the central monitoring device 20 of the present embodiment has a communication speed calculation function for calculating the communication speed of information transmitted from the monitoring terminal device 10, and when the calculated communication speed is equal to or less than a predetermined threshold, In addition to generating a monitoring image in which the first image having a relatively high resolution and the second image having a second resolution lower than the first resolution are mixed with the information to be transmitted under a predetermined composition condition. A command generation function for generating a monitoring information transmission command for transmitting the monitoring information including the generated monitoring image to the central monitoring device 20 and a generated monitoring information transmission command are transmitted to the selected monitoring terminal device 10. Command transmission function.

Therefore, the central monitoring device 20 includes a central control device 21, an image processing device 22, a communication device 23, a display 24, and an input device 25. The central monitoring device 20 according to the present embodiment has a database for storing monitoring information inside the central monitoring device 20, but may be provided outside the central monitoring device 20 as long as it is accessible.

The central control device 21 of the central monitoring device 20 is configured by a CPU, a ROM, and a RAM, and controls the image processing device 22, the communication device 23, and the display 24 to transmit the positional information, the monitoring image, and the information transmitted from the monitoring terminal device 10. The time information is received, subjected to image processing as necessary, and displayed on the display 24.

The central controller 21 calculates a communication speed when communication with a certain mobile unit V is performed. The communication speed of the present embodiment is a data transfer speed between the communication device 23 of the central monitoring device 20 and the communication device 13 of the monitoring terminal device 10. The communication speed of the present embodiment can be determined based on the frame rate of the monitoring image received from the monitoring terminal device 10 on the central monitoring device 20 side. The frame rate is the number of images processed (rewritten) per unit time in image display and moving image playback. If the communication speed between the communication device 23 of the central monitoring device 20 and the monitoring terminal device 10 communication device 13 is high, the frame rate related to the processing of the monitoring image received from the monitoring terminal device 10 on the central monitoring device 20 side becomes high. If the communication speed between the two is slow, the frame rate related to the monitoring image processing on the central monitoring device 20 side becomes low, so the communication speed can be evaluated based on the frame rate in the monitoring terminal device 10.

In this specification, the term “communication speed when the central monitoring device 20 and the monitoring terminal device 10 exchange information” includes “a frame rate related to processing of a monitoring image in the central monitoring device 20”. That is, in this embodiment, whether or not the communication speed is equal to or lower than the predetermined threshold can be determined based on whether or not the frame rate related to the monitoring image processing of the central monitoring device 20 is equal to or lower than the predetermined threshold.

The communication speed receives a flag indicating completion of data reception from the communication device 23 of the central monitoring device 20 from the timing when the communication device 13 of the monitoring terminal device 10 transmits data to the communication device 23 of the central monitoring device 20. This can be determined based on the time T until the timing. If the communication speed between the communication device 23 of the central monitoring device 20 and the monitoring terminal device 10 communication device 13 is high, the time T from information transmission to receipt of reception notification is shortened. Since the time T from transmission to receipt of the reception notification becomes longer, the communication speed can be evaluated based on the time T required for the information transmission completion in the monitoring terminal device 10.

Therefore, the term “communication speed when the central monitoring device 20 and the monitoring terminal device 10 exchange information” in this specification is “the time required for the monitoring terminal device 20 to transmit information to the central monitoring device 20”. Including. That is, in the present embodiment, whether or not the communication speed is equal to or lower than the predetermined threshold is determined until the monitoring terminal device 10 transmits information to the central monitoring device 20 and receives a signal of information reception completion from the central monitoring device 20. Can be determined by whether or not the time is equal to or greater than a predetermined threshold.

The predetermined threshold regarding the communication speed depends on the performance of the communication device 23 of the central monitoring device 20 and the communication device 13 of the monitoring terminal device 10, the communication environment around the location of the monitoring terminal device 10, and the communication environment around the location of the central monitoring device 20. And can be set appropriately. Note that the threshold for evaluating the frame rate is a threshold for determining whether or not the communication speed is decreasing, and the technical significance is the same as the threshold for evaluating the communication speed. Since the value has a meaning different from the speed, the threshold for evaluating the frame rate is set by independent determination separately from the threshold for evaluating the communication speed. For the same reason, the threshold for evaluating the time from the information transmission to the transmission completion confirmation in the monitoring terminal device 10 is set by independent determination separately from the threshold for evaluating the communication speed.

The central control device 21 exists in the vicinity of the moving body V and the moving body V when the communication speed when the central monitoring device 20 and the monitoring terminal device 10 exchange information is below a predetermined threshold. Mobile unit V or a mobile unit V belonging to a predefined area to which the mobile unit V belongs (such as an area in which communication is performed via the same base station) is selected, and a monitoring terminal device for the selected mobile unit V 10 generates a monitoring image under a predetermined composition condition, generates a monitoring information transmission command including a command for transmitting the generated monitoring image to the central monitoring device 20, and transmits the monitoring information to the selected monitoring terminal device 10. In this specification, when the communication speed when the central monitoring device 20 and the monitoring terminal device 10 exchange information is equal to or lower than a predetermined threshold, the frame rate related to the monitoring image processing in the central monitoring device 20 is predetermined. This includes a case where the value is equal to or less than the value and a case where the time from transmission of information to confirmation of transmission completion in the monitoring terminal device 10 is equal to or greater than a predetermined value.

Therefore, the monitoring information transmission command includes a communication ID and other identifiers for identifying the monitoring terminal device 10 of the selected moving object V. Selection of the moving body V can be performed based on a selection command for the moving body V input from the input device 25. The mobile body V whose communication speed is equal to or lower than a predetermined threshold, that is, the mobile body V that is a target for transmitting the monitoring information transmission command, is individually determined for each mobile body V based on the actual communication speed with the central monitoring device 20. Alternatively, since the communication environment tends to depend on the state at each time and place, it is highly likely that other mobile objects V in the vicinity of the mobile object V whose communication speed has decreased will also decrease in communication speed. It may be determined and a plurality of moving bodies V may be selected together based on the position information. Furthermore, all the mobile objects V existing in the communication area covered by the base station where the communication speed is reduced may be selected together. Further, in normal times, the moving body V existing in the vicinity area of the accident occurrence point input from the outside, the moving body V existing in the vicinity area of the moving body V reporting the occurrence of the accident, or monitoring in advance is important. One or a plurality of moving bodies V may be selected by automatically extracting moving bodies V and the like existing in the vicinity area of the priority monitoring point defined as the point to perform.

The monitoring terminal device 10 mounted on the moving body V selected by the monitor uses the cameras 1a to 1d provided at different positions of the moving body V and uses the monitoring image according to the composition condition included in the monitoring information transmission command. And the monitoring information including the monitoring image is transmitted to the central monitoring device 20. In this way, the central monitoring device 20 and the monitoring terminal device 10 of the moving object V cooperate with each other to provide monitoring information for monitoring the city.

The details of the monitoring information transmission command will be described later.

The image processing device 22 has a map database, displays map information from the map database on the display 24, and superimposes and displays position information detected by the position detection device 15 of the monitoring terminal device 10 on the map information. To do. In addition, image processing is performed to display the monitoring image captured by the camera 11 of the monitoring terminal device 10 and processed by the image processing device 12 on the display 24.

The display 24 can be composed of, for example, a liquid crystal display device having a size capable of displaying two or more window screens on one screen, or two or more liquid crystal display devices each displaying two or more window screens. One window screen displays a screen in which the position information of each moving body V is superimposed on the map information (see FIG. 1), and the other window screen is captured by the camera 11 of the moving body V. A monitoring image generated based on the captured image is displayed.

The input device 25 is an input device such as a keyboard, a mouse, or a touch panel. The input device 25 specifies a desired moving object V, outputs a monitoring information transmission command to the moving object V, and displays various information displayed on the display 24. This is used when inputting a processing command.

As described above, the communication device 23 is a communication means capable of wireless communication, and exchanges information with the communication device 13 of the monitoring terminal device 10 via the telecommunication network 30. When the telecommunications network 30 is a commercial telephone network, a mobile phone communication device can be used widely, and when the telecommunications network 30 is a dedicated telecommunications network for the monitoring system 1 of this example, it is dedicated to it. The

communication devices

13 and 23 can be used.

Subsequently, the monitoring terminal device 10 will be described. The monitoring terminal device 10 is a terminal device mounted on each of the plurality of moving bodies V. The monitoring terminal device 10 is mounted on each of the plurality of moving bodies V and a position detection function for detecting position information of each of the plurality of moving bodies V. The monitoring image generating function for generating a monitoring image based on the captured image around the moving object V captured by the camera 11, and the position information, the monitoring image and the time information acquired at a predetermined timing are stored in the central monitoring device 20. And a communication function for receiving a command from the central monitoring device 20. Therefore, each moving body V includes a plurality of cameras 11a to 11d, an image processing device 12, a communication device 13, a control device 14, and a position detection device 15. Note that the time information is mainly information used for post-event analysis, and may be omitted.

The plurality of cameras 11 mounted on the respective moving bodies V are constituted by CCD cameras or the like, take images of respective predetermined directions or predetermined areas around the moving body V, and output the image pickup signals to the image processing device 12. The camera 11 of this embodiment can set the resolution, and can image the surroundings of the vehicle V with a predetermined resolution. The image processing device 12 reads an imaging signal from the camera 11 and executes image processing for generating a monitoring image. Details of this image processing will be described later.

The position detection device 15 includes a GPS device and its correction device, and detects the current position of the moving object V and outputs it to the control device 14.

The control device 14 includes a CPU, a ROM, and a RAM, receives a monitoring information transmission command from the central monitoring device 20 received via the telecommunication network 30 and the communication device 13, and receives the camera 11, the image processing device 12, The communication device 13 and the position detection device 15 are controlled, the monitoring image generated by the image processing device 12, the position information of the moving object V detected by the position detection device 15, and the time information from the clock built in the CPU, Is output to the central monitoring device 20 via the communication device 13 and the telecommunication network 30.

In generating the monitoring image, the control device 14 follows the monitoring information transmission command acquired from the central monitoring device 20 via the communication device 13, and the resolution is relative based on the predetermined composition information included in the monitoring information transmission command. A monitoring image generation function for causing the image processing device 12 to generate a monitoring image including a first image having a first resolution higher than the first resolution and a second image having a second resolution lower than the first resolution; A monitoring information transmission function for causing the central monitoring device 20 to transmit the included monitoring information via the communication device 13 is executed. In the present embodiment of the present invention, a monitoring image including at least a first image having a relatively high first resolution and a second image having a relatively low second resolution will be described. It is also possible to further include images of other resolutions. Note that the control device 14 causes the image processing device 12 to generate a monitoring image having a predetermined resolution set in advance before receiving the monitoring information transmission command from the central monitoring device 20, and the central monitoring device via the communication device 13. 20 to send.

Next, the mounting positions and imaging ranges of the cameras 11a to 11d will be described. Here, a passenger car V will be described as an example of the moving body V. The cameras 11a to 11d are configured by using an image sensor such as a CCD, and the four cameras 11a to 11d are installed at different positions outside the passenger car V, respectively, and respectively photograph the four directions of front, rear, left and right around the moving body V.

For example, as shown in FIG. 3, the camera 11a installed at a predetermined position in front of the passenger car V such as a front grille is an object or road surface (front) in the area SP1 in front of the passenger car V and in the space in front thereof. Take a view). The camera 11b installed at a predetermined position on the left side of the passenger car V, such as the left side mirror portion, shows an object or road surface (left side view) existing in the area SP2 on the left side of the passenger car V and in the surrounding space. Take a picture. The camera 11c installed at a predetermined position in the rear part of the passenger car V, such as a rear finisher part or a roof spoiler part, shows an object or road surface (rear view) existing in the area SP3 behind the passenger car V and in the space behind it. Take a picture. The camera 11d installed at a predetermined position on the right side of the passenger car V, such as the right side mirror portion, shows an object or road surface (right side view) existing in the area SP4 on the right side of the passenger car V and in the surrounding space. Take a picture.

FIG. 4 is a view of the arrangement of the cameras 11a to 11d as viewed from above the passenger car V. As shown in the figure, the camera 11a that captures the area SP1, the camera 11b that captures the area SP2, the camera 11c that captures the area SP3, and the camera 11d that captures the area SP4 are the outer periphery VE of the body of the passenger car V. Along the counterclockwise (counterclockwise) or clockwise (clockwise). That is, when viewed along the outer periphery VE of the body of the passenger car V in the counterclockwise direction (counterclockwise) indicated by the arrow C in the same figure, the camera 11b is installed on the left side of the camera 11a, and the camera 11c is on the left side of the camera 11b. The camera 11d is installed on the left side of the camera 11c, and the camera 11a is installed on the left side of the camera 11d. Conversely, when viewed along the outer periphery VE of the body of the passenger car V in the direction opposite to the direction of the arrow C shown in the figure (clockwise), the camera 11d is installed on the right side of the camera 11a, and on the right side of the camera 11d. A camera 11c is installed, a camera 11b is installed on the right side of the camera 11c, and a camera 11a is installed on the right side of the camera 11b.

5A shows an example of an image GSP1 in which the front camera 11a images the area SP1, FIG. 5B shows an example of an image GSP2 in which the left side camera 11b images the area SP2, and FIG. 5C shows a rear camera. 11c shows an example of an image GSP3 obtained by imaging the area SP3, and FIG. 5D is an image diagram showing an example of an image GSP4 obtained by the right side camera 11d imaging the area SP4. Incidentally, the size of each image in the present embodiment is 480 vertical pixels × 640 horizontal pixels or 960 vertical pixels × horizontal 1280 pixels. In the present embodiment, an image having a relatively high first resolution of 960 pixels × width 128 is set as the first image, and an image having a relatively low second resolution of 480 pixels × width 640 pixels is the second image. And The image size is not particularly limited, and may be any size that can be played back by a general terminal device at two relatively different resolutions.

It should be noted that the number of cameras 11 and the positions of the cameras 11 can be appropriately determined according to the size, shape, detection area setting method, etc. of the passenger car V. The plurality of cameras 11 described above are assigned identifiers according to their arrangement, and the control device 14 can identify each of the cameras 11 based on each identifier. Further, the control device 14 can transmit an imaging command and other commands to the specific camera 11 by attaching an identifier to the command signal.

The control device 14 controls the image processing device 12 to acquire each image signal picked up by the camera 11, and the image processing device 12 processes the image pickup signal from each camera 11, and is shown in FIGS. 5A to 5D. Convert to surveillance image. Then, the control device 14 generates a monitoring image based on the four monitoring images shown in FIGS. 5A to 5D (monitoring image generation function), and projects the monitoring image set on the side of the projection model of the columnar body. Mapping information to be projected onto the surface is associated with the monitoring image (mapping information adding function) and output to the central monitoring device 20. Hereinafter, the monitoring image generation function and the mapping information addition function will be described in detail.

Note that the monitoring image is generated on the basis of the four monitoring images obtained by imaging the periphery of the passenger car V, and the process of associating the mapping information with the monitoring image is executed by the monitoring terminal device 10 as in this example, and also by the central monitoring device 20. It can also be executed. In this case, four monitoring images obtained by imaging the periphery of the passenger car V are transmitted as they are from the monitoring terminal device 10 to the central monitoring device 20, and are monitored by the image processing device 22 and the central control device 21 of the central monitoring device 20. It is only necessary to generate an image, associate mapping information, and perform projection conversion.

First, the monitoring image generation function will be described. The control device 14 of the monitoring terminal device 10 according to the present embodiment controls the image processing device 12 to acquire the imaging signals of the respective cameras 11a to 11d, and further turns clockwise or counterclockwise along the outer periphery of the body of the passenger car V. One monitoring image is generated so that the monitoring images of the cameras 11a to 11d installed in the direction of are arranged in the installation order of these cameras 11a to 11d.

As described above, in this embodiment, the four cameras 11a to 11d are installed in the order of the

cameras

11a, 11b, 11c, and 11d in the counterclockwise direction along the outer periphery VE of the body of the passenger car V. The control device 14 connects the four cameras 11a to 11d in a horizontal direction so that the four images captured by the cameras 11a to 11d are integrated in accordance with the installation order of these cameras 11a to 11d (cameras 11a → 11b → 11c → 11d). Then, one monitoring image is generated. In the monitoring image of the present embodiment, the images are arranged such that the ground contact surface (road surface) of the passenger vehicle V is the lower side, and the images are connected to each other at sides in the height direction (vertical direction) with respect to the road surface.

FIG. 6 is a diagram illustrating an example of the monitoring image K. As shown in the figure, the monitoring image K of the present embodiment includes a captured image GSP1 in which the front camera 11a images the area SP1 and a left camera 11b in the area P along the direction P from the left side to the right side of the drawing. A captured image GSP2 captured from SP2, a captured image GSP3 captured by the rear camera 11c capturing the area SP3, and a captured image GSP4 captured by the right side camera 11d capturing the area SP4 are arranged in this order in the horizontal direction. Four images are taken as a series of images. The monitoring image K generated in this way is displayed in order from the left end to the right side with the image corresponding to the road surface (the ground contact surface of the moving body V) facing down, so that the monitor can turn around the moving body V. It can be visually recognized on the display 24 in the same manner as when looking around clockwise. As described above, the monitoring image K in the mode shown in FIG. 6 can be reproduced in the arrangement in the expected order without causing the positional relationship to be out of order even when processing such as transmission / reception is performed.

It should be noted that when one monitoring image K is generated, four images acquired at substantially the same time as the photographing timings of the cameras 11a to 11d are used. Thereby, since the information included in the monitoring image K can be synchronized, the situation around the moving object V at a predetermined timing can be accurately expressed.

In addition, the monitoring image K generated from the respective captured images with substantially the same imaging timing of the camera 11 is stored over time, and a moving image monitoring image K including a plurality of monitoring images K per predetermined unit time is generated. You may do it. By generating the monitoring image K of the moving image based on the images with the same imaging timing, it is possible to accurately represent the change in the situation around the moving object V.

By the way, when the image of each imaging area is memorize | stored each time and the monitoring image K of the moving image produced | generated for every imaging area is transmitted to the central monitoring apparatus 20, depending on the function of the central monitoring apparatus 20, a several You may not be able to play videos at the same time. In such a conventional central monitoring apparatus 20, since a plurality of moving images cannot be reproduced and displayed at the same time, when reproducing each moving image, the moving images must be reproduced one by one by switching the screen. In other words, the conventional central monitoring device 20 has a disadvantage that it cannot simultaneously watch images (moving images) in a plurality of directions and cannot monitor the entire periphery of the moving object V on one screen.

On the other hand, since the control device 14 of the present embodiment generates one monitoring image K from a plurality of images, it can simultaneously play back moving images of images in different imaging directions regardless of the function of the central monitoring device 20. . That is, by continuously reproducing the monitoring image K (moving image reproduction), four images included in the monitoring image K are simultaneously reproduced (moving image reproduction), and the state change of the regions in different directions is displayed on one screen. Can be monitored.

In addition, the monitoring terminal device 10 of the present embodiment generates the monitoring image K by compressing the image data amount so that the number of pixels of the monitoring image K is substantially the same as the number of pixels of the images of the cameras 11a to 11d. You can also. While the size of each image shown in FIGS. 5A to 5D is 480 × 640 pixels, in this embodiment, compression processing is performed so that the size of the monitoring image K is 1280 × 240 pixels as shown in FIG. Do. As a result, the size of the monitoring image K (1280 × 240 = 307,200 pixels) becomes equal to the total size of each image (480 × 640 × 4 = 307,200 pixels). Regardless of the function on the monitoring device 20 side, image processing and image reproduction can be performed. Even when the size of each image shown in FIGS. 5A to 5D is 960 × 1280 pixels, it is possible to generate a compressed monitoring image K in a form in which the four images shown in FIG. 6 are connected.

Although not particularly limited, the first image of the first resolution of the present embodiment includes an image of 960 × 1280 pixels shown in FIGS. 5A to 5D, and these four images are made into one image as shown in FIG. , The monitoring image K compressed to 960 × 1280 pixels is included, and the second image of the second resolution includes the images of 480 × 640 pixels shown in FIGS. 5A to 5D, and these four images are shown in FIG. Thus, a monitoring image K compressed into 1280 × 240 pixels is included as one image.

The control device 14 of the monitoring terminal device 10 according to the present embodiment can also attach a line figure indicating the boundary between the arranged images to the monitoring image K. Taking the monitoring image K shown in FIG. 6 as an example, the control device 14 displays rectangular partition images Bb, Bc, Bd, Ba, Ba ′ between the images as line figures indicating the boundaries between the arranged images. It can be attached to the monitoring image K. In this manner, by arranging the partition images at the boundaries of the four images, it is possible to recognize each image having a different imaging direction in the series of monitoring images K. That is, the partition image functions as a frame of each captured image. In addition, since the image distortion is large in the vicinity of the boundary of each captured image, it is possible to hide the image of the region with large distortion or to suggest that the distortion is large by arranging the partition image at the boundary of the captured image. .

Further, the control device 14 according to the present embodiment can generate the monitoring image K after correcting the distortion when four images are projected on the projection plane set on the side surface of the projection model described later. In the peripheral area of the captured image, image distortion is likely to occur. In particular, in the case of the camera 11 using a wide-angle lens, the captured image tends to be largely distorted. Therefore, it is defined in advance to correct the image distortion. It is desirable to correct the distortion of the captured image using the image conversion algorithm and the correction amount.

Although not particularly limited, as shown in FIG. 7, the control device 14 reads information on the same projection model as the projection model for projecting the monitoring image K in the central monitoring device 20 from the ROM, and takes a captured image on the projection plane of the projection model. The distortion generated on the projection plane can be corrected in advance. The image conversion algorithm and the correction amount can be appropriately defined according to the characteristics of the camera 11 and the shape of the projection model. In this way, by correcting in advance the distortion when the image K is projected with respect to the projection plane of the projection model, it is possible to provide the monitoring image K with good visibility with less distortion. Further, by correcting the distortion in advance, it is possible to reduce the positional deviation between the images arranged side by side.

Next, the mapping information addition function will be described. In the monitoring terminal device 10 of the present embodiment, the control device 14 projects the generated monitoring image K on the projection plane set on the side surface of the columnar projection model M with the ground contact surface of the passenger car V as the bottom surface. The process of associating the mapping information with the monitoring image K is executed. The mapping information is information for allowing the central monitoring device 20 that has received the monitoring image K to easily recognize the projection reference position. FIG. 8 is a diagram showing an example of the projection model M of the present embodiment, and FIG. 9 is a schematic sectional view taken along the xy plane of the projection model M shown in FIG.

8 and 9, the projection model M of the present embodiment is a regular octagonal prism body having a regular octagonal bottom surface and a height along the vertical direction (z-axis direction in the figure). Note that the shape of the projection model M is not particularly limited as long as it is a column having side surfaces adjacent to each other along the boundary of the bottom surface, and is a cylinder, or a prism, such as a triangular column, a quadrangular column, or a hexagonal column, or An anti-rectangular column having a polygonal bottom surface and a triangular side surface can also be used.

Also, as shown in the figure, the bottom surface of the projection model M of this embodiment is parallel to the ground contact surface of the passenger car V. Projection surfaces Sa, Sb, Sc, and Sd (hereinafter collectively referred to as a projection surface S) that project an image around the passenger vehicle V that contacts the bottom surface of the projection model M are provided on the inner surface of the side surface of the projection model M. Is set. The projection surface S includes a part of the projection surface Sa and a part of the projection surface Sb, a part of the projection surface Sb and a part of the projection surface Sc, a part of the projection surface Sc and a part of the projection surface Sd, and the projection surface Sd. And a part of the projection surface Sa. The monitoring image K is projected on the projection plane S as an image of the passenger car V viewed from above the viewpoint R (R1 to R8, hereinafter referred to as viewpoint R) above the projection model M surrounding the passenger car V.

The control device 14 according to the present embodiment associates the reference coordinates of the captured image arranged at the right end or the left end with the monitoring image K as mapping information. Taking the monitoring image K shown in FIG. 6 as an example, the control device 14 is arranged at the right end as mapping information (reference coordinates) indicating the start end position or the end position of the monitoring image K when projected onto the projection model M. The coordinates A (x, y) of the upper left vertex of the captured image GSP1 and the coordinates B (x, y) of the upper right vertex of the captured image GSP2 arranged at the left end are attached to the monitoring image K. Note that the reference coordinates of the captured image indicating the start position or the end position are not particularly limited, and may be the lower left vertex of the monitoring image K arranged at the left end or the lower right vertex of the monitoring image K arranged at the right end. The mapping information may be attached to each pixel of the image data of the monitoring image K, or may be managed as a file different from the monitoring image K.

As described above, the information indicating the start position or the end position of the monitoring image K, that is, the reference coordinates used as a reference in the projection processing is associated with the monitoring image K as mapping information, whereby the central monitoring apparatus 20 that has received the monitoring image K Since the reference position at the time of the projection process can be easily recognized, the monitoring images K arranged in the order of arrangement of the cameras 11a to 11d are sequentially and easily projected onto the projection surface S on the side surface of the projection model M. Can do. That is, as shown in FIG. 9, the captured image GSP1 in front of the moving body V is projected onto the projection surface Sa located in the imaging direction of the camera 11a, and the right side of the moving body V is projected onto the projection surface Sb located in the imaging direction of the camera 11b. The captured image GSP2 is projected, the captured image GSP3 behind the moving body V is projected onto the projection surface Sc located in the imaging direction of the camera 11c, and the left side of the moving body V is projected onto the projection surface Sd located in the imaging direction of the camera 11d. One captured image GSP4 can be projected.

Thereby, the monitoring image K projected on the projection model M can show an image that can be seen as if looking around the passenger car V. In other words, the monitoring image K including four images arranged in a line in the horizontal direction in accordance with the installation order of the cameras 11a to 11d is projected on the side surfaces arranged in the horizontal direction in the column of the projection model M. An image around the passenger car V can be reproduced in the monitoring image K projected on the projection plane S of the body projection model M while maintaining the positional relationship.

Note that the control device 14 of the present embodiment can store the correspondence between the coordinate values of the monitoring image K and the coordinate values of the projection planes S of the projection model M as mapping information, and attach it to the monitoring image K. However, it may be stored in the central monitoring device 20 in advance.

Further, the positions of the viewpoint R and the projection plane S shown in FIGS. 8 and 9 are examples, and can be arbitrarily set. In particular, the viewpoint R can be changed by the operation of the operator. The relationship between the viewpoint R and the projection position of the monitoring image K is defined in advance, and when the position of the viewpoint R is changed, a predetermined coordinate transformation is performed, so that the viewpoint R is viewed from the newly set viewpoint R. The monitoring image K can be projected onto the projection surface S (Sa to Sd). A known method can be used for this viewpoint conversion processing.

As described above, the control device 14 according to the present embodiment generates the monitoring image K based on the monitoring image captured at a predetermined timing, and the monitoring image K includes line information (partitions) indicating mapping information, reference coordinates, and boundaries. Image) information is associated and stored over time according to the imaging timing. Although not particularly limited, the control device 14 may store the monitoring image K as one moving image file including a plurality of monitoring images K per predetermined unit time, or in a form that can be transferred / reproduced by a streaming method. The monitoring image K may be stored.

Moreover, the control apparatus 14 of this embodiment can include the movement speed acquired from the apparatus which detects the movement speed of the moving body V, and the vehicle speed sensor 16 with which a vehicle is provided in this example in monitoring information. This moving speed (vehicle speed) is used when the knitting condition included in the monitoring information transmission command is set on the central monitoring device 20 side.

On the other hand, the communication device 23 of the central monitoring device 20 receives the monitoring image K transmitted from the monitoring terminal device 10 and the mapping information associated with the monitoring image K. In the monitoring image K, as described above, the images of the four cameras 11 installed at different positions of the body of the passenger car V are installed along the clockwise or counterclockwise direction along the outer periphery of the body of the passenger car V. The cameras 11a to 11d are arranged according to the installation order (clockwise or counterclockwise order along the outer periphery of the body of the moving body V). The monitoring image K is associated with mapping information for projecting the monitoring image K onto the projection plane S of the octagonal prism projection model M. The communication device 23 transmits the acquired monitoring image K and mapping information to the image processing device 22.

The image processing apparatus 22 reads the projection model M stored in advance, and sets it on the side surface of the octagonal prism projection model M with the ground contact surface of the passenger car V shown in FIGS. 8 and 9 as the bottom surface based on the mapping information. A display image is generated by projecting the monitoring image K onto the projected planes Sa to Sd. Specifically, according to the mapping information, each pixel of the received monitoring image K is projected onto each pixel of the projection surfaces Sa to Sd. Further, when projecting the monitoring image K onto the projection model M, the image processing device 22 recognizes the start point of the monitoring image K (the right end or the left end of the monitoring image K) based on the reference coordinates received together with the monitoring image K. Then, the projection processing is performed so that the start point coincides with the start point (the right end or the left end of the projection surface S) defined in advance on the projection model M. Further, when projecting the monitoring image K onto the projection model M, the image processing device 22 arranges a line figure (partition image) indicating the boundary of each image on the projection model M. The partition image can be attached to the projection model M in advance, or can be attached to the monitoring image K after the projection processing.

The display 24 or the touch panel display 24 having a display function displays the monitoring image K projected on the projection surface S of the projection model M. 10 to 17 show examples of display images of the monitoring image K. FIG. Note that white double circles shown in FIG. 11 are attached to explain the method of inputting (specifying) the gaze direction, which will be described later, and are not included in the monitoring image K.

FIG. 10 shows a monitoring image K projected on the projection surfaces Sd, Sa, Sb viewed from the viewpoint R1 shown in FIGS. An image of the moving body V viewed from each viewpoint R is pasted on the bottom surface of the projection model M. Further, the portion where the image between the projection surfaces Sd, Sa, and Sb is not displayed is a “line figure indicating a boundary (partition image)”.

Similarly, FIG. 11 shows the monitoring image K viewed from the viewpoint R2, FIG. 12 shows the monitoring image K viewed from the viewpoint R3, FIG. 13 shows the monitoring image K viewed from the viewpoint R4, and FIG. 15 shows the monitoring image K viewed from the viewpoint R5, FIG. 15 shows the monitoring image K viewed from the viewpoint R6, FIG. 16 shows the monitoring image K viewed from the viewpoint R7, and FIG. The observed monitoring image K is shown.

As described above, the terminal device 800 according to the present embodiment transfers the captured image of each camera 1 along the x-axis direction or the y-axis direction (sideways) according to the installation order of the cameras 1 installed on the body of the moving body V. ) Since the arranged monitoring image K is mapped (laterally) along the side surface of the projection model M of the columnar body according to the arrangement order, the monitoring image K shown in the projection model M has a clock around the moving body V. It is possible to show an image that can be seen when looking around. That is, the supervisor can obtain the same information as when the user gets on the moving body V and looks around while watching the monitoring image K while staying at a position separated from the moving body V.

Particularly, as shown in FIGS. 10 to 17, by showing the image of the moving body V on the bottom surface of the projection model M, the positional relationship between the orientation of the moving body V and each captured image can be easily understood. In other words, the captured image GSP1 of the camera 1a provided on the front grille of the moving body V is projected onto the projection surface S facing the front grille of the moving body V, and the projection surface S facing the right side mirror of the moving body V is projected. Projects the captured image GSP4 of the camera 1d provided on the right side mirror of the moving object V, and the camera 1c provided on the rear part of the moving object V on the projection surface S facing the rear part of the moving object V. The captured image GSP3 is projected, and the captured image GSP2 of the camera 1b provided on the left side mirror of the moving body V can be projected onto the projection surface S facing the left side mirror of the moving body V. As described above, as shown in FIGS. 10 to 17, according to the present embodiment, it is possible to present the monitoring image K projected while maintaining the positional relationship of the video around the moving object V. The monitor can easily grasp what is happening around the moving body V.

Incidentally, since FIGS. 10 to 17 attached to the present application are still images, the actual display image reproduction state cannot be shown. However, in the central monitoring device 20 of the present embodiment, the display screen of the display 24 is not displayed. Each of the images shown on each projection plane S is a moving image. That is, the moving image of the imaging area SP1 in front of the moving body V is projected on the projection surface S facing the front grille of the moving body V, and the moving body V is projected on the projection surface S facing the right side mirror of the moving body V. A moving image of the imaging area SP4 on the right side of the V is displayed, and a moving image of the imaging area SP3 behind the moving object V is displayed on the projection surface S facing the rear part of the moving object V, and the left side of the moving object V is displayed. On the projection surface S facing the mirror, a moving image of the imaging region SP2 on the left side of the moving object V is projected. That is, a plurality of moving image monitoring images K based on captured images captured by different cameras 1 can be simultaneously reproduced on each projection plane S shown in FIGS.

It should be noted that by providing an input device 25 such as a mouse or a keyboard or adopting a touch panel type display 24 that is also the input device 25, the viewpoint can be freely set and changed by the operation of the supervisor. Since the correspondence relationship between the viewpoint position and the projection plane S is defined in advance in the image processing device 22 or the display 24 described above, the monitoring image K corresponding to the changed viewpoint is displayed on the display 24 based on this correspondence relationship. can do.

The resolution of each image included in each display image shown in FIG. 10 to FIG. 17 may be made common from the viewpoint of easy viewing, or may be a different resolution from the viewpoint of reducing the amount of communication data. In this embodiment, the resolution of each image constituting the monitoring image K at a certain timing may be different, but it is possible to simultaneously reproduce the monitoring images K of a plurality of moving images.

Next, the operation of the monitoring system 1 according to this embodiment will be described. 18 is a flowchart showing the operation on the monitoring terminal device 10 side, and FIGS. 19A, 19B, and 19C are flowcharts showing the operation on the central monitoring device 20 side.

As shown in FIG. 18, in the monitoring terminal device 10, the surrounding video and the indoor video are acquired from the in-vehicle camera 11 at a predetermined time interval (one routine shown in FIG. 18), and a monitoring image is obtained by the image processing device 12. Conversion is performed (step ST1). Further, the current position information of the passenger car V on which the monitoring terminal device 10 is mounted is detected from the position detection device 15 having GPS (step ST2).

In step ST3, it is determined whether or not the report button 16 for reporting the abnormality is pressed. If the report button 16 is pressed, the process proceeds to step ST4, and the monitoring image acquired in step ST1 and the acquired in step ST2. The positional information is associated with the CPU time information, and these are transmitted as monitoring information to the central monitoring device 20 via the communication device 13 and the telecommunications network 30 together with the abnormality information indicating that an abnormality has occurred. As a result, the occurrence of an abnormality related to security such as an accident or a crime is automatically transmitted to the central monitoring device 20 together with the position information of the passenger car V and the monitoring image around the passenger car V, thereby further strengthening the monitoring in the city. Will be. In this example, the monitoring image and the position information are acquired in the first steps ST1 and ST2, but the monitoring image and the position information may be acquired at a timing between steps ST3 and ST4.

Returning to step ST3, if the report button 16 is not pressed, the process proceeds to step ST5, where it communicates with the central monitoring device 20 and obtains a control command.

Subsequently, in step ST6, the monitoring terminal device 10 determines whether or not a monitoring information transmission command has been acquired from the central monitoring device 20, and proceeds to step ST7 if a monitoring information transmission command has been acquired. In step ST7, it is determined whether or not the acquired monitoring information transmission command includes a command for generating a monitoring image in which images of different resolutions are mixed under a predetermined composition condition. When the monitoring information transmission command includes a command for generating a monitoring image in which images of different resolutions are mixed under a predetermined composition condition, the process proceeds to step ST8 and the resolution is changed. The resolution change command is transmitted to the image processing apparatus 12 that performs image editing processing according to the resolution or the camera 11 that has a resolution setting function. If the acquired monitoring information transmission command does not include a command for generating a monitoring image in which images of different resolutions are mixed under a predetermined composition condition, the first resolution is relatively high without changing the resolution. A monitoring image is generated. That is, when the composition information is included in the monitoring information transmission command, the monitoring image is generated by mixing the second images having the relatively low second resolution.

After changing the resolution setting, the process proceeds to step ST9, where the control device 14 generates a first image of the first resolution and a second image of the second resolution according to the knitting conditions of the monitoring information transmission command, and the first image at a ratio according to the knitting conditions And the second image. In the present embodiment, the “ratio at which the first image and the second image are mixed” in the composition condition is adjusted by the interval, frequency, or cycle at which the first image and the second image are generated and transmitted. That is, the mixing ratio of the first image and the second image in the composition condition is the number of the first images of the first resolution generated per unit time or the number of the first images transmitted per unit time. It can be adjusted by the ratio of the number of second images of the second resolution generated per unit time or the number of second images transmitted per unit time.

Further, in the present embodiment, when each camera 11 has a resolution setting function for imaging the surroundings of the vehicle at a predetermined resolution, the control device 14 determines the first resolution or It is possible to capture a vehicle periphery at the second resolution and generate a monitoring image in which the first image and the second image are mixed under a predetermined composition condition.

In the present embodiment, the first image having the first resolution and the second image having the second resolution are generated by the image processing apparatus 12 having a function of processing a captured image captured by the camera 11 into an image having a predetermined resolution. Can be done. The control device 14 performs image processing on a monitoring image in which a first image having a first resolution and a second image having a second resolution lower than the first resolution are mixed according to a knitting condition according to the knitting condition of the monitoring information transmission command. The device 12 is generated.

Then, the control device 14 transmits monitoring information including the monitoring image, time information, and position information according to the composition condition to the central monitoring device 20 according to the monitoring information transmission command. The monitoring information may include the moving speed of the moving object V as necessary. When a storage command is included in the monitoring information transmission command, monitoring information such as a monitoring image, position information, and time information is stored in the memory of the monitoring terminal device 10.

Returning to step ST6, when the monitoring information transmission command is not acquired from the central monitoring device 20, the process proceeds to step ST10, where it is determined whether or not the passenger vehicle V exists in a predefined priority monitoring area. When the passenger car V is present in a pre-defined priority monitoring area, monitoring information including a monitoring image is transmitted. In this example, the monitoring information including the monitoring image that does not change the communication speed is transmitted from the viewpoint that a detailed monitoring image is preferable if it is within the priority monitoring area. You may perform between ST10 and step ST12. On the other hand, if the monitoring information transmission command is not acquired and is not the priority monitoring area, the process proceeds to step ST11, and monitoring information not including the monitoring image, that is, time information and position information is transmitted to the central monitoring device 20.

Next, processing on the central monitoring device 20 side will be described based on FIGS. 19A to 19C.
In step ST11 of FIG. 19A, position information and time information are acquired from all the passenger cars V and stored in the database 26. The database 26 stores monitoring information including monitoring images, position information, and time information acquired from the passenger car V (monitoring terminal device 10) in association with the position information. That is, if position information is designated, a series of monitoring information can be called. The monitoring information can include a mobile body ID (monitoring terminal device ID) for specifying the monitoring terminal device 10. The mobile object ID may be the address of the communication device 13 of the monitoring terminal device 10.

In step ST12, based on the position information acquired in step ST11, the position of the passenger car V is displayed by superimposing dots on the map information of the map database displayed on the display 24 as shown in the upper left of FIG. By looking at this map information, it is possible to see at a glance at which position the moving body V carrying the monitoring terminal device 10 is traveling. In other words, it is possible to identify the moving object V on which the monitoring terminal device 10 existing at the position to be monitored is mounted. Since the position information of the passenger car V is acquired and transmitted at a predetermined timing for each routine in FIG. 11, the supervisor can grasp the current position of the passenger car V in a timely manner.

In step ST13, it is determined whether or not abnormality information notified from the monitoring terminal device 10 of the passenger car V, that is, a notification that an abnormality relating to security such as an accident or a crime has occurred has been received. This abnormality information is output when the passenger of the passenger car V presses the notification button 16 of the monitoring terminal device 10.

If there is abnormality information, the passenger vehicle V for which the abnormality information is output is identified in step ST14, the monitoring image and the time information are received from the monitoring terminal device 10 of the passenger vehicle, and the monitoring image is displayed on the display 24. Further, as shown in the upper left of FIG. 1, highlighting is performed such as changing the color so that the passenger car displayed on the map information can be distinguished from other passenger cars. Thereby, the position where the abnormality has occurred can be visually recognized on the map information, and the abnormality content can be grasped on the display 24.

In the next step ST15, the passenger vehicle V traveling in the vicinity (within a predetermined distance) of the passenger vehicle V that has output the abnormality information is detected, and a monitoring information transmission command including a monitoring image and time information is output to the passenger vehicle V. . As a result, the monitoring information can be acquired from the passenger vehicle V that travels in the vicinity of the passenger vehicle V that has output the abnormality information. Therefore, in addition to the monitoring image from the passenger vehicle V that has output the abnormality information, the abnormality information Can be understood in detail. The monitoring information transmission command can include a composition condition described later.

In step ST16, the position information of the passenger vehicle V that has output the abnormality information is transmitted to emergency vehicles such as police cars, ambulances, and fire engines. In this case, a monitoring image may be attached and transmitted in order to notify the abnormal content. As a result, the emergency vehicle can be dispatched before a report from the site is entered, and a quick response to an accident or crime is possible.

In step ST17, all position information, monitoring images and time information received from the monitoring terminal device 10 are recorded on the recording medium. This record is used to resolve these after an accident or crime. If there is no abnormality information in step ST13, the process proceeds to step ST21 in FIG. 19B without performing the processes in steps ST14 to ST17.

In step ST21, it is determined whether there is an image information transmission command from an emergency passenger car such as a police car, an ambulance, or a fire engine. If an image transmission command is input, the process proceeds to step ST22. In step ST22, it is determined whether or not the passenger car V exists in the area specified by the image transmission command. If the passenger car V exists, the process proceeds to step ST23. In step ST23, a monitoring information transmission command is output to the passenger vehicle V existing in the area specified by the image transmission command. Thereby, the image information from the passenger car V can also be acquired in step ST11 of FIG. 19A in the next routine, and this can be transferred to the emergency passenger car, or the meaning of the transmission command from the emergency passenger car can be grasped. can do. If not corresponding to steps ST21 and ST22, the process proceeds to step ST24 without performing the processes of steps ST21 to ST23.

In step ST24, it is determined whether or not there is a passenger car V in the vicinity of a suspicious location such as a preset frequent crime delay, and if so, the process proceeds to step ST25 to include a monitoring image for the passenger car V. Output monitoring information transmission command. Suspicious areas are streets with poor security and streets. As a result, the monitoring of streets and streets that are suspicious places can be strengthened, and crime prevention can be expected. If the passenger vehicle V does not exist in the region near the suspicious part, the process proceeds to step ST26 without performing the process of step ST22.

In step ST26, it is determined whether or not there is a passenger vehicle V in the vicinity of the priority monitoring position where the priority monitoring object whose details should be monitored can be imaged. If the passenger vehicle V exists in the vicinity of the priority monitoring position, step ST27 is determined. To the passenger vehicle V and outputs a priority monitoring command for requesting transmission of monitoring information including a monitoring image in which the priority monitoring target is expanded. As a result, it is possible to monitor the priority monitoring target in detail, and to effectively detect a suspicious object that causes an incident or an accident in the specified priority monitoring target, so that prevention of crime can be expected. If there is no passenger vehicle V in the vicinity of the priority monitoring position, the process proceeds to step ST28 without performing the process of step ST27.

In step ST28, based on the position information received from each passenger car V, the passenger car V is not traveling within a predetermined time within a predetermined area that is required to be monitored (not limited to the suspicious location and the priority monitoring area). It is determined whether there is a route, and when there is such a route, it is monitored whether there is a passenger vehicle V traveling on the route. If there is a passenger car V traveling on the route most recently, the process proceeds to step ST29, and a monitoring information transmission command including a monitoring image is output to the passenger car V. As a result, it is possible to automatically acquire a monitoring image of a route that is in a region other than the suspicious location or the priority monitoring region and has a small traffic volume of the passenger car V. If there is no route that satisfies the condition of step ST28, the process returns to step ST11 of FIG. 19A without performing the process of step ST29.

The above-described monitoring information transmission command can include a command for requesting transmission of the monitoring image, position information, time information, and moving speed, and can further include an image resolution setting command applied when the monitoring image is created. Can be included.

Next, the process proceeds to the process of FIG. 19C. FIG. 19C is a flowchart showing a procedure of processing relating to generation and transmission of a monitoring information transmission command including a predetermined composition condition. Hereinafter, based on FIG. 19C and FIGS. 20 to 23, the generation and transmission processing of the monitoring information transmission command of this embodiment will be described. 20 to 23 are diagrams for explaining the transmission state of the monitoring image based on each composition condition.

First, in step ST30 of FIG. 19C, the central monitoring device 20 records the monitoring image, the position information, the time information, and the moving speed acquired from each passenger vehicle V in a recording medium in association with the identifier of each moving body V.

Step 31 and subsequent steps are processing related to generation and transmission processing of the next monitoring information transmission command based on information acquired up to the previous time. In step ST <b> 31, the central monitoring device 20 sets the monitor image of the acquired monitoring information on the touch panel display 24 having an input function so that it can be presented to the monitor. In step ST <b> 32, the central monitoring device 20 receives information specifying the moving object V present at the position to be monitored from the monitor. This operation is an operation such as clicking a dot corresponding to the moving object V displayed in the map information shown in the upper left of FIG. Subsequently, in step ST33, the central monitoring apparatus 20 receives an input of a gaze direction that the supervisor wants to gaze at. The gaze direction is input by touching, tapping, or sliding touching the screen of the touch panel display 24. When the monitor touches any point on the monitoring image, the direction of the touched point can be specified as the gaze direction. When the monitor pays attention to the left direction of the moving object V, the left side camera 11b of the monitoring image K performs input for selecting the captured image GSP2 obtained by imaging the area SP2. Specifically, in the monitoring image K shown in FIG. 11, the supervisor touches the point PRa in the area where the captured image GSP4 is presented in the touch panel display 24, so that the right direction of the moving object V is set as the gaze direction. Can be entered.

Referring back to the flowchart of FIG. 19C, in step ST33, the central monitoring device 20 calculates a communication speed in communication (information exchange) with the monitoring terminal device 10 mounted on the identified vehicle V. The communication speed in the present embodiment is the data amount per unit time of information received by the central monitoring device 20 from the monitoring terminal device 10 side. As described above, the communication speed can be determined based on the frame rate related to monitoring image processing in the central monitoring device 20 and the time required from transmission of information to confirmation of transmission completion in the monitoring terminal device 10.

In step ST34, the central monitoring device 20 determines whether or not the communication speed is equal to or lower than a predetermined threshold (whether the frame rate is equal to or lower than the predetermined threshold, the time required from transmission to transmission completion is equal to or higher than the predetermined threshold). Whether or not there is). In step ST34, the central monitoring apparatus 20 determines that the communication speed is equal to or lower than a predetermined threshold (the frame rate is equal to or lower than the predetermined threshold, or the time until information transmission is completed is equal to or higher than the predetermined value). Advances to step ST35. In step ST35, generation of the knitting condition for the monitoring image including images with different resolutions is started. The resolution can be specified for each frame. The knitting conditions in this embodiment include a matter of mixing a first image having a first resolution with a relatively high resolution and a second image having a second resolution lower than the first resolution under a predetermined knitting condition. The composition condition in the present embodiment defines that at least a first image and a second image having different resolutions are included at a predetermined ratio, but images of further different third resolution, fourth resolution to nth resolution are included. Inclusion can be defined at a predetermined rate.

Although not particularly limited, the composition condition of the present embodiment is that the first image of the first resolution that is the resolution of the image to be transmitted in the normal state where the communication speed is not less than or equal to the predetermined threshold is the second lower than the first resolution. A combination of the second images of resolution is defined. By causing the monitoring terminal device 10 to generate a monitoring image according to the knitting conditions, the amount of information transmitted and received can be reduced or maintained.

The composition conditions include the transmission frequency of the first image and the second image transmitted as the monitoring image, the existence ratio of the first image and the second image included in the monitoring image transmitted per unit time, the first image and the first image The transmission order of two images, and the transmission interval (cycle) of the first image and the second image sequentially transmitted according to the transmission order are included.

FIG. 20 is a diagram for explaining the transmission state of the monitoring image based on the first composition condition. As shown in FIG. 20, in the normal process (1) in which the communication speed between the central monitoring device 20 and the monitoring terminal device 10 does not decrease, the monitoring terminal device 10 has a relatively high first resolution. The first image is transmitted to the central monitoring apparatus 20 at timings t0 to t3 set at regular intervals. On the other hand, in the processes (2) and (3) when the communication speed between the central monitoring device 20 and the monitoring terminal device 10 is a predetermined threshold decrease, the monitoring terminal device 10 is relatively A first image with a high first resolution and a second image with a relatively low second resolution are alternately transmitted to the central monitoring device 20 at timings t0 to t3 set at regular intervals. In other words, the composition conditions shown in (2) and (3) of this example indicate that one or more second images of the second resolution are transmitted after a predetermined interval has elapsed after transmitting the first image of the first resolution. Define In the composition condition shown in (2) of this example, since the number of second images transmitted at t1 and t3 is two (640 × 480 pixels × 2), one first image (1280 × 960 pixels × 2) The amount of information to be communicated can be reduced compared to the case shown in (1) where only 1) is transmitted. In the composition condition shown in (3) of this example, since the number of second images transmitted at t1 and t3 is four (640 × 480 pixels × 4), one first image (1280 × 960) is obtained. The amount of information related to communication is the same as in the case of (1) transmitting only (pixels × 1), but in the case of (1), two images are transmitted between t0 and t1 ( In the case of 2), there is an advantage that as many as five images can be transmitted between t0 and t1. That is, a large number of monitoring images can be provided to the monitor while maintaining the amount of information.

As described above, when the communication speed is low, a monitoring image formed by mixing the first image having a high resolution and the second image having a low resolution on the monitoring terminal device 10 mounted on each moving body V is combined. Since transmission is performed to the central monitoring device 20, an increase in the amount of information related to communication can be suppressed while ensuring the transmission frequency of information. That is, when the communication speed is low, the central monitoring device 20 is not detailed but instead of acquiring one detailed image with a large amount of information related to communication. Since the monitoring terminal device 10 is controlled so that many (a plurality of) small images can be acquired, information can be collected in real time without encouraging congestion of the communication line that causes a decrease in communication speed.

Moreover, the central monitoring apparatus 20 of this embodiment can generate a monitoring information transmission command for each monitoring direction with the moving body V as a reference. Specifically, the central monitoring device 20 includes, in the monitoring information transmission command, information that specifies the monitoring direction (front, rear, right side, left side) with respect to the moving body, and the resolution and the monitoring direction in each monitoring direction. The transmission frequency can be associated. The central monitoring device 20 preliminarily stores information for specifying the monitoring direction with respect to the moving object V and information for associating the cameras 11a to 11d installed at a predetermined position of the moving object V and imaging the predetermined monitoring direction. In addition, information for identifying the cameras 11a to 11d can be included in the monitoring information transmission command. Of course, the monitoring direction included in the monitoring information transmission command is stored on the moving body V side in such a manner that the monitoring direction based on the moving body V and information associating the cameras 11a to 11d for imaging the predetermined monitoring direction are stored. Based on the direction, the monitoring terminal device 10 may identify the cameras 11a to 11d targeted by the monitoring information transmission command. The central monitoring device 20 of the present embodiment can create and execute a monitoring information transmission command for each direction around the moving object V.

For example, when the direction in which the monitor wants to monitor is determined in advance, specifically when the traveling direction (front) of the moving object V is monitored, the camera 11a installed at the front portion of the moving object V is specified. And a 1st image and a 2nd image can be produced | generated based on the captured image of this camera 11a. Thereby, it is possible to reduce the amount of information related to communication between the central monitoring device 20 and the monitoring terminal device 10 by adjusting the resolution while acquiring a monitoring image according to the intention of the monitoring person.

FIG. 21 is a diagram for explaining the transmission state of the monitoring image based on the second composition condition. As shown in (1) of FIG. 21, the same knitting condition can be defined for all the directions (front: Fr, rear Rr, right: Right, left: Left) or all the cameras 11a to 11d. That is, in the example of (1), the monitoring terminal device 10 has a relatively high first resolution image and a relatively low first image in all directions (or imaging directions of all the cameras 11a to 11d). The second image of two resolutions are alternately transmitted to the central monitoring device 20 at timings t0 to t3 set at a constant interval f0. On the other hand, in this embodiment, as shown in FIG. 21 (2), different knitting conditions are set for each direction (front: Fr, rear Rr, right: Right, left: Left) or for each camera 11a to 11d. Can be defined. That is, in the example of (2), the monitoring terminal device 10 has a relatively high first in each direction (or the imaging direction of all the cameras 11a to 11d) at timings t0 to t3 set at a constant interval f1. After the first image having the resolution is transmitted once, the second image is transmitted three times. At this time, as shown in the example (2), the first image transmission timing can be shifted in each direction (or the imaging direction of all the cameras 11a to 11d) so as not to overlap.

Under the knitting conditions shown in (1) of FIG. 21, after four first images are transmitted at the timing t0, the image is transmitted until the timing t1, for example, after the elapse of the transmission interval f0 required for transmitting the fourth images. Cannot be sent. On the other hand, under the knitting conditions shown in (2) of the figure, one first image and three second images are transmitted at the timing t0, and then a transmission time f1 shorter than the time from t0 to t1 has elapsed. At the timing of t0.5, one first image and three second images can be transmitted. In this way, the first image and the second image are mixed under a predetermined knitting condition, and the amount of information related to transmission is reduced or reduced, thereby shortening the image transmission interval, so that information can be collected in real time. Can do.

Here, it returns to step ST32 and progresses to step ST40 performed in parallel. In step ST <b> 40, it is determined whether or not the supervisor inputs the gaze direction that the supervisor wants to gaze among the surroundings of the moving body V on which the monitoring terminal device 10 is mounted to the central monitoring device 20 side. If the gaze direction is designated by the supervisor, the process proceeds to step ST36.

In step ST36, when the gaze direction is input, the central monitoring device 20 generates a monitoring information transmission command including a knitting condition according to the gaze direction. Since the gaze direction is a direction that the monitor wants to observe particularly carefully, it is preferable to provide the monitor with a relatively high resolution image for the gaze direction even when the communication speed decreases. For this reason, the central monitoring apparatus 20 of this embodiment produces | generates the knitting conditions according to a gaze direction. Specifically, the central monitoring device 20 is relatively based on the captured image of the camera 11 that captures the gaze direction when a specific gaze direction that the monitor wants to gaze is input around the moving body V. A first image having a high first resolution is generated, a second image having a relatively low second resolution is generated based on a captured image of the camera 11 that captures a direction other than the gaze direction, and a second image having a different resolution is generated. A knitting condition in which one image and the second image are mixed at a predetermined ratio is generated. This composition condition is included in a monitoring information transmission command transmitted to the monitoring terminal device 10.

FIG. 22 is a diagram for explaining a transmission state of the monitoring image based on the third composition condition. As shown in (1) of FIG. 22, the same knitting conditions can be defined for all the directions (front: Fr, rear Rr, right: Right, left: Left) or all the cameras 11a to 11d. That is, in the example shown in (1) of FIG. 22, the monitoring terminal device 10 compares the first image with a relatively high first resolution with respect to all directions (or the imaging directions of all the cameras 11a to 11d), Therefore, the second image of the second resolution having a lower resolution is alternately transmitted to the central monitoring apparatus 20 at timings t0 to t3 set at a constant interval f0. On the other hand, in the present embodiment, as shown in (2) of FIG. 22, the monitoring terminal device 10 captures the front that is the gaze direction for the gaze direction (for example, forward: Fr) designated by the monitor. Based on the captured image at the timing P1 of the camera 11a, the first image of the first resolution is generated, and the captured images at the timing P1 of the

cameras

11b, 11c, and 11d that capture the rear, right side, and left side that are directions other than the gaze direction. The second image of the second resolution is generated based on the above, and the monitoring image obtained by mixing them is transmitted to the central monitoring device 20 at the timing t0. Subsequently, a first image having a first resolution is generated based on a captured image at timing P2 of the camera 11a that captures the front in the gaze direction, and based on a captured image at timing P2 of the camera 11c that captures the rear that is not in the gaze direction. The first image having the first resolution is generated, and further, the second image having the second resolution is generated based on the captured images at the timing P2 of the

cameras

11b and 11d that capture the right side and the left side which are directions other than the gaze direction. The monitoring image in which these are mixed is transmitted to the central monitoring apparatus 20 at timing t0.5.

In this way, a first image having a relatively high first resolution is generated for the gaze direction, and a second image having a relatively low second resolution is generated for the direction other than the gaze direction, thereby monitoring a plurality of directions. While obtaining images uniformly, by giving light weight to the gaze direction and resolutions other than the gaze direction, the amount of information related to communication between the central monitoring device 20 and the monitoring terminal device 10 can be reduced.

After the processing of steps ST40 and ST36, further, in step ST41, the central monitoring apparatus 20 acquires the vehicle speed detected by the vehicle speed sensor 16 of the passenger vehicle V specified by the supervisor via the telecommunication network 30. In step ST42, the central monitoring device 20 determines whether the vehicle speed is equal to or higher than a predetermined threshold value. If the vehicle speed is greater than or equal to the predetermined threshold, the process proceeds to step ST37.

In step ST37, the central monitoring apparatus 20 generates a knitting condition that increases the proportion of the second image having the relatively low second resolution. When the vehicle speed is equal to or higher than the predetermined threshold value, that is, when the moving object V is traveling fast, the image around the vehicle also changes greatly. Since the video around the vehicle changes every moment, it is preferable to provide a large number of images to the monitor at short intervals when the vehicle speed is high even when the communication speed decreases. For this reason, the central monitoring apparatus 20 of this embodiment produces | generates the knitting conditions according to a vehicle speed. Specifically, when the vehicle speed of the moving body V is equal to or higher than a predetermined threshold, the central monitoring device 20 increases the ratio in which the second image in the knitting condition is included. This composition condition is included in a monitoring information transmission command transmitted to the monitoring terminal device 10.

FIG. 23 is a diagram for explaining a transmission state of the monitoring image based on the fourth composition condition. As shown in FIG. 23, in the process (1) in the case where the vehicle speed is less than the predetermined threshold, the monitoring terminal device 10 uses a relatively high first resolution first image and a relatively low second resolution second image. The images are alternately transmitted to the central monitoring device 20 at timings t0 to t3 set at regular intervals so that the images are included at the same rate. On the other hand, in the process (2) in the case where the vehicle speed is equal to or higher than the predetermined threshold, the monitoring terminal device 10 applies the first image of the first resolution at the timing t0 according to the knitting condition in which the ratio of the second image is increased. Then, the second image with the second resolution is transmitted twice at timings t1 and t2, and then the first image with the first resolution is transmitted at timing t3. In (1), the ratio between the first image and the second image is 1: 1, but in (2), the ratio between the first image and the second image is 1: 2.

As described above, when the vehicle speed is equal to or higher than the predetermined threshold, the ratio of the relatively low second resolution second image included in the monitoring image is increased, so that the vehicle speed is equal to or higher than the predetermined threshold. In some cases, by giving a light weight to the ratio of the second image, the amount of information required for communication between the central monitoring device 20 and the monitoring terminal device 10 is reduced, and the monitoring image around the vehicle that varies greatly according to the moving speed Can be provided in real time.

In subsequent step ST38, the central monitoring apparatus 20 generates a monitoring information transmission command including the knitting conditions generated in steps ST35 to ST37. Finally, in step ST39, a monitoring information transmission command including the composition conditions is transmitted to the vehicle previously identified in ST32. This monitoring information transmission command is a command for controlling the next generation of monitoring information.

In step ST34, if the communication speed is equal to or lower than the predetermined threshold (the frame rate is equal to or lower than the predetermined threshold, or the time required from the start of transmission to the completion of transmission is equal to or higher than the predetermined threshold), the previous monitoring information transmission command Therefore, the monitoring terminal device 10 may be processed based on the processing, so that the processing after step ST35 is terminated without performing the processing, and the processing returns to step ST30. Further, when the supervisor does not specify the gaze direction in step ST40, the processing after step ST41 is performed. Furthermore, when the vehicle speed is less than the predetermined threshold value in step ST42, the process proceeds to step ST38.

In step 39, the monitoring information transmission command is transmitted to the specified vehicle V. The monitoring terminal device 10 that has acquired the monitoring information transmission command generates a first image and a second image in order to obtain a monitoring image according to the composition condition. As described above, as a method of acquiring the first image and the second image having different resolutions, the image processing device 12 may be caused to change the resolution of the captured image, or the resolution of the camera 11 may be changed as desired. You may acquire the captured image of the resolution.

By the way, when the image processing apparatus 12 performs a process of processing a captured image captured by the camera 11 into an image with a predetermined resolution, first, a capture process is performed, a size change process is performed, and then a compression process is performed. Process, and finally the transmission process. FIG. 24 shows the time load related to each process when the generation of the first image and the second image is executed only by the image processing apparatus 12, and is distributed to the camera 11 and the image processing apparatus 12. It is a figure which shows the time load concerning each process in a case. In FIG. 24, a temporal load related to each process is indicated by a rectangular block.

As shown in (1) of FIG. 24, when the generation of the first image and the second image is executed only by the image processing device 12, it takes time tu1 to process one captured image, It takes time tq1 to process four captured images. On the other hand, as shown in (2) of FIG. 24, when the generation of the first image and the second image is distributed and executed in the camera 11 and the image processing device 12, one captured image is processed. It takes a time of tu2, and it takes a time of tq2 to process four captured images. Only the image processing device 12 processes a single captured image to generate a first image or a second image, and the time tu1, which is dispersed between the camera 11 and the image processing device 12, is converted into a single captured image. Although the time tu2 required for processing to generate the first image or the second image does not change much, the camera 11 and the image processing device 12 can execute processing of four captured images in parallel. Therefore, the time tq2 required to process the four captured images dispersedly in the camera 11 and the image processing device 12 to generate the first image or the second image is four times only for the image processing device 12. The time tq1 required for processing the captured image to generate the first image or the second image is shorter. As described above, since the resolution change (compression) is performed in the camera 11, the processing time for generating the first image or the second image is shortened. Therefore, the monitoring information including the monitoring image is stored in the central monitoring device 20. The time required to provide the data can also be shortened. As a result, a monitoring image can be provided to the monitoring person in real time.

As described above, the monitoring system of the present embodiment has the following effects.
(1) The monitoring system 1 according to the present embodiment of the present invention provides the first information with a relatively high first resolution to the monitoring terminal device 10 mounted on each mobile body when the communication speed is low. And the monitoring image formed by mixing the second information with the relatively low second resolution are transmitted to the central monitoring device 20, so that an increase in the amount of communication data can be suppressed while ensuring the information transmission frequency. . That is, the central monitoring device 20 can acquire rough information with a small amount of communication data a plurality of times instead of acquiring detailed information with a large amount of communication data once when the communication speed is low. Since the monitoring terminal device 10 is controlled as described above, information can be collected in real time without encouraging the congestion of the communication line that causes a decrease in the communication speed.

(2) Since the central monitoring device 20 of the monitoring system 1 according to the present embodiment of the present invention generates a monitoring information transmission command for each monitoring direction with the moving body V as a reference, for example, the progress of the moving body V in the monitor When there is an intention to monitor the direction (front), the camera 11a installed in the front part of the moving object V is specified, and the first image and the second image are generated based on the captured image of the camera 11a. Can be made. Thereby, it is possible to reduce the amount of information related to communication between the central monitoring device 20 and the monitoring terminal device 10 by adjusting the resolution while acquiring a monitoring image according to the intention of the monitoring person.

(3) The central monitoring device 20 of the monitoring system 1 according to the present embodiment of the present invention is a camera that captures a gaze direction when a specific gaze direction that the monitor wants to gaze is input around the moving body V. A first image having a relatively high first resolution is generated based on the captured image of 11 and a second image having a relatively low second resolution based on the captured image of the camera 11 that captures a direction other than the gaze direction. Since the knitting condition for generating the image is generated, the monitoring image in a plurality of directions can be obtained evenly, and the gaze direction and the resolution other than the gaze direction are given light weight, so that the central monitoring device 20 and the monitoring terminal device 10 are The amount of information related to the communication can be reduced.

(4) The central monitoring device 20 of the monitoring system 1 according to the present embodiment of the present invention is configured such that when the vehicle speed of the moving object V is equal to or higher than a predetermined threshold, a second image having a relatively low second resolution is used as the monitoring image. By increasing the included ratio, the communication between the central monitoring device 20 and the monitoring terminal device 10 is performed by giving a light weight to the ratio of the second image when the vehicle speed is equal to or higher than the predetermined threshold value and below the predetermined threshold value. While reducing the amount of information, it is possible to provide a real-time monitoring image around the vehicle that varies greatly according to the moving speed.

(5) The monitoring terminal device 10 of the monitoring system 1 according to the present embodiment of the present invention provides each camera 11 having a resolution setting function for imaging the surroundings of the vehicle with a predetermined resolution according to the knitting condition of the monitoring information transmission command. The surroundings of the vehicle can be imaged at the resolution or the second resolution, and a monitoring image in which the first image and the second image are mixed under a predetermined composition condition can be generated. Since the processing of four captured images can be executed in parallel by the camera 11 and the image processing device 12, the processing time for generating the first image or the second image is shortened. The time required for providing the monitoring information to be included to the central monitoring device 20 can also be shortened. As a result, a monitoring image can be provided to the monitoring person in real time.

(6) Since the monitoring terminal device 10 of the monitoring system 1 according to the present embodiment of the present invention can cause the image processing device 12 to generate the first image having the first resolution and the second image having the second resolution. The actions and effects (1) to (5) above can be achieved.

(7) When the monitoring information transmission command acquired from the central monitoring device 20 does not include a predetermined composition condition, the monitoring terminal device 10 of the monitoring system 1 according to the present embodiment of the present invention has the first resolution first. Since the monitoring image of the image is generated and transmitted to the central monitoring device 20, when the communication speed is higher than the predetermined threshold, the monitoring image having a relatively high resolution is transmitted to the monitoring person. Can monitor the city based on detailed monitoring images.

(8) Even when the monitoring method of the present embodiment of the present invention is used, the same effect as the monitoring system 1 can be obtained and the same effect can be obtained.

It should be noted that all the embodiments described above are described for easy understanding of the present invention, and are not described for limiting the present invention. Therefore, each element disclosed in the above embodiment is intended to include all design changes and equivalents belonging to the technical scope of the present invention.

In this specification, the monitoring system 1 including the monitoring terminal device 10 and the central monitoring device 20 will be described as an example of the monitoring system including the monitoring terminal device and the central monitoring device according to the present invention. It is not limited to this.

Further, in the present specification, as one aspect of the central monitoring device including the communication speed calculation unit, the command generation unit, and the command transmission unit according to the present invention, a communication speed calculation function, a command generation function, and a command transmission function The central monitoring device 20 including the control device 21, the image processing device 22, the communication device 23, the display 24, and the input device 25 will be described as an example. However, the present invention is not limited to this.

Moreover, in this specification, as one aspect | mode of the monitoring terminal device provided with the monitoring image generation means and monitoring information transmission means which concern on this invention, the control apparatus 14 which has a monitoring image generation function and the monitoring information transmission function, The monitoring terminal device 10 including the cameras 11a to 11d, the image processing device 12, the communication device 13, the position detection device 15, and the vehicle speed sensor 16 will be described as an example, but is not limited thereto.

In the above-described embodiment, the position information of the passenger car V and the monitoring images from the cameras 11a to 11d are acquired. However, in combination with the monitoring image from the fixed camera 11f installed in the city shown in FIG. You may get it. Moreover, as for the passenger car V which acquires a positional information and a monitoring image, it is desirable to use the taxi V1 and bus | bath which drive | work a predetermined area | region as shown in FIG. 1, but even if it uses private car V2 and emergency car V3 Good.

DESCRIPTION OF SYMBOLS 1 ... Monitoring system 10 ... Monitoring

terminal device

11, 11a-11d ... Camera 11f ... Fixed street camera 12 ... Image processing device 13 ... Communication device 14 ... Control device 15 ... Position detection device 16 ... Vehicle speed sensor 20 ... Central monitoring device 21 ... Central control device 22 ... Image processing device 23 ... Communication device 24 ... Display 25 ... Input device 30 ... Telecommunications network V, V1, V2, V3 ... Mobile object M ... Projection model S, Sa, Sb, Sc, Sd ... Projection Surfaces R1 to R8 ... view point K ... monitoring image

Claims

A monitoring system comprising a central monitoring device and a monitoring terminal device mounted on each of a plurality of mobile bodies and capable of transmitting and receiving information via the central monitoring device and wireless communication,
The central monitoring device is
A communication speed calculating means for calculating a communication speed when exchanging information with the monitoring terminal device;
When the calculated communication speed is equal to or lower than a predetermined threshold, the transmitted information includes a first image having a first resolution with a relatively high resolution and a second image having a second resolution lower than the first resolution. A command generation means for generating a monitoring image in which two images are mixed under a predetermined composition condition and generating a monitoring information transmission command for transmitting monitoring information including the generated monitoring image to the central monitoring device;
Command transmission means for transmitting the generated monitoring information transmission command to the selected monitoring terminal device,
The monitoring terminal device
A camera that is mounted on each of the movable bodies and images different directions around the movable body;
Monitoring image generation means for generating the monitoring image according to the predetermined composition condition included in the monitoring information transmission command acquired from the central monitoring device;
A monitoring system comprising: monitoring information transmitting means for transmitting monitoring information including the generated monitoring image to the central monitoring device in accordance with the monitoring information transmission command.
The monitoring system according to claim 1, wherein the command generation means of the central monitoring device generates the monitoring information transmission command for each monitoring direction with respect to the moving body.
The command generation unit of the central monitoring device is generated based on a captured image of a camera that captures the gaze direction when a specific gaze direction that the monitor wants to gaze is input from around the moving body. A monitoring image obtained by mixing the first image of the first resolution and the second image of the second resolution generated based on the captured image of the camera that captures a direction other than the gaze direction under a predetermined composition condition. The monitoring system according to claim 1, wherein a monitoring information transmission command including a command for generating the monitoring information is generated.
The monitoring information includes a vehicle speed of a vehicle on which the monitoring terminal device is mounted,
The command generation means of the central monitoring device includes monitoring information including a command to increase the ratio of the second information included in the information when the vehicle speed included in the monitoring information is equal to or higher than a predetermined threshold. The monitoring system according to any one of claims 1 to 3, wherein a transmission command is generated.
Each camera of the monitoring terminal device has a resolution setting function for imaging the surroundings of the vehicle at a predetermined resolution,
In accordance with the monitoring information transmission command, the monitoring image generation means causes the camera to capture an image at the first resolution included in the composition condition of the monitoring information transmission command, and at the second resolution included in the composition condition. Imaging the camera and generating the monitoring image in which the first image of the first resolution and the second image of the second resolution are mixed under a predetermined knitting condition in accordance with the knitting condition of the monitoring information transmission command The monitoring system according to any one of claims 1 to 4, wherein:
The monitoring image generation means of the monitoring terminal device includes an image processing function for processing a captured image captured by the camera into an image with a predetermined resolution,
In accordance with the monitoring information transmission command, the monitoring image generation means includes the first image of the first resolution included in the composition condition of the monitoring information transmission command and the second image of the second resolution lower than the first resolution. The monitoring system according to any one of claims 1 to 4, characterized in that the monitoring image is generated by mixing with a predetermined knitting condition.
When the monitoring information transmission command acquired from the central monitoring device does not include the predetermined knitting condition, the monitoring image generation unit of the monitoring terminal device has a first resolution with a relatively high first resolution. Generate a surveillance image of the image,
The monitoring system according to any one of claims 1 to 6, wherein the monitoring information transmitting unit transmits monitoring information including the generated monitoring image to the central monitoring device at a predetermined period.
When the communication speed when the central monitoring device and the monitoring terminal device mounted on the mobile body exchange information is equal to or lower than a predetermined threshold, the transmitted information includes a first resolution having a relatively high resolution. Generating a monitoring image obtained by mixing the first image of the first image and a second image having a second resolution lower than the first resolution under a predetermined composition condition, and monitoring information including the generated monitoring image is generated in the center. Generating a monitoring information transmission command to be transmitted to the monitoring device;
Transmitting the generated monitoring information transmission command to the selected monitoring terminal device, and transmitting the monitoring information acquired via the wireless communication based on a captured image captured by a camera provided on the moving body A monitoring method comprising: acquiring, via the wireless communication, monitoring information including the monitoring image generated by the monitoring terminal device according to the predetermined composition condition included in the command.