WO2023276050A1 - 画像処理システム及び撮像装置 - Google Patents
画像処理システム及び撮像装置 Download PDFInfo
- Publication number
- WO2023276050A1 WO2023276050A1 PCT/JP2021/024775 JP2021024775W WO2023276050A1 WO 2023276050 A1 WO2023276050 A1 WO 2023276050A1 JP 2021024775 W JP2021024775 W JP 2021024775W WO 2023276050 A1 WO2023276050 A1 WO 2023276050A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- image
- processing
- image processing
- unit
- result
- Prior art date
Links
- 238000012545 processing Methods 0.000 title claims abstract description 323
- 238000003384 imaging method Methods 0.000 title claims abstract description 38
- 238000007781 pre-processing Methods 0.000 claims description 34
- 230000005540 biological transmission Effects 0.000 claims description 8
- 238000012544 monitoring process Methods 0.000 abstract description 44
- 238000004891 communication Methods 0.000 abstract description 29
- 238000000034 method Methods 0.000 description 22
- 238000001514 detection method Methods 0.000 description 20
- 238000010586 diagram Methods 0.000 description 13
- 230000006399 behavior Effects 0.000 description 6
- 230000006870 function Effects 0.000 description 4
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/183—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a single remote source
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/94—Hardware or software architectures specially adapted for image or video understanding
- G06V10/95—Hardware or software architectures specially adapted for image or video understanding structured as a network, e.g. client-server architectures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/61—Control of cameras or camera modules based on recognised objects
- H04N23/611—Control of cameras or camera modules based on recognised objects where the recognised objects include parts of the human body
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30242—Counting objects in image
Definitions
- the present disclosure relates to an image processing system and an imaging device.
- Patent Document 1 discloses a method of distributing the processing load to other devices connected to the surveillance camera in order to make up for the lack of processing capability of the surveillance camera.
- part of the process is transferred to another device, and the part of the process is executed by the other device.
- the processing load of the device that executes part of the process increases, and the processing load of the device that was originally supposed to execute the part of the process has a margin, which causes unnecessary migration. I had a problem.
- An image processing system is an image processing system including an imaging device and a terminal device, wherein the imaging device captures an image and generates image data representing the image; a division processing unit that divides the image into a processed image and a target image when the processing load of image processing performed on the image is greater than a predetermined load; a first image processing unit that performs the image processing; and a first image processing result that is a result of the image processing performed on the processed image and target image data that indicates the target image to the terminal device.
- a transmitting unit for transmitting, the terminal device comprising: a receiving unit for receiving the first image processing result and the target image data; and performing the image processing on the target image indicated by the target image data.
- An imaging device includes an imaging unit that captures an image and generates image data representing the image; a division processing unit that divides the image into a processed image and a target image, an image processing unit that executes the image processing on the processed image, and a and a transmission unit configured to transmit an image processing result, which is a result of the image processing, and target image data representing the target image to a terminal device.
- FIG. 1 is a block diagram schematically showing the configuration of a monitoring camera system, which is an image processing system according to Embodiments 1 and 2;
- FIG. 3 is a block diagram schematically showing the configuration of a division processing unit according to Embodiment 1;
- FIG. (A) and (B) are block diagrams showing hardware configuration examples.
- FIG. 4 is a schematic diagram for explaining specific person recognition processing; 4 is a flow chart showing the operation of the surveillance camera in Embodiment 1.
- FIG. It is a schematic diagram showing an example of an image.
- FIG. 4 is a flowchart showing the operation of the terminal device according to Embodiment 1;
- FIG. 4 is a flowchart showing the operation of the terminal device according to Embodiment 1;
- FIG. 4 is a flowchart showing the operation of the terminal device according to Embodi
- FIG. 11 is a block diagram schematically showing the configuration of a division processing unit according to Embodiment 2;
- FIG. 11 is a schematic diagram for explaining a second example of dividing an image;
- FIG. 10 is a schematic diagram showing an image divided in Embodiment 2;
- FIG. 11 is a block diagram schematically showing the configuration of a division processing unit according to Embodiment 2;
- FIG. 11 is a schematic diagram for explaining a second example of dividing an image;
- FIG. 10 is a schematic diagram showing an image divided in Embodiment 2;
- FIG. 1 is a block diagram schematically showing the configuration of a monitoring camera system 100, which is an image processing system according to Embodiment 1.
- the surveillance camera system 100 includes a surveillance camera 110 as an imaging device and a terminal device 140 .
- the monitoring camera 110 and the terminal device 140 are connected to the network 101 , and the image data of the image captured by the monitoring camera 110 and the result of the image processing executed by the monitoring camera 110 are sent to the terminal device 140 . Control information and the like are also sent from the terminal device 140 to the monitoring camera 110 .
- the surveillance camera 110 photographs the surroundings where it is installed, performs predetermined image processing, or performs image processing according to the photographed image or according to instructions from the terminal device 140, and displays the photographed image on the terminal device 140. Image data and image processing results are transmitted.
- the image processing result is, for example, coordinate information indicating a rectangular area including a person included in the image, or an estimation result of an object appearing in the image.
- the surveillance camera 110 may be installed at a location away from the terminal device 140 .
- the monitoring camera 110 includes an imaging unit 111, a division processing unit 112, an image processing unit 113, a storage unit 114, and a communication unit 115.
- the imaging unit 111 captures an image and generates image data representing the image.
- the imaging unit 111 includes an imaging device that captures an image of the surroundings and an A/D conversion unit that converts the image into image data.
- the image data is given to the division processing section 112 .
- the division processing unit 112 By analyzing the image data from the imaging unit 111, the division processing unit 112 identifies images to be processed by the surveillance camera 110 according to the processing load when performing image processing on the image data.
- the division processing unit 112 regards the image as a processed image.
- a processed image is an image processed by the surveillance camera 110 .
- the target image is an image processed by the terminal device 140, and is a remaining image obtained by dividing the processed image from the image represented by the image data.
- the certain load may be a load that can be allocated to image processing among all the processing that the monitoring camera 110 is supposed to perform. It may be the load calculated from time to time from the total number of processes being executed in the .
- the division processing unit 112 passes the image data from the imaging unit 111 to the image processing unit 113 without dividing the image data.
- image processing unit 113 executes image processing on the image represented by the image data
- communication unit 115 transmits the result of the image processing executed on the image represented by the image data. It will be transmitted to the terminal device 140 .
- the division processing unit 112 Upon receiving the image processing result from the image processing unit 113 , the division processing unit 112 generates image processing result data indicating the image processing result, and causes the communication unit 115 to transmit the image processing result data to the terminal device 140 . Further, when the division processing unit 112 has a heavy processing load and divides an image, the division processing unit 112 performs target image data indicating a target image, which is a remaining image obtained by dividing a processed image from an image indicated by image data, and image data. It generates processing instruction data including image processing content data indicating the processing content, and causes the communication unit 115 to transmit it to the terminal device 140 . Note that if the image processing to be executed is predetermined, the image processing content data indicating the content of the image processing need not be transmitted to the terminal device 140 .
- FIG. 2 is a block diagram schematically showing the configuration of the division processing section 112.
- the division processing unit 112 includes a preprocessing unit 120 , a load determination unit 121 , a division area control unit 122 and an image division unit 123 .
- the preprocessing unit 120 performs preprocessing necessary for the image processing unit 113 to perform image processing on the image represented by the image data from the imaging unit 111, and preprocessing results, which are the results of the preprocessing, are performed. is passed to the load determination unit 121 .
- the result of preprocessing here is used to determine the processing load of the image processing.
- the load determination unit 121 determines whether or not the processing load, which is the load when image processing is performed, is greater than a predetermined load. For example, when the image capturing unit 111 captures an image including one or more subjects, the load determination unit 121 determines that the processing load is greater than a predetermined load.
- the division area control unit 122 determines that the processing load is greater than a predetermined load, it determines how to divide the image represented by the image data. Then, the division area control section 122 instructs the image dividing section 123 to divide according to the determination.
- the division instruction includes a division method indicating how to divide the image.
- the divided region control unit 122 determines to divide the image represented by the image data into the processed image and the target image.
- the divided area control unit 122 determines the processed image to be divided from the image so that the image processing performed on the processed image is completed within a predetermined time.
- the divided area control unit 122 divides the image so that the number of subjects included in the processed image is a predetermined number among one or more subjects included in the image represented by the image data. Determines the image to be processed.
- the image division unit 123 processes image data according to instructions from the division area control unit 122 .
- the image dividing unit 123 divides the image represented by the image data into the processed image and the target image according to the instruction, and divides the processed image into a processed image and a target image. and target image data representing the target image.
- the generated target image data is given to the image processing unit 113 .
- the image dividing unit 123 gives the image data from the imaging unit 111 to the image processing unit 113 .
- the image processing unit 113 executes image processing on the processed image indicated by the processed image data given from the division processing unit 112 or the image indicated by the image data from the imaging unit 111 .
- the image processing may be a process performed in one step or a process performed in a plurality of steps.
- the image processing unit 113 gives an image processing result, which is the result of the image processing, to the division processing unit 112 .
- the image processing unit 113 is also called a first image processing unit, and the result of image processing performed on the processed image by the image processing unit 113 is also called a first image processing result.
- the storage unit 114 stores programs and data necessary for processing in the surveillance camera 110 .
- the communication unit 115 communicates with the terminal device 140 via the network 101 .
- the communication unit 115 functions as a transmission unit that transmits the first image processing result, which is the result of image processing performed on the processed image, and the target image data to the terminal device 140 .
- the communication unit 115 also functions as a transmission unit that transmits to the terminal device 140 the result of image processing performed on the image indicated by the image data from the imaging unit 111 .
- Part or all of the division processing unit 112 and the image processing unit 113 described above execute a memory 10 and a program stored in the memory 10, as shown in FIG. It can be configured by a processor 11 such as a CPU (Central Processing Unit) that Such a program may be provided through a network, or recorded on a recording medium and provided. That is, such programs may be provided as program products, for example.
- a processor 11 such as a CPU (Central Processing Unit) that
- Such a program may be provided through a network, or recorded on a recording medium and provided. That is, such programs may be provided as program products, for example.
- part or all of the division processing unit 112 and the image processing unit 113 are, for example, as shown in FIG. , ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array). As described above, the division processing unit 112 and the image processing unit 113 can be configured by a processing circuit network.
- ASIC Application Specific Integrated Circuit
- FPGA Field Programmable Gate Array
- the storage unit 114 can be realized by a storage device such as a volatile or nonvolatile memory.
- the communication unit 115 can be implemented by a communication device such as a NIC (Network Interface Card).
- the terminal device 140 is a device that records image data transmitted from the monitoring camera 110 via the network 101 in a storage medium (not shown in FIG. 1) and displays images to the user using a monitor. is. Further, the terminal device 140 receives the target image data and the image processing content data transmitted from the monitoring camera 110, and performs the processing indicated by the image processing content data on the received target image data. Run.
- the terminal device 140 includes a communication section 141, an image processing section 142, a storage section 143, and a management section 144.
- the communication unit 141 communicates with the surveillance camera 110 via the network 101 .
- the communication unit 141 functions as a receiving unit that receives a first image processing result, which is the result of image processing performed on the processed image in the surveillance camera 110, and target image data.
- the image processing unit 142 executes predetermined processing on image data.
- the predetermined processing includes, in addition to the processing scheduled to be executed by the terminal device 140 , the processing content indicated by the image processing content data transmitted from the surveillance camera 110 .
- the image processing unit 142 executes image processing on the target image indicated by the target image data.
- the image processing unit 142 is also called a second image processing unit, and the result of image processing performed on the target image is also called a second image processing result.
- the storage unit 143 stores programs and data necessary for processing in the terminal device 140.
- the management unit 144 manages the operation of the terminal device 140 as a whole.
- the overall operation consists of recording the image data received by the communication unit 141 in an appropriate storage medium (not shown), instructing the user to display the data, and monitoring the image data by the communication unit 141.
- processing instruction data including target image data and image processing content data is received from the camera 110, the image processing unit 142 instructs the received target image data to perform image processing indicated by the image processing content data. Including.
- the management unit 144 also stores a first image processing result, which is the result of image processing performed on the processed image in the monitoring camera 110, and a second image processing result, which is the result of image processing performed on the target image. It functions as an acquisition unit that acquires one result by integrating the image processing result.
- the image processing result integrated into one result can be handled as a result equivalent to the image processing performed by the image processing unit 113 without dividing the image data captured by the imaging unit 111 .
- Some or all of the image processing unit 142 and the management unit 144 described above also execute the memory 10 and the program stored in the memory 10, as shown in FIG. It can be configured with a processor 11 such as a CPU.
- a program may be provided through a network, or recorded on a recording medium and provided. That is, such programs may be provided as program products, for example.
- the terminal device 140 can be realized by a so-called computer.
- part or all of the image processing unit 142 and the management unit 144 may be, for example, as shown in FIG. It can also be configured with a processing circuit 12 such as an ASIC or FPGA. As described above, the image processing unit 142 and the management unit 144 can be configured by a processing circuit network.
- the storage unit 143 can be realized by a storage device such as a volatile or nonvolatile memory.
- the communication unit 141 can be implemented by a communication device such as a NIC.
- the monitoring process is, for example, the specific person recognition process P1 shown in FIG.
- the specific person recognition process P1 includes person detection P1-1 for detecting a person, face position estimation P1-2 for estimating the position of the face of the detected person, and face authentication P1- for recognizing the face of the detected person. 3, database collation P1-4 for collating the face recognized from the detected person with the face stored in the database, and whether or not the detected person is a specific person according to the collation result. and person judgment P1-5 for judging whether or not.
- the specific person recognition process P1 is a process of extracting a person's face from image data and determining whether or not there is a corresponding person in a database held in advance. Although specific person recognition processing P1 will be described below as an example of monitoring processing, the present embodiment is not limited to such an example.
- person detection P1-1 is post-processing. This is preprocessing for estimating the processing load of image processing (P1-2 to P1-5).
- the pre-processing of the present embodiment is not limited to the person detection P1-1, and may be any processing as long as the processing load of image processing can be determined.
- FIG. 5 is a flow chart showing the operation of surveillance camera 110 according to the first embodiment.
- the imaging unit 111 generates image data by converting a signal obtained by the imaging device into image data (S10).
- the imaging unit 111 passes the image data to the division processing unit 112 .
- the preprocessing unit 120 of the division processing unit 112 executes person detection P1-1 as preprocessing on the image data from the imaging unit 111 (S11). For example, when the image data indicates the image IM1 shown in FIG. 6, the preprocessing unit 120 detects the number of persons and their positions as a result of executing the person detection P1-1. In the example of FIG. 6, four people and their positions are detected.
- the person detection P1-1 generally widely known techniques such as person detection using HOG (Histograms of Oriented Gradients) feature amount or person detection using Haar-like feature amount are used. should be used.
- the preprocessing unit 120 divides the image IM1 into four regions R1 to R4, which are a plurality of predetermined regions, and divides each of the regions R1 to R4 into a person and a human. Detect its position. Then, the preprocessing unit 120 identifies the number of people in the image IM1 based on the persons detected in each of the areas R1 to R4.
- the load determination unit 121 determines whether or not the processing load of image processing on the image data is larger than a certain threshold based on the detection result of the preprocessing unit 120 (S12).
- a certain threshold value is assigned as image processing to image data during a predetermined period of time during which image processing is performed on image data, or from the entire processing performed by surveillance camera 110 when the image processing is performed.
- the determination here is to determine whether or not the image processing for the image data will be completed within a predetermined time. Specifically, it is determined whether or not the number of persons detected by the preprocessing unit 120 is greater than a predetermined threshold value. For example, as shown in FIG.
- step S12 it may be determined whether or not the density of people in any of regions R1 to R4 divided from image IM1 is higher than a predetermined threshold. . If the processing load is equal to or less than the threshold (No in step S12), the process proceeds to step S13, and if the processing load is greater than the threshold (Yes in step S12), the process proceeds to step S14.
- step S13 it is determined that the image processing will be completed within the predetermined time. It is caused to be given to the processing unit 113 . Then, the image processing unit 113 executes image processing on the image indicated by the image data.
- the image processing unit 113 performs face position estimation P1-2, face authentication P1-3, database collation P1-4, and person determination other than the preprocessing of the specific person recognition processing P1 on the image represented by the image data. Execute P1-5. It is assumed that the database used in database collation P1-4 is stored in storage unit 114.
- the image processing unit 113 supplies the image processing result, which is the execution result of the image processing, to the divided area control unit 122, and the divided area control unit 122 generates image processing result data indicating the image processing result,
- the communication unit 115 is caused to transmit the processing result data to the terminal device 140 .
- step S14 it is determined that the image processing will not be completed within the predetermined time. It is decided to divide the image into the target image, which is the image of other regions, and the respective regions of the processed image and the target image are determined.
- the divided area control unit 122 A processed image may be determined such that the number of persons included in the processed image is equal to or less than a predetermined threshold. Specifically, when the threshold is "one person", the divided area control unit 122 sets the images of the areas R1 and R2 as the processed images, and sets the areas R3 and R4 as the target images. good. Although the images of the regions R2 and R3 may be processed images, it is assumed here that priority is given to horizontal regions over vertical regions.
- the divided area control unit 122 identifies the area with the fewest number of people as the determination area, and determines whether or not the number of persons in the determination area is equal to or less than the threshold. Then, when the number of persons in the determination area is equal to or less than the threshold, the divided area control unit 122 expands the determination area by adding an area with few people among the areas adjacent to the determination area to the determination area. Then, similarly, it is determined whether or not the number of persons in the determination area is equal to or less than the threshold. By repeating the above process, the divided area control unit 122 may set the largest image range in which the number of people included in the determination area is equal to or less than the threshold as the processed image.
- the image dividing unit 123 divides the image represented by the image data from the imaging unit 111 into a processed image and a target image according to the determination by the divided region control unit 122, and divides the processed image data and the target image into the processed image and the target image.
- Target image data representing the target image is generated (S15).
- the image dividing unit 123 uses the image shown in FIG. 8A as the processed image and the image shown in FIG. 8B as the target image.
- the processed image data is given to the image processing unit 113 .
- the divided area control unit 122 causes the communication unit 115 to transmit processing instruction data including target image data indicating the target image and image processing content data indicating the content of the image processing to the terminal device 140 (S16).
- the divided region control unit 122 performs the number and positions of persons obtained as a result of executing the person detection P1-1 in the preprocessing unit 120, and the face position estimation P1-2 other than the preprocessing in the specific person recognition processing P1. , face authentication P1-3, database collation P1-4, and person determination P1-5.
- the processing content may be a program describing the processing to be executed, or may be a symbol or character string designating the corresponding program if the terminal device 140 holds a program describing the processing to be executed.
- the image processing unit 113 executes image processing on the processed image data, and provides the divided region control unit 122 with the image processing result, which is the processing result.
- the divided area control unit 122 generates image processing result data indicating the image processing result, and causes the communication unit 115 to transmit it to the terminal device 140 (S17).
- FIG. 9 is a flow chart showing the operation of the terminal device 140 according to the first embodiment. Here, operations performed by the terminal device 140 when the image is divided by the monitoring camera 110 are shown.
- the communication unit 141 receives processing instruction data from the monitoring camera 110 and gives the processing instruction data to the image processing unit 142 (S20).
- the image processing unit 142 specifies the number and positions of persons indicated by the image processing content data included in the processing instruction data for the target image indicated by the target image data included in the processing instruction data.
- person recognition processing P1 face position estimation P1-2, face recognition P1-3, database collation P1-4, and person determination P1-5 other than preprocessing are executed, and an image processing result, which is the result of image processing on the target image. is obtained (S21).
- the image processing result of the target image is given to the management unit 144 . It is assumed that the database for performing database collation P1-4 is stored in storage unit 143.
- the communication unit 141 also receives image processing result data from the monitoring camera 110 and provides the image processing result data to the management unit 144 . Then, the management unit 144 combines the image processing result indicated by the image processing result data from the communication unit 141 and the image processing result of the target image, and integrates them into one image processing result.
- the image processing result integrated into one result can be treated as a result equivalent to the result of the specific person recognition processing P1 for the original image data (S22).
- the monitoring camera system 100 can appropriately perform processing that can be executed by the monitoring camera 110 and then distribute the load with the terminal device 140 .
- the surveillance camera by dividing the image data according to the processing capability of the surveillance camera 110, the surveillance camera can The processing performed by 110 can be appropriately assigned.
- the image data can be divided into areas according to the processing load that can be executed by the monitoring camera 110, so that the processing capacity of the monitoring camera 110 can be effectively used.
- the area of the image data to be executed by the surveillance camera 110 is processed in real time without any delay in network transmission, which is the same as when the image data is not divided. can be executed. As a result, even when performing image processing using the processing result of the previous image, it is possible to continue the processing without delay.
- the area of the image data to be executed by the monitoring camera 110 is not transmitted over the network 101, so that the portion requiring privacy is processed within the monitoring camera 110. settings can be made.
- the monitoring camera system 100 according to the first embodiment since the area of the image data to be executed by the monitoring camera 110 is not transmitted over the network 101, the amount of network transmission is suppressed, and even in an environment where the network bandwidth is not sufficient, high-level image processing can be performed. Image processing can be realized.
- the monitoring camera system 100 since the area of the image data to be executed by the monitoring camera 110 is not transmitted over the network 101, compared with executing all the image processing by the terminal device, Even if the performance is low, advanced image processing can be realized.
- a monitoring camera system 200 which is an image processing system according to the second embodiment, includes a monitoring camera 210 and a terminal device 140.
- FIG. The terminal device 140 of the surveillance camera system 200 according to the second embodiment is the same as the terminal device 140 of the surveillance camera system 100 according to the first embodiment.
- surveillance camera 210 includes imaging unit 111 , division processing unit 212 , image processing unit 113 , storage unit 114 , and communication unit 115 .
- the imaging unit 111, the image processing unit 113, the storage unit 114, and the communication unit 115 of the monitoring camera 210 according to Embodiment 2 are similar to the imaging unit 111, the image processing unit 113, the storage unit 114, and the communication unit 114 of the monitoring camera 110 according to Embodiment 1. It is the same as the part 115 .
- the division processing unit 212 divides the image to be processed by the monitoring camera 210 according to the processing load when executing image processing on the image data.
- the division processing unit 212 divides an image into a region captured near the monitoring camera 110 as a processed image and a region distant from the monitoring camera 110 as a target image. Spread the load. For example, the division processing unit 212 divides the processed image so that a predetermined number of subjects, from among one or more subjects included in the image represented by the image data, are included in the processed image in order from the subject closest to the imaging unit 111. , to split the processed image from the image.
- FIG. 10 is a block diagram schematically showing the configuration of division processing section 212 according to the second embodiment.
- the division processing unit 212 includes a preprocessing unit 120 , a load determination unit 121 , a division area control unit 222 and an image division unit 123 .
- the preprocessing unit 120, the load determination unit 121, and the image dividing unit 123 of the division processing unit 212 in Embodiment 2 are similar to the preprocessing unit 120, the load determination unit 121, and the image division unit 123 of the division processing unit 112 in Embodiment 1. is similar to
- the divided region control unit 222 determines how to divide the image data according to the distance to the person detected by the preprocessing unit 120 . Then, the divided area control section 122 instructs the image dividing section 123 according to the determination.
- the surveillance camera 210 is fixedly installed at a certain place, not carried around. Therefore, the distance to the person in the image captured by surveillance camera 210 can be specified based on the location where surveillance camera 210 is installed. For example, as shown in FIG. 6, when the monitoring camera 210 captures the ground obliquely from above, the distance is closer at the bottom of the image IM1, and the distance is greater at the top of the image IM1. Therefore, the divided area control unit 222 can roughly identify the distance to the person from the position of the person in the image.
- the division area control unit 222 divides the image IM1 from the bottom end of the image IM1 as shown in FIG. By moving the boundary L of , the maximum area including the number of persons that can be processed by the monitoring camera 210 can be used as the processed image, and the rest can be used as the target image.
- the divided area control unit 222 When the number of persons that can be processed by the monitoring camera 210 is, for example, "3", the divided area control unit 222 generates an image IM2 of an area containing three persons, as shown in FIG. is the image to be processed, and the image IM3 of the remaining area thereof is the target image, and it can be decided to divide the image IM1.
- the image dividing unit 123 divides the image IM1 according to such a determination to generate processed image data representing the processed image and target image data representing the target image.
- Embodiments 1 and 2 described above specific person recognition processing P1 is performed as system processing performed by surveillance camera systems 100 and 200, but Embodiments 1 and 2 are limited to such examples. not.
- an eye catch count may be performed as system processing.
- preprocessing performed by the preprocessing unit 120 the same person detection as described above is performed, and as image processing performed by the image processing units 113 and 142, face position estimation for estimating the position of the detected person's face is performed. It is only necessary to perform estimation, face feature amount estimation for estimating the feature amount of the face of the detected person, and face direction detection for detecting the direction of the face of the detected person.
- suspicious behavior analysis may be performed as system processing.
- preprocessing performed by the preprocessing unit 120 the same person detection as described above is performed, and as image processing performed by the image processing units 113 and 142, skeleton detection and detection for detecting the skeleton of the detected person are performed.
- Behavior analysis for analyzing the behavior from the skeleton of the detected person and suspicious behavior detection for detecting suspicious behavior from the detected behavior of the person may be performed.
- abandoned or left-behind detection may be performed.
- preprocessing performed by the preprocessing unit 120 abandoned object detection for detecting an abandoned object is performed.
- a notification process for notifying a predetermined destination such as a center of an object to be estimated and an abandoned object may be performed.
- Abandoned object detection may be performed, for example, by comparison with a previous image.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Signal Processing (AREA)
- Human Computer Interaction (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Oral & Maxillofacial Surgery (AREA)
- General Health & Medical Sciences (AREA)
- Closed-Circuit Television Systems (AREA)
- Alarm Systems (AREA)
Abstract
Description
特許文献1には、監視カメラの処理能力の不足を補うために、処理負荷を監視カメラに接続される他の装置に分散させる方法が開示されている。
次に、図面を用いて、実施の形態を説明する。図面において、同一の部分には、同一の符号が付されている。
また、図面は模式的なものであり、各寸法の比率等は現実のものとは異なる。従って、具体的な寸法等は、以下の説明を参酌して判断されるべきものである。さらに、図面相互間においても互いの寸法の関係又は比率が異なる部分が含まれていることは勿論である。
監視カメラシステム100は、撮像装置としての監視カメラ110と、端末装置140とを備える。
なお、監視カメラ110は、端末装置140から離れた場所に設置されてもよい。
ここで、ある一定の負荷とは、監視カメラ110が行うことになっている処理全体の内、画像処理に割り当てられることのできる負荷であってもよく、画像処理を実行する際に監視カメラ110で実行されている処理の合計から、その時々に応じて算出された負荷であってもよい。
また、分割処理部112は、処理負荷が重く、画像の分割を行った場合には、画像データで示される画像から処理画像を分割した残りの画像である対象画像を示す対象画像データと、画像処理の内容を示す画像処理内容データとを含む処理指示データを生成し、通信部115に、端末装置140へ送信させる。
なお、実行する画像処理が予め決まっている場合には、画像処理の内容を示す画像処理内容データについては、端末装置140に送信しなくてもよい。
分割処理部112は、前処理部120と、負荷判断部121と、分割領域制御部122と、画像分割部123とを備える。
例えば、撮像部111が、一又は複数の被写体が含まれるように画像を撮像している場合に、負荷判断部121は、その一又は複数の被写体の数が閾値よりも多い場合に、処理負荷が予め定められた負荷よりも大きいと判断する。
ここでは、分割領域制御部122は、処理画像に対して実行される画像処理が予め定められた時間内に完了するように、画像から分割する処理画像を決定する。
例えば、分割領域制御部122は、画像データで示される画像に含まれている一又は複数の被写体の内、処理画像に含まれる被写体の数が予め定められた数となるように、画像から分割する処理画像を決定する。
例えば、分割領域制御部122からの指示が分割を行う指示である場合には、画像分割部123は、その指示に従って、画像データで示される画像を処理画像及び対象画像に分割し、処理画像を示す処理画像データ及び対象画像を示す対象画像データを生成する。生成された対象画像データは、画像処理部113に与えられる。
なお、分割領域制御部122からの指示が分割を行わない指示である場合には、画像分割部123は、撮像部111からの画像データを画像処理部113に与える。
通信部115は、ネットワーク101を介して、端末装置140と通信を行う。例えば、通信部115は、処理画像に対して実行された画像処理の結果である第1の画像処理結果及び対象画像データを端末装置140に送信する送信部として機能する。また、通信部115は、撮像部111からの画像データで示される画像に対して実行された画像処理の結果を端末装置140に送信する送信部としても機能する。
以上のように、分割処理部112及び画像処理部113は、処理回路網で構成することができる。
通信部115は、NIC(Network Interface Card)等の通信装置により実現することができる。
例えば、画像処理部142は、対象画像データで示される対象画像に対して画像処理を実行する。ここで、画像処理部142を、第2の画像処理部ともいい、対象画像に対して実行された画像処理の結果を第2の画像処理結果ともいう。
一つの結果に統合された画像処理結果は、撮像部111で撮像された画像データを分割せずに画像処理部113で画像処理を実行した場合と同等の結果として扱うことができる。
以上のように、画像処理部142及び管理部144は、処理回路網で構成することができる。
通信部141は、NIC等の通信装置により実現することができる。
まず、撮像部111は、撮像素子で得られた信号を画像データに変換することで画像データを生成する(S10)。撮像部111は、その画像データを分割処理部112に渡す。
例えば、画像データが、図6に示されている画像IM1を示している場合、前処理部120は、人物検出P1-1を実行した結果として、人物の人数及び位置を検出する。図6の例では、四人の人物と、四人の人物の位置とが検出される。
なお、人物検出P1-1としては、HOG(Histograms of Oriented Gradients)特徴量を用いた人物検出、又は、Haar-like特徴量を用いた人物検出等、一般的に広く知られている公知の技術が使用されればよい。
そして、処理負荷が閾値以下である場合(ステップS12でNo)には、処理はステップS13に進み、処理負荷が閾値よりも大きい場合(ステップS12でYes)には、処理はステップS14に進む。
ここでは、監視カメラ110で画像の分割が行われ場合において、端末装置140が行う動作を示す。
まず、通信部141は、監視カメラ110からの処理指示データを受信し、画像処理部142にその処理指示データを与える(S20)。
一つの結果に統合された画像処理結果は、元の画像データに対する特定人物認識処理P1の結果と同等の結果として扱うことができる(S22)。
実施の形態1に係る監視カメラシステム100によれば、監視カメラ110で実行する画像データの領域は、ネットワーク101で伝送されないため、ネットワークの伝送量が抑えられネットワークの帯域が十分でない環境でも高度な画像処理を実現することができる。
実施の形態1に係る監視カメラシステム100によれば、監視カメラ110で実行する画像データの領域は、ネットワーク101で伝送されないため、画像処理すべてを端末装置で実行することと比較し、端末装置の性能が低くても高度な画像処理を実現することができる。
図1に示されているように、実施の形態2に係る画像処理システムである監視カメラシステム200は、監視カメラ210と、端末装置140とを備える。
実施の形態2に係る監視カメラシステム200の端末装置140は、実施の形態1に係る監視カメラシステム100の端末装置140と同様である。
実施の形態2における監視カメラ210の撮像部111、画像処理部113、記憶部114及び通信部115は、実施の形態1における監視カメラ110の撮像部111、画像処理部113、記憶部114及び通信部115と同様である。
一般的に、特定人物認識処理P1のうち、顔認証P1-3及びデータベース照合P1-4は、処理を実行する画像内に占める画素数が小さいほど認識が困難になる。このため、例えば、処理可能なサイズに拡張処理を行ったり、複数回処理を実行したりすることで、処理の精度を高める必要があり、処理負荷が大きくなる。
例えば、分割処理部212は、画像データで示される画像に含まれている一又は複数の被写体の内、撮像部111に近い被写体から順に予め定められた数の被写体が処理画像に含まれるように、画像から処理画像を分割する。
分割処理部212は、前処理部120と、負荷判断部121と、分割領域制御部222と、画像分割部123とを備える。
実施の形態2における分割処理部212の前処理部120、負荷判断部121及び画像分割部123は、実施の形態1における分割処理部112の前処理部120、負荷判断部121及び画像分割部123と同様である。
例えば、図6に示されているように、監視カメラ210が上方から斜めに地面を撮影している場合、画像IM1の下の方が近い距離となり、画像の上ほど、遠い距離となる。このため、分割領域制御部222は、画像に写っている人物の位置により、その人物までの距離を大まかに特定することができる。
例えば、システム処理として、アイキャッチカウントが行われてもよい。このような場合、前処理部120が行う前処理として、上記と同様の人物検出が行われ、画像処理部113、142が行う画像処理として、検出された人物の顔の位置を推定する顔位置推定、検出された人物の顔の特徴量を推定する顔特徴量推定、及び、検出された人物の顔の向きを検出する顔向き検出が行われればよい。
Claims (9)
- 撮像装置及び端末装置を備える画像処理システムであって、
前記撮像装置は、
画像を撮像して、前記画像を示す画像データを生成する撮像部と、
前記画像に対して実行される画像処理の処理負荷が予め定められた負荷よりも大きい場合に、前記画像を、処理画像と、対象画像とに分割する分割処理部と、
前記処理画像に対して前記画像処理を実行する第1の画像処理部と、
前記処理画像に対して実行された前記画像処理の結果である第1の画像処理結果及び前記対象画像を示す対象画像データを前記端末装置に送信する送信部と、を備え、
前記端末装置は、
前記第1の画像処理結果及び前記対象画像データを受信する受信部と、
前記対象画像データで示される前記対象画像に対して前記画像処理を実行する第2の画像処理部と、
前記第1の画像処理結果と、前記対象画像に対して実行された前記画像処理の結果である第2の画像処理結果とを統合することで、一つの結果を取得する取得部と、を備えること
を特徴とする画像処理システム。 - 前記処理負荷が前記予め定められた負荷以下である場合には、前記第1の画像処理部は、前記画像に対して前記画像処理を実行し、前記送信部は、前記画像に対して実行された前記画像処理の結果を前記端末装置に送信すること
を特徴とする請求項1に記載の画像処理システム。 - 前記対象画像は、前記画像から前記処理画像を分割した残りの画像であること
を特徴とする請求項1又は2に記載の画像処理システム。 - 前記分割処理部は、前記処理画像に対して実行される前記画像処理が予め定められた時間内に完了するように、前記画像から前記処理画像を分割すること
を特徴とする請求項1から3の何れか一項に記載の画像処理システム。 - 前記撮像部は、一又は複数の被写体が含まれるように前記画像を撮像し、
前記分割処理部は、前記一又は複数の被写体の数が閾値よりも多い場合に、前記処理負荷が前記予め定められた負荷よりも大きいと判断すること
を特徴とする請求項1から4の何れか一項に記載の画像処理システム。 - 前記分割処理部は、前記一又は複数の被写体の内、前記処理画像に含まれる被写体の数が予め定められた数となるように、前記画像から前記処理画像を分割すること
を特徴とする請求項5に記載の画像処理システム。 - 前記分割処理部は、前記一又は複数の被写体の内、前記撮像部に近い被写体から順に前記予め定められた数の被写体が前記処理画像に含まれるように、前記画像から前記処理画像を分割すること
を特徴とする請求項6に記載の画像処理システム。 - 前記分割処理部は、前記画像処理を行うために必要な前処理を前記画像に実行した結果により、前記処理負荷が前記予め定められた負荷よりも大きいか否かを判断すること
を特徴とする請求項1から7の何れか一項に記載の画像処理システム。 - 画像を撮像して、前記画像を示す画像データを生成する撮像部と、
前記画像に対して実行される画像処理の処理負荷が予め定められた負荷よりも大きい場合に、前記画像を、処理画像と、対象画像とに分割する分割処理部と、
前記処理画像に対して前記画像処理を実行する画像処理部と、
前記処理画像に対して実行された前記画像処理の結果である画像処理結果及び前記対象画像を示す対象画像データを端末装置に送信する送信部と、を備えること
を特徴とする撮像装置。
Priority Applications (5)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2023531246A JP7483140B2 (ja) | 2021-06-30 | 2021-06-30 | 画像処理システム、撮像装置、端末装置及び画像処理方法 |
PCT/JP2021/024775 WO2023276050A1 (ja) | 2021-06-30 | 2021-06-30 | 画像処理システム及び撮像装置 |
CN202180099794.XA CN117546461A (zh) | 2021-06-30 | 2021-06-30 | 图像处理系统和摄像装置 |
DE112021007910.7T DE112021007910T5 (de) | 2021-06-30 | 2021-06-30 | Bildverarbeitungssystem und bildgebungsvorrichtung |
US18/571,737 US20240119598A1 (en) | 2021-06-30 | 2021-06-30 | Image processing system, imaging device, terminal device, and image processing method |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/JP2021/024775 WO2023276050A1 (ja) | 2021-06-30 | 2021-06-30 | 画像処理システム及び撮像装置 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2023276050A1 true WO2023276050A1 (ja) | 2023-01-05 |
Family
ID=84691665
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2021/024775 WO2023276050A1 (ja) | 2021-06-30 | 2021-06-30 | 画像処理システム及び撮像装置 |
Country Status (5)
Country | Link |
---|---|
US (1) | US20240119598A1 (ja) |
JP (1) | JP7483140B2 (ja) |
CN (1) | CN117546461A (ja) |
DE (1) | DE112021007910T5 (ja) |
WO (1) | WO2023276050A1 (ja) |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2000099690A (ja) * | 1998-09-18 | 2000-04-07 | Toshiba Corp | 人物監視システム |
JP2010136032A (ja) * | 2008-12-04 | 2010-06-17 | Hitachi Ltd | 映像監視システム |
JP2015073191A (ja) * | 2013-10-02 | 2015-04-16 | キヤノン株式会社 | 画像処理システムおよびその制御方法 |
US9158974B1 (en) * | 2014-07-07 | 2015-10-13 | Google Inc. | Method and system for motion vector-based video monitoring and event categorization |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2014102691A (ja) | 2012-11-20 | 2014-06-05 | Toshiba Corp | 情報処理装置、通信機能を持つカメラ、および情報処理方法 |
-
2021
- 2021-06-30 JP JP2023531246A patent/JP7483140B2/ja active Active
- 2021-06-30 CN CN202180099794.XA patent/CN117546461A/zh active Pending
- 2021-06-30 DE DE112021007910.7T patent/DE112021007910T5/de active Pending
- 2021-06-30 US US18/571,737 patent/US20240119598A1/en active Pending
- 2021-06-30 WO PCT/JP2021/024775 patent/WO2023276050A1/ja active Application Filing
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2000099690A (ja) * | 1998-09-18 | 2000-04-07 | Toshiba Corp | 人物監視システム |
JP2010136032A (ja) * | 2008-12-04 | 2010-06-17 | Hitachi Ltd | 映像監視システム |
JP2015073191A (ja) * | 2013-10-02 | 2015-04-16 | キヤノン株式会社 | 画像処理システムおよびその制御方法 |
US9158974B1 (en) * | 2014-07-07 | 2015-10-13 | Google Inc. | Method and system for motion vector-based video monitoring and event categorization |
Also Published As
Publication number | Publication date |
---|---|
JPWO2023276050A1 (ja) | 2023-01-05 |
JP7483140B2 (ja) | 2024-05-14 |
DE112021007910T5 (de) | 2024-05-08 |
US20240119598A1 (en) | 2024-04-11 |
CN117546461A (zh) | 2024-02-09 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP5398341B2 (ja) | 物体認識装置及び物体認識方法 | |
US8938092B2 (en) | Image processing system, image capture apparatus, image processing apparatus, control method therefor, and program | |
JP6555906B2 (ja) | 情報処理装置、情報処理方法、およびプログラム | |
JP5484184B2 (ja) | 画像処理装置、画像処理方法及びプログラム | |
JP6270433B2 (ja) | 情報処理装置、情報処理方法、情報処理システム | |
US11132538B2 (en) | Image processing apparatus, image processing system, and image processing method | |
JP5693094B2 (ja) | 画像処理装置、画像処理方法及びコンピュータプログラム | |
EP2840557B1 (en) | Image processing system, server device, image pickup device and image evaluation method | |
JP2019062527A (ja) | エッジ演算を用いる多カメラシステムでのリアルタイムの対象再識別 | |
US20180144206A1 (en) | Setting apparatus, setting method, and storage medium | |
EP3249919B1 (en) | Image processing apparatus, image processing method, and program | |
US10691956B2 (en) | Information processing apparatus, information processing system, information processing method, and storage medium having determination areas corresponding to waiting line | |
JP6827790B2 (ja) | 画像処理装置およびその制御方法 | |
US8923552B2 (en) | Object detection apparatus and object detection method | |
WO2022041484A1 (zh) | 人体跌倒检测方法、装置、设备及存储介质 | |
JP5264457B2 (ja) | 物体検出装置 | |
WO2023276050A1 (ja) | 画像処理システム及び撮像装置 | |
WO2018037479A1 (ja) | 画像処理装置、ステレオカメラ装置及び画像処理方法 | |
US20180197000A1 (en) | Image processing device and image processing system | |
JP5769468B2 (ja) | 物体検出システム及び物体検出方法 | |
US20190102888A1 (en) | Image processing apparatus and method and monitoring system | |
KR20230060439A (ko) | 실물 촬영 이미지 여부 식별 방법 및 시스템 | |
JP2019192155A (ja) | 画像処理装置、撮影装置、画像処理方法およびプログラム | |
JP2019192154A (ja) | 画像処理装置、撮影装置、画像処理方法、およびプログラム | |
US20110090360A1 (en) | Image processing apparatus and object detecting method |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 21948357 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2023531246 Country of ref document: JP Kind code of ref document: A |
|
WWE | Wipo information: entry into national phase |
Ref document number: 18571737 Country of ref document: US |
|
WWE | Wipo information: entry into national phase |
Ref document number: 202180099794.X Country of ref document: CN |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 21948357 Country of ref document: EP Kind code of ref document: A1 |