JP2021027490A - Image processing device, imaging device, imaging system, control method of the image processing device, and program - Google Patents

Abstract

To provide a technique for enabling the reduction of a data size of an imaging image, while suppressing an influence for a subject detection accuracy.SOLUTION: An image processing device comprises: setting means of setting a detection processing of a subject to an imaging image; detection means of executing the set detection processing from the imaging image; gradation number control means of determining a gradation number of the imaging image after the reduction of the gradation number in accordance with a type of the set detection processing and a detection result of the detection processing, and converting the imaging image into the determined gradation number; and output means of outputting the imaging image obtained by converting the gradation number and the detection result when the detection processing is executed by the detection means.SELECTED DRAWING: Figure 1

Description

The present application relates to an image processing device, an imaging device, an imaging system, a control method of the image processing device, and a program.

In the situation of constant imaging by a network camera, if the data size of the captured image is large, the network distribution of the image (video) from the network camera to the external device is overloaded, and quality deterioration such as a decrease in frame rate occurs. On the other hand, an approach has been proposed in which the data size of the captured image is reduced by reducing the number of gradations of the image and the distribution load is reduced.

Patent Document 1 discloses a method of suppressing the amount of noise in an image and simultaneously reducing the data size when delivering the image by dynamically controlling the number of gradations of the image according to the illuminance of the imaging environment. To do. Patent Document 2 discloses a technique for reducing the data size of a captured image by reducing the number of gradations in a region where the amount of movement is large, focusing on the amount of movement of the subject.

JP-A-2017-135554 Japanese Unexamined Patent Publication No. 2012-73971

However, in the above-mentioned conventional technique, the reduction of the number of gradations may affect the detection accuracy of the subject. In particular, since the degree of influence on the detection accuracy differs depending on the type of the set detection method, it is required to control the number of gradations according to the type of the detection method.

Therefore, the present invention provides a technique for reducing the data size of a captured image while suppressing the influence on the subject detection accuracy.

The invention for solving the above problems is an image processing apparatus.
Setting means for setting subject detection processing for captured images, and
A detection means that executes the set detection process from the captured image, and
The number of gradations of the captured image after the number of gradations is reduced is determined according to the set type of detection processing and the detection result of the detection processing, and the image is converted to the determined gradation number. Key control means and
The image pickup image to which the number of gradations is converted and the output means for outputting the detection result when the detection process is executed by the detection means are provided.
The gradation number control means
When the detection process is not set by the setting means, or when the set detection process is executed and the detection result is not obtained, the first gradation number is reduced after the gradation number is reduced. Determine the number of gradations,
It is larger than the first gradation number when the detection process is set by the setting means and when the set detection process is executed and the predetermined detection result is obtained. The second gradation number is determined to be the gradation number after the gradation number reduction.

According to the present invention, it is possible to reduce the data size of the captured image while suppressing the influence on the subject detection accuracy.

The block diagram which shows the structural example of the imaging system which concerns on embodiment. The figure which shows an example of the functional structure of the image processing part which concerns on embodiment, and the figure which shows an example of the hardware structure of an image pickup apparatus. A table for explaining the relationship between the type of detection processing and the number of gradations according to the embodiment. The figure which shows an example of the gamma table which concerns on embodiment. The flowchart which shows an example of the process which concerns on embodiment. The flowchart which shows another example of the process which concerns on embodiment.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. The following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiment, not all of the plurality of features are essential to the invention, and the plurality of features may be arbitrarily combined. Further, in the attached drawings, the same or similar configurations are designated by the same reference numbers, and duplicate description is omitted.

[Embodiment 1]
Hereinafter, the configuration and processing flow of the imaging system according to the first embodiment will be described with reference to the attached drawings. First, the configuration of the imaging system according to the first embodiment of the invention will be described with reference to the block diagram of FIG. FIG. 1 is a block diagram showing a configuration example of an imaging system according to a first embodiment of the present invention. In FIG. 1, the imaging system 100 is configured such that an imaging device 110, an information processing device 120, and a display device 130 are connected to each other via a network 140. The components of the imaging system 100 shown in FIG. 1 are merely examples, and components other than these may be further added. Hereinafter, each component will be described.

The image pickup apparatus 110 may include an optical system 111, an image pickup section 112, a control section 113, an image processing section 114, an operation section 115, a display section 116, and a communication section 117.

The optical system 111 includes at least one of a zoom lens that determines the image magnification, a focus lens that adjusts the focal position, and a transmission filter that limits the wavelength range of the light to be imaged. The optical system 111 includes a predetermined drive mechanism such as a motor for moving the lens, and the operation of the drive mechanism is controlled in response to an instruction from the control unit 113.

The image pickup unit 112 can include an image pickup element, a development processing unit, a coding unit, and the like. The image pickup device can be configured to include an A / D converter so as to convert an analog image signal obtained by photoelectric conversion of an optical image formed on an image pickup surface into a digital image signal and output the signal. The development processing unit can perform development processing such as debayer processing, gamma correction processing, and gradation conversion processing on the digital image signal output from the image sensor. The development process can include a process of converting RGB format data into a format such as YUV. The coding unit compresses and encodes the digital image signal output by the development processing unit to generate image data. The image compression method can be based on a standard such as H.264, H.265, MJPEG or JPEG. Further, image data in any format including mp4 and avi formats may be generated. The imaging unit 112 can acquire, for example, an image for 30 frames per second and acquire a moving image (live image) of 30 fps in the monitoring area.

The control unit 113 adjusts imaging conditions such as the angle of view, imaging direction, frame rate, and type of image to be acquired of the imaging device 110. The adjustment of the imaging conditions can be performed according to an instruction from the user, or can be performed according to a set value stored in advance in the imaging device 110. When the user gives an instruction, the instruction may be given from the information processing device 120 or the display device 130 via the network 150, or may be given directly through the operation unit 115 of the image pickup device 110. Further, the control unit 113 can also control the image pickup unit 112 such as start / end of imaging in response to a command transmitted from the information processing device 120 or the display device 130.

The image processing unit 114 performs image processing such as processing for accepting the setting of the detection type, processing for determining the number of gradations according to the set detection type, and processing for detecting the subject or the like from the captured image according to the set detection type. I do. In addition, the movement of the subject can be detected in the captured image.

The operation unit 115 is a user interface such as a button, a key, a dial, and a touch panel, and receives setting input and instruction input for the image pickup apparatus 110. The operation unit 115 may be configured as a console terminal connected to the image pickup apparatus 110. The display unit 116 is a display for displaying a predetermined setting screen or displaying a captured image. The communication unit 117 is a communication interface for transmitting an image imaged and generated by the image pickup unit 112, a result of detection processing, and the like to the information processing device 120, the display device 130, and the like via the network 140. The communication unit 117 coordinates the exchange of data with the communication partner by using, for example, a network file system such as NFS or CIFS, or a network communication protocol such as UDP or TCP. Further, the communication unit 117 receives setting information and commands for controlling the image pickup device 110 from the information processing device 120 and the display device 130, and supplies them to the control unit 113. The control unit 113 controls the operation of the image pickup apparatus 110 according to the setting information and the command.

The storage unit 118 stores control information of the optical system 111 and the imaging unit 112, information on imaging conditions, a control program, etc. managed by the control unit 113, and information used for processing in the image processing unit 114, and processing. Memorize the results. The image captured by the image capturing unit 112 and the image processed by the image processing unit 114 are accumulated, and the processing result of the image analysis processing (detection processing of the subject or the like) in the image processing unit 114 is stored. In addition, the storage unit 118 stores various programs for the control unit 113 to control the operation of the image pickup apparatus 110, and also functions as a work area.

The information processing device 120 includes a control unit 121, a storage unit 122, an image processing unit 123, an operation unit 124, a display unit 125, and a communication unit 126. The control unit 121 controls the operation of each unit of the information processing device 120. The control unit 121 can manage the operation and setting of the image pickup apparatus 110 connected to the image pickup system 100. A plurality of imaging devices 110 can be connected to the imaging system 100, and the control unit 121 manages information regarding the type, installation location, installation environment, imaging parameters, and the like of each imaging device 110. The control unit 121 can select an imaging parameter to be set for the imaging device 110 and transmit it to the imaging device 110 via the communication unit 126 to set the imaging conditions. In addition, information on imaging conditions such as an imaging parameter set for the imaging apparatus 110 and an installation location and an installation environment can be stored and managed in the storage unit 122. The information regarding these imaging conditions can be appropriately updated even after being acquired and saved once. For example, when an image periodically transmitted from the imaging device 110 is analyzed and the imaging conditions (for example, illuminance, etc.) at the installation location of the imaging device 110 change, the control unit 121 provides information on the imaging conditions. Can be updated.

The storage unit 122 stores image data received from the image pickup apparatus 110, information used for processing in the image processing unit 123, and processing results. In addition, it is possible to store information and the like regarding imaging conditions managed by the control unit 121.

The image processing unit 123 performs predetermined image processing and analysis processing on the image received from the image pickup apparatus 110, and outputs the processing result. For example, it is possible to perform processing processing such as superimposing a frame for identifying an area in which a subject detected by the image pickup apparatus 110 exists on the received image. It is also possible to execute a process of detecting a subject on the received image.

The operation unit 124 can receive an operation from the operator of the information processing device 120. Specifically, it can accept operations for changing the settings of the image pickup device 110 and the information processing device 120, and can include a keyboard, a mouse, various operation buttons, a touch panel, and the like. When a plurality of image pickup devices 110 are connected to the network 140, the operation unit 124 can input instructions for displaying the image data of the plurality of image devices on the display unit 125 on one screen. It is also possible to select one image data on the screen and input instructions for displaying the image data on the display unit 125. In addition, it is possible to input instructions for setting and changing various setting information. Further, it is possible to input an evaluation for the result of the image analysis processing corresponding to the present embodiment.

In addition to displaying and outputting the captured image acquired from the image pickup device 110, the display unit 125 displays and outputs setting information of various processes in the information processing device 120, a screen for displaying the processing result in the information processing device 120, or the image pickup device 110. It is possible to display a setting screen for setting and changing setting information and imaging parameters, and a screen for notification output. Further, the display unit 125 may have a function of outputting audio in addition to image output. The display unit 125 can be composed of a display device such as a display and a display control unit that controls the display on the display device. Further, the display may be installed outside the information processing device 120 and may be used by connecting to the information processing device 120.

The communication unit 126 receives the image transmitted from the image pickup apparatus 110. Further, the communication unit 126 can transmit the setting information, the control information, and the distribution command set by the user of the information processing device 120 using the operation unit 124 and the display unit 125 to the image pickup device 110. Further, the result of image analysis processing by the information processing device 120 (including the above notification) can be output to the display device 130 connected to the network 140.

The display device 130 can display the image captured by the image pickup device 110 and output various image analysis results received from the information processing device 120. It is also possible to provide a user interface for instructing / changing the imaging conditions / imaging parameters to be captured by the imaging device 110, and to display the distributed video. The display unit 125 of the information processing device 120 may be configured as the display device 130.

The network 140 is a communication line capable of IP-based communication such as LAN and the Internet. In the network 140, regardless of the form of wire / wireless, a communication line in a required form is mounted according to the application and purpose. The image pickup apparatus 110 can capture a live image according to a designated imaging condition and distribute the obtained image data to each apparatus via the network 140.

In the configuration of the imaging system 100 shown in FIG. 1, the number of imaging devices 110 can be any number of one or more, and the number is not limited. The information processing device 120 and the display device 130 may be realized not only by a single device but also by a plurality of devices working together to realize each function.

Next, with reference to FIG. 2A, the functional configuration of the image processing unit 114 corresponding to the present embodiment will be described. The image processing unit 114 has a gradation number control unit 1141, a detection unit 1142, and a detection type setting unit 1143 as functions peculiar to the present embodiment.

The gradation number control unit 1141 determines the gradation number of the image to be detected by the detection unit 1142 according to the type of the detection method set by the detection type setting unit 1143, and each of the images is determined according to the gradation number. Convert the pixel value. The number of gradations is determined so as not to deteriorate the detection accuracy in the detection unit 1142.

The detection unit 1142 detects a subject or the like for the image to be processed according to the detection type set by the detection type setting unit 1143, and outputs information on the area including the subject, the type of the subject, the subject image, and the like. To do. The detection type setting unit 1143 sets the type of the detection method received via the operation unit 115 in the detection unit 1142.

Hereinafter, when the type of detection processing in the detection unit 1142 is set via the detection type setting unit 1143, the gradation number control unit 1141 determines the number of gradations of the image according to the set detection processing type. , The case of converting the number of gradations of the captured image will be described.

Next, with reference to FIG. 2B, the hardware configuration of each device corresponding to the present embodiment will be described. Hereinafter, an example of the schematic configuration of the image pickup apparatus 110 will be described, but the information processing apparatus 120 and the display apparatus 130 may also be configured as the same or equivalent hardware configuration. However, in FIG. 2B, the hardware configuration of the optical system 111 and the imaging unit 112 is omitted.

In FIG. 2B, the CPU 200 can operate as the control unit 113 and executes an application program, an operating system (OS), a control program, etc. stored in the hard disk device (hereinafter referred to as HD) 205. , RAM 202 is controlled to temporarily store information, files, etc. necessary for executing the program. It is also possible to operate as the image processing unit 114, and execute processing, analysis processing, and the like of the image provided by the imaging unit 112. Further, it controls data transmission / reception with an external information processing device 120 or a display device 130 via the interface (I / F) 208.

The ROM 201 internally stores various data such as an application program that executes a predetermined process in addition to the basic I / O program. The RAM 202 temporarily stores various data and functions as a main memory, a work area, or the like of the CPU 200. In addition, the information received from the information processing device 120 is temporarily stored. The external storage drive 203 is an external storage drive for realizing access to the recording medium, and stores the image captured by the imaging unit 112 in the medium (recording medium) 204, a program stored in the media 204, or the like. Can be loaded into this computer system. As the media 204, for example, a CD-ROM, a CD-R, a CD-RW, a DVD, a Blu-ray (registered trademark), an SD card, an IC memory card, a memory stick, or the like can be used.

In the present embodiment, the hard disk device 205 uses an HD (hard disk) that functions as a large-capacity memory. The HD205 stores an application program, an OS, a control program, a related program, and the like. A non-volatile storage device such as a flash (registered trademark) memory may be used instead of the hard disk. The ROM 201, RAM 202, and hard disk device 205 can be configured to function as the storage unit 118.

The instruction input device 206 corresponds to a key, a button, a touch panel, etc., and functions as an operation unit 115. The output device 207 outputs a command input from the instruction input device 206, a response output to the command, and the like. The output device 207 may include a display or the like. The output device 207 can further include an audio output device such as a speaker. The system bus 209 controls the flow of data in the image pickup apparatus 110.

The interface (hereinafter referred to as I / F) 208 functions as a communication unit 117 to play a role of mediating the exchange of data with an external device. Specifically, the I / F 208 can include a wireless communication module, which includes an antenna system, an RF transmitter / receiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a CODEC chipset. , A well-known circuit mechanism, including a subscriber identification module card, memory, and the like. It can also include a wired communication module for wired connections. Wired communication modules allow communication with other devices via one or more external ports. It can also include various software components that process the data. The external port couples with other devices, either directly or indirectly via a network, via Ethernet, USB, IEEE 1394, etc. It should be noted that the software that realizes the same functions as each of the above devices can be configured as a substitute for the hardware device.

Every time the corresponding program is operated to execute the process corresponding to the present embodiment, the program may be loaded from the HD 205 to the RAM 202 in which the program is already installed. It is also possible to record the program according to this embodiment in ROM 201, configure it to form a part of the memory map, and execute it directly on the CPU 200. Further, the corresponding program and related data can be directly loaded into the RAM 202 from the media 204 and executed.

Next, the details of the detection process in the imaging system 100 according to the present embodiment will be described. The detection process executed by the detection unit 1142 in the present embodiment is roughly classified based on two detection algorithms. The first is a first detection algorithm that evaluates background subtraction, and the second is a second detection algorithm that evaluates the feature amount of an imaged subject.

Generally, the first detection algorithm based on background subtraction is a method of detecting the presence / absence or movement of a subject of interest, and is used in subject detection processing such as abandonment detection, take-away detection, and intrusion / passage detection. Will be done. These detection processes are called the first subject detection process. In the first subject detection process, the difference in brightness between the reference image called the background image and the captured image (hereinafter referred to as the brightness difference) is evaluated, and whether or not a predetermined brightness difference is satisfied is determined in the captured image. It is possible to detect the presence / absence of the subject of interest and a predetermined operation of the subject of interest.

Next, the second detection algorithm for evaluating the feature amount of the captured subject evaluates whether or not the captured subject is the subject to be detected. The second detection algorithm is used in, for example, a second subject detection process for the purpose of identifying an object of the same type such as a human body or a vehicle, face recognition, or the like. In the second subject detection process, a predetermined luminance difference according to the feature amount evaluation method is used as a criterion for determining whether or not the subject is a detection target based on the feature amount evaluation. In the second subject detection process, when a subject having a predetermined feature amount (a specific person, a specific vehicle, or the like) is included in the captured image, the subject can be detected.

Comparing the above two detection processes, the difference in brightness in the second type of detection process used for identifying the subject, such as the evaluation of the feature amount, is more detected than the difference in brightness in the first type of detection process. The required brightness difference tends to be small. In the present embodiment, the data size of the captured image can be reduced by controlling the number of gradations, but the second type of detection processing for evaluating the feature amount reduces the number of gradations of pixels. It can be said that it is strongly influenced by.

As described above, since the required luminance difference differs depending on the type of detection processing, in the present embodiment, the required number of gradations is individually prepared according to the type of detection processing. Here, in the present embodiment, the gradation number is expressed as a range in which one pixel value is expressed or a number of steps in which the pixel value can be taken. For example, when one pixel is represented by a value from 0 to 255, 256 (8 bits) is the number of gradations. In this embodiment, 256 is set as a reference gradation number.

Next, the relationship between the type of detection process and the number of gradations in the present embodiment will be described with reference to FIG. FIG. 3 is a table 300 showing the relationship between the type of subject detection processing and the set number of gradations. The table 300 can be stored in, for example, a storage unit 118. Examples of the detection type include abandonment detection, take-away detection, and intrusion / passage detection from the first subject detection process described above, and object detection and face recognition from the second subject detection process.

In the present embodiment, the original gradation number before gradation adjustment is set to 256 as the reference gradation number, and the first gradation number for the purpose of reducing the data size among the gradation numbers after the gradation number reduction is 60. And. By applying the first number of gradations, it is possible to reduce the number of bits representing one pixel from 8 bits to 6 bits by 2 bits in terms of the number of bits. In such an image with the first gradation number, the data size can be reduced by reducing the gradation number, but when the brightness difference between pixels is insufficient, the presence of a predetermined subject included in the image is detected. It becomes difficult to judge, and the detection accuracy is lowered. Therefore, in the present embodiment, the second gradation number after the gradation number reduction is increased from the first gradation number according to the type of the subject to be detected, in other words, the type of the detection process to be executed. Set to the number of gradations.

When the first gradation number is 60, the second gradation number is at least a value larger than the first gradation number, while the reference gradation number is maintained in order to maintain the effect of reducing the data size. The value shall be smaller than 256. In the example of FIG. 3, the abandonment detection is 80, the take-away detection is 80, the intrusion / passage detection is 100, the object detection is 150, and the face recognition is 200. However, these setting values are just examples. It is not limited.

When applying gradation control to an image, the range of the gamma curve is controlled. FIG. 4 is an example of a graph 400 showing the magnitude relationship between the range of the gamma curve, the number of first gradations, and the number of second gradations. In FIG. 4, the output luminance range of the gamma curve 402 (solid line) when the second gradation number is set is larger than that of the gamma curve 401 (dotted line) when the first gradation number is set. It's getting bigger. The gamma curve corresponding to the second number of gradations shown in FIG. 4 is an example, and the gamma curve can be set according to the detection type. The conversion of the number of gradations is executed based on the set gamma curve.

Next, a method of determining specific values for the first gradation number and the second gradation number will be described. The first gradation number and the second gradation number may be predetermined fixed values (hereinafter referred to as presets), may be determined manually by the user, or may be determined by the user's manual input. The number of gradations may be dynamically calculated according to the detected (or to be detected) subject. In the case of a preset, the number of gradations may be uniquely prepared according to the type of detection processing, or a plurality of levels are provided for the second number of gradations in order to correspond to various scenes. In this mode, the user may be allowed to select a value according to the scene. Considering the case where the user manually inputs, the input method may be a mode of inputting by a slide bar on the user interface or a mode of directly inputting a numerical value. Further, the value may be set by using the operation unit 115 such as a button attached to the exterior of the image pickup apparatus 110.

Further, in the case of dynamically calculating according to the subject, the image processing unit 114 can further have a functional configuration block for extracting a moving object region. By extracting the moving object region, when a moving subject is captured, the image can be separated into a moving object region and a background region other than the moving object region. The brightness of the subject area and the background area is evaluated individually, using this moving object area as the area where the subject of interest is captured (hereinafter referred to as the subject area). The second number of gradations can be set so that the brightness difference between the subject area and the background area obtained at this time becomes a predetermined brightness difference required for detection. The setting method can also be used when the second gradation number is set in advance. For example, the brightness difference between the subject to be detected and the virtual subject area and the background area obtained by assuming the area where the subject is detected is calculated in advance, and the second gradation is calculated based on the brightness difference. You can set the number. At this time, a certain range can be given to the luminance difference so as to compensate for the deviation from the assumption.

Further, it is possible to further have a function of adjusting the set second gradation number according to the scene and the user's setting. For example, when the illuminance of the imaging environment is low (dark) (hereinafter referred to as low illuminance), the contrast of the image may be lower than when the imaging environment is bright. Therefore, in the present embodiment, it is possible to add a functional configuration of illuminance estimation for evaluating the illuminance of the imaging environment. In this case, a second number of gradations is set according to the estimated illuminance so as not to deteriorate the detection accuracy of the detection unit 1142.

Here, the illuminance estimation can be performed based on, for example, the output value from an outer sensor such as an optical sensor, the exposure amount in the image pickup apparatus 110, and the brightness information of the image. As a specific process, when the environmental illuminance obtained by illuminance estimation falls below a predetermined threshold value, a predetermined coefficient (+ 10% = 1.10) as shown in FIG. 3 is multiplied by the second gradation number. The illuminance can be corrected with. In addition to this, a function or a table for determining the second gradation number from the illuminance may be given.

Further, it is considered that the image after the second gradation number is set may also be missed due to the change in the imaging environment. Therefore, in the imaging system 100, in consideration of the problem of detection omission, the user may select whether to prioritize the detection accuracy or the data size reduction in the detection unit 1142. In this case, the second gradation number may be increased in the mode in which the detection accuracy is prioritized, and the second gradation number may be decreased in the mode in which the data size reduction is prioritized. At this time, as shown in FIG. 4, when the detection accuracy is prioritized, a coefficient of + 5% (1.05) is multiplied, and when the data size is prioritized, a coefficient of -5% (0.95) is applied. You may hang it. These predetermined coefficients can be arbitrary values, and may be fixed values or values determined by manual input by the user.

Next, the flow of processing in the image pickup apparatus 110 corresponding to the present embodiment will be described with reference to the flowchart of FIG.

First, in S501, the detection type setting unit 1143 sets the detection type. The user of the image pickup apparatus 110 can set the detection type via the operation unit 115. For example, in the present embodiment, at least one detection type can be selected and set from left-behind detection, take-away detection, intrusion / passage detection, object detection, and face recognition. Further, the setting may be such that the detection type is not selected and the monitoring area is imaged and the image is output without performing the subject detection process. At this time, the user can additionally select either a mode in which the detection accuracy is prioritized or a mode in which the data size is prioritized. The information is stored in the storage unit 118 as mode selection information.

Next, in S502, the gradation number control unit 1141 determines whether or not the detection type is set by the detection type setting unit 1143 based on the information stored in the storage unit 118. If it is determined that the detection type is not set, the process proceeds to S503. In S503, the gradation number control unit 1141 determines the gradation number to be the first gradation number, and performs gradation conversion processing of the captured image. The gradation conversion process is executed based on the gamma curve based on the determined first gradation number. At this time, when the captured image is RGB, the gradation number control unit 1141 performs gradation conversion for each color component of RGB, and after YUV conversion, only the luminance image or each component of YUV is used. On the other hand, gradation conversion is performed. After performing the gradation conversion process, the process proceeds to S508.

When it is determined that the detection type has been set, the process proceeds to S504. In S504, the gradation number control unit 1141 determines the gradation number from the second gradation number based on the set detection type and the registration information of the table 300 shown in FIG. For example, when the abandonment detection is selected, 80 is selected as the second gradation number. Further, for example, when a plurality of detection types such as intrusion / passage detection and object detection are selected, a larger gradation number is selected from the number of gradations for realizing a predetermined luminance difference required for detection. To do.

In the subsequent S505, the gradation number control unit 1141 corrects the gradation number selected in S504 based on the mode selection information held in S501 and the registration information in the table 300 of FIG. Further, the above-mentioned illuminance correction using the coefficient for illuminance correction is performed based on the illuminance in the imaging environment and the registration information of the table 300.

In the subsequent S506, the gradation number control unit 1141 performs gradation conversion processing of the captured image based on the corrected gradation number. The gradation conversion process is executed based on the gamma curve based on the set number of gradations. When the captured image is RGB, the gradation number control unit 1141 performs gradation conversion for each color component of RGB, and after YUV conversion, only the luminance image or the floor for each component of YUV. Perform key conversion.

In the subsequent S507, the detection unit 1142 performs subject detection processing corresponding to the detection type set in S501 on the image after gradation conversion. In the following S508, the detection result including the image after the gradation conversion and the information of the detected subject is output. The image and subject information are transmitted from the communication unit 117 to the information processing device 120 and the display device 130 via the network 140.

According to the process corresponding to the embodiment described above, the process of detecting a desired subject or the like from the image is executed while reducing the data size of the captured image and contributing to the reduction of the distribution load when the image is distributed. When the setting is made, the number of gradations can be set so as not to deteriorate the detection accuracy. Here, in the present embodiment, since the number of gradations is prepared according to the set type of detection processing, the data size can be reduced as much as possible according to the type of detection processing while ensuring the accuracy of the detection processing. ..

[Embodiment 2]
In the first embodiment, when the detection type is set, the number of gradations is maintained at the second number of gradations regardless of whether or not a desired subject or the like is actually detected. For example, when there is no subject in the monitoring area, or when there is no change in the subject, no significant detection result can be obtained, so that there is no difference from simply photographing the area to be imaged. In such a situation where no significant information about the subject is detected, even if the captured image is output with the first gradation number, the problem related to the detection accuracy cannot occur. Rather, there is no reason why the data size must be maintained at a size based on the second number of gradations, so it can be said that there is room for further reduction of the data size.

Therefore, in the present embodiment, the number of gradations is determined by further referring to the information related to subject detection in the captured image. For example, the functional configuration of the motion detection unit is further added to the image processing unit 114, and the second gradation number is set at the timing when the motion of the subject is detected by the motion detection unit. As a result, the detection accuracy is ensured by setting the number of gradations as the second gradation number at the timing when the subject exists and the detection process should be executed, while the first gradation number when the subject does not exist. Achieve a higher data size reduction effect.

The motion detection unit in the image processing unit 114 is a unit that detects the movement of the subject captured within the angle of view. For example, the difference is based on the difference between the frame image to be processed and the background image and the difference between frames. The presence or movement of the subject can be detected based on whether or not the value is equal to or greater than the threshold value. Further, the motion information between frames may be acquired and determined not from the image processing unit 114 but from the coding unit included in the image pickup unit 112, for example. In either case, since the motion detection process itself is a known technique, further details will be omitted.

In the process corresponding to the present embodiment, in S502 of the flowchart of FIG. 5, the branch destination can be determined based on whether or not there is movement of the subject in addition to the presence or absence of the setting. Specifically, even when the detection type is set, if the movement of the subject is not detected in the captured image, the process proceeds to S503 and the gradation number is set to the first gradation number. Gradation conversion is performed. On the other hand, when the detection type is set and the movement of the subject is detected in the captured image, the process proceeds to S504 and the number of gradations is set to the second number of gradations.

According to the above embodiment, in a situation where the subject does not exist in the captured image or the subject does not change, the subject exists while reducing the data size by selecting the first gradation number. Alternatively, when there is a change in the subject, the detection accuracy can be ensured by selecting the number of gradations according to the set detection type. Thereby, the data size can be reduced more efficiently.

[Embodiment 3]
In the above two embodiments, a configuration in which the detection process is performed after determining the number of gradations and converting the gradations has been described. In this configuration, the data size is reduced while controlling the number of gradations so as not to affect the detection accuracy by the detection unit 1142. In addition, in the case of mode selection and low illuminance, although the processing to prevent detection omission is taken by correcting the gradation value, it is caused by the reduction in the number of gradations depending on the imaging scene and the state of the camera. There is still a risk that omission of detection will still be unavoidable. On the other hand, in the present embodiment, the order of processing is changed, the image is analyzed by the detection unit 1142, the detection processing is performed, and then the gradation number control unit 1141 controls the gradation number. Deal with situations like.

Specifically, also in the present embodiment, as in the first embodiment, when the detection type is set by the detection type setting unit 1143, the gradation number of the image according to the set detection type is set to the gradation number control unit 1141. Is set as the number of gradations of the captured image. On that premise, the order of the gradation number setting and the detection process is changed in order to reduce the detection omission caused by the gradation number control.

Next, the flow of processing in the image pickup apparatus 110 corresponding to the present embodiment will be described with reference to the flowchart of FIG. In the flowchart of FIG. 6, a corresponding process number is assigned to the process of performing the same processing as that of FIG. The description of the process will be omitted as appropriate.

After the detection type setting unit 1143 accepts the detection type setting in S501, if it is determined in S502 that the detection type has been set, the process proceeds to S601. In S601, the detection unit 1142 performs subject detection processing corresponding to the detection type set in S501 for the captured image (number of gradations: 256) before gradation conversion. In the following S602, the detection unit 1142 determines whether or not a predetermined detection result is obtained as a result of the detection process. Here, the predetermined detection result means a significant detection result in the detection process. For example, in the case of abandonment detection, it means that an object (person, object, vehicle, etc.) is detected to be left behind in the monitoring space. In the case of take-away detection, it means that the object that existed in the monitoring space has been taken away. In the case of intrusion / passage detection, it means that a person or vehicle has invaded or passed through the monitoring space. Furthermore, in the case of object detection or face recognition, it means that a specific person, vehicle, etc. has been identified. On the other hand, when a predetermined detection result cannot be obtained, it means that the significant detection result was not obtained.

If it is determined that a predetermined detection result has been obtained, the process proceeds to S504, and if it is determined that a predetermined detection result has not been obtained, the process proceeds to S503. After that, the same processing as described in the first embodiment is performed in relation to FIG. However, since the detection process in S507 has already been performed in S601, it is omitted.

In this way, in the present embodiment, the subject is detected in the image having the lower gradation number before the gradation conversion is performed, so that the detection omission can be prevented. Further, when the detection unit 1142 does not detect a predetermined subject, the first number of gradations is set to reduce the data size, and the detection type is set according to the detection type set when the detection unit 1142 detects the predetermined subject. A second number of gradations is set. Here, since the detection is performed first, it may be uniformly converted to the first gradation number, but when a subject is detected, the first is to ensure the visibility of the predetermined subject. The number of gradations is changed to the second number of gradations for each detection type according to the type of the detected subject without reducing the number of gradations. As a result, when a subject is detected, the data size can be efficiently reduced while ensuring the visibility of the subject. When the mode in which the data size is prioritized is selected, the number of gradations may be reduced to the first number of gradations even when the subject is detected.

According to the above embodiment, the detection process is performed on the original captured image before the gradation number is converted, and the gradation number is switched according to the result. This makes it possible to efficiently reduce the data size while preventing omission of subject detection.

[Embodiment 4]
In the above-described first to third embodiments, the detection process according to the set detection type is executed on the image pickup apparatus 110 side. However, when a high-load calculation is required for the detection process, it is necessary to mount a high-quality processor on the image pickup apparatus 110 side. Otherwise, most of the calculation resources will be devoted to the detection process, and as a result, the distribution load such as a decrease in the frame rate may increase. Therefore, in the present embodiment, the detection process for the captured image described in the first and second embodiments is executed on the information processing apparatus 120 side, and the calculation load on the imaging apparatus 110 is reduced.

In the process corresponding to the present embodiment, the process of S507 in the flowchart of FIG. 5 is executed on the information processing apparatus 120 side, and in S508, only the image after the gradation number is converted is transmitted from the communication unit 117 via the network 140. Is transmitted to the information processing apparatus 120. When the information processing device 120 receives the captured image from the image pickup device 110, the image processing unit 123 performs subject detection processing corresponding to the detection type set in S501 on the image after gradation conversion. The set detection type information may be received from the image pickup apparatus 110 together with the image, or the setting information required at the start of the process may be received from the image pickup apparatus 110. Further, if the setting information about the detection type cannot be acquired from the image pickup apparatus 110, the detection process may be omitted.

According to the present embodiment, it is possible to reduce the calculation load on the image pickup apparatus 110 side while reducing the data size of the image distributed between the image pickup apparatus 110 and the information processing apparatus 120. Further, on the information processing apparatus 120 side, the accuracy of the detection process is ensured while reducing the data size by acquiring the image of the number of gradations according to the set type of the detection process and performing the detection process. Can be done.

The invention is not limited to the above embodiments, and various modifications and modifications can be made without departing from the spirit and scope of the invention. Therefore, a claim is attached to make the scope of the invention public.

(Other Examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

100: Imaging system, 110: Imaging device, 120: Information processing device, 130: Display device, 140: Network

Claims (16)

It is an image processing device
Setting means for setting subject detection processing for captured images, and
A gradation number control means that determines the number of gradations of the captured image after reducing the number of gradations according to the set type of detection processing and converts the captured image to the determined gradation number.
A detection means that executes the set detection process from the captured image to which the number of gradations has been converted, and
The image pickup image to which the number of gradations is converted and the output means for outputting the detection result when the detection process is executed by the detection means are provided.
The gradation number control means
When the detection process is not set by the setting means, the first gradation number is determined to be the gradation number after the gradation number reduction.
An image characterized in that, when the detection process is set by the setting means, a second gradation number larger than the first gradation number is determined as the gradation number after the gradation number reduction. Processing equipment. A motion detecting means for detecting the motion of the subject in the captured image is further provided.
The gradation number control means
When the detection means does not detect the movement of the subject and when the detection process is not set by the setting means, the first gradation number is reduced to the gradation number after the gradation number is reduced. Decided to
When the motion detecting means detects the motion of the subject and the detection process is set by the setting means, the second gradation number is determined to be the gradation number after the gradation number is reduced. The image processing apparatus according to claim 1. It is an image processing device
Setting means for setting subject detection processing for captured images, and
A detection means that executes the set detection process from the captured image, and
The number of gradations of the captured image after the number of gradations is reduced is determined according to the set type of detection processing and the detection result of the detection processing, and the image is converted to the determined gradation number. Key control means and
The image pickup image to which the number of gradations is converted and the output means for outputting the detection result when the detection process is executed by the detection means are provided.
The gradation number control means
When the detection process is not set by the setting means, or when a predetermined detection result is not obtained by executing the set detection process, the first gradation number is reduced. Decide on the number of gradations later,
It is larger than the first gradation number when the detection process is set by the setting means and when the set detection process is executed and the predetermined detection result is obtained. An image processing device characterized in that the second gradation number is determined to be the gradation number after the gradation number reduction. The setting means performs the setting from a plurality of detection processes, and then performs the setting.
A corresponding number of gradations is assigned to each of the plurality of detection processes.
The image processing according to any one of claims 1 to 3, wherein the gradation number control means sets the gradation number corresponding to the set detection processing as the second gradation number. apparatus. When the setting means sets two or more detection processes, the gradation control means sets the largest gradation number among the gradations assigned to the set detection processes as the second. The image processing apparatus according to claim 4, wherein the number of gradations is used. The plurality of detection processes include a first subject detection process based on a first detection algorithm for evaluating background subtraction, and a second detection algorithm based on a second detection algorithm for evaluating the feature amount of a subject included in the captured image. The image processing apparatus according to claim 4 or 5, further comprising subject detection processing. The image processing apparatus according to claim 6, wherein the number of gradations assigned to the first subject detection process is smaller than the number of gradations assigned to the second subject detection process. 1. The gradation number control means is characterized in that the second gradation number is determined based on the brightness difference between the subject region in which the subject is detected in the captured image and another region. 7. The image processing apparatus according to any one of 7. When setting the detection process, the setting means has a first mode in which the detection accuracy of the detection process is prioritized and a second mode in which the data size of the captured image after converting the number of gradations is prioritized. Select one of
The gradation number control means
When the first mode is selected, the correction is made so as to increase the number of the second gradations.
When the second mode is selected, the correction is made so as to reduce the number of the second gradations.
The image processing apparatus according to any one of claims 1 to 8, wherein the captured image is converted into a second gradation number after correction. The ninth aspect of the present invention is characterized in that the gradation number control means further corrects the second gradation number so as to increase when the illuminance in the imaging environment of the captured image is lower than a predetermined illuminance. Image processing equipment. Imaging means and
An imaging device including the image processing device according to any one of claims 1 to 10, which processes an captured image captured by the imaging means. An imaging system composed of an imaging device and an information processing device communicating with each other.
The image pickup device
Imaging means and
A setting means for setting a subject detection process for an captured image captured by the imaging means, and
A gradation number control means that determines the number of gradations of the captured image after reducing the number of gradations according to the set type of detection processing and converts the captured image to the determined gradation number.
It is provided with a communication means for transmitting the captured image to which the number of gradations is converted to the information processing apparatus.
The gradation number control means
When the detection process is not set by the setting means, the first gradation number is determined to be the gradation number after the gradation number reduction.
When the detection process is set by the setting means, a second gradation number larger than the first gradation number is determined as the gradation number after the gradation number reduction.
The information processing device
A communication means for receiving the captured image in which the number of gradations transmitted from the imaging device is converted, and
An imaging system including a detection means for executing the set detection process from the received image to which the number of gradations is converted. The image pickup device
A motion detecting means for detecting the motion of the subject in the captured image is further provided.
The gradation number control means
When the detection means does not detect the movement of the subject and when the detection process is not set by the setting means, the first gradation number is reduced to the gradation number after the gradation number is reduced. Decided to
When the motion detecting means detects the motion of the subject and the detection process is not set by the setting means, the second gradation number is determined to be the gradation number after reducing the gradation number. The imaging system according to claim 12, wherein the image pickup system is made. It is a control method of an image processing device.
The setting means sets the subject detection process for the captured image, and
The gradation number control means determines the gradation number of the captured image after the gradation number is reduced according to the type of the set detection process, and converts the captured image into the determined gradation number. Number control process and
A detection step in which the detection means executes the set detection process from the captured image to which the number of gradations has been converted.
The output means includes an output step of outputting the captured image to which the number of gradations is converted and a detection result when the detection process is executed in the detection step.
In the gradation number control step,
When the detection process is not set in the setting step, the first gradation number is determined to be the gradation number after the gradation number reduction.
When the detection process is set in the setting step, an image characterized in that a second gradation number larger than the first gradation number is determined as the gradation number after the gradation number reduction. How to control the processing device. It is a control method of an image processing device.
The setting means sets the subject detection process for the captured image, and
A detection step in which the detection means executes the set detection process from the captured image, and
The gradation number control means determines the gradation number of the captured image after reducing the gradation number according to the set type of detection processing and the detection result of the detection processing, and obtains the determined gradation number. The gradation number control step for converting the captured image and
The output means includes an output step of outputting the captured image to which the number of gradations is converted and a detection result when the detection process is executed in the detection step.
In the gradation number control step,
When the detection process is not set in the setting step, or when a predetermined detection result is not obtained by executing the set detection process, the first gradation number is reduced. Decide on the number of gradations later,
When the detection process is set in the setting step and when the set detection process is executed and the predetermined detection result is obtained, the number of gradations is larger than the first gradation number. A control method of an image processing apparatus, characterized in that the second gradation number is determined to be the gradation number after the gradation number is reduced. A program for operating a computer as the image processing device according to any one of claims 1 to 10.

Images