JP7310926B2 - Analysis device and analysis program - Google Patents

Description

The present invention relates to an analysis device and an analysis program.

In general, when recording or transmitting image data, the data size is reduced by image compression processing, thereby reducing recording costs and transmission costs.

On the other hand, in recent years, there has been an increasing number of cases where image data is recorded or transmitted for the purpose of being used for image recognition processing by AI (Artificial Intelligence). Typical AI models include, for example, models using deep learning and machine learning.

JP 2018-101406 A JP 2019-079445 A JP 2011-234033 A

However, conventional compression processing is performed based on human visual characteristics, not based on AI motion analysis. For this reason, there have been cases where compression processing has not been performed at a sufficient compression level for regions that are not required for image recognition processing by AI.

An object of one aspect is to realize compression processing suitable for image recognition processing by AI.

According to one aspect, the analysis device comprises:
Recognition of each region of each decoded data calculated by performing recognition processing on decoded data obtained by decoding each compressed data when compression processing is performed on image data at different compression levels. a storage unit for storing information indicating the degree of influence on the result;
a determination unit that determines the compression level of each region of the image data based on information indicating the degree of influence of each region of the decoded data corresponding to the different compression levels on the recognition result.

Compression processing suitable for image recognition processing by AI can be realized.

FIG. 1 is a first diagram showing an example of the system configuration of a compression processing system. FIG. 2 is a diagram illustrating an example of a hardware configuration of an analysis device or an image compression device; FIG. 3 is a first diagram illustrating an example of a functional configuration of an analysis device; FIG. 4 is a diagram showing a specific example of the aggregate result. FIG. 5 is a first diagram showing a specific example of processing by the quantization value determination unit. FIG. 6 is a first diagram showing an example of the functional configuration of the image compression device. FIG. 7 is a first flowchart showing an example of the flow of image compression processing by the compression processing system. FIG. 8 is a second diagram illustrating an example of the functional configuration of the analysis device; FIG. 9 is a second diagram showing a specific example of processing by the quantization value determination unit. FIG. 10 is a second flowchart showing an example of the flow of image compression processing by the compression processing system. FIG. 11 is a third diagram illustrating an example of the functional configuration of the analysis device; FIG. 12 is a third diagram showing a specific example of processing by the quantization value determination unit. FIG. 13 is a third flowchart showing an example of the flow of image compression processing by the compression processing system. FIG. 14 is a fourth diagram illustrating an example of the functional configuration of the analysis device; FIG. 15 is a diagram illustrating a specific example of processing by the quantization value setting unit; FIG. 16 is a fourth flowchart showing an example of the flow of image compression processing by the compression processing system. FIG. 17 is a fourth diagram showing a specific example of processing by the quantization value determination unit. FIG. 18 is a fifth flowchart showing an example of the flow of image compression processing by the compression processing system. FIG. 19 is a fifth diagram showing a specific example of processing by the quantization value determination unit. FIG. 20 is a sixth flowchart showing an example of the flow of image compression processing by the compression processing system. FIG. 21 is a fifth diagram illustrating an example of the functional configuration of the analysis device; FIG. 22 is a diagram showing a specific example of processing by the invalid area determination unit. FIG. 23 is a diagram showing a specific example of invalidation image data. FIG. 24 is a seventh flowchart showing an example of the flow of image compression processing by the compression processing system. FIG. 25 is a sixth diagram illustrating an example of the functional configuration of the analysis device; FIG. 26 is a diagram illustrating a specific example of processing by the effective area determination unit; FIG. 27 is an eighth flowchart showing an example of the flow of image compression processing by the compression processing system. FIG. 28 is a seventh diagram showing an example of the functional configuration of the analysis device; FIG. 29 is a second diagram showing a specific example of processing by the valid area determination unit. FIG. 30 is a ninth flowchart showing an example of the flow of image compression processing by the compression processing system.

Each embodiment will be described below with reference to the accompanying drawings. In the present specification and drawings, constituent elements having substantially the same functional configuration are denoted by the same reference numerals, thereby omitting redundant description.

[First embodiment]
<System configuration of compression processing system>
First, the system configuration of the entire compression processing system including the analysis device according to the first embodiment will be described. FIG. 1 is a first diagram showing an example of the system configuration of a compression processing system. In the first embodiment, the processing executed by the compression processing system is roughly divided into a phase of determining the compression level (quantization value) and a phase of performing compression processing based on the determined compression level (quantization value). can do.

In FIG. 1, 1a shows the system configuration of the compression processing system in the phase of determining the compression level (quantization value), and 1b is the phase of performing compression processing based on the determined compression level (quantization value). 1 shows the system configuration of a compression processing system in .

As shown in FIG. 1a, the compression processing system in the phase of determining the compression level (quantization value) includes an imaging device 110, an analysis device 120, and an image compression device .

The image capturing device 110 captures images at predetermined frame intervals and transmits image data to the analysis device 120 . Note that the image data includes an object to be recognized.

The analysis device 120 has a trained model that performs recognition processing, and inputs image data or decoded data obtained by decoding compressed data obtained when compression processing is performed on image data at different compression levels to the trained model. By doing so, recognition processing is performed and the recognition result is output.

In addition, the analysis device 120 generates a map (referred to as an important feature map) indicating the degree of influence on the recognition result by performing motion analysis of the trained model using, for example, the error backpropagation method, and for each predetermined region. In addition, the analysis device 120 instructs the image compression device 130 to perform compression processing at different compression levels (quantization values). Similar processing is repeated for each piece of compressed data when compression processing is performed at the level.

Each time the analysis device 120 instructs the image compression device 130 to perform compression processing at a different compression level, the analysis device 120 calculates a total value of the degree of influence of each block, and changes the total value for each compression level (each quantization value). determines the optimal compression level (quantization value) for each block. The optimum compression level (quantization value) refers to the maximum compression level (quantization value) at which an object included in image data can be correctly recognized.

In this way, by analyzing the behavior of the trained model and calculating the degree of influence on the recognition result, the analysis device 120 can optimize compression processing suitable for image recognition processing using the trained model. compression level can be determined.

On the other hand, as shown in 1b of FIG. 1, the compression processing system in the phase of performing compression processing based on the determined compression level (quantization value) includes an analysis device 120, an image compression device 130, and a storage device 140. .

The analysis device 120 transmits the optimum compression level (quantization value) determined for each block and the image data to the image compression device 130 .

The image compression device 130 compresses the image data using the determined optimum compression level (quantization value) and stores the compressed data in the storage device 140 .

Thus, the analysis device 120 according to this embodiment uses a compression level suitable for image recognition processing using a trained model. In other words, the analyzing apparatus 120 according to the present embodiment can realize compression processing suitable for image recognition processing using a trained model by having the following differences from conventional compression processing.
・In the first place, conventional compression processing is not based on the feature part focused on inference (it is based on the shape, property, object of interest, etc. that can be grasped by human concepts), and the feature part focused on inference (not necessarily Characteristic parts that cannot be demarcated by human concepts) are not used.
・In conventional compression processing, the internal operation of the CNN unit 320, which is the process of outputting the recognition result (for example, the signal/processing result propagation process from inputting image data to outputting the recognition result, signal/ Propagation intensity of the processing result) is not analyzed.

<Hardware Configuration of Analysis Device or Image Compression Device>
Next, hardware configurations of the analysis device 120 and the image compression device 130 will be described. Note that since the analysis device 120 and the image compression device 130 have the same hardware configuration, the two devices will be collectively described here using FIG.

FIG. 2 is a diagram illustrating an example of a hardware configuration of an analysis device or an image compression device; The analysis device 120 or the image compression device 130 has a processor 201 , a memory 202 , an auxiliary storage device 203 , an I/F (Interface) device 204 , a communication device 205 and a drive device 206 . Each piece of hardware of the analysis device 120 or the image compression device 130 is interconnected via a bus 207 .

The processor 201 has various computing devices such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The processor 201 reads out various programs (for example, an analysis program or an image compression program to be described later) onto the memory 202 and executes them.

The memory 202 has main storage devices such as ROM (Read Only Memory) and RAM (Random Access Memory). The processor 201 and the memory 202 form a so-called computer, and the processor 201 executes various programs read on the memory 202, thereby realizing various functions (details of the various functions will be described later).

The auxiliary storage device 203 stores various programs and various data used when the various programs are executed by the processor 201 .

The I/F device 204 is a connection device that connects the operation device 210 and the display device 220, which are examples of external devices, with the analysis device 120 or the image compression device 130. FIG. The I/F device 204 receives operations for the analysis device 120 or the image compression device 130 via the operation device 210 . The I/F device 204 also outputs the result of processing by the analysis device 120 or the image compression device 130 and displays it via the display device 220 .

A communication device 205 is a communication device for communicating with other devices. The analysis device 120 communicates with the imaging device 110 and the image compression device 130 via the communication device 205 . Also, in the case of the image compression device 130 , it communicates with the analysis device 120 and the storage device 140 via the communication device 205 .

A drive device 206 is a device for setting a recording medium 230 . The recording medium 230 here includes media for optically, electrically or magnetically recording information such as a CD-ROM, a flexible disk, a magneto-optical disk, and the like. The recording medium 230 may also include a semiconductor memory or the like that electrically records information, such as a ROM or flash memory.

Various programs to be installed in the auxiliary storage device 203 are installed by, for example, setting the distributed recording medium 230 in the drive device 206 and reading the various programs recorded in the recording medium 230 by the drive device 206. be done. Alternatively, various programs installed in the auxiliary storage device 203 may be installed by being downloaded from the network via the communication device 205 .

<Functional configuration of analysis device>
Next, the functional configuration of the analysis device 120 will be described. FIG. 3 is a first diagram illustrating an example of a functional configuration of an analysis device; As described above, the analysis program is installed in the analysis device 120, and when the program is executed, the analysis device 120 includes the input unit 310, the CNN unit 320, the quantization value setting unit 330, the output unit 340. The analysis device 120 also functions as an important feature map generation unit 350 , totalization unit 360 and quantization value determination unit 370 .

The input unit 310 acquires image data transmitted from the imaging device 110 or compressed data transmitted from the image compression device 130 . The input unit 310 notifies the CNN unit 320 and the output unit 340 of the obtained image data, decodes the obtained compressed data using a decoding unit (not shown), and notifies the CNN unit 320 of the decoded data.

The CNN unit 320 has a trained model, receives image data or decoded data, performs recognition processing on an object included in the image data or decoded data, and outputs a recognition result.

The quantization value setting unit 330 sequentially notifies the output unit 340 of the compression levels (from the minimum quantization value (initial value) to the maximum quantization value) used when the image compression device 130 performs compression processing. Together with this, it is stored in the tally result storage unit 380, which is an example of a storage unit.

The output unit 340 transmits the image data acquired by the input unit 310 to the image compression device 130 . Also, each quantization value notified from the quantization value setting unit 330 is sequentially transmitted to the image compression device 130 . Furthermore, the quantization value determined by the quantization value determination unit 370 (determined quantization value) is transmitted to the image compression device 130 .

The important feature map generation unit 350 is an example of a map generation unit, acquires CNN unit structure information when a trained model performs recognition processing on image data or decoded data, and based on the acquired CNN unit structure information A key feature map is generated by using the error backpropagation method.

The important feature map generation unit 350 generates an important feature map by using, for example, a BP (Back Propagation) method, a GBP (Guided Back Propagation) method, or a selective BP method.

In the BP method, the error of each label is calculated from the classification probability obtained by performing recognition processing on the image data (or decoded data) whose recognition result is the correct label. This is a method of visualizing the characteristic portion by imaging the magnitude of the gradient that is applied. The GBP method is a method of visualizing a characteristic portion by imaging only positive values of gradient information as a characteristic portion.

Furthermore, the selective BP method is a method of maximizing only the error of the correct label and then backpropagating using the BP method or the GBP method. In the selective BP method, the visualized features are those that affect only the score of the correct label.

In this way, the important feature map generating unit 350 uses the BP method, the GBP method, or the selective BP method, so that the inside of the CNN unit 320 from the input of image data or decoded data to the output of the recognition result is Analyze the signal flow and strength of each path. As a result, the important feature map generator 350 can visualize which part of the input image data or decoded data affects the recognition result and to what degree. Therefore, for example, when an AI that does not apply (or cannot apply) the BP method, the GBP method, or the selective BP method is used as the CNN unit 320, the important feature map generation unit 350 analyzes similar information, Generate an important feature map.

Note that the method of generating the important feature map by the error backpropagation method is, for example,
"Selvaraju, Ramprasaath R., et al. "Grad-cam: Visual explanations from deep networks via gradient-based localization." The IEEE International Conference on Computer Vision (ICCV), 2017, pp. 618-626",
et al.

Aggregation unit 360 aggregates the degree of influence on the recognition result for each block based on the important feature map, and calculates the aggregate value of the degree of influence for each block. In addition, the totalization unit 360 stores the calculated totalized value of each block in the totalization result storage unit 380 in association with the quantized value.

The quantization value determination unit 370 is an example of a determination unit, and based on the total value of each block (the number of total values corresponding to the number of quantization values) stored in the total result storage unit 380, Determine the optimal quantization value. In addition, the quantization value determination unit 370 notifies the output unit 340 of the determined optimal quantization value for each block.

In this way, in the analysis device 120, the degree of tolerance (quantization value) of deterioration (effect on recognition accuracy) due to the compression processing of the characteristic portion that is important when the CNN unit 320 performs the recognition processing is determined by human is calculated based on the concept recognized by the CNN unit 320 rather than the concept recognized by the CNN unit 320 .

<Specific example of aggregated results>
Next, a specific example of the tally result stored in the tally result storage unit 380 will be described. FIG. 4 is a diagram showing a specific example of the aggregate result. Among them, 4a shows an arrangement example of blocks in the image data 410. FIG. As shown in 4a, in this embodiment, all blocks in the image data 410 are assumed to have the same size for simplification of explanation. Also, let the block number of the upper left block of the image data be "block 1" and the block number of the lower right block be "block m".

As shown in 4b, the counting result 420 includes "block number" and "quantization value" as information items.

“Block number” stores the block number of each block in the image data 410 . The "quantization value" includes "no compression" indicating that the image compression device 130 does not perform compression processing, and the minimum quantization value (" _Q1 ") to the maximum quantized value ("Q _n ") is stored.

Also, in the area specified by "block number" and "quantization value",
- performing a compression process on the image data 410 using the corresponding quantized values;
・By inputting the decoded data obtained by decoding the acquired compressed data, the trained model performs recognition processing,
・Based on the important feature map calculated when recognition processing was performed, aggregated in the corresponding block,
Aggregate value is stored.

<Specific example of processing by the quantization value determination unit>
Next, a specific example of processing by the quantization value determination unit 370 will be described. FIG. 5 is a first diagram showing a specific example of processing by the quantization value determination unit. In FIG. 5, graphs 510_1 to 510_m are graphs generated by plotting the total values of each block included in the total result 420, with the horizontal axis representing the quantization value and the vertical axis representing the aggregate value.

Note that the total value of each block used to generate the graphs 510_1 to 510_m is, for example,
- It may be adjusted using an offset value common to all blocks.
- The absolute value may be taken and aggregated.
- The aggregate values of other blocks may be processed based on the aggregate values of blocks that have not received attention.

As shown in graphs 510_1 to 510_m, changes in aggregated values when changing from the minimum quantization value (Q ₁ ) to the maximum quantization value (Q _n ) differ for each block. In the quantization value determination unit 370, for example,
・When the size of the aggregated value exceeds a predetermined threshold, or
・If the amount of change in the aggregate value exceeds a predetermined threshold, or
・If the slope of the aggregate value exceeds a predetermined threshold, or
・If the change in the slope of the aggregate value exceeds a predetermined threshold,
Determine the optimum quantization value for each block if any of the conditions of are satisfied.

In FIG. 5, reference numeral 530 indicates how B ₁ Q to B _m Q are determined as the optimum quantization values for blocks 1 to m and set to the corresponding blocks.

It should be noted that the size of blocks used for aggregation and the size of blocks used for compression processing do not have to match. In that case, the quantization value determining section 370 determines the quantization value as follows, for example.
・When the size of the block used for compression processing is larger than the size of the block used for aggregation The average value of the quantization values (or , minimum value, maximum value, and values processed by other indexes) is used as the quantization value of each block used for compression processing.
・When the size of the block used for compression processing is smaller than the size of the block used for aggregation. Use value-based quantization values.

Also, the quantized value indicated by reference numeral 530 may be additionally evaluated by the analysis device 120 . Specifically, first, the analysis device 120 decodes the compressed data that has been compressed using the quantization value indicated by reference numeral 530, and performs recognition processing on the decoded data. Subsequently, the analysis device 120 adds the quantization value (for example, 1 addition) to the minimum value among the quantization values indicated by reference numeral 530 to change the quantization value indicated by reference numeral 530 . At this time, if there are a plurality of minimum values among the quantized values indicated by reference numeral 530, similar addition is performed.

Subsequently, the analysis device 120 decodes the compressed data compressed using the changed quantization value indicated by reference numeral 530, and performs recognition processing on the decoded data.

In the analysis device 120, these processes are repeated until the maximum value of the quantized values indicated by reference numeral 530 is reached, and a plurality of sets of the changed quantized values indicated by reference numeral 530 and the corresponding recognition results are acquired. .

Subsequently, the analysis device 120 selects a set whose recognition accuracy exceeds the allowable lower limit from among the plurality of sets and whose minimum quantization value is the maximum, and the selected set includes a change The quantized value indicated by reference numeral 530 is used to replace the quantized value indicated by reference numeral 530 (before change).

By additionally evaluating the quantized value indicated by reference numeral 530 in this manner, a quantized value with a higher compression rate than the quantized value indicated by reference numeral 530 can be determined.

<Functional Configuration of Image Compression Apparatus>
Next, the functional configuration of the image compression device 130 will be described. FIG. 6 is a first diagram showing an example of the functional configuration of the image compression device. As described above, the image compression program is installed in the image compression device 130 , and the image compression device 130 functions as the encoding unit 620 by executing the program.

The encoder 620 is an example of a compressor. The coding section 620 has a difference section 621 , an orthogonal transformation section 622 , a quantization section 623 , an entropy coding section 624 , an inverse quantization section 625 and an inverse orthogonal transformation section 626 . The encoding unit 620 also has an addition unit 627 , a buffer unit 628 , an in-loop filter unit 629 , a frame buffer unit 630 , an intra-screen prediction unit 631 and an inter-screen prediction unit 632 .

A difference unit 621 calculates a difference between image data (for example, the image data 410) and predicted image data, and outputs a prediction residual signal.

The orthogonal transform section 622 performs orthogonal transform processing on the prediction residual signal output from the difference section 621 .

A quantization unit 623 quantizes the orthogonally transformed prediction residual signal to generate a quantized signal. The quantization section 623 generates a quantized signal using the quantized value indicated by reference numeral 530 (the quantized value transmitted from the analysis device 120 or the determined optimum quantized value).

The entropy coding unit 624 generates compressed data by performing entropy coding processing on the quantized signal.

An inverse quantization unit 625 inversely quantizes the quantized signal. The inverse orthogonal transform unit 626 performs inverse orthogonal transform processing on the inversely quantized quantized signal.

The addition unit 627 adds the signal output from the inverse orthogonal transformation unit 626 and the predicted image data to generate reference image data. A buffer unit 628 stores the reference image data generated by the addition unit 627 .

The in-loop filter unit 629 filters the reference image data stored in the buffer unit 628 . The in-loop filter unit 629 includes
- Deblocking filter (DB),
・Sample Adaptive Offset filter (SAO),
- Adaptive loop filter (ALF),
is included.

The frame buffer unit 630 stores the reference image data filtered by the in-loop filter unit 629 in units of frames.

The intra-screen prediction unit 631 performs intra-screen prediction based on the reference image data to generate predicted image data. The inter-screen prediction unit 632 performs motion compensation between frames using the input image data (for example, image data 410) and reference image data to generate predicted image data.

Note that the predicted image data generated by the intra-screen prediction unit 631 or the inter-screen prediction unit 632 is output to the difference unit 621 and addition unit 627 .

Note that in the above description, the encoding unit 620 uses MPEG-2, MPEG-4, H.264, and MPEG-2. It is assumed that compression processing is performed using an existing moving picture encoding method such as H.264 and HEVC. However, the compression processing by the encoding unit 620 is not limited to these moving image encoding methods, and may be performed using any encoding method that controls the compression rate by parameters such as quantization.

<Flow of image compression processing by compression processing system>
Next, the flow of image compression processing by the compression processing system 100 will be described. FIG. 7 is a first flowchart showing an example of the flow of image compression processing by the compression processing system.

In step S701, the quantization value setting unit 330 initializes the compression level (sets the minimum quantization value (Q ₁ )) and sets the upper limit of the compression level (maximum quantization value (Q _n )).

In step S702, the input unit 310 acquires image data or compressed data in units of frames. Further, when the input unit 310 acquires compressed data, it decodes the acquired compressed data to generate decoded data.

In step S703, the CNN unit 320 performs recognition processing on image data (or decoded data) and outputs recognition results.

In step S704, the important feature map generator 350 generates an important feature map indicating the degree of influence of each region on the recognition result based on the CNN structure information.

In step S705, the tabulation unit 360 tabulates the degree of influence of each region on a block-by-block basis based on the important feature map. Also, the counting unit 360 stores the counting result in the counting result storage unit 380 in association with the current compression level (quantization value).

In step S<b>706 , the output unit 340 transmits the image data and the current compression level (quantization value) to the image compression device 130 . Also, the image compression device 130 compresses the transmitted image data at the current compression level (quantization value) to generate compressed data.

In step S707, the quantization value setting unit 330 increases the compression level (here, sets the quantization value (Q ₂ )).

In step S708, the quantization value determination unit 370 determines whether the current compression level has exceeded the upper limit (whether the current quantization value has exceeded the maximum quantization value (Q _n )). If it is determined in step S708 that the current compression level does not exceed the upper limit (No in step S708), the process returns to step S702.

In this case, in step S702, the compressed data generated in step S706 is acquired, and the process of steps S703 to S707 is performed on the decoded data obtained by decoding the acquired compressed data.

On the other hand, if it is determined in step S708 that the current compression level has exceeded the upper limit (if Yes in step S708), the process proceeds to step S709.

In step S<b>709 , the quantization value determination unit 370 determines the optimum compression level (optimum quantization value) for each block based on the totalization result stored in the totalization result storage unit 380 . Also, the output unit 340 transmits the determined optimal quantization value to the image compression device 130 .

In step S<b>710 , the image compression device 130 performs compression processing on the image data using the determined optimum quantization value, and stores the compressed data in the storage device 140 .

As is clear from the above description, the analysis apparatus according to the first embodiment acquires each piece of compressed data when compression processing is performed on image data using different quantization values. Further, the analysis device according to the first embodiment inputs decoded data obtained by decoding each compressed data to a trained model, and based on the CNN unit structure information when recognition processing is performed, the degree of influence on the recognition result Generate a key feature map showing Further, the analysis device according to the first embodiment aggregates the degree of influence for each block based on the important feature map, and calculates the degree of influence of each block of the image data based on the aggregated value of each block corresponding to different compression levels. Determine compression level.

Thus, according to the first embodiment, compression processing can be performed using the optimum quantization value determined based on the degree of influence on the recognition result. That is, according to the first embodiment, compression processing suitable for image recognition processing by AI can be realized.

[Second embodiment]
In the above-described first embodiment, all values from the minimum quantization value to the maximum quantization value that can be set in the image compression device 130 are used in determining the optimum quantization value based on the degree of influence on the recognition result. described as a thing.

On the other hand, in the second embodiment, a case will be described in which an optimal quantization value is determined by performing compression processing using a predetermined quantization value. The second embodiment will be described below, focusing on differences from the first embodiment.

<Functional configuration of analysis device>
First, the functional configuration of an analysis device 120 according to the second embodiment will be described. FIG. 8 is a second diagram illustrating an example of the functional configuration of the analysis device; The difference from the functional configuration shown in FIG. This is a point different from the function of the unit 370 . Another difference is that the analysis device 120 has a group information storage unit 830 instead of the tally result storage unit 380 .

Maximum quantization value setting section 810 notifies output section 340 of the maximum quantization value (Q _n ). The quantization value determination unit 820 determines the group to which the aggregate value of each block notified by the aggregation unit 360 belongs from the group information stored in the group information storage unit 830, which is an example of the storage unit. In addition, quantization value determination section 820 notifies output section 340 of the optimum quantization value associated in advance with the determined group.

<Specific example of processing by the quantization value determination unit>
Next, a specific example of processing by the quantization value determination unit 820 will be described. FIG. 9 is a second diagram showing a specific example of processing by the quantization value determination unit.

As shown in FIG. 9, group information 910 includes a plurality of typical patterns of total values when changing from the minimum quantized value to the maximum quantized value (graphs 911 to 913 in the example of FIG. A group containing three patterns shown in ) is defined. Also, the group information 910 defines an optimum quantization value for each group. The example in FIG. 13 is
- For group 1, the optimal quantization value G ₁ Q is
- For group 2, the optimal quantization value G ₂ Q is
- For group 3, the optimal quantization value G ₃ Q is
It indicates that they are associated with each other.

In the quantization value determination unit 820, recognition processing is performed on the decoded data obtained by decoding the compressed data when the image data is compressed using the maximum quantization value (Q _n ) from the counting unit 360. Get the total value of each block, calculated by what is done. Also, the quantization value determination unit 820 determines to which group the total value of each block belongs.

Furthermore, the quantization value determination unit 820 notifies the output unit 340 of the quantization value associated with the determined group as the optimum quantization value for each block.

Although only one type of group information 910 is shown in the example of FIG. 9, there may be multiple types of group information. For example, different group information may be prepared for each type of object to be recognized. Alternatively, different group information may be prepared for each complexity of image data.

In the example of FIG. 9, the group information 910 includes graphs 911 to 913, but may include approximate functions, deep learning models, and the like.

In addition, in the _{example of FIG. 9, the maximum quantization value (Q n )} is used to determine the group. Multiple quantization values may be used that do not include the quantization value (Q _n ).

<Flow of image compression processing by compression processing system>
Next, the flow of image compression processing by the compression processing system 100 will be described. FIG. 10 is a second flowchart showing an example of the flow of image compression processing by the compression processing system.

In step S1001, maximum quantization value setting section 810 sets the maximum compression level (maximum quantization value (Q _n )).

In step S1002, the input unit 310 acquires image data on a frame-by-frame basis.

In step S<b>1003 , the output unit 340 transmits the image data and the maximum compression level (maximum quantization value (Q _n )) to the image compression device 130 . The image compression device 130 also performs compression processing on the transmitted image data at the maximum compression level (maximum quantization value (Q _n )) to generate compressed data.

In step S<b>1004 , the input unit 310 acquires and decodes the compressed data generated by the image compression device 130 . Also, the CNN unit 320 performs recognition processing on the decoded data and outputs the recognition result.

In step S1005, the important feature map generator 350 generates an important feature map indicating the degree of influence on the recognition result based on the CNN structure information.

In step S1006, the tabulation unit 360 tabulates the degree of influence of each region on a block-by-block basis based on the important feature map. Aggregation unit 360 also notifies quantization value determination unit 820 of the aggregation result.

In step S1007, the quantization value determination unit 820 refers to the group information stored in the group information storage unit 830, and determines to which group the total value of each block notified by the totalization unit 360 belongs. . Accordingly, the quantization value determination unit 820 groups each block.

In step S1008, the quantization value determination unit 1220 determines the optimum quantization value associated with each group determined for each block as the optimum quantization value for each block. Also, the output unit 340 transmits the determined optimal quantization value to the image compression device 130 .

In step S<b>1009 , the image compression device 130 performs compression processing on the image data using the determined optimal quantization value, and stores the compressed data in the storage device 140 .

As is clear from the above description, the analysis apparatus according to the second embodiment acquires compressed data when compression processing is performed on image data using the maximum quantization value. Further, the analysis device according to the second embodiment determines the degree of influence on the recognition result based on the CNN structure information when the decoded data obtained by decoding the compressed data is input to the trained model and the recognition process is performed. Generate a key feature map shown. Further, the analysis device according to the second embodiment aggregates the degree of influence for each block based on the important feature map, and determines the group to which the aggregated value belongs, so that the quantized value associated with the group is Determine the optimum quantization value.

Thus, according to the second embodiment, compression processing can be performed using the optimum quantization value determined based on the degree of influence on the recognition result. That is, according to the second embodiment, the same effects as those of the first embodiment are obtained. In addition, according to the second embodiment, it is possible to determine the optimum quantization value with a smaller number of compression processes than in the first embodiment.

[Third Embodiment]
In the above-described second embodiment, in determining the group to which the aggregated value belongs, it is assumed that the recognition process is performed on the decoded data obtained by decoding the compressed data when the compression process is performed using the maximum quantized value. bottom. In contrast, in the third embodiment, pseudo-compressed data (pseudo-compressed data) is generated by performing image processing having the same effect as compression processing using the maximum quantization value. Then, recognition processing is performed on the pseudo-compressed data. As a result, according to the third embodiment, the optimal quantization value can be determined with a smaller number of compression processes than in the second embodiment. The third embodiment will be described below, focusing on differences from the second embodiment.

<Functional configuration of analysis device>
First, the functional configuration of an analysis device 120 according to the third embodiment will be described. FIG. 11 is a third diagram illustrating an example of the functional configuration of the analysis device; The difference from the functional configuration shown in FIG. 8 is that the maximum quantization value setting section 810 is not included and the image processing section 1110 is included.

The image processing unit 1110 performs filtering processing on the image data acquired by the input unit 310 using, for example, a low-pass filter. As a result, the image processing unit 1110 generates pseudo-compressed data having the same effect as compressing image data using the maximum quantization value.

Also, the image processing unit 1110 inputs the generated pseudo-compressed data to the CNN unit 320 . As a result, the CNN unit 320 performs recognition processing on the pseudo-compressed data, and the important feature map generation unit 350 generates an important feature map based on the CNN structure information. Furthermore, the aggregation unit 360 aggregates the important feature map for each block, and the quantization value determination unit 820 determines the group to which the aggregation value of each block belongs from the group information stored in the group information storage unit 830. , the optimum quantization value is notified to the output unit 340 .

<Specific example of processing by the quantization value determination unit>
Next, a specific example of processing by the quantization value determination unit 820 will be described. FIG. 12 is a third diagram showing a specific example of processing by the quantization value determination unit. The difference from FIG. 9 is that the quantization value determination unit 820 acquires the total value of each block when recognition processing is performed on pseudo-compressed data filtered using a low-pass filter. is.

Note that quantization value determination section 820 determines which group each block belongs to based on the obtained aggregated value of each block, and assigns the optimum quantization value associated with the determined group to each The output unit 340 is notified as the optimum quantized value of the block.

<Flow of image compression processing by compression processing system>
Next, the flow of image compression processing by the compression processing system 100 will be described. FIG. 13 is a third flowchart showing an example of the flow of image compression processing by the compression processing system. The difference from the second flowchart shown in FIG. 10 is that the process of step S1001 is not included, and that the processes of steps S1301 and S1302 are included instead of steps S1003 and S1004.

In step S<b>1301 , the image processing unit 1110 generates pseudo image data by filtering using a low-pass filter, and inputs the pseudo image data to the CNN unit 320 .

In step S1302, the input unit 310 acquires pseudo image data, the CNN unit 320 performs recognition processing on the acquired pseudo image data, and outputs a recognition result.

As is clear from the above description, the analysis apparatus according to the third embodiment performs filtering processing on image data to acquire pseudo-compressed data. Further, the analysis apparatus according to the third embodiment provides an important feature map indicating the degree of influence on the recognition result based on the CNN part structure information when the pseudo-compressed data is input to the trained model and the recognition process is performed. to generate Further, the analysis apparatus according to the third embodiment aggregates the degree of impact for each block based on the important feature map, and determines the group to which the aggregated value belongs, so that the quantized value associated with the group is Determine the optimum quantization value.

Thus, according to the third embodiment, compression processing can be performed using the optimum quantization value determined based on the degree of influence on the recognition result. That is, according to the third embodiment, the same effects as those of the first embodiment are obtained. In addition, according to the third embodiment, the optimal quantization value can be determined with a smaller number of compression processes than the first and second embodiments.

[Fourth embodiment]
In the first embodiment, the compression process is performed using a different quantization value each time image data in units of frames is input, and the optimum quantization value is determined. In contrast, in the fourth embodiment, compression processing is performed using different quantization values while a plurality of frames of image data are input, and the optimum quantization value is determined. The fourth embodiment will be described below, focusing on differences from the first embodiment.

<Functional configuration of analysis device>
First, the functional configuration of an analysis device 120 according to the fourth embodiment will be described. FIG. 14 is a fourth diagram illustrating an example of the functional configuration of the analysis device; The difference from the functional configuration shown in FIG. 370 and the tally result storage unit 380 are not included.

Position determination section 1410 extracts position information of an object included in decoded data obtained by decoding image data or compressed data from the recognition result output from CNN section 320 . In addition, position determination section 1410 notifies quantization value setting section 1420 of the extracted position information.

The quantization value setting unit 1420 notifies the output unit 340 of the compression level (quantization value) used when the image compression device 130 performs compression processing. The quantization value setting section 1420 sequentially notifies the output section 340 of the quantization values obtained by starting from the minimum quantization value and adding them in predetermined increments.

Also, the quantization value setting unit 1420 monitors the total value of each block notified from the totalization unit 360 each time the quantization value is notified, and if the total value of each block exceeds a predetermined threshold, , lower the quantization value. In this manner, the quantization value setting section 1420 can control the quantization value to be notified so that the total value does not exceed the predetermined threshold.

Note that the quantization value setting unit 1420 identifies blocks whose aggregate values are to be monitored based on the object position information notified by the position determination unit 1410, and sets the quantization values of the identified blocks to the aggregate values of the identified blocks. Control based on value.

<Specific example of processing by the quantization value setting unit>
Next, a specific example of processing by the quantization value setting unit 1420 will be described. FIG. 15 is a diagram illustrating a specific example of processing by the quantization value setting unit; In FIG. 15, decoded data 1511 to 1514 obtained by decoding compressed data respectively indicate decoded data obtained by decoding compressed data obtained by the input unit 310 at times=t ₁ to t ₄ .

Decoded data 1511 to 1514 obtained by decoding compressed data each include an object 1521 . The example in FIG. 15 shows how an object 1521 moves over time from the lower left to the upper right within decoded data 1511 to 1514 obtained by decoding compressed data.

Quantization value setting section 1420 identifies the position of object 1521 in decoded data 1511 to 1514 obtained by decoding compressed data based on the position information notified from position determination section 1410 .

Also, the quantization value setting unit 1420 acquires the aggregate value of each block included in the specified position from the aggregation unit 360 . In FIG. 15, reference numerals 1531 to 1534 indicate total values of blocks included in the specified position, which the quantization value setting unit 1420 is notified from the totalizing unit 360. In FIG.

The example of FIG. 15 shows how the quantization value setting section 1420 notifies the quantization values Q _x+1 , Q _x+2 , and Q _x+3 in predetermined increments (however, Q _x+1 <Q _x+2 <Q _x+3 ). .

Here, the total number of blocks included in the object 1521 calculated by performing the recognition processing on the decoded data 1513 obtained by decoding the compressed data when the compression processing is performed using the quantization value Q _x+3. Suppose a value (1533) exceeds a predetermined threshold 1530;

In this case, the quantization value setting unit 1420 sets the quantization value to be notified next to a quantization value smaller than the quantization value Qx ₊₃ (in the example of FIG. 15, the quantization value Qx ₊₂ is notified). shown).

In this way, by controlling the quantization value to be notified so that the total value of each block included in the object does not exceed a predetermined threshold value, the quantization value setting unit 1420 can continue the optimum quantization value. can be notified by

<Flow of image compression processing by compression processing system>
Next, the flow of image compression processing by the compression processing system 100 will be described. FIG. 16 is a fourth flowchart showing an example of the flow of image compression processing by the compression processing system. Note that steps S1601 to S1606 differ from the first flowchart shown in FIG.

In step S1601, the tabulation unit 360 tabulates the degree of influence of each region on a block-by-block basis based on the important feature map.

In step S1602, the quantization value setting unit 1420 specifies the position of the object based on the position information notified from the position determination unit 1410, and the aggregate value of each block included in the specified position of the object is a predetermined value. Determine whether or not the threshold is exceeded.

If it is determined in step S1602 that the predetermined threshold is not exceeded (No in step S1602), the process proceeds to step S1603.

In step S<b>1603 , quantization value setting section 1420 adds quantization values in predetermined increments, and notifies output section 1430 of the quantization values after the addition.

On the other hand, if it is determined in step S1602 that the predetermined threshold has been exceeded (Yes in step S1602), the process proceeds to step S1604.

In step S1604, quantization value setting section 1420 subtracts the quantization value by a predetermined step width, and notifies output section 1430 of the quantization value after the subtraction.

In step S<b>1605 , the image compression device 130 performs compression processing on the image data using the quantization value transmitted from the output unit 1430 and stores the compressed data in the storage device 140 .

In step S1606, the input unit 310 determines whether or not to end the image compression processing, and if it is determined not to end (No in step S1606), the process returns to step S702. On the other hand, if it is determined to end in step S1606 (if Yes in step S1606), the image compression process ends.

As is clear from the above description, the analysis apparatus according to the fourth embodiment acquires each piece of compressed data when performing compression processing using different quantization values for each of a plurality of image data. Further, the analysis device according to the fourth embodiment inputs the decoded data obtained by decoding each compressed data to the trained model, and based on the CNN unit structure information when performing the recognition process, the degree of influence on the recognition result Generate a key feature map showing Also, the analysis apparatus according to the fourth embodiment aggregates the important feature map for each block, and obtains the aggregate value of the blocks included in the position of the object. Furthermore, the analysis device according to the fourth embodiment controls the quantization value so that the acquired total value does not exceed a predetermined threshold.

In this way, by controlling the quantization value so that the total value of each block included in the object does not exceed a predetermined threshold value, the analysis apparatus according to the fourth embodiment realizes optimal quantization Values can be output continuously.

[Fifth embodiment]
In the first to third embodiments, the total value is calculated for each block, and the optimum quantization value is determined for each block. On the other hand, in the fifth embodiment, the total value of the reference block is compared, and the optimum quantization value is determined based on the comparison result. The fifth embodiment will be described below, focusing on differences from the first embodiment.

<Specific example of processing by the quantization value determination unit>
FIG. 17 is a fourth diagram showing a specific example of processing by the quantization value determination unit. In FIG. 17, graphs 510_1 to 510_m are the same as the graphs 510_1 to 510_m already described using FIG.

Here, in the example of FIG. 17, the block number=“block 1” is used as a reference block, the total value in this block is “v ₁ ”, and the optimum quantization value in this block is “B ₁ Q”. Suppose there is

In this case, the quantization value determination unit, for example,
- For block 2, optimal quantization value = _B1Q * _v2 / _v1 ,
- For block 3, optimal quantization value = _B1Q * _v3 / _v1 ,
・・・
- For block m, optimal quantization value = _B1Q * _vm / _v1 ,
are calculated respectively. As a result, the quantization value determination unit determines the optimum quantization value 1700 .

<Flow of image compression processing by compression processing system>
FIG. 18 is a fifth flowchart showing an example of the flow of image compression processing by the compression processing system. The difference from the first flowchart shown in FIG. 7 is step S1801.

In step S1801, the quantization value determination unit compares the total value of the reference block with the total value of each block, and determines the optimal quantization value of each block based on the comparison result and the optimum quantization value of the reference block. Determine the optimal quantization value.

In this way, by comparing with the aggregated value of the reference block and determining the optimum quantization value based on the comparison result, according to the fifth embodiment, it is possible to achieve a predetermined compression level regardless of the image data. Compression processing can be performed at the above compression levels. Further, according to the fifth embodiment, quantization values can be matched between blocks.

[Sixth embodiment]
In the first to third embodiments described above, the total value is calculated for each block, and the quantization value is determined based on the calculated total value. In contrast, in the sixth embodiment, the quantization value preset in the image compression device 130 (the quantization value set based on the human visual characteristics) is corrected using the calculated total value. to determine the optimum quantization value. The sixth embodiment will be described below, focusing on differences from the first embodiment.

<Specific example of processing by the quantization value determination unit>
FIG. 19 is a fifth diagram showing a specific example of processing by the quantization value determination unit. In FIG. 19, a quantization value 1900 is a quantization value that is preset in the image compression device 130, and is a quantization value that is set based on human visual characteristics.

Also, in FIG. 19, the tally result 1910 is the tally result when the decoded data obtained by decoding the predetermined compressed data is subjected to recognition processing. The predetermined compressed data here means the quantization value set immediately before the quantization value is set when an erroneous recognition result is output in the recognition processing by the CNN unit 320 for the decoded decoded data. Compressed data when compression processing is performed by

Also, in FIG. 19, the optimum quantized value 1920 is a quantized value calculated based on the quantized value 1900 and the tally result 1910 . As shown in FIG. 19, the optimal quantization value 1920 is calculated based on the following formula (Formula 1).
(Equation 1) Qa (x, y) = Qpb (x, y) + P (x, y) x weighting coefficient In Equation 1, Qa (x, y) is specified by coordinates (x, y) Points to the optimal quantization value for the block. In Expression 1, Qpb(x, y) is the quantization value of the block specified by the coordinates (x, y), which is preset in the image compression device 130 . Further, in Equation 1, P(x, y) is the aggregate result when recognition processing is performed on decoded data obtained by decoding predetermined compressed data of a block specified by coordinates (x, y). Point.

<Flow of image compression processing by compression processing system>
Next, the flow of image compression processing by the compression processing system 100 will be described. FIG. 20 is a sixth flowchart showing an example of the flow of image compression processing by the compression processing system. Differences from the first flowchart shown in FIG. 7 are steps S2001 and steps S2002 to S2005.

In step S2001, the quantization value determination unit determines whether or not the correct recognition result is output from the CNN unit. If it is determined in step S2001 that a correct recognition result has been output (if Yes in step S2001), the process proceeds to step S704.

In step S704, the important feature map generator 350 generates an important feature map indicating the degree of influence of each region on the recognition result based on the CNN structure information.

In step S705, the tabulation unit 360 tabulates the degree of influence of each region on a block-by-block basis based on the important feature map. Also, the counting unit 360 stores the counting result in the counting result storage unit 380 in association with the current compression level (quantization value).

In step S2002, quantization value setting section 330 increases the compression level (quantization value).

In step S<b>2003 , the output unit 340 transmits the image data and the current compression level (quantization value) to the image compression device 130 . The image compression device 130 also performs compression processing on the transmitted image data using the current compression level (quantization value) to generate compressed data.

On the other hand, if it is determined in step S2001 that an erroneous recognition result has been output (No in step S2001), the process proceeds to step S2004.

In step S<b>2004 , the quantization value determination unit multiplies the total value of the finally recognizable decoded data by a weighting factor, and adds it to the quantization value preset in the image compression device 130 .

In step S<b>2005 , the image compression device 130 compresses the image data using the quantization value calculated in step S<b>2004 , and stores the compressed data in the storage device 140 .

In this way, by correcting the quantization value preset in the image compression device (the quantization value set based on the human visual characteristics) using the calculated total value, the sixth embodiment can be achieved. can determine the optimal quantization value.

[Seventh embodiment]
In the first to sixth embodiments, cases have been described in which the degree of influence on the recognition result is aggregated for each block and the optimal quantization value is determined based on the aggregated result. On the other hand, in the eighth embodiment, image data is divided into a valid area and an invalid area based on the totalization result, blocks included in the invalid area are invalidated, and then the valid area is compressed. process.

Note that invalidating a block included in the invalid area means, for example, setting the pixel value of each pixel in the block included in the invalid area to "0". is hereinafter referred to as "invalidation image data".

By performing compression processing on invalidated image data (valid regions included in invalidated image data) in this way, compared to the case where compression processing is performed on the entire image data, the compressed data data size can be further reduced.

Note that when performing compression processing on invalidated image data, a predetermined quantization value may be used. Quantized values may also be used. Further, in the case of a compression method capable of compressing data of any shape, data from which invalid regions are removed from the invalidated image data may be compressed. The seventh embodiment will be described below, focusing on differences from the first embodiment.

<Functional configuration of analysis device>
First, the functional configuration of an analysis device 120 according to the seventh embodiment will be described. FIG. 21 is a fifth diagram illustrating an example of the functional configuration of the analysis device; The difference from the functional configuration shown in FIG. 3 is that instead of the quantization value determination section 370, an invalid area determination section 2110 and an invalid image generation section 2120 are provided.

The invalid area determination unit 2110 determines whether each block is a It is determined whether or not the block belongs to the invalid area.

Note that the invalid area determination unit 2110 first acquires the recognition result from the CNN unit 320 when determining whether or not each block belongs to the invalid area. identify the normalization value. Subsequently, in the invalid area determination unit 2110, based on whether or not the difference between the aggregated value corresponding to the minimum quantized value and the aggregated value of the specified quantized value is equal to or greater than a predetermined threshold, each block is It is determined whether or not the block belongs to the invalid area.

In addition, the invalid area determination unit 2110 notifies the invalid image generation unit 2120 of the blocks determined to belong to the invalid area.

The invalidated image generating unit 2120 generates invalidated image data by invalidating the blocks notified by the invalid area determination unit 2110 among the blocks included in the image data. In addition, the invalidation image generation unit 2120 notifies the output unit 340 of the invalidation image data thus generated.

<Specific example of processing by the invalid area determination unit>
Next, a specific example of processing by the invalid area determination unit 2110 will be described. FIG. 22 is a diagram showing a specific example of processing by the invalid area determination unit. Graphs 510_1 to 510_m in FIG. 22 are the same as the graphs 510_1 to 510_m shown in FIG. However, the graphs 510_1 to 510_m shown in FIG. 22 clearly show the quantized values (unrecognizable quantized values) when the correct recognition result is not output in the recognition processing by the CNN unit 320 (chain line reference).

The invalid area determination unit 2110 calculates the difference between the aggregated value corresponding to the minimum quantized value and the aggregated value corresponding to the unrecognizable quantized value. The example of FIG. 22 indicates that the differences calculated for blocks 1 to m are Δ ₁ to _Δm .

The invalid area determination unit 2110 determines whether the corresponding block belongs to the invalid area based on whether the calculated difference is equal to or greater than a predetermined threshold.

The example of FIG. 22 shows how the invalid area determination unit 2110 determines that block 1 belongs to the invalid area because Δ ₁ is less than the predetermined threshold. On the other hand, the example of FIG. 22 shows how the invalid area determination unit 2110 determines that block 2 belongs to the valid area because _Δ2 is greater than or equal to the predetermined threshold. Also, the example of FIG. 22 shows how the invalid area determination unit 2110 determines that block 3 belongs to the invalid area because _Δ3 is less than the predetermined threshold.

<Specific example of invalidation image data>
Next, a specific example of invalidation image data generated by the invalidation image generation unit 2120 will be described. FIG. 23 is a diagram showing a specific example of invalidation image data.

In invalidated image data 2300 shown in FIG. 23, a hatched area 2301 is an area determined as an invalid area by the invalid area determination unit 2110 . On the other hand, a non-hatched area 2302 in the invalidated image data 2300 is an area determined by the invalid area determination unit 2110 to be a valid area.

The output unit 340 invalidates each block included in the area 2301 and transmits image data (invalidated image data 2300 ) composed of each block included in the area 2302 to the image compression device 130 .

As a result, the image compression device 130 performs compression processing on the invalidated image data 2300 to generate compressed data. Therefore, the data size of the compressed data can be further reduced compared to the case where the compression processing is performed on the entire image data using the optimum quantization value.

Note that when the image compression device 130 performs compression processing on the invalidated image data 2300, the analysis device 120 determines an optimum quantization value for each block included in the region 2302 according to the degree of influence on the recognition result. It may be calculated and transmitted to the image compression device 130 .

As a result, the data size of the compressed data can be further reduced as compared with the case where the invalidation image data 2300 is compressed using a predetermined quantization value.

<Flow of image compression processing by compression processing system>
Next, the flow of image compression processing by the compression processing system 100 will be described. FIG. 24 is a seventh flowchart showing an example of the flow of image compression processing by the compression processing system. The difference from the first flowchart shown in FIG. 7 is steps S2401 to S2404.

In step S2401, invalid area determination section 2110 determines whether or not CNN section 320 has output a correct recognition result. If it is determined in step S2401 that a correct recognition result has been output (if Yes in step S2401), the process returns to step S702.

On the other hand, if it is determined in step S2401 that the correct recognition result was not output (No in step S2401), the process proceeds to step S2402.

In step S2402, invalid area determination section 2110 calculates the difference between the total value associated with the minimum quantization value and the total value associated with the quantization value at the time of unrecognizability for each block. Also, the invalid area determination unit 2110 determines whether or not each block belongs to the invalid area based on the calculated difference.

In step S2403, the invalidation image generating unit 2120 generates invalidation image data by invalidating blocks belonging to the invalid area.

In step S<b>2404 , output unit 340 transmits the invalidated image data to image compression device 130 . The image compression device 130 also performs compression processing on the invalidated image data and stores the compressed data in the storage device 140 . Note that the image compression device 130 performs compression processing using the quantized value when the correct recognition result was output immediately before it was determined that the correct recognition result was not output.

As is clear from the above description, the analysis apparatus according to the seventh embodiment acquires each piece of compressed data when compression processing is performed on image data using different quantization values. In addition, the analysis device according to the seventh embodiment inputs the decoded data obtained by decoding each compressed data to the trained model and performs the recognition processing based on the CNN part structure information, the degree of influence on the recognition result Generate an important feature map showing Further, the analysis device according to the seventh embodiment, based on the difference between the total value corresponding to the quantization value when the correct recognition result is not output and the total value corresponding to the minimum quantization value, Determine whether each block belongs to the invalid area. Furthermore, the analysis apparatus according to the seventh embodiment performs compression processing on invalidated image data in which blocks belonging to invalid areas are invalidated.

In this manner, by performing compression processing on image data in which invalid regions determined based on the degree of influence on recognition results are invalidated, the same effects as those of the first embodiment can be obtained, and at the same time, The data size of the compressed data can be further reduced compared to the embodiment of .

[Eighth embodiment]
In the seventh embodiment described above, blocks belonging to the invalid area are determined based on the degree of influence on the recognition result. In contrast, in the eighth embodiment, blocks belonging to the effective area are determined based on the degree of influence on the recognition result.

In the eighth embodiment, when determining blocks belonging to the effective area, the minimum effective area is set first, and according to the change in the total value of each block when the quantization value is increased, The effective area is determined by gradually expanding the effective area. As described above, in the eighth embodiment, a larger quantization value is determined as the optimum quantization value by covering the decrease in recognition accuracy caused by increasing the quantization value by expanding the effective region. can be done. The eighth embodiment will be described below, focusing on differences from the seventh embodiment.

<Functional configuration of analysis device>
First, the functional configuration of an analysis device 120 according to the eighth embodiment will be described. FIG. 25 is a sixth diagram illustrating an example of the functional configuration of the analysis device; The difference from the functional configuration shown in FIG. 21 is that it has an initial disabled image generating section 2510 and that it has a valid area determining section 2520 instead of the invalid area determining section 2110 . Another difference is that the function of the invalidation image generation unit 2530 is different from the function of the invalidation image generation unit 2120 in FIG.

The initial invalidation image generation unit 2510 generates invalidation image data (referred to as initial invalidation image data) including a preset minimum valid area. Further, the initial invalidation image generation unit 2510 notifies the output unit 340 of the generated initial invalidation image data.

Valid area determination section 2520 reads out the tally result from tally result storage section 380, and determines whether or not to expand the valid area based on the amount of change in the tally value of each block with respect to the change in quantization value. In addition, when valid area determining section 2520 determines to extend the valid area, it notifies invalid image generating section 2530 of the valid area after the extension.

The invalidation image generation unit 2530 invalidates blocks belonging to an area (invalid area) other than the expanded valid area notified by the valid area determination unit 2520, and creates invalidation image data. In addition, the invalidation image generation unit 2530 notifies the output unit 340 of the generated invalidation image data.

<Specific example of processing by effective area determination unit>
Next, a specific example of processing by the valid area determination unit 2520 will be described. FIG. 26 is a diagram illustrating a specific example of processing by the effective area determination unit; In FIG. 26 , initial invalidation image data 2610 indicates initial invalidation image data generated by the initial invalidation image generation unit 2510 .

In the initial invalidated image data 2610 , the hatched area is the invalid area 2611 . On the other hand, the non-hatched area 2612 in the initial invalidated image data 2610 is the minimum valid area.

Here, the image compression device 130 performs compression processing on the initial invalidated image data 2610 based on different quantization values. As a result, the CNN unit 320 performs recognition processing on the decoded data obtained by decoding the compressed data corresponding to each quantization value, and the summation unit 360 recognizes the influence on the recognition result corresponding to each quantization value. Aggregate degrees in blocks.

In FIG. 26, a graph 2641 shows aggregate values of block 2612_1 (block number=“block X”) corresponding to respective quantization values. Also, the graph 2642 shows the total values of the block 2612_2 (block number=“block X+1”) corresponding to each quantization value.

For example, for the block 2612_1, the valid area determination unit 2520 calculates the difference _Δx between the total value corresponding to the current quantization value and the total value corresponding to the minimum quantization value. Accordingly, valid area determination section 2520 determines whether or not the valid area should be extended to blocks adjacent to block 2612_1.

Similarly, the valid area determining section 2520 calculates the difference Δx ₊₁ between the aggregated value corresponding to the current quantized value and the aggregated value corresponding to the minimum quantized value for the block 2612_2. Accordingly, valid area determination section 2520 determines whether or not the valid area should be extended to blocks adjacent to block 2612_2.

It should be noted that the valid area determining section 2520 performs the same determination for all blocks inside the boundary position between the valid area and the invalid area.

The example of FIG. 26 shows that for block 2612_1, _Δx is less than the predetermined threshold, so it is determined that it is not necessary to expand the valid area to adjacent blocks. Also, the example of FIG. 26 shows that for block 2612_2, since Δx ₊₁ is greater than or equal to the predetermined threshold, it is determined that the effective area needs to be extended to the adjacent block.

Note that the valid area determination unit 2520 notifies the invalidation image generation unit 2530 of the extended valid area including the block adjacent to the block 2612_2 in the valid area. Disabled image data is generated based on the later effective region.

In FIG. 26, invalidated image data 2620 indicates invalidated image data generated by the invalidated image generation unit 2530 based on the effective area after expansion notified by the effective area determination unit 2520 .

As shown in FIG. 26, valid area 2622 of invalidated image data 2620 includes block 2631 adjacent to block 2612_2. Also, the invalid area 2621 of the invalidated image data 2620 is smaller than the invalid area 2611 of the initial invalidated image data 2610 by expanding the valid area.

In this manner, the valid area determination unit 2520 determines the valid area by gradually expanding the valid area in accordance with the change in the total value of each block when the quantization value is increased. By including the adjacent blocks in the valid area, the effective area determination unit 2520 reduces the total value of the block located inside the boundary position between the valid area and the invalid area, and the total value corresponding to the minimum quantized value is reduced. is less than a predetermined threshold, the extension of the effective area is continued.

On the other hand, in the valid area determination section 2520, although the adjacent blocks were included in the valid area, the aggregate value of the block located inside the boundary position between the valid area and the invalid area did not decrease, and the aggregate value corresponding to the minimum quantized value did not decrease. If the difference from the value remains equal to or greater than the predetermined threshold, the extension of the valid area is terminated.

<Flow of image compression processing by compression processing system>
Next, the flow of image compression processing by the compression processing system 100 will be described. FIG. 27 is an eighth flowchart showing an example of the flow of image compression processing by the compression processing system.

In step S2701, the input unit 310 acquires image data in units of frames.

In step S2702, the CNN unit 320 outputs recognition results by performing recognition processing on the image data, and the important feature map generation unit 350 generates an important feature map. In addition, the aggregation unit 360 aggregates the degree of impact for each block. As a result, a total value corresponding to the minimum quantized value is calculated for each block.

In step S2703, quantization value setting section 330 initializes the compression level and sets the upper limit of the compression level. Also, the initial invalidation image generation unit 2510 generates initial invalidation image data.

In step S2704, the image compression device 130 uses the current quantization value to perform compression processing on the invalidated image data (initial invalidated image data here) to generate compressed data.

In step S2705, the CNN unit 320 performs recognition processing on the decoded data obtained by decoding the compressed data, and outputs a recognition result, and the important feature map generation unit 350 generates an important feature map. In addition, the aggregation unit 360 aggregates the degree of impact for each block.

In step S2706, valid area determination section 2520 determines the difference between the aggregated value corresponding to the current quantized value and the aggregated value corresponding to the minimum quantized value for the block inside the boundary position between the valid area and the invalid area. is greater than or equal to a predetermined threshold.

If it is determined in step S2706 that it is less than the predetermined threshold (No in step S2706), the process proceeds to step S2712.

On the other hand, if it is determined in step S2706 that it is equal to or greater than the predetermined threshold value (Yes in step S2706), the process proceeds to step S2707.

In step S2707, the valid area determination unit 2520 includes the blocks adjacent to the block whose difference is equal to or greater than a predetermined threshold in the valid area, and notifies the invalid image generation unit 2530 of the extended valid area.

In step S2708, the invalidation image generation unit 2530 generates invalidation image data based on the valid area after expansion.

In step S2709, the image compression device 130 performs compression processing on the invalidated image data using the current quantization value to generate compressed data.

In step S2710, CNN section 320 outputs recognition results by performing recognition processing on decoded data obtained by decoding compressed data, and important feature map generation section 350 generates an important feature map. In addition, the aggregation unit 360 aggregates the degree of impact for each block.

In step S2711, valid area determination section 2520 determines whether or not the total value for the block determined to be equal to or greater than the predetermined threshold in step S2706 has decreased and the difference has become less than the predetermined threshold.

If it is determined in step S2711 that the value is less than the predetermined threshold value (Yes in step S2711), the process proceeds to step S2712.

In step S2712, quantization value setting section 330 increases the compression level (quantization value) and returns to step S2704.

On the other hand, if it is determined in step S2711 that it remains equal to or greater than the predetermined threshold value (No in step S2711), the process proceeds to step S2713.

In step S2713, invalidation image generation section 2530 generates invalidation image data based on the valid area immediately before the valid area is expanded in step S2707.

In step S2714, the image compression apparatus 130 performs compression processing on the invalidated image data generated in step S2713 using the compression level (quantization value) immediately before expanding the valid area in step S2707. Store data.

As is clear from the above description, the analysis apparatus according to the eighth embodiment first sets the minimum effective area, and according to the change in the total value of each block when the quantization value is increased, , gradually expanding the effective area.

As a result, according to the analysis apparatus according to the eighth embodiment, it is possible to compensate for the decrease in recognition accuracy due to the increase in the quantization value by extending the effective region, so that a larger quantization value can be optimized. Compression processing can be performed as a quantized value.

As a result, according to the eighth embodiment, the same effects as those of the first embodiment can be obtained, and the data size of the compressed data can be further reduced than that of the first embodiment.

[Ninth Embodiment]
In the above-described eighth embodiment, in extending the valid area, attention is paid to the total value of the blocks inside the boundary position between the valid area and the invalid area. On the other hand, in the ninth embodiment, when expanding the valid area, the total value of the blocks adjacent to each other across the boundary position (the aggregate value of the blocks inside the boundary position between the valid area and the invalid area and the blocks outside and ). The ninth embodiment will be described below, focusing on differences from the eighth embodiment.

<Functional configuration of analysis device>
First, the functional configuration of an analysis device 120 according to the eighth embodiment will be described. FIG. 28 is a seventh diagram showing an example of the functional configuration of the analysis device; The difference from the functional configuration shown in FIG. This is different from function. Also, instead of the initial invalidation image generation unit 2510, an initial valid area setting unit 2820 is provided.

Initial valid area setting section 2820 first sets the minimum valid area for valid area determining section 2810 .

Valid area determination unit 2810 reads out the total result from total result storage unit 380, and determines whether or not to expand the effective area based on the total value of each block in each quantization value.

Specifically, the valid area determination unit 2810 calculates the total value of each block for each compressed data generated each time the quantization value is increased for the entire image data, and stores the total value in the total result storage unit 380 . Gets the aggregate value for each block, if any.

At that time, the valid area determination unit 2810 calculates the difference in total value between the block inside the boundary position between the initial valid area and the invalid area and the block outside the boundary position (blocks adjacent to each other via the boundary position). calculate. Then, when the valid area determination unit 2810 determines that the calculated difference is equal to or greater than a predetermined threshold value, the blocks outside the boundary position are included in the valid area.

After that, the total value of each block is similarly obtained for each piece of compressed data generated each time the quantization value is increased for the entire image data. At this time, the valid area determination unit 2810 calculates the difference in the total values between the blocks inside and outside the boundary position between the valid area and the invalid area after expansion. Then, when the valid area determination unit 2810 determines that the calculated difference is equal to or greater than a predetermined threshold value, the blocks outside the boundary position are included in the valid area.

The invalidation image generation unit 2830 generates invalidation image data based on the effective area when the effective area determination unit 2810 completes the extension of the effective area. In addition, the invalidation image generation unit 2830 notifies the output unit 340 of the generated invalidation image data.

<Specific example of processing by effective area determination unit>
Next, a specific example of processing by the valid area determination unit 2810 will be described. FIG. 29 is a second diagram showing a specific example of processing by the valid area determination unit. In FIG. 29, image data 2910 is image data compressed by the image compression device 130 . An initial valid area 2912 in the image data 2910 indicates an initial valid area set by the initial valid area setting section 2820 .

Here, the image compression device 130 performs compression processing on the image data 2910 using each quantization value to generate compressed data. As a result, the CNN unit 320 performs recognition processing on the decoded data obtained by decoding the compressed data corresponding to each quantization value, and the summation unit 360 recognizes the influence on the recognition result corresponding to each quantization value. Aggregate degrees in blocks.

In FIG. 29, a graph 2931 shows aggregate values of block 2921 (block number=“block X”) corresponding to each quantization value. Note that the block 2921 is a block inside the boundary position between the initial valid area 2912 and the invalid area 2911 .

Also, a graph 2932 shows aggregated values of block 2922 (block number=“block X+1”) corresponding to each quantization value. Note that the block 2922 is a block outside the boundary position between the initial valid area 2912 and the invalid area 2911 and adjacent to the block 2921 .

The valid area determination unit 2810 calculates the difference between the aggregated value of block 2921 and the aggregated value of block 2922 corresponding to the current quantized value, and determines whether the calculated difference is equal to or greater than a predetermined threshold. By doing so, it is determined whether or not the block 2922 is included in the valid area.

The example of FIG. 29 shows how block 2922 is determined to be included in the valid area. Note that the valid area determination unit 2810 performs the same processing for all blocks inside the boundary position between the initial valid area and the invalid area.

In FIG. 29, the image data 2940 shows how the effective area determining section 2810 sets the effective area 2942 after expansion. In FIG. 29, block 2922 is a block newly included in the valid area.

After that, the image compression device 130 similarly obtains the total value of each block for each compressed data generated each time the quantization value is increased for the entire image data. At this time, the valid area determination unit 2810 calculates the difference in total value between the block inside and outside the boundary position between the valid area 2942 and the invalid area 2941 after expansion. Then, when the valid area determination unit 2810 determines that the calculated difference is equal to or greater than a predetermined threshold value, the blocks outside the boundary position are included in the valid area.

When the expansion of the valid area is completed, the valid area determination unit 2810 notifies the invalidation image generation unit 2830 of the valid area at the time of completion. Generate image data.

<Flow of image compression processing by compression processing system>
Next, the flow of image compression processing by the compression processing system 100 will be described. FIG. 30 is a ninth flowchart showing an example of the flow of image compression processing by the compression processing system. The difference from the eighth flowchart shown in FIG. 27 is steps S3001 to S3009.

In step S3001, initial valid area setting section 2820 sets an initial valid area.

In step S3002, the image compression device 130 compresses the image data using the current quantization value to generate compressed data.

In step S3003, the CNN unit 320 outputs recognition results by performing recognition processing on the decoded data obtained by decoding the compressed data, and the important feature map generation unit 350 generates an important feature map. In addition, the aggregation unit 360 aggregates the degree of impact for each block.

In step S3004, valid area determination section 2810 calculates the difference in the total values between the blocks inside the boundary position and the blocks outside the boundary position for the current valid area and the invalid area. It is determined whether or not it is equal to or greater than the threshold.

If it is determined in step S3004 that it is less than the predetermined threshold value (No in step S3004), the process proceeds to step S3006.

On the other hand, if it is determined in step S3004 that it is equal to or greater than the predetermined threshold value (Yes in step S3004), the process proceeds to step S3005.

In step S3005, valid area determination section 2810 includes blocks outside the boundary position in the valid area.

In step S3006, quantization value setting section 330 increases the compression level (quantization value), and proceeds to step S3007.

In step S3007, quantization value setting section 330 determines whether or not the compression level (quantization value) exceeds the upper limit. ) and returns to step S3002.

On the other hand, if it is determined in step S3007 that the upper limit has been exceeded (if Yes in step S3007), the process proceeds to step S3008.

In step S3008, invalidation image generation section 2830 generates invalidation image data based on the current valid area.

In step S3009, the image compression device 130 compresses the invalidated image data and stores the compressed data. Note that the image compression device 130 performs compression processing on invalidated image data using, for example, a quantization value when the valid area is expanded.

As is clear from the above description, the analysis apparatus according to the ninth embodiment first sets the minimum effective area, and when the quantization value is increased, the total value between adjacent blocks at the boundary position The effective area is gradually expanded in accordance with the difference between .

As a result, according to the analysis apparatus according to the ninth embodiment, it is possible to compensate for the decrease in recognition accuracy due to the increase in the quantization value by extending the effective region. Compression processing can be performed as a quantized value.

As a result, according to the ninth embodiment, the same effects as those of the first embodiment can be obtained, and the data size of the compressed data can be further reduced than that of the first embodiment.

[Other embodiments]
In the first embodiment described above, compression processing is performed using all quantized values from the minimum quantized value to the maximum quantized value. However, the quantization value used for compression processing is not limited to this, and compression processing may be performed using a predetermined number of quantization values included between the minimum quantization value and the maximum quantization value. The predetermined number of quantization values refers to the number of quantization values that can determine the optimum quantization value, and refers to at least two quantization values.

Further, in the first embodiment, the image data includes one object. However, the image data may contain multiple objects. In this case, the CNN structure information may be obtained for multiple objects in the image data at the same time, and the compression levels may be determined for the multiple objects at the same time. Alternatively, for a plurality of objects in the image data, the CNN structure information is individually acquired, the compression level is determined for each object, and then the compression level of each object is merged to obtain the compression level of the entire image data. may decide.

Further, in the above-described third embodiment, filtering processing using a low-pass filter has been described as an example of image processing for generating pseudo-compressed data. However, the image processing for generating pseudo-compressed data is not limited to this.

For example, the entire image data may be Fourier-transformed, high-frequency components cut, and then inverse Fourier-transformed. Alternatively, the image data may be Fourier-transformed on a block-by-block basis, high-frequency components cut, and then inverse Fourier-transformed.

Alternatively, the entire image data may be DCT-transformed, quantized, and then inversely DCT-transformed. Alternatively, image data may be DCT-transformed block by block, quantized, and then inversely DCT-transformed.

Further, in any one of the seventh to ninth embodiments, or an embodiment combining any of the seventh to ninth embodiments, an area having a large influence on the recognition result and an area having a small influence separate,
- Any one of the first to sixth embodiments or an embodiment combining any one of the first to sixth embodiments is applied to an area that has a large influence on the recognition result. ,
- A highly compressed quantization value may be applied to an area having a small influence on the recognition result (or may be an invalid area).

It should be noted that the compression level calculated in each of the above embodiments and the information indicating the effective area or the ineffective area are used to determine the details of preprocessing for image data that can be expected to reduce the data size by performing compression processing. You may use it as information of The preprocessing referred to here includes, for example, processing for reducing color information from image data, processing for reducing high-frequency components from image data, and the like.

It should be noted that the present invention is not limited to the configurations shown here, such as combinations with other elements, etc., in the configurations described in the above embodiments. These points can be changed without departing from the gist of the present invention, and can be determined appropriately according to the application form.

100: Compression processing system 120: Analysis device 130: Image compression device 310: Input unit 320: CNN unit 330: Quantization value setting unit 340: Output unit 350: Important feature map generation unit 360: Aggregation unit 370: Quantization value determination Unit 420: Aggregation result 810: Maximum quantization value setting unit 820: Quantization value determination unit 910: Group information 1110: Image processing unit 1410: Position determination unit 1420: Quantization value setting unit 1430: Output unit 2110: Invalid area determination Unit 2120: invalidation image generation unit 2300: invalidation image data 2510: initial invalidation image generation unit 2520: effective area determination unit 2530: invalidation image generation unit 2810: effective area determination unit 2820: initial effective area setting unit 2830: Invalidation image generator

Claims (11)

Recognition of each region of each decoded data calculated by performing recognition processing on decoded data obtained by decoding each compressed data when compression processing is performed on image data at different compression levels. a storage unit for storing information indicating the degree of influence on the result;
and a determination unit that determines the compression level of each area of the image data based on information indicating the degree of influence of each area of the decoded data corresponding to the different compression levels on the recognition result. a map generation unit that generates a map indicating the degree of influence of each area of the decoded data on the recognition result, which is calculated by performing recognition processing on the decoded data;
a tallying unit that tallies, based on the generated map, the degree of influence of each region of the decoded data on the recognition result for each block used when compression processing is performed;
2. The analysis apparatus according to claim 1, wherein said storage unit stores the aggregated value aggregated for each block as information indicating the degree of influence of each area on the recognition result. The storage unit is
Group information associated with a compression level is stored for each group when the information indicating the degree of influence on the recognition result of each region of the decoded data corresponding to the different compression levels is classified into a plurality of groups. death,
The decision unit
It is determined to which of the groups the information indicating the degree of influence of each region on the recognition result corresponding to the predetermined compression level belongs, and the compression level associated with the determined group is stored in the image data. 3. The analyzing apparatus according to claim 1, wherein the compression level of each area is determined. The decision unit
Information indicating the degree of influence of each region of pseudo-compressed data on the recognition result, calculated by performing recognition processing on pseudo-compressed data generated by performing image processing on image data 4. The analyzing apparatus according to claim 3, further comprising: determining to which of the groups each of the image data belongs, and determining a compression level associated with the determined group as the compression level of each area of the image data. The decision unit
Each region of each decoded data calculated by performing recognition processing on decoded data obtained by decoding each compressed data when compression processing is performed at different compression levels for different image data. 2. The analyzing apparatus according to claim 1, wherein the compression level of each area of each of said different image data is determined so that the information indicating the degree of influence on the recognition result does not exceed a predetermined threshold. The decision unit
Of the aggregated values aggregated in units of predetermined blocks, the aggregated value of the reference block is compared with the aggregated value of other blocks, and based on the compression level of the reference block and the comparison result, other 3. The analysis apparatus of claim 2, determining a compression level for blocks of . The decision unit
Multiplying information indicating the degree of influence of each region of the decoded data on the recognition result with respect to the correct recognition result output immediately before the erroneous recognition result is output by a weighting factor and adding the result to a preset compression level. 2. The apparatus of claim 1, wherein determining a compression level for each region of said image data. an invalid area determination unit that acquires information indicating the degree of influence on the recognition result of each area of the decoded data with respect to the correct recognition result output immediately before the erroneous recognition result is output, and determines an invalid area;
2. The analysis apparatus according to claim 1, further comprising: an invalidated image generating unit that generates invalidated image data by invalidating the area determined as the invalid area in the image data. further comprising a disabled image generation unit that generates disabled image data by disabling a predetermined disabled area in the image data;
The storage unit is
Disabled image data calculated by performing recognition processing on decoded data obtained by decoding respective compressed data when compression processing is performed on the disabled image data at different compression levels. stores the aggregate value of each block contained in the effective area of ,
The invalidation image generation unit
Among the aggregated values of the blocks included in the valid area of the invalidated image data, if the aggregated value of the blocks located inside the boundary position between the valid area and the invalid area satisfies a predetermined condition, the valid area is newly expanded to expand the aggregated value. 3. The analysis device according to claim 2, which generates invalidated image data. an effective area determination unit that determines an effective area in image data;
a disabled image generating unit that generates disabled image data by disabling an invalid region other than the determined valid region in the image data;
The storage unit is
Aggregation of each block of each decoded data calculated by performing recognition processing on decoded data obtained by decoding each compressed data when compression processing is performed at different compression levels for image data store the value,
The effective area determination unit
expanding the valid area when the aggregate value of blocks adjacent to each other across a boundary position between the valid area and the invalid area, among the aggregate values of each block of the decoded data, satisfies a predetermined condition;
The invalidation image generation unit
3. The analysis apparatus according to claim 2, wherein invalidated image data is generated by invalidating an invalid area other than the valid area after expansion of the valid area. Recognition of each region of each decoded data calculated by performing recognition processing on decoded data obtained by decoding each compressed data when compression processing is performed on image data at different compression levels. Get information that indicates the degree of influence on the result,
Determining the compression level of each region of the image data based on information indicating the degree of influence of each region of the decoded data corresponding to the different compression levels on the recognition result;
An analysis program for making a computer perform processing.

Images