WO2016002245A1

WO2016002245A1 - Image processing system, and image processing method

Info

Publication number: WO2016002245A1
Application number: PCT/JP2015/053725
Authority: WO
Inventors: 影山　昌広; 奥　万寿男; 野里　浩司; 前　愛州; 山下　和也
Original assignee: 株式会社日立製作所
Priority date: 2014-07-04
Filing date: 2015-02-12
Publication date: 2016-01-07
Also published as: WO2016002063A1; JPWO2016002245A1; JP6416255B2

Abstract

Provided is a technology which achieves image transmission with improved image quality by flexibly performing signal processing related to image enlargement and sharpening, in accordance with the capacity of computing resources.

Description

Image processing system and image processing method

The present invention relates to an image processing system and an image processing method, and more particularly to an image processing system and an image processing method using an image signal encoding technique and a sharpening technique.

ITU-T recommendation H. Video encoding technology represented by H.264 and still image coding technology represented by JPEG are indispensable technologies for applications such as videophones, remote conferences, and network surveillance cameras that transmit images.

In an application that encodes and transmits such an image, an image captured by a plurality of cameras installed at a number of locations is simultaneously transmitted via a communication network such as the Internet, an intranet, or a public network. Generally, these images are collectively displayed on a single screen at a receiving terminal. At this time, encoded data having a data size obtained by multiplying the amount of encoding generated in one image by the total number of images is transmitted to one terminal via the communication network. When an image having a data size exceeding the limit of the transmission bandwidth of the communication network is transmitted, a communication error or data delay may occur in an application that transmits a moving image, and a received image may be deteriorated or frozen.

Further, when the transmitted image is recorded and stored in a medium such as a hard disk (magnetic disk) or an optical disk, if the data size is large, a large-capacity recording medium is required, which increases the cost of the recording apparatus. . Alternatively, when a medium having a limited recording capacity is used, there is a problem that only a short recording can be performed.

Therefore, when providing an application for transmitting or recording an image to a user, the system that realizes the application needs to encode and compress the image in accordance with the transmission band and the recording capacity to reduce the data size. is there. However, in general, the encoded data size of the image and the image quality are in a contradictory relationship, and the image quality deteriorates as the encoded data size is reduced. Therefore, it is desired to realize a technique that can maintain a desired image quality for the user even if the encoded data size is reduced.

In such an application, the number of pixels of a camera used for imaging has increased with the advance of technology, and currently, for example, an image of horizontal 1920 pixels × vertical 1080 pixels (hereinafter abbreviated as full HD size) is generally used. Yes. However, in view of the transmission bandwidth of the communication network and the limitation of the recording medium capacity as described above, the number of pixels at the time of imaging is intentionally reduced, for example, horizontal 704 pixels × vertical 480 pixels (hereinafter abbreviated as D1 size). In some cases, the data size may be reduced by using the above-mentioned image as an encoding target. At this time, a part of the captured image frame may be trimmed to be an encoding target image, but the entire captured image frame is reduced to reduce the number of pixels to be an encoding target image. There are many cases. When a transmitted or recorded image is reproduced, the reduced image is generally displayed in an enlarged manner.

It is well known that when an image is enlarged, the image after enlargement is blurred only by increasing the number of pixels using an interpolation filter or the like. Therefore, many techniques for suppressing blurring by using image sharpening in combination have been proposed.

As a technique for sharpening an image, a technique for amplifying a high-frequency component included in an image and enhancing an edge or the like has been used for a long time. In recent years, a signal processing technique called super-resolution has been actively developed. For example, in the technique described in Patent Document 1, a Nyquist frequency representing the resolution limit of a transmitted or recorded image is obtained by performing nonlinear processing on a high-frequency component included in the image and then adding it to the original image. Generate higher harmonic components. Further, in the technique described in Patent Document 2, a high-frequency component exceeding the Nyquist frequency is used as an aliasing component in the image, and a plurality of image frames are used as input images, and the subject shown in each image frame After accurately aligning the position of, the baseband component and the aliasing component are separated, demodulated (inverted) to the frequency component before the aliasing, and the aliasing component after demodulation is added to the baseband signal, Get an image. In the technique described in Patent Document 3, the aliasing component is separated from the image of one frame in the same manner as the technique described in Patent Document 2. Non-Patent Document 1 introduces a number of techniques for obtaining a high-resolution image by using a plurality of image frames and performing successive approximation with iterative calculation. Non-Patent Document 2 describes a technique for obtaining a high-resolution image from one frame image by successive approximation processing.

Japanese Patent No. 5320538 Japanese Patent No. 5103314 Japanese Patent No. 5250233

As described above, a personal computer (hereinafter abbreviated as PC), a mobile terminal, or the like is used as a means for realizing an application for transmitting, recording, or displaying an image such as a videophone, a remote conference, or a network surveillance camera. It is often done. In these terminals, special hardware is used by performing signal processing by software processing using a general-purpose CPU (Central Processing Unit), MPU (Micro Processing Unit), GPGPU (General Purpose Computing on Graphics Processing Unit), etc. Without encoding, it is possible to realize image encoding, decoding, reduction, enlargement, sharpening, synthesis of a plurality of images, and the like.

On the other hand, the above-described super-resolution processing requires a large amount of calculation for processing compared to edge enhancement processing that only amplifies high-frequency components included in an image. For example, in the super-resolution technique using the successive approximation process described in Non-Patent Document 1 and Non-Patent Document 2, as will be described later, in order to obtain one output image, the input image is enlarged and enlarged The image reduction, the difference detection with the input image, and the output image correction processing need to be repeated several times, which greatly consumes calculation resources such as a CPU and a memory necessary for calculation. In particular, as described above, images taken by a plurality of cameras installed at a number of locations are simultaneously transmitted via a communication network, and these images are combined into one screen by one image receiving terminal. In the case of display, if this image receiving terminal is to be realized on a PC, a plurality of images are decoded, enlarged, and sharpened at the same time. There are problems such as frame dropping that makes the movement jerky, the entire process stopped, and that the user input from the mouse, keyboard, etc. is not accepted and no response is made. Therefore, it is not easy to sharpen an image in parallel while receiving a plurality of image data.

The present invention has been made in view of such a situation, and the image quality can be improved by flexibly executing image processing of image enlargement and sharpening according to the performance of calculation resources provided in the image receiving terminal. A technique for realizing enhanced image transmission is provided.

In order to solve the above problems, for example, the configuration described in the claims is adopted. The present application includes a plurality of means for solving the above problems. To give an example, an image processing system includes a plurality of first image processing devices and an image encoded from the first image processing device. An image processing system including a second image processing device that receives a signal, the first image processing device including an imaging unit that captures an image, an encoding unit that encodes an image and outputs a signal, and the signal A transmission unit that transmits to the second image processing device, and the second image processing device receives a signal and outputs a decoded decoded image, and performs a sharpening process on the decoded image to clarify the decoded image. A sharpening processing unit that outputs an image; a display unit that displays a decoded image or a sharpened image; and a control unit that switches an image to be displayed on the display unit. Generate image, high-frequency from enlarged image The second processed image is generated by synthesizing the first processed image extracted from the components and subjected to nonlinear processing and the enlarged image, and an image obtained by removing the aliasing component from the second processed image is output as a sharpened image. It is characterized by doing.

Alternatively, there is an image processing method in which images captured by a plurality of first image processing devices are processed by a second image processing device, the first step of capturing images by the first image processing device, and the image is encoded and a signal A second step of outputting a signal, a third step of transmitting a signal to the second image processing device, a fourth step of receiving a signal and outputting a decoded image, and a sharpening process of the decoded image, A fifth step of outputting a sharpened image, a sixth step of displaying the decoded image or the sharpened image on the display unit, and a seventh step of switching the image to be displayed on the display unit. In the fifth step, An enlarged image obtained by enlarging the decoded image is generated, a high-frequency component is extracted from the enlarged image, and the first processed image obtained by performing nonlinear processing and the enlarged image are combined to generate a second processed image, and the second processed Fold from image And outputs an image obtained by removing components as sharpening the image.

According to the present invention, it is possible to flexibly execute image enlargement and sharpening signal processing according to the performance of calculation resources, and to realize image transmission with improved image quality.

It is a figure for demonstrating schematic structure of the 1st image processing system by embodiment of this invention. It is a figure for demonstrating a general image sharpening process (example). It is a flowchart for demonstrating the procedure (example) of the 1st display switching process by embodiment of this invention. It is a flowchart for demonstrating the procedure (example) of the image reception process by embodiment of this invention. It is a figure for demonstrating the structure (example) of the image expansion and sharpening process by embodiment of this invention. It is a figure for demonstrating the operation | movement (example) of the image expansion and sharpening process by embodiment of this invention. It is a figure for demonstrating the structure of the image reduction by embodiment of this invention. It is a figure for demonstrating the outline of the 2nd image processing system by embodiment of this invention. It is a figure for demonstrating the 2nd display switching process by embodiment of this invention. It is a figure for demonstrating schematic structure of the 3rd image processing system by embodiment of this invention. It is a figure for demonstrating operation | movement of the 3rd image processing system by embodiment of this invention. It is a figure for demonstrating the other image display form (example) by embodiment of this invention.

The present invention provides a technique for realizing image transmission with improved image quality.

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. Also, the same reference numerals are assigned to common components in the drawings. The attached drawings show specific embodiments and implementation examples based on the principle of the present invention, but these are for understanding the present invention and are not intended to limit the present invention. Not used.

This embodiment has been described in sufficient detail for those skilled in the art to practice the present invention, but other implementations and configurations are possible without departing from the scope and spirit of the technical idea of the present invention. It is necessary to understand that the configuration and structure can be changed and various elements can be replaced. Therefore, the following description should not be interpreted as being limited to this.

Furthermore, the embodiment of the present invention may be implemented by software running on a general-purpose computer, or may be implemented by a combination of software and hardware.

In addition, the term “removing” used in the following description includes the meaning of “reducing” or “weakening” in addition to the meaning of “completely eliminating”.

In addition, as an example of the number of pixels constituting the image frame, a full HD size (horizontal 1920 pixels × vertical 1080 pixels) and a D1 size (horizontal 704 pixels × vertical 480 pixels) will be described below as an example. It is clear that the present invention can be applied to the case of handling image frames having the same, and is not used to limit the present invention.

<Schematic configuration of image processing system>
FIG. 1 is a diagram showing a schematic configuration of an image processing system according to an embodiment of the present invention. The image processing system includes n (where n is an integer equal to or greater than 1) image transmission devices (101- # 1 to #n), a communication network (105), an image reception device (106), and a user who receives an image reception device ( 106) and the display unit (112) for displaying the output image of the image receiving device (106).

The image transmission device (101) includes a camera (102), an image reduction unit (103), and an encoding unit (104). The camera (102) includes, for example, a general imaging unit including a photoelectric conversion element (not shown) such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor) and a lens, signal level adjustment, contrast adjustment, and brightness adjustment. And a general signal processing unit that performs white balance adjustment and the like. The image reduction unit (103) performs processing to reduce the number of pixels of the image obtained by the camera (102), and may trim the part of the captured image frame to reduce the number of pixels. When a camera with a small number of pixels is used, the image reduction unit (103) may be omitted. The encoding unit (104) is a generally known MPEG (Moving Picture Expert Group) -1, MPEG-2, MPEG-4, H.264. H.264, VC-1, JPEG (Joint Photographic Experts Group), Motion JPEG, JPEG-2000, etc. may be standardized encoding, or non-standard encoding may be performed. Note that the processing configuration for connecting the image transmission apparatus (101) and the communication network (105) can be realized by a general technique, and is not shown.

The communication network (105) may be either wired or wireless, and is a network for communicating digital data using a communication protocol such as a general IP (Internet Protocol).

The image receiving device (106) includes a decoder (107- # 1 to #n) for decoding each image transmitted from the image transmitting device (101- # 1 to #n), and an image selection described later. Alternatively, the processing unit (108) that performs image composition, the processing unit (109) that performs image enlargement and sharpening described later, and the processing unit (108, 109) are controlled based on signals from the operation unit (111). It is comprised from the control part (110) which performs. Note that the processing configuration for connecting the communication network (105) and the image receiving device (106) can be realized by a general technique, and is not shown.

The image reproduced by the image receiving device (106) is displayed on the display unit (112). At this time, when n = 1, one image is displayed on the entire screen of the display unit as shown in (a) image selection mode of the display form (113) in FIG. On the other hand, in the case of n ≧ 2, only one image is selected and displayed from n images (a) an image selection mode and a plurality of images up to n (in FIG. 1). , 9 images are illustrated) are combined into one image and displayed (b) The image composition mode is switched by a signal from the user operation unit (111).

With the configuration of the image processing system described above, each image captured by n (where n is an integer of 1 or more) cameras can be displayed on a single image receiving device via a communication network. At this time, it is possible to selectively display only one image designated by the user from among the n images and to display a plurality of images up to the upper limit of n images. In addition, by using a process for enlarging and sharpening an image, a clear image with less blur can be displayed even when the image is enlarged and displayed.

<Overview of general image sharpening processing and its problems>
FIG. 2 shows a configuration example of a general image sharpening process disclosed so far. The image sharpening process shown in FIG. 2 can be used as the processing unit (109) that performs image enlargement and sharpening shown in FIG.

In the image sharpening process (109 (a)) having the configuration shown in FIG. 2 (a), the input image is enlarged by the image enlargement unit (201) and then included in the enlarged image by the horizontal / vertical high-pass filter (202). A high frequency component is extracted, and the multiplier (203) multiplies the high frequency component by a coefficient K to adjust the signal strength of the high frequency component. The nonlinear processing unit (204) performs nonlinear processing of the signal strength, and adds the adder (205 ) And the high-frequency component after the non-linear processing are added to obtain an output image. Here, if the operation of the nonlinear processing unit (204) is omitted, a general high-frequency component amplification process (edge enhancement) is performed. The nonlinear processing of the signal intensity in the nonlinear processing unit (204) includes, for example, processing for generating odd harmonic components by raising the signal value to the third power, small amplitude signal removal (coring) for reducing noise, A large amplitude signal removal (limiter) that suppresses excessive edge enhancement may be provided. This image sharpening process has an advantage that blurring of an image can be removed with a small amount of computation compared with the image sharpening process shown in FIGS. 2B and 2C described later, but the number of pixels of the image is reduced. There is no effect of removing the aliasing component that occurs when the number is increased or decreased. Note that the image sharpening process (109 (a)) having the configuration shown in FIG. 2A can be realized by, for example, the technique described in Patent Document 1, and thus the detailed configuration and operation of each part are not described. To do.

In the image sharpening process (109 (b)) having the configuration shown in FIG. 2 (b), after the input image is enlarged by the image enlargement unit (206), one or more enlargements are made by the folding separation unit (207). After the baseband component and the aliasing component are separated from the image and demodulated (inverted) to the frequency component before the aliasing component is aliased by the demodulator (208), the demodulated aliasing component and the baseband signal are added (209). To obtain an output image. In this image sharpening process, by removing the aliasing component, the high-frequency component exceeding the Nyquist frequency is reproduced and the image is sharpened while suppressing image quality deterioration such as jaggedness and moire of lines included in the image. On the other hand, this image sharpening process has no effect of amplifying the attenuated high frequency component. Note that the image sharpening process (109 (b)) having the configuration shown in FIG. 2B can be realized by the technique described in, for example, Patent Document 2 or Patent Document 3, and thus the detailed configuration and operation of each unit. Description of is omitted.

In the image sharpening process (109 (c)) having the configuration shown in FIG. 2 (c), the temporarily set output image (initial image) is reduced by the image reduction unit (210) so as to have the same number of pixels as the input image. The difference detection unit (211) detects the difference between the reduced image and the input image to generate a correction value for each pixel, and the adder (212) adds the input image and the correction value and inputs them to the image enlargement unit (213). The output is used as a new temporary output image. These processes are repeated many times so that the correction value gradually decreases, and when the correction values corresponding to all the pixels become smaller than a predetermined threshold value, or the number of repetitions is predetermined. The output of the image enlargement unit (213) is set as an output image when the number of times exceeds the predetermined number. At this time, the input image may be one frame, or the input image may be a plurality of frames, and the average image after the position correction is performed so that the position of the subject on each frame coincides with the difference detection unit (211) and the adder ( 212). This image sharpening process can simultaneously realize the removal of the aliasing component and the blur removal of the image (amplification of the high frequency component). On the other hand, since this image sharpening process requires repetitive operations, high-performance computing resources are required to realize real-time processing. Note that the image sharpening process (109 (c)) having the configuration shown in FIG. 2C can be realized by the techniques described in Non-Patent Document 1 and Non-Patent Document 2, for example. The description of the operation is omitted.

By the above-described image sharpening process, an enlarged image with reduced blur can be obtained. However, these image sharpening processes require a large amount of calculation for processing compared to edge enhancement processes that only amplify high-frequency components included in an image. Therefore, when the image sharpening process is performed on all the images obtained by the plurality of cameras, the processing limit of the calculation resource provided in the image receiving apparatus (106) is greatly exceeded, and the movement of the image is jerky. Frame dropping occurs, the entire process stops, or user input from a mouse, keyboard, etc. is not accepted, resulting in a non-responsive state. It is not easy to perform sharpening in parallel.

<Procedure for display switching processing (example)>
In order to solve the above-described problems, the embodiment of the present invention proposes a display switching process having a procedure as described below.

FIG. 3 is a flowchart showing an example of the procedure of the display switching process according to the embodiment of the present invention. In FIG. 3, the display switching process is started from step (301), the display mode designated by the user is identified in step (302), and if (a) the image selection mode is selected, the process proceeds to step (303); If it is the image composition mode, the process proceeds to step (305).

(A) The image selection mode is a processing mode in which one image that the user wants to display is selected from n decoded images (where n is an integer of 1 or more) and displayed. At this time, for example, assuming that a decoded image of D1 size (horizontal 704 pixels × vertical 480 pixels) is displayed on a full HD size (horizontal 1920 pixels × vertical 1080 pixels) display unit, The image is enlarged by 2.7 times (= 1920/704) and about 2.3 times (= 1080/480) in the vertical direction, and the image blur is remarkable. In other words, the effect of blur removal by the sharpening process is great. In addition, since only one image is displayed, the sharpening process target may be limited to only this image. That is, a sharpening process for an image that is not to be displayed is not necessary, and calculation resources required for the process can be reduced as compared to performing a sharpening process for all the decoded images. Accordingly, in step (303), based on an instruction from the user, an image that the user wants to display is selected from n decoded images (where n is an integer equal to or greater than 1), and the process proceeds to step (304). Then, the image enlargement and sharpening processes are performed, and the display switching process is terminated in step (307).

On the other hand, (b) the image composition mode is a processing mode in which a plurality of images (up to nine images are illustrated in FIG. 1) with an upper limit of n images are combined and displayed as one image. At this time, for example, the decoded nine images of D1 size (horizontal 704 pixels × vertical 480 pixels) are combined into one image and displayed on the display unit of full HD size (horizontal 1920 pixels × vertical 1080 pixels). Assuming that display is performed, the number of display pixels per decoded image is 640 horizontal pixels (= 1920/3) × 360 vertical pixels (= 1080/3), which is larger than the transmitted D1 size. Get smaller. Accordingly, since the image enlargement and sharpening processes are unnecessary, in step (305), a plurality of images to be displayed are combined into one image, and the image enlargement and sharpening processes are performed in step (306). Each part is controlled so as not to be performed, and the display switching process is ended in step (307). Even when the number of display pixels per image after image synthesis is larger than the number of pixels transmitted as an encoded image, it is smaller than the number of display pixels in the above-described (a) image selection mode. (B) Display image blur in the image composition mode is relatively small. Therefore, since the effect of blur removal by the sharpening process is also relatively small, even if it is controlled to stop the sharpening process in the (b) image composition mode, no significant problem occurs.

Note that the display switching process shown in FIG. 3 may be executed when a mode switching operation by the user occurs. At this time, the switching operation from the (a) image selection mode to the (b) image composition mode by the user is performed by, for example, a specific key operation (for example, pressing down of the escape key) using the keyboard of the operation unit (111) shown in FIG. ) Or a specific operation with the mouse of the operation unit (111) (for example, click of the right button) may be used as a trigger to perform mode switching. In addition, the switching operation from the (b) image composition mode to the (a) image selection mode by the user is performed by, for example, a code corresponding to an image that the user wants to select while viewing a plurality of images displayed on the display unit (112). Is displayed on the display unit (112) in correspondence with the mouse movement of the operation unit (111). After moving the cursor over the image you want to select, double-click the left mouse button to select the image and switch modes simultaneously.

According to the procedure (example) of the display switching process described above, in (a) the image selection mode, all the decoded images are reduced by narrowing the sharpening process target to only the display image and stopping the sharpening process for the image that is not displayed. Rather than performing the sharpening process on the image, it is possible to reduce the calculation resources necessary for the process. In (b) image composition mode, calculation resources can be greatly reduced by not performing any sharpening process.

<Image reception processing procedure (example)>
In the procedure (example) of the display switching process described above, the calculation resources are reduced by switching the presence / absence of the sharpening process for each display mode. However, the calculation resources necessary for the sharpening process itself have not been reduced. Therefore, in order to solve this problem (reduction of calculation resources necessary for the sharpening process), an embodiment of the present invention proposes an image reception process having the following procedure.

FIG. 4 is a flowchart showing an example of the procedure of image reception processing according to the embodiment of the present invention. In FIG. 4, image reception processing is started from step (401), and reception processing and decoding processing of n pieces of image data (where n is an integer of 1 or more) transmitted in step (402) are performed. The subsequent steps (301 to 307) are the procedure of the display switching process shown in FIG. 3, and according to the display mode set by the user operation, (a) in the image selection mode, the process proceeds to step (404). b) In the image composition mode, image display is performed in step (409), and the image reception process is terminated in step (410).

In step (404), measurement of the time required for the image enlargement and sharpening processing executed in step (405) is started, and in step (406), the time measurement is terminated, and the process proceeds to step (407). At this time, it is only necessary to start the timer in step (404), read the elapsed time indicated by the timer in step (406) and set it as the processing time in step (405), and then terminate the operation of the timer. Alternatively, the elapsed time from the start of the timer may be read in step (404) and step (406), respectively, and the time difference between the two may be used as the processing time in step (405). Alternatively, the step (404) may be omitted, and the difference between the time read in the previous step (406) and the time read in the current step (406) may be used as the processing time in the step (405). In step (405), an image enlargement and sharpening process is executed by performing the process shown in FIG.

In step (407), the processing time of step (405) measured in step (406) is compared with a predetermined set value, and the contents of the next sharpening process in step (405) are determined as will be described later. Specifically, the content of the next sharpening processing is determined from the processing content shown in FIG. 6 described later, for example, so that the processing time of step (405) is less than a predetermined set value. Thereafter, an image is displayed in step (409), and the image receiving process is terminated in step (410). At this time, as a predetermined setting value, for example, when step (405) is executed in units of one frame of an image, a value that takes into account a margin to the reciprocal of the average frame rate when displaying the image is used as the setting value. Good. That is, for example, when it is desired to display 5 frames per second, the predetermined set value may be set to a value of 200 milliseconds or less.

According to the procedure (example) of the image reception process described above, it is possible to grasp the degree of margin in the calculation resources provided in the image reception device (106) shown in FIG. 1 according to the time taken for image enlargement and sharpening. By determining the content of the next sharpening process based on this result, the processing limit of the computational resource is greatly exceeded, and frame dropping may occur where the motion of the image is jerky, or the entire process It is possible to avoid problems such as stopping or not receiving user input from a mouse, keyboard, etc., and flexibly executing image enlargement and sharpening signal processing according to the performance of computing resources become.

<Configuration of image enlargement and sharpening processing (example)>
FIG. 5 is a diagram showing a configuration example of image enlargement and sharpening processing according to the embodiment of the present invention. As described above, each of the image enlargement and sharpening processes shown in FIGS. 2A, 2B, and 2C has advantages and disadvantages, and there are options for the contents of each process as described later. . In view of this, a plurality of values are obtained in accordance with the time taken for image enlargement and sharpening measured by the procedure of the image reception processing shown in FIG. 4, that is, in accordance with the degree of margin in the calculation resources provided in the image reception device (106). By combining these image enlargement and sharpening processes and selecting the contents of each process, the image enlargement and sharpening signal processing is flexibly executed in accordance with the performance of the computational resources.

In FIG. 5, the processing unit (501) for enlarging and sharpening the image is different from the processing unit (109 (a)) for enlarging and sharpening the image shown in FIGS. 2 (a) and 2 (b). And a processing unit (109 (b)) that performs sharpening is combined in series. This combination makes it possible to compensate for the disadvantages of the two processing units described above, removing blurring of the image by the processing unit (109 (a)) and removing the aliasing component by the processing unit (109 (b)). Will be able to. In addition, the switching units (502) and (503) can be switched to use the processing unit (109 (c)) that performs image enlargement and sharpening shown in FIG. 2 (c). This configuration shows an example of a combination of image enlargement and sharpening processing, and is not limited to the configuration of FIG. For example, the order of the processing unit (109 (a)) and the processing unit (109 (b)) is reversed, the processing unit (109 (a)) and the processing unit (109 (b)), or the processing unit (109 ( Only one of c)) may be used.

Here, in the processing unit (109 (a)) that performs image enlargement and sharpening, there are processing content options (examples) as shown in FIG. That is, by increasing the number of taps of the horizontal / vertical filter (202) in FIG. 2A to improve the image quality by finely setting the frequency characteristics of the filter, or conversely, by reducing the number of taps, the image quality is slightly sacrificed. However, the processing time can be shortened by reducing the calculation amount. In addition, the nonlinear processing (204) shown in FIG. 2 (a) may be performed to generate odd-order harmonic components. Conversely, by not performing the nonlinear processing (204), the image quality is slightly sacrificed. However, it is possible to shorten the processing time by reducing the calculation amount. Also, the amount of calculation can be greatly reduced by not performing the entire processing unit (109 (a)).

Similarly, the processing unit (109 (b)) that performs image enlargement and sharpening also has processing content options (examples) as shown in FIG. That is, the image can be sharpened by demodulating (208) and adding (209) the aliasing component in FIG. 2 (b), but when the computing resources are insufficient, this demodulation (208) and adding (209). Even if it does not perform, the baseband signal which isolate | separated the folding component becomes an output image, and the jaggedness and moire resulting from the folding component can be suppressed. In addition, by increasing the number of frames of the input image used for this processing, it is possible to improve the separation component separation accuracy and to improve the image quality. Conversely, by setting the number of frames of the input image to 1, the subject between frames can be improved. Therefore, the amount of calculation can be greatly reduced. Also, the amount of calculation can be greatly reduced by not performing the entire processing unit (109 (b)). When the processing unit (109 (a)) and the processing unit (109 (b)) are used in combination, the processing unit (109 (a)) enlarges the image in the processing unit (109 (a)). The image enlargement unit (206) of (109 (b)) is unnecessary.

Similarly, the processing unit (109 (c)) that performs image enlargement and sharpening also has processing content options (examples) as shown in FIG. That is, the correction value is sufficiently increased by increasing the number of times each process of the image reduction unit (210), the difference detection unit (211), the adder (212), and the image enlargement unit (213) in FIG. The result of convergence to a small value may be used as the output image. Conversely, by reducing the number of iterations, the image quality is slightly sacrificed, but the amount of computation can be reduced and the processing time can be shortened. Further, the image quality can be improved by increasing the number of frames of the input image used for this processing, and conversely, by setting the number of frames of the input image to 1, it is not necessary to align the subject between frames, and the calculation amount Can be greatly reduced.

Using the control unit (504) in FIG. 5, one processing content is selected from the many options described above based on the processing content determined in step (407) in FIG. 4, and each unit (109 (a ), 109 (b), 109 (c), 502, 503). This specific operation will be described later.

The above-described image enlargement and sharpening processing enables flexible sharpening processing according to the performance of computing resources.

<Image enlargement and sharpening processing (example)>
FIG. 6 collectively shows the operation (example) of the control unit (504) in FIG. 5 described above. As described above, the processing units (109 (a), 109 (b), and 109 (c)) that perform image enlargement and sharpening in FIG. 5 have options for each processing content, and are necessary for each processing. The amount of computation is also different. Therefore, as shown in FIG. 6, the level is set according to the amount of calculation necessary for the processing, and the processing contents of each processing unit (109 (a), 109 (b), 109 (c)) are set. Set in advance. Note that the processing contents and processing parameters shown in FIG. 6 are examples for explaining the operation, and are not limited to the processing contents.

In FIG. 6, from level 1 to level 4, only the image enlargement and sharpening is performed by the processing unit (109 (a)), and the processing of the processing unit (109 (b)) and processing unit (109 (c)) is performed. Indicates not to do. Further, the calculation amount of the processing unit (109 (a)) from level 1 to level 4 is set so as to gradually increase and achieve high image quality. Similarly, for the other levels, the processing content of each processing unit (109 (a), 109 (b), 109 (c)) is set so that the calculation amount and the image quality have a positive correlation. When the processing time of step (405) in FIG. 4 is larger than the predetermined set value, the level of FIG. 6 is changed so as to decrease. Conversely, when the processing time of step (405) is smaller than the predetermined set value, FIG. If the level 6 is changed to be larger, the processing time of step (405) can be controlled in the vicinity of a predetermined set value, and an image quality suitable for the calculation resource can be obtained. The processing time obtained in one measurement may be used as the processing time in step (405), or frequent level changes may be made by using the average value of the processing times obtained in a plurality of measurements. It may be suppressed.

In this embodiment, an image processing apparatus in which a camera identification function is added to the configuration of the first embodiment will be described. In addition, about the structure similar to Example 1, description is abbreviate | omitted suitably.
<Schematic configuration of second image processing system>
FIG. 8 is a diagram showing a schematic configuration of the second image processing system according to the embodiment of the present invention. As described with reference to FIGS. 1 and 7, in the image processing system according to the embodiment of the present invention, the image transmission apparatus 101 (first image processing apparatus) having various characteristics is connected via the communication network 105. , Connected to the image receiving device 106 (second image processing device). Similar to the first embodiment, the image recording / reproducing device 114 records the encoded image data captured by the image transmission device (101) on a recording medium, and after performing time shift and archiving, the encoded image data from the recording medium. Is transmitted to the image receiving device (106). At this time, a general hard disk (magnetic disk), an optical disk such as a DVD (Digital Versatile Disc) or CD (Compact Disc), a magnetic tape, or the like is used alone as a recording medium, or a plurality of recording media are arrayed. It can be used in a connected state. Further, the recording operation and the reproducing operation may be performed independently, or another image data may be reproduced at the same time while recording the encoded image data on the recording medium. The recording medium and configuration, the control method of the image recording / reproducing device (114), and the processing configuration for connecting the image recording / reproducing device (114) and the communication network (105) can be realized by general techniques. For this reason, illustration is omitted. In FIG. 1, the image recording / reproducing apparatus (114) is configured as an independent apparatus, and is illustrated to be connected to the image transmitting apparatus (101) and the image receiving apparatus (106) via the communication network (105). However, the present invention is not limited to this configuration. The image recording / reproducing device (114) is built in the image transmitting device (101), or the image recording / reproducing device (114) is installed in the image receiving device (106). It may be configured to be built in.

FIG. 8 shows three types of image transmission apparatuses (101- # p) (101- # q) (101- # r) described below as a typical example of the image transmission apparatus 101. An image transmission device (101- # p) is an image reduction unit whose characteristics are optimized between a camera (102- # p) having a large number of pixels such as a full HD size and an image reception device (806) described later. (103- # p), an encoding unit (104), and an identification signal transmission unit (801) described later. The image transmission apparatus (101- # q) includes a camera (102- # q) having a large number of pixels such as a full HD size, an image reduction unit (103- # q) whose characteristics are not optimized, and an encoding unit (104). The image transmission device (101- # r) includes a camera (102- # r) having a small number of pixels such as a D1 size and an encoding unit (104). The encoding unit (104) is common to each image transmission apparatus.

The identification signal transmission unit (801) encodes an identification signal indicating that the characteristics are optimized with the image receiving device (806) (that is, the characteristics match), and the camera (102 -# P) means for multiplexing the image from the encoded data and transmitting it to the image receiving device (806) via the communication network (105). An identification signal encoding processing configuration, a data multiplexing processing configuration, and a processing configuration for connecting the identification signal transmission unit (801) and the communication network (105) can be realized by a general technique. Is omitted. The identification signal may be determined in advance by using, for example, specific flag data indicating that the characteristics have been optimized, or using digital data indicating the model name or serial number of the camera.

The image receiving device (806) is obtained by adding an identification signal receiving unit (802) to the configuration of the image receiving device (106) shown in FIG. The control unit (810) in the image reception device (806) is a first processing unit that performs image selection or image synthesis based on the signal from the identification signal reception unit (802) and the signal from the operation unit (111). 108 and the operation of the second processing unit 109 for enlarging and sharpening the image. The processing configuration for connecting the communication network (105) and the identification signal receiving unit (802) or separating the identification signal from the transmitted encoded data can be realized by a general technique. The illustration is omitted. Also, a decoding unit (107), a processing unit (108) that performs image selection or image synthesis, a processing unit (109) that performs image enlargement and sharpening, an operation unit (111), a display unit (112), image recording / playback The device (114) is the same as that shown in FIG. 1 or FIG.

Here, when data from each of the image transmission apparatuses (101- # p) (101- # q) (101- # r) is input to the image reception apparatus (806) via the communication network (105) In addition, the image receiving device (806) uses the identification signal receiving unit (802) to determine whether or not a predetermined identification signal is multiplexed for each transmitted data. As described above, the characteristics of the image reduction unit in the image transmission device (101- # p) are optimized with the image reception device (806), and other image transmission devices (101- # q) ( It can be determined that the characteristics of the image reduction unit in 101- # r) are not optimized or the image reduction unit is not provided. As described with reference to FIG. 7, when the characteristics of the image reduction unit are not optimized or the image reduction unit is not provided, the image sharpening is performed in the processing unit (109) that performs image enlargement and sharpening. Since the processing may be difficult, the processing unit (109) that performs image enlargement and sharpening is forcibly sharpened regardless of whether the display mode is (a) image selection mode or (b) image composition mode. Is controlled by the control unit (810). On the other hand, the data determined that the predetermined identification signal is multiplexed with the transmission data, that is, the image transmitted from the image transmitting apparatus (101- # p) whose characteristics are optimized are described above. The presence / absence of the sharpening process is switched according to the display mode.

It should be noted that when the image data recorded in the image recording / reproducing apparatus (114) is reproduced and received, the frame rate of the image at the time of reproduction may be arbitrarily set. For example, there are cases in which an image can be stopped for each frame or played slowly. Thus, lowering the frame rate at the time of reproduction corresponds to reducing the amount of calculation required per unit time. Therefore, when the frame rate at the time of reproduction is lowered, the sharpening process may be performed even in the image synthesis mode (b).

In view of the above, the image processing system of the difference in the present embodiment includes an image having a plurality of first image processing devices and a second image processing device that receives an image signal encoded from the first image processing device. In the processing system, the first image processing device includes an imaging unit that captures an image, an encoding unit that encodes an image and outputs a signal, and a transmission unit that transmits the signal to the second image processing device; The second image processing apparatus includes: a decoding unit that receives a signal and outputs a decoded image; a sharpening processing unit that sharpens the decoded image and outputs a sharpened image; and a decoded image Alternatively, the image processing apparatus includes a display unit that displays a sharpened image and a control unit that switches an image displayed on the display unit. The sharpening processing unit generates an enlarged image obtained by enlarging the decoded image, and extracts a high-frequency component from the enlarged image. First process that performed nonlinear processing And-image-larger image to generate a second processed image by synthesizing, and outputs an image obtained by removing an aliasing component from the second processed image as a clear image.

By performing the first processing and the second processing in this way, an image from which aliasing components are removed can be output, and the image quality of an image signal sent from a remote location can be displayed with an improved quality.

Also, unlike the first embodiment, by receiving the identification signal from the image transmission device, the image transmission device (101) having various characteristics can be connected to the image reception device (806) via the communication network (105). ), The image sharpening process can be appropriately performed.

<Second Display Switching Procedure (Example)>
In order to realize the operation of the control unit (810) described above, the embodiment of the present invention proposes a display switching process having a procedure as described below.

FIG. 9 is a flowchart showing an example of the procedure of the second display switching process according to the embodiment of the present invention. Here, the contents of steps (301), (302), (303), (304), (305), (306) and (307) in FIG. 9 are the same as the contents of each step in FIG. .

After selecting an image to be displayed in step (303) in FIG. 9, it is determined in step (901) whether or not the aforementioned identification signal corresponding to the selected image has been received, and the identification signal is determined. If received, the process proceeds to step (304), and if no identification signal is received, the process proceeds to step (902). In step (902), image enlargement is performed, but sharpening processing is not performed. As a result, the image that has received the identification signal (that is, the image whose characteristics match the image sharpening) is subjected to the image sharpening process, and the image that does not receive the identification signal (that is, whether the characteristics of the image sharpening and the characteristics match) With respect to (unknown image), it is possible to control so as not to perform the image sharpening process.

In view of the above, the image processing method described in the present embodiment is an image processing method in which images captured by a plurality of first image processing devices are processed by the second image processing device, and the images are processed by the first image processing device. A second step of encoding an image and outputting a signal; a third step of transmitting a signal to the second image processing device; and receiving a signal and outputting a decoded image A fourth step of sharpening the decoded image and outputting a sharpened image, a sixth step of displaying the decoded image or the sharpened image on the display unit, and an image displayed on the display unit. A seventh step of switching, and in the fifth step, an enlarged image obtained by enlarging the decoded image is generated, and the first processed image obtained by extracting a high-frequency component from the enlarged image and performing nonlinear processing and the enlarged image are synthesized. Second place It generates already image, and outputs an image obtained by removing an aliasing component from the second processed image as a clear image.

Also, when the image data recorded in the image recording / reproducing apparatus (114) is reproduced and received, the frame rate at the time of reproduction can be lowered as described above. In this case, the predetermined set value described above can be set to a value larger than the set value at the time of normal real-time reproduction, and blurring with greater effect can be performed without causing a shortage of calculation resources. become able to.

Also, unlike the first embodiment, by receiving the identification signal from the image transmission device, the image sharpening process can be appropriately performed even when an image having various characteristics is received. Become.

In the present embodiment, a description will be given with reference to an image processing apparatus that performs camera identification by a method different from that in the second embodiment. In addition, about the structure similar to Example 1, 2, description is abbreviate | omitted suitably.

<Schematic Configuration and Operation of Third Image Processing System>
FIG. 10 is a diagram showing a schematic configuration of a third image processing system according to the embodiment of the present invention. In the figure, each image transmission device (101- # p) (101- # q) (101- # r) connected to the image reception device (1006) via the communication network (106) is connected to the image reception device. In order to identify whether or not the characteristics have been optimized with the sharpening process in (1006), the image receiving apparatus (1006) includes an image transmitting apparatus identifying unit (1001) and a processing method determining unit (1002). ). The image transmission device identification unit (1001) is means for identifying individual image transmission devices (101- # p) (101- # q) (101- # r), and this operation will be described later with reference to FIG. To do. The processing method determination unit (1002) determines the image based on the attributes of the image transmission devices (101- # p) (101- # q) (101- # r) identified by the image transmission device identification unit (1001). The processing unit (109) that performs enlargement and sharpening determines whether or not sharpening is performed, and sends the result to the control unit (810). The other units and the control unit (810) in FIG. 10 are the same as the configurations and operations of the other units and the control unit (810) shown in FIG. 1 and FIG.
FIG. 11 is an explanatory diagram of operations of the image transmission device identification unit (1001) and the processing method determination unit (1002) described above. As a means for identifying each image transmission device, for example, an IP (Internet Protocol) address or a MAC (Media Access Control) address that is uniquely set for each device is used to identify each device on the communication network. To do. Alternatively, a unique name (for example, lobby, passage, entrance / exit, etc.) that can be easily identified by a person according to the location where the image transmission device is installed may be assigned to each image transmission device in advance. Good. Characteristics are optimized between the IP address or MAC address (logical address) or installation location (physical address) for each device and the image receiving apparatus (1006) shown in FIG. If the flag indicating whether or not (that is, whether or not to perform sharpening) is managed as a list or a list and the characteristics are optimized for each image transmission device, the image sent from the image transmission device The processing method is determined such that the sharpening process is not performed unless the characteristics are optimized.

When the image transmission apparatus (101) having various characteristics is connected to the image reception apparatus (806) via the communication network (105) by the configuration and operation of the third image processing system described above. The image sharpening process can be appropriately performed without using a means for transmitting and receiving the identification signal.
<Other image display forms (example)>
In the embodiment of the present invention described above, in order to simplify the description, two types of images, that is, (a) an image selection mode and (b) an image composition mode as in the image display form (113) shown in FIG. Although the display form has been described as an example, it is not limited to this image display form. For example, as shown in FIG. 12, (c) image selection / combination mode, i.e., i images among (i + j) images (where i and j are each an integer of 1 or more) (FIG. 12 (c)). In FIG. 12, there is provided a mode in which image # 1) is displayed large as a main screen and the other j images (images # 2 to # 6 in FIG. 12C) are displayed small as sub-screens. You may make it switch mutually between selection mode and (b) image composition mode. (C) In the image selection / combination mode, control is performed so that the image sharpening process is performed for i images (image # 1 in FIG. 12C) selected as the parent screen, and the other j images are selected. (Images # 2 to # 6 in FIG. 12C) can be processed more clearly than performing the sharpening process on all decoded images if the control is performed so that the image sharpening process is not performed. The required computing resources can be reduced. At this time, whether or not to perform the image sharpening process may be determined based on the size of the image to be displayed (that is, the number of horizontal pixels and the number of vertical pixels). That is, a threshold value for the number of pixels may be determined in advance, and an image sharpening process may be performed on an image displayed with a larger number of pixels than the threshold value. In the case of this (c) image selection / combination mode, the ratio of the sharpened image is desirably about 60-80, assuming that the area of the sharpened image, the decoded image that has not been sharpened, and the total screen is 100. If the sharpened image is too small, the details are not visible even if the sharpening process is performed, and the effect is weak. Also, if the sharpened image is too large, (a) the load of the image selection mode and the sharpening processing is not much different, but rather the amount of processing increases by the amount combined with the decoded image, and the merit of combining images is thin. is there.

Further, image sharpening processing may be performed on an image displayed with a larger number of pixels than the transmitted image size. The (c) image selection / combination mode is regarded as (a) a special case of the image selection mode, that is, (a) a case where j images of the child screen are combined and displayed on the image in the image selection mode. You can also.

<Summary>
(I) According to the embodiment of the present invention described above, in the “image selection mode” in which one image is displayed on the entire screen of the display unit, the image is sharpened, and a plurality of images are converted into one image. In the “image composition mode” where the images are combined and displayed, control is performed so as not to sharpen the image, so that the calculation resources required for processing are reduced rather than performing sharpening processing on all decoded images. Can be reduced. In addition, by measuring the time taken for the sharpening process and determining the content of the next sharpening process so that this time is less than or equal to the preset value, flexible sharpening according to the performance of the computing resources Processing can be performed.

(Ii) The present invention can be realized by software program code that implements the functions of the embodiments. In this case, a storage medium in which the program code is recorded is provided to the system or apparatus, and the computer (or CPU or MPU) of the system or apparatus reads the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the program code itself and the storage medium storing the program code constitute the present invention. As a storage medium for supplying such program code, for example, a flexible disk, CD-ROM, DVD-ROM, hard disk, optical disk, magneto-optical disk, CD-R, magnetic tape, nonvolatile memory card, ROM Etc. are used.

Also, based on the instruction of the program code, an OS (operating system) running on the computer performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing. May be. Further, after the program code read from the storage medium is written in the memory on the computer, the computer CPU or the like performs part or all of the actual processing based on the instruction of the program code. Thus, the functions of the above-described embodiments may be realized.

Further, by distributing the program code of the software that realizes the functions of the embodiment via a network, the program code is stored in a storage means such as a hard disk or a memory of a system or apparatus, or a storage medium such as a CD-RW or CD-R And the computer (or CPU or MPU) of the system or apparatus may read and execute the program code stored in the storage means or the storage medium when used.

Finally, it should be understood that the processes and techniques described herein are not inherently related to any particular equipment, and can be implemented by any suitable combination of components. In addition, various types of devices for general purpose can be used in accordance with the teachings described herein. It may prove useful to build a dedicated device to perform the method steps described herein. Various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the embodiments. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined. Although the present invention has been described with reference to specific examples, these are in all respects illustrative rather than restrictive. Those skilled in the art will appreciate that there are numerous combinations of hardware, software, and firmware that are suitable for implementing the present invention. For example, the described software can be implemented in a wide range of programs or script languages such as assembler, C / C ++, perl, shell, PHP, Java (registered trademark).

Furthermore, in the above-described embodiment, the control lines and information lines indicate what is considered necessary for the explanation, and not all the control lines and information lines on the product are necessarily shown. All the components may be connected to each other.

In addition, other implementations of the invention will become apparent to those skilled in the art from consideration of the specification and embodiments of the invention disclosed herein. Various aspects and / or components of the described embodiments can be used singly or in any combination in a computerized storage system capable of managing data.

DESCRIPTION OF SYMBOLS 101 ... Image transmission apparatus 106 ... Image reception apparatus 109,501 ... An example of the process part which performs an image expansion and sharpening 111 ... Operation part 113 ... An example of a display form.

Claims

An image processing system comprising a plurality of first image processing devices and a second image processing device for receiving a signal of an image encoded from the first image processing device,
The first image processing apparatus includes:
An imaging unit that captures an image;
An encoding unit that encodes the image and outputs a signal;
A transmission unit for transmitting the signal to the second image processing device,
The second image processing device includes:
A decoding unit that receives the signal and outputs a decoded image;
A sharpening processing unit that sharpens the decoded image and outputs a sharpened image;
A display unit for displaying the decoded image or the sharpened image;
A control unit that switches an image to be displayed on the display unit,
The sharpening processing unit generates an enlarged image obtained by enlarging the decoded image, combines a first processed image obtained by extracting a high frequency component from the enlarged image and performing nonlinear processing, and the enlarged image to perform a second process. An image processing system for generating a finished image and outputting an image obtained by removing the aliasing component from the second processed image as a sharpened image.
The image processing system according to claim 1,
The second image processing device includes:
A time measuring unit for measuring a processing time in the sharpening processing unit;
An image processing system, further comprising: a process determining unit that determines the content of the sharpening process for the next input image based on the processing time.
The image processing system according to claim 1,
The first image processing apparatus includes:
An image reduction unit that reduces the image and outputs the reduced image to the encoding unit;
The transmission unit adds an identification signal for identifying the first image processing apparatus that has encoded the image to the signal and transmits the signal to the second image processing apparatus,
The second image processing device includes:
A signal identification unit that receives the identification signal and identifies the first image processing apparatus that has transmitted the identification signal;
An image processing system, further comprising: a process determining unit that determines a content of the sharpening process based on a result identified by the signal identifying unit.
The image processing system according to claim 1,
The first image processing apparatus includes:
An image reduction unit that reduces the image and outputs the reduced image to the decoding unit;
The transmission unit adds a characteristic identification signal indicating whether or not the processing characteristics of the image reduction unit and the sharpening processing unit are optimized to the signal, and transmits the signal to the second image processing apparatus. And
The second image processing device includes:
An image processing system, further comprising: a process determining unit that determines contents of the sharpening process based on a result of identifying the characteristic identification signal.
The image processing system according to claim 1,
The display unit is configured to switch and display a sharpened image, a decoded image, and a combined image of the sharpened image and the decoded image based on an instruction from a user.
An image processing method for processing an image captured by a plurality of first image processing devices with a second image processing device,
A first step of capturing an image with the first image processing device;
A second step of encoding the image and outputting a signal;
A third step of transmitting the signal to the second image processing device;
A fourth step of receiving the signal and outputting a decoded image;
A fifth step of sharpening the decoded image and outputting the sharpened image;
A sixth step of displaying the decoded image or the sharpened image on a display unit;
A seventh step of switching an image to be displayed on the display unit,
In the fifth step, an enlarged image obtained by enlarging the decoded image is generated, a first processed image obtained by extracting a high-frequency component from the enlarged image and subjected to nonlinear processing and the enlarged image are synthesized and second processed. An image processing method characterized in that an image is generated and an image obtained by removing a aliasing component from the second processed image is output as a sharpened image.
The image processing method according to claim 6,
An eighth step of measuring the processing time in the fifth step;
And a ninth step of determining the content of the sharpening process for the next input image based on the processing time.
The image processing method according to claim 6, wherein
A tenth step of reducing the image before the second step;
An eleventh step of receiving an identification signal for identifying the first image processing device that has encoded the image, and identifying the first image processing device;
Based on the result specified in the eleventh step, further includes a twelfth step for determining the processing content in the fifth step,
In the third step, the identification signal is added to the signal and transmitted to the second image processing apparatus.
The image processing method according to claim 6, wherein
A tenth step of reducing the image before the second step;
A thirteenth step of adding a characteristic identification signal indicating whether or not the characteristics of the processing in the tenth step and the sixth step are optimized with respect to the signal output in the second step;
An image processing method, further comprising: a fourteenth step of determining a content of the process in the sixth step based on a result of identifying the characteristic signal.