JP2009033629A - Imaging device, its control method, program, medium, and image processing device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging device and an image processing device capable of generating appropriate image data suitable for the purpose of use, depending on a shooting direction, and outputting and recording the image data. <P>SOLUTION: The imaging device includes a direction information acquiring means for acquiring direction information indicating a shooting direction of an input image, and a direction determining means for determining whether an imaging unit 101 is directed to an object to be supervised or an object to be appreciated, depending on the direction information acquired by the direction information acquiring means. The imaging device also includes a process control means for controlling a camera image processing unit 102 based on the result of determination by the direction determining means. The function of each means is possessed by a system control unit 110. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an imaging apparatus and an image processing apparatus, and more particularly to an imaging apparatus and an image processing apparatus that are characterized by various control techniques based on shooting direction information of an input image.

  Conventionally, a camera (camera system) in which a camera (camera body) and a camera turntable are integrated is commercially available. This camera is capable of performing both horizontal rotation (pan) and vertical rotation (tilt) by the operation of a turntable, and is called a pan-tilt camera (see, for example, Patent Document 1). ).

  An example of the operation of the pan / tilt camera is shown in FIG. FIG. 14A shows the horizontal movement of the camera. As the horizontal movement, there is, for example, one capable of endless rotation of 360 degrees. FIG. 14B shows the vertical movement of the camera. For example, it can be rotated 180 degrees.

This pan-tilt camera is installed on the ceiling of a public facility and can take a desired direction by operating a controller. Further, a user at a remote location can view an image captured by the camera and control the movement of the camera via a control terminal via a cable or a network.
JP-A-9-205574

  Conventional pan-tilt cameras are installed in public facilities and are used as live cameras for ornamental use to convey the real-time state of the site to users at remote locations, or as surveillance cameras for monitoring remote locations. .

  Here, for example, when photographing landscapes as an ornamental application, and when photographing a monitoring target such as a person as a surveillance application, the same processing is performed on image data.

  However, the required image quality and format differ between the monitoring application and the ornamental application, and it has been difficult to obtain a desired image according to the application with a conventional camera.

For example, as an item particularly required for image data for ornamental use,
(1) The image is preferable for appearance.
(2) reducing the recording capacity;
Etc.

On the other hand, for image data for surveillance purposes,
(1) Information is recorded without being damaged,
(2) The format is easy to search after recording,
Etc. are required.

  Moreover, even if the cameras are fixed in the same place, the application may differ depending on the shooting direction.

  For example, although it is generally open to the public as an ornamental camera for photographing an outdoor landscape, a monitor may check a room or the like for monitoring purposes at regular time intervals. At this time, it has been troublesome to change the camera to a monitoring setting every time the monitoring person confirms.

  An object of the present invention is to provide an imaging apparatus and an image processing apparatus capable of generating, outputting or recording appropriate image data suitable for a purpose of use according to a shooting direction.

  In order to achieve the above object, an imaging apparatus according to claim 1 is an imaging apparatus including an imaging unit, an image processing unit, and an output unit, and direction information acquisition unit that acquires direction information indicating a shooting direction of an input image. A direction determination unit that determines whether the imaging unit is directed in a monitoring purpose direction or an ornamental purpose direction from the direction information acquired by the direction information acquisition unit; and determination by the direction determination unit And controlling the image processing means to execute one of the image processing associated with the direction of the monitoring purpose and the image processing associated with the direction of the viewing purpose based on the result of And a control means.

  The imaging apparatus according to claim 2 is an imaging apparatus including an imaging unit, an image processing unit, a compression encoding unit, and an output unit, and direction information acquisition unit that acquires direction information indicating a shooting direction of an input image; Based on the direction information acquired by the information acquisition means, the direction determination means for determining whether the imaging means is directed in a monitoring purpose direction or an ornamental purpose direction, and the determination result of the direction determination means. Based on this, the compression coding means is controlled to execute one of the compression coding associated with the monitoring purpose direction and the compression coding associated with the viewing purpose direction. And a processing control means.

  According to a third aspect of the present invention, there is provided an imaging apparatus comprising: an imaging unit, an image processing unit, a compression encoding unit, a recording unit, and an output unit; and a direction information acquisition unit that acquires direction information indicating a shooting direction of an input image. A direction determination unit that determines whether the imaging unit is directed in a monitoring purpose direction or an ornamental purpose direction from the direction information acquired by the direction information acquisition unit; and determination by the direction determination unit And a process control means for performing control to switch the compression coding method of the compression coding means and whether to record to the recording means based on the result.

  The method for controlling an imaging apparatus according to claim 4 is a method for controlling an imaging apparatus including an imaging unit, an image processing unit, and an output unit, and a direction information acquisition step for acquiring direction information indicating a shooting direction of an input image; From the direction information acquired in the direction information acquisition step, a direction determination step for determining whether the imaging means is directed in a monitoring purpose direction or an ornamental purpose direction, and a result of the determination in the direction determination step A process control step of controlling the image processing means to execute one of the image processing associated with the direction of the monitoring purpose and the image processing associated with the direction of the viewing purpose based on It is characterized by providing.

  The imaging apparatus control method according to claim 5, wherein the imaging apparatus control method includes an imaging unit, an image processing unit, a compression encoding unit, and an output unit, and direction information for acquiring direction information indicating a shooting direction of the input image. An acquisition step; a direction determination step for determining whether the imaging means is directed in a monitoring purpose direction or an ornamental purpose direction from the direction information acquired in the direction information acquisition step; and the direction determination. Based on the determination result of the step, the compression is performed such that one of the compression coding associated with the direction of the monitoring purpose and the compression coding associated with the direction of the viewing purpose is executed. And a process control step for controlling the encoding means.

  The imaging apparatus control method according to claim 6 is an imaging apparatus control method including an imaging unit, an image processing unit, a compression encoding unit, a recording unit, and an output unit, and acquires direction information indicating a shooting direction of an input image. A direction determination step for determining whether the imaging means is directed in a monitoring purpose direction or a viewing purpose direction from the direction information obtained in the direction information acquisition step; And a process control step of performing control to switch the compression coding method of the compression coding means and to switch whether or not to record to the recording means based on the determination result of the direction determination step. And

  The program according to claim 7 is a program for causing a computer to execute a control method of an image pickup apparatus including an image pickup unit, an image processing unit, and an output unit, wherein the image pickup device control method includes direction information indicating a shooting direction of an input image. A direction information acquisition step for determining whether the imaging means is directed to a monitoring purpose direction or an ornamental purpose direction from the direction information obtained by the direction information acquisition step. Based on the determination result of the direction determination step, one of the image processing associated with the monitoring purpose direction and the image processing associated with the viewing direction is executed. And a process control step for controlling the image processing means.

  The program according to claim 8 is a program for causing a computer to execute a control method for an image pickup apparatus including an image pickup means, an image processing means, a compression encoding means, and an output means. From the direction information acquisition step for acquiring direction information indicating the direction and the direction information acquired by the direction information acquisition step, it is determined whether the imaging unit is facing a monitoring purpose direction or an ornamental purpose direction. Based on the determination result of the direction determination step and the determination of the direction determination step, one of the compression encoding associated with the direction of the monitoring purpose and the compression encoding associated with the direction of the viewing purpose And a process control step of controlling the compression encoding means so as to execute compression encoding.

  The program according to claim 9 is a program for causing a computer to execute a control method of an image pickup apparatus including an image pickup means, an image processing means, a compression encoding means, a recording means, and an output means. From the direction information acquisition step for acquiring the direction information indicating the shooting direction of the image, and the direction information acquired by the direction information acquisition step, the imaging means is directed to the monitoring purpose direction or the viewing purpose direction. A direction determining step for determining whether or not to perform the control, and a control for switching the compression encoding method of the compression encoding means based on the determination result of the direction determining step and switching whether or not to record to the recording means. And a processing control step to be performed.

  A computer-readable storage medium according to a tenth aspect stores the program according to any one of the seventh to ninth aspects.

  The image processing apparatus according to claim 11 is an image processing unit that performs predetermined image processing on input image data, a compression encoding unit that compresses and encodes the image processed image data, and a compression encoded unit. In an image processing apparatus comprising output / recording means for outputting or recording image data on a recording medium, direction information acquisition means for acquiring direction information indicating a shooting direction of an input image, and the direction information acquisition means acquired by the direction information acquisition means Based on the direction information, the direction determining means for determining whether the direction of the monitoring purpose is directed or the direction of the viewing purpose, and the direction of the monitoring purpose are associated with each other according to the determination result of the direction determining means. Of the image processing means, the compression encoding means, and the output / recording means so as to execute one of the movement actions and the movements associated with the viewing direction. Characterized in that it comprises a system control means for controlling the Chi any one or more.

  According to the imaging apparatus and the image processing apparatus of the present invention, it is possible to generate, output, or record appropriate image data according to the shooting direction.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(First embodiment)
As a first embodiment to which the present invention can be applied, an imaging apparatus that changes an image processing method in accordance with a shooting direction will be described. A first embodiment will be described with reference to FIGS.

  FIG. 1 is a block diagram of an imaging apparatus according to the first embodiment of the present invention.

  In FIG. 1, an imaging apparatus includes an imaging unit 101 including an optical system such as a lens and an imaging device such as a CMOS or a CCD, a camera image processing unit 102 that processes captured image data, and an output unit (output) that outputs image data. A recording means component 103); The imaging apparatus also includes a system control unit 110 that controls the entire system and a pan / tilt driving unit 111 that performs pan / tilt driving.

  The camera image processing unit 102 includes a luminance / color signal generation unit 121 that generates a luminance signal and a color signal from image data, a contour enhancement processing unit 122 that performs contour enhancement processing, and a luminance that performs gamma processing (correction) on the luminance signal. A gamma processing unit 123 is provided.

  The camera image processing unit 102 generates a white balance processing unit 124 that controls white balance, a color gamma processing unit 125 that performs gamma processing on the color signal, and color difference signals (RY, BY). A color difference signal generation unit 126 is provided. The camera image processing unit 102 includes a color correction processing unit 127 that corrects color reproduction and a noise removal processing unit 128 that removes noise components.

  FIG. 2 is a diagram illustrating a connection relationship between the imaging apparatus of FIG. 1 and an external information terminal.

  In FIG. 2, an imaging device 201 capable of pan / tilt control in FIG. 1 and an information terminal 202 (202 a, 202 b, 202 c) such as a PC (Personal Computer) are connected via a network.

  Application software for communicating with the imaging device 201 is installed in the information terminal 202, and the user operates the application software to perform various operations of the imaging device 201 and images taken by the imaging device 201. It is possible to see.

  FIG. 3 is a diagram illustrating an example of an application for operating the imaging apparatus in FIG.

  In FIG. 3, an image display area 301 displays an image output from the imaging device 201. The pan / tilt control button 302 operates the pan / tilt of the imaging apparatus 201. A zoom control button 303 controls zooming of the imaging apparatus 201.

  When the information terminal 202 receives an operation by the user through the application software, the information terminal 202 transmits the operation information to the imaging device 201.

  Next, returning to FIG. 1, the operation of the imaging apparatus having the above-described configuration will be described.

  The imaging unit 101 forms an optical image of a subject through an optical system on an imaging surface of an imaging element, and generates image data. The imaging unit 101 can switch the frame rate (for example, 30 frames / second and 60 frames / second). The generated image data is output to the camera image processing unit 102.

  In the camera image processing unit 102, first, a luminance / color signal generation unit 121 generates a luminance signal (Y) and a color signal (RGB). For the luminance signal, the contour enhancement processing unit 122 performs contour enhancement processing, and the luminance gamma processing unit 123 performs gamma processing (correction) (details of these processing will be described later).

  On the other hand, with respect to the color signal, the Wheat Ballath processing unit 124 applies a gain to the RGB signal and corrects the achromatic object so that it can be reproduced correctly. Next, the color gamma processing unit 125 performs gamma processing, and the color difference signal generation unit 126 converts the RGB signals into color difference signals (RY, BY). The color correction processing unit 127 performs a correction process for changing the hue / saturation on the generated color difference signal.

  The noise removal processing unit 128 performs filter processing using a low-pass filter or the like on the luminance signal and color difference signals (Y, RY, BY) processed as described above. The noise component of the image is removed (details of color correction processing and noise removal will be described later).

  The image signal processed as described above is output to the output unit 103. The output unit 103 outputs image data to the external information terminal 202 or the like.

  The system control unit 110 controls the entire imaging apparatus in response to an operation request from the outside. When an operation request is received from the information terminal 202 as described above, the system control unit 110 controls the pan / tilt driving unit 111.

  The pan / tilt driving unit 111 performs pan / tilt driving based on control from the system control unit 110. That is, it is possible to change the shooting direction as shown in FIG. As a result of the pan / tilt drive, the pan / tilt driving unit 111 acquires direction information (imaging direction information) indicating which direction the imaging unit 101 is facing and outputs the acquired direction information to the system control unit 110.

  The direction information is information indicating the horizontal and vertical rotation angles of the imaging unit 101. The system control unit 110 controls the imaging unit 101 and the camera image processing unit 102 based on the acquired direction information.

  Next, the control of the system control unit 110 based on the direction information will be described in detail.

  FIG. 4 is a flowchart showing a procedure of control processing based on direction information executed by the system control unit in FIG.

  In FIG. 4, in step S401, direction information (shooting direction information) indicating the shooting direction of the input image is acquired from the pan / tilt driving unit 111 as described above (direction information acquiring unit). Here, for simplification of explanation, only the rotation angle in the horizontal direction is acquired as the direction information.

  In step S402, it is determined from the direction information whether the imaging device 201 is facing the viewing purpose direction or the monitoring purpose direction (direction determination unit).

  FIG. 5 is a diagram illustrating a determination boundary in the horizontal direction of the imaging apparatus in FIG.

  FIG. 5 shows an angle reference (0 °) L500, and boundary angles L501 (180 °) and L502 (30 °) between the direction for viewing and the direction for monitoring. That is, it is judged as a monitoring purpose when facing the direction of 30 ° to 180 ° with reference to the boundary angles L501 and L502, and when it faces the direction of 0 ° to 30 ° and 180 ° to 360 °, to decide. The boundary angles L501 and L502 can be arbitrarily set by the user from the information terminal 202.

  Returning to FIG. 4, in S403, the branch is made based on the direction determined as described above (direction determination means). That is, when it is determined that the direction of the monitoring purpose is directed, the corresponding steps S404 to S408 are controlled (processing control means), and when it is determined that the direction of the viewing purpose is directed, the corresponding step S409 is performed. To S413 (processing control means).

  In step S404 or S409, the imaging unit 101 is controlled to change the frame rate. If it is determined in step S402 that the direction is the monitoring purpose, in step S404, the frame rate is set to be high (for example, 60 frames / second). On the other hand, if it is determined that the direction is for viewing, control is performed in step S409 so that the frame rate is lower (for example, 30 frames / second) than when it is determined to be monitoring.

  The reason for controlling in this way is to place an emphasis on reliably shooting an object that moves at high speed in the case of a surveillance application, while in the case of an ornamental application, it is necessary to keep the load on the transmission band low. This is for emphasis.

  In step S405 or S410, the contour enhancement processing unit 122 is controlled. The outline enhancement process will be briefly described.

  FIG. 6 is a diagram illustrating a waveform example in the process of edge enhancement processing of the luminance signal by the edge enhancement processing unit in FIG.

  FIG. 6A shows a luminance signal to be subjected to contour enhancement. A contour compensation signal as shown in FIG. 6B is generated by amplifying the high-frequency component of the luminance signal shown in FIG. When the contour compensation signal shown in FIG. 6B is added to the luminance signal shown in FIG. 6A, a luminance signal with an emphasized contour is obtained as shown in FIG. 6C.

  That is, the luminance signal shown in FIG. 6C has a larger amplitude difference between the rising edge and the falling edge of the waveform than the original luminance signal shown in FIG. .

  If it is determined in step S403 that the direction is a monitoring purpose, the contour enhancement processing is turned off in step S405. On the other hand, if it is determined that the direction is for viewing purposes, the contour emphasis process is turned on in step S410.

  The reason for controlling in this way is that it is desirable not to process the original image data as much as possible so that the captured image can be analyzed in detail later in the monitoring application, but it is easier to see in the viewing application. This is because it is considered better to emphasize the outline.

  In step S406 or S411, the luminance gamma processing unit 123 is controlled to switch the gamma processing characteristics.

  FIG. 7 is a characteristic diagram of gamma processing by the luminance gamma processing unit in FIG.

  In the graph of FIG. 7, the horizontal axis represents the input signal and the vertical axis represents the output signal. The two graphs in FIGS. 7A and 7B show different input / output characteristics. In the portion where the signal is close to black (0%) (P701a, P701b), (a) is steeper, so that (a) is the output image of the dark part after gamma processing. Is expressed more brightly.

  On the other hand, the portion (a) is steeper in the portion (P702a, P702b) where the signal is close to white (100%). Therefore, as the output image of the bright part after the gamma processing, (a) is assigned more gradations, and it becomes easier to recognize the bright part object. On the other hand, the characteristic (b) is a characteristic in which black is tightened and emphasis is placed not only on the bright portion but also on reproduction of intermediate gradation.

  If the system control unit 110 determines in step S403 that the object is to be monitored, in step S406, the system control unit 110 performs gamma processing of the characteristic of FIG. On the other hand, if it is determined that the object is for viewing, gamma processing of the characteristic shown in FIG. 7B is performed in step S411.

  The reason for controlling in this way is to generate an image that makes it easier for the supervisor to find objects, people, etc. that are present in the dark part by making the dark part slightly brighter in the case of monitoring applications. In addition, by assigning a large number of gradations to the bright part, it is possible to prevent the object in the bright part from being saturated and information being lost. On the other hand, in the case of ornamental use, the dark portion is tightened, and many gradations are assigned to the intermediate gradation, so that image characteristics preferable for appearance are obtained.

  In step S407 or S412, the color correction processing unit 127 is controlled to change the color reproduction characteristics.

  FIG. 8 is a characteristic diagram of color correction by the color correction processing unit in FIG.

  Specifically, FIG. 8 is a vector diagram showing a color difference (RY, BY) plane. As an example, a case where a chart of a reference color bar is photographed will be described. Six chromatic colors (red, yellow, blue, green, magenta, cyan) 801a to 801f of the color bar are shown.

  FIG. 8A shows an input signal to the color correction processing unit 127. FIG. 8B shows the color signal after color correction. In FIG. 8B, the hue and saturation are changed as compared with FIG. Specifically, the saturation is increased and brilliant as a whole so as to be visually appealing, and the hue of red 801a is changed from α to β.

  If the system control unit 110 determines in step S403 that the direction is a monitoring purpose, it turns off the color correction processing in step S407. On the other hand, if it is determined that the direction is for viewing, the color correction process is turned ON in step S412.

  The reason for controlling in this way is, in the case of surveillance applications, for example, it is desirable to reproduce the color of the subject as faithfully as possible in order to accurately identify the color of the suspicious object. This is because it is considered better to emphasize the colors so as to be preferable.

  In step S408 or S413, the noise removal processing unit 128 is controlled. The noise removal processing unit 128 performs low-pass filter processing for removing spatial and temporal high-frequency components of the image.

  If it is determined in step S403 that the direction is a monitoring purpose, the noise removal process is turned off in step S408. On the other hand, when it is determined that the direction is for viewing, the noise removal process is turned on in step S413.

  The reason for controlling in this way is to prevent changes in the signal due to noise removal so that the recorded image can be analyzed in detail in the case of monitoring applications. On the other hand, in the case of ornamental use, it is considered that it is better to remove noise so that it is preferable to the appearance, so the noise removal processing is turned on.

  As described above, the system control unit 110 controls the image processing parameters of the camera image processing unit 102 based on the shooting direction information of the input image, and generates image data. The generated image data is output to the output unit 103, and is output from the output unit 103 to the external information terminal 202.

  As described above, the imaging apparatus (image processing apparatus) according to the present embodiment includes the camera image processing unit 102, the bantilt driving unit 111 that acquires shooting direction information, and the system control unit 110 that inputs the shooting direction information. Prepare. The camera image processing unit 102 is controlled according to the shooting direction.

  With this configuration, when the imaging apparatus 201 is directed in the monitoring direction, processing is performed so that a subject in a dark part or a bright part is easy to see, and unnecessary image processing is prevented later. It is possible to output an image that can be analyzed in detail.

  On the other hand, when the imaging apparatus 201 is facing the direction of viewing, it is possible to generate a visually preferable image for viewing by performing various processes such as contour enhancement and color correction.

  In the above system, the system control unit 110 controls the imaging unit 101 and the camera image processing unit 102 according to the direction information. However, in addition to the above, the system control unit 110 can be configured to control the output unit 103. is there.

  For example, when it is determined that the direction of the monitoring purpose is being photographed, the image may be limited so as not to be output to an unspecified number of users. For example, it is possible to add a restriction such that an image is output only to the information terminal 202a in FIG. 2 and not output to the information terminals 202b and 202c.

  In the above example, the case where three information terminals 202a to 202c shown in FIG. 2 are connected is shown as an example, but the number of connectable information terminals 202 is not limited to this number. Further, although the case where the information terminal 202 is connected via a network has been described as an example, a configuration in which the information terminal 202 is connected using other means such as a wired cable can be employed.

  Further, in the above example, the case has been described in which the system control unit 110 controls all the processes (contour emphasis, gamma color correction, noise removal) and the output unit 103 of the imaging unit 101 and the camera image processing unit 102. However, it is also possible to adopt a configuration in which only some of these processes are controlled.

  In the above example, the case where the contour enhancement, color correction, and noise removal are turned off has been described as an example when it is determined that the direction is the purpose of monitoring, but it is not necessary to completely turn it off. In this case, each correction is made weaker than when the direction is determined to be an ornamental purpose. As a result, it is possible to prevent unnecessary image processing and obtain an image with little deterioration for monitoring purposes.

  In the above example, the pan / tilt driving unit 111 and the imaging device are described as an example. However, if the imaging device is installed on the camera platform and the direction information can be acquired from the camera platform, the direction information can be acquired. It can be realized in any form.

  Further, in the above example, for simplification of explanation, an example in which only the horizontal direction is used has been described. However, a configuration in which the direction for monitoring and the direction for viewing are determined in consideration of the vertical direction is taken. Is also possible.

(Second Embodiment)
As a second embodiment to which the present invention can be applied, a description will be given of an imaging apparatus that changes the control of compression encoding in accordance with the shooting direction.

  FIG. 9 is a block diagram of an imaging apparatus according to the second embodiment of the present invention.

  The imaging apparatus according to the second embodiment compresses and encodes a captured image using the MPEG2 method and outputs the image. The imaging apparatus is the same as the imaging apparatus of the first embodiment in that it is connected to an external information terminal 202 as shown in FIG. The imaging apparatus according to the present embodiment includes a compression encoding unit 904 that performs compression encoding processing in addition to the imaging apparatus according to the first embodiment, and controls the compression encoding unit 904 based on the direction information. This is different from the imaging device of the first embodiment.

  In FIG. 9, an imaging unit 101, a camera image processing unit 102, an output unit 103, and a pan / tilt driving unit 111 are the same as those in the first embodiment. The present embodiment includes a compression encoding unit 904 that compresses and encodes image data using the MPGE2 method, and a system control unit 910 having a function different from that of the first embodiment.

  The compression encoding unit 904 includes an image frame buffer 921 for accumulating and rearranging image data, a subtracting unit 922 for obtaining a difference between the input image and the local decoded image, a feature extracting unit 923 for extracting features of the image data, and a DCT. A conversion unit 924 and a quantization unit 925 are provided. The compression encoding unit 904 includes an inverse quantization unit 926, an inverse DCT transform unit 927, a motion prediction / compensation unit 928, a variable length encoding unit 929, an output buffer 930, and a quantization control unit 931.

  Next, the operation of the imaging apparatus having the above configuration will be described.

  The imaging unit 101 generates subject image data, and the operation of performing image processing in the camera image processing unit 102 is the same as in the first embodiment, and thus detailed description thereof is omitted.

  The camera image processing unit 102 outputs the image data to the image frame buffer 921, and the image data is accumulated in the image frame buffer 921. The image frame buffer 921 rearranges the image frames in the order of encoding according to the picture type.

  Here, the coded picture type will be described. In the MPEG2 system, image data is encoded with three types of pictures: I, P, and B. An I picture is an image that is encoded only with information in one frame, a P picture is encoded by performing prediction from the I or P picture, and a B picture is an image that is encoded by bidirectional prediction.

  Since the B picture refers to the temporally subsequent picture, the encoding order is after the reference picture is encoded. The images rearranged in the encoding order are divided into macroblock units, which are small areas having a predetermined size, and output to the DCT transform unit 924 for I pictures and to the subtractor 922 for B and P pictures. Is done.

  The subtraction unit 922 takes a difference between a prediction image based on inter-frame prediction described later and the current image, and outputs the difference image to the DCT conversion unit 924. The DCT conversion unit 924 performs DCT conversion in units of DCT blocks and converts them to frequency components.

  The quantization unit 925 quantizes the input frequency component data according to a quantization parameter described later. The image data quantized by the quantization unit 925 is output to the variable length coding unit 929 and the inverse quantization unit 926 for local decoding.

  The variable length encoding unit 929 performs variable length encoding so that a shorter code is assigned to data with a higher appearance frequency, temporarily stores the data in the output buffer 930, and then outputs the encoded data.

  On the other hand, the inverse quantization unit 926 inversely quantizes the image data quantized by the quantization unit 925 and decodes it into frequency components. The inverse DCT transform unit 927 decodes the frequency component image data into a prediction error image (difference image) by inverse orthogonal transform.

  The motion prediction / compensation unit 928 searches the local decoded image for a reference image having the smallest difference from the input image, and calculates a motion vector. Further, the calculation indicated by the motion vector and the reference direction information is performed, and the motion compensated image is output to the subtraction unit 922.

  The feature extraction unit 923 calculates the complexity of the image data for each macroblock, and outputs the complexity information to the quantization control unit 931. The quantization control unit 931 grasps the code amount by monitoring the output buffer 930 and determines a quantization parameter (quantization scale) that matches the target bit rate.

  If this quantization scale is increased, it is possible to roughly quantize (assign a small amount of code), and decrease it to finely quantize (assign a large amount of code).

  Further, the quantization scale determined by the code amount control is corrected according to the complexity output from the feature extraction unit 923. Specifically, since a pattern with a high degree of complexity is hardly noticeable, correction is performed so that the quantization scale becomes large. On the other hand, since a pattern such as a flat part having a low complexity is easily deteriorated, correction is performed so that a large amount of code is allocated.

  The above is the basic operation flow in the compression encoding unit 904. In the present embodiment, the system control unit 910 controls the compression encoding unit 904 based on the shooting direction information.

  The operation of the pan / tilt driving unit 111 is the same as that of the first embodiment, and the shooting direction information is output to the system control information 910. The system control unit 910 controls the compression encoding unit 904 based on the shooting direction information. A flow of processing controlled by the system control unit 910 will be described with reference to FIG.

  FIG. 10 is a flowchart showing the procedure of the control process based on the direction information executed by the system control unit in FIG.

  In FIG. 10, in step S1001, direction information is acquired from the pan / tilt driving unit 111 as in the first embodiment. In step S <b> 1002, it is determined from the direction information whether the imaging apparatus is facing the viewing purpose direction or the monitoring purpose direction. This is also the same as in the first embodiment.

  In S1003, the process branches based on the direction determined as described above. If it is determined that the direction is directed to the monitoring purpose, the control in steps S1004 and S1005 is performed, and if it is determined that the direction is directed to the viewing purpose, the control in steps S1006 and S1007 is performed.

  In step S1004 or S1006, the picture type is controlled. If it is determined in step S1003 that the direction is the monitoring purpose, in step S1004, encoding is performed using only the I picture.

  FIG. 11 is a diagram illustrating a configuration example of a picture at the time of encoding by the compression encoding unit in FIG. 9.

  In FIG. 11, (a) is an example of encoding only with an I picture. On the other hand, if it is determined that the direction is for viewing purposes, in step S1006, encoding is performed using I, B, and P pictures. A configuration example in the case of encoding using an IBP picture is as shown in FIG.

  The reason for changing the picture type configuration in this way will be described. A B picture using bi-directional prediction has a delay because it needs to be buffered. In addition, if inter-frame prediction is used, the reference relationship becomes complicated, which may cause a decrease in search speed.

  Therefore, for the purpose of monitoring, encoding is performed only with an I picture that is intra-frame encoding in order to emphasize real-time characteristics and search speed. On the other hand, for viewing purposes, encoding is performed using B and P pictures in order to place importance on encoding efficiency.

  Next, the target bit rate is controlled in step S1005 or S1007. If it is determined in step S1003 that the direction is a monitoring purpose, in step S1005, the target bit rate is set to be relatively high. On the other hand, if it is determined that the direction is for viewing purposes, the target bit rate is set to be relatively low in step S1007.

  That is, the target bit rate is controlled such that B1> B2, where B1 is the bit rate when it is determined as the step monitoring purpose direction and B2 is the bit rate when it is determined as the viewing purpose direction.

  By increasing the target bit rate, the quantization scale set by the quantization control unit 931 is reduced, and as a result, the amount of generated code is increased, but an image with little deterioration is obtained.

  As described above, the system control unit 910 controls the compression encoding unit 904, and the compression encoded image data is output to the output unit 103. The output unit 103 outputs the encoded image data to the external information terminal 202 or the like.

  As described above, the imaging apparatus (image processing apparatus) according to the present embodiment includes the compression encoding unit 904 that compresses and encodes an image, the bantilt drive unit 111 that acquires imaging direction information, and the imaging direction information thereof. The system control part 110 which inputs is provided. And it was set as the structure which controls the compression encoding part 904 according to an imaging | photography direction.

  Here, the parameter of the compression encoding process is a ratio of a picture to be encoded within a frame and between frames. The parameter of the compression encoding process is a target bit rate.

  When the imaging device is in the direction of the monitoring purpose, encoding with only I pictures makes it easy to search later, and does not use bi-directional prediction, thereby enabling output without delay. In addition, when the direction is for viewing purposes, it is possible to reduce the amount of generated code by using an IBP picture.

  In addition, when the imaging device is facing the monitoring purpose, it is possible to generate image data with less deterioration of the subject by increasing the target bit rate compared to when the imaging device is facing the viewing purpose. It becomes. As a result, it is possible to obtain an image that can be analyzed in detail with little deterioration for monitoring purposes.

  In the above system, the example in which the system control unit 910 controls both the picture type and the target bit rate in accordance with the shooting direction has been described. However, it is possible to adopt a configuration in which only the picture type or the target bit rate is controlled. It is.

  Further, the process controlled by the system control unit 910 is not limited to the picture type and the target bit rate. It is also possible to add processing for controlling the quantization control unit 931 according to the shooting direction.

  For example, the feature extraction unit 923 determines whether the macroblock to be processed includes a human face (or a part of the human face). When the shooting direction is directed to the monitoring direction, the quantization control unit 931 is controlled so as to allocate a larger amount of code to the macroblock including the human face (decreasing the quantization step).

  This makes it possible to prevent the face of a person from being deteriorated due to encoding when facing the monitoring purpose direction, which is effective when searching for a suspicious person or the like from a captured image later.

  Further, in the above example, when the picture type is controlled, the control is performed by using only the I picture when the monitoring is directed in the direction of the monitoring purpose. However, if the purpose is to reduce the delay, it is possible to adopt a configuration in which encoding is performed using an I picture and a P picture instead of using only a B picture of bidirectional prediction that causes a large delay. It is.

  In the above example, the case where the system control unit 910 controls only the compression encoding unit 904 has been described as an example. However, as in the first embodiment, the camera image processing unit 102 is controlled according to the shooting direction. It is also possible to do.

(Third embodiment)
As a third embodiment to which the present invention can be applied, a description will be given of an imaging apparatus that changes a compression encoding method and recording control in accordance with a shooting direction.

  FIG. 12 is a block diagram of an imaging apparatus according to the third embodiment of the present invention.

  The imaging device according to the third embodiment is the same as the first embodiment and the second embodiment in that it is connected to an external information terminal 202 as shown in FIG. The present embodiment is different from the second embodiment in that a plurality of compression encoding units are provided and the compression encoding method is switched based on the direction information.

  In FIG. 12, an imaging unit 101, a camera image processing unit 102, an output unit 103, and a pan / tilt driving unit 111 are the same as those in the first embodiment. The image pickup apparatus according to the present embodiment further controls a compression encoding unit 1204 that compresses and encodes image data, a recording unit (output / recording unit constituent element) 1205 that accumulates image data such as a hard disk drive, and the entire system. A system control unit 1210 is provided.

  The compression encoding unit 1204 includes a Motion JPEG compression unit 1221 that compresses the Motion JPEG method and an MPEG2 compression unit 1222 that compresses and encodes image data using the MPEG2 method.

  Next, the operation of the imaging apparatus having the above configuration will be described.

  Since the operation of generating subject image data in the imaging unit 101 and performing image processing in the camera image processing unit 102 is the same as that in the first embodiment, detailed description thereof is omitted.

  Image data processed by the camera image processing unit 102 is input to the compression encoding unit 1204. The compression encoding unit 1204 compresses and encodes image data using the Motion JPEG method or the MPEG2 method, and outputs the image data to the output unit 103 and the recording unit 1205. The output unit 103 outputs image data to the external information terminal 202 or the like. The recording unit 1205 accumulates (records) image data.

  The operation of the pan / tilt drive unit 111 is the same as that of the first embodiment, and outputs shooting direction information to the system control unit 1210. The system control unit 1210 controls the compression encoding unit 904 and the recording unit 1205 based on the shooting direction information. A processing flow of the system control unit 1210 will be described with reference to FIG.

  FIG. 13 is a flowchart showing a procedure of control processing based on direction information executed by the system control unit in FIG.

  In FIG. 13, in step S1301, direction information is acquired from the pan / tilt drive unit 111, as in the first embodiment. Similar to the first embodiment, it is determined in step S1302 from the direction information whether the imaging apparatus is facing the viewing purpose or the monitoring purpose.

  In S1303, the process branches based on the direction determined as described above. If it is determined that the direction of the monitoring purpose is facing, the control of steps S1304 and S1305 is performed, and if it is determined that the direction of the viewing purpose is facing, the control of steps S1306 and S1307 is performed.

  In step S1304 or S1306, the encoding method of the compression encoding unit 1204 is switched according to the shooting direction. If it is determined in step S1303 that the direction is the monitoring purpose, in step S1304, compression encoding is performed using the Motion JPEG method. The Motion JPEG method is characterized in that each frame of a moving image is compressed in JPEG format and does not use correlation between frames.

  On the other hand, when it is determined in step S1303 that the direction is for viewing, compression encoding is performed using the MPEG2 method in step S1306. The MPEG2 system is the same as the compression encoding unit 904 described in the second embodiment, and compresses and encodes image data using correlation between frames.

  In step S1305 or S1307, the recording unit 1205 is switched according to the shooting direction. If it is determined in step 1303 that the direction is the monitoring purpose, in step S1305, the recording unit 1205 is controlled so that image data is always recorded on the recording medium while the recording unit 1205 is photographing.

  If it is determined in step S1303 that the direction is for viewing, the recording unit 1205 is controlled so that image data is recorded on the recording medium only when a recording request is received from the user or the like in step S1307. .

  As described above, the imaging apparatus (image processing apparatus) according to the present embodiment inputs the compression encoding unit 1204 that compresses and encodes an image, the bantilt drive unit 111 that acquires imaging direction information, and the imaging direction information thereof. The system control unit 110 is provided. And it was set as the structure which switches a compression encoding system according to an imaging | photography direction.

  When the imaging device is in the direction of the monitoring purpose, it is encoded by the Motion JPEG method that does not use inter-frame correlation. This makes it easy to search later, and does not use bidirectional prediction, enabling output in a format without delay. In addition, when it is directed to the viewing purpose, it is encoded by the MPEG2 method using inter-frame correlation. As a result, the encoding efficiency can be improved as compared with the case of monitoring purposes.

  In addition, when the image pickup apparatus is directed to the monitoring purpose, the image data is always recorded on the recording medium (recording unit 1205), thereby preventing a subject from being recorded that is important for the monitoring purpose. Is possible.

  In the above description, the system control unit 1210 is configured to control both the compression encoding unit 1204 and the recording unit 1205. However, a configuration in which only one of them is controlled is also possible.

  In the above description, Motion JPEG has been described as an example of a compression encoding method that does not use inter-frame correlation. However, other methods such as Motion JPEG 2000 and DV may be used as long as the method does not use inter-frame correlation. I do not care. Also, the format using inter-frame correlation is not limited to the MPEG2 system. H.264, VC1 method, or the like may be used.

  The object of the present invention is achieved by executing the following processing. That is, a storage medium that records a program code of software that realizes the functions of the above-described embodiments is supplied to a system or apparatus, and a computer (or CPU, MPU, etc.) of the system or apparatus is stored in the storage medium. This is the process of reading the code.

  In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the program code and the storage medium storing the program code constitute the present invention.

  Moreover, the following can be used as a storage medium for supplying the program code. For example, floppy (registered trademark) disk, hard disk, magneto-optical disk, CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD + RW, magnetic tape, nonvolatile memory card, ROM or the like. Alternatively, the program code may be downloaded via a network.

  Further, the present invention includes a case where the function of the above-described embodiment is realized by executing the program code read by the computer. In addition, an OS (operating system) running on the computer performs part or all of the actual processing based on an instruction of the program code, and the functions of the above-described embodiments are realized by the processing. Is also included.

  Furthermore, a case where the functions of the above-described embodiment are realized by the following processing is also included in the present invention. That is, the program code read from the storage medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. Thereafter, based on the instruction of the program code, the CPU or the like provided in the function expansion board or function expansion unit performs part or all of the actual processing.

1 is a block diagram of an imaging apparatus according to a first embodiment of the present invention. It is a figure which shows the connection relationship of the imaging device of FIG. 1, and an external information terminal. It is a figure which shows the example of the application which operates the imaging device in FIG. It is a flowchart which shows the procedure of the control process based on the direction information performed by the system control part in FIG. It is a figure which shows the determination boundary of the horizontal direction of the imaging device in FIG. It is a figure which shows the example of a waveform in the process of the outline emphasis process of the luminance signal by the outline emphasis processing part in FIG. FIG. 2 is a characteristic diagram of gamma processing by a luminance gamma processing unit in FIG. 1. FIG. 2 is a characteristic diagram of color correction by a color correction processing unit in FIG. 1. It is a block diagram of the imaging device concerning a 2nd embodiment of the present invention. It is a flowchart which shows the procedure of the control process based on the direction information performed by the system control part in FIG. FIG. 10 is a diagram illustrating a configuration example of a picture at the time of encoding by a compression encoding unit in FIG. 9. It is a block diagram of the imaging device concerning a 3rd embodiment of the present invention. It is a flowchart which shows the procedure of the control processing based on the direction information performed by the system control part in FIG. It is a figure which shows a motion of a pan / tilt camera.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 Image pick-up part 102 Camera image process part 103 Output part 110 System control part 111 Pan / tilt drive part 121 Luminance and color signal generation part 122 Outline emphasis process part 123 Brightness gamma process part 124 White balance process part 125 Color gamma process part 126 Color difference signal generation 127 Color correction processing unit 128 Noise removal processing unit

Claims (30)

In an imaging apparatus comprising an imaging means, an image processing means, and an output means,
Direction information acquisition means for acquiring direction information indicating the shooting direction of the input image;
Direction determining means for determining from the direction information acquired by the direction information acquiring means whether the imaging means is directed in a monitoring purpose direction or an ornamental purpose direction;
Based on the determination result of the direction determination unit, the image processing is performed so that one of the image processing associated with the monitoring target direction and the image processing associated with the viewing purpose direction is executed. Processing control means for controlling the processing means;
An imaging apparatus comprising: In an imaging apparatus comprising an imaging means, an image processing means, a compression encoding means, and an output means,
Direction information acquisition means for acquiring direction information indicating the shooting direction of the input image;
Direction determining means for determining from the direction information acquired by the direction information acquiring means whether the imaging means is directed in a monitoring purpose direction or an ornamental purpose direction;
Based on the result of determination by the direction determination means, one of the compression encoding associated with the direction of the monitoring purpose and the compression encoding associated with the direction of the viewing purpose is executed. Processing control means for controlling the compression encoding means;
An imaging apparatus comprising: In an imaging apparatus comprising an imaging means, an image processing means, a compression encoding means, a recording means, and an output means,
Direction information acquisition means for acquiring direction information indicating the shooting direction of the input image;
Direction determining means for determining from the direction information acquired by the direction information acquiring means whether the imaging means is directed in a monitoring purpose direction or an ornamental purpose direction;
A process control means for performing control to switch the compression coding method of the compression coding means and to switch whether or not to record to the recording means based on the determination result of the direction determination means;
An imaging apparatus comprising: In a control method for an imaging apparatus comprising an imaging means, an image processing means, and an output means,
A direction information acquisition step for acquiring direction information indicating the shooting direction of the input image;
A direction determination step for determining whether the imaging means is directed in a monitoring purpose direction or an ornamental purpose direction from the direction information acquired in the direction information acquisition step;
Based on the determination result of the direction determination step, the image processing is performed such that one of image processing associated with the monitoring target direction and image processing associated with the viewing purpose direction is executed. A process control step for controlling the processing means;
An image pickup apparatus control method comprising: In a control method for an imaging apparatus comprising an imaging means, an image processing means, a compression encoding means, and an output means,
A direction information acquisition step for acquiring direction information indicating the shooting direction of the input image;
A direction determination step for determining whether the imaging means is directed in a monitoring purpose direction or an ornamental purpose direction from the direction information acquired in the direction information acquisition step;
Based on the result of the determination in the direction determination step, one of the compression encoding associated with the direction of the monitoring purpose and the compression encoding associated with the direction of the viewing purpose is executed. A process control step for controlling the compression encoding means;
An image pickup apparatus control method comprising: In a control method for an imaging apparatus comprising an imaging means, an image processing means, a compression encoding means, a recording means, and an output means,
A direction information acquisition step for acquiring direction information indicating the shooting direction of the input image;
A direction determination step for determining whether the imaging means is directed in a monitoring purpose direction or an ornamental purpose direction from the direction information acquired in the direction information acquisition step;
A process control step for performing control to switch the compression coding method of the compression coding means and to switch whether or not to record to the recording means, based on the determination result of the direction determination step;
An image pickup apparatus control method comprising: In a program for causing a computer to execute a control method of an imaging apparatus including an imaging means, an image processing means, and an output means,
The control method of the imaging device is:
A direction information acquisition step for acquiring direction information indicating the shooting direction of the input image;
A direction determination step for determining whether the imaging means is directed in a monitoring purpose direction or an ornamental purpose direction from the direction information acquired in the direction information acquisition step;
Based on the determination result of the direction determination step, the image processing is performed such that one of image processing associated with the monitoring target direction and image processing associated with the viewing purpose direction is executed. A process control step for controlling the processing means;
A program comprising: In a program for causing a computer to execute a control method of an imaging apparatus including an imaging unit, an image processing unit, a compression encoding unit, and an output unit,
The control method of the imaging device is:
A direction information acquisition step for acquiring direction information indicating the shooting direction of the input image;
A direction determination step for determining whether the imaging means is directed in a monitoring purpose direction or an ornamental purpose direction from the direction information acquired in the direction information acquisition step;
Based on the result of the determination in the direction determination step, one of the compression encoding associated with the direction of the monitoring purpose and the compression encoding associated with the direction of the viewing purpose is executed. A process control step for controlling the compression encoding means;
A program comprising: In a program for causing a computer to execute a control method of an imaging apparatus including an imaging unit, an image processing unit, a compression encoding unit, a recording unit, and an output unit,
The control method of the imaging device is:
A direction information acquisition step for acquiring direction information indicating the shooting direction of the input image;
A direction determination step for determining whether the imaging means is directed in a monitoring purpose direction or an ornamental purpose direction from the direction information acquired in the direction information acquisition step;
A process control step for performing control to switch the compression coding method of the compression coding means and to switch whether or not to record to the recording means, based on the determination result of the direction determination step;
A program comprising:   A computer-readable storage medium storing the program according to claim 7. Image processing means for performing predetermined image processing on the input image data;
Compression encoding means for compressing and encoding image processed image data;
Output / recording means for outputting the compression-encoded image data or recording it on a recording medium;
In an image processing apparatus comprising:
Direction information acquisition means for acquiring direction information indicating the shooting direction of the input image;
Direction determining means for determining from the direction information acquired by the direction information acquiring means whether it is directed to a monitoring purpose direction or an ornamental purpose direction;
The image processing unit is configured to execute one of an operation associated with the monitoring target direction and an operation associated with the viewing purpose direction according to a determination result of the direction determination unit. System control means for controlling at least one of the compression encoding means and the output / recording means;
An image processing apparatus comprising: An image pickup means for picking up image data;
The image processing apparatus according to claim 11, wherein the image processing unit receives image data output from the imaging unit.   The image processing apparatus according to claim 1, wherein the system control unit controls an image processing parameter of the image processing unit.   The image processing apparatus according to claim 13, wherein the image processing parameter is a frame rate.   When the direction determining means determines that the shooting direction is facing the direction of the monitoring purpose, the frame rate is made higher than when it is determined that the shooting direction is facing the direction of the viewing purpose. The image processing apparatus according to claim 13.   The image processing apparatus according to claim 3, wherein the image processing parameter is a parameter of gamma processing.   When the direction determining means determines that the shooting direction is in the direction of the monitoring purpose, the gamma processing parameter that makes the dark portion brighter than in the case of determining that the direction of the viewing is in the direction of the viewing purpose is used. The image processing apparatus according to claim 16, wherein gamma processing is performed.   The gamma processing parameter in which when the direction determining means determines that the shooting direction is in the direction of the monitoring purpose, the gradation of the bright part remains more than in the case of determining that the direction of the viewing is in the direction of the viewing purpose. The image processing apparatus according to claim 16, wherein gamma processing is performed using an image processing apparatus.   The image processing apparatus according to claim 13, wherein the image processing parameter is a parameter for color correction.   When the direction determining means determines that the shooting direction is in the direction of the monitoring purpose, the saturation and hue corrections are weaker than in the case where it is determined that the shooting direction is in the direction of the viewing purpose. The image processing apparatus according to claim 19.   The image processing apparatus according to claim 13, wherein the image processing parameter is a parameter for contour enhancement processing.   When it is determined by the direction determination means that the shooting direction is in the direction of the monitoring purpose, the edge enhancement processing is weaker than in the case where it is determined that the shooting direction is in the direction of the viewing purpose. The image processing apparatus according to claim 19.   The image processing apparatus according to claim 13, wherein the image processing parameter is a parameter for noise removal processing.   When it is determined by the direction determination means that the shooting direction is oriented in the direction of the monitoring purpose, the noise removal processing is applied more weakly than in the case where it is determined that the shooting direction is directed in the direction of the viewing purpose. The image processing apparatus according to claim 23.   The image processing apparatus according to claim 11, wherein the system control unit controls a parameter of the compression encoding processing unit.   The image processing apparatus according to claim 16, wherein the parameter of the compression encoding process is a ratio of a picture encoded within a frame and between frames.   When it is determined by the direction determination means that the shooting direction is in the direction of the monitoring purpose, the ratio of the encoded pictures in the frame is larger than in the case where it is determined that the shooting direction is in the direction of the viewing purpose. 24. The image processing apparatus according to claim 23, wherein the number is increased.   26. The image processing apparatus according to claim 25, wherein the compression coding processing parameter is a target bit rate.   When the direction determining means determines that the shooting direction is facing the monitoring purpose direction, the target bit rate is set higher than when it is determined that the shooting direction is facing the viewing purpose direction. The image processing apparatus according to claim 28.   26. The image processing apparatus according to claim 25, wherein the parameter of the compression encoding process is an allocation of a code amount within one frame.

Images