JP7080086B2 - Imaging device - Google Patents

Description

The embodiment of the present invention relates to an image pickup apparatus.

There is a method of displaying an electronic line by superimposing it on an image acquired by an image pickup device such as a camera and displayed on a monitor. Generally, the electronic line is displayed as a vertical line and a horizontal line, and can move horizontally or vertically in the screen of the monitor in units of one pixel or one line according to the input from the operator.

However, the conventional electronic line display is a function for the purpose of aligning a subject or marking on the screen, and is a function independent of processing related to imaging and image quality.

Japanese Unexamined Patent Publication No. 2013-140042

An object of the present embodiment is to improve the convenience of operation.

The image pickup apparatus according to the present embodiment includes an image sensor, an image generation unit, a display control unit, a specific unit, an electron line control unit, and a control unit . The image sensor is arranged in a matrix and includes a plurality of pixels that generate an electric signal by receiving light. The image generation unit generates an image based on the electric signal. The display control unit displays the image on the display unit and displays an electronic line on the displayed image. The specific unit identifies a part of the image based on the display position of the electronic line . The image sensor outputs an electric signal only from the pixels corresponding to a part of the specified image by the partial readout process. The electronic line control unit changes the display position of the electronic line and changes the angle of the electronic line according to the angle of the subject. The control unit controls the angle of the image pickup unit including the image sensor according to the angle.

FIG. 1 is a block diagram showing an image pickup device. FIG. 2 is a block diagram showing details of the image processing circuit. FIG. 3 is a diagram showing a display example of an electronic line. FIG. 4 is a diagram showing an example of a detection area specifying process. FIG. 5 is a diagram showing another example of the specific processing of the detection area. FIG. 6 is a diagram showing an example of a detection area cutting process. FIG. 7 is a diagram showing an example of rotation processing of the detection area. FIG. 8 is a diagram showing an example of the size change processing of the detection area.

Hereinafter, the image pickup apparatus according to the present embodiment will be described with reference to the drawings. In the following embodiments, the parts with the same reference numerals perform the same operation, and duplicate description will be omitted as appropriate.

The image pickup apparatus according to the present embodiment is assumed to be, for example, an industrial inspection camera, a measurement camera, a surveillance camera, or a microscope camera, but may be a home-use camera or a medical-use camera. Further, although the image pickup apparatus according to the present embodiment is assumed to shoot a moving image, a still image may be shot.

The image pickup apparatus according to this embodiment will be described with reference to the block diagram of FIG.
The image pickup apparatus 1 according to the present embodiment includes an image pickup unit 11, an image generation circuit 12, a display control circuit 13, an operation reception circuit 14, an image processing circuit 15, a control circuit 16, a monitor 17, and a USB output. Includes circuit 18.

The image pickup unit 11 is a camera including, for example, a lens 111, a shutter 113, and an image sensor 115. The image pickup unit 11 generates an image pickup signal based on the input light. More specifically, the lens 111 collects light to generate input light. The shutter 113 adjusts the amount of light that reaches the image sensor 115 in the subsequent stage. Examples of the image sensor 115 include a CCD (Charge-Coupled Device) sensor and a CMOS (Complementary Metal-Oxide Semiconductor) sensor, but other image pickup devices may be used. The image sensor 115 is arranged in a matrix and includes a plurality of pixels that generate an electric signal by receiving light. The image sensor 115 generates an image pickup signal obtained by amplifying an electric signal.

When outputting a color image, although not shown here, a prism may be provided between the lens 111 and the shutter 113. The prism separates the input light into an R signal, a G signal and a B signal. Each of the R signal, the G signal, and the B signal may be processed by a separate image sensor 115, and the three processed electrical signals may be combined to generate one imaging signal (RGB signal). Alternatively, a color filter may be provided in front of the image sensor 115 instead of the prism. R light, G light, and B light may be extracted by the color filter, and each light may be received by the image sensor 115 and processed by the image generation circuit 12 in the subsequent stage so as to be one image pickup signal (RGB signal). Since the processing for the separated R signal, G signal, and B signal is general processing, a specific description thereof will be omitted.

The image generation circuit 12 receives an image pickup signal from the image pickup unit 11 and performs general signal processing such as gain adjustment, analog-digital conversion, color balance adjustment, and gamma correction on the image pickup signal to generate an image signal.

The image pickup unit 11 and the image generation circuit 12 may be formed integrally or may be separate bodies. When the image pickup unit 11 and the image generation circuit 12 are integrally mounted on the camera, for example, the camera alone generates an image signal. An image generation circuit 12 included in the camera is a personal computer equipped with a CPU (Central Processing Unit) including an image processing circuit 15 in a subsequent stage for displaying an image signal via a communication interface (not shown) mounted in the camera. You can send it by wire or wirelessly. On the other hand, when the image pickup unit 11 and the image generation circuit 12 are configured as separate bodies, for example, the image pickup unit 11 is arranged at a predetermined position, and the image pickup unit 11 is mounted on the image pickup unit 11 (shown). The image pickup signal may be transmitted by wire or wirelessly to the image generation circuit 12 arranged away from the image pickup unit 11 via (1).

The display control circuit 13 receives an image signal from the image generation circuit 12. The display control circuit 13 displays an image on the monitor 17 and displays an electronic line on the displayed image. The electronic line may be displayed by adding a pixel value (luminance value) representing the electronic line to the pixels constituting the image, or the image and the electronic line are superimposed at the pixel position on the screen of the monitor 17. It may be displayed. Further, the display control circuit 13 may receive the image before image processing for recording from the image generation circuit 12, and control the output so that the image is displayed on the display unit.

The operation reception circuit 14 receives an input from an input device (not shown) such as a keyboard, a mouse, or a touch panel display. Specifically, the operation reception circuit 14 receives a user instruction input by the user to the input device.

The image processing circuit 15 receives signals related to images and electronic lines from the display control circuit 13. The image processing circuit 15 performs image processing on an image and an electronic line according to a user instruction. The image and the electronic line after image processing are output to the display unit. When the image processing circuit 15 does not perform image processing, the image and the electronic line may be output to the monitor 17 as they are. The details of the image processing circuit 15 will be described later with reference to FIG.

The control circuit 16 controls the posture of the image pickup unit 11, the shutter 113, and the image sensor 115 based on the processing result of the image processing circuit 15. The control circuit 16 causes the image sensor 115 to execute a partial readout process based on the processing result of the image processing circuit 15.

The monitor 17 receives the output of the image and the electronic line from the display control circuit 13 or the image processing circuit 15, and displays the image and the electronic line on the screen of the monitor 17.

The USB (Universal Serial Bus) output circuit 18 controls to output images and electronic lines to USB.

Next, the details of the image processing circuit 15 according to the present embodiment will be described with reference to the block diagram of FIG.
The image processing circuit 15 includes an electronic line control function 161, a specific function 162, a cutting function 163, a rotation function 164, a size changing function 165, and an analysis function 166.

The electronic line control function 161 changes the display position of the electronic line according to the user's instruction. Specifically, the electron line control function 161 performs processing such as generation or deletion of an electron line, rotation, movement, and designation of an electron line.

The specific function 162 identifies a part of the image based on the display position of the electronic line. Specifically, the partial image belonging to the detection area is specified by specifying the area partitioned by the electron line or the area surrounded by the plurality of electron lines as the detection area.

The cutout function 163 cuts out a partial image belonging to the detection area specified by the specific function 162.
The rotation function 164 rotates a partial image belonging to the detection area. The electron line may be rotated together with the detection area. When the detection area and the electronic line are rotated together, the image is rotated around the detection area while maintaining the positional relationship between the electronic line and the detection area displayed on the screen of the monitor 17.

The size change function 165 changes the image size so as to enlarge or reduce the image captured by the image pickup unit 11 or the partial image belonging to the detection area.

The analysis function 166 analyzes the pixel value, contrast, etc. of the partial image belonging to the detection area, and controls the control of the brightness (Auto Exposure: AE), white balance (WB), and focus adjustment (AF) of the image. Determine the parameters.
The control circuit 16 receives control parameters as the processing result of the image processing circuit 15, and controls the aperture of the lens 111 and the exposure time by the shutter 113 based on the control parameters to obtain the brightness, white balance, and focus of the image. Control adjustments, etc. In the white balance control, the gain (sensitivity) of the R signal and the B signal is specifically controlled.

Next, a display example of the electronic line according to the present embodiment will be described with reference to FIG.
In FIG. 3, the electron line 301 shown by the broken line indicates the position where the horizontal electron line is displayed. On the other hand, the electron line 302 shown by the solid line is a line rotated clockwise around the midpoint of the electron line 301 by the rotation function 164.

As the method of displaying the electronic line described below, the pixel value indicating the electronic line is superimposed (for example, added) on the pixel value of the image at the pixel position of the screen of the monitor 17 on which the electronic line should be displayed, and the image is transparent. It is assumed that the display is as follows. It should be noted that the image may not be displayed at the pixel position of the screen displaying the electronic line, and only the pixel value indicating the electronic line, for example, if the electronic line is white, the white pixel value may be displayed.

Further, in the present embodiment, the electron line can be arbitrarily moved and rotated. For example, using a GUI (Graphic User Interface), the user inputs a desired angle. By executing the electronic line control function 161 the image processing circuit 15 may be rotated around the midpoint of the electronic line displayed on the screen as a rotation axis so that the rotation angle corresponds to the angle desired by the user. ..

The rotation axis can be arbitrarily set on the electronic line. Further, by executing the electronic line control function 161 the image processing circuit 15 may freely move and rotate the electronic line in response to a user's drag operation of the electronic line on the GUI. For example, the operation reception circuit 14 may receive the drag operation, and the electronic line control function 161 may change the display position of the electronic line in real time according to the instruction signal based on the drag operation.

By making the electronic line rotatable, for example, the electronic line can be rotated by the user's operation or automatically with respect to the subject tilted with respect to the screen of the monitor 17, and the electronic line can be adjusted to the tilt of the subject. In the case of automatic control, the image processing circuit 15 may extract the outer shape of the tilted subject by a general image recognition process and rotate the electronic line so as to circumscribe it substantially parallel to the outer shape of the extracted subject. ..

As a result, the image processing circuit 15 measures the inclination of the subject from the angle of the electronic line. If the inclination (angle) of the subject can be measured, the control circuit 16 receives the processing result angle from the image processing circuit 15 and controls an actuator (not shown) that drives the image pickup unit 11 with the angle as a reference. , The posture of the image pickup unit 11 (the posture of the camera) can also be adjusted.

Next, an example of the detection area specifying process will be described with reference to FIG.
In the example of FIG. 4, the image processing circuit 15 detects a part of the image specified by being surrounded by the four electronic lines 401 as the detection area 402 by executing the specific function 162. For example, by user input, the operation reception circuit 14 receives a user instruction to generate and move an electronic line so as to match the outer shape of the target area to be set as the detection area 402. By executing the electronic line control function 161 the image processing circuit 15 rotates and moves the four electronic lines 401, respectively, in response to a user instruction, so that the region may be surrounded. The target area can be processed in any shape. If the target area is a polygon, the polygon may be surrounded by a plurality of electron lines parallel to each side. If the target area has a shape including a curved line, the area including the target area may be surrounded by three or more electron lines.

The image processing circuit 15 that executes the specific function 162 is not limited to specifying the area surrounded by the electronic line 401 as the detection area 402, but also specifies the area partitioned by the electronic line 401 as the detection area 402. May be good.

Another example of the specific processing of the detection area by the specific function 162 will be described with reference to FIG.
For example, as shown in FIG. 5, the image processing circuit 15 that executes the specific function 162 may detect one region as the detection area 402 by dividing the image into two by one electronic line 401. .. That is, the image processing circuit 15 that executes the specific function 162 divides the image displayed on the monitor 17 into two by the electronic line 401, and detects the area surrounded by the contour of the monitor 17 and the electronic line 401 as the detection area 402. You may.

The default display position of the electronic line 401 or the display position specified by the user may be predetermined, and the area surrounded or partitioned at the stage where the electronic line 401 is displayed may be specified as the detection area 402. For example, it is set to display one electronic line 401 as shown in FIG. In this case, if the area on the left side of the area partitioned by the electronic line 401 is set in advance as the detection area 402, the detection area 402 is specified together with the display of the electronic line 401.

Next, an example of the cutting process of the detection area 402 will be described with reference to FIG.
As shown in FIG. 6, by executing the cutout function 163, the image processing circuit 15 cuts out the partial image belonging to the detection area 402 as the output partial image 601. The image processing circuit 15 outputs the output partial image 601 to the display unit so as to be displayed on the screen of the monitor 17.

The image sensor 115 may execute partial readout so that the image sensor 115 outputs only the output partial image 601.

For example, the image processing circuit 15 sends the pixel position information of the image sensor 115 corresponding to the detection area 402 to the control circuit 16. The image pickup unit 11 receives the pixel position information from the control circuit 16 and performs a partial readout process of outputting only the electric signal from the pixel corresponding to the pixel position information by the image sensor 115. The partial readout process in the image sensor 115 may be performed by using a general method, and the description thereof is omitted here. Further, the detection area may be further cut out from the partial image after the partial readout process by the image sensor 115.

By performing the partial readout process that outputs only the partial image belonging to the detection area 402, the amount of data to be output is smaller than that in the case of outputting the data of the entire image from the image sensor 115. For example, when performing a partial readout process for outputting a partial image corresponding to a 1/4 area of the entire image, the amount of data of the partial image per frame is 1/4 of the entire image. The amount of data that can output one frame of the entire image corresponds to the amount of data that can output four frames of the partial image. Therefore, the transfer rate (frame rate) that can be output can be improved.

Next, an example of the rotation process of the detection area 402 will be described with reference to FIG. 7.
By executing the rotation function 164, the image processing circuit 15 rotates the partial image belonging to the detection area 402 together with the electron line 401 around the center of gravity 403 of the partial image belonging to the detection area 402. A display example before rotation is shown on the left side of FIG. 7, and a display example after rotation is shown on the right side of FIG. 7. As a specific rotation process, for example, when the user designates an electronic line on the screen of the monitor 17, the image processing circuit 15 changes the color of the designated electronic line. By executing the rotation function 164, the image processing circuit 15 may rotate the color-changed electronic line horizontally. The electronic line specified by the user may be changed so that the color, thickness, type (for example, solid line, broken line) and the like can be distinguished from other electronic lines.

Further, by selecting in advance whether to make the electron line vertical or horizontal and then designating the electron line, the rotation process may be performed along the designated electron line.

By making the image rotatable in this way, it is possible to easily visually recognize the subject without having to rotate the image pickup unit (camera). It should be noted that the black level 701 (luminance value zero) may be displayed on the screen of the monitor 17 in the portion where there is no image.

Further, in the rotation process by the rotation function 164, it is not necessary to rotate only the partial image of the detection area 402 and not the image other than the detection area 402.

Next, an example of the size change process of the detection area 402 will be described with reference to FIG.
By executing the size change function 165, the image processing circuit 15 changes the image size of the output partial image 601 with respect to the cut out output partial image 601. In the example of FIG. 8, as shown in the right figure of FIG. 8, the output partial image 601 is enlarged. In the example of FIG. 8, prior to the image resizing process, the partial image of the detection area 402 is cut out and rotated.

This makes it possible to enlarge the partial image, which is the area of interest, and make it easier to see. When the output partial image 601 is enlarged, the smaller of the vertical and horizontal sizes of the partial image may be enlarged so as to be the maximum size of the screen, and the other portion whose aspect ratio does not match may be displayed at the black level 701. .. The image size may be changed so as to reduce the output partial image 601. Further, only the size change process may be performed without performing the cutout process and the rotation process of the detection area.

Further, in the example of the above-described embodiment, the case of one output to the monitor is shown, but two or more outputs may be used. One output may be subjected to signal processing and image processing using the above-mentioned electronic line for confirmation, and the other may not display the electronic line for recording.

The above-mentioned electronic line is generated by the display control circuit 13 and superimposed on the image, but an electronic line generation circuit is separately provided as a hardware configuration, and the electronic line signal (electronic line signal) generated by the electronic line generation circuit is used. It may be superimposed on the image signal generated by the image generation circuit 12.
The rotation display of the electronic line when the hardware configuration is taken into consideration will be described.

The resolution of the monitor screen is set to "horizontal 1980 x vertical 1080". The electron line is assumed to be in the state of the electron line 302 shown in FIG.
As a method of displaying the electronic line, the input signal to the monitor may be switched between the image signal from the image generation circuit 12 and the electronic line signal from the electronic line generation circuit according to the following rules.

For example, when the vertical line number (line No.) in which 1080 pixels are lined up and the counter value H shown in the following equation (1) match, an electronic line signal is output, and in other cases, an image signal is output. Switch to output to the monitor.

Counter value H = A- {(AB) / 1080 × n} ... (1)
In the formula (1), A is a display position of the end point of the electronic line 302 in the horizontal direction when the line number is "1" (that is, the top of the screen). B is the display position of the start point of the electronic line 302 in the horizontal direction when the line number is "1080" (that is, at the bottom of the screen). n is a line number in the vertical direction, and here, a value from 1 to 1080 is taken.

When the electronic line is displayed diagonally with respect to the screen, a division circuit is required for the calculation of the counter value H, so that the hardware scale becomes large. However, by calculating the counter value H in a circuit with high computing power (for example, CPU or MPU (Micro Processing Unit)) and referring to the counter value H calculated by other hardware, the overall circuit scale Can be made smaller.

According to the present embodiment shown above, the electron line can be freely operated by moving and rotating the electron line, and the area divided by the electron line is detected as a detection area. The detection area is used for arbitrary processing such as brightness control, white balance and autofocus, detection area cutting processing, rotation processing, and resizing processing. From the user's point of view, just by setting the electronic line for observing the subject in the image, brightness control, white balance and autofocus, detection area cutout processing, rotation processing, resizing processing, etc. are optional. The image area for processing can also be set at the same time. Therefore, it is possible to improve the convenience of operation and the like as compared with the case where the image area is selected and operated in another menu for the arbitrary processing separately from the setting of the electronic line.

Further, each of the above-mentioned circuits includes an application specific integrated circuit (ASIC) incorporating these dedicated hardware circuits, a field programmable logic device (FPGA), and other composites. It may be realized by a programmable logic device (Complex Programmable Logic Device: CPLD) or a simple programmable logic device (Simple Programmable Logic Device: SPLD).
Further, the processing related to each circuit may be integrated into the CPU, and the CPU may execute each processing.

Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other embodiments, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the scope of the invention described in the claims and the equivalent scope thereof.

1 ... Image pickup device 11 ... Image pickup unit 12 ... Image generation circuit 13 ... Display control circuit 14 ... Operation reception circuit 15 ... Image processing circuit 16 ... Control circuit 17 ... Monitor 18 ... USB output circuit 111 ... Lens 113 ... Shutter 115 ... Image sensor 161 ... Electronic line control function 162 ... Specific function 163 ... Cutting function 164 ... Rotation function 165 ... Size change function 166 ... Analysis function 301, 302, 401 ... Electronic line 402 ... Detection area 403 ... Center of gravity 601 ... Output part image 701 ... Black level

Claims (5)

An image sensor that is arranged in a matrix and contains multiple pixels that generate electrical signals when it receives light.
An image generator that generates an image based on the electrical signal,
A display control unit that displays the image on the display unit and displays an electronic line on the displayed image.
A specific part that identifies a part of the image based on the display position of the electronic line, and
The image sensor outputs an electric signal only from the pixels corresponding to a part of the specified image by the partial readout process.
An electronic line control unit that changes the display position of the electronic line is further provided.
The electronic line control unit changes the angle of the electronic line according to the angle of the subject.
An image pickup apparatus further provided with a control unit for controlling the angle of the image pickup unit including the image sensor according to the angle . An image sensor that is arranged in a matrix and contains multiple pixels that generate electrical signals when it receives light.
An image generator that generates an image based on the electrical signal,
A display control unit that displays the image on the display unit and displays an electronic line on the displayed image.
A specific part that identifies a part of the image based on the display position of the electronic line, and
An analysis unit that determines control parameters related to brightness, white balance, or focus by analyzing a part of the identified image.
Equipped with
An electronic line control unit that changes the display position of the electronic line is further provided.
The electronic line control unit changes the angle of the electronic line according to the angle of the subject.
An image pickup apparatus further provided with a control unit that controls the angle of the image pickup unit including the image sensor according to the angle . The image pickup apparatus according to claim 1 or 2 , further comprising a cutout portion for cutting out a part of the specified image. The image pickup apparatus according to any one of claims 1 to 3 , further comprising a rotating portion for rotating the image around a part of the specified image. The image pickup apparatus according to any one of claims 1 to 4 , further comprising a changing unit for changing the size of a part of the specified image to be displayed on the display unit.

Images