KR102609415B1 - Image view system and operating method thereof - Google Patents

Abstract

One embodiment of the imaging system includes a plurality of cameras with the same view angle and resolution; a control unit that communicates with the camera and controls the operation of the camera; And it may include a display unit that is electrically connected to the control unit and plays an image transmitted from the control unit.

Description

Image view system and operating method thereof}

Embodiments relate to an imaging system including a camera and a method of operating the same.

The content described in this section simply provides background information on the embodiments and does not constitute prior art.

Vehicles can be equipped with cameras for various purposes. For example, a camera that can secure the side and front view of the vehicle required while the vehicle is driving may be mounted on the front and side portions of the vehicle. Additionally, a camera module that can secure a rear view when parking the vehicle may be mounted at the rear of the vehicle.

Images reproduced from all of these cameras can be combined to create an around view image.

In addition, a camera module can also be used in the case of a vehicle black box, which is very useful in tracking the details of the accident and the cause of the accident in recent years when a traffic accident occurs. In addition, there is a gradual increase in the use of cameras as recognition devices to clearly and easily identify blind spot situations that are difficult for the driver or passenger of the vehicle to see with the naked eye.

Recently, so-called smart cars, that is, collision warning systems that detect and prepare for the possibility of front and rear collisions in advance when driving a vehicle, prevent collisions between driving vehicles by a control device mounted on the vehicle, without relying on the driver to operate the control device. The production of vehicles equipped with collision avoidance systems that can directly avoid collisions is increasing, and the development of related technologies is increasing.

As the use of cameras as a means of recognizing external situations in smart cars is increasing, the production and technology development of vehicle cameras are also increasing accordingly.

Embodiments relate to an imaging system including a camera and a method of operating the same.

The technical problems to be achieved by the embodiment are not limited to the technical problems mentioned above, and other technical problems not mentioned can be clearly understood by those skilled in the art from the description below.

One embodiment of the imaging system includes a plurality of cameras with the same view angle and resolution; a control unit that communicates with the camera and controls the operation of the camera; And it may include a display unit that is electrically connected to the control unit and plays an image transmitted from the control unit.

Each of the plurality of cameras includes an image sensor for forming an external image; an image signal processor that is electrically connected to the image sensor and processes the image received from the image sensor; a communication unit electrically connected to the image signal processor and communicating with the control unit; And it may include a memory that is electrically connected to the image signal processor and provides information about image processing to the image signal processor.

Each of the plurality of cameras is mounted on a vehicle to photograph objects located in front, sides, and rear of the vehicle, and the image signal processor mounted on each camera has the number of pixels and image distortion set by the memory. ) may be provided with information about.

The image signal processor may control the number of pixels of an image formed on the image sensor.

The image signal processor may receive a portion of the image of the surrounding view of the vehicle from the image sensor.

The memory may store the set number of pixels and image distortion of the camera, and store at least one of the shooting direction, gain, brightness, and contrast of the camera.

The control unit may include a main processor, and the main processor may select or combine each image transmitted from the image signal processor of each of the plurality of cameras and transmit the selected or combined image to the display unit.

When combining each image transmitted from the image signal processor, the main processor may transmit a combined image having the same resolution as the lowest resolution image among each image to the display unit.

The image signal processor may transmit an image at a rate corresponding to the display unit to the control unit.

Another embodiment of the imaging system includes a plurality of cameras with the same angle of view and resolution; a control unit that communicates with the camera and controls the operation of the camera; and a display unit electrically connected to the control unit and reproducing an image transmitted from the control unit, wherein each of the plurality of cameras includes: an image sensor for forming an external image; an image signal processor that is electrically connected to the image sensor and processes the image received from the image sensor; a communication unit electrically connected to the image signal processor and communicating with the control unit; and a memory electrically connected to the image signal processor and providing information about image processing to the image signal processor, wherein the image signal processor controls the number of pixels of the image formed on the image sensor. You can.

One embodiment of a method of operating an imaging system includes an image forming step of forming an image on an image sensor provided in each of a plurality of cameras having the same angle of view and resolution; An image processing step in which an image signal processor processes the image by adjusting the number of pixels of the image received from the image sensor; An image processing step in which a control unit receives a processed image and selects or combines the processed image; and a display step in which the selected or combined image is reproduced on a display unit, and the image processing step may include receiving information about the set number of pixels from a memory electrically connected to the image signal processor.

In the embodiment, the same type of camera with the same angle of view and resolution is used for the front, side, and rear cameras, so that the same application can be used for each camera, thereby simplifying the role of the control unit, suppressing overload in the control unit, and improving price competitiveness. It can be raised.

In an embodiment, by pre-processing the image played on the display unit in the image signal processor, the role and configuration of the control unit including the main processor can be simplified, and the occurrence of overload in the control unit can be suppressed.

1 is a diagram showing an imaging system of one embodiment.
Figure 2 is a diagram showing the configuration of a camera according to an embodiment.
FIG. 3 is a diagram illustrating a method of adjusting the number of pixels of an image formed on an image sensor in an imaging system according to an embodiment of the present invention.
Figure 4 is a flowchart showing a method of operating an imaging system in one embodiment.

Hereinafter, the embodiment will be described in detail with reference to the attached drawings. Since the embodiments can be subject to various changes and have various forms, specific embodiments will be illustrated in the drawings and described in detail in the text. However, this is not intended to limit the embodiment to a specific disclosed form, and should be understood to include all changes, equivalents, and substitutes included in the spirit and technical scope of the embodiment.

Terms such as “first”, “second”, etc. may be used to describe various components, but the components should not be limited by the terms. The above terms are used for the purpose of distinguishing one component from another component. Additionally, terms specifically defined in consideration of the configuration and operation of the embodiment are only for explaining the embodiment and do not limit the scope of the embodiment.

In the description of the embodiment, in the case where each element is described as being formed "on or under", ) includes two elements that are in direct contact with each other or one or more other elements that are formed (indirectly) between the two elements. Additionally, when expressed as “up” or “on or under,” it can include not only the upward direction but also the downward direction based on one element.

In addition, relational terms such as "top/top/top" and "bottom/bottom/bottom" used below do not necessarily require or imply any physical or logical relationship or order between such entities or elements, It may be used to distinguish one entity or element from another entity or element.

1 is a diagram showing an imaging system of one embodiment. The imaging system of the embodiment may be used in a vehicle, for example, and may include a camera 100, a control unit 200, and a display unit 300.

A plurality of cameras 100 may be provided, and each of the plurality of cameras 100 may be mounted on a vehicle to view objects located in the front view, side view, and rear view of the vehicle, respectively. You can shoot.

In Figure 1, for example, a total of four cameras 100 are shown, including two cameras 100 mounted on the front and rear sides of the vehicle and two cameras 100 mounted on both sides of the vehicle, respectively. It is not limited to this, and the number of cameras 100 may be five or more.

The plurality of cameras 100 may be provided as cameras 100 of the same type. That is, the plurality of cameras 100 may be equipped with the same type of camera 100 having the same view angle and resolution.

Conventionally, front, side, and rear cameras mounted on vehicles each consisted of cameras with different viewing angles. For example, the front and rear cameras had a view angle of about 130° to 140°, and the side camera had a view angle of about 70°.

Additionally, when a surrounding view camera is installed separately in a vehicle, a surrounding view camera having a viewing angle of approximately 180° or more is used.

Therefore, the surrounding cameras installed separately from the front, side, and rear cameras mounted on conventional vehicles used different types of cameras with different viewing angles, and because they were different types of cameras, the resolutions of each camera were different.

In addition, since it is a heterogeneous camera, different applications, which are software that controls the operation of these cameras, are also used.

Ultimately, when multiple cameras of different types are installed and each camera is controlled using a different application, the control unit 200 that controls the cameras as a whole is configured in a complicated manner, and the vehicle's ECU (electronic control unit) is connected to the control unit ( 200), there was a problem of overloading the ECU.

Therefore, in the embodiment, the same type of camera 100 with the same angle of view and resolution is used for the front, side, and rear cameras 100, so that the same application can be used for each camera 100, thereby playing the role of the control unit 200. can simplify, suppress overload in the control unit 200, and increase price competitiveness.

The control unit 200 can communicate with the camera 100 and control the operation of the camera 100. At this time, the control unit 200 and the camera 100 communicate by wire or wirelessly, and the control unit 200 transmits a control signal for operation to the camera 100, and sends a plurality of images from each camera 100. signals can be received.

The control unit 200 may be, for example, an ECU mounted on a vehicle. Accordingly, the control unit 200 can control various systems necessary for vehicle operation and, in addition, can process images received from the camera 100.

The control unit 200 may include a main processor 210 and may communicate with an image signal processor 120 (see FIG. 2).

The main processor 210 selects or combines each image transmitted from the image signal processor 120 of each of the plurality of cameras 100 and transmits the selected or combined image to the display unit 300. there is.

For example, if the driver of the vehicle wants to view any one image of the external environment of the front, side, and rear of the vehicle on the display unit 300, the main processor 210 selects one of the images and It can be transmitted to the display unit 300.

In addition, when the driver of the vehicle wants to view images combining two or more of the external environments of the front, side, and rear of the vehicle on the display unit 300, the main processor 210 displays the combination of each image desired by the driver. One image can be transmitted to the display unit 300.

In addition, when the driver of the vehicle wants to view an around view image combining the external environment of the vehicle, the main processor 210 uses a plurality of cameras 100 mounted on the front, side, and rear sides of the vehicle. One image in which all images taken from can be combined to be transmitted to the display unit 300.

Meanwhile, when the main processor 210 combines each image transmitted from the image signal processor 120, it sends the combined image with the same resolution as the lowest resolution image among each image to the display unit 300. Can be transmitted.

As will be described later, at least some of the images transmitted from the image signal processor 120 may have different numbers of pixels, and the resolution of the images may vary depending on the number of pixels.

Accordingly, the main processor 210 transmits a combined image having the same resolution as the lowest resolution image among images with different resolutions transmitted from the image signal processor 120 to the display unit 300 so that the driver can see an overall uniform resolution. You can view the combined images you have.

The display unit 300 is electrically connected to the control unit 200 and can play images transmitted from the control unit 200. The display unit 300 is equipped with a built-in type, head-up display device, etc. inside the vehicle so that the driver of the vehicle can conveniently view the images played on the display unit 300. It can be.

Figure 2 is a diagram showing the configuration of the camera 100 of one embodiment. As shown in FIG. 2, the camera 100 may include an image sensor 110, an image signal processor 120, a communication unit 130, and a main processor 210. At this time, the image sensor 110, image signal processor 120, communication unit 130, and main processor 210 may be provided in each of the plurality of cameras 100.

The image sensor 110 may form an external image by receiving light that has passed through a lens. The image formed on the image sensor 110 may be transmitted to an image processor for processing.

The image signal processor 120 is electrically connected to the image sensor 110 and can process the image received from the image sensor 110.

The communication unit 130 is electrically connected to the image signal processor 120 and can communicate with the control unit 200 by wire or wirelessly. Through the communication unit 130, the image signal processor 120 and the control unit 200 can transmit and receive information to each other.

The memory 140 is electrically connected to the image signal processor 120 and can provide information about image processing to the image signal processor 120.

The memory 140 stores the set number of pixels and image distortion of the camera 100, and includes the shooting direction, gain, brightness, and contrast of the camera 100. You can save at least one.

Additionally, the memory 140 may store information regarding auto exposure, AWB (auto white balance), HDR (high dynamic range), etc. of the camera 100, and the above information may be stored in a look-up table. It may be stored in the memory 140 in the form of a table).

If necessary, the image signal processor 120 can receive each piece of information stored in the memory 140 and use it for image processing.

In an embodiment, the image signal processor 120 may receive information about the set number of pixels and image distortion from the memory 140. At this time, since the image signal processor 120 and the memory 140 are provided in each of the plurality of cameras 100, the set number of pixels and image distortion provided to each image signal processor 120 may be different.

The plurality of cameras 100 each serve as the front, side, and rear cameras 100 of the vehicle, and therefore the mounting positions of each camera 100 in the vehicle are different, so the images captured by each camera 100 The degree of image distortion may be different.

Therefore, each image signal processor 120 receives information about the image distortion degree set differently according to each camera 100 from each memory 140, corrects the distortion of the image, and generates the distortion-corrected image. can be transmitted to the control unit 200.

The image signal processor 120 may receive a portion of the surrounding view image of the vehicle from the image sensor 110.

That is, images of the front, side, and rear sides of the vehicle are transmitted from a plurality of cameras 100, respectively, and the transmitted images are combined in the main processor 210 to display images in all directions of the vehicle, that is, the front and side sides of the vehicle. and the rear object may be converted into one integrated surrounding image and displayed on the display unit 300.

FIG. 3 is a diagram illustrating a method of adjusting the number of pixels of an image formed on the image sensor 110 in an imaging system according to an embodiment.

The image signal processor 120 can control the number of pixels of the image formed on the image sensor 110.

In FIG. 3, A×B and C×D represent the number of pixels of the image formed on the image sensor 110. That is, A×B has A horizontally and B vertically, and C×D has C horizontally and D pixels vertically.

A×B represents the maximum number of pixels of each camera 100, that is, the maximum number of pixels. Since the A and B values are the same for each camera 100, the maximum number of pixels is also the same for each camera 100. Accordingly, each camera 100 in the embodiment has the same angle of view and resolution.

The C×D value may vary depending on each camera 100. The image signal processor 120 may receive differently set C×D values for each camera 100 from the memory 140 and transmit an image corresponding to the set C×D value to the main processor 210. At this time, the C×D value may be selected based on, for example, the center of the image formed on the image sensor 110.

At this time, when displaying a wide-angle image, for example, a surrounding view image, some or all of the images taken by each of the plurality of cameras 100 are stored in A × B state, which is the maximum number of pixels, and are processed by the main processor ( 210).

Conversely, when displaying a narrow-angle image, for example, an image captured by a specific camera 100 among a plurality of cameras 100, the number of pixels of the image captured by the specific camera 100 in the image signal processor 120 is It can be processed in A×B state and transmitted to the main processor 210.

The main processor 210 receives images processed by the image signal processor 120 in the manner described above, selects or combines these images, and transmits them to the display unit 300, and the display unit 300 receives the images processed by the image signal processor 120. You can play images received from ).

In an embodiment, the role and configuration of the control unit 200 including the main processor 210 can be simplified by pre-processing the image played on the display unit 300 in the image signal processor 120, and the control unit 200 ) can suppress the occurrence of overload.

Meanwhile, the image signal processor 120 may serve to transmit an image with a ratio corresponding to that of the display unit 300 to the control unit 200. That is, the role of the control unit 200 can be reduced by the image signal processor 120 expanding the size of the image at a rate corresponding to the screen size of the display unit 300 and transmitting it. Accordingly, the role of the control unit 200 can be reduced. The occurrence of overload can be suppressed.

When a vehicle is parked or driving at a low speed because there are many obstacles around it, the imaging system needs to implement and reproduce the wide-angle image to eliminate as many dead zones around the vehicle as possible.

Meanwhile, when the vehicle is traveling at high speed, it is necessary to implement and reproduce the narrow-angle image so that the driver can easily recognize the situation in a specific area on the front side of the vehicle, and if necessary, zoom up the lens mounted on the camera 100. -up) function can be used to help the driver operate the vehicle.

Figure 4 is a flowchart showing a method of operating an imaging system in one embodiment. The imaging system operating method of the embodiment may include an image forming step (S110), an image processing step (S120), an image processing step (S130), and a display step (S140).

In the image forming step (S110), an image may be formed on the image sensor 110 provided in each of the plurality of cameras 100 having the same angle of view and resolution.

In the image processing step (S120), the image signal processor 120 may process the image by adjusting the number of pixels of the image received from the image sensor 110.

At this time, as described above, the image signal processor 120 receives the number of pixels and image distortion set for each camera 100 provided from the memory 140 electrically connected thereto and adjusts the number of pixels in the image. And image distortion can be corrected.

In the image processing step (S130), the control unit 200 receives the processed image from the image signal processor 120 and selects or combines the processed image. The main processor 210 provided in the control unit 200 can select or combine these images.

At this time, when a specific area of the external object of the vehicle is played on the display unit 300, some of the processed images are selected. For example, when a surrounding view image is played, the processed images are combined. It can be.

In the display step (S140), the image selected or combined by the main processor 210 may be reproduced on the display unit 300.

Although only a few examples have been described as described above in relation to the embodiments, various other forms of implementation are possible. The technical contents of the above-described embodiments can be combined in various forms unless they are incompatible technologies, and through this, can be implemented as a new embodiment.

100: Camera
110: image sensor
120: Image signal processor
130: Department of Communications
140: memory
200: control unit
210: main processor
300: Display unit

Claims (11)

A plurality of cameras with the same view angle and resolution;
a control unit that communicates with the camera and controls the operation of the camera; and
A display unit electrically connected to the control unit and playing an image transmitted from the control unit,
Each of the plurality of cameras,
An image sensor that forms an external image;
an image signal processor that is electrically connected to the image sensor, receives a portion of the surrounding view image of the vehicle from the image sensor, processes it into an image with a ratio corresponding to the display unit, and transmits it to the control unit;
a communication unit electrically connected to the image signal processor and communicating with the control unit; and
An imaging system electrically connected to the image signal processor and including a memory that provides information about image processing to the image signal processor. delete According to paragraph 1,
Each of the plurality of cameras is mounted on a vehicle to photograph objects located in front, sides, and rear of the vehicle, respectively,
The image signal processor mounted on each of the cameras receives information on the number of pixels and image distortion set for each from the memory. According to paragraph 3,
The image signal processor,
An imaging system that adjusts the number of pixels of an image formed on the image sensor. delete According to paragraph 1,
The memory is,
Stores the set number of pixels and image distortion of the camera,
An imaging system that stores at least one of the camera's shooting direction, gain, brightness, and contrast. According to paragraph 1,
The control unit includes a main processor,
The main processor selects or combines each image transmitted from the image signal processor of each of the plurality of cameras, and transmits the selected or combined image to the display unit. In clause 7,
The main processor is,
An imaging system that, when combining each image transmitted from the image signal processor, transmits the combined image with the same resolution as the lowest resolution image among each image to the display unit. delete delete An image forming step in which an image is formed on an image sensor provided in each of a plurality of cameras having the same angle of view and resolution;
An image processing step in which an image signal processor adjusts the number of pixels of the image of the surrounding view of the vehicle received from the image sensor to process the image at a ratio corresponding to the display unit;
An image processing step in which a control unit receives a processed image and selects or combines the processed image; and
A display step in which the selected or combined image is reproduced on the display unit,
The image processing step is,
A method of operating an imaging system in which information on the number of pixels set is provided from a memory electrically connected to the image signal processor.

Images