KR101650810B1 - Car black box including fish-eye lens and car black box system including the same - Google Patents

Car black box including fish-eye lens and car black box system including the same Download PDF

Info

Publication number
KR101650810B1
KR101650810B1 KR1020150056787A KR20150056787A KR101650810B1 KR 101650810 B1 KR101650810 B1 KR 101650810B1 KR 1020150056787 A KR1020150056787 A KR 1020150056787A KR 20150056787 A KR20150056787 A KR 20150056787A KR 101650810 B1 KR101650810 B1 KR 101650810B1
Authority
KR
South Korea
Prior art keywords
image
roi
rendered
extracted
hemispherical
Prior art date
Application number
KR1020150056787A
Other languages
Korean (ko)
Inventor
조성열
Original Assignee
(주)유디웍스
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by (주)유디웍스 filed Critical (주)유디웍스
Application granted granted Critical
Publication of KR101650810B1 publication Critical patent/KR101650810B1/en

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B62—LAND VEHICLES FOR TRAVELLING OTHERWISE THAN ON RAILS
    • B62D—MOTOR VEHICLES; TRAILERS
    • B62D41/00—Fittings for identifying vehicles in case of collision; Fittings for marking or recording collision areas
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/02—Rear-view mirror arrangements
    • B60R1/04—Rear-view mirror arrangements mounted inside vehicle
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01D—MEASURING NOT SPECIALLY ADAPTED FOR A SPECIFIC VARIABLE; ARRANGEMENTS FOR MEASURING TWO OR MORE VARIABLES NOT COVERED IN A SINGLE OTHER SUBCLASS; TARIFF METERING APPARATUS; MEASURING OR TESTING NOT OTHERWISE PROVIDED FOR
    • G01D9/00—Recording measured values
    • H04N5/225—
    • H04N2005/225—

Abstract

According to an embodiment of the present invention, a car black box comprises: a camera module generating a hemispherical image by using one fish-type lens; and an image signal processor generating a panorama image rendered from the hemispherical image. The image signal processor may comprise: a region of interest (ROI) extraction module extracting ROI from the hemispherical image by responding to a ROI selection signal; and an image treatment module rendering the extracted ROI and generating the rendered panorama image.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a black box for a vehicle including a fisheye lens,

An embodiment according to the concept of the present invention relates to a vehicle black box, and more particularly to an image signal processor capable of generating a rendered panoramic image from a hemispherical image output from a fisheye lens and devices including the same.

The use of automotive black boxes is increasing to prevent disputes over car accidents. The conventional vehicle black box has a problem in that it is not possible to photograph the side or rear side of the camera because the front side of the camera can be photographed and stored. Further, in order to photograph the side or rear of the vehicle, additional cameras must be installed on the side or rear of the vehicle, so that the installation cost of additional cameras may increase.

Japanese Patent Application Laid-Open No. 10-2013-0064169 (June 17, 2013)

SUMMARY OF THE INVENTION It is an object of the present invention to provide an image signal processor capable of generating a panoramic image rendered from a hemispherical image output from a fisheye lens and an apparatus including the same.

A vehicle black box according to an embodiment of the present invention includes a camera module for generating a hemispherical image using one fisheye lens and an image signal processor for generating a panorama image rendered from the hemispherical image, An ROI extraction module for extracting an ROI from the hemispherical image in response to a region of interest (ROI) selection signal, and an image processing module for rendering the extracted ROI and generating the rendered panorama image.

The image processing module may convert the circumferential direction of the extracted ROI into the horizontal direction of the rendered panorama image and convert the radial direction of the extracted ROI into the vertical direction of the rendered panorama image.

The image processing module may determine a vertical resolution of the rendered panorama image based on the vertical resolution, the difference between the maximum altitude and the minimum altitude of the extracted ROI, and the reference altitude.

The image processing module generates the rendered panorama image from the extracted ROI through the rendering that maps the coordinates of each pixel of the extracted ROI and the coordinates of each pixel of the rendered panorama image in a one- can do.

Wherein the image signal processor further comprises a memory for storing a mapping table, wherein the image processing module uses coordinates of the pixels of the extracted ROI and coordinates of pixels of the rendered panoramic image using the mapping table, To-one mapping.

Wherein the image processing module maps the pixel value of each pixel of the extracted ROI and the pixel value of each pixel of the rendered panorama image in a one-to-one mapping manner, and outputs the rendered panorama image from the extracted ROI Can be generated.

The image processing module may correct the rendered panorama image using a low-pass filter, and generate a corrected panorama image.

Wherein the vehicle black box further comprises a display for displaying the rendered panorama image generated by the image signal processor, wherein the image signal processor controls the display to simultaneously display the hemispherical image and the rendered panorama image .

In a vehicle black box system including a vehicle black box according to an embodiment of the present invention and a display for outputting images stored in the vehicle black box, the vehicle black box includes a camera for generating a hemispherical image using one fisheye lens, And an image signal processor for generating a panorama image rendered from the hemispherical image, the image signal processor comprising: an ROI extraction module for extracting an ROI from the hemispherical image in response to an ROI selection signal; And an image processing module for generating the rendered panorama image.

The image processing module may convert the circumferential direction of the extracted ROI into the horizontal direction of the rendered panorama image and convert the radial direction of the extracted ROI into the vertical direction of the rendered panorama image. The image signal processor may control the display to simultaneously display the hemispherical image and the rendered panoramic image.

According to an embodiment of the present invention, an image signal processor that generates a panoramic image rendered from the hemispherical image output from a camera module that generates a hemispherical image using one fisheye lens includes a region of interest (ROI) An ROI extraction module for extracting an ROI from the hemispherical image in response to a selection signal, and an image processing module for rendering the extracted ROI and generating the rendered panorama image, Direction to the horizontal direction of the rendered panorama image, and converts the radial direction of the extracted ROI into the vertical direction of the rendered panorama image.

The image processing module determines a vertical resolution of the rendered panoramic image based on the extracted vertical resolution, the difference between the maximum altitude and the minimum altitude, and the reference altitude.

The black box for a vehicle according to the embodiment of the present invention is capable of generating a rendered panoramic image from a hemispherical image output from a fish-eye lens.

The vehicle black box can generate a wide panoramic image from the hemispherical image output from the fish-eye lens, and can utilize the panoramic image as an around view image.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS In order to more fully understand the drawings recited in the detailed description of the present invention, a detailed description of each drawing is provided.
1 shows a schematic block diagram of a vehicle black box system according to an embodiment of the present invention.
2 is a block diagram illustrating a method of operating a black box for a vehicle according to an embodiment of the present invention.
3 is a block diagram illustrating a method of operating a black box for a vehicle according to another embodiment of the present invention.
Fig. 4 shows the wide angle of the fish-eye lens shown in Fig.
5 is a block diagram illustrating an operation of rendering a hemispherical image as a panoramic image according to an embodiment of the present invention.
6 is a block diagram illustrating an operation of rendering a hemispherical image into a panoramic image according to an embodiment of the present invention.
7 is a block diagram illustrating an operation of rendering a hemispherical image into a panoramic image according to an embodiment of the present invention.
8 shows a table stored in the memory shown in Fig.

It is to be understood that the specific structural or functional description of embodiments of the present invention disclosed herein is for illustrative purposes only and is not intended to limit the scope of the inventive concept But may be embodied in many different forms and is not limited to the embodiments set forth herein.

The embodiments according to the concept of the present invention can make various changes and can take various forms, so that the embodiments are illustrated in the drawings and described in detail herein. It should be understood, however, that it is not intended to limit the embodiments according to the concepts of the present invention to the particular forms disclosed, but includes all modifications, equivalents, or alternatives falling within the spirit and scope of the invention.

The terms first, second, etc. may be used to describe various elements, but the elements should not be limited by the terms. The terms may be named for the purpose of distinguishing one element from another, for example, without departing from the scope of the right according to the concept of the present invention, the first element may be referred to as a second element, The component may also be referred to as a first component.

It is to be understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, . On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between. Other expressions that describe the relationship between components, such as "between" and "between" or "neighboring to" and "directly adjacent to" should be interpreted as well.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, the terms "comprises ", or" having ", and the like, specify that the presence of the features, numbers, steps, operations, elements, Should not be construed to preclude the presence or addition of one or more features, integers, steps, operations, components, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries are to be interpreted as having a meaning consistent with the meaning of the context in the relevant art and, unless explicitly defined herein, are to be interpreted as ideal or overly formal Do not.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings attached hereto.

In this specification, an application program (application or APP) transmits or receives a signal (or data) includes a wireless communication module (e.g., a transmitter and a receiver) implemented in a computing device in which the application program is executed, Means transmitting or receiving signals (or data) with a wireless communication device under the control of an application program or a processor.

As used herein, the term " module "may refer to a hardware component capable of performing functions and operations according to the respective names described herein, (E. G., A processor) having computer program code (or firmware) capable of performing certain functions and operations. In other words, a module may mean a functional and / or structural combination of hardware components for carrying out the technical idea of the present invention and / or software for driving the hardware components.

1 shows a schematic block diagram of a vehicle black box according to an embodiment of the present invention. Referring to FIG. 1, a vehicle black box 100 may include a camera module 110, an image signal processor 140, a bus 160, a second memory 170, and a display 180.

The vehicle black box 100 is installed in a vehicle and photographs a peripheral image of the vehicle using the camera module 110, processes a photographed image (for example, a moving image or a moving image stream) (170) or display it via the display (180). In this specification, an image may mean image data.

The camera module 110 can photograph a peripheral image of a vehicle in which the vehicle black box 100 is installed and generate a photographed image. The camera module 110 may include a lens 120 and an image sensor 130.

The lens 120 may transmit the optical image to the image sensor 130. [ The lens 120 may be, for example, a wide-angle lens or a fish-eye lens, but the technical idea of the present invention is not limited thereto. If the lens 120 is a fisheye lens, the image produced by the lens 120 may be a hemispheric image. The wide angle of the fisheye lens may be 180 degrees or 240 degrees, but the wide angle of the fisheye lens according to the embodiment of the present invention is not limited to the above example.

The image sensor 130 may convert an image distorted by the lens 120 into an image signal HI which is an electric signal and output the image signal HI to the image signal processor 140.

2, the image sensor 130 receives the optical image f output from the fisheye lens 120 and processes the image f to generate a hemispherical image HI. For example, the image sensor 130 may be a complementary metal-oxide-semiconductor (CMOS) image sensor or a charge-coupled device (CCD) image sensor.

The image signal processor 140 may receive and process the image output from the camera module 110 and store the processed image on the bus 160 to the second memory 170 or to the display 180. The image signal processor 140 may be implemented as an integrated circuit or a system on chip (SoC).

In accordance with an embodiment, the image signal processor 140 may generate a rendered panoramic image from the hemispherical image output from the image sensor 130. In the present specification, "rendering " may refer to a process or process of converting a hemispherical image into a rectangular image, and the rendering may be performed by converting a distorted hemispherical image into a rectangular panorama image, It can mean.

The image signal processor 140 receives instructions and / or data from each component 110, 170, and 180 via the bus 160 or through the bus 160 to each component 110, 170, and 180, Lt; RTI ID = 0.0 > and / or < / RTI > The image signal processor 140 includes an encoder 142, a decoder 144, a CPU 150, a region of interest (ROI) extraction module 152, an image processing module 154, and a first memory 156 can do.

The encoder 142 may encode the image received from the image sensor 130 and transmit the encoded image to the ROI extraction module 152 under the control of the CPU 150. [ Encoder 142 may also transmit the encoded image to memory 156 or 170. [

The decoder 144 can receive and decode the encoded image output to the memory 156 or 170 under the control of the CPU 150. [ The decoder 144 may send the decoded image to the ROI extraction module 152.

The CPU 150 may control the overall operation of the image signal processor 140. According to an embodiment, when an image is transmitted from the image sensor 130 to the ROI extraction module 152, the CPU 150 may transmit an ROI selection signal to the ROI extraction module 152. For example, the ROI selection signal may be generated by the CPU 150 based on the user input input from the user.

The ROI extraction module 152 may extract the ROI from the image received from the image sensor 130. The ROI extraction module 152 may extract the ROI from the image transmitted from the image sensor 130 in response to the ROI selection signal output from the CPU 150. [ The ROI extraction module 152 may transmit the extracted ROI to the image processing module 154.

The image processing module 154 may render the extracted ROI and generate the rendered image. In accordance with an embodiment, the image processing module 154 may render the extracted ROI and generate a rendered panorama image. As illustrated in FIGS. 5 and 6, the rendering may refer to a process of making a donut-shaped ROI into a rectangular shape. The image processing module 154 may transmit the rendered image to the second memory 170 or the display 180.

The first memory 156 may store a mapping table. The first memory 156 may transmit the mapping table to the image processing module 154 under the control of the CPU 150. [

For example, the first memory 156 may be implemented as volatile memory. The volatile memory may be implemented as a random access memory (RAM), a dynamic RAM (DRAM), or a static RAM (SRAM). The first memory 156 may be implemented as a non-volatile memory such as a read only memory (ROM) or a flash memory.

The second memory 170 may be implemented as volatile memory and / or nonvolatile memory. Although a single second memory 170 is illustrated by way of example in FIG. 1, the second memory 170 may refer to a collective memory including volatile memory and non-volatile memory.

At this time, the volatile memory may be implemented as a DRAM, and the non-volatile memory may be implemented as a NAND flash memory. The nonvolatile memory may store a boot image and a mapping table of the black box 100 for a vehicle. For example, when the vehicle black box 100 is booted, the boot image and the mapping table may be loaded into the memory 156 from the non-volatile memory. As another example, when the second memory 170 includes a DRAM and a NAND flash memory, the boot image and the mapping table may be loaded into the DRAM or memory 156 from the non-volatile memory.

The second memory 170 may store the processed data or data to be processed in the image signal processor 140. [ For example, the second memory 170 may include an embedded SSD (eSSD), a multimedia card (MMC), an embedded MMC (eMMC), or a universal flash storage (UFS)), but is not limited thereto. Also, the second memory 170 may be implemented as a secure digital (SD) card or a multimedia card (MMC).

The display 180 may display the image received from the image signal processor 140. For example, the display 180 may display the rendered panorama image transmitted from the image signal processor 140.

2 is a block diagram illustrating a method of operating a black box for a vehicle according to an embodiment of the present invention. Referring to Figs. 1 and 2, a vehicle black box 100-1 may include a lens 120, an image sensor 130, an image signal processor 140-1, and a display 180. Fig.

The optical image f generated by the lens 120 may be transmitted to the image sensor 130. [ Assume that the lens 120 is a fisheye lens.

The image sensor 130 may convert the optical image f to image data HI which is an electrical signal and output the image data HI to the image signal processor 140-1. According to the embodiment, the image sensor 130 may convert the optical image f into hemispherical image, that is, hemispherical image data HI. The image sensor 130 may transmit the hemispherical image HI to the image signal processor 140-1.

The image signal processor 140-1 may convert the hemispherical image HI, that is, the hemispherical image data HI, into the rendered panorama image PS, that is, the rendered panorama image data PS. The image signal processor 140-1 may transmit the rendered panoramic image PS to the display 180. [

The image signal processor 140-1 may include a CPU 150, an ROI extraction module 152, an image processing module 154, and a first memory 156.

The ROI extraction module 152 can extract the ROI (RS) from the hemispherical image HI according to the ROI selection signal CS transmitted from the CPU 150. [ For example, the extracted ROI (RS) may be a donut-shaped image, as exemplarily shown in FIGS. The ROI extraction module 152 may transmit the extracted ROI (RS) to the image processing module 154.

The image processing module 154 may render the extracted ROI (RS) and generate the rendered panoramic image (PS). According to an embodiment, the image processing module 154 receives the mapping table TB from the first memory 156, renders the extracted ROI (RS) using the mapping table TB, (PS).

When the image processing module 154 includes a low pass filter, the image processing module 154 corrects (e.g., filters) the rendered panorama image PS using the low pass filter, Filtered) panoramic image.

According to an embodiment, the image processing module 154 may transmit the rendered panoramic image (PS) to the display 180. According to another embodiment, the image processing module 154 may transmit the calibrated panoramic image to the display 180. The image processing module 154 may transmit the rendered panoramic image PS and the hemispherical image HI simultaneously (or in parallel) or sequentially to the display 180 under the control of the CPU 150.

The display 180 may display the rendered panorama image PS or the corrected panorama image. The display 180 may display the hemispherical image HI and the rendered panorama image PS simultaneously or in parallel.

3 is a block diagram illustrating a method of operating a black box for a vehicle according to another embodiment of the present invention. 1 to 3, a vehicle black box 100-2 includes a lens 120, an image sensor 130, an image signal processor 140-2, a second memory 170, and a display 180, . ≪ / RTI >

The optical image f generated by the lens 120 may be transmitted to the image sensor 130. [ Assume that the lens 120 is a fisheye lens.

The image sensor 130 converts the optical image f into image data HI which is an electrical signal and outputs the image data HI to the image signal processor 140-1. According to an embodiment, the image sensor 130 may convert the optical image f into a hemispherical image HI, i.e. hemispherical image data HI. The image sensor 130 may transmit the hemispherical image HI to the image signal processor 140-1.

The image signal processor 140-2 may receive and encode the hemispherical image HI1 and store the encoded hemispherical image ID or the encoded hemispherical image data ID in the second memory 170 .

The image signal processor 140-2 may read the encoded hemispherical image ID from the second memory 170 and convert the read encoded hemispherical image ID into a rendered panoramic image PS have. The image signal processor 140-2 may transmit the rendered panoramic image PS to the display 180. [

The image signal processor 140-2 may include an encoder 142, a decoder 144, a CPU 150, an ROI extraction module 152, an image processing module 154, and a first memory 156 .

The encoder 142 can encode the hemispherical image HI1 received from the image sensor 130 and generate an encoded hemispherical image ID according to the control of the CPU 150. [ The encoding used in the encoder 142 may be a joint picture expert group (JPEG), a motion picture expert groups (MPEG), MPEG-2, MPEG-4, VC-1, H.264, H.265, or HEVC High Efficiency Video Coding), but the present invention is not limited thereto. The encoder 142 may store the encoded hemispherical image (ID) in the second memory 170.

The decoder 144 reads the encoded hemispherical image ID from the second memory 170 under the control of the CPU 150, decodes the encoded hemispherical image ID, and outputs the decoded hemispherical image ID (HI2).

The ROI extraction module 152 can extract the ROI (RS) from the decoded hemispherical image HI2 in accordance with the ROI selection signal CS transmitted from the CPU 150. [ For example, the ROI (RS) may be a donut-shaped image. The ROI extraction module 152 may transmit the extracted ROI (RS) to the image processing module 154.

The image processing module 154 may render the extracted ROI (RS) and generate the rendered panoramic image (PS). According to an embodiment, the image processing module 154 receives the mapping table TB from the first memory 156, renders the extracted ROI (RS) using the mapping table TB, (PS). The image processing module 154 may transmit the rendered panoramic image PS to the display 180.

According to the embodiment, the image processing module 154 displays the rendered panoramic image PS and the decoded hemispherical image HI2 simultaneously, in parallel, or sequentially, according to the control of the CPU 150 ). The display 180 may display the rendered panoramic image PS.

According to an embodiment, the display 180 may display the decoded hemispherical image HI2 and the rendered panoramic image PS simultaneously or in parallel. For example, the decoded hemispherical image HI2 and the rendered panoramic image PS may be simultaneously displayed on one display 180. [ For example, the decoded hemispherical image HI2 and the rendered panoramic image PS may be displayed up or down or on the left and right.

Fig. 4 shows the wide angle of the fish-eye lens shown in Fig. Referring to FIGS. 1 to 4, the wide angle of the lens 120 may be 180 degrees or 240 degrees, but the technical idea of the present invention is not limited to the above-described optical. Although the fisheye lens 120 having a wide angle of 240 degrees is shown in FIG. 4, the technical idea of the present invention is not limited thereto.

In this specification, the resolution of the image sensor 130 is 1920 * 1080, and the resolution of the hemispherical image obtained through the fisheye lens 120 is 1500 * 1500. And the 0-degree position indicates the position indicating the front face of the fisheye lens 120. [ Therefore, the fisheye lens 120 should be arranged so that the zero-degree position is vertically upward.

When the 0 degree position is vertically oriented, the area 200 where the image can be formed using the fisheye lens 120 is divided into a region from 0 degrees to 120 degrees to the left and a region from 0 degrees to 120 degrees to the right . That is, the fisheye lens 120 can acquire an image for a region corresponding to the rear portion.

According to the embodiment, each of the ROIs 210 and 220 is divided into regions from the left 45 degrees (φmin = 45 degrees) region to the left 105 degrees (φmax = 105 degrees) and the right 45 (? Min = 45 degrees) to the right side of 105 degrees (? Max = 105 degrees). That is, the ROI (RS) extracted from the hemispherical image HI by the ROI extraction module 152 may be the ROI 210 and the ROI 220. Although the minimum altitude (min) and the maximum altitude (max) are limited in FIG. 4, the technical idea of the present invention is not limited thereto.

5 is a block diagram illustrating an operation of rendering a hemispherical image as a panoramic image according to an embodiment of the present invention. Referring to FIGS. 1 to 5, the ROI extraction module 152 may extract a donut-shaped ROI 320 from the hemispherical image 300.

In the hemispherical image 300, when the 0-degree position of the region 200 in which the image can be obtained in FIG. 4 is set as the origin of the hemispherical image 300, the region 200 in which the image can be acquired, And may be an area ranging from 0 degrees to 120 degrees of the image 300.

The ROI extraction module 152 may determine the ROI (RS or 320) based on the maximum altitude phi max and the minimum altitude phi min. The ROI extraction module 152 can determine the maximum altitude (? Max) and the minimum altitude (? Min) under the control of the CPU (150). For example, under the control of the CPU 150, the ROI extraction module 152 may determine the maximum altitude phi max at 105 degrees and the minimum altitude phi min at 45 degrees. That is, the ROI region 320 may be determined as an area corresponding to 45 to 105 degrees. For example, the maximum altitude phi max and the minimum altitude phi min may be determined based on the user input entered from the user. For example, the user input may be transmitted to the CPU 150 via the touch screen.

5, the image processing module 154 may convert the circumference direction of the extracted ROI 320 to the horizontal direction of the rendered panorama image 350. [ The image processing module 154 may convert the radial direction of the extracted ROI 320 to the vertical direction of the rendered panorama image 350.

The image processing module 154 may determine the horizontal resolution WR of the rendered panoramic image 350. [ For example, the image processing module 154 may determine the horizontal resolution WR of the rendered panoramic image 350 to be the same as the horizontal resolution of the image sensor 130. For example, the image processing module 154 may determine the horizontal resolution WR of the rendered panoramic image 350 as 1920. [

The image processing module 154 generates the rendered panoramic image PS or 350 based on the vertical resolution of the hemispherical image HI, the difference between the maximum altitude? Max1 and the minimum altitude? Min1, and the reference altitude? Ref, The vertical resolution (HS) of the image can be determined. For example, the image processing module 154 can determine the vertical resolution (HS) of the rendered panoramic image 350 using Equation (1).

[Equation 1]

HR = (HS / 2) * (? Max1 -? Min1) / (90 -? Ref)

Represents the vertical resolution of the rendered panorama image 350, "HS" represents the vertical resolution of the extracted ROI 320, "phi max1" is the maximum altitude, "phi min1" is the minimum altitude, represents the reference altitude.

For example, when the extracted ROI 320 has a vertical resolution (HS) of 1080, a maximum altitude (phi max1) of 45, a minimum altitude (phi min1) of -15 and a reference altitude (phi ref) of 10, The vertical resolution (HR) of the image 350 becomes 405.

The maximum altitude (? Max1) and minimum altitude (? Min1) can be increased or decreased by 90 degrees in the vertical direction when the horizontal height is 0 degree in the extracted ROI (320).

According to the embodiment, the maximum altitude phi max1 may be equal to phi min, and the minimum altitude phi min1 may be equal to (90-phi min). That is, if? Min = 45, the maximum altitude? Max1 is 45 degrees, and if? Max = 105, the minimum altitude? Min1 is -15 degrees. The reference altitude? Ref refers to a reference altitude corresponding to an upper or lower end of the extracted ROI 320.

The image processing module 154 may render the extracted ROI 320 and generate the rendered panorama image 350 using the determined horizontal and vertical resolutions WR and HR.

6 is a block diagram illustrating an operation of rendering a hemispherical image into a panoramic image according to an embodiment of the present invention. Referring to FIGS. 1 through 6, the image processing module 154 may render the extracted ROI 400 and generate a rendered panorama image 450.

6, it is assumed that the extracted ROI 400 includes a first area IM1, a second area IM2, a third area IM3, and a fourth area IM4, It is assumed that the image 450 includes a first area IM1 ', a second area IM2', a third area IM3 ', and a fourth area IM4'.

The image processing module 154 may convert the first region IM1 of the extracted ROI 400 to the first region IM1 'of the rendered panorama image 450, The second region IM2 of the extracted panoramic image 450 can be converted into the second region IM2 'of the rendered panoramic image 450 and the third region IM3 of the extracted ROI 400 can be converted into the second region IM2' 3 region IM3 'and convert the fourth region IM4 of the extracted ROI 400 into the fourth region IM4' of the rendered panorama image 450. [

The image processing module 154 may convert a part or all of the extracted ROI 400 into a rendered panorama image 450 under the control of the CPU 150. [ For example, the image processing module 154 may render only the second area IM2 of the extracted ROI 400 and convert it into the rendered second area IM2 '.

For example, the user can use a touch screen or an input device included in the black box 100 for a vehicle to input user input capable of selecting at least one region out of the regions IM1, IM2, IM3, and IM4 to the CPU 150 ). Accordingly, the CPU 150 outputs an ROI selection signal CS for selecting at least one region out of the regions IM1, IM2, IM3, and IM4 based on the user input to the ROI extraction module 152 and / Or to the image processing module 154.

Although the ROI 400 including the regions IM1, IM2, IM3, and IM4 is shown in FIG. 6, the number and shape of the regions included in the ROI 400 are different from those shown in FIG. 6 But are not limited to shapes.

7 is a block diagram illustrating an operation of rendering a hemispherical image into a panoramic image according to an embodiment of the present invention. Referring to FIGS. 1 to 7, the image processing module 154 may render the ROI 520 extracted from the hemispherical image 500 and generate the rendered panorama image 550.

The first step in the rendering according to the embodiment of the present invention is to set coordinates. The image processing module 154 may map the coordinates of each pixel included in the ROI 520 and the coordinates of each pixel included in the rendered panorama image 550 in a one-to-one correspondence.

7, it is assumed that the coordinates of the pixels of the ROI 520 are (xs, ys), and the coordinates of the pixels of the rendered panorama image 550 are (xr, yr).

According to an embodiment, the image processing module 154 maps the coordinates of each pixel of the ROI 520 and the coordinates of each pixel of the rendered panoramic image 550 one-to-one using Equation (2) .

&Quot; (2) "

xs = r * sin? + (WS / 2)

ys = r * cos? + (HS / 2)

Where WS represents the x coordinate of the pixel of the ROI 520 ys represents the y coordinate of the pixel of the ROI 520 WS represents the horizontal resolution of the ROI 520 and HS represents the vertical extent of the ROI 520 R represents the first variable, and &thetas; represents the second variable. In this case, the second variable? May mean an angle in the circumferential direction.

According to the embodiment, the image processing module 154 may calculate the first variable r, the second variable? And the third variable? Using Equation (3). In this case, the third parameter? May mean an angle in the radial direction.

&Quot; (3) "

&thetas; = 360 * xr / WR-180

φ = φmax1- (φmax1-φmin1) * yr / HR

r = (HS / 2) * (? max -?) / (? max -

Here, xr represents the x coordinate of the pixel of the rendered panoramic image 550, yr represents the y coordinate of the pixel of the rendered panoramic image 550, WR represents the horizontal resolution of the rendered panoramic image 550 , HR denotes the vertical resolution of the rendered panoramic image 550,? Max1 denotes the maximum altitude,? Min1 denotes the minimum altitude, and? Ref denotes the reference altitude.

The image processing module 154 can use the equations 2 and 3 to map the coordinates of each pixel of the ROI 520 and the coordinates of each pixel of the rendered panoramic image 550 in a one- have. The process of mapping the coordinates (xr, yr) of the pixels of the rendered panoramic image 550 and the coordinates (xs, xs) of the pixels of the ROI in a one-to-one manner includes a step of generating a luminance component and a chrominance component It can be applied to all. For example, the image processing module 154 may use the YUV420 format.

According to an embodiment, the ROI 520 may include a first coordinate xs1, ys1, a second coordinate xs2, ys2, and a third coordinate xs3, ys3. The panoramic image 550 may include a first coordinate xr1, yr1, a second coordinate xr2, yr2, and a third coordinate xr3, yr3.

The image processing module 154 may map the first coordinates xs1 and ys1 of the extracted ROI 520 to the first coordinates xr1 and yr1 of the rendered panorama image 550 and the extracted ROI 400 (Xs2, ys2) of the extracted ROI 520 to the second coordinates (xr2, yr2) of the rendered panorama image 450 and to render the third coordinates (xs3, ys3) of the extracted ROI 520 To the third coordinates (xr3, yr3) of the panoramic image 550.

The second step in the rendering according to the embodiment of the present invention is to perform a one-to-one mapping of the pixel value of each pixel of the ROI 520 and the pixel value of each pixel of the rendered panorama image 550 based on the set coordinates . That is, the image processing module 154 may map the pixel value of each pixel of the extracted ROI 520 and the pixel value of each pixel of the rendered panorama image one-to-one.

The image processing module 154 uses bi-linear interpolation when the coordinate values (xs, ys) of the ROI 520 calculated using Equations (2) and (3) are not integer values The coordinates (xs, ys) of the ROI 520 can be interpolated.

If the coordinate value (xs, ys) of the ROI 520 calculated using Equations (2) and (3) is not an integer value, the image processing module 154 calculates the value of the nearest neighbor pixel .

The image processing module 154 may correct the rendered panorama image 550 using a low-pass filter. Although the mapping has been described using three coordinates in FIG. 7, the technical idea of the present invention is not limited to the number or position of coordinates to be mapped.

8 shows a table stored in the memory shown in Fig. 1, 7, and 8, the image processing module 154 maps the coordinates of each pixel of the ROI 520 and the coordinates of each pixel of the rendered panoramic image 550 one-to-one .

The image processing module 154 reads the table 600 stored in the first memory 156 and reads the coordinate of each pixel of the ROI 520 and the coordinates of each pixel of the rendered panorama image 550 using the read table 600 The coordinates of each pixel can be mapped on a one-to-one basis.

For example, the image processing module 154 uses the table 600 to convert the first coordinates xs1, ys1 of the extracted ROI 520 to the first coordinates xr1, yr1 of the rendered panorama image 550, The second coordinates xs2 and ys2 of the extracted ROI 400 may be mapped to the second coordinates xr2 and yr2 of the rendered panorama image 450 and the extracted ROI 520 may be mapped to the second coordinates xr2 and yr2 of the rendered panorama image 450, (Xs3, ys3) of the rendered panorama image 550 to the third coordinates (xr3, yr3) of the rendered panorama image 550.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is evident that many alternatives, modifications and variations will be apparent to those skilled in the art. Accordingly, the true scope of the present invention should be determined by the technical idea of the appended claims.

100: Car black box
110: camera module
120: lens
130: Image sensor
140: Image signal processor
160: Bus
170: memory
180: Display

Claims (13)

A camera module for generating a hemispheric image using one fisheye lens; And
And an image signal processor for generating a panoramic image rendered from the hemispherical image,
The image signal processor comprising:
An ROI extraction module for extracting an ROI from the hemispherical image in response to a region of interest (ROI) selection signal; And
And an image processing module for rendering the extracted ROI and generating the rendered panorama image,
The image processing module includes:
Transforming the circumferential direction of the extracted ROI into a horizontal direction of the rendered panorama image, transforming the radiating direction of the extracted ROI into a vertical direction of the rendered panorama image, and converting the extracted vertical direction resolution, And determining a vertical resolution of the rendered panoramic image based on the difference between the minimum altitude and the reference altitude. delete delete The image processing apparatus according to claim 1,
Wherein the rendered panorama image is generated from the extracted ROI through the rendering by mapping the coordinates of each pixel of the extracted ROI and the coordinates of each pixel of the rendered panorama image one-to-one. 5. The method of claim 4,
Wherein the image signal processor further comprises a memory for storing a mapping table,
Wherein the image processing module maps the coordinates of each pixel of the extracted ROI and the coordinates of each pixel of the rendered panoramic image in a one-to-one correspondence using the mapping table. The image processing apparatus according to claim 1,
A vehicle black box for generating the rendered panorama image from the extracted ROI through the one-to-one mapping of the pixel value of each pixel of the extracted ROI and the pixel value of each pixel of the rendered panorama image; . The image processing apparatus according to claim 1,
And correcting the rendered panorama image using a low-pass filter to generate a corrected panorama image. The method according to claim 1,
Wherein the vehicle black box further comprises a display for displaying the rendered panorama image generated by the image signal processor,
Wherein the image signal processor controls to simultaneously display the hemispherical image and the rendered panorama image. A vehicle black box system comprising a vehicle black box and a display for outputting images stored in the vehicle black box,
The vehicle black box includes:
A camera module for generating a hemispherical image using one fisheye lens; And
And an image signal processor for generating a rendered panoramic image from the hemispherical image,
The image signal processor comprising:
An ROI extraction module for extracting an ROI from the hemispherical image in response to an ROI selection signal; And
And an image processing module for rendering the extracted ROI and generating the rendered panorama image,
The image processing module includes:
Transforming the circumferential direction of the extracted ROI into a horizontal direction of the rendered panorama image, converting the radiating direction of the extracted ROI into a vertical direction of the rendered panorama image, and outputting the extracted vertical direction resolution, And determines a vertical resolution of the rendered panoramic image based on the difference between the minimum altitude and the reference altitude. delete 10. The method of claim 9,
Wherein the image signal processor controls the display such that the hemispherical image and the rendered panoramic image are simultaneously displayed. An image signal processor for generating a panorama image rendered from a hemispherical image output from a camera module for generating a hemispherical image using a fisheye lens,
An ROI extraction module for extracting an ROI (RS) from the hemispherical image in response to a region of interest (ROI) selection signal; And
And an image processing module for rendering the extracted ROI and generating the rendered panorama image,
The image processing module includes:
Transforming the circumferential direction of the extracted ROI into a horizontal direction of the rendered panorama image, converting the radiating direction of the extracted ROI into a vertical direction of the rendered panorama image, and outputting the extracted vertical direction resolution, And determining a vertical resolution of the rendered panoramic image based on the difference between the minimum altitude and the reference altitude. delete
KR1020150056787A 2015-03-24 2015-04-22 Car black box including fish-eye lens and car black box system including the same KR101650810B1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR20150040903 2015-03-24
KR1020150040903 2015-03-24

Publications (1)

Publication Number Publication Date
KR101650810B1 true KR101650810B1 (en) 2016-08-24

Family

ID=56884410

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020150056787A KR101650810B1 (en) 2015-03-24 2015-04-22 Car black box including fish-eye lens and car black box system including the same

Country Status (1)

Country Link
KR (1) KR101650810B1 (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101871096B1 (en) 2017-01-09 2018-08-02 주식회사 옵트론텍 Imaging lens optical system
KR20180117867A (en) 2017-04-20 2018-10-30 스크린커플스(주) 360 degrees Fisheye Rendering Method for Virtual Reality Contents Service
KR102158528B1 (en) 2019-04-15 2020-09-22 (주)에이치엠티 An Image Quality Improved Type of a Lens Assemblyfor a Black Box
KR20200120420A (en) 2019-04-12 2020-10-21 (주)에이치엠티 A Lens Assembly for a Black Box

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20070066660A (en) * 2005-12-22 2007-06-27 엘지전자 주식회사 Apparatus for recording car accident information and apparatus for monitoring rear view
JP2011066860A (en) * 2009-09-18 2011-03-31 Loarant Corp Panoramic image generation method and panoramic image generation program
KR20120008827A (en) * 2010-07-20 2012-02-01 허병도 Synthetic simultaneous monitoring camera with pan/tilt/zoom function and monitoring system using the same
KR20130064169A (en) 2011-12-08 2013-06-18 아진산업(주) An apparatus for generating around view image of vehicle using multi look-up table
KR20140137485A (en) * 2013-05-22 2014-12-03 아이오솔루션(주) System for multi channel display to use a fish-eye lens

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20070066660A (en) * 2005-12-22 2007-06-27 엘지전자 주식회사 Apparatus for recording car accident information and apparatus for monitoring rear view
JP2011066860A (en) * 2009-09-18 2011-03-31 Loarant Corp Panoramic image generation method and panoramic image generation program
KR20120008827A (en) * 2010-07-20 2012-02-01 허병도 Synthetic simultaneous monitoring camera with pan/tilt/zoom function and monitoring system using the same
KR20130064169A (en) 2011-12-08 2013-06-18 아진산업(주) An apparatus for generating around view image of vehicle using multi look-up table
KR20140137485A (en) * 2013-05-22 2014-12-03 아이오솔루션(주) System for multi channel display to use a fish-eye lens

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101871096B1 (en) 2017-01-09 2018-08-02 주식회사 옵트론텍 Imaging lens optical system
KR20180117867A (en) 2017-04-20 2018-10-30 스크린커플스(주) 360 degrees Fisheye Rendering Method for Virtual Reality Contents Service
KR20200120420A (en) 2019-04-12 2020-10-21 (주)에이치엠티 A Lens Assembly for a Black Box
KR102158528B1 (en) 2019-04-15 2020-09-22 (주)에이치엠티 An Image Quality Improved Type of a Lens Assemblyfor a Black Box

Similar Documents

Publication Publication Date Title
US11659282B2 (en) Image processing system and image processing method
KR102385360B1 (en) Electronic device performing image correction and operation method of thereof
CN107493432B (en) Image processing method, image processing device, mobile terminal and computer readable storage medium
US8379976B2 (en) Image processing apparatus and method and a computer-readable recording medium on which an image processing program is stored
KR101650810B1 (en) Car black box including fish-eye lens and car black box system including the same
TWI549519B (en) High resolution array camera
CN109690628B (en) Image processing apparatus
US20120154541A1 (en) Apparatus and method for producing 3d images
EP3707671B1 (en) Electronic device and method for correcting image corrected in first image processing scheme in external electronic device in second image processing scheme
CN113170061B (en) Image sensor, imaging device, electronic apparatus, image processing system, and signal processing method
CN107704798B (en) Image blurring method and device, computer readable storage medium and computer device
JP7282798B2 (en) Imaging device, image processing method, and program
CN109951641B (en) Image shooting method and device, electronic equipment and computer readable storage medium
KR20170067634A (en) Image capturing apparatus and method for controlling a focus detection
CN109981983B (en) Augmented reality image processing method and device, electronic equipment and storage medium
CN106233717B (en) System and method for transmitting camera-based parameters without a dedicated back channel
CN110995982B (en) Image processing apparatus, control method therefor, imaging apparatus, and recording medium
CN112866554B (en) Focusing method and device, electronic equipment and computer readable storage medium
JP6563694B2 (en) Image processing apparatus, image processing system, image composition apparatus, image processing method, and program
JP2013150071A (en) Encoder, encoding method, program and storage medium
CN114257733A (en) Method and system for image processing of omni-directional image with viewpoint offset
JP2016040883A (en) Image processing device, image processing method, image processing system, and program
JP2020088419A (en) Imaging apparatus, image correction method, and program
WO2018078222A1 (en) Multiple view colour reconstruction
US11451699B2 (en) Device for correcting and compressing image error data, and method related thereto

Legal Events

Date Code Title Description
GRNT Written decision to grant
FPAY Annual fee payment

Payment date: 20190919

Year of fee payment: 4