KR20180051422A - Electronic apparatus and control method thereof - Google Patents

Abstract

An electronic apparatus and a control method thereof are disclosed. According to the present invention, the control method of the electronic apparatus comprises the steps of: receiving video data; acquiring feature information representing an object from the video data by using a plurality of filters; detecting the object included in the video data by using the feature information acquired by at least two of the plurality of filters; and providing information on the detected object. As a result, the electronic apparatus can more accurately detect a surrounding vehicle and a pedestrian even under a dark road condition (such as at night time and bad weather) or the like as well as under a general road condition.

Description

[0001] The present invention relates to an electronic apparatus and a control method thereof,

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an electronic device and a control method thereof, and more particularly, to an electronic device for detecting an object from image data input to an electronic device and a control method thereof.

BACKGROUND ART [0002] With the recent development of autonomous driving-related technologies and the like, an electronic device such as a head-up display mounted in a vehicle and providing driving information, or a black box mounted in a vehicle and photographing a driving situation in real time, Lt; / RTI > In addition, the electronic device provides the running information including the situation of the road on which the vehicle is running. Specifically, the electronic device captures the road conditions on which the vehicle travels, detects objects including nearby vehicles, pedestrians, and the like around the vehicle from the photographed image data, and displays the type information of the detected object, Or the like.

Accordingly, the driver can prevent the risk of an accident between the vehicles or between the vehicle and the pedestrian by checking the distance between the preceding vehicle, the risk of a blind spot, the presence of a pedestrian crossing the road, and the like through various travel information provided through the electronic device.

However, such a conventional electronic device has a problem that a plurality of nearby vehicles are parked around the vehicle, or a road condition under which a plurality of pedestrians cross the vehicle road together, a road situation of bad weather (snow, rain, fog) There arises a problem in that, in the road situation, the surrounding vehicles and the pedestrian can not be detected from the photographed image data.

The present invention has been made in order to solve the above-described problems and to respond to the above-described technology development request. The present invention is intended to facilitate detection of nearby vehicles and pedestrians even in the case of night roads and bad weather conditions as well as general road conditions. .

A method of controlling an electronic device according to an embodiment of the present invention includes receiving image data, acquiring feature information indicating an object from the image data using a plurality of filters, Detecting an object included in the image data using the feature information acquired through the filter, and providing information on the detected object.

According to another aspect of the present invention, there is provided an electronic device including an input unit for inputting image data, an output unit for outputting information on the object of the image data, And controlling the output unit to output information about the detected object by detecting an object included in the image data using the feature information acquired through at least two filters among the plurality of filters, .

According to the various embodiments as described above, the electronic device according to the present invention can more accurately detect nearby vehicles and pedestrians, not only in general road conditions but also in nighttime and bad weather conditions.

1 is an exemplary view showing an environment in which an object is not recognized in a general electronic device,
2 is a schematic block diagram of an electronic device for detecting objects from image data according to an embodiment of the present invention;
3 is a detailed block diagram of an electronic device for detecting an object from image data according to an embodiment of the present invention;
4 is an exemplary diagram showing an object detection process and a detection result in a conventional electronic device,
5 is an exemplary diagram illustrating an object detection process and a detection result in an electronic device according to an embodiment of the present invention.
6 is an exemplary diagram for adjusting the size of the reference confidence map used for generating the final confidence map in the electronic device according to the present invention,
7 is an exemplary diagram showing a detection result of an object detected from image data in an electronic device and a conventional electronic device according to the present invention,
8 is a diagram illustrating an example of providing travel information according to an object detection result of an electronic device according to an embodiment of the present invention.
9 is a detailed block diagram of an electronic device according to an embodiment of the present invention;
10 is a flowchart of a method of controlling an electronic device for detecting an object from photographed image data according to an embodiment of the present invention;
11 is a flowchart of a method for acquiring feature information representing an object from image data in an electronic device according to an embodiment of the present invention;
12 is a flowchart of a method of detecting an object in an electronic device according to an embodiment of the present invention.

Before describing the present invention in detail, a method of describing the present specification and drawings will be described.

First, the terms used in the specification and claims have chosen generic terms in light of their function in various embodiments of the present invention. However, these terms may vary depending on the intentions of the skilled artisan, the legal or technical interpretation, and the emergence of new technologies. In addition, some terms are arbitrarily selected by the applicant. These terms may be construed in the meaning defined herein and may be interpreted based on the general contents of this specification and the ordinary technical knowledge in the art without specific terms definition.

In addition, the same reference numerals or signs in the drawings attached to the present specification indicate components or components that perform substantially the same function. For ease of explanation and understanding, different embodiments will be described using the same reference numerals or symbols. That is, even though all of the elements having the same reference numerals are shown in the plural drawings, the plural drawings do not mean one embodiment.

Further, in the present specification and claims, terms including ordinal numbers such as "first "," second ", etc. may be used for distinguishing between elements. These ordinals are used to distinguish between identical or similar elements, and the use of such ordinal numbers should not be construed as limiting the meaning of the term. For example, components associated with such an ordinal number should not be limited in the order of use, placement order, or the like. If necessary, each ordinal number may be used interchangeably.

As used herein, the singular forms "a", "an" and "the" include plural referents unless the context clearly dictates otherwise. In this application, the terms "comprise", "comprising" and the like are used to specify that there is a stated feature, number, step, operation, element, component, or combination thereof, But do not preclude the presence or addition of features, numbers, steps, operations, components, parts, or combinations thereof.

In the embodiments of the present invention, terms such as "module", "unit", "part", and the like are terms used to refer to components that perform at least one function or operation, Or may be implemented as a combination of hardware and software. It should also be understood that a plurality of "modules "," units ", "parts ", etc. may be integrated into at least one module or chip, (Not shown).

Further, in an embodiment of the present invention, when a part is connected to another part, this includes not only a direct connection but also an indirect connection through another medium. Also, the inclusion of a component in a component means that the component may include other components, not the exclusion of any other component, unless specifically stated otherwise.

Hereinafter, various embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is an exemplary diagram showing an environment in which an object is not recognized by a general electronic device.

As shown in FIG. 1, an electronic device 100 provided in a transportation apparatus such as a vehicle may be, for example, a black box, a smart phone, or the like, which photographs the road conditions on which the transportation apparatus travels in real time. Such an electronic device 100 can photograph a front region and a rear region of a road on which a transportation apparatus travels in real time, and can detect objects such as people and vehicles on photographed image data using an object detection algorithm.

Then, the electronic device 100 provides at least one of the type information of the detected object, the position information of the object, and the movement information of the object. Accordingly, the driver who operates the transportation apparatus can recognize the road environmental conditions in the forward direction or the backward direction on the basis of the information provided through the electronic device 100, thereby enabling safe running.

In this way, the electronic device 100 providing information on various objects existing on the road can not detect the object under the following environmental conditions.

Specifically, as shown in Fig. 1, in the first driving environment 10 in which the transportation apparatus travels on the night road, the electronic device 100 does not detect an object for a pedestrian from photographed image data. In addition, the electronic device 100 can not detect an object for a pedestrian or a nearby vehicle from the photographed image data in the second driving environment 20 where the transportation device travels on the road in the rainy weather . In addition, in the third driving environment 30 in which a part of the pedestrian is covered by the umbrella, the electronic device 100 does not detect the object for the pedestrian obstructed by the umbrella from the photographed image data.

These environmental conditions can be environmental conditions that make the transportation equipment difficult to operate. If the electronic device 100 can not detect an object existing on the road under such an environmental condition, the probability of occurrence of an accident by a driver driving the transportation device can be increased.

Hereinafter, each configuration of the electronic device 100 will be described in detail.

2 is a schematic block diagram of an electronic device for detecting objects from image data according to an embodiment of the present invention.

2, the electronic device 100 includes an input unit 110, an output unit 120, a storage unit 130, and a processor 140. The input unit 110, the output unit 120,

The input unit 110 receives image data photographed through a camera. The output unit 120 outputs at least one of video and audio from the image data input through the input unit 110. [ Also, the output unit 120 outputs information about the object detected from the image data through at least one of video and audio. Here, the information about the object detected from the image data may include at least one of type information of the object, position information, and movement information.

The storage unit 130 may store a set of filters including a plurality of filters for detecting feature points representing objects included in the photographed image data.

Accordingly, the processor 140, which will be described later, can detect the object from the image data input through the input unit 110 using the filter set stored in the storage unit 130. [

Specifically, the processor 140 acquires feature information indicating an object from the image data input through the input unit 110 using a plurality of filters included in the filter set. Thereafter, the processor 1409 detects an object included in the corresponding image data using the feature information acquired through at least two filters among the plurality of filters, and outputs the information on the detected object. .

Here, the feature information may include a probability value indicating the reliability of the object and position information on the position of the object.

Specifically, the processor 140 generates a confidence map corresponding to each of the plurality of filters using each of the characteristic information obtained from the plurality of filters constituting the filter set. Thereafter, the processor 140 acquires first and second confidence maps of the respective confidence maps corresponding to the plurality of filters, and generates the final confidence map using the acquired first and second confidence maps. After that, the processor 140 analyzes the final confidence map and can detect the object from the input image data through the input unit 110. [

Here, the first confidence map is a map generated using the feature information acquired from the filter after the predetermined filter among the plurality of filters, and the second confidence map is generated using the feature information acquired from the last filter among the plurality of filters Gt; map < / RTI >

The present invention is limited to generating the final confidence map using the first and second confidence maps. However, the present invention is not limited to this, and the first and second confidence maps and the feature information The final confidence map can be generated using the generated confidence map.

More specifically, the processor 140 acquires first characteristic information indicating an object included in the image data from the image data input through the input unit 110 using the first filter among the plurality of filters constituting the filter set . Thereafter, the processor 140 generates a confidence map based on the first feature information, as described above. Thereafter, the processor 140 uses the second filter among the plurality of filters to acquire second characteristic information indicating the object from the confidence map generated based on the first characteristic information. Thereafter, the processor 140 polls the confidence map generated based on the second feature information to a predetermined size. Thereafter, the processor 140 uses the third filter to obtain third feature information representing the object from the polled confidence map.

If the third filter is a last filter, the processor 140 determines whether or not the third feature information acquired from the last filter and the confidence map generated based on the feature information obtained from at least one of the first and second filters It is possible to generate the final confidence map using the generated confidence map based on the confidence map.

Thereafter, the processor 140 compares the probability value indicating the reliability of the object with the predetermined threshold value from the preliminarily generated final confidence map, and detects the object from the image data if the probability value is equal to or greater than a preset threshold value. Thereafter, the processor 140 may obtain the type information of the object based on the type of the detected object.

Accordingly, the processor 140 may control the output unit 120 to output at least one of type information, position information, and movement information of the detected object from the input image data.

In addition, the processor 140 may use at least one of the type information, the position information, and the movement information of the object detected from the input image data to calculate distance information between the electronic device 100 and the object, dangerous situation information, Information. ≪ / RTI > Then, the processor 140 transmits the information to another electronic device (not shown) provided in the transportation apparatus through the communication unit 150, which will be described later, and provides the traveling information of the transportation apparatus. Accordingly, the other electronic device (not shown) can provide at least one of distance information, dangerous situation information and blind zone notification information to the previously detected object based on the information received from the electronic device 100. [

Hereinafter, the operation of detecting an object from image data in the electronic device 100 according to the present invention will be described in more detail.

3 is a detailed block diagram of an electronic device for detecting an object from image data according to an embodiment of the present invention.

3, the processor 140 includes first to third operation units 141 to 143, and the storage unit 130 includes a filter storage unit 131, a feature result storage unit 132, And an information storage unit 133.

More specifically, when the first image data is input through the input unit 110, the first calculation unit 141 uses the parameters set in the first filter among the plurality of filters included in the filter set previously stored in the filter storage unit 131 Acquires the first characteristic value for the object from the image data, and stores the first characteristic value in the characteristic result storage unit 132.

The second calculation unit 142 obtains the first characteristic information including the probability value representing the reliability of the object and the position information of the object based on the first characteristic value stored in the characteristic result storage unit 132, (133). Accordingly, the third calculation unit 143 generates the first confidence map corresponding to the first filter based on the first feature information stored in the feature information storage unit 133. [

When the first confidence map is generated, the first calculation unit 141 acquires the second characteristic value for the object from the first confidence map using the parameter set in the second filter, and stores the acquired second characteristic value in the characteristic result storage unit 132 .

Accordingly, the second calculation unit 142 acquires second characteristic information including a probability value and positional information indicating the reliability of the object based on the second characteristic value stored in the characteristic result storage unit 132, (133). Accordingly, the third computing unit 143 generates the second confidence map corresponding to the second filter based on the second characteristic information stored in the characteristic information storage unit 133. [

If a third confidence map corresponding to the third filter, which is the last filter among the plurality of filters, is generated through the above-described series of processes, the third computing unit 143 uses the generated first to third confidence maps to determine the final confidence You can create a map.

However, the present invention is not limited to this, and the third calculation unit 143 may use the third confidence map generated corresponding to one of the generated first and second confidence maps and the third filter, You can create a confidence map.

When the final confidence map is generated, the third computing unit 143 compares the probability value indicating the reliability of the object with the preset threshold value from the final confidence map, and if the probability value is equal to or greater than a preset threshold value, And transmits information about the object to the output unit 120. [ Accordingly, the output unit 120 can output information about the object detected from the input image data through at least one of video and audio.

Hereinafter, a process of detecting an object from image data input from an electronic device according to the present invention and a conventional electronic device, and a detection result are compared and described.

4 is an exemplary diagram showing an object detection process and a detection result in a conventional electronic device.

4, in the conventional electronic device 100 ', when the image data 410 is inputted, a feature value of the object is obtained from the input image data using a plurality of filters included in the filter set .

In this case, when the image data 410 is inputted, the conventional electronic device 100 'acquires the first characteristic information on the object from the image data 410 using the parameter set in the first filter. Thereafter, the electronic device 100 'acquires a first confidence map (conv 1_1) 411 corresponding to the first filter based on the first feature information.

Thereafter, the electronic device 100 'acquires the second feature information from the first confidence map 411 using the parameter set in the second filter, and based on the acquired second feature information, 2 conviance map (conv 1 - 2) 412.

Thereafter, the electronic device 100 'polls the second confidence map 412 to a predetermined size (pool 1) 413 to obtain the polled confidence map. Thereafter, the electronic device 100 'acquires the third feature information from the polled confidence map using the third filter, and based on the acquired third feature information, generates a third confidence map (conv 2_1 ) ≪ / RTI >

When the n-th confidence map (conv 5 - 3) 415 corresponding to the last n-th filter is obtained through the series of processes, the conventional electronic device 100 'acquires the n-th confidence map 415, And determines it as the confidence map 420.

Then, when the final feature information is obtained from the predetermined final confidence map 420, the conventional electronic device 100 'generates image data (hereinafter referred to as " final feature information ") from the final feature information estimated using the position predictor 410) of the object. Here, the above-described characteristic information may be a characteristic value for an object.

Accordingly, the conventional electronic device 100 'can acquire the position information on the object from the feature value included in the final feature information estimated through the position predictor (RPN).

Thereafter, the conventional electronic device 100 'obtains the type information of the object from the feature values contained in the final feature information acquired. Thereafter, the conventional electronic device 100 'may provide the object detection result information 430 indicating the type and position of the object based on the type information of the object and the position information of the object.

That is, the conventional electronic device 100 'provides only the object detection result information 430 in which the first object corresponding to the car 431 is detected through the object detection result information 430.

On the input image data, a first object corresponding to the car 431 and a second object corresponding to a person partially overlapping the first object may be included. However, the conventional electronic device 100 'can not detect the second object overlapped with the first object, and only the object detection result information 430 in which the first object corresponding to the automobile 431 is detected do.

5 is an exemplary diagram illustrating an object detection process and a detection result in an electronic device according to an exemplary embodiment of the present invention.

5, when the image data 510 is input, the electronic device 100 extracts characteristics of the object from the input image data using the parameters set in the first filter among the plurality of filters included in the filter set, ≪ / RTI > Thereafter, the electronic device 100 acquires the feature information including the probability value indicating the reliability of the object and the positional information at which the object is located, based on the feature value acquired. Then, the electronic device 100 generates a first confidence map (conv 1 _ 1) 511 corresponding to the first filter based on the characteristic information obtained by the process.

When the first confidence map 511 is generated, the electronic device 100 acquires the feature value for the object from the first confidence map 511 using the parameter set in the second filter. Thereafter, the electronic device 100 acquires the feature information including the probability value indicating the reliability of the object and the positional information at which the object is located, based on the feature value acquired. Thereafter, the electronic device 100 generates a second confidence map (conv 1 - 2) 512 corresponding to the second filter based on the characteristic information acquired.

When the second confidence map 512 is generated, the electronic device 100 polls the generated second confidence map 512 to a preset size, and generates a polled confidence map 513. [ Then, the electronic device 100 acquires the feature value for the object from the polled confidence map 513 using the parameter set in the third filter. Thereafter, the electronic device 100 acquires the feature information including the probability value indicating the reliability of the object and the positional information at which the object is located, based on the feature value acquired. Thereafter, the electronic device 100 generates a third confidence map (conv 2_1) 514 corresponding to the third filter based on the feature information that has been estimated.

When the n-th confidence map (conv 8_2) 515 corresponding to the last n-th filter is obtained through the series of processes, the electronic device 100 generates an n-th confidence map (conv 8_2) 515, And obtains a confidence map corresponding to the predetermined filter in the confidence map.

Then, the electronic device 100 determines the n-th confidence map (conv 8_2) 515 and the confidence map corresponding to the predetermined filter as the reference confidence map (hereinafter referred to as first to fourth reference confidence maps) do. Then, the electronic device 100 adjusts the sizes of the first to fourth reference confidence maps 521 to 525 to the same size. Then, the electronic device 100 generates the final confidence map 530 using the first to the reference confidence maps 521 to 525 adjusted to the same size.

However, the present invention is not limited to this, and the electronic device 100 can adjust the size of each of the confidence maps to the same predetermined size when a confidence map is generated corresponding to each of the plurality of filters.

When the final confidence map 530 is generated, the electronic device 100 acquires feature information indicating an object from the generated final confidence map 530, and based on the acquired feature information, Object is detected. Then, the electronic device 100 provides the object detection result 540 for the input image data based on the detected object position and type information.

4, the first object corresponding to the automobile and the second object corresponding to the person partially overlapping the first object may be included in the input image data 510. [

In this case, the electronic device 100 according to the present invention includes a first object 541 corresponding to a car and a second object corresponding to a person partially overlapping the first object from the input image data 510 through a series of the above- And the first and second objects 541 and 542 may provide the object detection result information 540 that has been detected.

Hereinafter, an operation of adjusting the sizes of the first to fifth reference confidence maps 521 to 525 used to generate the above-described final confidence map 530 in the electronic device 100 according to the present invention will be described .

6 is an exemplary diagram for adjusting the size of the reference confidence map used for generating the final confidence map in the electronic device according to the present invention.

5, the electronic device 100 transmits a confidence map corresponding to a predetermined filter out of the confidence maps corresponding to each of the plurality of filters and a confidence map corresponding to the last n filter to the first through fifth reference confidences Maps 521 to 525 can be determined.

Each of the confidence maps determined to be determined by the first to fifth reference confidence maps 521 to 525 can be polled to a predetermined size.

For example, the confidence map determined by the first reference confidence map 521 may be a map that is resized in units of 80 * 48 pixels, and the confidence map determined by the second reference confidence map 522 may be 80 * 48 May be a map whose size is adjusted by 40 * 24 pixels in pixel units.

The confidence map determined by the third reference confidence map 523 may be a map in which the size is adjusted in units of 20 * 12 pixels in units of 40 * 24 pixels, and the reliability determined in the fourth reference confidence map 524 The map can be a map resized in units of 20 * 12 pixels, or 10 * 6 pixels. Then, the confidence map corresponding to the last nth filter determined by the fifth reference confidence map 525 may be a map in which the size is adjusted in 5 * 3 pixel units in 10 * 6 pixel units.

Therefore, the electronic device 100 adjusts the size of each of the confidence maps determined by the first to fifth reference confidence maps 521 to 525 to the same predetermined size.

According to the embodiment, the electronic device 100 is provided with the first reference confidence map 521 corresponding to the first reference confidence map 521 represented by the largest size among the sizes of the respective confidence maps determined by the first to fifth reference confidence maps 521 to 525 The sizes of the confidence maps corresponding to the remaining second to fifth confidence maps 522 to 525 can be adjusted by the size of the confidence map.

That is, the electronic device 100 adjusts each of the confidence maps corresponding to the second to fifth confidence maps 522 to 525 so as to be expressed by a size of 80 * 48 pixels. Accordingly, the reliability map corresponding to each of the first to fifth reference confidence maps 521 to 525 can be a size of 80 * 48 pixels.

Then, the electronic device 100 can generate the final confidence map 540 as described in FIG. 5 using the first to fifth reference confidence maps 521 to 525 adjusted to the same size as described above have.

Fig. 7 is an exemplary diagram showing detection results of an object detected from image data in an electronic device and a conventional electronic device according to the present invention. Fig.

As shown in Fig. 7 (a), when the plurality of objects are superimposed on each other on the input image data, the conventional electronic device 10 can not detect the overlapped objects. For example, when a bus stopped in front of a crosswalk is partially obscured by a person crossing the crosswalk, or when a vehicle and a bus stopped in front of the crosswalk overlap each other, or when a crowd crosses the crosswalk, the conventional electronic device 10 Does not detect an object for a crowd of people crossing the bus, the vehicle, and the pedestrian crossing from the image data 710 that photographed such road environment.

On the other hand, the electronic device 10 according to the present invention can detect an object for each of a crowd of people crossing a bus, a vehicle, and a pedestrian crossing from the image data 710 photographed from the road environment through the above-described execution process.

7B, the conventional electronic device 10 can not detect an object for a running vehicle from image data 720 obtained by photographing a road environment at night, The other electronic device 10 can detect the object for the vehicle under running from the image data 720 photographed at the nighttime road environment.

8 is a diagram illustrating an example of providing driving information according to an object detection result of an electronic device according to an embodiment of the present invention.

8, the electronic device 100 uses the at least one of type information, position information, and movement information of the object detected from the photographed image data to calculate the distance information with respect to the previously detected object, And generates the at least one of the blind zone notification information and transmits it to the other electronic devices 200 provided in the transportation apparatus.

Accordingly, the other electronic device 200 may provide at least one of distance information with respect to the object received from the electronic device 100, dangerous situation information, and blind zone notification information.

8, the other electronic device 200 is connected to the electronic device 100 via an electronic mirror (E-Mirror) 810, which visually provides a risk factor for the circumstance of the transportation device. For example, Lt; RTI ID = 0.0 > notifications. ≪ / RTI >

Specifically, when the blind zone notification information is received from the electronic device 100, the other electronic device 200 determines that one zone of the mirror 810 is displayed differently from the remaining zone based on the received blind zone notification information Thereby controlling the mirror 810. Accordingly, one area of the electronic mirror 810 is indicated by the red color 811, so that the driver of the transportation apparatus can recognize that the vehicle is close to the transportation apparatus based on the notification information displayed on the electronic mirror 810 .

In addition, the other electronic device 200 may provide blind zone notification information received from the electronic device 100 via a head-up display (HUD) 820 that provides driving information.

Specifically, when the blind zone notification information is received from the electronic device 100, the other electronic device 200 controls the head-up display to indicate that the neighboring vehicle is close to the transportation vehicle based on the received blind zone notification information . Accordingly, the head-up display 820 indicates to the one area 821 of the screen that the nearby vehicle is close to the transportation vehicle, so that the driver of the transportation device can display, It is possible to recognize that the vehicle is close to the surroundings.

In addition, the other electronic device 200 can provide dangerous situation information on the neighboring vehicle received from the electronic device 100 through a head-up display (HUD) 820 that provides driving information.

Specifically, when the dangerous situation information is received from the electronic device 100, the other electronic device 200 displays the surrounding vehicle information 822 indicating that the nearby vehicle is in the vicinity of the transportation vehicle on the basis of the received dangerous situation information Up display 820 for display. Accordingly, the head-up display 820 displays a nearby vehicle in a region 821 of the screen differently from another nearby vehicle in the vicinity of the transportation vehicle, so that the driver of the transportation device can display the transportation vehicle It can be recognized that the vehicle is close to the preceding vehicle in the front.

Meanwhile, the electronic device 100 according to the present invention uses at least one of the type information, the position information, and the movement information of the detected object to determine at least one of the distance information, the dangerous situation information, As shown in FIG.

Hereinafter, the detailed configuration of the electronic device 100 according to the present invention will be described in detail.

9 is a detailed block diagram of an electronic device according to an embodiment of the present invention.

9, the electronic device 100 includes a communication unit 150, a photographing unit 160, and a sensing unit 160 in addition to the input unit 110, the output unit 120, the storage unit 130, 170).

First, as described above, the input unit 110 receives image data photographed through a camera. In addition, the input unit 110 may be input means for receiving various user commands and transmitting the user commands to the processor 140. In this case, the input unit 110 may be implemented as a touch pad capable of touch input or a key pad having various function keys, numeric keys, and character keys.

The output unit 120 includes a display unit 121 and an audio output unit 122. The display unit 121 displays the driving information of the transportation vehicle and the video image included in the photographed image data on the screen. Also, the display unit 121 displays at least one of the distance information, the dangerous situation information, and the blind zone notification information of the object generated based on at least one of the type information, the position information, and the movement information of the object detected from the photographed image data Can be visually provided.

The audio output unit 122 can output driving information of the vehicle and audio signals included in the photographed video data in the form of audible sound through a speaker. The audio output unit 122 may output at least one of the distance information, the dangerous situation information, and the blind zone notification information of the object generated based on at least one of the type information, the position information, and the movement information of the object detected from the photographed image data It is possible to output an audio signal for one.

Meanwhile, the display unit 121 may be implemented as a liquid crystal display (LCD), an organic light emitting diode (OLED), or the like. In particular, when the input unit 110 includes a touch input unit (not shown) capable of touch input by the user, the display unit 121 may be implemented as a touch screen having a mutual layer structure together with a touch input unit .

The processor 140 described above controls overall operation of each configuration of the electronic device 100. [ In particular, the processor 140 may detect an object from the image data input through the input unit 110 and provide at least one of type information, position information, and movement information for the detected object, as described above.

Such a processor 140 may include a CPU 141, a GPU 142, a ROM 143, and a RAM 144 and may include a CPU 141, a GPU 142, a ROM 143, and a RAM 144 May be connected to each other via a bus 145.

The CPU 141 accesses the storage unit 130 and performs booting using the OS stored in the storage unit 130. [ The CPU 141 also performs various operations using various programs, contents, data stored in the storage unit 130, and the like.

GPU 142 creates a display screen that includes various objects such as icons, images, text, and the like. Specifically, based on the received control command, the GPU 142 calculates attribute values such as coordinate values, shapes, sizes, and colors to be displayed by the respective objects according to the layout of the screen, And generates a display screen of various layouts including the display screen.

The ROM 143 stores a command set for booting the system and the like. When the turn-on command is input and power is supplied, the CPU 141 copies the OS stored in the storage unit 130 to the RAM 144 according to the instruction stored in the ROM 143, and executes the OS to boot the system. When the booting is completed, the CPU 141 copies various programs stored in the storage unit 130 to the RAM 144, executes the program copied to the RAM 144, and performs various operations.

The processor 140 may be implemented as a system-on-a-chip (SOC) or a system-on-chip (SoC) in combination with the above-described components.

Meanwhile, the operation of the processor 140 may be performed by a program stored in the storage unit 130. Here, the storage unit 130 may include a ROM 143, a RAM 144, or a memory card (e.g., SD card, memory stick) removable / attachable to the electronic device 100, a nonvolatile memory, (HDD) or a solid state drive (SSD).

The communication unit 150 includes a terminal device (not shown) such as a smart phone in the transportation vehicle, an electronic device 100 provided in the nearby vehicle, And can communicate with other electronic devices 200 and a server (not shown) collecting traffic conditions and the like.

The communication unit 150 may be implemented as a communication module such as a short-range wireless communication module (not shown), a wireless communication module (not shown), and the like. Here, the short-range wireless communication module (not shown) is a communication module that performs wireless communication with another electronic device 200 located in a short distance or a terminal device (not shown) in the transportation vehicle, Zigbee, and Near Field Communication (NFC).

The wireless communication module (not shown) accesses the mobile communication network according to various mobile communication standards such as WiFi, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Ling Term Evolution) (Not shown) or the like which collects data.

In addition, the communication unit 150 may include a connector (not shown) including at least one of wired communication modules such as High-Definition Multimedia Interface (HDMI), Universal Serial Bus (USB), IEEE (Institute of Electrical and Electronics Engineers) As shown in FIG. Such a connector (not shown) receives content data transmitted from an external server (not shown) through a wired cable connected to a connector (not shown) according to a control command of the processor 140, Media. Further, the connector (not shown) can receive power from a power source through a wired cable physically connected to a connector (not shown).

The photographing unit 160 photographs a still image or a moving image according to a user command or an event, and may be implemented as a plurality of cameras such as a front camera and a rear camera.

The sensing unit 170 senses the ambient brightness, the external temperature, and the movement of the electronic device 100. The sensing unit 170 may include an illumination sensor (not shown), a temperature sensor (not shown), a motion sensor (not shown), a geomagnetic sensor (not shown), a gravity sensor (not shown), and a gyro sensor .

The illuminance sensor (not shown) senses the brightness of the surrounding environment, and the temperature sensor (not shown) may be a sensor that senses the external temperature.

An accelerometer sensor (not shown) is an acceleration sensor that measures the acceleration or impact strength of the moving electronic device 100. A magnetic sensor (not shown) is a sensor that can detect the azimuth using a geomagnetic field. A gravity sensor (not shown) detects a direction in which gravity acts. And automatically rotates according to the moving direction of the transportation vehicle equipped with the electronic device 100 to sense the direction. Finally, a gyroscope sensor (not shown) is a sensor that recognizes the 6-axis direction by rotating each of the existing motion sensors (not shown) to recognize more detailed and precise motion.

The operation of detecting an object from image data photographed in the electronic device 100 according to the present invention has been described in detail. Hereinafter, a method for detecting an object from image data photographed by the electronic device 100 according to the present invention will be described in detail.

10 is a flowchart of a method of controlling an electronic device for detecting an object from photographed image data according to an embodiment of the present invention.

As shown in Fig. 10, the electronic device 100 receives image data photographed through a camera (S1010). When the image data is input, the electronic device 100 acquires feature information indicating an object from the input image data using a plurality of filters constituting the pre-stored filter set (S1020). Thereafter, the electronic device 100 detects an object included in the image data using the feature information acquired through at least two filters among the plurality of filters (S1030). Then, the electronic device 100 provides information to the detected object (S1040).

Here, the feature information may include a probability value indicating the reliability of the object and position information on the position of the object.

Specifically, the electronic device 100 generates a confidence map corresponding to each of the plurality of filters using each of the feature information obtained from the plurality of filters constituting the filter set. Thereafter, the electronic device 100 obtains the first and second confidence maps corresponding to each of the predetermined two or less filters of the confidence maps corresponding to each of the plurality of filters, and acquires the acquired first and second confidence maps To generate a final confidence map. Then, the electronic device 100 can analyze the final confidence map generated by using the first and second confidence maps, and detect the object from the previously input image data.

Hereinafter, a method of acquiring feature information representing an object from image data using a plurality of filters in the electronic device 100, and a method of detecting an object included in the image data based on the feature information of interest will be described in detail do.

11 is a flowchart of a method for acquiring feature information representing an object from image data in an electronic device according to an embodiment of the present invention.

As shown in FIG. 11, when the image data is input, the electronic device 100 acquires the first characteristic information indicating the object from the image data previously input using the first filter among the plurality of filters (S1110).

Thereafter, the electronic device 100 acquires second feature information indicating an object from the confidence map generated based on the first feature information, using the second filter (S1120). Thereafter, the electronic device 100 polls the confidence map generated based on the second feature information to a predetermined size (S1130). Thereafter, the electronic device 100 acquires third feature information indicating an object from the confidence map polled at a predetermined size using the third filter (S1140).

Specifically, when the original image data is input, the electronic device 100 acquires the first characteristic value for the object from the image data using the parameters set in the first filter among the plurality of filters. Thereafter, the electronic device 100 acquires the first feature information including the probability value indicating the reliability of the object and the position information on the object based on the first feature value. Then, the electronic device 100 generates a first confidence map corresponding to the first filter based on the acquired first feature information.

Thereafter, the electronic device 100 acquires the second characteristic value for the object from the first confidence map generated using the parameter set in the first filter. Thereafter, the electronic device 100 acquires second characteristic information including a probability value and positional information indicating the reliability of the object based on the second characteristic value, and acquires, based on the acquired second characteristic information, And generates a second confidence map.

Thereafter, the electronic device 100 polls the second confidence map at a predetermined size. For example, if the second confidence map is a map expressed in units of 80 * 48 pixels, the electronic device 100 may poll the second confidence map in units of 40 * 24 pixels. When the second confidence map is polled to a preset size, the electronic device 100 uses the parameter set in the third filter to obtain a third characteristic value for the object from the second confidence map polled at a preset size . Thereafter, the electronic device 100 obtains second characteristic information including a probability value and positional information indicating the reliability of the object based on the third characteristic value, and supplies the second characteristic information to the second filter based on the acquired second characteristic information And generates a third confidence map.

When a confidence map corresponding to each of the plurality of filters is generated through the series of processes, the electronic device 100 generates a final confidence map using at least two of the plurality of confidence maps, The object can be detected from the confidence map.

12 is a flowchart of a method of detecting an object in an electronic device according to an embodiment of the present invention.

As shown in FIG. 12, the electronic device 100 generates a confidence map corresponding to each of the plurality of filters using each of the feature information obtained from the plurality of filters (S1210). Then, the electronic device 100 acquires the first and second confidence maps of the plurality of the confidence maps corresponding to the respective filters (S1220)

Here, the first confidence map is a map generated using the feature information acquired from the filter after the predetermined filter among the plurality of filters, and the second confidence map is generated using the feature information acquired from the last filter among the plurality of filters Gt; map < / RTI >

When such first and second confidence maps are obtained, the electronic device 100 adjusts the first and second confidence maps to the same size (S1220, S1230). Thereafter, the electronic device 100 generates a final confidence map using the first and second confidence maps adjusted to the same size, analyzes the generated final confidence map, and detects the objects included in the image data (S1240, S1250).

Specifically, the electronic device 100 can acquire the feature information representing the object from the final confidence map, and can detect the object included in the pre-input image data based on the acquired feature information.

More specifically, when the characteristic information indicating the object is obtained from the final confidence map, the electronic device 100 compares the probability value representing the reliability of the object included in the characteristic information acquired with the predetermined threshold with a preset threshold value. As a result of the comparison, if the probability value is greater than or equal to a predetermined threshold value, the electronic device 100 detects the object based on the positional information of the object included in the characteristic information acquired. Thereafter, the electronic device 100 obtains the type information of the detected object from the final confidence map. Thereafter, the electronic device 100 provides the object detection result for the previously input image data based on at least one of the position information of the detected object, the type information of the object, and the movement information of the object.

Meanwhile, the electronic device 100 is an apparatus provided in the transportation apparatus, and can communicate with another electronic apparatus 200 provided in the transportation apparatus and providing travel information. In this case, the electronic device 100 may receive at least one of distance information, dangerous situation information, and blind zone notification information with respect to the object using at least one of type information, position information, and movement information of the object detected from the image data To the electronic device (200). Accordingly, the other electronic device 200 can provide notification information about the object through an electronic mirror (E-Mirror) or a head-up display based on the information received from the electronic device 100.

On the other hand, the control method of the electronic device 100 as described above may be implemented as at least one executable program, which executable program may be stored in a non-transitory computer readable medium.

A non-transitory readable medium is a medium that stores data for a short period of time, such as a register, cache, memory, etc., but semi-permanently stores data and is readable by the apparatus. Specifically, the above-described programs may be stored in a computer-readable recording medium such as a RAM (Random Access Memory), a flash memory, a ROM (Read Only Memory), an EPROM (Erasable Programmable ROM), an EEPROM (Electronically Erasable and Programmable ROM) Card, a USB memory, a CD-ROM, or the like.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is clearly understood that the same is by way of illustration and example only and is not to be construed as limiting the scope of the invention as defined by the appended claims. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present invention.

100: electronic device 110: input
120: output unit 121: display unit
122: Audio output unit 130:
140: processor 150:
160: photographing unit 170:

Claims (20)

A method of controlling an electronic device,
Receiving image data;
Obtaining characteristic information representing an object from the image data using a plurality of filters;
Detecting an object included in the image data using feature information acquired through at least two filters among the plurality of filters; And
Providing information about the detected object;
≪ / RTI > The method according to claim 1,
The feature information
A probability value indicating the reliability of the object, and position information on which the object is located. The method according to claim 1,
Wherein the acquiring comprises:
Obtaining first feature information representing the object from the image data using a first filter of the plurality of filters;
Acquiring second feature information representing the object from a confidence map generated based on the first feature information using a second filter;
The method comprising: pooling a confidence map generated based on the second feature information to a predetermined size; And
Obtaining third feature information representing the object from the polled confidence map using a third filter;
The control method comprising the steps of: 3. The method of claim 2,
Wherein the detecting comprises:
Generating a confidence map corresponding to each of the plurality of filters using feature information obtained from the plurality of filters;
Obtaining a first and a second one of a plurality of confidence maps corresponding to the plurality of filters;
Generating a final confidence map using the first and second confidence maps; And
Analyzing the final confidence map to detect the object;
The control method comprising the steps of: 5. The method of claim 4,
Wherein the first confidence map comprises:
A map generated by using feature information acquired from a filter next to a predetermined filter among the plurality of filters,
Wherein the second confidence map comprises:
Wherein the map is generated using feature information obtained from the last filter among the plurality of filters. 5. The method of claim 4,
Adjusting the sizes of the first and second confidence maps to the same size;
Further comprising the steps of: 5. The method of claim 4,
Wherein the detecting the object comprises:
And the object is detected based on whether or not a probability value indicating the reliability of the object is equal to or greater than a predetermined threshold value from the final confidence map. The method according to claim 1,
Wherein the providing step comprises:
And providing at least one of type information, position information, and movement information of the detected object. 9. The method of claim 8,
The electronic device is provided in the transportation device and performs communication with other electronic devices providing driving information,
Wherein the providing step comprises:
And provides at least one of distance information, dangerous situation information, and blind zone notification information to the other electronic device using at least one of type information, position information, and movement information of the detected object Way. The method according to claim 1,
The electronic device is a device provided in a transportation device,
Wherein the image data is image data for a front region or a rear region of a road on which the transportation apparatus travels. In an electronic device,
An input unit for receiving image data;
An output unit for outputting information on an object of the image data; And
Acquiring feature information indicating an object from the image data using a plurality of filters,
A processor for detecting an object included in the image data using feature information acquired through at least two filters among the plurality of filters and controlling the output unit to output information about the detected object;
≪ / RTI > 12. The method of claim 11,
The feature information
A probability value indicating the reliability of the object, and position information on which the object is located. 12. The method of claim 11,
The processor comprising:
Acquiring first characteristic information representing the object from the image data using a first filter among the plurality of filters,
Acquiring second feature information representing the object from a confidence map generated based on the first feature information using a second filter,
The method comprising the steps of: pooling a confidence map generated based on the second feature information to a preset size;
And acquires third feature information indicating the object from the polled confidence map using a third filter. 13. The method of claim 12,
The processor comprising:
Generates a confidence map corresponding to each of the plurality of filters using each of the feature information acquired from the plurality of filters,
Acquiring first and second confidence maps of a confidence map corresponding to each of the plurality of filters,
Generates a final confidence map using the first and second confidence maps,
And the object is detected by analyzing the final confidence map. 15. The method of claim 14,
Wherein the first confidence map comprises:
A map generated by using feature information acquired from a filter next to a predetermined filter among the plurality of filters,
Wherein the second confidence map comprises:
Wherein the map is generated using feature information obtained from the last filter among the plurality of filters. 15. The method of claim 14,
The processor comprising:
And adjusts the sizes of the first and second confidence maps to the same size. 15. The method of claim 14,
The processor comprising:
And detects the object based on whether the probability value indicating the reliability of the object is equal to or greater than a predetermined threshold value from the final confidence map. 12. The method of claim 11,
The processor comprising:
And controls the output unit to output at least one of type information, position information, and movement information of the detected object. 19. The method of claim 18,
And a communication unit provided in the transportation device and performing communication with another electronic device providing traveling information,
The processor comprising:
And controls the communication unit to transmit at least one of distance information, dangerous situation information, and blind zone notification information to the other electronic device using at least one of type information, position information, and movement information of the detected object . 12. The method of claim 11,
The electronic device is a device provided in a transportation device,
Wherein the image data is image data for a front region or a rear region of a road on which the transportation apparatus travels.

Images