KR20120103330A - Video processing apparatus and method thereof - Google Patents

Abstract

PURPOSE: An image processing apparatus and a method thereof are provided to detect a difference image corresponding to a depth image area divided by depth information from a three-dimensional image. CONSTITUTION: A depth detector(401) divides an image area having depth according to depth information from a three-dimensional image. A difference image detector(402) detects a difference image between a previous image frame and a current image frame. An object detector(403) detects an image area having motion within the detected difference image as an object. [Reference numerals] (401) Depth detector; (402) Difference image detector; (403) Object detector; (AA) Three-dimensional image

Description

Image processing apparatus and its method {VIDEO PROCESSING APPARATUS AND METHOD THEREOF}

The present specification relates to an image processing apparatus and a method thereof.

In general, the image processing apparatus according to the related art displays a stereoscopic image captured by a camera or a stereoscopic image received through a wired / wireless communication network on a display unit.

It is an object of the present specification to provide an image processing apparatus and a method capable of quickly and accurately tracking object (s) in a stereoscopic image.

An image processing apparatus according to embodiments of the present disclosure may include a depth detector configured to segment an image region having a depth according to depth information from a stereoscopic image; A difference image detector detecting a difference image of a previous and current image frame corresponding to the divided image region; The apparatus may include an object detector configured to determine, as an object, an image area having a motion in the detected difference image.

As an example related to the present invention, the depth detector may segment the image area from the stereoscopic image through a probability density function.

As an example related to the present invention, the difference image detector may set a predetermined threshold value to the detected difference image, and detect a difference image having a motion based on the predetermined threshold value.

As an example related to the present invention, the difference image detector may remove noise of the difference image having the movement through a median filter.

As an example related to the present disclosure, the object detector may determine, as the object, an image region having a motion in the difference image from which the noise is removed.

As an example related to the present disclosure, the object detector may detect a boundary of an image region having the movement and reduce the boundary of the image region based on the continuity of the difference information of the difference image.

As an example related to the present invention, the object detector may determine an image area having the reduced boundary as the object.

As an example related to the present invention, the object detector may determine a target object to be tracked according to a ratio of each object included in the image area having the reduced boundary.

As an example related to the present invention, the object detector determines whether an object is included in the image area having the reduced boundary by a preset ratio or more, and includes the object detector in the image area having the reduced boundary by the preset ratio or more. Can be determined as the target object to be tracked.

As an example related to the present disclosure, the object detector may exclude an object included in the image area having the reduced boundary below the predetermined ratio from the target object to be tracked.

An image processing method according to embodiments of the present disclosure includes: dividing an image region having a depth according to depth information from a stereoscopic image; Detecting difference images of previous and current image frames corresponding to the divided image region; The method may include determining an image area having a motion in the detected difference image as an object.

An image processing apparatus and a method thereof according to an embodiment of the present invention detect a difference image corresponding to a depth image region segmented according to depth information from a stereoscopic image, and detect an image region having a motion in the detected difference image. As a result, the object (s) in the stereoscopic image can be tracked quickly and accurately.

An image processing apparatus and method according to embodiments of the present invention detect a difference image corresponding to a depth image region divided according to depth information from a stereoscopic image, and boundary of an image region having a motion within the detected difference image. There is also an effect that can quickly and accurately track the object (s) in the stereoscopic image by reducing.

The image processing apparatus and the method according to the embodiments of the present invention can determine the target object to be tracked according to the ratio of the objects included in the image area having the reduced boundary, so that the object (s) in the stereoscopic image can be made faster. There is also an effect that can be tracked accurately.

1 is a block diagram illustrating a configuration of a mobile communication terminal to which an image processing apparatus according to embodiments of the present invention is applied.
2 is a block diagram illustrating a configuration of an image display apparatus to which an image processing apparatus according to an exemplary embodiment of the present invention is applied.
3 is a block diagram illustrating an image processing apparatus according to a first embodiment of the present invention.
4 is a flowchart illustrating an image processing method according to a first embodiment of the present invention.
5 is an exemplary view illustrating a stereoscopic image for explaining an image processing apparatus according to an exemplary embodiment of the present invention.
6 is an exemplary diagram illustrating a depth image region divided according to a depth of an object existing in a stereoscopic image according to an exemplary embodiment of the present invention.
7 is a flowchart illustrating an image processing method according to a second embodiment of the present invention.
8 is an exemplary view illustrating a difference image detected according to a predetermined threshold value according to embodiments of the present invention.
9 is an exemplary view illustrating a difference image from which noise is removed according to embodiments of the present invention.
10 is an exemplary diagram illustrating a boundary of an image region in which a motion within a difference image is present according to embodiments of the present invention.
FIG. 11 is an exemplary view illustrating an image region having a motion having a reduced boundary according to embodiments of the present invention.
12 is a flowchart illustrating an image processing method according to a third embodiment of the present invention.

Hereinafter, the present invention detects a difference image corresponding to a depth image region divided according to depth information from a stereoscopic image, and detects an image region having a motion in the detected difference image as an object. Will be described with reference to FIGS. 1 to 12. The suffix "module" and " part "for components used in the following description are given merely for convenience of description, and do not give special significance or role in themselves. Accordingly, the terms "module" and "part" may be used interchangeably.

1 is a block diagram illustrating a configuration of a mobile communication terminal 100 to which an image processing apparatus according to embodiments of the present invention is applied. The mobile communication terminal (mobile phone) 100 may be implemented in various forms. For example, the mobile communication terminal 100 includes a mobile phone, a smart phone, a notebook computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), and the like.

As shown in FIG. 1, the mobile communication terminal 100 includes a wireless communication unit 110, an A / V (Audio / Video) input unit 120, a user input unit 130, a sensing unit 140, and an output unit 150. ), A memory 160, an interface unit 170, a controller 180, a power supply unit 190, and the like. Not all components of the mobile communication terminal 100 shown in FIG. 1 are essential components, and the mobile communication terminal 100 may be implemented by more components than those shown in FIG. The mobile communication terminal 100 may also be implemented by components.

The wireless communication unit 110 includes one or more components for performing wireless communication between the mobile communication terminal 100 and a wireless communication system or wireless communication between the mobile communication terminal 100 and a network in which the mobile communication terminal 100 is located. It may include. For example, the wireless communication unit 110 may include a broadcast receiving module 111, a mobile communication module 112, a wireless internet module 113, a short range communication module 114, a location information module 115, and the like. .

The broadcast receiving module 111 receives a broadcast signal and / or broadcast related information from an external broadcast management server through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. The broadcast management server may mean a server that generates and transmits a broadcast signal and / or broadcast related information or a server that receives a pre-generated broadcast signal and / or broadcast related information and transmits the same to the mobile communication terminal 100. . The broadcast-related information may refer to a broadcast channel, a broadcast program, or information related to a broadcast service provider. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, a data broadcast signal, and a broadcast signal in which a data broadcast signal is combined with a TV broadcast signal or a radio broadcast signal.

Meanwhile, the broadcast related information may be provided through a mobile communication network, and in this case, may be received by the mobile communication module 112. The broadcast related information may exist in various forms. For example, it may exist in the form of Electronic Program Guide (EPG) of Digital Multimedia Broadcasting (DMB) or Electronic Service Guide (ESG) of Digital Video Broadcast-Handheld (DVB-H).

The broadcast receiving module 111 receives broadcasting signals using various broadcasting systems. In particular, the broadcasting receiving module 111 may be a Digital Multimedia Broadcasting-Terrestrial (DMB-T), a Digital Multimedia Broadcasting-Satellite (DMB-S) Only Digital Broadcast-Handheld (DVB-H), Integrated Services Digital Broadcast-Terrestrial (ISDB-T), and the like. Of course, the broadcast receiving module 111 is configured to be suitable for all broadcasting systems that provide broadcasting signals as well as the digital broadcasting system described above. The broadcast signal and / or broadcast related information received through the broadcast receiving module 111 may be stored in the memory 160.

The mobile communication module 112 transmits and receives a radio signal with at least one of a base station, an external terminal, and a server on a mobile communication network. The wireless signal may include various types of data according to transmission and reception of a voice call signal, a video call signal, and / or a text / multimedia message.

The wireless internet module 113 refers to a module for wireless internet access, and the wireless internet module 113 may be built in or external to the mobile communication terminal 100. Here, the wireless Internet technology includes a wireless LAN (WLAN), a Wi-Fi, a wireless broadband (Wibro), a WiMAX, a high speed downlink packet access (HSDPA) Can be used.

The short range communication module 114 means a module for short range communication. As a short range communication technology, Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, and the like may be used.

The position information module 115 is a module for checking or obtaining the position of the mobile terminal (it is possible to check the position of the vehicle when the mobile terminal is mounted on the vehicle). One example is the Global Position System (GPS) module. The GPS module receives position information from a plurality of satellites. Here, the location information may include coordinate information represented by latitude and longitude. For example, the GPS module can measure the exact time and distance from three or more satellites and accurately calculate the current position by triangulating three different distances. A method of obtaining distance and time information from three satellites and correcting the error with one satellite may be used. In particular, the GPS module can obtain not only the location of latitude, longitude, and altitude but also accurate time together with three-dimensional speed information from the location information received from the satellite. As the location information module 115, a Wi-Fi Positioning System and / or a Hybrid Positioning System may be applied.

The location information module 115 may further include a geomagnetic sensor and / or a gravity sensor for detecting a direction. For example, the location information module 115 detects the direction of the mobile communication terminal (for example, east, west, north and south) to implement navigation using augmented reality through the geomagnetic sensor (electronic compass). The location information module 115 detects which direction gravity is acting through the gravity sensor (G sensor), and shows a vertical screen when the user is holding the mobile communication terminal vertically, and lifts it horizontally. When it is present, the screen is rotated 90 degrees automatically to show a wide screen. In addition, when the user watches a video, the location information module 115 rotates the screen according to the direction in which the user is holding the mobile communication terminal through the gravity sensor (G sensor) so that the user can comfortably watch the video.

The A / V input unit 120 is for inputting an audio signal or a video signal, and the A / V input unit 120 includes a camera 121 and a microphone 122. May be included. The camera 121 processes an image frame such as a still image or a video obtained by an image sensor in a video call mode or a photographing mode. The processed image frame may be displayed on the display unit 151.

The image frame processed by the camera 121 may be stored in the memory 160 or transmitted to the outside through the wireless communication unit 110. Two or more cameras 121 may be configured according to the configuration of the mobile terminal.

The microphone 122 receives an external sound signal by a microphone in a call mode, a recording mode, a voice recognition mode, etc., and processes the external sound signal into electrical voice data. In the call mode, the processed voice data may be converted into a form transmittable to the mobile communication base station through the mobile communication module 112 and output. The microphone 122 may implement various noise removing algorithms for removing noise generated in the process of receiving an external sound signal.

The user input unit 130 generates input data for the user to control the operation of the mobile terminal. The user input unit 130 may include a key pad dome switch, a touch pad (static pressure / capacitance), a jog wheel, a jog switch, and the like. In particular, when the touch pad forms a mutual layer structure with the display unit 151, this may be referred to as a touch screen.

The sensing unit 140 may be an open / closed state of the mobile communication terminal 100, a location of the mobile communication terminal 100, a user contact presence, a bearing of the mobile communication terminal 100, an acceleration / deceleration of the mobile communication terminal 100, and the like. As described above, the current state of the mobile communication terminal 100 is sensed to generate a sensing signal for controlling the operation of the mobile communication terminal 100. For example, when the mobile communication terminal 100 is in the form of a slide phone, whether the slide phone is opened or closed may be sensed. In addition, the sensing unit 140 is responsible for sensing functions related to whether the power supply unit 190 is supplied with power, whether the interface unit 170 is coupled to an external device, and the like.

The interface unit 170 serves as an interface with all external devices connected to the mobile communication terminal 100. For example, the interface unit 170 may include a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device equipped with an identification module, An audio input / output (I / O) port, a video input / output (I / O) port, and an earphone port may be configured. Here, the identification module is a chip that stores a variety of information for authenticating the usage rights of the mobile communication terminal 100, and includes a user identification module (UIM) and a subscriber identify module (SIM). ), A Universal Subscriber Identity Module (“USIM”), and the like. In addition, the device equipped with the identification module (hereinafter, 'identification device') may be manufactured in the form of a smart card. Therefore, the identification module may be connected to the mobile communication terminal 100 through a port. The interface unit 170 receives data from an external device or receives power to transmit the data to each component inside the mobile communication terminal 100 or transmit data within the mobile communication terminal 100 to an external device.

The output unit 150 is for outputting an audio signal, a video signal, or an alarm signal. The output unit 150 includes a display unit 151, an audio output module 152, an alarm unit 153, and the like. This may be included.

The display unit 151 displays and outputs information processed by the mobile communication terminal 100. For example, when the mobile communication terminal 100 is in a call mode, the mobile terminal 100 displays a user interface (UI) or a graphic user interface (GUI) related to the call. When the mobile communication terminal 100 is in a video call mode or a photographing mode, the mobile communication terminal 100 displays a photographed and / or received image, a UI, or a GUI.

The display unit 151 may be a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, or a three-dimensional display. It may include at least one of (3D display). In addition, two or more display units 151 may exist according to an implementation form of the mobile communication terminal 100. For example, an external display unit (not shown) and an internal display unit (not shown) may be simultaneously provided in the mobile communication terminal 100.

When the display unit 151 and a sensor for detecting a touch operation (hereinafter referred to as a touch sensor) form a mutual layer structure (hereinafter referred to as a touch screen), the display unit 151 outputs the touch screen. In addition to the device can also be used as an input device. The touch sensor may have the form of, for example, a touch film, a touch sheet, a touch pad, or the like.

In addition, the touch sensor may be configured to convert a change in pressure applied to a specific portion of the display unit 151 or capacitance generated at a specific portion of the display unit 151 into an electrical input signal. The touch sensor may be configured to detect not only the position and area of the touch but also the pressure at the touch. When there is a touch input to the touch sensor, the corresponding signal (s) is sent to a touch controller (not shown). The touch controller processes the signal (s) and then transmits the corresponding data to the controller 180. As a result, the controller 180 can determine which area of the display unit 151 is touched.

Proximity-touch in the present invention means that the pointer is approached a predetermined distance away from the screen without actually touching the screen.

The proximity sensor 141 may be disposed in the inner region of the mobile communication terminal 100 surrounded by the touch screen or near the touch screen. The proximity sensor 141 refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or an object present in the vicinity without using mechanical force by using an electromagnetic force or infrared rays. The proximity sensor 141 has a longer life and higher utilization than a contact sensor.

Examples of the proximity sensor 141 include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor. When the touch screen is capacitive, the touch screen is configured to detect the proximity of the pointer by a change in an electric field according to the proximity of the pointer. In this case, the touch screen (touch sensor) may be classified as a proximity sensor.

Hereinafter, for convenience of explanation, the act of allowing the pointer to be recognized without being in contact with the touch screen so that the pointer is located on the touch screen is referred to as "Proximity Touch", and the touch The act of actually touching the pointer on the screen is called "Contact Touch". The position of the proximity touch with the pointer on the touch screen means a position where the pointer is perpendicular to the touch screen when the pointer is in proximity touch.

In addition, the proximity sensor 141 detects a proximity touch and a proximity touch pattern (for example, a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, and a proximity touch movement state). do. Information corresponding to the sensed proximity touch operation and proximity touch pattern may be output on the touch screen.

The sensing unit 140 may include an acceleration sensor 142. The acceleration sensor 142 converts an acceleration signal in one direction into an electrical signal, and is widely used with the development of micro-electromechanical systems (MEMS) technology. The acceleration sensor 142 measures the acceleration of a large value embedded in an airbag system of a vehicle and used to detect a collision, and recognizes the minute movement of a human hand and uses the acceleration of a minute value used as an input means such as a game. There are many different types of measurements. The acceleration sensor 142 is usually configured by mounting two or three axes in one package, and in some cases, only one Z axis is required depending on the use environment. Therefore, when the acceleration sensor in the X-axis direction or the Y-axis direction is used instead of the Z-axis direction for some reason, the acceleration sensor may be mounted on the main substrate by using a separate piece substrate.

The sound output module 152 outputs audio data received from the wireless communication unit 110 or stored in the memory 160 in a call signal reception, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, and the like. In addition, the sound output module 152 outputs a sound signal related to a function (eg, a call signal reception sound, a message reception sound, etc.) performed in the mobile communication terminal 100. The sound output module 152 may include a speaker, a buzzer, and the like.

The alarm unit 153 outputs a signal for notifying occurrence of an event of the mobile communication terminal 100. Examples of events occurring in the mobile terminal include call signal reception, message reception, and key signal input. The alarm unit 153 may output a signal for notifying occurrence of an event in a form other than an audio signal or a video signal. For example, it is possible to output a signal in a vibration mode. When a call signal is received or a message is received, the alarm unit 153 may vibrate the mobile terminal through a vibration means. Alternatively, when the key signal is input, the alarm unit 153 may vibrate the mobile communication terminal 100 through the vibration means in response to the key signal input. The user can recognize the occurrence of the event through the vibration as described above. Of course, the signal for notification of event occurrence may be output through the display unit 151 or the voice output module 152.

The haptic module 154 generates various tactile effects that the user can feel. A typical example of the haptic effect generated by the haptic module 154 is vibration. The intensity and pattern of vibration generated by the haptic module 154 can be controlled. For example, different vibrations may be synthesized and output or sequentially output.

The haptic module 154, in addition to the vibration, the pin arrangement to move vertically with respect to the contact skin surface, the blowing force or suction force of the air through the injection or inlet, the grazing to the skin surface, the contact of the electrode (eletrode), the stimulation such as electrostatic force Various tactile effects can be produced, such as the effects of the heat-absorbing effect and the effect of reproducing a sense of cold using the elements capable of absorbing heat or heat.

The haptic module 154 may not only deliver the haptic effect through direct contact, but may also be implemented to allow the user to feel the haptic effect through a muscle sense such as a finger or an arm. The haptic module 154 may be provided with two or more according to the configuration aspect of the telematics terminal. The haptic module 154 may be provided where the contact with the user is frequent in the vehicle. For example, it may be provided in a steering wheel, a shift gear lever, a seat seat, and the like.

The memory 160 may store a program for processing and controlling the controller 180 and temporarily stores input / output data (for example, map data, phonebook, message, still image, video, etc.). It can also perform a function for.

The memory 160 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory). Random Access Memory (RAM) Static Random Access Memory (SRAM), Read-Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Programmable Read-Only Memory (PROM) magnetic memory, Magnetic It may include a storage medium of at least one type of disk, optical disk. In addition, the mobile communication terminal 100 may operate a web storage that performs a storage function of the memory 150 on the Internet.

The interface unit 170 serves as an interface with all external devices connected to the mobile terminal 100. For example, the interface unit 170 may include a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device equipped with an identification module, An audio input / output (I / O) port, a video input / output (I / O) port, and an earphone port may be configured. Here, the identification module is a chip that stores various information for authenticating the use authority of the mobile terminal 100, and includes a user identification module (UIM), a subscriber identify module (SIM), and universal user authentication. And a module (Universal Subscriber Identity Module (USIM)). In addition, the device equipped with the identification module (hereinafter referred to as an identification device) may be manufactured in the form of a smart card. Therefore, the identification module may be connected to the mobile terminal 100 through a port. The interface unit 170 may receive data from an external device or receive power and transmit the data to each component inside the mobile terminal 100 or transmit data within the mobile terminal 100 to an external device.

The interface unit 170 may be a passage through which power from the cradle is supplied to the mobile terminal 100 when the mobile terminal 100 is connected to an external cradle, or various commands input by the user from the cradle. It may be a passage through which a signal is transmitted to the mobile terminal 100. Various command signals or power input from the cradle may be operated as signals for recognizing that the mobile terminal 100 is correctly mounted on the cradle.

The controller 180 typically controls the overall operation of the mobile communication terminal 100. For example, the controller 180 performs related control and processing for voice call, data communication, video call, and the like. In addition, the controller 180 may include a multimedia module 181 for multimedia playback. The multimedia module 181 may be implemented in the controller 180 or may be implemented separately from the controller 180.

The controller 180 may perform a pattern recognition process for recognizing a writing input or a drawing input performed on a touch screen as text and an image, respectively.

The power supply unit 190 receives an external power source and an internal power source under the control of the controller 180 to supply power for operation of each component.

The functions of the components applied to the mobile communication terminal 100 may be implemented in a computer-readable recording medium using software, hardware, or a combination thereof. Hardware implementations include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPCs), and controllers (FPGAs). It may be implemented using at least one of controllers, micro-controllers, microprocessors, electrical units for performing functions, etc. In some cases such embodiments are implemented by the controller 180. In a software implementation, embodiments such as a procedure or function may be implemented with a separate software module that performs at least one function or operation Software code may be written in a software application written in a suitable programming language. Software code is also stored in the memory 160 and controlled. It may be executed by the unit 180.

The voice recognition module 182 recognizes the voice spoken by the user and performs a corresponding function according to the recognized voice signal.

The navigation session 300 applied to the mobile communication terminal 100 displays a driving route on map data.

Meanwhile, in the image processing apparatus applied to the mobile communication terminal 100 according to the embodiments of the present invention, a depth for dividing a depth image region (image region having a depth) from a stereoscopic image according to depth information (or binocular disparity information) A detector; A difference image detector detecting a difference image of a previous and current image frame corresponding to the divided depth image region; It may be configured as an object detector for detecting (determining) an image area having a motion in the difference image as an object, and the depth detector, the difference image detector, and the object detector may be included in the controller 180.

Detailed description of the components of the image processing apparatus applied to the mobile communication terminal 100 according to the embodiments of the present invention will be described with reference to FIGS.

Hereinafter, a configuration of an image display apparatus 200 to which an image processing apparatus according to embodiments of the present invention is applied will be described with reference to FIG. 2.

2 is a block diagram illustrating a configuration of an image display apparatus 200 to which an image processing apparatus according to an exemplary embodiment of the present invention is applied.

As shown in FIG. 2, the video display device 200 according to the embodiments of the present invention includes a tuner 210, a demodulator 220, an external device interface 230, a network interface 235, and storage. The unit 240, a user input interface unit 250, a controller 270, a display 280, an audio output unit 285, and a 3D viewing apparatus 295 may be included.

The tuner 210 selects an RF broadcast signal corresponding to a channel selected by a user or all pre-stored channels from a radio frequency (RF) broadcast signal received through an antenna. Also, the selected RF broadcast signal is converted into an intermediate frequency signal, a baseband image, or a voice signal.

For example, if the selected RF broadcast signal is a digital broadcast signal, it is converted into a digital IF signal (DIF). If the selected RF broadcast signal is an analog broadcast signal, it is converted into an analog baseband image or voice signal (CVBS / SIF). That is, the tuner 210 may process a digital broadcast signal or an analog broadcast signal. The analog baseband video or audio signal CVBS / SIF output from the tuner 210 may be directly input to the controller 270.

In addition, the tuner 210 may receive an RF broadcast signal of a single carrier according to an Advanced Television System Committee (ATSC) scheme or an RF broadcast signal of multiple carriers according to a digital video broadcasting (DVB) scheme.

Meanwhile, in the present invention, the tuner 210 sequentially selects RF broadcast signals of all broadcast channels stored through a channel memory function among RF broadcast signals received through an antenna and converts them into intermediate frequency signals or baseband video or audio signals. I can convert it.

The demodulator 220 receives the digital IF signal DIF converted by the tuner 210 and performs a demodulation operation.

For example, when the digital IF signal output from the tuner 210 is an ATSC scheme, the demodulator 220 performs 7-VSB (7-Vestigal Side Band) demodulation. In addition, the demodulator 220 may perform channel decoding. To this end, the demodulator 220 includes a trellis decoder, a de-interleaver, a reed solomon decoder, and the like. Soloman decryption can be performed.

For example, when the digital IF signal output from the tuner 210 is a DVB scheme, the demodulator 220 performs coded orthogonal frequency division modulation (COFDMA) demodulation. In addition, the demodulator 220 may perform channel decoding. To this end, the demodulator 220 may include a convolution decoder, a deinterleaver, a reed-soloman decoder, and the like to perform convolutional decoding, deinterleaving, and reed soloman decoding.

The demodulator 220 may output a stream signal TS after performing demodulation and channel decoding. In this case, the stream signal may be a signal multiplexed with a video signal, an audio signal, or a data signal. For example, the stream signal may be an MPEG-2 TS (Transport Stream) multiplexed with an MPEG-2 standard video signal, a Dolby AC-3 standard audio signal, or the like. Specifically, the MPEG-2 TS may include a header of 4 bytes and a payload of 184 bytes.

The demodulator 220 can be provided separately according to the ATSC system and the DVB system. That is, it can be provided as an ATSC demodulation unit and a DVB demodulation unit.

The stream signal output from the demodulator 220 may be input to the controller 270. The controller 270 performs demultiplexing, image / audio signal processing, and the like, and then outputs an image to the display 280 and outputs an audio to the audio output unit 285.

The external device interface unit 230 may transmit or receive data with the connected external device. To this end, the external device interface unit 230 may include an A / V input / output unit (not shown) or a wireless communication unit (not shown).

The external device interface 230 may be connected to an external device (not shown) such as a DVD (Digital Versatile Disk), a Blu-ray, a game device, a camera, a camcorder, a computer (laptop), or the like by wire / wireless connection. Can be. The external device interface unit 230 transmits an image, audio, or data signal input from the outside to the controller 270 of the image display device 200 through the connected external device. In addition, the controller 270 may output the processed video, audio, or data signal to a connected external device. To this end, the external device interface unit 230 may include an A / V input / output unit (not shown) or a wireless communication unit (not shown).

The A / V input / output unit may be connected to a USB terminal, a CVBS (Composite Video Banking Sync) terminal, a component terminal, an S-video terminal (analog), and a DVI so that video and audio signals of an external device can be input to the video display device 200. (Digital Visual Interface) terminal, HDMI (High Definition Multimedia Interface) terminal, RGB terminal, D-SUB terminal and the like.

The wireless communication unit can perform short-range wireless communication with other electronic devices. The image display device 200 may communicate with Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, and Digital Living Network Alliance (DLNA). Depending on the specification, it can be networked with other electronic devices.

In addition, the external device interface unit 230 may be connected through various set top boxes and at least one of the various terminals described above to perform input / output operations with the set top box.

Meanwhile, the external device interface unit 230 may transmit / receive data with the 3D viewing device 295.

The network interface unit 235 provides an interface for connecting the image display apparatus 200 to a wired / wireless network including an internet network. The network interface unit 235 may include an Ethernet terminal for connection with a wired network, and for connection with a wireless network, a WLAN (Wi-Fi) or a Wibro (Wireless). Broadband, Wimax (World Interoperability for Microwave Access), High Speed Downlink Packet Access (HSDPA) communication standards, and the like may be used.

The network interface unit 235 may receive content or data provided by the Internet or a content provider or a network operator through a network. That is, it can receive contents and related information such as movies, advertisements, games, VOD, broadcasting signals, etc., provided from the Internet, a content provider, and the like through a network. In addition, the update information and the update file of the firmware provided by the network operator can be received. It may also transmit data to the Internet or content provider or network operator.

In addition, the network interface unit 235 is connected to, for example, an IP (Internet Protocol) TV, and receives a video, audio, or data signal processed by an IPTV set-top box to enable bidirectional communication. The signal processed by the controller 270 may be transmitted to the set top box for the IPTV.

Meanwhile, the above-described IPTV may mean ADSL-TV, VDSL-TV, FTTH-TV, etc. according to the type of transmission network, and include TV over DSL, Video over DSL, TV overIP (TVIP), and Broadband TV ( BTV) and the like. In addition, IPTV may also mean an Internet TV capable of accessing the Internet, or a full browsing TV.

The storage unit 240 may store a program for processing and controlling each signal in the controller 270, or may store a signal-processed video, audio, or data signal.

In addition, the storage unit 240 may perform a function for temporarily storing an image, audio, or data signal input to the external device interface unit 230. In addition, the storage unit 240 may store information about a predetermined broadcast channel through a channel storage function such as a channel map.

The storage unit 240 may include a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory), It may include at least one type of storage medium such as RAM, ROM (EEPROM, etc.). The video display device 200 may provide a user with a file (video file, still image file, music file, document file, etc.) stored in the storage unit 240.

1 illustrates an embodiment in which the storage unit 240 is provided separately from the control unit 270, but the scope of the present invention is not limited thereto. The storage unit 240 may be included in the controller 270.

The user input interface unit 250 transmits a signal input by the user to the controller 270 or transmits a signal from the controller 270 to the user.

For example, the user input interface unit 250 may power on / off, select a channel, and display a screen from the remote controller 260 according to various communication methods such as a radio frequency (RF) communication method and an infrared (IR) communication method. A user input signal such as a setting may be received, or a signal from the controller 270 may be transmitted to the remote controller 260.

Also, for example, the user input interface unit 250 may transmit a user input signal input from a local key (not shown) such as a power key, a channel key, a volume key, and a set value to the controller 270.

Also, for example, the user input interface unit 250 may transmit a user input signal input from a sensing unit (not shown) that senses a user's gesture to the control unit 270, or a signal from the control unit 270. May be transmitted to a sensing unit (not shown). Here, the sensing unit (not shown) may include a touch sensor, a voice sensor, a position sensor, an operation sensor, and the like.

The control unit 270 demultiplexes the input stream or processes the demultiplexed signals through the tuner 210, the demodulator 220, or the external device interface unit 230, and outputs a video or audio signal. You can create and output.

The image signal processed by the controller 270 may be input to the display 280 and displayed as an image corresponding to the image signal. In addition, the image signal processed by the controller 270 may be input to the external output device through the external device interface 230.

The voice signal processed by the controller 270 may be sound output to the audio output unit 285. In addition, the voice signal processed by the controller 270 may be input to the external output device through the external device interface 230.

Although not shown in FIG. 1, the controller 270 may include a demultiplexer, an image processor, and the like. The controller 270 may control overall operations of the image display apparatus 200. For example, the controller 270 may control the tuner 210 so as to select an RF broadcast corresponding to a channel selected by a user or a pre-stored channel.

In addition, the controller 270 may control the image display apparatus 200 by a user command or an internal program input through the user input interface unit 250.

For example, the controller 270 controls the tuner 210 to input a signal of a selected channel according to a predetermined channel selection command received through the user input interface 250. Then, video, audio, or data signals of the selected channel are processed. The controller 270 allows the channel information selected by the user to be output through the display 280 or the audio output unit 285 together with the processed video or audio signal.

As another example, the controller 270 may be provided from an external device, for example, a camera or a camcorder, input through the external device interface unit 230 according to an external device image playback command received through the user input interface unit 250. Video signal or audio signal may be output through the display 280 or the audio output unit 285.

The controller 270 may control the display 280 to display an image. For example, a broadcast image input through the tuner 210, an external input image input through the external device interface unit 230, or an image input through the network interface unit 235, or an image stored in the storage unit 240. May be controlled to be displayed on the display 280.

In this case, the image displayed on the display 280 may be a still image or a video, and may be a 2D image or a 3D image.

Meanwhile, the controller 270 may generate and display a 3D object with respect to a predetermined object in the image displayed on the display 280. For example, the object may be at least one of a connected web screen (newspaper, magazine, etc.), an EPG (Electronic Program Guide), various menus, widgets, icons, still images, videos, and text.

Such a 3D object may be processed to have a depth different from that of the image displayed on the display 280. Preferably, the 3D object may be processed to protrude compared to an image displayed on the display 280.

On the other hand, the control unit 270 recognizes the position of the user based on the image photographed from the photographing unit (not shown). For example, the distance (z-axis coordinate) between the user and the image display apparatus 200 may be determined. In addition, the x-axis coordinates and the y-axis coordinates in the display 280 corresponding to the user position can be identified.

On the other hand, although not shown in the figure, it may be further provided with a channel browsing processing unit for generating a thumbnail image corresponding to the channel signal or the external input signal. The channel browsing processor may receive a stream signal TS output from the demodulator 220 or a stream signal output from the external device interface 230, extract a video from the input stream signal, and generate a thumbnail image. Can be. The generated thumbnail image may be encoded as it is or may be input to the controller 270. In addition, the generated thumbnail image may be encoded in a stream form and input to the controller 270. The controller 270 may display a thumbnail list including a plurality of thumbnail images on the display 280 using the input thumbnail image. In this case, the thumbnail list may be displayed in a simple viewing manner displayed in a partial region while a predetermined image is displayed on the display 280 or in an overall viewing manner displayed in most regions of the display 280. The thumbnail images in the thumbnail list may be updated sequentially.

The display 280 converts an image signal, a data signal, an OSD signal, a control signal, or an image signal, a data signal, a control signal received from the external device interface unit 230 processed by the controller 270 to drive a drive signal. Create

The display 280 may be a PDP, an LCD, an OLED, a flexible display, or the like, and in particular, according to an embodiment of the present invention, a 3D display may be possible.

For viewing the 3D image, the display 280 may be divided into an additional display method and a single display method.

The independent display method may implement a 3D image by the display 280 alone without an additional display, for example, glasses, for example, a lenticular method, a parallax barrier, or the like. Various methods can be applied.

Meanwhile, the additional display method may implement a 3D image by using an additional display in addition to the display 280. For example, various methods such as a head mounted display (HMD) type and glasses type may be applied. In addition, the spectacles type can be further divided into a passive scheme such as a polarized glasses type and an active scheme such as a shutter glass type. On the other hand, the head mounted display type can be divided into passive and active methods.

In the embodiment of the present invention, the 3D additional display 295 focuses on the 3D glasses for stereoscopic image viewing. The 3D glass 295 may include a passive polarized glass or an active shutter glass, and is described with the concept of including the aforementioned head mount type.

The display 280 may be configured as a touch screen and used as an input device in addition to the output device.

The audio output unit 285 receives a signal processed by the controller 270, for example, a stereo signal, a 3.1 channel signal, or a 5.1 channel signal, and outputs a voice signal. The voice output unit 185 may be implemented by various types of speakers.

Meanwhile, in order to detect a gesture of the user, as described above, a sensing unit (not shown) including at least one of a touch sensor, a voice sensor, a position sensor, and a motion sensor may be further provided in the image display apparatus 200. Can be. The signal detected by the sensing unit (not shown) is transmitted to the controller 170 through the user input interface unit 150.

The controller 270 may detect a user's gesture by combining or combining the image photographed by the photographing unit (not shown) or the detected signal from the sensing unit (not shown).

The remote controller 260 transmits a user input to the user input interface unit 250. To this end, the remote control unit 260 may use Bluetooth, RF (Radio Frequency) communication, infrared (IR) communication, UWB (Ultra Wideband), ZigBee (ZigBee) method and the like. In addition, the remote controller 260 may receive an image, an audio or a data signal output from the user input interface unit 250, and display or output the audio from the remote controller 260.

The video display device 200 described above is a fixed type of ATSC (7-VSB) digital broadcasting, DVB-T (COFDM) digital broadcasting, ISDB-T (BST-OFDM) digital broadcasting, and the like. It may be a digital broadcast receiver capable of receiving at least one. In addition, as a mobile type, digital broadcasting of terrestrial DMB system, digital broadcasting of satellite DMB system, digital broadcasting of ATSC-M / H system, digital broadcasting of DVB-H system (COFDM system) and media flow link only system It may be a digital broadcast receiver capable of receiving at least one of digital broadcasts. It may also be a digital broadcast receiver for cable, satellite communications, or IPTV.

On the other hand, the video display device described in the present specification is a TV receiver, a mobile phone, a smart phone (notebook computer), a digital broadcasting terminal, PDA (Personal Digital Assistants), PMP (Portable Multimedia Player), etc. May be included.

Meanwhile, a configuration diagram of the image display device 200 illustrated in FIG. 1 is a configuration diagram for embodiments of the present invention. Each component of the configuration diagram may be integrated, added, or omitted according to the specifications of the image display apparatus 200 that is actually implemented. That is, two or more constituent elements may be combined into one constituent element, or one constituent element may be constituted by two or more constituent elements, if necessary. In addition, the functions performed in each block are intended to illustrate the embodiments of the present invention, and the specific operations and apparatuses do not limit the scope of the present invention.

The video signal decoded by the video display device 200 may be a 3D video signal of various formats. For example, the image may be a 3D image signal including a color image and a depth image, or may be a 3D image signal including a plurality of view image signals. The plurality of viewpoint image signals may include, for example, a left eye image signal and a right eye image signal. Here, the format of the 3D video signal includes a side by side format in which the left eye video signal L and the right eye video signal R are arranged left and right, and top and down arranged up and down. Format, frame sequential arrangement in time division, interlaced format that mixes the left and right eye signals by line, and checker box that mixes the left and right eye signals by box Format and the like.

On the other hand, the image processing apparatus applied to the image display device 200 according to the embodiments of the present invention, the depth for dividing the depth image area (image area having a depth) according to the depth information (or binocular parallax information) from the stereoscopic image A detector; A difference image detector detecting a difference image of a previous and current image frame corresponding to the divided depth image region; The object detector may be configured to detect (determine) an image area having a motion in the difference image as an object, and the depth detector, the difference image detector, and the object detector may be included in the controller 270.

Detailed description of the components of the image processing apparatus according to the embodiments of the present invention will be described with reference to FIGS. 3 to 12.

Hereinafter, an image processing apparatus and a method thereof according to a first embodiment of the present invention will be described with reference to FIGS. 3 to 6. An image processing apparatus and method thereof according to embodiments of the present invention are applied to a mobile terminal such as the mobile communication terminal 100, the telematics terminal, a navigation device, and a 3D video television (3D TV) displaying 3D images. It may also be applied to terminals such as video devices, smart phones, notebook computers, personal digital assistants (PDAs), portable multimedia players (PMPs), projectors, and home theaters.

3 is a block diagram illustrating an image processing apparatus according to a first embodiment of the present invention.

As shown in FIG. 3, the image processing apparatus 400 according to the first exemplary embodiment divides a depth image region (image region having a depth) from a stereoscopic image according to depth information (or binocular disparity information). A depth detector 401; A difference image detector 402 for detecting a difference image of a previous and current image frame corresponding to the divided depth image region; The object detection unit 403 may detect an image area having a motion in the difference image as an object.

Hereinafter, an image processing method according to a first embodiment of the present invention will be described with reference to FIGS. 3 to 6.

4 is a flowchart illustrating an image processing method according to a first embodiment of the present invention.

First, the depth detector 401 divides an image area having a depth (depth image area) from the received stereoscopic image according to image depth information (S11). For example, the depth detector 401 detects a depth image region (or binocular parallax image) from the received stereoscopic image, and detects a histogram (eg, parallax histogram) from the depth image region. The histogram is discrete and has a plurality of local maxima values and local minima values. The depth detector 401 may divide a depth image area for distinguishing distributions of objects through the histogram. Since the histogram is a prior art disclosed in Korean Patent No. 10-0690295, a detailed description thereof will be omitted.

In addition, the depth detector 401 may segment the depth image area by calculating a probability density function according to the prior art. When the depth image area is divided based on the partial minimum values of the probability density functions, it is possible to extract a specific object from each of the divided depth image areas. Since the probability density function is a prior art disclosed in Korean Laid-Open Patent No. 1998-015866, a detailed description thereof will be omitted.

The depth detector 401 may receive a stereoscopic image captured by a stereo camera or a time-of-flight (TOF) camera or a stereoscopic image received from a terminal or a server through a wired or wireless communication network.

5 is an exemplary view illustrating a stereoscopic image for explaining an image processing apparatus according to an exemplary embodiment of the present invention. That is, FIG. 5 is a diagram illustrating an image expressed according to a depth value of an object existing in a stereoscopic image. A general stereoscopic image is composed of objects 5-1 and 5-2 having depth, and the objects 5 The image including -1 and 5-2 is displayed on the display unit (not shown) according to the depth information.

6 is an exemplary diagram illustrating a depth image region divided according to a depth of an object existing in a stereoscopic image according to an exemplary embodiment of the present invention.

Area information on an object existing in each of the divided depth image areas 6-1 and 6-2 may be detected through a connected component analysis (CCA) of each object. The connection element analysis may be implemented in the same manner as the connection element analysis method widely used by those skilled in the art, which is disclosed in Korean Patent Registration No. 10-0836197. Therefore, detailed description of the connection element analysis is omitted.

Since the area (area) information on the detected object includes a wide variety of information including a background image, it is necessary to select a meaningful object (eg, an object having a motion). Accordingly, a process of detecting (selecting) a difference image having a meaningful object (for example, an object that moves) using the difference information between frames corresponding to the divided image area is necessary.

The difference image detector 402 detects a difference image between a previous image frame and a current image frame corresponding to each of the divided depth image regions (S12).

The difference image detector 402 sets a predetermined threshold on the detected difference image (S13). The threshold may be variously set according to the intention of a designer or a user, and the threshold may be a reference value for removing an image region having an object without movement and selecting an image region including the object with movement. It may be set as, or as a reference value for removing an image region having no motion or a small number of objects, and selecting an image region including an object having many movements.

The difference image detector 402 detects (selects) a difference image having a motion based on the predetermined threshold value (S14). For example, the difference image detector 402 determines that there is little or no movement of an object included in the difference image when the difference image is less than or equal to the predetermined threshold value, or that the difference image (or difference image is less than or equal to the predetermined threshold value). The specific image area within the predetermined threshold value in the image is excluded (excluded). On the other hand, the difference image detector 402 determines that there is or is a movement of an object included in the difference image when the difference image exceeds the predetermined threshold value (or difference) that exceeds the predetermined threshold value. Detects (selects) a particular image area within the image that exceeds the predetermined threshold.

The difference image detector 402 may further remove noise of the detected difference image through the median filter. The difference image detector 402 removes noise existing in the detected difference image through the median filter to detect the main object region in the difference image. The median filter removes noise by arranging the values of a given mask area in order of magnitude and selecting a medium value. Since the median filter is a known filter, a detailed description thereof will be omitted.

The object detector 403 receives a difference image (or a difference image from which the noise is removed) detected by the difference image detector 402, and has an image area (or a high-motion image) having motion in the difference image. Area) is detected (S15). For example, the object detector 403 receives a difference image (or a difference image from which the noise is removed) detected by the difference image detector 402, and has an image area (or a motion) within the difference image. The boundary of the image area having a lot of motion is detected. The object detector 403 may determine an image region having a motion in the difference image (or an image region having a lot of movement) as an object having a depth (S16).

Accordingly, the image processing apparatus and the method according to the first exemplary embodiment of the present invention detect a difference image corresponding to a depth image region divided according to depth information from a stereoscopic image, and have an image having a motion in the detected difference image. By detecting a region as an object, it is possible to quickly and accurately track the object (s) in the stereoscopic image.

Hereinafter, an image processing apparatus and a method thereof according to a second exemplary embodiment of the present invention will be described with reference to FIGS. 3 and 7 to 11.

7 is a flowchart illustrating an image processing method according to a second embodiment of the present invention.

First, the depth detector 401 divides an image area (depth image area) having a depth from the received stereoscopic image according to image depth information (S21). For example, the depth detector 401 may segment the depth image area by calculating a probability density function according to the prior art. When the depth image area is divided based on the partial minimum values of the probability density functions, it is possible to extract a specific object from each of the divided depth image areas.

The difference image detector 402 detects a difference image between a previous image frame and a current image frame corresponding to each of the divided depth image regions (S22).

The difference image detector 402 sets a predetermined threshold on the detected difference image. The threshold may be variously set according to the intention of a designer or a user, and the threshold may be a reference value for removing an image region having an object without movement and selecting an image region including the object with movement. It may be set as, or as a reference value for removing an image region having no motion or a small number of objects, and selecting an image region including an object having many movements.

The difference image detector 402 detects (selects) a difference image having a motion based on the predetermined threshold value.

8 is an exemplary view illustrating a difference image detected according to a predetermined threshold value according to embodiments of the present invention.

As shown in FIG. 8, if the difference image is less than or equal to the predetermined threshold value, the difference image detection unit 402 determines that there is little or no movement of an object included in the difference image, and thus the difference image is less than or equal to the predetermined threshold value. (Or a specific image area that is less than or equal to the predetermined threshold value in the difference image) is excluded (excluded). On the other hand, the difference image detector 402 determines that there is or is a movement of an object included in the difference image when the difference image exceeds the predetermined threshold value (or difference) that exceeds the predetermined threshold value. Detects (selects) a specific image area 8-1 that exceeds the predetermined threshold in the image.

The difference image detector 402 may further remove noise of the detected difference image through the median filter.

9 is an exemplary view illustrating a difference image from which noise is removed according to embodiments of the present invention.

As shown in FIG. 9, the difference image detector 402 removes noise existing in the detected difference image through the median filter so as to detect a main object region in the difference image. . The median filter removes noise by arranging the values of a given mask area in order of magnitude and selecting a medium value. Since the median filter is a known filter, a detailed description thereof will be omitted.

The difference image detector 402 may omit the step of detecting a difference image having a motion and / or removing noise of the detected difference image based on the predetermined threshold value.

The object detector 403 receives a difference image (or a difference image from which the noise is removed) detected by the difference image detector 402, and has an image area (or a high-motion image) having motion in the difference image. Area). For example, the object detector 403 receives a difference image (or a difference image from which the noise is removed) detected by the difference image detector 402, and has an image area (or a motion) within the difference image. In operation S23, a boundary of the image area having a lot of motion is detected.

10 is an exemplary diagram illustrating a boundary of an image region in which a motion within a difference image is present according to embodiments of the present invention.

As shown in FIG. 10, a case where an image area larger than an image area having a motion may be detected due to noise that is not removed from the boundary of the image area having a motion in the difference image. Accordingly, the object detector 403 detects a boundary 10-1 of an image region (existing region) having the movement in the difference image.

The object detector 403 reduces the boundary of the image area (existing area) having the movement in the difference image based on the continuity of the difference information of the difference image (S24).

FIG. 11 is an exemplary view illustrating an image region having a motion having a reduced boundary according to embodiments of the present invention.

As illustrated in FIG. 11, the object detecting unit 403 may perform the difference image with respect to an image area (existing area) having a motion in the difference image (for example, inward from an outermost angle of the image area having a motion). By comparing the continuity of the difference information with each other, the boundary of the image region having the motion is reduced. That is, the object detector 403 may select the partial region 11-1 when the partial region 11-1 of the image region (existing region) having motion in the difference image is low or no continuity of the difference information. By removing the boundary, the boundary of the image area (existing area) having the motion in the difference image can be reduced.

The object detector 403 determines the image region 11-2 having the reduced boundary as an object (S25).

Accordingly, the image processing apparatus and the method according to the second exemplary embodiment of the present invention detect a difference image corresponding to a depth image region divided according to depth information from a stereoscopic image, and have an image having a motion in the detected difference image. By reducing the boundary of the region, the object (s) in the stereoscopic image can be tracked more quickly and accurately.

Hereinafter, an image processing apparatus and a method thereof according to a third exemplary embodiment of the present invention will be described with reference to FIGS. 3 and 12.

12 is a flowchart illustrating an image processing method according to a third embodiment of the present invention.

First, the depth detector 401 divides an image area having a depth (depth image area) from the received stereoscopic image according to image depth information (S31). For example, the depth detector 401 may segment the depth image area by calculating a probability density function according to the prior art. When the depth image area is divided based on the partial minimum values of the probability density functions, it is possible to extract a specific object from each of the divided depth image areas.

The difference image detector 402 detects a difference image between a previous image frame and a current image frame corresponding to each of the divided depth image regions (S32).

The difference image detector 402 sets a predetermined threshold on the detected difference image. The threshold may be variously set according to the intention of a designer or a user, and the threshold may be a reference value for removing an image region having an object without movement and selecting an image region including the object with movement. It may be set as, or as a reference value for removing an image region having no motion or a small number of objects, and selecting an image region including an object having many movements.

The difference image detector 402 detects (selects) a difference image having a motion based on the predetermined threshold value. The difference image detector 402 determines that there is little or no movement of an object included in the difference image when the difference image is less than or equal to the predetermined threshold value, so that the difference image (or the predetermined image within the difference image) is less than or equal to the predetermined threshold value. The specific image area below the threshold) is excluded (excluded). On the other hand, the difference image detector 402 determines that there is or is a movement of an object included in the difference image when the difference image exceeds the predetermined threshold value (or difference) that exceeds the predetermined threshold value. Detects (selects) a specific image area 8-1 that exceeds the predetermined threshold in the image.

The difference image detector 402 may further remove noise of the detected difference image through the median filter. The difference image detector 402 removes noise existing in the detected difference image through the median filter to detect the main object region in the difference image. The median filter removes noise by arranging the values of a given mask area in order of magnitude and selecting a medium value. Since the median filter is a known filter, a detailed description thereof will be omitted.

The difference image detector 402 may omit the step of detecting a difference image having a motion and / or removing noise of the detected difference image based on the predetermined threshold value.

The object detector 403 receives a difference image (or a difference image from which the noise is removed) detected by the difference image detector 402, and has an image area (or a high-motion image) having motion in the difference image. Area). For example, the object detector 403 receives a difference image (or a difference image from which the noise is removed) detected by the difference image detector 402, and has an image area (or a motion) within the difference image. In operation S33, a boundary of the video region having a lot of motion is detected.

The boundary of an image region having a motion in the difference image may be detected due to noise that is not removed, such that an image region larger than the image region having a motion may be detected. Accordingly, the object detector 403 detects a boundary of an image region (existing region) having the movement in the difference image.

The object detector 403 reduces the boundary of the image area (existing area) having the movement in the difference image based on the continuity of the difference information of the difference image (S34). For example, the object detecting unit 403 may determine the difference information of the difference image with respect to an image region (existing region) having a motion in the difference image (for example, from the outermost angle of the image region having a movement inward). By comparing the continuity with each other, the boundary of the image region having the movement is reduced. That is, the object detecting unit 403 removes the partial area when the partial area of the image area (existing area) having the motion in the difference image has low or no continuity of the difference information, thereby having the motion in the difference image. The boundary of the image area (existing area) can be reduced.

The object detector 403 examines an inclusion relationship between the image region having the reduced boundary and the objects (object (s) present in the divided depth image region) and includes the image detection unit 403 in the image region having the reduced boundary. You can also decide which objects to track based on the percentage of objects that are tracked. In this case, area information on an object existing in the divided depth image area may be detected through a connected component analysis (CCA) of each object. For example, the object detector 403 determines whether an object is included in the image area having the reduced boundary at a predetermined ratio (for example, 80% or more) (S35), and determines the reduced boundary. The object included in the image region having a predetermined ratio (for example, 80% or more) or more is included in the target object to be tracked (S36).

On the other hand, the object detector 403 may exclude an object included in the image area having the reduced boundary below a predetermined ratio (for example, 80% or more) from the target object to be tracked (S37). Accordingly, the object detector 403 may prevent the object having a background form, which is an image area in which most of the movement is not determined, as a target object to be tracked.

Therefore, the image processing apparatus and the method according to the third embodiment of the present invention, the object (s) in the stereoscopic image by determining the target object to be tracked according to the ratio of the objects included in the image area having the reduced boundary You can also track more quickly and accurately.

As described above, the image processing apparatus and the method according to the embodiments of the present invention detect a difference image corresponding to a depth image region divided according to depth information from a stereoscopic image, and then move a motion within the detected difference image. By detecting an image region having an object as an object, the object (s) in the stereoscopic image can be tracked quickly and accurately.

An image processing apparatus and method according to embodiments of the present invention detect a difference image corresponding to a depth image region divided according to depth information from a stereoscopic image, and boundary of an image region having a motion within the detected difference image. By zooming in, the object (s) in the stereoscopic image can be tracked more quickly and accurately.

The image processing apparatus and the method according to the embodiments of the present invention can determine the target object to be tracked according to the ratio of the objects included in the image area having the reduced boundary, so that the object (s) in the stereoscopic image can be made faster. It can also be tracked accurately.

It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the embodiments disclosed in the present invention are intended to illustrate rather than limit the scope of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be interpreted by the following claims, and all technical ideas within the equivalent scope should be interpreted as being included in the scope of the present invention.

401: depth detector 402: difference image detector
403: object detector

Claims (20)

A depth detector for dividing an image area having a depth from the stereoscopic image according to the depth information;
A difference image detector detecting a difference image of a previous and current image frame corresponding to the divided image region;
And an object detector configured to determine, as an object, an image area having a motion in the detected difference image. The method of claim 1, wherein the depth detector,
And dividing the image region from the stereoscopic image through a probability density function. The method of claim 1, wherein the difference image detector,
And a predetermined threshold value is set in the detected difference image, and a difference image having a motion is detected based on the predetermined threshold value. The method of claim 3, wherein the difference image detector,
And a noise of the difference image having the movement through a median filter. The method of claim 4, wherein the object detector,
And an image region having a motion in the difference image from which the noise is removed is determined as the object. The method of claim 1, wherein the object detector,
And detecting a boundary of the image region having the movement and reducing the boundary of the image region based on the continuity of the difference information of the difference image. The method of claim 6, wherein the object detector,
And an image area having the reduced boundary as the object. The method of claim 6, wherein the object detector,
And determining a target object to be tracked according to a ratio of each object included in the image area having the reduced boundary. The method of claim 8, wherein the object detector,
It is determined whether the object is included in the image area having the reduced boundary by more than a preset ratio, and determining the object included in the image area having the reduced boundary by more than the preset ratio as the target object to be tracked. An image processing apparatus. The method of claim 9, wherein the object detector,
And excluding an object included in the image area having the reduced boundary below the preset ratio from the target object to be tracked. Dividing an image area having a depth from the stereoscopic image according to the depth information;
Detecting difference images of previous and current image frames corresponding to the divided image region;
And determining an image area having a motion in the detected difference image as an object. The method of claim 11, wherein dividing the image area comprises:
And dividing the image region from the stereoscopic image through a probability density function. The method of claim 11, wherein the detecting of the difference image comprises:
Setting a predetermined threshold on the detected difference image;
And detecting a difference image having a motion on the basis of the predetermined threshold value. The method of claim 13,
And removing noise of the difference image having the movement through a median filter. The method of claim 14, wherein determining the object,
And determining, as the object, an image region having a motion in the difference image from which the noise is removed. The method of claim 11,
Detecting a boundary of an image region having the movement;
And reducing the boundary of the image area based on the continuity of the difference information of the difference image. The method of claim 16, wherein determining the object,
And determining the image area having the reduced boundary as the object. The method of claim 16,
And determining a target object to be tracked according to a ratio of each object included in the image area having the reduced boundary. The method of claim 18, wherein determining the target object to track,
Determining whether an object is included in the image area having the reduced boundary by more than a preset ratio;
And determining, as the target object to be tracked, an object included in the image area having the reduced boundary at or above the predetermined ratio. The method of claim 19, wherein the determining of the target object to track,
And excluding an object included in the image area having the reduced boundary below the predetermined ratio from the target object to be tracked.

Images