KR101350068B1 - Electronic device for outputting region of interest image - Google Patents

Abstract

An electronic device of the present invention includes a camera outputting view section images and interest section images corresponding to a part of the view section images according to setting information of an interest section; a first image signal processor setting the interest section in the view section images and outputting the setting information of the interest section; a second image signal processor converting a size or a resolution of the interest section image transmitted from the camera and outputting the converted interest section image; and a display unit displaying the converted interest section image for a user. [Reference numerals] (170) Camera

Description

ELECTRICAL DEVICE FOR OUTPUTTING REGION OF INTEREST IMAGE}

The present invention relates to an electronic device having a camera, and more particularly, to an electronic device for selectively outputting a region of interest automatically set in a pixel array of a camera or set by a user.

The angle of view of a typical camera is about 50 ° (human vision is about 50 °). As a result, a pan / tilt / zoom operation of the camera is performed to capture a subject of interest in a wide background. The pan / tilt / zoom operation may be performed by a person or a separate mechanical device may be mounted on the camera to perform the pan / tilt / zoom operation.

Another method for pan / tilt / zoom operation is the electronic implementation using a wide angle (75 ° or more) lens and a high pixel image sensor.

1 is a view for explaining a mechanical pan / tilt / zoom method.

When implementing a pan / tilt / zoom function mechanically in a camera that targets 1080p (1920 × 1080) output, for example, an FHD pixel (1920 × 1080) sensor and a 45 ° angle of view lens are mounted on the camera. In this case, the camera captures the view area 11 in the background 10 and outputs it as the image 13. The user moves the view area 11 of the camera mechanically (or manually) in the direction of the up, down, left, and right arrows 12. The zoom function may be implemented by attaching a separate lens.

2 is a diagram for explaining an electronic pan / tilt / zoom method.

When implementing a pan / tilt / zoom function electronically in a camera that targets 1080p (1920 × 1080) output, for example, an 8M pixel sensor (3264 × 2448) and a lens with an angle of view of 75 ° are mounted on the camera. In this case, the camera captures the view area 21 in the background 20 and outputs the region of interest 22 as the image 23. The region of interest 22 may be moved up and down ± 14.4 °, left and right ± 14.2 °, and the zoom function may implement X1.7 times without using a separate lens.

In the electronic pan / tilt / zoom method, the wider the angle of view, the larger the pan / tilt angle, and the higher the image sensor, the higher the magnification. The implementation of the zoom function here refers to a pseudo or virtual zoom method, not a digital zoom in which quality deterioration occurs.

For example, the digital zoom method implements a zoom function by generating 320 × 240 pixels and generating a missing pixel with reference to surrounding pixels in a camera that outputs 640 × 480. In the digital zoom method, there is a difference in image quality depending on an algorithm for generating insufficient pixels.

In implementing the pan / tilt / zoom, the camera cannot detect the entire background in the mechanical method, so the user must directly grasp the subject of interest in the entire background and control the camera's view area to move to the area of the subject. .

However, the electronic method allows the camera to detect a significant portion of the entire background, automatically identifying the subject of interest (based on a computer-implemented algorithm), and allowing the region of interest (that is, the output region) You can control it to move.

In order to implement an automatic control of the output area in the electronic pan / tilt / zoom method, the camera must perform image processing of the view area and the ROI together. Common CMOS cameras include CMOS Image Sensors (CIS) and Image Signal Processors (ISPs). Due to the electronic pan / tilt / zoom method, high pixel CIS should be used. Since the image processing of the view region and the region of interest must be performed together, it requires a very high performance.

3 is a view for explaining the operation of the image signal processor in the electronic pan / tilt / zoom method.

The ISP 32 processes the view region 33 output from the CIS 31, detects a subject of interest in the view region 33, and detects the region of interest 34 including the subject of interest in the view region 33. And the image 36 obtained by converting the ROI image 35.

The ISP has to do much more processing compared to the role that only handles the view area output from the existing CIS. Therefore, even if a high-performance ISP is applied, the output speed is considerably lowered. In particular, considering that at least 30 fps is recommended for the image output from the ISP for the motion recognition processing, the overload of the ISP is a serious problem.

Functions such as screen output and motion recognition are functions that require a high frame rate for quality and recognition processing. However, as the amount of information in the CIS increases and functions such as object recognition and distortion correction are added, it is difficult to implement a high frame rate unless the ISP is a high performance device. By the way, the amount of sensing information of CIS is increasing very rapidly (eg, 8M, 13M, 20M, etc.), but the amount of expression information of image output devices (TV, monitor, etc.) is increasing slowly (eg, 720p, 1080p). Etc). By using these differences, only a specific region of interest may be output from the detectable region (view angle) of the CIS to indicate the effect of zoom and camera angle of view movement. In addition, since an object for motion recognition or object recognition processing does not always occupy the entire field of view, it is possible to increase the efficiency of image processing by processing only a specific region of interest in which the object exists. In order to find the region of interest of such output and recognition, it is necessary to search the entire region, if not the high frame rate. However, in the related art, a very high performance ISP is required, and therefore, the price of the entire system is high.

It is an object of certain embodiments of the present invention to at least partially solve, alleviate or eliminate at least one of the problems and / or disadvantages associated with the prior art.

One object of the present invention is to reduce the bottleneck of the entire system by distributing load concentration on the image signal processor when the pan / tilt / zoom is automatically controlled electronically.

An electronic device according to an aspect of the present invention includes a camera for outputting a region of interest image corresponding to a portion of the view region image according to a view region image and setting information of the region of interest; A first image signal processor configured to set a region of interest in the view region image and output setting information of the region of interest; A second image signal processor configured to convert a size or resolution of the ROI image received from the camera and output the converted ROI image; And a display unit configured to display the converted ROI image to the user.

According to the present invention, when the pan / tilt / zoom is automatically controlled electronically, it is possible to reduce the bottleneck of the entire system by distributing load concentration on the image signal processor, and by using a plurality of entry-level image signal processors. The advantage is that you can implement the functionality you want.

1 is a view for explaining a mechanical pan / tilt / zoom method,
2 is a view for explaining an electronic pan / tilt / zoom method,
3 is a view for explaining the operation of the image signal processor in the electronic pan / tilt / zoom method,
4 is a diagram illustrating a configuration of an electronic device according to an embodiment of the present disclosure;
5 is a diagram illustrating a configuration of an image sensor;
6 is a view for explaining a method of driving the image sensor;
7 is a view for explaining a circuit configuration of a driving unit according to an embodiment of the present invention;
8 is a signal diagram illustrating a method of outputting a region of interest image according to an embodiment of the present invention;
9 is a main configuration diagram illustrating a method of outputting a region of interest image according to an embodiment of the present invention;
10 is a main configuration diagram for describing a method of outputting a region of interest image according to another embodiment of the present invention;
11 and 12 are diagrams for describing a video call method using the electronic device of the present invention.

As the inventive concept allows for various changes and numerous embodiments, particular embodiments will be described in detail with reference to the accompanying drawings. It should be understood, however, that the invention is not intended to be limited to the particular embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention.

Terms including ordinals, such as first, second, etc., may be used to describe various elements, but the elements are not limited to these terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component. And / or < / RTI > includes any combination of a plurality of related listed items or any of a plurality of related listed items.

When a component is referred to as being "connected" or "connected" to another component, it may be directly connected to or connected to that other component, but it may be understood that other components may be present in between. Should be. On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between.

The terminology used in this application is used only to describe a specific embodiment and is not intended to limit the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprise" or "have" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in commonly used dictionaries are to be interpreted as having a meaning consistent with the meaning in the context of the relevant art and are to be construed as ideal or overly formal in meaning unless explicitly defined in the present application Do not.

4 is a diagram illustrating a configuration of an electronic device according to an exemplary embodiment of the present disclosure.

The electronic device 100 may include a sensor unit 110, a memory 120, a user interface 130, a communication unit 140, first and second image signal processors (ISPs 160, 165) and a camera. And a display unit 150.

The electronic device 100 may perform an image display operation by interworking with an external device, and the electronic device 100 may transmit an image captured by the camera 170 to an external device through a network. For example, the electronic device 100 may operate under the control of an external device, and the control by the external device may be implemented in various ways. Examples of the network include, but are not limited to, a local area network (LAN), a wireless local area network (WLAN), a wide area network (WAN), the Internet, and a small area network (Small). Area Network (SAN). The electronic device 100 may also be connected to an external device through the network. The electronic device 100 may be directly connected to an external device or through the network. The external device is used to play back and display an image captured by the camera 170 of the electronic device 100. Examples of the external device connected to the electronic device 100 through the network include, but are not limited to, a Youtube player, an iTunes player, an MPlayer player and a bareshare player. (Bearshare player), TV, mobile phone and the like.

The electronic device 100 may be any device having a camera, and the electronic device 100 and the external device may be a smartphone, a mobile phone, a game machine, a TV, a display device, a vehicle head unit, a laptop, a laptop, a tablet, respectively. (Tablet) PC, PMP (Personal Media Player), PDA (Personal Digital Assistants) and the like. The electronic device 100 may be implemented as a portable communication terminal having a wireless communication function.

The communication unit 140 is provided for a direct connection with an external device or a connection through a network, and may be a wired or wireless communication unit, and wires data from the controller 180, the memory 120, or the camera 170. Alternatively, wirelessly, or wired or wirelessly receives data from an external communication line or the air and transmits the data to the controller 180 or stores the data in the memory 120.

The communication unit 140 may include at least one of a mobile communication module, a wireless LAN module, and a short range communication module according to performance. Examples of the communication unit 140 include, but are not limited to, an integrated services digital network (ISDN) card, a modem, a local area network (LAN) card, an infrared module, a Bluetooth module, and a zigbee. Module and wireless module.

The mobile communication module allows the electronic device 100 to be connected to an external device through mobile communication using at least one, one, or a plurality of antennas (not shown) under the control of the controller 180. The mobile communication module uses a mobile phone (not shown), a smart phone (not shown), a tablet PC or another device (not shown) having a phone number input to the electronic device 100 to make a voice call, a video call, or a text message. Send / receive radio signals for SMS or multimedia messages.

The WLAN module may be connected to the Internet at a place where a wireless access point (not shown) is installed under the control of the controller 180. The wireless LAN module supports the IEEE 802.11x standard of the Institute of Electrical and Electronics Engineers (IEEE). The short range communication module may wirelessly perform short range communication between the electronic device 100 and an image forming apparatus (not shown) under the control of the controller 180. The local area communication method may include bluetooth, infrared data association (IrDA), and the like.

The user interface 130 may include a plurality of buttons, a microphone, a speaker, a vibration motor, a connector, a keypad, and the like as a means for receiving a user input or informing the user of information. Examples of the user interface 130 include, but are not limited to, a cursor control such as a mouse, trackball, joystick, or cursor direction keys communicating information with the controller 180 and on the display 150. It may be provided for controlling the cursor movement.

The button may be formed on the front, side, or back of the electronic device 100, and may include a power / lock button (not shown), a volume button (not shown), a menu button, a home button, and a back button (back button). ) And a search button.

The microphone receives an audio or sound under the control of the controller 180 to generate an electrical signal.

The speaker may output sound corresponding to various signals (eg, a wireless signal, a broadcast signal, a digital audio file, a digital video file, or a photo capture) to the outside of the electronic device 100 under the control of the controller 180. The speaker may output a sound corresponding to a function performed by the electronic device 100. One or more speakers may be formed at appropriate locations or locations of the electronic device 100.

The vibration motor may convert an electrical signal into mechanical vibration under the control of the controller 180. For example, when the electronic device 100 in the vibration mode receives a voice call from another device (not shown), the vibration motor operates. One or more vibration motors may be formed in the electronic device 100. The vibrating motor may operate in response to a user's touch action on the touchscreen and the continuous movement of the touch on the touchscreen.

The connector may be used as an interface for connecting the electronic device 100 to an external device or a power source (not shown). Under the control of the controller 180, data stored in the memory 120 of the electronic device 100 may be transmitted to an external device or may receive data from an external device through a wired cable connected to the connector. A wired cable connected to the connector allows power to be input from the power source or charge the battery.

The keypad may receive a key input from a user for controlling the electronic device 100. The keypad includes a physical keypad formed on the electronic device 100 or a virtual keypad displayed on a touch screen.

The sensor unit 110 includes at least one sensor that detects a state (position, orientation, movement, etc.) of the electronic device 100. For example, the sensor unit 110 may be a proximity sensor that detects whether a user approaches the electronic device 100, or an operation of the electronic device 100 (eg, rotation, acceleration, Motion sensors for detecting deceleration, vibration, etc.). In addition, the motion sensor may include an acceleration sensor, a gravity sensor, an impact sensor, a GPS, a compass sensor, an acceleration sensor, and the like. The sensor unit 110 may detect a state of the electronic device 100, generate a signal corresponding to the detection, and transmit the signal to the controller 180. For example, the GPS module receives radio waves from a plurality of GPS satellites (not shown) on Earth orbit, and sets a time of arrival from the GPS satellites (not shown) to the electronic device 100. The location of the electronic device 100 may be calculated by using the same.

The camera 170 includes a lens system 174, a driver 172, and an image sensor 176, and may further include a flash. The camera 170 converts and outputs an optical signal input (or photographed) through the lens system 174 into an electrical image signal or data, and a user may capture a moving image or a still image through the camera 170. . In other words, the camera 170 forms an optical image of the subject and detects the formed optical image as an electrical signal.

The lens system 174 forms an image of a subject by converging light incident from the outside. The lens system 174 may include at least one lens, and each lens may be a convex lens, an aspheric lens, or the like. The lens system 174 is symmetric about an optical axis passing through its center, and the optical axis is defined as this central axis. The image sensor 176 is defined by external light incident through the lens system 174. The formed optical image is detected as an electrical image signal.

The image sensor 176 may include a plurality of pixels arranged in an M × N matrix structure, and the pixels may include a photodiode and at least one transistor. The pixel accumulates (i.e., exposes) the charge generated by the incident light upon application of the on-signal from the horizontal scanner, outputs the voltage by the accumulated charge (i.e., outputs the pixel data), and The voltage represents the illuminance of the incident light. This exposure and output process is performed by the operation of one pixel.

In the case of processing an image constituting a still image or a moving image, the image data output from the image sensor 176 is composed of a set of voltages (ie, pixel values) output from the pixels, and the image The data represents one image (ie still image). According to an embodiment, the image sensor 176 outputs view area image data including M × N pixels to the first image signal processor 160 and includes P × Q pixels corresponding to some of the M × N pixels. The ROI image data is output to the second image signal processor 165. In this case, the view area may be referred to as the entire area. In another embodiment, the image sensor 176 stores view image data composed of M × N pixels in the memory 120. The image sensor 176 may be a charge-coupled device (CCD) image sensor, a complementary metal-oxide semiconductor (CMOS) image sensor, or the like.

The driver 172 drives the image sensor 176 under the control of one of the first and second image signal processors 160 and 465 and the controller 180. The driver 172 operates all the pixels of the image sensor 176 according to the first control signal received from the first image signal processor 160, and performs a first operation on the view area image data output from the all pixels. Output to the image signal processor 160. In addition, the driver 172 operates only pixels of the ROI among all the pixels of the image sensor 176 according to the second control signal received from the first image signal processor 160, and controls the pixels of the ROI. The region of interest image data output from the output signal is output to the second image signal processor 165. In this example, although the first image signal processor 160 controls the driver 172, the driver 172 may be directly controlled by the controller 180.

The display unit 150 displays an image input from any one of the first and second image signal processors 160 and 165 and the controller 180 on the screen. The display unit 150 may be a liquid crystal display (LCD), a touch screen, or the like. The touch screen displays an image according to the control of the controller 180, generates a key contact interrupt when a user input means such as a finger, a stylus pen, or the like comes into contact with the surface of the touch screen. Under the control of 180, user input information including input coordinates and an input state is output to the controller 180.

The touch screen may provide a user with a graphic user interface corresponding to various services (eg, call, data transmission, broadcasting, photographing / video recording). The touch screen may transmit user input information corresponding to at least one touch input to the graphic user interface to the controller 180. The touch screen can receive at least one touch via a user's body (e.g., a finger including a thumb) or a touchable input means (e.g., a stylus pen). Also, the touch screen may receive a continuous movement of one touch among at least one touch. The touch screen may transmit user input information corresponding to continuous movement of an input touch to the controller 180.

In the present invention, the touch is not limited to the contact between the touch screen and the user's body or touchable input means, and includes non-contact (for example, a detectable distance between the touch screen and the user's body or touchable input means is 1 mm or less). can do. The touch screen may be implemented by, for example, a resistive method, a capacitive method, an infrared method, or an acoustic wave method.

The memory 120 includes databases related to images, user information, documents, etc. for providing applications of various functions such as a video call, a game, and the like, and a graphical user interface (GUI) related thereto, and the electronic device ( Background images (menu screen, standby screen, etc.) or operating programs, images captured by a camera, data related to a method of outputting a region of interest image, etc. necessary for driving 100 may be stored. The memory 120 is a machine readable medium, and the term machine readable medium may be defined as a medium for providing data to the machine so that the machine can perform a specific function. The machine-readable medium may be a storage medium. The memory 120 may include non-volatile media and volatile media. All such media must be tangible so that the instructions carried by the media can be detected by a physical mechanism that reads the instructions into the machine.

Examples of such machine-readable media include, but are not limited to, floppy disks, flexible disks, hard disks, magnetic tapes, compact disc read-only memory (CD-ROM), Optical discs, punchcards, papertape, RAM, Programmable Read-Only Memory (PROM), Erasable PROM (EPROM) and FLASH-EPROM.

The controller 180 executes an application according to user input information, and the application performs a program operation according to the user input information. At this time, the user input includes an input through a keypad, a touch screen, or the like, or a camera-based input. The controller 180 may include a bus for information communication and a processor connected to the bus for information processing. The controller 180 may also include a second memory (eg, random access memory (RAM)) coupled with the bus to store information required by the processor. The second memory may be used to store temporary information required by the processor. The electronic device 100 may further include a read only memory (ROM) connected to the bus to store static information required by the processor. The controller 180 controls the overall operation of the electronic device 100 as a central processing unit and performs a method of outputting a region of interest image according to the present invention. The controller 180 controls the first and second image signal processors 160 and 165 to output an image of the ROI according to user input information or a default setting.

Each of the first and second image signal processors 160 and 165 controls the image input from the camera 170 or the image stored in the memory 120 on a frame basis under the control of the controller 180. Process.

The first image signal processor 160 is used to set a region of interest in the viewing area, and the second image signal processor 165 is adapted to fit the screen characteristics (size, image quality, resolution, etc.) of the display unit 150. Output the transformed ROI image.

The second image signal processor 165 may output the view region image converted to match the screen characteristics (size, image quality, resolution, etc.) of the display unit 150.

5 is a diagram illustrating a configuration of an image sensor. 5 illustrates a pixel array 210 of an image sensor 176 constituting the camera 170 and a Bayer color filter 220 disposed on the pixel array 210. Each pixel 212 outputs a value corresponding to the brightness of incident light. The image sensor 176 may be configured so that each pixel detects red (R), green (G), and blue (B), but for each pixel to detect only one color using a color filter due to a price increase or the like. It is customary to construct. The Bayer color filter 220 has filter units of three colors (ie, R, G, and B), and the filter units of each color are arranged one across the pixel for each of the row direction and the column direction. In addition, the filter units correspond one-to-one with the pixels. For example, the R channel passing through the R filter unit has a red color, and the corresponding pixel aligned to correspond to the R filter unit detects the R channel. The Bayer color filter 220 may be selectively provided. Hereinafter, although the pixel array 210 is described, the pixel array 210 itself may be regarded as the image sensor 176.

The first and second image signal processors 160 and 165 may be integrated into the controller 180. In the following description, a method of outputting a region of interest image according to the present invention may include the first image signal processor 160. It will be described as controlling the drive unit 172.

6 is a view for explaining a method of driving the image sensor.

The driver 172 applies signals to the image sensor 176 according to a first control signal for outputting a view area image input from the first image signal processor 160 and from the image sensor 176. Read the view area image data. In addition, the driver 172 applies signals to the image sensor 176 according to a second control signal for outputting the ROI image input from the first image signal processor 160, and the image sensor 176. The region of interest image data is read. The driver 172 may include a first horizontal scanner 320 and a first vertical scanner 310 for outputting a view area image, and a second horizontal scanner for outputting an ROI image. 325, and a second vertical scanner 315. The first horizontal scanner 320 supplies signals that are sequentially turned on to the first horizontal lines 322 corresponding to the view area, and the second horizontal scanner 325 to the second horizontal lines corresponding to the ROI. The signals 327 are sequentially supplied. The first vertical scanner 310 sequentially reads pixel data from the first vertical lines 312 corresponding to the view area, and the second vertical scanner 315 reads the second vertical lines corresponding to the ROI. Pixel data is read out sequentially from 317. This sequential reading of pixel data is called a scan.

The pixel array 210 includes a plurality of pixels 212, and each pixel 212 is connected to corresponding horizontal lines 322 and 327 and vertical lines 312 and 317. Each of the pixels 212 is exposed when an on-signal is applied through the horizontal lines 322 and 327, and the pixels are generated by the exposure according to the operation of the signal analysis device connected to the vertical lines 312 and 317. Output the data.

7 is a diagram illustrating a circuit configuration of a driving unit according to an exemplary embodiment of the present invention. In FIG. 7, only three rows and four columns of pixels 212 of the image sensor 176 are illustrated. For example, the pixels 212 in three rows and one column are referred to as 3-1 pixels.

The first horizontal scanner 320 includes the first signal applying elements 410 by the number of all rows of the pixels 212 constituting the image sensor 176. For example, the first signal applying element 410 corresponding to the third pixel row is called an H1-3 element. The first signal applying elements 410 are connected one-to-one with the first horizontal lines 322. The pixel row refers to the set of pixels forming one row, and the pixel column refers to the set of pixels forming one column.

The second horizontal scanner 325 includes the second signal applying elements 420 by the number of all rows of the pixels 212 constituting the image sensor 176. For example, the second signal applying element 420 corresponding to the third pixel row is called an H2-3 element. The second signal applying elements 420 are connected one-to-one with the second horizontal lines 327.

Each pixel 212 operates for one unit of time, and each pixel row operates for one cycle. In this case, one cycle corresponds to a time obtained by multiplying the number of pixels constituting one pixel row by a unit time.

The first horizontal scanner 320 sequentially applies signals that are sequentially cycled to pixel rows of the view area.

The second horizontal scanner 325 sequentially applies signals that are sequentially cycled to pixel rows of the ROI.

The first vertical scanner 310 includes the first signal analyzing elements 430 by the number of all columns of the pixels 212 constituting the image sensor 176. For example, the first signal analysis element 430 corresponding to the third pixel column is called a V1-3 element. The first signal analysis elements 430 are connected one-to-one with the first vertical lines 312.

The second vertical scanner 315 includes the second signal analysis elements 440 by the number of all columns of the pixels 212 constituting the image sensor 176. For example, the second signal analysis element 440 corresponding to the third pixel column is referred to as a V2-3 element. The second signal analysis elements 440 are connected one-to-one with the second vertical lines 317.

The first vertical scanner 310 sequentially reads pixel data from the pixel columns of the view area in cycle units.

The second vertical scanner 315 sequentially reads pixel data from the pixel columns of the ROI in units of cycles.

8 is a signal diagram illustrating a method of outputting a region of interest image according to an exemplary embodiment.

In this example, an example of setting a region of interest (corresponding to 2-2 pixels) at a size of 1/12 of a view region and outputting a region of interest image at 30 fps is illustrated.

The frame rate of the view area is calculated as shown below, and in this example, the frame rate of the view area image is 2.5 fps.

In FIG. 8, the region of interest is smaller than the view region, Data1 represents view region image data, and Data2 represents region of interest image data. As the ratio of the region of interest in the view region increases, the frame rate of the view region becomes faster.

The time points t1 to t12 are spaced at unit time intervals, and four unit times form one cycle. Each of the H1-1 to H1-3 devices operates to output a high signal for one cycle, and the H1-1 to H1-3 devices operate sequentially in cycle units. Each of the V1-1 to V1-4 devices operates to read corresponding pixel data at a corresponding point in time, and the V1-1 to V1-4 devices operate sequentially in unit time units.

The H2-2 device operates to output a high signal at time points t1 to t12, and the V2-2 device outputs pixel data of the second pixel column (as a result, pixel data of 2-2 pixels) at time points t1 to t12. Works to read

At the time t1, the H1-1 device applies a high signal to the first pixel row, the V1-1 device reads pixel data of the first pixel column (as a result, pixel data of 1-1 pixels), and H2. The -2 device applies a high signal to the second pixel row, and the V2-2 device reads pixel data (consequently pixel data of 2-2 pixels) in the second pixel column.

At the time t6, the H1-2 device applies a high signal to the second pixel row, the V1-2 device reads pixel data of the second pixel column (as a result, pixel data of 2-2 pixels), and H2. The -2 device applies a high signal to the second pixel row, and the V2-2 device reads pixel data (consequently pixel data of 2-2 pixels) in the second pixel column.

9 is a block diagram illustrating a method of outputting a region of interest image according to an exemplary embodiment.

The camera 170 generates a view area image 510 and outputs the view area image 510 to the first image signal processor 160 at a low speed first frame rate (eg, 2.5 fps).

The first image signal processor 160 automatically detects a subject from the view area image 510, sets a region of interest 520 including the subject, and sets coordinates of the set region of interest 520 to the camera 170. )

The camera 170 generates a region of interest image 530 based on the coordinates of the region of interest 520, and converts the region of interest image 530 to a second frame rate (eg, 30 fps) at a high speed. The image signal is output to the processor 165.

The second image signal processor 165 processes (or converts) the ROI image 530 to match the screen characteristics (size, image quality, resolution, etc.) of the display unit 150, and processes the ROI image 540 processed. ) Is output to the display unit 150. The second image signal processor 165 may reduce the resolution or size of the ROI image 530 received from the camera 170.

Unlike the present example, in order to avoid the constraints of Equation 1, the view area image and the ROI image may be output in a time division manner.

For example, assuming that the camera 170 outputs 30 frames per second, the first frame corresponding to the view region image is output to the first image signal processor 160, and the second frame corresponding to the ROI images is output. The 30 th frame may be output to the second image signal processor 165.

FIG. 10 is a diagram illustrating an example of a method of outputting a region of interest image according to another exemplary embodiment.

The camera 170 generates a view area image 610 and stores the view area image 160 in the memory 120 at a high speed second frame rate (eg, 30 fps).

The first image signal processor 160 reads the view area image 610 from the memory 120 at a low speed third frame rate (eg, 5 fps), and automatically the subject in the view area image 610. Detects an image, sets a region of interest 620 including the subject, and stores the coordinates of the region of interest 620 in the memory 120 or outputs the coordinates of the region of interest 620 to the second image signal processor 165.

The second image signal processor 165 converts the region of interest image 630 (ie, a portion of the view region image) from the memory 120 based on the coordinates of the region of interest 620 to a high-speed second frame rate (eg, For example, at 30fps, the ROI image 630 is processed (or converted) to fit the screen characteristics (size, image quality, resolution, etc.) of the display unit 150, and the processed ROI image 640 is processed. Is output to the display unit 150. The second image signal processor 165 may reduce the resolution or size of the ROI image 630 received from the memory 120. Unlike the present example, the second image signal processor 165 may read the view area image 610 from the memory 120 and extract the region of interest image 630 from the view area image 610.

11 and 12 are diagrams for describing a video call method using the electronic device of the present invention.

11 shows a view area image 710 captured by the camera 170. Since the default view angle of the camera 170 is large, the background occupies most of the view area image 710 in addition to the user who wants to make a video call. The controller 180 may include a driving unit 172 and a second unit so that the second image signal processor 165 outputs an image of the region of interest 720 when a user's video call command is input through the user interface 130. The first and second image signal processors 160 and 15 are controlled.

The present ROI setting method is described for one image, but the present method may be applied to images sequentially input or read. In addition, the method may be performed on an image input in real time from the camera 170 while the electronic device 100 performs a one-to-one video call or a multi-party video conference, or input in real time through the camera 170 in another application mode. Can be performed on the image being imaged.

The first image signal processor 160 determines whether the face of the user is included in the view area image 710 and sets the region of interest 720 to include the face part if the face of the user is included. Here, face detection is performed using a conventional face detection method, and a face extraction technique using a contour of a face, a color and / or texture of a face skin, a template, or the like may be used. For example, the first image signal processor 160 may perform face learning through a plurality of face images and detect a face of the user from the input view area image 710 using the accumulated face learning data. .

Thereafter, the second image signal processor 165 or the controller 180 displays the video call screen 810 as shown in FIG. 12 to the user, and displays the ROI image 820 on the screen 810. .

It will be appreciated that embodiments of the present invention may be implemented in hardware, software, or a combination of hardware and software. Such arbitrary software may be stored in a memory such as, for example, a volatile or non-volatile storage device such as a storage device such as ROM or the like, or a memory such as a RAM, a memory chip, a device or an integrated circuit, , Or a storage medium readable by a machine (e.g., a computer), such as a CD, a DVD, a magnetic disk, or a magnetic tape, as well as being optically or magnetically recordable. It will be appreciated that a memory that can be included in an electronic device is an example of a machine-readable storage medium suitable for storing a program or programs including instructions for implementing embodiments of the present disclosure. Accordingly, the present invention includes a program comprising code for implementing the apparatus or method described in any claim herein and a machine-readable storage medium storing such a program. In addition, such a program may be electronically transported through any medium such as a communication signal transmitted via a wired or wireless connection, and the present invention appropriately includes the same.

In addition, the electronic device may receive and store the program from a program providing device connected by wire or wirelessly. The program providing apparatus may include a program including instructions for causing the electronic device to perform a preset method of outputting a region of interest image, a memory for storing information necessary for a method of outputting a region of interest image, and a wired line with the electronic device. Or a communication unit for performing wireless communication, and a controller for automatically transmitting a request or a corresponding program to the electronic device.

While the present invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments. Therefore, the scope of the present invention should not be defined by the described embodiments, but should be determined by the equivalent of claims and claims.

100: electronic device, 110: sensor unit, 120: memory, 130: user interface, 140: communication unit, 150: display unit, 160: first image signal processor, 165: second image signal processor, 170; Camera, 172: driving unit, 174: lens system, 176: image sensor, 180: control unit

Claims (6)

In an electronic device,
A camera for outputting a region of interest image corresponding to a part of the view region image according to a view region image and setting information of the region of interest;
A first image signal processor configured to set a region of interest in the view region image and output setting information of the region of interest;
A second image signal processor configured to convert a size or resolution of the ROI image received from the camera and output the converted ROI image;
A display unit configured to display the converted region of interest image to a user;
The camera comprises:
An image sensor having a plurality of pixels arranged in a matrix structure;
And a driving unit configured to output the view region image by operating pixels corresponding to the view region among the plurality of pixels, and to output the region of interest image by operating pixels corresponding to the ROI among the plurality of pixels. An electronic device for outputting a region of interest image. delete The driving apparatus according to claim 1,
A first horizontal scanner for applying signals on pixels corresponding to the view area;
A first vertical scanner which reads pixel data from pixels in said view area;
A second horizontal scanner configured to apply on signals to the pixels corresponding to the ROI among the plurality of pixels;
And a second vertical scanner configured to read pixel data from the pixels of the ROI. The method of claim 3,
The first horizontal scanner includes first signal applying elements each connected to rows of pixels in the view area,
The first vertical scanner includes first signal analysis elements each connected to columns of pixels of the view area,
The second horizontal scanner includes second signal applying elements each connected to rows of pixels of the ROI,
And the second vertical scanner includes second signal analysis elements connected to columns of pixels of the ROI, respectively. In an electronic device,
A camera for outputting a view area image;
A memory for storing the view area image;
A first image signal processor for receiving the view area image from the memory, setting a region of interest in the view region image, and outputting setting information of the region of interest;
A second image that receives a region of interest image corresponding to a portion of the view region image from the memory, converts a size or resolution of the region of interest image, and outputs the converted region of interest image according to setting information of the region of interest; A signal processor;
A display unit configured to display the converted region of interest image to a user;
And the memory outputs the view region image at a first frame rate, and outputs the region of interest image at a second frame rate that is greater than the first frame rate. 6. The method according to claim 1 or 5,
The first image signal processor detects a face of the user from the view region image, sets a portion of the view region image including the face of the user as the region of interest,
And the display unit displays the ROI image on a video call screen.

Images