KR20150086385A - Object of interest based image processing - Google Patents

Abstract

Apparatus, method and system are provided for encoding and / or compressing one or more objects of interest within individual image frames with bit densities higher than the bit density used to encode and / or compress their background Encoding engine. The system may further include a status engine for identifying an area of interest that includes at least a portion of one or more objects of interest, and for scaling the area of interest within the individual image frames to highlight the objects of interest.

Description

[0001] OBJECT OF INTEREST BASED IMAGE PROCESSING [0002]

Field of the Invention [0002] The present invention relates generally to data processing, and more specifically, to object-based image processing of interest.

Unless otherwise indicated herein, the contents set forth in this section are not prior art to the claims of the present application and are not to be construed as prior art by the inclusion therein.

Generally, imaging processing applications, specifically video conferencing solutions, can receive or capture live video image frames from one or more cameras, compress captured video image frames, and send the compressed video image frames to one or more recipients, The one or more recipients may then decompress the received video image frames. Today, more and more image processing applications, such as video conferencing, are implemented using mobile devices over networks that may have a relatively lower bandwidth than conventional networks configured to support applications using fixed devices. Thus, the quality of the transmitted video and the user experience of these applications may not be ideal.

BRIEF DESCRIPTION OF THE DRAWINGS Embodiments of the present invention will now be described, by way of example only, with reference to the accompanying drawings, in which: FIG.
Figure 1 is a block diagram illustrating an exemplary image processing apparatus in accordance with various embodiments of the present invention.
2 is a block diagram illustrating exemplary individual image frames of a video signal in accordance with various embodiments of the present invention.
3 is a flow diagram illustrating a portion of the operations of an encoding engine of an image processing apparatus in accordance with various embodiments of the present invention.
4 is a flow diagram illustrating a portion of the operations of the context engine of an image processing apparatus according to various embodiments of the present invention.
5 is a flow diagram illustrating a portion of the operations of an image processing apparatus in accordance with various embodiments of the present invention.
Figure 6 is a block diagram illustrating an exemplary computer system suitable for use in implementing various aspects of the methods and / or devices described in accordance with embodiments of the present invention.
7 is a block diagram illustrating an article of manufacture having programming instructions configured to cause the apparatus to perform various operations of the image processing apparatus, in accordance with embodiments of the present invention.

Various embodiments of the present invention include an encoding engine configured to encode and / or compress one or more objects of interest within individual image frames with a bit density higher than the bit density used to encode and / or compress the background of image frames Desc / Clms Page number 5 > devices, methods and systems. The image processing system may further comprise a status engine configured to identify a region of interest that includes at least a portion of the one or more objects of interest, and to scale the region of interest within each frame to highlight the objects of interest.

In various embodiments, the apparatus receives a plurality of image frames of a video signal and, based on one or more bit densities higher than the bit density used to encode and / or compress the background of each image frame, The image processing system may include an encoding engine configured to encode and / or compress one or more regions associated with one or more objects of interest in the background, and wherein the background and one or more regions form respective image frames. The apparatus may further comprise a transmitter coupled to the encoding engine and configured to transmit a plurality of encoded / compressed image frames to one or more receivers.

In various embodiments, the one or more objects of interest may include one or more faces of the participants of the video conference.

In various embodiments, the apparatus may further comprise a context engine coupled to the encoding engine and configured to identify one or more objects of interest within each image frame.

In various embodiments, the context engine may be further configured to identify a region of interest within each image frame, wherein the region of interest may include, at least in part, one or more objects of interest.

In various embodiments, the context engine may be further configured to scale up the region of interest within each image frame, thereby increasing the saliency of the region of interest within each image frame.

In various embodiments, the context engine may be further configured to adjust the region of interest and center at least one of the one or more objects of interest in each image frame.

In various embodiments, the context engine may be further configured to adjust the region of interest based on contextual information to place at least one of the one or more objects of interest at off-center locations within each image frame.

In various embodiments, one or more objects of interest of the device may include one or more faces of participants of a video conference, and the contextual information may include one or more face orientations.

In various embodiments, the transmitter may be further configured to separately transmit one or more objects of interest and a background.

In various embodiments, the method includes receiving a plurality of image frames of a video signal; And encoding and / or compressing one or more regions associated with one or more objects of interest in each image frame based on one or more bit densities higher than the bit density used to encode and / or compress the background of each image frame And the background and at least one region form respective image frames.

In various embodiments, the method may further comprise identifying one or more objects of interest within each image frame.

In various embodiments, the method may further comprise identifying a region of interest within each image frame, wherein the region of interest includes at least in part at least one object of interest.

In various embodiments, the method may further comprise scaling up the region of interest within each image frame to increase the saliency of the region of interest within each image frame.

In various embodiments, the method may further comprise adjusting the region of interest within each image frame to center at least one of the one or more objects of interest in each image frame.

In various embodiments, the method may further comprise adjusting at least one of the one or more objects of interest at a location off center in each image frame by adjusting the region of interest based on contextual information.

In various embodiments, one or more objects of interest of the method may include one or more faces of participants of a video conference, and the contextual information may include one or more face orientations.

In various embodiments, the method may further comprise transmitting a plurality of encoded / compressed image frames to one or more recipients, said transmitting comprising separately transmitting one or more objects of interest and a background can do.

In various embodiments, the system includes a camera configured to capture a video signal having a plurality of image frames; A plurality of image frames, operatively coupled to the camera, configured to receive the plurality of captured image frames and to generate one of the plurality of image frames based on one or more bit densities higher than the bit density used to encode and / And may include an encoding engine configured to encode and / or compress the object of interest, wherein the background and one or more objects of interest form respective image frames. The system may further comprise a transmitter coupled to the encoding engine and configured to transmit a plurality of encoded / compressed image frames to one or more receivers.

In various embodiments, the contextual information may include a viewing angle of one of the one or more recipients, and the contextual engine of the system may be adapted to adjust the captured video signal based on the viewing angle of one of the one or more recipients, Lt; / RTI >

In various embodiments, the article of manufacture may be of a type of non-volatile computer-readable storage medium; And a plurality of programming instructions stored on a storage medium, wherein the programming instructions cause the device to: receive a plurality of image frames of the video signal in response to execution of programming instructions; And encoding and / or compressing one or more objects of interest in each image frame based on one or more bit densities higher than the bit density used to encode and / or compress the background of each image frame Wherein the background and one or more objects of interest form image frames.

In the following detailed description, reference is made to the accompanying drawings that form a part hereof, and in which is shown, by way of illustration, embodiments in which the invention may be practiced. It will be appreciated that other embodiments may be utilized and structural or logical changes may be made without departing from the scope of the present invention. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the embodiments according to the invention is defined by the appended claims and their equivalents.

While various operations may be described one after another as a number of separate operations in a manner that may facilitate understanding of embodiments of the present invention, the order of description should not be construed as implying that such operations are order dependent.

For purposes of explanation, the phrase "A / B" or "A and / or B" means (A), (B) or (A and B). For purposes of illustration, the phrase "at least one of A, B, and C" is used to refer to a combination of (A), (B), (C), (A and B), (A and C) (A, B, and C). For purposes of illustration, the phrase "(A) B" means (B) or (AB), that is, A is an optional element.

The description may use the phrases "in an embodiment" or "in embodiments ", each of which may refer to one or more of the same or different embodiments. Moreover, terms such as " comprising ", "comprising ", and" having ", as used in connection with the embodiments of the present invention, are synonymous. The description may refer to the "object of interest" and "background" of each image frame. For purposes of the present application, including the claims, the term "background" refers to the remainder of the image frame except for one or more "objects of interest", unless expressly indicated otherwise.

Figure 1 is a block diagram illustrating an exemplary image processing apparatus in accordance with various embodiments of the present invention. As shown in FIG. 1, the image processing apparatus 100 may include an encoding engine 110, a context engine 120, and a transmitter 130 coupled together via a bus 140. Although FIG. 1 illustrates various components of the device 100 coupled through the bus 140, in various embodiments, the encoding engine 110, the context engine 120, and the transmitter 130 may include one or more point- Or via any suitable mechanisms such as a hierarchy of buses.

Although FIG. 1 illustrates specific components, device 100 includes other suitable components for facilitating image processing and / or transmission, such as an antenna, camera, decoding engine, display, etc., can do. 1 are shown as separate blocks within device 100, the functions performed by some of these blocks may be incorporated within a single component, or may be further subdivided using two or more separate components have. For example, a decoding engine (not shown) and an encoding engine 110 may be integrated into a single decoding / encoding engine. In addition, the device 100 including all or a portion of the encoding engine 110, the context engine 120, and the transmitter 130 may be implemented in software or hardware, or a combination thereof.

In various embodiments, the image processing apparatus 100 may be a desktop computer, a laptop computer, a handheld computer, a tablet, a cellular telephone, a pager, an audio and / or video player (e.g., an MP3 player or a DVD player) (E. G., A heart rate monitor, a blood pressure monitor, etc.), a set-top box (e. G., A digital camera, a digital camera, a navigation device Boxes, and / or other suitable relatively fixed, portable or mobile electronic devices.

In various embodiments, the image processing apparatus 100 may provide the processed images to one or more wired or wireless network connections, such as a private network, a personal area network (PAN), a local area network (LAN), a virtual private network ), An urban area network (MAN), a wide area network (WAN), a proprietary network, or an open network generally referred to as the Internet, or a combination thereof.

In various embodiments, the image processing apparatus 100 may be configured to perform spread spectrum modulation (e.g., direct sequence code division multiple access (DS-CDMA) and / or frequency hopping code division multiple access (FH-CDMA) (OFDM), Orthogonal Frequency Division Multiplexing (OFDM), Orthogonal Frequency Division Multiplexing (OFDM), Orthogonal Frequency Division Multiplexing (OFDMA), SC-FDMA (Single Carrier FDMA), and / Or any other suitable modulation technique for communicating over wireless links, using a variety of modulation techniques. In one example, the video conferencing device 100 may be a Bluetooth, Zigbee, short-range (WLAN), or wireless local area network (WLAN) for implementing a Wireless Personal Area Network (WPAN), a Wireless Local Area Network May operate in accordance with suitable wireless communication protocols requiring very low power such as near field communication (NFC), ultra wide band (UWB) and / or radio frequency identification (RFID).

In various embodiments, the image processing apparatus 100 may be part of a video conferencing application or system, and may include one or more other image processing devices (not shown), which may or may not be similar to the image processing apparatus 100, The conference can be facilitated.

In various embodiments, the encoding engine 110 may receive a video signal comprising a series of image frames. In various embodiments, the video signal may comprise image frames of uncompressed raw data format. In various embodiments, the video signal is transmitted by H.261, published by the International Telecommunication Union Telecommunication Standardization Sector (ITU-T) in 1984, by the International Organization for Standardization (ISO) May have already been encoded / compressed with some lossy or lossless encoding / compression schemes such as the announced MPEG-1 Part 2 and the H.264 / MPEG-4 AVC published by ISO in 1998. [ Encoding engine 110 may receive a video signal from a communication interface (not shown) of device 100 that may receive a video signal from an external source. Alternatively, the encoding engine 110 may receive the video signal via the bus 140 from a video camera attached to or otherwise integrated with the device 100.

The encoding engine 110 may be configured to encode and / or compress image frames one by one as a stream or in parallel. The encoding engine 110 may transcode the video signal (e.g., decode the video signal and re-encode the video signal based on a different scheme) if the video signal is already encoded / compressed. Encoding engine 110 may encode or add to the video signal additional information, such as information related to the retrieved forward, reverse or random access, subtitle, and digital rights management, etc. of the video signal. Encoding engine 110 may use any known video / image compression schemes or methods such as interframe compression, intra-frame compression, discrete cosine transform (DCT), fractal compression, matching tracking, discrete wavelet transform Image frames can be encoded / compressed.

In various embodiments, the encoding engine 110 may have information associated with one or more object of interest (OOI) in one or more image frames. The OOI may be associated with applications or usage scenarios of the device 100. Exemplary OOIs may include a face and / or shoulder area of the speaker during a video conference session, a car riding through a racing track during a television broadcast, a moving object captured and / or tracked by a surveillance camera, and the like. In various embodiments, the number of OOI regions included in an image frame of a video signal may vary from frame to frame. For example, a single OOI may exist within an image frame, and there may be multiple OOIs in subsequent or other image frames.

2 is a block diagram illustrating exemplary individual image frames of a video stream in accordance with various embodiments of the present invention. As shown, the image frame 200 includes an OOI region 210, an OOI region 212, an OOI region 214, and a shadow region within the image frame 200, and the OOI regions 210-214 May include an enclosing background (230). Although Figure 2 shows an image frame 200 with three OOI regions, in various embodiments the image frame 200 may include more or fewer OOI regions. Although FIG. 2 shows multiple OOI regions 210-214 having the same size and shape, the size, shape, and location of the various OOI regions 210-214 are exemplary only and may be different in various embodiments . In addition, the illustrative drawing shows a background 230 surrounding OOI areas 210-214, in which background 230 partially surrounds OOI areas 210-214, Respectively.

In various embodiments, the encoding engine applies different quantization parameters to encode / compress the background 230 that constitutes the image frame with one or more OOI regions 210-214 and OOI regions 210-214 . In some embodiments, the encoding engine 110 may be configured to encode / compress one or more OOI regions 210-214 using one or more bit densities higher than the bit density used to encode / compress the background 230 . For example, in a video conferencing application using the device 100, the areas around the face and shoulder of the speaker may be identified as two OOI regions, respectively. The encoding engine 110 encodes / compresses the area around the face of the speaker using a high bit density that allows other meeting attendees to see the face representation of the speaker more clearly, Encoding / compressing the area around the shoulder, and encoding / compressing the background of the image frame using low bit density. Thus, the face region of the speaker (e.g., one or more OOI regions) may be allocated more bits than non-face regions (e.g., background) in each image frame. Configuring high, medium, or low bit densities may vary from application to application.

In various embodiments, the encoding engine 110 may encode / decode the image frame 200 using region-based coding techniques in addition to or in place of the bit-density preferential encoding and / Can be compressed. For example, OOI regions 210-214 and background 230 may be separately encoded / compressed using different encoding / compression models. Encoding engine 110 may encode / compress background 230 based on a non-parametric background model. Encoding engine 110 may encode / compress and compress OOI regions 210-214 based on a separate encoding and / or compression model. The encoded / compressed OOI regions 210-214 of the image frame 200 are transmitted to one or more recipients (not shown in FIG. 1) via the transmitter 130 separately from the encoded / compressed background 230 . One or more receivers can decompress and decode the background and OOI regions individually as received, and combine the regions and background to reconstruct the complete image frame.

In various embodiments, the encoding engine 110 may encode / compress a particular image frame 200 based on previous or subsequent image frames to achieve more efficient compression, for example, by use of inter-frame compression, can do.

In various embodiments, such as video conferencing applications, the background 230 may be static or substantially static for most of the time, and may be unchanged or minimized for each frame. Thus, the transmitter 130 may periodically transmit the background 230 for every two or more image frames, rather than transmitting the background in each image frame. In other embodiments, the transmitter 130 may dynamically transmit the background 230 when a (substantial) change in background is detected via one or more previous image frames. The coding and / or transmission efficiency of the device 100 may be improved by using bit-density optimized encoding and / or region-based encoding / compression techniques that individually encode, compress, and transfer background and OOI regions. Thus, the user experience of the video conferencing application based on the device 100 may be improved in video conferencing, particularly performed using mobile devices where network bandwidth may be limited.

In various embodiments, the context engine 120 may detect, identify and / or track one or more OOI regions 210-214 within an image frame 200 or in a video stream, and may include OOI regions 210-214 and And provide the associated information to the encoding engine 110. The context engine 120 may detect the OOI regions 210-214 based on various techniques known for object tracking or face recognition. One such object tracking technique may be to thoroughly scan the window across the image frame based on the cascade of AdaBoost classifiers. In various embodiments, the context engine 120 may provide OOI information to help the encoding engine 110 to encode and / or compress image frames.

In various embodiments, the context engine 120 may be further configured to identify a region of interest (ROI) within an image frame that may include at least a portion of one or more OOI regions. An exemplary ROI may be an ROI 220 illustrated as an area surrounded by a dotted line in FIG. The location and size of the ROI 220 may be identified based on the bounding box represented by coordinates such as (Rx, Ry). In various embodiments, ROI 220 may or may not include portions of background 230.

In various embodiments, the context engine 120 may also identify a central point (P _c ) in the ROI region 220. In certain embodiments, P _c may be defined as the weighted center (weighted center) of all OOI region in the ROI. For example, P _c can be identified based on the following equation:

Where P _i is the center position of each of the detected OOI regions and N is the number of detected OOI regions. In other embodiments, P _c can be identified as the center point of the maximum OOI region within ROI, such as the closest object of interest in the camera. For example, P _c

, Where arg max (size (P _i )) is the center point of the OOI region with the maximum size. In some other embodiments, P _c may be the center point of the OOI from which changes from one or more previous frames are detected. For example, P _c may be used to indicate an OOI that represents one of the moving persons being tracked by a surveillance camera, or a person currently speaking with multiple participants in a video conference. In various embodiments, P _c may or may not be the true "center" of ROI 220, as described above.

In various embodiments, as shown, the context engine 120 may identify the ROI region 220 by a bounding box that includes all of the OOI regions 210-214. In various embodiments, when ROI 220 is identified, context engine 120 performs ROI adjustment, including adjustment of size, position, and scaling of ROI 220 to determine center point (P _c ) It may be at or near the center of the frame 200. [ The area surrounding the ROI 220 may also be moved and / or scaled along with the ROI 220 to provide a spatial correlation between the ROI 220 and the portion of the background 230 surrounding the ROI 220. In other embodiments, Lt; / RTI >

In some embodiments, the situation engine 120 may scale up the ROI 220 as much as possible to the dimensions (or raw size) of the image frame 200, which may distort or distort the image within the ROI 220 . In some embodiments, the dimensions (or aspect ratio) of the ROI 220 may not scale up to match the dimensions (or aspect ratio) of the image frame 200. Thus, the context engine may optionally include a portion of the background region 230 or remove a portion of the ROI 220 to display the image within the ROI 220 with the correct aspect ratio. Similarly, the situation engine 120 may selectively remove / include portions of the ROI 220 and the background 230 to move the center point P _c to or near the center position of the image frame 200 have. As a result, at least one of the OOIs in the ROI 220, e.g., the OOI 210, may appear larger and may be focused at or near the center portion of the image frame 200. [

In various embodiments, the context engine 120 may provide the encoding engine 110 with an image frame that includes the scaled / modified ROI 220 to enable the encoding engine 110 to encode and / And / or compressed and subsequently transmitted.

In various embodiments, a camera (not shown) may be attached to or integrated with apparatus 100 to capture a video signal having a stream of image frames. The situation engine 120 may be functionally coupled to the camera and configured to control the camera. Upon identification of the ROI 220 within each image frame, the context engine 120 does not scale up or move the ROI 220 but controls the camera to zoom in or out, or panning the camera angle, At least one of the OOIs in the ROI 220 may be located in the central portion of the image frame 200. [

In various embodiments, the context engine 120 may be further configured to adjust the size and position of the ROI 220 to deliberately place at least one of the OOIs in an off-center location based on the one or more contextual information . The context information may depend on the application of the device 100 or the usage scenario. For example, in a video conference session, the context engine 120 may use the face orientation (e.g., pose) of the current speaker as context information.

In various embodiments, facial orientation information can be used to estimate potential ROIs in addition to the identified ROIs and can be used to stabilize ROI extraction. For example, if the speaker is looking straight at the camera, the speaker may appear at or near the center of the video. However, if the speaker speaks while looking at his right side, in various embodiments context engine 120 may optionally move or pan the window of ROI 220 to the "right" side of the speaker, or scaling ROI 220 Up or down, or otherwise adjustments, to display more background 230 on the "right" side of the speaker. As another example, a video conference may include multiple participants. The context engine 120 may initially recognize all participants as objects of interest and draw an ROI to include all participants. However, face orientation information can indicate that one of the participants is reading (perhaps reading) his head and not looking at the camera. In various embodiments, the context engine 120 may decide to temporarily exclude its participant (or OOI) from the ROI, so that the person currently engaged in the conversation may appear more prominently in the center of the screen. In various embodiments, upon detection of a (substantial) change in the face orientation of the participant, the context engine 120 may re-adjust the ROI 220 to include its participants in the image.

In various embodiments, the context engine 120 may be configured to analyze the image frame 200 and generate contextual information. For example, in a video conferencing application, the context engine 120 may be configured to analyze the face orientation of the speaker to include the face orientation of the speaker as contextual information. The context engine 120 may analyze the face orientation of the speaker based on measurement and analysis of three different angles of the face, e.g., pitch, yaw and roll.

In various embodiments, instead of the context engine 120 analyzing the image frame to generate contextual information, contextual information may be provided to the contextual engine 120. [ For example, the transmitter 130 of the device 100 may transmit video over the network to one or more recipients. One or more cameras installed on the recipient side may track the viewpoints (or viewing angles) of one or more recipients based on face orientation tracking methods or other known methods similar to those described above. Alternatively, in addition to face orientation tracking, cameras may similarly identify and / or identify gestures of one or more recipients using any known method, such as the Xbox 360 Kinect < ' > camera kit available from Washington Redmond, / Or can be tracked. Thus, in some embodiments, the context information may include one or more recipient's views of the video and / or gestures. For example, if the recipient is viewing the left side of the video, such point-of-view information may be fed back to the situation engine 120 over the network, and the situation engine 120 may provide feedback to the ROI region 220 of the video, To display more information towards the left side of the video. Gestures of recipients may also be fed back to the situation engine 120 for a similar purpose. Alternatively, the recipients of the video may provide contextual information to the context engine 120 via other input methods such as keyboard, mouse, voice input via a microphone, and the like. Thus, by receiving status information from one or more recipients, the device 100 may be able to provide virtual reality effects to attendees of the video conference or remote control capabilities to the recipients of the video.

In various embodiments, the situation engine 120 may provide the encoding engine 110 with an image frame that includes the adjusted ROI 220 based on contextual information, so that encoding and / or compression of image frames, , And subsequent transmission by the transmitter 130. [

3 is a flow diagram illustrating a portion of the operations of an encoding engine of an image processing apparatus in accordance with various embodiments of the present invention. At block 310, the encoding engine 110 may receive a stream of image frames and information associated with one or more OOI regions inserted into each image frame. At block 320, the encoding engine 110 may encode / compress each image frame using bit-density differential encoding / compression and / or region-based differential encoding / compression as described above. In region-based encoding / compression, the transmitter 130 may transmit the background and one or more OOI regions separately to one or more recipients. Transmitter 130 may dynamically transmit the background on a periodic basis for every two or more image frames or upon detection of a background change. The encoding engine 110 may repeat the above operations until all of the images have been processed. One or more OOI regions may be transmitted in each image frame. The one or more recipients can then decompress and reconstruct images as they are received.

4 is a flow diagram illustrating a portion of the operations of the context engine of an image processing apparatus according to various embodiments of the present invention. At block 410, the situation engine 120 may receive a stream of image frames and may be configured to process the streams of image frames one by one or in parallel. The context engine 120 may receive a stream of image frames in parallel with the encoding engine 110, or alternatively may process image frames prior to the encoding engine 110. At block 420, the situation engine 120 may identify one or more OOI regions within an image frame and provide OOI region information to the encoding engine 110 for encoding and / or compression as described above. At block 430, the context engine 120 may set an ROI within an image frame that includes at least one of the OOI regions. The context engine 120 scales the ROI and can reduce regions outside the ROI in the image frame, so that one or more of the OOI regions can be marked significantly and appear focused in the center of the image in the image. The context engine 120 may adjust the size and position of the ROI so that at least one OOI may be centered within the image frame. At block 440, the context engine 120 may analyze the image frame to generate contextual information associated with the image frame. At block 450, the situation engine 120 may adjust the ROI based on contextual information to place at least one of the one or more OOI regions at a location off center in the image frame. The context engine 120 may send the OOI and / or ROI information to the encoding engine 110, which may or may not be adjusted according to context information. The context engine 120 may repeat the above steps until all of the images have been processed.

5 is a flow diagram illustrating a portion of the operations of an image processing apparatus in accordance with various embodiments of the present invention. At block 510, one or more video cameras (not shown in FIG. 1) attached to the image processing apparatus 100 may capture video images in a two-dimensional (2D) or three-dimensional (3D) format. At block 520, the context engine may analyze each image frame to identify one or more OOI regions. At block 530, the context engine 120 may set the ROI within the image frame and scale the ROI within the image frame to place at least one OOI in the central portion of the image frame. At block 540, the context engine may analyze the image frame, generate one or more contextual information including an estimate of the face orientation information, and modify the ROI according to the contextual information. The context engine 120 may send an image frame containing the modified ROI to the encoding engine 110. At block 550, the encoding engine 110 may encode / compress the streams of image frames one by one or in parallel based on bit-density differential or region-based differential encoding and / or compression. At block 560, the transmitter 130 may transmit the encoded / compressed image frames to one or more recipients. The context engine 120, the encoding engine 110, and the transmitter 130 may repeat the above steps until all image frames have been processed.

Figure 6 is a block diagram illustrating an exemplary computer system suitable for use in implementing various aspects of the methods and / or devices described in accordance with embodiments of the present invention. As shown, the computer system 600 may include a power unit 601, a plurality of processors or processor cores 602, a system memory 604, a mass storage device 606, and a communication interface 610 . For purposes of this description including the claims, the terms "processor" and "processor core" may be considered synonymous unless the context clearly dictates otherwise.

In addition, computing system 600 may include one or more types of non-volatile computer readable mass storage devices 606 (e.g., diskettes, hard drives, compact disk read only memory (CDROM) (E. G., Keyboard, cursor control, etc.). In various embodiments, I / O devices 608 may include one or more cameras 618. The elements may be coupled to the elements previously listed via the system bus 612 to represent one or more buses and to each other. In the case of multiple buses, they may be bridged by one or more bus bridges (not shown). Data may be transferred from the I / O devices 608, e.g., from the camera 618, to the processors 602 via the system bus 612. [

The system memory 604 and mass storage device 606 store executable and permanent copies of programming instructions that implement one or more operating systems, firmware modules or drivers, applications, etc., collectively represented herein as 622 Can be used. The programming instructions may cause the computing system 600 to perform operations of the encoding engine 110, the context engine 120, and / or the transmitter 130 as described above when executed by the processors 602. [ The camera 618 may capture a stream of video images and may be under the control of the context engine 120 as described above.

A permanent copy of the programming instructions may be stored permanently at the factory or in the field, for example, via a distribution medium (not shown) such as a compact disk (CD) or via a communication interface 610 (from a distribution server May be located within the device 606. [ That is, one or more distribution media having an implementation of the agent program may be used to distribute the agent and program to the various computing devices.

The remainder of the configuration of these elements 601-622 is known and is therefore not further described.

Figure 7 illustrates an article of manufacture having programming instructions configured to enable a device to perform various operations of an image processing apparatus, in accordance with embodiments of the present invention. As shown, the article 700 may include a computer readable non-volatile storage medium 710. Storage medium 710 may include programming instructions 720 configured to implement an image processing apparatus.

Storage medium 710 represents a wide variety of persistent storage media known in the art, including, but not limited to, flash memory, optical or magnetic disks. Programming instructions 720 may include, in response to their execution by an image processing device, an operation that causes the device to receive a plurality of image frames of a video signal; And encoding and / or compressing one or more objects of interest in each image frame based on one or more bit densities higher than the bit density used to encode / compress the background of each image frame And the background and one or more objects of interest form respective image frames.

Although specific embodiments have been illustrated and described herein, those of ordinary skill in the art will recognize that various alternatives and / or equivalent implementations may be substituted for the specific embodiments shown and described without departing from the scope of the embodiments of the invention You will know. This application is intended to cover any adaptations or variations of the embodiments described herein. Accordingly, it is manifestly intended that embodiments of the invention are limited only by the claims and their equivalents.

Claims (1)

The method according to claim 1,
Apparatus for image processing.

Images