GB2516486A

GB2516486A - Auto focus determination apparatus

Info

Publication number: GB2516486A
Application number: GB1313224.6A
Authority: GB
Inventors: Markus Tapio Vartiainen; Ali Karaoglu
Original assignee: Nokia Oyj
Current assignee: Nokia Oyj
Priority date: 2013-07-24
Filing date: 2013-07-24
Publication date: 2015-01-28
Also published as: GB201313224D0

Abstract

An apparatus, means of generation and method, comprising a focus window generator configured to generate a focus window (or region of interest ROI) for use in a contrast autofocus apparatus, the focus window generator comprising a focus window border with no substantially continuous vertical or horizontal components, and may comprise for example a quantized rhombus or diamond. The focus window generator may comprise a microblock determiner 401 configured to define a number of microblocks each defined by size and location, an image signal processor configured to receive an image and configured to determine at least one focus value for the defined microblocks with respect to the image, a macroblock grouper 407 configured to determine at least one macroblock comprising a further number of microblocks from the number of microblocks, and a shape filter 409 configured to apply a special shaping filtering 409 to at least one focusing value for the determined further number microblocks associated with at least one microblock to generate the focus window. The special shape filtering 409 may apply a centre biased (weighted) shaping filtering 409 to at least one focus value of the determined microblocks and may comprise a mask generator configured to generate a mask of values applied to the determined microblock focus values. A special low pass filter 405 may be included to output a filtered focus value for a microblock and also for neighbouring microblock focus values. The macroblock grouper may be configured to determine at least two macroblocks comprising at least one common microblock.

Description

AUTO FOCUS DETERMINATION APPARATUS R&d

The present apphcation r&ates to apparatus for autofocus determination. The appHcation further relates to, but is not limited to, portable or mobile apparatus for autofocus determination.

Background

Camera hardware has become the norm for mobHe phones and other portable devices or apparatus such as audio devices, Furthermore a rapid devebpment in mobe camera hardware technologies and image processing algorithms has enabled a profes&onal like experience for end users operating such camera hardware. From the camera sensor image capture to the presentation of the image to the end user, a series of steps or operations are required to obtain a high quality image. For example colour correction, noise fUtering, auto white balance, auto exposure are such operations which have to be employed in order to generate a high quaHty image as typicay a user is generay interested in producing an image with proper colour, light, noise and intensity levels. Another aspect of a high quaHty image is sharpness, which is directly proporrional wtth detafis in the image.

R would be understood that the lens coHects the light from the scene and projects the Hght onto an image sensor. To capture a sharp image or object, the camera is required to be focused properly. Although manual or semiautomatic focussing of the camera is known typicafly focusing a camera is performed by an auto focus function or operation in digital camera systems. There are two main technologies that are implemented to provide an auto4ocus function, which are active focusing and passive focusing.

An active focussing system measures the distance between an object to the camera without using the ens system. To achieve this measurement the device activ&y measures the distance by transmitting and then receiving a suitable wave. For example known active focussing systems implement ultrasonic sound waves or infrared fight to measure the distance. The distance can then be ca'culated by trianguation between source, object and sensor, or estimating the depth from time of ffight of the Ughtlsound wave.

In passive focussing systems, there are two known ways to determine the focus point. These are focus determmatkon by phase detect on and focus determination by contrast delection. In focus determination by phase detection, a ens system is employed to spt the Hght into pairs of images and by comparing the pairs of images determining the direction and focal distance. In focus determination by contrast detection, the focus range is scanned by moving the ens and capturing images, measuring the sharpness of them and finding the best location for the lens to stay stifi during the image capture. A contrast detection focus determination requires less hardware than the other auto-focus systems. The other focus determination systems require extra hardware, which increases the weight, size and more importantly the cost of the device.

oftheAIication Aspects of this application thus provide a focus determination capability to enable more flexible image capturing.

According to a first aspect there is provided an apparatus comprising: a focus window generator configured to generate a focus window for use in a contrast autofocus apparatus, the focus window generator comprising a focus window border with no substantially continuous vertical or horizontal components.

The focus window generator may comprise: a microblock determiner configured to define a number of microblocks, each microblock deFined by a size and location; an image signal processor configured to receive an image and further configured to determine at east one focus value for the defined microblocks with respect to the image; a macroblock grouper configured to determine at least one macroblock comprising a further number of microblocks from the number of microblocks; and a shape filter configured to apply a spatial shaping filtering to the at least one focus value for the determined further number microblocks associated with the at roast one macrobbck to generate the focus window.

The shape filter configured to apply a spatial shaping fHtering to the at least one focus value for the determined microblocks associated with the at least one macroblock to generate the focus window may be further configured to apply a centre biased spatial shaping filtering to the at east one focus value for the determined microblocks associated with the at least one macrobbck.

The shape fiRer may comprise a mask generator configured to generate a mask of values to be applied to the determined microblock focus values associated with the at least one macroblock.

The macroblock grouper may he configured to determine at least two rnacroblocks, wherein the at least two macroblocks comprise at least one microblock in common.

The focus window generator may comprise a spatial low pass lifter configured to output a filtered focus value for a microblock, wherein the macroblock grouper is configured to determine at least one macroblock comprising a further number of rnicroblock filtered focus values.

The spatial ow pass fiRer may be configured to output a filtered focus value for a microblock based on the microblock focus vaiue and neighbouring microblock focus values.

The focus window generator comprising a focus window border with no substantiaUy continuous vertical or horizontal components may be configured to generate a spatially quantized rhombus focus window.

According to a second aspect there is provided an apparatus comprising at least one processor and at least one memory including computer code for one or more programs, the at least one memory and the computer code configured to with the at least one processor cause the apparatus to: generate a focus window for use in a contrast autofocus apparatus, the focus window generator comprising a focus window border with no substantially continuous vertical or horizontal components.

Generating the focus window may cause the apparatus to: define a number of mlcroblocks, each microblock defined by a size and location; receive an image and further configured to determine at least one focus value for the defined microblocks with respect to the Image; determine at least one macroblock comprising a further number of microbiocks from the number of microblocks; and apply a spatial shaping filter to the at least one focus value for the determined further number microblocks assocIated with the at least one macroblock to generate the focus window.

Applying a spatial shaping filtering to the at least one focus value for the determined microblocks associated with the at least one macroblock to generate the focus window may further cause the apparatus to apply a centre biased spatial shaping filter to the at least one focus value for the detemiined microblocks associated with the at least one macroblock.

Applying a spatial shaping filtering to the at least one focus value hr the determined microblocks associated with the at least one maciubiock to generate the focus window may further cause the apparatus to generate a mask of values to be applied to the determined microblock focus values associated with the at least one macroblock.

Determining at least one macroblock comprising a further number of microblocks from the number of microblocks may cause the apparatus to detemilne at least t macrobiocks, wherein the at least two macroblocks comprise at least one microblock in common.

Generating the focus window causes the apparatus to spatial low pass filter the at least one microblock to output a filtered focus value for a microblock, wherein determining at least one macroblock may cause the apparatus to determine a macroblock comprising a further number of microblock filtered focus values.

Spatial low pass filtering the at least one microblock causes the apparatus to output a filtered focus value for a microblock based on the microblock focus value and neighbouring microblock focus values, GeneratIng a focus window for use in a contrast autofocus apparatus, the focus window generator comprising a focus window border with no substantially continuous vertical or horizontal components may cause the apparatus generate a focus window comprising a spatially quantized rhombus focus bonier.

According to a third aspect there Is provided an apparatus comprising: means for generating a focus window for use in a contrast autofocus apparatus, the focus window generator comprising a focus window border with no substantially continuous vertical or horizontal components.

The means for generating the focus window may comprise: means for defining a number of microbiocks, each microblock defined by a size and location; means for receiving an image and further configured to determine at least one focus value for the defined microblocks with respect to the image; means for determining at least one macroblock comprising a further number of microblocks from the number of microblocks; and means for applying a spatial shapIng filter to the at least one focus value for the determined further number microblocks associated with the at least one macrobiock to generate the focus window.

The means for applying a spatial shaping fiftering to the at least one focus value for the determined microbiocks associated with the at least one macroblock to generate the focus window may further comprise means for applying a centre biased spatial shaping filter to the at least one focus value for the determined microblocks associated with the at feast one macroblock.

The means for applying a spatial shaping filtering to the at least one focus value for the determined microbiocks associated with the at least one macroblock to generate the focus window may further comprise means for generating a mask of values to be applied to the determined microblock focus values associated with the at least one macroblock.

The means for determining at east one macroblock comprising a further number of microblocks from the number of microblocks may comprise means for determining at east two macrohbcks, wherein the at least two macroblocks comprise at east one microblock in common, The means for generating the focus window may comprise means for spatial low pass fiftering the at least one microblock to output a filtered focus value for a microblock, wherein the means for determining at least one macroblock may comprise means for determining a macroblock comprising a further number of microblock filtered focus values.

The means for spatial low pass filtering the at least one microblock may comprise means for outputting a filtered focus value for a microblock based on the microblock focus value and neighbouring microblock focus values.

The means for generating a focus window for use in a contrast autofocus apparatus, the focus window generator comprising a focus window border with no substantiay continuous vertical or horizontal components may comprises means for generating a focus window comprising a spatiafly quantized rhombus focus border.

According to a fourth aspect there is provided a method comprising: generating a focus window for use in a contrast autofocus apparatus, the focus window generator comprising a focus window border with no substantially continuous vertical or horizontal components.

Generating the focus window may comprise: defining a number of microblocks, each microblock defined by a size and location; receiving an image and further configured to determine at least one focus value for the defined microblocks with respect to the image; determining at least one macroblock comprising a further number of microblocks from the number of microblocks; and applying a spatial shaping filter to the at least one focus value for the determined further number microblocks associated with the at east one macroblock to generate the focus window.

Applying a spatial shaping filtering to the at least one focus value for the determined microblocks associated with the at least one macroblock to generate the focus window may further comprise applying a centre biased spatial shaping filter to the at least one focus value for the determined microblocks associated with the at least one macroblock.

Applying a spatial shaping filtering to the at least one focus value for the determined microblocks associated with the at least one macroblock to generate the focus window may further comprise generating a mask of values to be applied to the determined microbiock focus values associated with the at least one macroblock.

DetermIning at least one macroblock comprising a further number of microblocks from the number of micioblocks may comprise determining at least two macrobiocks, wherein the at least two macroblocks comprise at least one microblock In common.

Generating the focus window may comprise spatial low pass filtering the at least one microblock to output a filtered focus value for a microblock, wherein determining at least one macroblock may comprise determining a macroblock comprising a further number of microblock filtered focus values.

Spatial low pass filtering the at least one microblock may comprise outputting a filtered focus value for a microbiock based on the microblock focus value and neighbouring microblock focus values.

Generating a focus window for use in a contrast autofocus apparatus, the focus window generator comprising a focus window bonier with no substantially continuous vertical or horlzontai components may comprises generating a focus window comprising a spatially quantized rhombus focus border.

AccordIng to a fifth aspect there is provided an apparatus comprising: a microblock determiner configured to define a number of microblocks, each microblock defined by a size and location; an image signal processor configured to receive an image and further configured to determine at least one focus value for the defined microblocks with respect to the image; a macroblock grouper configured to determine at least one macroblock comprising a further number of microblocks from the number of microblocks; and a shape filter configured to apply a spatial shaping filtering to the at least one focus value for the determined further number microblocks associated with the at least one macroblock to generate a focus window for contrast auto focus determination.

According to a sixth aspect there is provided an apparatus comprising at least one processor and at least one memory including computer code for one or more programs, the at least one memory and the computer code configured to wfth the at least one processor cause the apparatus to: define a number of microblocks, each microblock defined by a size and location; receive an image and further configured to determine at least one focus value for the defined microblocks with respect to the image; determine at least one macrohiock comprising a further number of microblocks from the number of microblocks; and apply a spatial shaping filter to the at least one focus value for the determined further number microblocks associated with the at least one macroblock to generate a focus window for contrast auto focus determination.

According to a seventh aspect there is provided an apparatus comprising: means for defining a number of microblocks, each microblock defined by a size and location; means for receiving an image and further configured to determine at least one focus value for the defined microblocks with respect to the image; means for determining at least one macroblock comprising a further number of microblocks from the number of microblocks; and means for applying a spatial shaping filter to the at least one focus value for the determined further number microblocks associated with the at least one macroblock to generate a focus window for contrast autofocus determination.

According to an eighth aspect there is provided a method comprising: defining a number of microblocks, each microhlock defined by a size and location; receiving an image and further configured to determine at least one focus value for the defined microblocks with respect to the image; determining at least one macroblock comprising a further number of microblocks from the number of microblocks; and applying a spatial shaping filter to the at least one focus value for the determined further number microbiocks associated with the at least one macroblock to generate the focus window.

A computer program product stored on a medium may cause an apparatus to 6 perform the method as described herein.

An electronic device may comprise apparatus as described herein.

A chlpset may comprise apparatus as described herein.

Embodiments of the present application aim to address problems associated with the state of the art.

Summary of the Figures

For better understanding of the present application, reference will now be made by way of example to the accompanying drawings In which: Figure 1 shows schematically an apparatus suitable for Implementing embodiments; Figure 2 shows schematically a conventional contrast detection autofocus method; Figure 3 shows an example 3x3 grid contrast detection autofocus example; Figures 4a and lb show example graphs of the output of the example 3x3 grid contrast detection autofocus example shown in Figure 3 under static and simulated hand shake motion conditions; Figure 5 shows schematically contrast detection autofocus apparatus according to some embodiments; Figure 6 shows a flow diagram of the operation of the contrast detection autofocus apparatus shown In Figure 5; Figures la and lb show an example rhombus shaped focus window and a centre weIghted rhombus shaped focus window as employed in some embodiments by the contrast detection autofocus apparatus; FIgure 8 shows an example 3x3 spaced representations of centre weighted rhombus shaped focus windows within an image as employed in some embodiments by the contrast detection autofocus apparatus; Figure 9 shows an example overlapping pattern as employed in some embodiments by the contrast detection autofocus apparatus; Figure 10 shows an example combination of the overlapping pattern as shown In Figure 9 with the example centre weighted rhombus shaped focus windows shown in Figure 8 as employed in some embodiments by the contrast detection autofocus apparatus in some embodiments; and Figure 11 shows an example graph of the output oF the combined overlapping pattern and centre weighted rhombus shaped focus windows under simulated hand shake motion conditions.

Embodiments of the Application The following describes in further detail suitable apparatus and possible mechanisms for the provision of effective contrast based autofocus operations and apparatus suitable for Implementing the effective contrast based autofocus operations.

As described herein a contrast based passive autofocus system is one where the focus range is scanned by moving the lens and capturing images, measuring the sharpness of the images and finding the best location for the lens to stay still during the image capture. The operation of a conventional contrast determining autofocus method is shown with respect to Figure 2. Figure 2 In particular shows a flow diagram of the contrast determination operations.

In such an operation the camera Is set to capture an image. The operation of capturing an image is shown in Figure 2 by step 101.

The image data can then be processed to determine or calculate a focus value with a given focus wIndow. The operation of calculating a focus value with a gIven focus window is shown in Figure 2 by step 103.

The next operation Is then to determine the next lens position. The operation of determining the next lens position is shown in Figure 2 by step 105.

The final operation in the loop is then to change the ens position so that when the operation loops back to capturing the image it does so using the new lens position.

The operation of sethng or changing the lens position is shown in Figure 2 by step 107.

The operation of calculating or determining the focus value is significant in producing a good or high quaUty image. In the operation of determining the focus value the main input is the region of interest in the image that is used for calculating the change in sharpness as the ens moves across the focusing range. The region of interest is typically cafled a focus window. A focus window must be big enough to provide meaningful amount of data and small enough to represent a part of the image that is Ukely to have content which is singularly distant, in other words does not contain a large range of distances from the camera.

A major problem in contrasthased focusing is that as it requires several image frames to be captured during focusing which obviously takes a nonzero time interval to perform. In this time interval there is a possibility for either the subject or the the camera to move between exposing focusing frames. If the content inside and at the immediate vicinity of a focus window is somewhat comparable, in other words provides a similar level of contrast, then the focusing content drift does not pose a problem, However, if the contrast of the image content at the immediate vicinity (outside) of the focus window is very different from inside the focus window, (for example there is a strong line or change in colour or lightness inside or immediately outside the focus window), there is a risk that high contrast elements only transit the focus window during focusing, causing an artificial change in sharpness during focusing, which may confuse the focusing algorithm.

The content drift in hand held cameras during the auto4ocusing is generally caused by the motion of the end user. However, an end user wants to be able to capture in every condition and doesn't want to carry a tripod everywhere. Furthermore user motion can degrade the performance of the content autofocussing apparatus where high contrast objects are in the images or scene and the motion causes the objects to move in and out of the focus window during the auto$ocus process. In such circumstances the focus values can change in an unexpected way. Typically strong contrast elements are generally presented as verflcal or horizontal Unes and are produced by conjunctions of objects such as walls, doorways, windows, and furniture.

As described herein the concept is to apply at east one of the following to attempt to improve the performance of a contrastbased autofocus apparatus. It would be understood that the combinabon of the following aspects can be made in any logical manner.

A first aspect as described herein is that by aligning a focus window border such that it is not the same as expected strong contrast elements, drifting content wUl have much smaller impact in the subsequent sharpness values of the focus window. For example, as described herein, a rhombus-shaped focus window with borders at 45 degrees angles in relation to the expected high-contrast horizontal and vertical content reduces the probabllity of such high-contrast elements to enter or leave the focus window all at once. For example Figure 7a shows an example rhombus or rhomboid focus window 601, where a high contrast horizontal or vertical content element (or a high contrast element approximately horizontal or vertical) would gradually leave or enter the focus window. In other words the focus window as a rhombus allows the horizontal or vertical high contrast element to fade in' or fade out'. It would be understood that in such examples the rhombus or rhomboid focus window allows much more reliable sharpness data for an autofocusing operation, and especially when focu&ng is slow for a reason or another (for example when there is little light, or the exposure time is long and frame rate is slow).

In the following examples a rhombus focus window is described, however it would be understood that in some embodiments the focus window can he any suitable shape which prevents high contrast (approximately) horizontal or vertical content from suddenly entering or leaving the window, In other words generating a focus window border with no substantially continuous vertical or horizontal components. The shape can be regular or irregular. For example in the following example the rhombus is generated from combining micro-blocks to form a digitized or quantized rhomboid like focuswindow hut it would be understood that by using a different arrangement of micro-blocks any suitable shape regular or irregular can be approximated.

A second aspect further addresses the same issue of content drift and in particular to decrease the amount of contrast that can get in and out of the focus window. For example in some embodiments the focus window can he weighted in a manner that by decreasing the weight on the edges of the focus window, the amount of the contrast on the edges is decreased and therefore the amount of contrast which leaves the focus window during the auto$ocus process because of motion of the apparatus is decreased.

This can be applied to any suitable focus window, such as a conventional block focus window but as shown in Figure lb can be applied to the rhombus focus window example shown in Figure 7a. The centre weighted rhombus focus window 611 is shown where the darkness of the rhombus represents the weighting of the focus window such that the centre is black indicaUng the greatest weighting and the vertices of the rhombus are the lightest shade indicating the lowest weighting. In the examples shown herein the weighting is symmetrical or circthar weighting however it would be understood that in some embodiments the weighting can be biased in a specific orientation or direction.

A third aspect can be shown with respect to the example focus windows shown in Figure 3. Figure 3 shows example block or square focus windows arranged in a grid pattern or target region 201. The focus windows shown in Figure 3 are a top left focus window 203, a top centre focus window 205, a top right focus window 207, a centre left focus window 209, a centre centre focus window 211, a centre right focus window 213, a bottom left focus window 215, a bottom centre focus window 217, and a bottom right focus window 219. When the focus window blocks are arranged in such a grid pattern (a 3by-3 block grid shown in Figure 3) high contrast content can move from one block to another. This movement can cause the focus values of one block to suddenly increase and the other block focus value to suddenly decrease. In some embodiments the focus windows are overlapped. The overlapping window system allows a high contrast object on the edge of one focus block to be also within (for example in the middle range) another focus block, A fourth aspect as embodied In the description herein it to apply a spatial low-pass fitter to the focus window blocks. In such embodiments the effect of contrast drift in the image can be reduced by spreading the focus value information to the surrounding blocks. As shown in the examples herein the focus window can be generated by combining many micro-block focus windows to form macro-blocks. The divided focus window in the form of small blocks can provide uncorrelated sharpness values for each block to represent the pixels. in some embodiments the sharpness of the block is achieved by running a high pass filter over the contents of the focus blocks, and then running a low pass filter over the focus blocks results in smoothed sharpness data. This keeps the micro blocks as corresponding to the surrounding blocks so we decrease the possibility of false peak focus values in the grid.

With respect to Figures 4a and 4b is shown graphs of the focal values for simulated focal windows applied to the example image shown in Figure 3 to show the effect of simulated hand shaking when implementing a typical contrast autolocus operation implementing block focus windows and without applying the aspects as described herein. In the example shown in Figure 3 the image shows a target region 201 and the grid of 3x3 focus wIndows 203, 205, 201, 209, 211, 213, 215, 217, and 219.

Figure Ia shows the example focus curves 301 labelled I to 9 which are the focal values for the focus windows from the top left to the bottom right on a row by row basis where a simulated camera Is held in a static manner. Figure 4b shows the example focus curves 303 also labelled I to 9 which are the focal values for the focus windows from the top left to the bottom right on a row by row basis where a simulated camera Is affected by a simulated hand shaking. It can be seen from the Figures 4a and 4b that the simulated hand shaking generated significantly different focus curves for the same overall image.

in this regard reference Is first made to FIgure 1 which shows a schematic block diagram of an exemplary apparatus or electronic devIce 10, which may be used to capture and process the image data for contrast autofocusing applIcations.

The apparatus 10 can for example be a mobile terminal or user equipment of a wireless communication system. In some embodiments the apparatus can be an audio player or audio recorder, such as an MP3 player, a media recorder/player (also known as an MP4 player), or any suitable portable device requiring user Interface inputs.

In some embodiments the apparatus can be part of a personal computer system an electronIc document reader, a tablet computer, or a laptop.

The apparatus 10 can in some embodiments comprise an audio subsystem. The audio subsystem for example can include in some embodiments a microphone or array of mIcrophones 11 for audio signal capture. In some embodiments the microphone (or at least one of the array of microphones) can be a solid state microphone, in other words capable of capturing acoustic signals and outputting a suitable digital format audio signal. in some other embodiments the microphone or array of microphones 11 can comprise any suitable microphone or audio capture means, for example a condenser microphone, capacitor microphone, electrostatic microphone, electret condenser microphone, dynamic microphone, ribbon microphone, carbon microphone, plezoelectrlc microphone, or microelechical-mechanical system (MEMS) microphone. The microphone 11 or array of microphones can In some embodiments output the generated audio signal to an analogue-to-digital converter (ADC) 14.

in some embodiments the apparatus and audio subsystem includes an analogue-to-digital converter (ADC) 14 configured to receive the analogue captured audio signal from the microphones and output the audio captured signal in a suitable digital form.

The analogue-to-digital converter 14 can be any suitable analogue-to-digital 26 conversion or processing means.

In some embodiments the apparatus 10 and audio subsystem further Includes a digital-to-analogue converter 32 for converting digital audio signals from a processor 21 to a suitable analogue format The digital-to-analogue converter (DAC) or signal processIng means 32 can in some embodiments be any suitable DAC technology.

Furthermore the audio subsystem can Include in some embodiments a speaker 33.

The speaker 33 can in some embodiments receive the output from the digital-to-analogue converter 32 and present the analogue audio signal to the user. In some embodiments the speaker 33 can be representative of a headset, for example a set of headphones, or cordless headphones.

Although the apparatus 10 is shown having both audio capture and audio presentation components, it would be understood that in some embodiments the apparatus 10 can comprise the audio capture only such that in some embodiments of the apparatus the microphone (for audio capture) and the anaIogue-todigftal converter are present.

In some embodiments the apparatus audiovideo subsystem comprises a camera 51 or image capturing means configured to supply to the processor 21 image data, In some embodiments the camera can be configured to supply multiple images or frames over time to provide a video stream.

In sonic embodiments the apparatus audiovideo subsystem comprises a display 52.

The display or image display means can he configured to output visual images of video frames which can be viewed by the user of the apparatus. In some embodiments the display can be a touch screen display suitable for supplying input data to the apparatus. The display can be any suitable display technology, for example the display can be implemented by a flat panel comprising cells of LCD, LED, OLED, or plasma' display implementations.

Although the apparatus 10 is shown having both audio/video capture and audio/video presentation components, it would be understood that in some embodiments the apparatus 10 can comprise only the audio capture and audio presentation parts of the audio subsystem such that in some embodiments of the apparatus the microphone (for audio capture) or the speaker (for audio presentation) are present. Similarly in some embodiments the apparatus 10 can comprise one or the other of the video capture and video presentation parts of the video subsystem such that in some embodiments the camera 51 (for video capture) or the display 52 (for video presentation) is present.

In some embodiments the apparatus 10 comprises a processor 21. The processor 21 is coupled to the audio subsystem and specifically in some examples the analogue4odigrtal converter 18 for receiving digital signals representing audio signals from the microphone 11, and the digitakoanalogue converter (DAC) 12 configured to output processed digital audb &gnals, the camera 51 for receiving digital signals representing video signals, and the display 52 configured to output processed digital video signals from the processor 21.

The processor 21 can he configured to execute various program codes. The implemented program codes can comprise for example image capture control, and image processing.

In some embodiments the apparatus further comprises a memory 22. In some embodiments the processor 21 is coupled to memory 22. The memory 22 can be any suitable storage means, In some embodiments the memory 22 comprises a program code section 23 for storing program codes implementable upon the processor 21 such as those code routines described herein, Furthermore in some embodiments the memory 22 can further comprise a stored data section 24 for storing data, for example image data that has been captured in accordance with the application or image data to be processed with respect to the embodiments described herein. The implemented program code stored within the program code section 23, and the data stored within the stored data section 24 can be retrieved by the processor 21 whenever needed via a memoryprocessor coupling.

In some further embodiments the apparatus 10 can comprise a user interface 15.

The user interface 15 can be coupled in some embodiments to the processor 21. In some embodiments the processor can control the operation of the user interface and receive inputs from the user interface 15. In some embodiments the user interface can enable a user to input commands to the electronic device or apparatus 10, for example via a keypad, andfor to obtain information from the apparatus 10, for example via a display which is part of the user interface 15, The user interface 15 can in some embodiments comprise a touch screen or touch interface capable of both enabling information to be entered to the apparatus 10 and further displaying information to the user of the apparatus 10.

In some embodiments the apparatus further comprises a transceiver 13, the transceiver in such embodiments can he coupled to the processor and configured to enable a communication wtth other apparatus or &ectronic devices, for example via a wir&ess communications network, The transceiver 13 or any suitable transceiver S or transmitter and/or receiver means can in some embodiments be configured to communicate with other electronic devices or apparatus via a wire or wired coupUng.

The transceiver 13 can communicate with further devices by any suitable known communications protocol, for example in some embodiments the transceiver 13 or transceiver means can use a suitable universal mobile telecommunications system (UMTS) protocoL a wireless local area network (WLAN) protocol such as for example IEEE 802.X, a suitable shortrange radio frequency communication protocol such as Bluetooth, or infrared data communicaUon pathway (IRDA).

In some embodiments the transceiver is configured to transmit and/or receive the image signals for processing according to some embodiments as discussed herein.

In some embodiments the apparatus comprises a position sensor 16 configured to estimate the position of the apparatus 10. The position sensor 16 can in some embodiments be a sateflite positioning sensor such as a GPS (Global Positioning System), GLONASS or Galileo receiver.

In some embodiments the positioning sensor can be a cellular ID sysiem or an assisted GPS system.

In some embodiments the apparatus 10 further comprises a direction or orientation sensor. The orientation/direction sensor can in some embodiments be an electronic compass, accelerometer, a gyroscope or be determined by the motion of the apparatus using the positioning estimate.

It is to he understood again that the structure of the apparatus 10 could be supplemented and varied in many ways.

With respect to Figure 5 an example contrast detection autofocus apparatus according to some embodiments is shown and in particuar a focus window generator apparatus or means for generating a focus window. Furthermore with respect to Figure 6 a flow diagram is shown detalUng the operation of the contrast detection autofocus apparatus shown in Figure 5 or method of generating a focus window according to some embodiments. The example apparatus shown in Figure 5 shows the incorporation of aH four aspects as described herein. However it would be understood that in some embodiments the functional blocks or apparatus performing some of the aspects can be removed or bypassed to form an apparatus configured to implement at east one of the aspects described herein. Simflarly it would be understood that in some embodiments the functional blocks can be switched on or off thus generating or producing focus windows according to at least one of the aspects described herein.

In some embodiments the contrast detection autofocus apparatus is configured to receive the image data from the camera 51.

The contrast detection autofocus apparatus in some embodiments comprises an image signal processor 403. The image signal processor 403 is configured to receive the image data from the camera 51.

The operation of receiving the image data is shown in Figure 6 by step 501.

Furthermore in some embodiments the contrast detection autofocus apparatus comprises a micmoblock determiner 401 or suitable means for determining a number of microblocks. The microblock determiner 401 is configured to generate parameters which can be used to define a grid of microblock focus windows and which can be used by the image signal processor to generate the suitable microblock focus window focus values, For example, such as shown in Figure 5, the microblock determiner 401 can be configured to generate a parameter determining a number of microblocks to be analysed (num blocks), the size of the microblock to be analysed (block size) and the location of the microblock to he analysed (block location) and pass this information to the image signal processor.

The operation of generating microblock parameters is shown in Figure 6 by step 503.

In the following example the microblock determiner 401 can be configured to generate parameters which define microblocks over a target area or region which dIvide the target area Into a non-overlapping 14x14 grid or array of microblocks.

However it would be understood that in embodiments the number, size, and location of the microblocks can be chosen differently.

The image signal processor 403, having received the image data from the camera and the microblock parameter data from the microblock determiner 401 can be configured to generate a suitable image processing focus value for each of the microbiocks. This can for example be within a suitable means for determining at least one focus value. The microblocks in the example shown herein are rectangular or square focus window blocks and as such enables the image signal processor to implement any suitable Image signal processor focus window focus value determination.

The image signal processor 403 can In some embodiments be configured to output the microblock focus values to a low pass filter 405.

The operation of generating a grid of focus microblock values is shown in Figure 6 by step 505.

In some embodiments the contrast detectIon autolbcus apparatus comprises a low pass fIlter 405 or suitable means for spatial low pass filtering. The low pass filter 405 can be configured to apply a spatial low pass filtering to the microblock focus values.

An example low pass filtering operation can for example be a two dimensional low pass filtering for calculating the central value from the central value and the direct neighbours using the following mask values: 0.71.00.7 1.02.2 1.0 0.7 1.0 0.7 The output of the low pass filter 405 can then be passed to a macroblock grouper 407.

The operation of ow pass filiering the microblocks is shown in Figure 6 by step 507.

In some embodiments the contrast detection autofocus apparatus comprises a macroblock grouper 407 or suitable means for determining at least one macroblock comprising a number of microblocks, The macroblock grouper 407 is configured to receive the ow pass filtered microblock focus values and select some of these values to pass to a shape filter 409. In other words the macroblock grouper 407 can be configured to define or determine a macroblock comprising a grid or array of the microblocks which are passed to the shape filter 409.

In some embodiments the macroblock grouper 407 can generate or define macroblocks which overlap. In other words select define at least two macroblocks with at least some of the same microbtocks (and their associated microblock focus values). For example Figure 9 shows the target area within which the 14x14 array of microblocks are defined and within the target area a 3x3 array of overlapping macro blocks. The overlapping macroblocks shown in Figure 9 are a top left macroblock 801, a top centre macroblock 803, a top right macroblock 805, a centre left macroblock 807, a centre centre macroblock 809, a centre right macroblock 8111 a bottom left macroblock 813, a bottom centre macroblock 8151 and a bottom right macroblock 817.

In the example shown in Figure 9 each macroblock is formed from a 8x8 selection of microblocks which generates an overlapping area for a macroblock of 1/3 with any neighbouring macroblock in a horizontal or vertical direction and 1/9 with any diagonal neighbouring macroblock, however it would be understood that the degree of overlapping can he greater than or less than that described herein. Furthermore the macroblock grouper can select any suitable shape of configuration of microblocks (and their associated microblock focus values). For example in some embodiments the macroblock grouper can perform a uniform shaping filtering by selecting the microblocks using a defined shape pattern or mask. To generate the overlapping macroblocks the defined shape pattern or mask can be moved over the microblock away a defined number of microbtocks at a lime.

The output of the macroblock grouper 407 can then be passed to a shape filter 409.

The operation of grouping the filtered microblocks to generate macroblocks is shown in Figure 6 by step 509.

In some embodiments the contrast detection autofocus apparatus comprises a shape filter 409 or means for applying a spatial shaping filter for shaping the mactoblock, The shape filter 409 is configured to receive the selected microblocks on a macrobiock by macroblock basis and apply a weighting value to each of the microblocks within the macroblock to generate a suItable shape.

In some embodiments the shape filter 409 can further be configured to perform centre weighting by applying a centre weighting variation to the microblock weighting values, for example by a suitable means for applying a centre weighting filtering.

For example a rhombus type shape filter can be Implemented on a selected macroblock comprising an 8 x 8 grid of microblock values by applying the following weighting mask a, a, a, b, b, a, a, a, a,a,b,c,c,b,a,a, a, b, c, c, c, c, b, a, b, c, c, ci, ci,c, c, b, b, a, c,d, ci, a, a, b, a,b,c,c,c,c,b,a, a, a, b, a, a, b, a, a, a, a, a, b, b, a, a, a Where a uniform weighting has the values a=0 and b=c=d=1. This uniform weighting Is similar to q) c However in some embodiments a centre weighting filtering operation varies the values of b. c and d such that d > a> b. An example of which would be o.oo b1.O5 c"1.89 d2i5 The operation of shape fUtering the macroblock comprising the micrablocks to farm shaped focus values, and furthermore optionaUy to centre weight the shape focus values is shown in Figure 6 by step 511.

The macroblock shaped focus values can then be output and a suitable autofocus operation performed on the focus window focus values, With respect to Figure 8 example (quantized) rhombus shaped focus windows with a centre weighting function is shown for each of the 9 example overlapping macroblocks as shown in Figure 9. Thus Figure 8 shows a top left rhombus shaped focus window 701, a top centre rhombus shaped focus window 703, a top right rhombus shaped focus window 705, a centre left rhombus shaped focus window 707, a centre centre rhombus shaped focus window 709, a centre right rhombus shaped focus window 7111 a bottom left rhombus shaped focus window 713, a bottom centre rhombus shaped focus window 715, and a bottom right rhombus shaped focus window 717.

With respect to Figure 10 the combined centres of the nine rhombus shaped focus windows formed from the macro blocks across the target area are shown.

Furthermore with respect to Figure lithe application of embodiments as described herein for an example image such as shown in Figure 3 is shown. The graph as shown in Figure 11 show the focus curves created by the focus windows according to the embodiments described herein produce a result with simulated hand shaking which are significantly more similar to the static camera operation and thus would provide a reliable and more accurate results.

In the descriplion herein the components can be considered to be implernentable in some embodiments at least partially as code or rouUnes operating within at least one processor and stored in at least one memory.

It shall be appreciated that the term user equipment is intended to cover any sudable type of wireless user equipment, such as moblle telephones, portable data processing devices or portable web browsers.

Furthermore elements of a pubhc land mobfle network (PLMN) may also comprise apparatus as described above.

In general, the various embodiments of the invention may be implemented in hardware or special purpose circuits, software, logic or any combination thereof. For example, some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software which may be executed by a controHer, microprocessor or other computing device, although the invention is not limited thereto. Whe various aspects of the invention may be illustrated and described as block diagrams, flow charts, or using some other pictorial representation, it is well understood that these blocks, apparatus, systems, techniques or meLhods deschbed herein may be implemented in, as nonlimiting examples, hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof.

The embodiments of this invention may be implemented by computer software executable by a data processor of the mobile device! such as in the processor entity, or by hardware, or by a combination of software and hardware. Further in this regard it should he noted that any blocks of the logic flow as in the Figures may represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions, The software may be stored on such physical media as memory chips, or memory blocks implemented within the processor, magnetic media such as hard disk or floppy disks! and optical media such as for example DVD and the data variants thereof, CD.

The memory may be of any type suitable to the local technical environment and may be implemented using any suitable data storage technology, such as semiconductorThased memory devices, magnetic memory devices and systems, optical memory devices and systems, fixed memory and removable memory. The data processors may be of any type suitable to the local technical environment, and may include one or more of general purpose computers, special purpose computers, microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASIC), gate level circuits and processors based on multicore processor architecture, as nonHmiting examples.

Embodiments of the inventions may he practiced in various components such as integrated circuit modules. The design of integrated circuits is by and large a highly automated process. Complex and powerful software tools are available for converting a logic level design into a semiconductor circuit design ready to be etched and formed on a semiconductor substrate.

Programs, such as those provided by Synopsys, Inc. of Mountain View, California and Cadence Design, of San Jose, California automatically route conductors and locate components on a semiconductor chip using well established rules of design as well as Ubraries of pr&stored design modules. Once the design for a semiconductor circuit has been completed, the resultant design, in a standardized electronic format (e.g., Opus, GDSII, or the like) may be transmitted to a semiconductor fabrication facility or fab for fabrication.

The foregoing description has provided by way of exemplary and nonUmiting examples a full and informative description of the exemplary embodiment of this invention. However, various modifications and adaptations may become apparent to those skilled in the relevant arts in view of the foregoing description, when read in conjunction with the accompanying drawings and the appended claims. However, all such and similar modifications of the teachings of this invention will still fall within the scope of this invention as defined in the appended claims.

Claims

CLMMS: 1. An apparatus comprising: a focus wndow generator configured to generate a focus window for use in a contrast autofocus apparatus, the focus window generator comprising a focus window border with no substantiaUy continuous vertical or horizontal components.
2. The apparatus as claimed in claim 1, wherein the focus window generator comprises: a microblock determiner configured to define a number of microblocks, each microblock defined by a size and location: an image signal processor configured to receive an image and further configured to determine at east one focus value for the defined microbbcks with respect to the image; a macroblock grouper configured to determine at east one macroblock comprising a further number of microblocks from the number of microblocks; and a shape ifiter configured to apply a spatial shaping filtering to the at least one focus value for the determined further number microblocks associated with the at least one macroblock to generate the focus window.
3. The apparatus as claimed in daim 2, wherein the shape filter configured to apply a spatial shaping filtering to the at least one focus value for the determined microblocks associated with the at least one macroblock to generate the focus window is further configured to apply a centre biased spatial shaping filtering to the at least one focus value for the determined microblocks assocated with the at least one macroblock,
4. The apparatus as claimed in claims 2 and 3, wherein the shape filter comprises a mask generator configured to generate a mask of values to be apped to the determined microblock focus values associated with the at least one macro block.
5. The apparatus as claimed in claims 2 to 4, whereIn the macroblock grouper is configured to determine at least two macroblocks, wherein the at least two macroblocks comprise at least one microblock in common.
6. The apparatus as claimed in claims 2 to 5, wherein the focus window generator comprises a spatial low pass filter configured to output a filtered focus value for a microblock, wherein the macroblock grouper is configured to determine at least one macroblock comprising a further number of microblock filtered focus values.ID7. The apparatus as claimed In claim 6, wherein the spatial low pass filter Is configured to output a filtered focus value for a microblock based on the microblock focus value and neighbouring microblock focus values.8. The apparatus as claimed in claims I to 7 wherein the focus window generator comprising a focus window border with no substantially continuous vertical or horizontal components comprises a spatially quantized rhombus.9. An apparatus comprising at least one processor and at least one memory including computer code for one or more programs, the at least one memory and the computer code configured to with the at least one processor cause the apparatus to: generate a focus window for use in a contrast autofocus apparatus, the focus window generator comprising a focus window boiter with no substantially continuous vertical or horizontal components.10. The apparatus as claimed In claim 9, wherein generating the focus window causes the apparatus to: define a number of microblocks, each microblock defined by a size and location; receive an Image and further configured to determine at least one focus value for the defined mlcroblocks with respect to the image: determine at least one macroblock comprising a further number of microblocks from the number of microblocks; and apply a spatial shaping filter to the at least one focus value for the determined further number microblocks associated with the at least one macroblock to generate the focus window.II. The apparatus as claimed in claim 10, whereIn applying a spatial shaping filtering to the at least one focus value for the determined microblocks associated with the at least one macroblock to generate the focus window further causes the apparatus to apply a centre biased spatial shaping filter to the at least one focus value for the determined mlcroblocks associated with the at least one macrobiock.12. An apparatus comprising: means for generating a focus window for use in a contrast autofocus apparatus, the focus window generator comprising a focus window border with no substantially continuous vertical or horizontal components.13. The apparatus as claimed in claim 12, wherein the means for generating the focus window comprises: means for defining a number of microblocks, each microblock defined by a size and location; means for receiving an image and further configured to determine at least one focus value for the defined microblocks with respect to the image; means for determining at least one macroblock comprising a further number of microblocks from the number of microblocks; and means for applying a spatial shaping filter to the at least one focus value for the determined further number microblocks associated with the at least one macroblock to generate the focus window.14. The apparatus as claimed in claim 13, whereIn the means for applying a spatial shaping filtering to the at least one focus value for the determined microblocks associated with the at least one macroblock to generate the focus window further comprises means for applying a centre biased spatial shaping filter to the at least one focus value for the determined microblocks associated with the at least one macroblock.15. A method comprising: generaUng a focus window for use in a contrast autofocus apparatus, the focus window generator comprising a focus window border with no substantially continuous vertical or hohzontal components.16, The method as daimed in claim 15, wherein generating the focus window comprises: defining a number of microblocks, each microblock defined by a &ze and location; receiving an image and further configured to determine at east one focus value for the defined rnicrobbcks with respect to the image; determining at least one macroblock comprising a further number of microblocks from the number of microblocks; and applying a spatial shaping fflter to the at east one focus value for the determined further number microblocks associated with the at least one macroblock to generate the focus window.17. The method as claimed in claim 16, wherein applying a spatial shaping fiftering to the at least one focus value for the determined microblocks associated with the at east one macroblock to generate the focus window further comprises applying a centre biased spatial shaping filter to the at least one focus value for the determined microblocks associated with the at east one macroblock.