CN109224291B

CN109224291B - Image processing method and device of retina stimulator and retina stimulator

Info

Publication number: CN109224291B
Application number: CN201810924696.5A
Authority: CN
Inventors: 王追; 陈大伟; 陈志�; 钟灿武; 夏斌; 赵瑜
Original assignee: Shenzhen Sibionics Technology Co Ltd
Current assignee: Shenzhen Silicon Bionics Technology Co ltd
Priority date: 2017-12-29
Filing date: 2018-08-14
Publication date: 2021-03-02
Anticipated expiration: 2038-08-14
Also published as: CN112972889A; CN112972889B; CN109224291A

Abstract

The present disclosure provides an image processing method of a retinal stimulator having a predetermined number of stimulation electrodes, characterized in that: the method comprises the following steps: an image acquisition step for acquiring an initial image; a graying step, which is used for carrying out graying processing on the initial image to obtain a grayscale image; a low-pixelation step, which is used for compressing the pixels of the gray level image to obtain a low-pixel gray level image, wherein the number of the pixels of the low-pixel gray level image is less than or equal to the specified number of the stimulation electrodes; and a binarization step, which is used for carrying out binarization processing on the low-pixel gray level image to obtain a binary image, wherein the stimulation electrode generates an electrical stimulation signal according to the binary image. According to the present disclosure, more useful information of an image can be provided by optimizing the process of capturing an image.

Description

Image processing method and device of retina stimulator and retina stimulator

Technical Field

The disclosure relates to the technical field of bionics, in particular to an image processing method and device of a retina stimulator and the retina stimulator.

Background

Normal vision is developed by the photoreceptor cells on the retina within the eyeball converting external light signals into visual signals. Visual signals reach the cerebral cortex via bipolar cells and ganglion cells, creating light sensation. However, in life, patients have blocked light sensing pathways due to retinal diseases such as RP (retinitis pigmentosa), AMD (age-related macular degeneration), and the like, resulting in visual deterioration or blindness.

With the research and development of technology, there has been a technical means for repairing the above-mentioned retinal diseases using a retinal stimulator or the like, by which the brain can receive an external stimulus signal and obtain improved vision. In order to restore partial vision to a patient, it is generally necessary to implant an implant within the eyeball of the patient and to arrange an image processing device and a camera device outside the patient's body, which communicate with the implant. After capturing an external image, the in-vitro camera device processes the external image through the image processing device and sends a processed image signal to the implant. The implant further converts these image signals into electrical stimulation signals to stimulate ganglion cells or bipolar cells on the retina, thereby producing light perception to the patient.

However, in the existing retinal stimulator, the image captured by the camera device includes parameters such as brightness, hue and saturation, and the number of pixels of the image is often relatively large, in this case, on one hand, the amount of image information captured by the camera device is relatively large, the image processing device needs to process the image information relatively large, and the complexity is relatively high, on the other hand, the stimulation electrodes disposed on the implant of the eyeball of the patient are very limited, for example, 60 electrodes, and the information receiving capability is very limited, and if the image information captured by the camera device of the retinal stimulator directly corresponds to the limited stimulation electrodes implanted in the patient, a large amount of data loss is caused, and image distortion is caused. The lost data may be basic information of the image, such as a basic outline of an obstacle in the image, so that the obstacle is difficult to be identified by the patient, and inconvenience is brought to the life of the patient.

Disclosure of Invention

The present disclosure has been made in view of the above-mentioned state of the art, and an object thereof is to provide an image processing method and apparatus of a retinal stimulator capable of providing more useful information of an image by optimizing a process of capturing an image, and a retinal stimulator.

To this end, a first aspect of the present disclosure provides an image processing method of a retinal stimulator having a prescribed number of stimulation electrodes, characterized in that: the method comprises the following steps: an image acquisition step for acquiring an initial image; a graying step, which is used for carrying out graying processing on the initial image to obtain a grayscale image; a low-pixelation step of compressing pixels of the gray-scale image to obtain a low-pixel gray-scale image, wherein the number of pixels of the low-pixel gray-scale image is less than or equal to the specified number of the stimulation electrodes; and a binarization step, configured to perform binarization processing on the low-pixel grayscale image to obtain a binary image, where the stimulation electrode generates an electrical stimulation signal according to the binary image.

In the present disclosure, a binary image is obtained by performing a graying process, a low pixelation process, and a binarization process on an initial image. The number of pixels of the low-pixel gray image obtained after the low-pixelation process is less than or equal to the prescribed number of the stimulation electrodes. In this case, the process of capturing the image can be optimized and each pixel of the low-pixel binary image can be made to correspond to a stimulation electrode within the implant device of the retinal stimulator, thereby providing the patient with more useful information of the image that helps to resolve objects or obstacles.

In the image processing method of a retinal stimulator according to the first aspect of the present disclosure, optionally, the step of low-pixelating includes: carrying out partition processing on the gray level image to obtain a plurality of gray level image areas, wherein each gray level image area comprises a plurality of pixels; calculating an average gray value of pixels for any one gray image area among the plurality of gray image areas, and taking the average gray value as a gray value of the gray image area; each gray scale image area of the gray scale image is taken as a pixel with an average gray scale value to obtain a low pixel gray scale image. In this case, the gray image is subjected to low pixelation processing to extract useful information of the image in a subsequent step.

In the image processing method of a retinal stimulator according to the first aspect of the present disclosure, optionally, the step of low-pixelating includes: calculating gradient values of the gray level images along a preset direction; determining pixels of which the gradient values in the preset direction of the gray image are greater than or equal to a preset gradient value, and taking the pixels of which the gradient values in the preset direction of the gray image are greater than or equal to the preset gradient value as effective pixels; partitioning effective pixels in the gray level image to obtain a plurality of pixel areas, wherein each pixel area comprises a plurality of effective pixels; calculating an average gray value of pixels for any one of the plurality of pixel regions, and taking the average gray value as a gray value of the pixel region; each pixel region of the grayscale image is treated as a pixel having an average grayscale value to obtain a low-pixel grayscale image. In this case, the gray image is subjected to low pixelation processing to extract useful information of the image in a subsequent step.

In the image processing method of a retinal stimulator according to the first aspect of the present disclosure, optionally, the step of low-pixelating includes: carrying out partition processing on the gray level image to obtain a plurality of gray level image areas, wherein each gray level image area comprises a plurality of pixels; calculating an average gray value of pixels for any gray image area in the plurality of gray image areas, comparing the average gray value in the gray image with a preset average gray value, and determining an effective gray image area in the gray image area; and taking each gray image area of the effective gray image as a pixel with an average gray value to obtain a low-pixel gray image. In this case, the gray image is subjected to low pixelation processing to extract useful information of the image in a subsequent step.

Further, in the image processing method of a retinal stimulator according to the first aspect of the present disclosure, optionally, the binarizing step includes: comparing the gray value of each pixel in the low-pixel gray image with a preset gray value; according to the comparison result, the gray values in the low-pixel gray image can be set into two types, namely a maximum gray value and a minimum gray value, and the binary image can be obtained after the gray values are changed. In this case, the maximum gray scale value and the minimum gray scale value of the pixels of the binary image can be represented by high and low levels, and thus can better correspond to the stimulation signals of the stimulation electrodes.

A second aspect of the present disclosure provides an image processing apparatus of a retinal stimulator having a prescribed number of stimulation electrodes, characterized by comprising: an acquisition unit for acquiring an initial image; the gray processing unit is used for carrying out gray processing on the initial image to obtain a gray image; a pixel processing unit, configured to perform compression processing on pixels of the grayscale image to obtain a low-pixel grayscale image, where the number of pixels of the low-pixel grayscale image is less than or equal to the specified number of the stimulation electrodes; and the binarization processing unit is used for carrying out binarization processing on the low-pixel gray level image to obtain a binary image, wherein the stimulation electrode generates an electrical stimulation signal according to the binary image.

In the present disclosure, the initial image is processed by a gradation processing unit, a pixel processing unit, and a binarization processing unit to obtain a binary image, wherein the number of pixels of the low-pixel gradation image obtained by the pixel processing unit is less than or equal to the prescribed number of the stimulation electrodes. In this case, the process of capturing the image can be optimized and each pixel of the low-pixel binary image can be made to correspond to a stimulation electrode within the implant device of the retinal stimulator, thereby providing the patient with more useful information of the image that helps to resolve objects or obstacles.

In the image processing apparatus of the retinal stimulator according to the second aspect of the present disclosure, the pixel processing unit may optionally include: a first partitioning subunit, configured to perform partitioning processing on the grayscale image to obtain a plurality of grayscale image regions, where each grayscale image region includes a plurality of pixels; a first acquisition subunit, configured to calculate an average grayscale value of pixels for any grayscale image region among the plurality of grayscale image regions, and use the average grayscale value as a grayscale value of the grayscale image region; a first pixel processing subunit for treating each gray image area of the gray image as one pixel with an average gray value to obtain a low pixel gray image. In this case, the gray image is subjected to low pixelation processing to extract useful information of the image in a subsequent step.

In the image processing apparatus of the retinal stimulator according to the second aspect of the present disclosure, the pixel processing unit may optionally include: a calculation subunit, configured to perform gradient value calculation on the grayscale image along a preset direction; a determining subunit, configured to determine pixels of the grayscale image having a gradient value in the preset direction greater than or equal to a preset gradient value, and to take the pixels of the grayscale image having a gradient value in the preset direction greater than or equal to the preset gradient value as effective pixels; the second partitioning subunit is used for partitioning effective pixels in the grayscale image to obtain a plurality of pixel areas, and each pixel area comprises a plurality of effective pixels; a second acquisition subunit, configured to calculate an average grayscale value of pixels for any one of the plurality of pixel regions, and use the average grayscale value as a grayscale value of the pixel region; a second pixel processing subunit for treating each pixel region of the grayscale image as one pixel having an average grayscale value to obtain a low-pixel grayscale image. In this case, the gray image is subjected to low pixelation processing to extract useful information of the image in a subsequent step.

In the image processing apparatus of the retinal stimulator according to the second aspect of the present disclosure, the pixel processing unit may optionally include: a third partitioning subunit, configured to perform partitioning processing on the grayscale image to obtain a plurality of grayscale image regions, where each grayscale image region includes a plurality of pixels; a third obtaining subunit, configured to calculate an average gray scale value of pixels for any one of the gray scale image regions, compare the average gray scale value in the gray scale image with a preset average gray scale value, and determine an effective gray scale image region in the gray scale image region; a third pixel processing subunit, configured to treat each gray image area of the effective gray image as a pixel having an average gray value to obtain a low-pixel gray image. In this case, the gray image is subjected to low pixelation processing to extract useful information of the image in a subsequent step.

In addition, in the image processing device of a retinal stimulator according to a second aspect of the present disclosure, the binarization processing unit may optionally include: the comparison subunit is used for comparing the gray value of each pixel in the low-pixel gray image with the preset gray value; and the processing subunit is used for setting the gray values in the low-pixel gray image into two types according to the comparison result, namely a maximum gray value and a minimum gray value, and obtaining the binary image after changing the gray values. In this case, the maximum gray value and the minimum gray value of the pixels of the binary image can be represented by high and low levels, and can better correspond to the stimulation signals of the stimulation electrodes.

Further, a third aspect of the present disclosure provides a retinal stimulator characterized by comprising an image pickup device, a video processing device, and an implantation device, wherein: the camera device is used for capturing a video image and converting the video image into a visual signal; the video processing device at least comprises the image processing device, the video processing device is connected with the camera device, and the video processing device is used for processing the visual signals and sending the visual signals to the implantation device through a transmitting antenna; the implant device is used for converting the received visual signals into bidirectional pulse current signals serving as electric stimulation signals, so that the bidirectional pulse current signals are distributed to ganglion cells or bipolar cells of retina to generate light sensation.

According to the present disclosure, it is possible to provide an image processing method and apparatus of a retinal stimulator capable of providing more useful information of an image by optimizing a process of capturing an image, and the retinal stimulator.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present disclosure, and it is obvious for those skilled in the art to obtain other drawings based on the drawings without creative efforts.

Fig. 1 is a schematic view of a retina stimulator according to the present disclosure.

Fig. 2 is a schematic configuration diagram of an image processing apparatus of a retinal stimulator according to the present disclosure.

Fig. 3A is a block diagram of an example of a pixel processing unit in the image processing apparatus of the retinal stimulator according to the present disclosure.

Fig. 3B is a block diagram of modification 1 of the pixel processing unit in the image processing apparatus of the retinal stimulator according to the present disclosure.

Fig. 3C is a block diagram of modification 2 of the pixel processing unit in the image processing apparatus of the retinal stimulator according to the present disclosure.

Fig. 4 is a schematic structural diagram of a binarization processing unit in the image processing device of the retinal stimulator according to the present disclosure.

Fig. 5 is a schematic configuration diagram of an image processing device of a retinal stimulator according to the present disclosure.

Fig. 6 is a flowchart illustrating an image processing method of the retinal stimulator according to the present disclosure.

Fig. 7 is a flow chart illustrating the low pixelation step of the image processing method of the retinal stimulator according to the present disclosure.

Fig. 8A is a schematic diagram of an image processing process from a gray image to a low-pixel gray image based on fig. 7.

Fig. 8B is a schematic diagram of the image processing from the grayscale image to the binary image based on fig. 8A.

Fig. 9 is a flowchart schematically illustrating a modification 1 of the low pixelation step of the image processing method of the retinal stimulator according to the present disclosure.

Fig. 10A is a schematic diagram of an image processing process from a gray image to a low-pixel gray image based on fig. 9.

Fig. 10B is a schematic diagram of an image processing process from a grayscale image to a binary image based on fig. 10A.

Fig. 11 is a flowchart schematically illustrating a modification 2 of the low pixelation step of the image processing method of the retinal stimulator according to the present disclosure.

Fig. 12A is a schematic diagram of an image processing process from a gray image to a low-pixel gray image based on fig. 11.

Fig. 12B is a schematic diagram of the image processing from the grayscale image to the binary image based on fig. 12A.

Fig. 13 is a schematic flowchart of the binarization step in the image processing method of the retinal stimulator according to the present disclosure.

Fig. 14 is a schematic diagram of the image processing effect of the retinal stimulator according to the present disclosure.

Detailed Description

Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the following description, the same components are denoted by the same reference numerals, and redundant description thereof is omitted. The drawings are schematic and the ratio of the dimensions of the components and the shapes of the components may be different from the actual ones.

In addition, the headings and the like referred to in the following description of the present disclosure are not intended to limit the content or scope of the present disclosure, but merely serve as a reminder for reading. Such a subtitle should neither be understood as a content for segmenting an article, nor should the content under the subtitle be limited to only the scope of the subtitle.

(retina stimulator)

Fig. 1 is a schematic view of a retina stimulator according to the present disclosure. The retinal stimulator 1 of the present disclosure may be suitable for patients who have retinopathy leading to blindness, but whose visual pathways remain intact, such as bipolar cells, ganglion cells, etc. In the present disclosure, the retinal stimulator 1 is also sometimes referred to as "artificial retina", "artificial retinal system", or the like.

In some examples, as shown in fig. 1, the retinal stimulator 1 may include an implant device 10, a camera device 20, and a video processing device 30. The implant device 10 may receive the visual signal and generate an electrical stimulation signal based on the visual signal to create a sensation of light in the patient. Wherein, the visual signal can be collected by the camera device 20 and processed by the video processing device 30.

In some examples, the implant device 10 may include a prescribed number of stimulation electrodes. Stimulation electrodes (sometimes simply referred to as "electrodes") may generate electrical stimulation signals based on the visual signals. In particular, the implant device 10 may receive visual signals and the stimulation electrodes may convert the received visual signals into bi-directional pulsed current signals as electrical stimulation signals, thereby delivering bi-directional pulsed current signals to ganglion cells or bipolar cells of the retina to produce light sensation. Alternatively, the implant device 10 may be implanted in a human body, such as an eyeball.

In some examples, the visual signals received by the implant device 10 may be captured and processed by the camera device 20 and the video processing device 30.

In some examples, the camera 20 may be used to capture video images and convert the video images into visual signals. For example, the camera 20 may capture video images of the environment in which the patient is located.

In some examples, the image capture device 20 may be an apparatus having an image capture function, such as a video camera, a still camera, or the like. For ease of use, a camera of smaller volume may be designed on (e.g., embedded in) the eyewear.

In other examples, the patient may also capture video images by wearing lightweight camera-enabled glasses as the camera 20. The imaging device 20 may be implemented by google glasses or the like. In addition, the camera device 20 may be mounted on smart wearable devices such as smart glasses, smart headsets, and smart bracelets.

In some examples, the video processing device 30 may receive visual signals generated by the camera device 20. The video processing device 30 may process the visual signal and send it to the implanted device 10 via the transmitting antenna.

In some examples, the camera device 20 is connected to the video processing device 30. The image pickup device 20 and the video processing device 30 may be connected by wire or wirelessly.

In some examples, the wired connection may be a data line connection, the wireless connection may be a bluetooth connection, a WiFi connection, an infrared connection, an NFC connection, or a radio frequency connection, among others.

In some examples, the camera device 20 and the video processing device 30 may be configured outside the patient's body. For example, the patient may wear the imaging device 20 on glasses. The patient may also wear the camera device 20 on a wearable accessory such as a headgear, hair band, or brooch. The patient can wear the video processing device 30 on the waist, and the patient can wear the video processing device 30 on the arm, leg, or the like. Examples of the present disclosure are not limited thereto, and for example, the patient may also place the video processing device 30 in, for example, a handbag or backpack that is carried around.

(image processing apparatus)

Fig. 2 is a schematic configuration diagram of an image processing apparatus of a retinal stimulator according to the present disclosure. The image processing apparatus 300 of the retinal stimulator 1 (which may be simply referred to as the image processing apparatus 300) according to the present disclosure may be used for the retinal stimulator 1 as a functional block of image processing. Specifically, the image processing apparatus 300 may be included in the video processing apparatus 30 of the retinal stimulator 1. In the present disclosure, by the image processing apparatus 300, it is possible to optimize the processing of an image and provide more useful information of the image.

In the present disclosure, "image useful information" refers to information useful for patients who use the image processing apparatus of the present disclosure, for example, for these patients, contour information of objects or obstacles appearing in the image is useful, and such information can help the patients and the like avoid the objects or obstacles. It is useful for current retinal stimulators or artificial retinas to help patients identify and avoid objects or obstacles.

In some examples, as shown in fig. 2, the image processing apparatus 300 may include an acquisition unit 310. The acquisition unit 310 may be used to acquire an initial image.

In some examples, the initial image may be acquired by the camera 20. The pixels of the imaging lens (not shown) of the imaging device 20 may be, for example, 30, 100, 200, 500, 1200, or the like. In the present embodiment, the pixels of the initial image are determined by the pixels of the imaging lens, and the number of the pixels of the initial image may be, for example, 30 ten thousand, 100 ten thousand, 200 ten thousand, 500 ten thousand, 1200 ten thousand, or the like corresponding to the number of the pixels matching the lens.

In some examples, the initial image may be an image captured by the camera 20 without any processing. In general, the initial image obtained by capturing the surrounding environment by the image capturing device 20 is a color image. That is, the initial image captured by the image capturing apparatus 20 without any processing may be a color image. In some examples, the color image may be an HSI image. An HSI image is an image model that reflects the morphological features of an image. The color image may also be an RGB image. An RGB image is an image model in which color adjustment is performed in principle from optics.

In some examples, an HSI image can perceive color through three basic characteristic quantities of hue, saturation (i.e., color saturation), and brightness. Based on the HSI image model, image information such as brightness, hue, and saturation may be included in each pixel of the color image.

In some examples, based on the RGB image model, the color of each pixel in the initial image may be determined by three components, red (R), green (G), and blue (B). That is, each pixel comprises R, G, B three pixel sub-units, and if each color component can be represented by, for example, an 8-bit binary number, then 0-255 values for each component can be selected. Examples of the present disclosure are not limited thereto, and each color component may also be represented by, for example, a 16-bit binary number, and each color component may also be represented by, for example, a binary number of 24 bits and more.

Generally, the appearance of objects or obstacles in an image is the information of major interest to a patient, and in particular, recognizing the outline of an object or obstacle is advantageous for the blind or low-vision patient. On the other hand, since information such as color features in a color image is not always used to reflect morphological features of an object in the image, the information can relatively well retain the contour of the object or an obstacle even if a portion of the color image is removed. On the other hand, the number of electrodes of the implant device 10 of the retinal stimulator 1 is still relatively small at present, and the number of electrodes is, for example, 60, 100, 150 or 256. Relatively few electrodes generally have difficulty in fully conveying all of the information of the initial image, and often have difficulty conveying information such as the contours of objects or obstacles in the initial image. In such a case, since the information of a large number of pixels of the initial image is directly associated with the extremely limited number of stimulation electrodes in the implant device 10 of the retinal stimulator 1, the amount of information of the pixels of the initial image cannot be completely reflected by the stimulation electrodes, and thus the image is easily distorted to a great extent. Based on this, the inventors found from experience of practical use that, by performing the gradation processing on the initial image, even when the number of electrodes is small and the ability to receive information is limited, the processing of the image can be optimized, and useful information of the image such as the outline of an object or an obstacle can be retained as much as possible.

In some examples, as shown in fig. 2, the image processing apparatus 300 may further include a grayscale processing unit 320. The gray processing unit 320 may perform a graying process on the initial image to obtain a gray image. The grayscale image may be R, G, B a special color image with the same size of three components (i.e., R ═ G ═ B), and the amount of information is less than that of a normal color image. Each pixel of a grayscale image has a corresponding grayscale value. In some examples, each gray value may be represented using, for example, an 8-bit binary number, i.e., the gray value of the gray image ranges from 0-255. In other examples, each gray value may be represented by a 16-bit binary number, for example, or may be represented by a binary number of 24 bits or more, for example.

In some examples, the graying process mainly processes color information of an image, and initial image information other than the color information is not changed. For example, the graying process can help to provide more useful information of the image, such as morphological feature information of objects or obstacles in the image.

In some examples, the graying processing method may be a component method, i.e., a value of any one of the three components may be selected R, G, B as the grayscale value. For example, if R ═ 70, G ═ 110, and B ═ 150 for one pixel, then 70, for example, may be selected as the grayscale value of the pixel, i.e., R ═ G ═ B ═ 70 may be set as the grayscale value of the pixel; for example, 110 may be selected as the gradation value of the pixel, and for example, 150 may be selected as the gradation value of the pixel.

In addition, in some examples, the graying processing method may also be a maximum value method, i.e., the maximum value of the three components may be selected R, G, B as the grayscale value. For example, if R is 70, G is 110, and B is 150 for one pixel, 150 may be selected as the grayscale value of the pixel.

In addition, in some examples, the graying processing method may also be an average value method, that is, an average value of R, G, B three components may be selected as the grayscale value. For example, if R is 70, G is 110, and B is 150 for one pixel, the average of the R, G, B values is 110, and 110 may be selected as the grayscale value of the pixel.

In addition, in some examples, the graying processing method may also be a weighting method, that is, R, G, B three components may be weighted according to different weighting coefficients to obtain the grayscale value. For example, if R is 70, G is 110, and B is 150, R and G are 0.3, G is 0.5, and B is 0.2, then the gray-scale value of the pixel is 0.3 +0.5 + 110+0.2 and 150 is 106. In the above example, the graying processing can reduce the data amount (or information amount) of the initial image, facilitate the subsequent processing of the image, and contribute to highlighting useful information in the image at the time of the subsequent processing.

In some examples, as shown in fig. 2, the image processing apparatus 300 may further include a pixel processing unit 330. The pixel processing unit 330 may be configured to perform compression processing on the pixels of the grayscale image to obtain a low-pixel grayscale image. In some examples, the number of pixels of the low-pixel grayscale image may be less than or equal to the prescribed number of stimulation electrodes. By the pixel processing unit 330, the number of pixels of the grayscale image can be reduced, resulting in a low-pixel grayscale image.

In some examples, graying the original image results in a grayscale image, but the grayscale image still includes much redundant information relative to the information needed for the contours of objects or obstacles, e.g., spatial redundancy due to correlation between adjacent pixels in the image. The pixel processing unit 330 can reduce the number of pixels of the grayscale image and reduce the complexity of subsequent image processing, so as to extract useful information of the image in subsequent steps.

Fig. 3A is a block diagram of an example of a pixel processing unit in the image processing apparatus of the retinal stimulator according to the present disclosure. Fig. 3B is a block diagram of modification 1 of the pixel processing unit in the image processing apparatus of the retinal stimulator according to the present disclosure. Fig. 3C is a block diagram of modification 2 of the pixel processing unit in the image processing apparatus of the retinal stimulator according to the present disclosure.

In some examples, as shown in fig. 3A, pixel processing unit 330 may include a first partition subunit 3310, a first acquisition subunit 3311, and a first pixel processing subunit 3312. The first partition subunit 3310 may be configured to partition the grayscale image into multiple grayscale image regions. Wherein each gray scale image region may comprise a plurality of pixels.

In some examples, the first obtaining subunit 3311 may be configured to calculate an average grayscale value of the pixels for any one of the plurality of grayscale image regions, taking the average grayscale value as the grayscale value of that grayscale image region. That is, the first obtaining subunit 3311 may select any one of the plurality of grayscale image regions as the target grayscale image region, and obtain the average grayscale value of the plurality of pixels of the target grayscale image region for the target grayscale image region. Wherein the average gray value may be taken as the gray value of the target gray image area.

In some examples, the first pixel processing subunit 3312 may be used to treat each grayscale image region of the grayscale image as one pixel having an average grayscale value to obtain a low-pixel grayscale image. That is, the first pixel processing subunit 3312 may be configured to combine each of the grayscale image regions of the grayscale image as one pixel of the low-pixel grayscale image, with the respective pixels being combined in order to form the low-pixel grayscale image. The gray value of each pixel may be an average gray value of the corresponding gray image area, in which case the gray image is subjected to low pixelation processing to extract useful information of the image in the subsequent step.

In some examples, as shown in fig. 3B, pixel processing unit 330 may include a computation subunit 3320, a determination subunit 3321, a second partitioning subunit 3322, a second acquisition subunit 3323, and a second pixel processing subunit 3324. The calculation subunit 3320 may be configured to perform gradient value calculation on the grayscale image along a preset direction.

In some examples, the determining subunit 3321 may be configured to determine pixels in which a gradient value of the grayscale image in a preset direction is greater than or equal to a preset gradient value. Pixels of the gray image having a gradient value in a preset direction greater than or equal to a preset gradient value may be taken as effective pixels.

In some examples, the second partition subunit 3322 may be configured to partition the effective pixels in the grayscale image into a plurality of pixel regions. Each pixel region may include a plurality of effective pixels.

In some examples, the second obtaining subunit 3323 may be configured to calculate an average grayscale value of the pixels for any one of the plurality of pixel regions, and take the average grayscale value as the grayscale value of the pixel region. That is, the second obtaining subunit 3323 may select any one of the plurality of pixel regions as the target pixel region, and obtain the average gray scale value of the plurality of effective pixels of the target pixel region for the target pixel region. The average gray value is taken as the gray value of the target pixel region.

In some examples, the second pixel processing subunit 3324 may be configured to treat each pixel region of the grayscale image as one pixel having an average grayscale value to obtain a low-pixel grayscale image. That is, the second pixel processing subunit 3324 may be configured to take each pixel region of the grayscale image as one pixel of the low-pixel grayscale image, and combine the respective pixels in order to form the low-pixel grayscale image. The gray value of each pixel may be an average gray value of the corresponding gray image area, in which case the gray image is subjected to low pixelation processing to extract useful information of the image in the subsequent step.

In other examples, as shown in fig. 3C, pixel processing unit 330 may include a third partition subunit 3330, a third acquisition subunit 3331, and a third pixel processing subunit 3312. The third partitioning subunit 3330 may be configured to partition the grayscale image into a plurality of grayscale image regions. Each gray scale image region may include a plurality of pixels.

In some examples, the third acquisition subunit 3331 may be configured to calculate an average grayscale value of the pixels for any grayscale image region among the plurality of grayscale image regions. The average gray value in the gray image may be compared to a preset average gray value to determine an effective gray image area in the gray image area. That is, the third obtaining subunit 3331 may select any one of the grayscale image regions as the target grayscale image region, and obtain the average grayscale value of the plurality of pixels of the target grayscale image region for the target grayscale image region. The average gray value in the gray image may be compared to a preset average gray value to determine an effective gray image area in the gray image area.

In some examples, the third pixel processing subunit 3312 may treat each grayscale image region of the effective grayscale image as one pixel having an average grayscale value to obtain a low pixel grayscale image. That is, the third pixel processing subunit 3312 may be configured to combine each effective grayscale image region of the grayscale image as one pixel of the low-pixel grayscale image, combining the pixels in order to form the low-pixel grayscale image. The gray value of each pixel may be an average gray value of the corresponding gray image area, in which case the gray image is subjected to low pixelation processing to extract useful information of the image in the subsequent step.

In this case, the low-pixel grayscale image obtained by the pixel processing unit 330 described above has a reduced number of pixels as compared with the grayscale image before the low-pixelation processing, and it is possible to reduce redundancy of image data caused by correlation between adjacent pixels in the grayscale image accordingly. Thereby, useful information of the image can be extracted in the subsequent steps.

In some examples, as shown in fig. 2, the image processing apparatus 300 may further include a binarization processing unit 340. The binarization processing unit 340 may perform binarization processing on the low pixel grayscale image to obtain a binary image. Each pixel of the binary image can correspond to a stimulation electrode within the implant device 10 of the retinal stimulator 1. The stimulation electrode may generate an electrical stimulation signal from the binary image.

In some examples, as shown in fig. 4, the binarization processing unit 340 may include a comparison sub-unit 3410 and a processing sub-unit 3411. The comparing subunit 3410 may be configured to compare the gray scale value of each pixel in the low-pixel gray scale image with a preset gray scale value.

In some examples, the processing subunit 3411 may set the gray-scale values in the low-pixel gray-scale image to two types, i.e., a maximum gray-scale value and a minimum gray-scale value, according to the comparison result. And after the gray value is changed, a binary image can be obtained.

In some examples, the number of pixels of the low-pixel grayscale image is less than or equal to the prescribed number of stimulation electrodes. That is, the pixels of the low pixel gray scale image match the number of stimulation electrodes of the implant device 10 of the retinal stimulator 1. In other words, each pixel of the low pixel gray scale image may correspond to one electrode. However, each pixel of the low-pixel gray-scale image may be at least 8 bits, for example, in which case there are at least 256 possible values for each pixel, and it is generally difficult to achieve at least 256 different results by one stimulation electrode, so that the low-pixel gray-scale image is binarized to obtain a binary image. Wherein the gray value of each pixel may be the maximum gray value 255 or the minimum gray value 0. But examples of the present disclosure are not limited thereto, for example, each pixel may be, for example, 16 bits, and the gray scale value of each pixel of the binary image may be 65535 or 0.

In this case, with the binary image obtained by the binarization processing unit 340, each pixel may correspond to one stimulation electrode, and the gray value (i.e., the maximum gray value or the minimum gray value) of each pixel may be represented by high and low levels. This allows better response to the electrical stimulation signal of the stimulation electrode.

In some examples, the image processing apparatus 300 may turn down the number of pixels of the image and perform binarization processing, resulting in a low-pixel binary image. With the implant device 10 of the retinal stimulator 1, since it needs to be implanted into the eyeball, the size of the implant device 10 is severely limited, and the number of stimulation electrodes in the implant device 10 is also small. In this case, it is necessary that the number of pixels in the binary image is less than or equal to the number of stimulation electrodes within the implant device 10 of the retinal stimulator 1. Thereby, each pixel of the low-pixel binary image is enabled to correspond to a stimulation electrode within the implant device 10 of the retinal stimulator 1 to efficiently transfer the information of the pixel to the respective stimulation electrode. Therefore, the patient wearing the retinal stimulator 1 can perceive information (light sensation) from the binary image pixels by being stimulated by the stimulation electrodes, thereby helping the patient recognize and avoid an object or an obstacle from the processed image as much as possible.

Here, the functions of the respective units of the above-mentioned image processing apparatus 300 including the acquisition unit 310, the gradation processing unit 320, the pixel processing unit 330, and the binarization processing unit 340 can be realized by the image processing apparatus 300 of fig. 5 described below. This is explained in detail below.

Fig. 5 is a schematic configuration diagram of an image processing device of a retinal stimulator according to the present disclosure. In some examples, as shown in fig. 5, the image processing apparatus 300 may include a processor 410, a memory 420, and a communication interface 430.

In some examples, processor 410 may be used to control and manage actions performed by image processing apparatus 300. For example, the processor 410 may be used to implement the functions of the respective units of the image processing apparatus 300 of embodiment 1 described above. Additionally, processor 410 may also be used to support image processing device 300 in performing steps S100-S400 in fig. 6 and/or other processes for the techniques described herein.

In some examples, Processor 410 may be a Central Processing Unit (CPU), a general purpose Processor, a Digital Signal Processor (DSP), an Application-Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, transistor logic, hardware components, or any combination thereof. Which may implement or perform the various illustrative logical blocks, modules, and circuits described in connection with the disclosure. The processor 410 may also be a combination of computing functions, e.g., comprising one or more microprocessors, a combination of a DSP and a microprocessor, or the like.

In some examples, communication interface 430 may be used to support communication of image processing apparatus 300 with other devices (e.g., camera apparatus 20).

Additionally, in some examples, communication interface 430 may be a communication interface, a transceiver, a transceiving circuit, and/or the like. The communication interface is a generic term, and may include one or more interfaces.

In some examples, memory 420 may be used to store program codes and data for image processing apparatus 300.

Additionally, in some examples, the image processing apparatus 300 may further include a communication bus 440. The communication bus 440 may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus 440 may also be divided into an address bus, a data bus, a control bus, etc. There may be one or more communication buses 440. For ease of illustration, only one line is shown in FIG. 5, but this does not represent only one bus or one type of bus.

The above is the image processing apparatus 300 of the retinal stimulator 1 according to the present disclosure, and the image processing method of the retinal stimulator 1 according to the present disclosure is described in detail below with reference to a flowchart. The respective steps of the image processing method may correspond to the respective units of the image processing apparatus 300.

Fig. 6 is a flowchart illustrating an image processing method of the retinal stimulator according to the present disclosure. The image processing method of the retinal stimulator 1 can be applied to the image processing apparatus 300 in the retinal stimulator 1. The retinal stimulator 1 has a prescribed number of stimulating electrodes. The image processing method of the retinal stimulator 1 may be simply referred to as an image processing method.

In some examples, as shown in fig. 6, the image processing method may include an image acquisition step (step S100). Step S100 may be used to acquire an initial image.

In step S100, an initial image may be acquired by the image capture device 20. The camera 20 may be similar to the camera 20 of the retinal stimulator 1 described above. The camera 20 may capture the external environment in which the patient is located to obtain an initial image. The initial image may be a color image. A color image may be composed of a large number of pixels. For example, the number of pixels of a color image may be, for example, 30, 100, 200, 500, 1200, etc.

In some examples, in step S100, when the initial image is a color image of the HSI image model, each pixel may contain image information such as brightness, hue, and saturation. When the initial image is an RGB image model, the color information in the initial image can be determined by three components, red (R), green (G) and blue (B). If each component is represented by, for example, an 8-bit binary number, then 0-255 values per component can be selected. Examples of the present disclosure are not limited thereto, and each component may also be represented by, for example, a 16-bit binary number, and each component may also be represented by, for example, a 24-bit and above binary number.

In some examples, the information such as color features in the color image may not all be used to reflect morphological features of the object in the image, and it is generally difficult for the patient to completely transmit all the information of the initial image and often difficult to transmit information such as the outline of the object or obstacle in the initial image with the relatively small number of stimulation electrodes of the retinal stimulator 1. Based on this, the inventors found from experience of practical use that, by performing the gradation processing on the initial image, even when the number of electrodes is small and the ability to receive information is limited, the processing of the image can be optimized, and useful information of the image such as the outline of an object or an obstacle can be retained as much as possible.

In some examples, as shown in fig. 6, the image processing method may further include a graying step (step S200). Step S200 may perform graying processing on the initial image to obtain a grayscale image. The grayscale image may be a special color image of R, G, B with the same size of the three components (i.e., R-G-B values). When the initial image is a color image, the initial image is grayed, that is, the initial image is processed to make the values of the three components consistent. In this case, the graying process can reduce the color information in the initial image and retain the basic information such as morphological feature information of the initial image (particularly, an object or an obstacle in the image). Therefore, the calculation amount of subsequent image processing can be reduced, and more useful information of the image, such as morphological feature information of a middle object, an obstacle and the like of the image, can be provided subsequently.

In some examples, the graying processing method may be a component method, i.e., a value of any one of the three components may be selected R, G, B as the grayscale value. The graying processing method may also be a maximum value method, that is, the maximum value of R, G, B three components may be selected as the grayscale value. The graying processing method may be an average value method, that is, an average value of R, G, B three components may be selected as the grayscale value. The graying processing method may be a weighting method, that is, the graying value may be obtained by weighting R, G, B three components with different weighting coefficients.

In some examples, the grayscale image contains a relatively large number of pixels, e.g., 30 ten thousand pixels, whereas the number of stimulation electrodes of the implant device 10 in the retinal stimulator 1 is limited, e.g., 60 stimulation electrodes. Since the number of pixels of an image is much greater than the number of electrodes, most pixels do not have corresponding electrodes, and information contained in the pixels is lost during transmission, so that the picture is severely distorted. Therefore, the low pixelation process can be performed on the grayscale image.

In some examples, as shown in fig. 6, the image processing method may further include a low pixelation step (step S300). Step S300 may perform compression processing on the pixels of the grayscale image to obtain a low-pixel grayscale image. In some examples, the number of pixels of the low-pixel grayscale image is less than or equal to the prescribed number of stimulation electrodes.

In step S300, the number of pixels of the low pixel gradation image is reduced as compared with the gradation image, and accordingly, redundancy of image data due to correlation between adjacent pixels in the gradation image can be reduced. Therefore, the complexity of subsequent image processing is reduced, so that useful information of the image can be extracted in the subsequent steps.

Fig. 7 is a flow chart illustrating the low pixelation step of the image processing method of the retinal stimulator according to the present disclosure. Fig. 8A is a schematic diagram of an image processing process from a gray image to a low-pixel gray image based on fig. 7.

In some examples, step S300 may perform compression processing on the pixels of the grayscale image, that is, perform low pixelation processing on the grayscale image, so that the number of pixels of the image after the low pixelation processing is less than or equal to the specified number of stimulation electrodes. Thus, each pixel that can satisfy a low pixel grayscale image corresponds to one stimulation electrode within the implant device 10. Next, a grayscale image including "1" shown in fig. 8A, 10A, and 12A will be described in detail with reference to step S300.

In some examples, as shown in fig. 7, the low pixelation step (step S300) may include step S310, step S311, and step S312.

In step S310, the grayscale image may be subjected to a partition process to obtain a plurality of grayscale image regions. Wherein each gray scale image region may comprise a plurality of pixels. For example, a grayscale image including "1" shown in fig. 8A is subjected to a partition process on the grayscale image in fig. 8A, and a plurality of grayscale image regions are obtained.

In step S311, an average gradation value of pixels may be calculated for any one of the plurality of gradation image areas. Wherein the average gray value can be taken as the gray value of the gray image area. In other words, step S311 may select any one of the plurality of grayscale image regions as the target grayscale image region. An average gray value of a plurality of pixels of the target gray image area may be obtained for the target gray image area. Wherein the average gray value may be taken as the gray value of the target gray image area. For example, in the gradation image including "1" shown in fig. 8A, the average gradation value of each gradation image region is calculated.

In step S312, each gray image region of the gray image may be treated as a pixel having an average gray value to obtain a low pixel gray image. That is, step S312 may combine each gray scale image region of the gray scale image as one pixel of the low pixel gray scale image, and form the low pixel gray scale image by sequentially combining the respective pixels. For example, in the grayscale image shown in fig. 8A including "1", each grayscale image region in fig. 8A is regarded as one pixel, and the grayscale value of the pixel is the average grayscale value of the grayscale image regions.

In some examples, the low pixelation process may achieve the purpose of reducing pixels of a grayscale image by performing a partition process on the grayscale image and then treating one grayscale image area as one pixel. The number of pixels of the gray image may be, for example, 30 ten thousand, and the number of pixels of the gray image may be reduced from 30 ten thousand to 60 by dividing the gray image into, for example, 60 gray image regions. Examples of the present disclosure are not limited thereto, and the pixels of the gray image may be, for example, 100 ten thousand, 200 ten thousand, 500 ten thousand, 2000 ten thousand. The divided gray image areas may be less than 60, e.g. 55, 50, 30.

Fig. 9 is a flowchart schematically illustrating a modification 1 of the low pixelation step of the image processing method of the retinal stimulator according to the present disclosure. Fig. 10A is a schematic diagram of an image processing process from a gray image to a low-pixel gray image based on fig. 9.

In other examples, as shown in fig. 9, the low pixelation step (step S300) may include step S320, step S321, step S322, step S323, and step S324.

In step S320, gradient value calculation may be performed on the grayscale image along a preset direction. The preset direction may be any direction set by a person. For example, the grayscale image shown in fig. 10A including "1" is subjected to gradient value calculation along a preset direction for the grayscale image in fig. 10A.

In step S321, a pixel in which a gradient value of the gray image in a preset direction is greater than or equal to a preset gradient value may be determined. Pixels of the gray image having a gradient value in a preset direction greater than or equal to a preset gradient value may be taken as effective pixels. The preset gradient value may be set manually. For example, in the grayscale image shown in fig. 10A including "1", the gradient value obtained from the grayscale image in fig. 10A is compared with the preset gradient value to determine the effective pixel of the grayscale image in fig. 10A.

In step S322, the effective pixels in the grayscale image may be subjected to partition processing to obtain a plurality of pixel regions. Each pixel region may include a plurality of effective pixels. For example, in a grayscale image including "1" shown in fig. 10A, the effective pixels in fig. 10A are subjected to partition processing to obtain a plurality of pixel regions.

In step S323, an average gradation value of pixels may be calculated for any one pixel region among the plurality of pixel regions. Wherein, the average gray value is used as the gray value of the pixel area. In other words, step S323 may further include acquiring, for the target pixel region, an average grayscale value of a plurality of effective pixels of the target pixel region. The average gradation value may be taken as the gradation value of the target pixel region. The target pixel region may be any one of a plurality of pixel regions. For example, in a gradation image including "1" shown in fig. 10A, the average gradation value of each pixel region is calculated.

In step S324, each pixel region of the grayscale image may be treated as a pixel having an average grayscale value to obtain a low-pixel grayscale image. For example, in the gradation image including "1" shown in fig. 10A, each pixel region in fig. 10A is regarded as one pixel, and the gradation value of the pixel is the average gradation value of the respective pixel regions.

In some examples, the low pixelation process may achieve the purpose of reducing the number of pixels of the grayscale image by performing a partition process on the effective pixels and by regarding one pixel region as one pixel. The number of pixels of the grayscale image may be, for example, 100 ten thousand, and the grayscale image is divided into 60 grayscale image regions, so that the number of pixels capable of realizing the grayscale image is reduced from 100 ten thousand to 60. But examples of the present disclosure are not limited thereto, and the pixels of the gray image may be, for example, 500 ten thousand, 2000 ten thousand. The divided gray image areas may be less than 60, e.g. 55, 30.

Fig. 11 is a flowchart schematically illustrating a modification 2 of the low pixelation step of the image processing method of the retinal stimulator according to the present disclosure. Fig. 12A is a schematic diagram of an image processing process from a gray image to a low-pixel gray image based on fig. 11.

In other examples, as shown in fig. 11, the low pixelation step (step S300) may include step S330, step S331, and step S332.

In step S330, the grayscale image may be subjected to a partition process to obtain a plurality of grayscale image regions. Each gray scale image region may include a plurality of pixels. For example, a grayscale image including "1" shown in fig. 12A is subjected to a partition process on the grayscale image in fig. 12A, and a plurality of grayscale image regions are obtained.

In step S331, an average gradation value of pixels may be calculated for any one of a plurality of gradation image areas. The average gray value in the gray image may be compared to a preset average gray value to determine an effective gray image area in the gray image area. In other words, step S331 may acquire, for the target grayscale image region, an average grayscale value of a plurality of pixels of the target grayscale image region. The average gray value in the gray image may be compared to a preset average gray value to determine an effective gray image area in the gray image area. Wherein any one of the grayscale image regions is set as a target grayscale image region. For example, in the gray image including "1" shown in fig. 12A, the average gray value of each gray image region is calculated, and each average gray value is compared with the preset average gray value to determine the effective gray image region in fig. 12A.

In step S332, each gray image region of the effective gray image may be treated as a pixel having an average gray value to obtain a low-pixel gray image. That is, step S332 may combine each effective grayscale image region of the grayscale image as one pixel of the low-pixel grayscale image, and combine the pixels in order to form the low-pixel grayscale image. For example, in the gray scale image shown in fig. 12A including "1", each effective gray scale image region in fig. 12A is regarded as one pixel, and the gray scale value of the pixel is the average gray scale value of the respective effective gray scale image regions.

In some examples, a portion of the grayscale image region with the lower grayscale value may be removed, and the valid grayscale image region with the grayscale value greater than or equal to the preset average grayscale value may be processed. This can improve the image processing efficiency. However, examples of the present disclosure are not limited thereto, and for example, when the number of pixels of the grayscale image is reduced, a part of the grayscale image region with a higher grayscale value may be removed, and an effective grayscale image region with a grayscale value smaller than a preset average grayscale value may be processed.

In addition, in the above three examples, step S300, which is a different low-pixelation processing method, relates to a process of partitioning an image. The image partitioning is actually a process of reducing the number of pixels, and the reduction of the number of pixels of the grayscale image may be performed in a successive reduction manner or in a one-time reduction manner.

In some examples, for example, when the number of pixels of the grayscale image is, for example, 30 ten thousand, that is, for example, 640 × 480 pixels, if the number of pixels of the grayscale image needs to be adjusted from 30 ten thousand to, for example, 60, if the one-time adjustment is adopted, the number of pixels, for example, 640 × 480 pixels, can be directly adjusted to, for example, 10 × 6 pixels; if a successive dimming approach is used, 640 × 480 pixels may be first dimmed to, for example, 10 × 12 pixels, and then 10 × 12 pixels may be dimmed to, for example, 10 × 6 pixels. Examples of the present disclosure are not limited thereto, and the number of pixels of the grayscale image after the pixels are reduced may be less than or equal to 60, such as 55, 50, or 30, while the number of pixels of the grayscale image is, for example, 100 ten thousand, 200 ten thousand, 500 ten thousand, or 2000 ten thousand.

In addition, in some examples, the division of the regions may be equal or unequal.

In addition, in some examples, the low-pixel grayscale image obtained after the low-pixelation processing has a reduced number of pixels compared to the grayscale image before the low-pixelation processing, and the redundancy of image data caused by correlation between adjacent pixels in the grayscale image can be correspondingly reduced, whereby the complexity in the subsequent image processing can be reduced to extract useful information of the image in the subsequent step.

In addition, in some examples, as shown in fig. 6, the image processing method further includes a binarization step (step S400). Step S400 may perform binarization processing on the low-pixel grayscale image to obtain a binary image. Each pixel of the binary image can correspond to a stimulation electrode within the implant device 10 of the retinal stimulator 1. The stimulation electrode may generate an electrical stimulation signal from the binary image.

Fig. 13 is a schematic flowchart of the binarization step in the image processing method of the retinal stimulator according to the present disclosure. As shown in fig. 13, step S400 may include comparing the gray scale value of each pixel in the low pixel gray scale image with a preset gray scale value (step S410). The preset gray scale value can be set by human.

In some examples, as shown in fig. 13, step S400 may further include setting gray values in the low-pixel gray image to two categories, a maximum gray value and a minimum gray value, respectively, according to the result of the comparison. After the gray scale value is changed, a binary image is obtained (step S411). In some examples, each pixel may be, for example, 8-bit, then the maximum grayscale value is 255 and the minimum grayscale value is 0. But examples of the present disclosure are not limited thereto, and each pixel may be, for example, 16 bits, and then the maximum gray value is 65535 and the minimum gray value is 0.

In some examples, step S400 may change a gray value in a low pixel gray image, in which the gray value of the pixel is greater than or equal to a preset gray value, to a maximum gray value, and change a gray value in a low pixel gray image, in which the gray value of the pixel is less than the preset gray value, to a minimum gray value. However, examples of the present disclosure are not limited thereto, and for example, a gray value in a low pixel gray image in which a gray value of a pixel is less than or equal to a preset gray value may be changed to a maximum gray value, and a gray value in a low pixel gray image in which a gray value of a pixel is greater than a preset gray value may be changed to a minimum gray value.

In some examples, the number of pixels of the low-pixel binary image (which may also be referred to as a target grayscale image) is less than or equal to the prescribed number of stimulation electrodes within the implant device 10. That is, the number of pixels of the binary image of low pixels matches the number of stimulation electrodes of the implant device 10 of the retinal stimulator 1. In other words, each pixel of the binary image of the low pixel may correspond to one electrode. In this case, when each pixel may be, for example, 8-bit, two kinds of gray values can be represented by high and low levels. For example, the electrical stimulation signal may be at a low level and may correspond to a pixel having a gray scale value of 0, and the electrical stimulation signal may be at a high level and may correspond to a pixel having a gray scale value of 255. Examples of the present disclosure are not limited thereto, and for example, the electrical stimulation signal may be at a high level and may correspond to a pixel having a gray scale value of 0, and the electrical stimulation signal may be at a low level and may correspond to a pixel having a gray scale value of 255.

Fig. 8B is a schematic diagram of the image processing from the grayscale image to the binary image based on fig. 8A. Fig. 10B is a schematic diagram of an image processing process from a grayscale image to a binary image based on fig. 10A. Fig. 12B is a schematic diagram of the image processing from the grayscale image to the binary image based on fig. 12A.

In some examples, based on the low-pixel grayscale images obtained by the different low-pixelation processing methods in the above three examples, as shown in fig. 8B, 10B, or 12B, the low-pixel grayscale images may be respectively subjected to binarization processing to obtain binary images.

Fig. 14 is a schematic diagram of the image processing effect of the retinal stimulator according to the present disclosure. As shown in fig. 14, the grayscale image may be subjected to a pixel reduction process to obtain a low-pixel grayscale image. The low-pixel gray level image can be subjected to binarization processing, and a binary image can be obtained. As can be seen from fig. 14, after the number "1" in fig. 14 is subjected to the pixel reduction and binarization processing, the obtained target grayscale image can still be well recognized.

In some examples, the graying processing, the low pixelation processing, and the binarization processing are performed on the initial image to obtain a binary image. The number of pixels of the binary image is less than or equal to the prescribed number of stimulation electrodes within the implant device 10. Thereby, each pixel of the binary image is enabled to correspond to a stimulation electrode within the implant device 10 of the retinal stimulator 1 to efficiently transfer the information of the pixel to the respective stimulation electrode. Therefore, the patient wearing the retinal stimulator 1 can perceive information (light sensation) from the binary image pixels by being stimulated by the stimulation electrodes, thereby helping the patient recognize and avoid an object or an obstacle from the processed image as much as possible.

While the present disclosure has been described in detail in connection with the drawings and examples, it should be understood that the above description is not intended to limit the disclosure in any way. Those skilled in the art can make modifications and variations to the present disclosure as needed without departing from the true spirit and scope of the disclosure, which fall within the scope of the disclosure.

Claims

1. An image processing method of a retinal stimulator having a prescribed number of stimulation electrodes, characterized by:

the method comprises the following steps:

an image acquisition step, which is used for acquiring an initial image, wherein the initial image is a video image of the environment where the patient is located;

a graying step, which is used for performing graying processing on the initial image to obtain a grayscale image, wherein the graying processing comprises the step of respectively performing weighted calculation on R, G, B three components of each pixel according to different weighting coefficients and taking the components as the grayscale value of the pixel;

a low pixelation step of compressing pixels of the grayscale image to obtain a low-pixel grayscale image, the number of pixels of the low-pixel grayscale image being less than or equal to the predetermined number of stimulation electrodes, the low pixelation step including: calculating gradient values of the gray level images along a preset direction; determining pixels of which the gradient values in the preset direction of the gray image are greater than or equal to a preset gradient value, and taking the pixels of which the gradient values in the preset direction of the gray image are greater than or equal to the preset gradient value as effective pixels; partitioning effective pixels in the gray level image to obtain a plurality of pixel areas, wherein each pixel area comprises a plurality of effective pixels; calculating an average gray value of pixels for any one of the plurality of pixel regions, and taking the average gray value as a gray value of the pixel region; taking each pixel region of the gray image as a pixel with an average gray value to obtain the low-pixel gray image; and is

A binarization step for performing binarization processing on the low-pixel grayscale image to obtain a binary image with a minimum grayscale value and a maximum grayscale value, wherein the maximum grayscale value corresponds to a high level, the minimum grayscale value corresponds to a low level,

and the stimulation electrode generates an electrical stimulation signal which is delivered to the ganglion cells or bipolar cells of the retina according to the binary image, wherein the electrical stimulation signal is at a high level or a low level.

2. The image processing method according to claim 1,

the binarization step comprises the following steps:

comparing the gray value of each pixel in the low-pixel gray image with a preset gray value;

according to the comparison result, the gray values in the low-pixel gray image can be set into two types, namely a maximum gray value and a minimum gray value, and the binary image can be obtained after the gray values are changed.

3. An image processing apparatus of a retinal stimulator having a prescribed number of stimulation electrodes,

the method comprises the following steps:

an acquisition unit for acquiring an initial image, the initial image being a video image of an environment in which a patient is located;

the gray processing unit is used for carrying out gray processing on the initial image to obtain a gray image, and the gray processing comprises the steps of respectively carrying out weighting calculation on R, G, B three components of each pixel according to different weighting coefficients and taking the components as the gray value of the pixel;

a pixel processing unit configured to perform compression processing on pixels of the grayscale image to obtain a low-pixel grayscale image, wherein the number of pixels of the low-pixel grayscale image is less than or equal to the predetermined number of the stimulation electrodes, and the pixel processing unit includes: a calculation subunit, configured to perform gradient value calculation on the grayscale image along a preset direction; a determining subunit, configured to determine pixels of the grayscale image having a gradient value in the preset direction greater than or equal to a preset gradient value, and to take the pixels of the grayscale image having a gradient value in the preset direction greater than or equal to the preset gradient value as effective pixels; the second partitioning subunit is used for partitioning effective pixels in the grayscale image to obtain a plurality of pixel areas, and each pixel area comprises a plurality of effective pixels; a second acquisition subunit, configured to calculate an average grayscale value of pixels for any one of the plurality of pixel regions, and use the average grayscale value as a grayscale value of the pixel region; a second pixel processing subunit, configured to treat each pixel region of the grayscale image as a pixel having an average grayscale value to obtain the low-pixel grayscale image; and

a binarization processing unit for performing binarization processing on the low-pixel grayscale image to obtain a binary image with a minimum grayscale value and a maximum grayscale value, wherein the maximum grayscale value corresponds to a high level and the minimum grayscale value corresponds to a low level,

4. The image processing apparatus according to claim 3,

the binarization processing unit includes:

a comparison subunit, configured to compare a grayscale value of each pixel in the low-pixel grayscale image with a preset grayscale value;

and the processing subunit is used for setting the gray values in the low-pixel gray image into two types according to the comparison result, namely a maximum gray value and a minimum gray value, and obtaining the binary image after changing the gray values.

5. A retinal stimulator, characterized in that,

including camera device, video processing apparatus and implantation device, wherein:

the camera device is used for capturing a video image and converting the video image into a visual signal;

the video processing device at least comprises the image processing device of any one of claims 3 to 4, the video processing device is connected with the camera device, and the video processing device is used for processing the visual signals and sending the visual signals to the implantation device through a transmitting antenna; and

the implant device is used for converting the received visual signals into bidirectional pulse current signals serving as electric stimulation signals, so that the bidirectional pulse current signals are distributed to ganglion cells or bipolar cells of retina to generate light sensation.