CN112860060B - Image recognition method, device and storage medium - Google Patents
Image recognition method, device and storage medium Download PDFInfo
- Publication number
- CN112860060B CN112860060B CN202110024773.3A CN202110024773A CN112860060B CN 112860060 B CN112860060 B CN 112860060B CN 202110024773 A CN202110024773 A CN 202110024773A CN 112860060 B CN112860060 B CN 112860060B
- Authority
- CN
- China
- Prior art keywords
- image
- gazing
- range
- recognition method
- virtual image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/255—Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
Abstract
The invention discloses an image recognition method, equipment and a storage medium, wherein the image recognition method is applied to an eyeball tracking system and comprises the following steps of S1: acquiring image data, and establishing a virtual image model according to the image data; step S2: the method comprises the steps of collecting eye movement data at least comprising a gazing coordinate and gazing duration, generating a reference identification frame on the surface of a virtual image model by taking the gazing coordinate as an original point when the gazing duration on any gazing coordinate is longer than a preset duration, and amplifying the reference identification frame along the surface of the virtual image model according to the gazing duration to obtain a final identification range; step S3: and identifying the image in the final identification range to generate and display the characteristic information of the image in the final identification range. The invention can improve the interactivity of the eyeball tracking technology, improve the use experience of the user and improve the interestingness at the same time.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to an image recognition method and apparatus based on eye tracking, and a storage medium.
Background
At present, the conventional image recognition method generally introduces image data into software, and then the software performs image recognition on an image frame to recognize all objects in the image frame and feature information such as motion or expression of the objects, and if the local position in the image frame is to be recognized, it is necessary to frame the local position of the image frame by using a mouse cursor and then perform image recognition on the framed local area to acquire the feature information in the image.
The eyeball tracking technology is generally used for reflecting the position of an attention point of a user when the user sees a certain object through eyeball tracking, but the process is unidirectional, the computer cannot be controlled to perform corresponding operation according to eyeball motion, the user cannot read characteristic information in an image when the user pays attention to objects in the image, and the use experience of the user is poor.
Disclosure of Invention
In order to overcome the defects of the prior art, an object of the present invention is to provide an image recognition method, which can improve the interactivity of the eye tracking technology, improve the user experience, and improve the interestingness.
Another object of the present invention is to provide an electronic device.
It is a further object of the present invention to provide a storage medium.
One of the purposes of the invention is realized by adopting the following technical scheme:
an image recognition method applied in an eyeball tracking system comprises the following steps:
step S1: acquiring image data, and establishing a virtual image model according to the image data;
step S2: the method comprises the steps of collecting eye movement data at least comprising a gazing coordinate and gazing duration, generating a reference identification frame on the surface of a virtual image model by taking the gazing coordinate as an original point when the gazing duration on any gazing coordinate is longer than a preset duration, and amplifying the reference identification frame along the surface of the virtual image model according to the gazing duration to obtain a final identification range;
step S3: and identifying the image in the final identification range to generate and display the characteristic information of the image in the final identification range.
Further, the step S3 further includes: and amplifying the image in the final identification range, and identifying the amplified image.
Further, after the image amplification is performed on the image within the final recognition range, the method further includes: and carrying out pixel restoration on the amplified image, and then identifying the amplified image after pixel restoration.
Further, the image data includes two-dimensional image data and three-dimensional image data, a two-dimensional virtual image model is established according to the two-dimensional image data, and a three-dimensional virtual image model is established according to the three-dimensional image data.
Further, the eye movement data comprise an eye movement path, a movement identification line matched with the eye movement path is created in the virtual image model according to the eye movement path, and the movement identification line has a preset transparency and a preset standard width.
Further, before the step S2 of enlarging the reference frame, the method further includes: setting the ratio of the watching duration to the magnification of the reference identification frame in advance; and when the reference identification frame is amplified, the watching time length is converted into an amplification factor according to a preset proportion, so that the reference identification frame is amplified according to the amplification factor.
Further, the method for recognizing the image in the final recognition range in step S3 is as follows:
and carrying out outline recognition on all objects in the final recognition range in the virtual image model, carrying out outline comparison on the outline of the objects and the standard model of each object in the preset database so as to recognize the types of all the objects in the final recognition range, and generating corresponding characteristic information according to the types of the objects in the final recognition range.
Further, still include: and judging whether a modification instruction is received or not in real time, generating a coverage range in the final identification range according to the modification instruction, and updating the area outside the coverage range into a new final identification range.
The second purpose of the invention is realized by adopting the following technical scheme:
an electronic device comprising a processor, a memory and a computer program stored on the memory and executable on the processor, the processor implementing the image recognition method as described above when executing the computer program.
The third purpose of the invention is realized by adopting the following technical scheme:
a storage medium having stored thereon a computer program which, when executed, implements the image recognition method described above.
Compared with the prior art, the invention has the beneficial effects that:
the method comprises the steps of marking out a reference identification frame in a virtual image model through an eyeball tracking technology, amplifying the reference identification frame according to the watching duration in proportion, and accordingly identifying the image features in the reference identification frame to generate feature information corresponding to the image in the reference identification frame, improving interactivity and interestingness of a user using the technology, enabling the user to read more feature information in the image watching process, even knowing details which are easily ignored by naked eyes in the image, and improving the use experience of the user.
Drawings
FIG. 1 is a flowchart illustrating an image recognition method according to the present invention.
Detailed Description
The present invention will be further described with reference to the accompanying drawings and the detailed description, and it should be noted that any combination of the embodiments or technical features described below can be used to form a new embodiment without conflict.
Example one
The embodiment provides an image recognition method, by which more detailed features that cannot be noticed by naked eyes in an image can be read, the interaction degree of an eyeball tracking technology can be improved, and the use experience and interestingness of a user are improved.
As shown in fig. 1, the image recognition method of the present embodiment specifically includes the following steps:
step S1: and acquiring image data, and establishing a virtual image model according to the image data.
The image data comprises two-dimensional image data and three-dimensional image data, a two-dimensional virtual image model is established according to the two-dimensional image data, and a three-dimensional virtual image model is established according to the three-dimensional image data. The image data may be image data of a single object or a combined image in which a plurality of objects are combined together; the image data is imported into the appointed software, and the appointed software can generate the corresponding virtual image model, so that the user can check various detailed characteristics of the virtual image model through an eyeball tracking technology.
Step S2: the method comprises the steps of collecting eye movement data at least comprising a gazing coordinate and gazing duration, generating a reference identification frame on the surface of a virtual image model by taking the gazing coordinate as an original point when the gazing duration on any gazing coordinate is larger than a preset duration, and amplifying the reference identification frame along the surface of the virtual image model according to the gazing duration to obtain a final identification range.
The eye movement data are generated by an eye movement instrument, and the user can wear the eye movement instrument and check the virtual image model to generate the corresponding eye movement data. When the virtual image model is a three-dimensional model, the user can view the three-dimensional virtual image model by using the eye tracker combined with the vr technology, so as to perform feature recognition on any position of the three-dimensional virtual image model.
The eye movement data comprises an eye movement path, a watching coordinate and a watching duration, wherein the eye movement path is formed by connecting a plurality of watching coordinates and is a movement path of eyeballs of the user. The watching duration is the time of the eyeballs of the user staying on any watching coordinate. In addition, in this embodiment, a moving identification line matched with the eye movement path may be created in the virtual image model according to the eye movement path, and the moving identification line has a preset transparency and a preset standard width; the user can clearly know whether the moving route of the eyeballs of the user is consistent with the trend of the moving identification line in the virtual image model or not in the virtual image model. When the user repeatedly watches the same coordinate point, the transparency of the position of the coordinate point on the mobile identification line is increased, so that the position of a key area watched by the user can be known at any time.
When the watching duration of a user on any watching coordinate is longer than a preset duration, a reference identification frame is generated on the surface of the virtual image model by taking the watching coordinate as an original point, the shape of the reference identification frame can be preset to be circular or square, and the initial area of the reference identification frame can be defaulted to be a preset area. The watching duration and the magnification of the reference identification frame are in a direct proportional relation, and the ratio of the watching duration and the magnification of the reference identification frame can be set in advance; converting the watching duration into the magnification factor of the reference identification frame according to a preset proportion, and amplifying the reference identification frame according to the magnification factor; and the reference identification frame is amplified along the surface of the virtual image model in the amplification process, so that the reference identification frame is always attached to the outer surface of the virtual image model, and the reference identification frame amplified according to the watching duration is the final identification range.
Step S3: and identifying the image in the final identification range to generate and display the characteristic information of the image in the final identification range.
After the final recognition range is determined, amplifying the image of the local area of the virtual image model in the final recognition range, and displaying the amplified image; if the amplified image is fuzzy, the amplified image can be subjected to pixel reduction, so that a user can see a local image of the amplified clear virtual image model; the pixel restoration process adjusts the pixels of the amplified image into the original pixels of the virtual image module, and the original pixels are determined by the image data, so that the definition of the amplified image after the pixel restoration is still the definition of the original image in the image data, and the object in the image is prevented from being sent and deformed in the pixel restoration process.
The amplified and pixel-restored image can be viewed by a user and can also be used as a basis for image recognition; the embodiment is provided with a preset database, wherein a large number of standard models of different objects are stored in the preset database in advance, and meanwhile, a large number of data such as object types, object names, object outline images, object purposes and the like of the standard models are also stored. After a final recognition range is determined in the virtual image model, performing outline recognition on all objects in the final recognition range in the virtual image model through an image recognition technology, performing outline comparison on the outline of each object and a standard model of each object in a preset database, determining the type of the object in the final recognition range when the outline of the object in the final recognition range is matched with any standard model, and displaying relevant data of the standard model as feature information. In addition, when the object in the final recognition range is judged to be a human, the human in the final recognition range can be subjected to face recognition according to a face recognition technology, and corresponding face characteristic information is generated and displayed; meanwhile, the standard model can be bound with prestored information which can be various information related or unrelated to the object, and when the object in the final identification range is judged to be matched with any standard model, the prestored information corresponding to the standard model is displayed. In this embodiment, the display mode of the feature information may be displayed in the software in a pop-up window mode, so that the user can view the feature information of the final recognition range while viewing the virtual image model, thereby improving the user experience.
In addition, after the final recognition range is determined, it is further required to determine whether a modification instruction is received in real time, where the modification instruction may be generated by an external device such as a mouse, and the embodiment generates a coverage range in the final recognition range according to the modification instruction, and updates an area outside the coverage range to a new final recognition range, so that a user may modify the size of the final recognition range to improve the accuracy of area selection.
Example two
The embodiment provides an electronic device, which comprises a processor, a memory and a computer program stored on the memory and capable of running on the processor, wherein the processor implements the image recognition method in the first embodiment when executing the computer program; in addition, the present embodiment also provides a storage medium on which a computer program is stored, the computer program implementing the image recognition method described above when executed.
The apparatus and the storage medium in this embodiment are based on two aspects of the same inventive concept, and the method implementation process has been described in detail in the foregoing, so that those skilled in the art can clearly understand the structure and implementation process of the system in this embodiment according to the foregoing description, and for the sake of brevity of the description, details are not repeated here.
The above embodiments are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited thereby, and any insubstantial changes and substitutions made by those skilled in the art based on the present invention are within the protection scope of the present invention.
Claims (9)
1. An image recognition method applied to an eye tracking system includes:
step S1: acquiring image data, and establishing a virtual image model according to the image data;
step S2: the method comprises the steps of collecting eye movement data at least comprising gazing coordinates and gazing duration, generating a reference identification frame on the surface of a virtual image model by taking the gazing coordinates as an original point when the gazing duration on any gazing coordinate is longer than a preset duration, amplifying the reference identification frame along the surface of the virtual image model according to the gazing duration, wherein the reference identification frame is amplified along the surface of the virtual image model in the amplifying process, so that the reference identification frame is always attached to the outer surface of the virtual image model, and the reference identification frame amplified according to the gazing duration is a final identification range;
step S3: and carrying out outline recognition on all objects in the final recognition range in the virtual image model, carrying out outline comparison on the outline of the objects and the standard model of each object in the preset database so as to recognize the types of all the objects in the final recognition range, generating corresponding characteristic information according to the types of the objects in the final recognition range, and displaying the characteristic information.
2. The image recognition method according to claim 1, wherein the step S3 further includes: and amplifying the image in the final identification range, and identifying the amplified image.
3. The image recognition method according to claim 2, wherein after the image enlargement of the image in the final recognition range, the method further comprises: and performing pixel restoration on the amplified image, and then identifying the amplified image after pixel restoration.
4. The image recognition method according to claim 1, wherein the image data includes two-dimensional image data and three-dimensional image data, a two-dimensional virtual image model is created from the two-dimensional image data, and a three-dimensional virtual image model is created from the three-dimensional image data.
5. The image recognition method according to claim 1, wherein the eye movement data includes an eye movement path, a movement identification line matching the eye movement path is created in the virtual image model according to the eye movement path, and the movement identification line has a preset transparency and a preset standard width.
6. The image recognition method according to claim 1, wherein before the enlarging the reference frame in step S2, the method further comprises: setting the ratio of the watching duration to the magnification of the reference identification frame in advance; and when the reference identification frame is amplified, the watching time length is converted into an amplification factor according to a preset proportion, so that the reference identification frame is amplified according to the amplification factor.
7. The image recognition method according to claim 1, further comprising:
and judging whether a modification instruction is received in real time, generating a coverage range in the final recognition range according to the modification instruction, and updating the area outside the coverage range into a new final recognition range.
8. An electronic device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the image recognition method according to any one of claims 1 to 7.
9. A storage medium having stored thereon a computer program which, when executed, implements the image recognition method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110024773.3A CN112860060B (en) | 2021-01-08 | 2021-01-08 | Image recognition method, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110024773.3A CN112860060B (en) | 2021-01-08 | 2021-01-08 | Image recognition method, device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112860060A CN112860060A (en) | 2021-05-28 |
| CN112860060B true CN112860060B (en) | 2022-07-01 |
Family
ID=76005612
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110024773.3A Active CN112860060B (en) | 2021-01-08 | 2021-01-08 | Image recognition method, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112860060B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117788742A (en) * | 2022-09-20 | 2024-03-29 | 顺丰科技有限公司 | Irregular part identification method, irregular part identification device, computer equipment and storage medium |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109271982A (en) * | 2018-09-20 | 2019-01-25 | 西安艾润物联网技术服务有限责任公司 | Multiple identification region recognition methods, identification terminal and readable storage medium storing program for executing |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1312635C (en) * | 2004-03-17 | 2007-04-25 | 致伸科技股份有限公司 | Image zooming method |
| US11747895B2 (en) * | 2013-03-15 | 2023-09-05 | Intuitive Surgical Operations, Inc. | Robotic system providing user selectable actions associated with gaze tracking |
| CN107797664B (en) * | 2017-10-27 | 2021-05-07 | Oppo广东移动通信有限公司 | Content display method and device and electronic device |
| CN109086726B (en) * | 2018-08-10 | 2020-01-14 | 陈涛 | Local image identification method and system based on AR intelligent glasses |
| CN112034977B (en) * | 2019-06-04 | 2022-07-19 | 陈涛 | Method for MR intelligent glasses content interaction, information input and recommendation technology application |
-
2021
- 2021-01-08 CN CN202110024773.3A patent/CN112860060B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109271982A (en) * | 2018-09-20 | 2019-01-25 | 西安艾润物联网技术服务有限责任公司 | Multiple identification region recognition methods, identification terminal and readable storage medium storing program for executing |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112860060A (en) | 2021-05-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20190139297A1 (en) | 3d skeletonization using truncated epipolar lines | |
| CN108876934B (en) | Key point marking method, device and system and storage medium | |
| RU2524836C2 (en) | Information processor, processing method and programme | |
| CN110716645A (en) | Augmented reality data presentation method and device, electronic equipment and storage medium | |
| US10186084B2 (en) | Image processing to enhance variety of displayable augmented reality objects | |
| CN106127552B (en) | Virtual scene display method, device and system | |
| CN108734078B (en) | Image processing method, image processing apparatus, electronic device, storage medium, and program | |
| CN109547753A (en) | The method and system of at least one image captured by the scene camera of vehicle is provided | |
| CN104574267A (en) | Guiding method and information processing apparatus | |
| KR20150105479A (en) | Realization method and device for two-dimensional code augmented reality | |
| US11159717B2 (en) | Systems and methods for real time screen display coordinate and shape detection | |
| CN112506340A (en) | Device control method, device, electronic device and storage medium | |
| CN112860059A (en) | Image identification method and device based on eyeball tracking and storage medium | |
| CN108597034B (en) | Method and apparatus for generating information | |
| CN111598996B (en) | Article 3D model display method and system based on AR technology | |
| CN113160231A (en) | Sample generation method, sample generation device and electronic equipment | |
| CN111429338A (en) | Method, apparatus, device and computer-readable storage medium for processing video | |
| CN108430032B (en) | Method and equipment for realizing position sharing of VR/AR equipment | |
| CN112860060B (en) | Image recognition method, device and storage medium | |
| CN112181141A (en) | AR positioning method, AR positioning device, electronic equipment and storage medium | |
| CN115008454A (en) | Robot online hand-eye calibration method based on multi-frame pseudo label data enhancement | |
| CN108664948B (en) | Method and apparatus for generating information | |
| CN111767456A (en) | Method and device for pushing information | |
| CN116452745A (en) | Hand modeling, hand model processing method, device and medium | |
| CN115840550A (en) | Angle-adaptive display screen display method, device and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: Room 238, room 406, 1 Yichuang street, Huangpu District, Guangzhou, Guangdong 510700 Applicant after: Guangzhou langguo Electronic Technology Co.,Ltd. Address before: Room 238, room 406, 1 Yichuang street, Huangpu District, Guangzhou, Guangdong 510700 Applicant before: GUANGZHOU LANGO ELECTRONIC SCIENCE & TECHNOLOGY Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |