CN112291445A - Image processing method, device, equipment and storage medium - Google Patents
Image processing method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN112291445A CN112291445A CN202011174740.9A CN202011174740A CN112291445A CN 112291445 A CN112291445 A CN 112291445A CN 202011174740 A CN202011174740 A CN 202011174740A CN 112291445 A CN112291445 A CN 112291445A
- Authority
- CN
- China
- Prior art keywords
- color image
- image information
- information
- depth information
- target object
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/00002—Diagnosis, testing or measuring; Detecting, analysing or monitoring not otherwise provided for
- H04N1/00007—Diagnosis, testing or measuring; Detecting, analysing or monitoring not otherwise provided for relating to particular apparatus or devices
- H04N1/00023—Colour systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/00002—Diagnosis, testing or measuring; Detecting, analysing or monitoring not otherwise provided for
- H04N1/00071—Diagnosis, testing or measuring; Detecting, analysing or monitoring not otherwise provided for characterised by the action taken
- H04N1/00082—Adjusting or controlling
- H04N1/00087—Setting or calibrating
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/00795—Reading arrangements
- H04N1/00798—Circuits or arrangements for the control thereof, e.g. using a programmed control device or according to a measured quantity
- H04N1/00819—Self-calibrating reading means
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/04—Scanning arrangements, i.e. arrangements for the displacement of active reading or reproducing elements relative to the original or reproducing medium, or vice versa
Abstract
The present disclosure relates to the field of computer technologies, and in particular, to a method, an apparatus, a device, and a storage medium for image processing. The image processing method provided by the present disclosure includes: acquiring color image information through a color image acquisition unit; determining depth information corresponding to the color image information according to the shooting height and the shooting angle of the color image acquisition unit; acquiring target key characteristics of a target object based on the depth information and the color image information; a perspective-based conversion is performed on the color image information according to depth information and target key features to produce corrected image information. According to the image processing method provided by the disclosure, the identification accuracy is improved and the conversion distortion degree is reduced by carrying out the identification of the key features of the target and the conversion of the visual angle based on the depth information, and the color image acquisition unit is not required to shoot the target object and is not required to be repeatedly adjusted when in use.
Description
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a method, an apparatus, a device, and a storage medium for image processing.
Background
With the development of network communication technology, online education is becoming more and more popular. In the scene of online education, a teacher can remotely acquire the conditions of class attendance and writing of students through network video and perform real-time tutoring. In this regard, some related arts provide a high-speed photographing apparatus which needs to be disposed right above a desktop and a lens facing the desktop for photographing a book right below the same. When the user uses the high-speed shooting instrument, the position of the high-speed shooting instrument or the book has to be adjusted repeatedly, so that the high-speed shooting instrument is opposite to the book, and the high-speed shooting instrument occupies the central position of a desktop, which is inconvenient for the user to use.
Disclosure of Invention
This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
According to one or more embodiments of the present disclosure, there is provided a method of image processing, applied to an image capturing apparatus including a color image acquisition unit, the method including:
acquiring color image information through the color image acquisition unit;
determining depth information corresponding to the color image information according to the shooting height and the shooting angle of the color image acquisition unit;
acquiring target key features of a target object based on the depth information and the color image information;
performing perspective-based conversion on the color image information according to the depth information and the target key features to generate corrected image information.
According to one or more embodiments of the present disclosure, there is provided an apparatus for image processing, including:
the color image acquisition unit is used for acquiring color image information through the color image acquisition unit;
the color image acquisition unit is used for acquiring color image information through the color image acquisition unit;
the depth information determining unit is used for determining depth information corresponding to the color image information according to the shooting height and the shooting angle of the color image acquisition unit;
a feature acquisition unit configured to acquire a target key feature of a target object based on the depth information and the color image information;
a perspective conversion unit for performing perspective-based conversion on the color image information according to the depth information and the target key feature to generate corrected image information.
According to one or more embodiments of the present disclosure, there is provided an image capturing apparatus including:
at least one memory, at least one processor, and a color image acquisition unit;
wherein the memory is configured to store program code, and the processor is configured to call the program code stored in the memory to perform a method of image processing provided according to one or more embodiments of the present disclosure.
According to one or more embodiments of the present disclosure, there is provided a non-transitory computer storage medium storing program code executable by a computer device to cause the computer device to perform a method of image processing provided according to one or more embodiments of the present disclosure.
According to the image processing method provided by the embodiment of the disclosure, the target key feature of the target object is acquired based on the depth information obtained according to the shooting height and the shooting angle of the color image acquisition unit, and the view angle conversion is performed on the color image information, so that the corrected image information can be obtained, and the viewing angle of the target object in the corrected image information is changed, so that the color image acquisition unit is not required to be shot right at the target object, and the color image acquisition unit is not required to be adjusted repeatedly during use, thereby facilitating the use of a user, and moreover, the identification accuracy is improved and the conversion distortion is reduced by performing the target key feature identification and the view angle conversion based on the depth information. In addition, the image processing method provided by the embodiment of the disclosure does not need an additional depth sensor, reduces the hardware cost, is integrated in the image capturing device, can enable the image capturing device to be matched with any video system on the market, and does not need to additionally install specific software or network.
Drawings
The above and other features, advantages and aspects of various embodiments of the present disclosure will become more apparent by referring to the following detailed description when taken in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numbers refer to the same or similar elements. It should be understood that the drawings are schematic and that elements and features are not necessarily drawn to scale.
FIG. 1 is a flow diagram of a method of image processing provided in accordance with an embodiment of the present disclosure;
fig. 2 is a schematic view of a usage scenario of a method of image processing provided according to an embodiment of the present disclosure;
FIG. 3 is a flow diagram of a method of image processing provided in accordance with another embodiment of the present disclosure;
fig. 4 is a schematic structural diagram of an apparatus for image processing according to an embodiment of the present disclosure;
fig. 5 is a schematic structural diagram of an image capturing apparatus for implementing an embodiment of the present disclosure.
Detailed Description
Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is to be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but rather are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the disclosure are for illustration purposes only and are not intended to limit the scope of the disclosure.
It should be understood that the steps recited in the apparatus embodiments of the present disclosure may be performed in a different order and/or performed in parallel. Moreover, device embodiments may include additional steps and/or omit performing the illustrated steps. The scope of the present disclosure is not limited in this respect.
The term "include" and variations thereof as used herein are open-ended, i.e., "including but not limited to". The term "based on" is "based, at least in part, on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions for other terms will be given in the following description.
It should be noted that the terms "first", "second", and the like in the present disclosure are only used for distinguishing different devices, modules or units, and are not used for limiting the order or interdependence relationship of the functions performed by the devices, modules or units.
It is noted that references to "a", "an", and "the" modifications in this disclosure are intended to be illustrative rather than limiting, and that those skilled in the art will recognize that "one or more" may be used unless the context clearly dictates otherwise.
For the purposes of this disclosure, the phrase "a and/or B" means (a), (B), or (a and B).
The names of messages or information exchanged between devices in the embodiments of the present disclosure are for illustrative purposes only, and are not intended to limit the scope of the messages or information.
Referring to fig. 1, fig. 1 shows a flowchart of a method 100 of image processing provided by an embodiment of the present disclosure, where the method 100 is applied to an image capturing device including a color image acquisition unit. The method 100 comprises steps S101-S104:
step S101: and acquiring color image information through the color image acquisition unit.
The color image information includes original color image information captured by the color image capturing unit, and image information generated after encoding and preprocessing are performed according to the original color image information, such as MPEG encoding information, 1080P video stream, and the like.
Illustratively, the color image capture unit includes, but is not limited to, an RGB camera.
Step S102: and determining depth information corresponding to the color image information according to the shooting height and the shooting angle of the color image acquisition unit.
Where the depth information represents the distance of objects in the scene captured by the sensor from the sensor, the depth information for a color image may consist of a depth value for each pixel, which may be expressed in units of centimeters, millimeters, and so on.
The image capturing apparatus provided according to the present embodiment is used for photographing a target object on a desktop while being placed on the desktop. As shown in fig. 2, the present embodiment assumes that the image capturing apparatus 10 is used by the user 20 and the target object 30 is located on the table 30, and the photographing height and photographing angle of the image capturing apparatus 10 are configured to mainly photograph the target object 20 on the table 40. On the basis of the above-described assumption, the depth information of the color image information captured by the image capturing apparatus 10 can be calculated by the photographing height and the photographing angle thereof.
As a specific embodiment, the image capturing apparatus has a support for supporting the color image pickup unit, the support being configured to be able to adjust a photographing height and a photographing angle of the color image pickup unit, and a sensor configured to detect the photographing height and the photographing angle of the support. Further, the correspondence of the camera height and camera angle to the depth information may be stored in advance in the image capturing apparatus.
As another specific embodiment, the height and camera angle of the image capturing apparatus are not adjustable, and the corresponding depth information is fixed, which can be obtained in advance.
Step S103: and acquiring target key characteristics of the target object based on the depth information and the color image information.
The target object comprises paper, books, handwriting boards, tablet computers and other paper or electronic and mechanical products which can be used for writing or reading. In some embodiments, the target key feature of the target object comprises a vertex or an edge of the target object, such as a corner or an edge of a sheet of paper. The target key feature of the target object may be obtained through image recognition technology related in the art, which is not limited in this embodiment. In some embodiments, the target key features of the target object may be identified from the color image information by a trained classifier.
When the target key features of the target object are acquired, the object near the target object or other features of the target object may interfere with the identification process of the target key features. Taking an example that the target object is a book and the target key feature is a corner of a top page of the book, features such as a white table corner, a table edge, a corner of a bottom page of the book and the like may be mistakenly recognized as the target key features, which causes an error in subsequently generated corrected image information.
In this regard, the present embodiment can more accurately acquire the target key feature of the target object and reduce the degree of distortion of the corrected image by based on the depth information. On one hand, because the target object usually has a certain thickness, taking the book as an example, a certain height difference exists between the top-layer page and the desktop and the bottom-layer page of the book on the desktop, so that in the color image information, the depth values of the top-layer page and the desktop or the bottom-layer page of the target object are also different, and therefore the written text can be distinguished from the desktop, the top-layer page and the bottom-layer page through the depth information, and the interference information is eliminated; on the other hand, whether the key features are located on the plane and the relation between the planes where the key features are located can be determined through the depth information, so that the obtained target key features can reflect the real shape of the target object more accurately, and the distortion degree of the corrected image is reduced. Therefore, according to the method for processing an image provided by the embodiment, in step S103, the target key feature of the target object can be acquired more accurately based on the depth information, and the distortion degree of the target object in the corrected image after the perspective conversion can be reduced.
In some embodiments, the target key features are all located on the same plane of the target object.
In the embodiment, the key target features which are obtained and located on the same plane can more accurately reflect the real shape of the target object on the plane, so that the distortion degree of subsequently generated and corrected image information can be reduced.
In some embodiments, step S103 comprises:
step A1: acquiring key characteristics of a target object based on the color image information;
step A2: determining target key features located on the same plane of the target object from the key features based on depth information.
In some embodiments, the key feature recognition model is trained using the target key point features of the target object and the corresponding depth information and color image information thereof located in the same plane (e.g., top plane) as the expected output and the expected input, respectively, so that the trained key feature model can obtain the key features of the target object located in the same plane based on the detected depth information and color image information.
Step S104: performing perspective-based conversion on the color image information according to the depth information and the target key features to generate corrected image information.
In the present embodiment, since the color image capturing unit is not directed toward the writing object on the desk, the written shape and the writing contents thereon are distorted in the color image information captured by the color image capturing unit due to the perspective principle. Therefore, according to the method of image processing provided by the present embodiment, in step S104, perspective-based conversion is performed on the color image information according to the depth information and the target key feature to generate corrected image information, thereby changing the viewing perspective of the target object in the corrected image information. Preferably, the angle of view in which the target object shown in fig. 2 is obliquely photographed may be converted into the angle of view that is directly opposite, so that the target object is restored to or close to its original shape in the corrected image information, for example, the target object having a trapezoid shape in the color figure information is rectangular in the target figure information by the angle of view conversion.
As a specific implementation, a perspective transformation matrix may be determined based on the depth information and the keypoint features, and the color image information may be subjected to perspective transformation according to the perspective transformation matrix to obtain corrected image information. The perspective transformation matrix may map the coordinates of the original image onto a new predetermined plane. In this embodiment, step S104 may also adopt other related perspective transformation technologies provided in the art, for example, a warp perspective method of OpenCV, and the disclosure is not limited herein.
In this way, according to the image processing method provided by the embodiment of the disclosure, the target key feature of the target object is acquired based on the depth information obtained according to the shooting height and the shooting angle of the color image acquisition unit, and the view angle conversion is performed on the color image information, so that the corrected image information can be obtained, and the viewing angle of the target object in the corrected image information is changed, so that the color image acquisition unit is not required to be shot right at the target object, and the color image acquisition unit is not required to be adjusted repeatedly during use, thereby facilitating the use of a user. In addition, the image processing method provided by the embodiment of the disclosure does not need an additional depth sensor, reduces the hardware cost, is integrated in the image capturing device, can enable the image capturing device to be matched with any video system on the market, and does not need to additionally install specific software or network.
In some embodiments, referring to fig. 2, the image capturing apparatus 10 includes a support 12 for supporting the color image capturing unit 11, and the support 12 is configured to be capable of adjusting a support height and a support angle to adjust a photographing height and a photographing angle of the color image capturing unit.
Illustratively, the support part has a height telescopically adjustable support rod and a rotatably adjustable support shaft for adjusting the support height and the support angle, respectively, but the present application is not limited thereto.
Further, in some embodiments, step S102 includes:
step B1: acquiring a support height and a support angle;
step B2: and determining depth information corresponding to the color image information according to the support height and the support angle.
In this embodiment, the camera height and the camera angle of the color image acquisition unit can be obtained by obtaining the support height and the support angle of the support portion, so that the depth information can be determined quickly. The mapping relation between the support height angle gear of the support part and the depth information can be stored in the image capturing device in advance, so that the depth information which can be determined by acquiring the height gear and the angle gear of the support part through the sensor can be obtained.
In some embodiments, the support portion 11 has at least one height gear corresponding to the imaging height of the color image capturing unit and at least one angle gear corresponding to the imaging angle of the color image capturing unit. In this embodiment, the camera shooting height and the camera shooting angle of the image capturing device can be adjusted, so that the user can conveniently use the camera shooting height and the camera shooting angle, which are convenient to determine, to correspond to the depth information.
Further, in some embodiments, step B1 includes: acquiring a height gear and an angle gear of the supporting part; step B2 includes: and determining depth information corresponding to the color image information according to the height gear and the angle gear.
In some embodiments, the method 100 further comprises: acquiring the direction characteristic of the target object based on the color image information; step 104 further comprises: performing perspective-based conversion on the color image information according to the depth information, the target key feature, and the direction feature to generate corrected image information.
The directional characteristic refers to a characteristic that can represent the orientation of the target object. For example, the target object is usually written, printed or displayed with text, and the current orientation of the target object is obtained by acquiring the orientation of the text. In addition, when the user uses the target object, the writing arm usually covers the bottom edge or the lower right corner of the target object, and therefore, the current orientation of the target object can be obtained by acquiring the edge and/or the corner of the target object covered by the writing arm of the user. Therefore, the present embodiment can know the current orientation of the target object by acquiring the directional characteristic of the target object, so that the perspective-based conversion can be performed on the color image information according to the orientation of the target object in the color image.
In some embodiments, the method 100 further comprises: and performing character recognition on the corrected image information to obtain a character recognition result. The character recognition result obtained by character recognition on the basis of correcting the image information has higher recognition accuracy, and the user experience is improved.
In some embodiments, step 103 further comprises:
step C1: acquiring at least one key feature of a target object based on the depth information and the color image information;
step C2: and fitting the at least one key feature to obtain at least one target key feature.
In an actual application scenario, it may happen that some target key features of a target object are located outside a picture, are occluded by a user hand, are distorted, and the like, thereby affecting subsequent view angle conversion processing. In this regard, the present embodiment may overcome the above problem by utilizing the characteristic that the target object generally has a regular shape, and fitting the identified partial key features to obtain at least one target key feature. For example, when the target object is a book, based on two edges of the identified book page, that is, key features, corners of the book page formed by the two edges, that is, target key features, may be fitted.
In some embodiments, the method 100 further comprises: and transmitting the color image information and the correction image information. In this embodiment, by sending the color image information and the corrected image information at the same time, the video receiver can obtain the real image captured by the color image capturing unit and the corrected image of the target object at the same time. For example, in an application scenario of online education, a teacher may simultaneously obtain a listening and speaking image of a student and a homework book image of the student when the student makes a question.
In some embodiments, the color image information and/or the corrected image information may be output based on a user's setting, for example, only the color image information or only the corrected image information may be output, or images of both viewing angles may also be output simultaneously.
Referring to fig. 3, fig. 3 shows a flowchart of a method 300 of image processing provided by an embodiment of the present disclosure. The method 300 is applied to an image capturing apparatus comprising a color image capturing unit and a support for supporting the color image capturing unit, the support having at least one height gear corresponding to a camera height of the image capturing apparatus and at least one angular gear corresponding to a camera angle of the image capturing apparatus. The image capturing device further comprises a sensor for acquiring the height gear and the angle gear. Comprising the steps S301-S305:
step S301: acquiring color image information through the color image acquisition unit;
step S302: acquiring a height gear and an angle gear of the supporting part through a sensor;
step S303: determining depth information corresponding to the color image information according to a preset mapping relation between a height gear and an angle gear and the depth information;
step S304: acquiring target key features of a target object based on the depth information and the color image information;
step S305: performing perspective-based conversion on the color image information according to the depth information and the target key features to generate corrected image information.
Accordingly, as shown in fig. 4, an embodiment of the present disclosure provides an apparatus 400 for image processing, including:
a color image obtaining unit 401, configured to obtain color image information through the color image collecting unit;
a depth information determining unit 402, configured to determine depth information corresponding to the color image information according to the shooting height and the shooting angle of the color image acquisition unit;
a feature obtaining unit 403, configured to obtain a target key feature of a target object based on the depth information and the color image information;
a view angle conversion unit 404, configured to perform view angle-based conversion on the color image information according to the depth information and the target key feature to generate corrected image information.
According to the image processing device provided by the embodiment of the disclosure, the target key feature of the target object is acquired based on the depth information obtained according to the shooting height and the shooting angle of the color image acquisition unit, and the view angle conversion is performed on the color image information, so that the corrected image information can be obtained, and the viewing angle of the target object in the corrected image information is changed, so that the color image acquisition unit is not required to be shot just over the target object, and the color image acquisition unit is not required to be adjusted repeatedly during use, the use by a user is facilitated, and the identification accuracy and the view angle conversion are improved and the conversion distortion degree is reduced by performing the target key feature identification and the view angle conversion based on the depth information. In addition, the image processing device provided by the embodiment of the disclosure can match the image capturing device with any video system on the market without additionally installing specific software or a network.
In some embodiments, the image capture device comprises a support for supporting the color image acquisition unit; the support portion is configured to be capable of adjusting a support height and a support angle to adjust a photographing height and a photographing angle of the color image capturing unit.
In some embodiments, the depth information determining unit 402 includes:
a support parameter acquiring subunit, configured to acquire the support height and the support angle;
and the depth information determining subunit is used for determining the depth information corresponding to the color image information according to the support height and the support angle.
In some embodiments, the target key feature comprises at least one vertex and/or at least one edge of the target object.
In some embodiments, the apparatus 400 further comprises:
a direction feature acquisition unit configured to acquire a direction feature of the target object based on the color image information;
the view conversion unit 404 is further configured to perform a view-based conversion on the color image information according to the depth information, the target key feature and the direction feature to generate corrected image information.
In some embodiments, the directional feature includes text on the target object, an edge and/or a corner of the target object that is covered by a user's writing arm.
In some embodiments, the view angle conversion unit 404 includes:
a matrix determining subunit, configured to determine a perspective transformation matrix according to the depth information;
and the perspective transformation subunit is used for carrying out perspective transformation on the color image information according to the perspective transformation matrix so as to obtain corrected image information.
In some embodiments, the apparatus 400 further comprises:
and the character recognition unit is used for carrying out character recognition on the corrected image information so as to obtain a character recognition result.
In some embodiments, the feature acquisition unit 403 includes:
a key feature obtaining subunit, configured to obtain at least one key feature of the target object based on the depth information and the color image information;
and the target feature fitting unit is used for fitting the at least one key feature to obtain at least one target key feature.
In some embodiments, the apparatus 400 further comprises:
an output unit configured to output the color image information and the correction image information.
For the embodiments of the apparatus, since they correspond substantially to the method embodiments, reference may be made to the partial description of the method embodiments for relevant points. The above-described apparatus embodiments are merely illustrative, in that modules illustrated as separate modules may or may not be separate. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Accordingly, in accordance with one or more embodiments of the present disclosure, there is provided an image capturing apparatus including:
at least one memory, at least one processor, and a color image acquisition unit;
wherein the memory is used for storing program codes, and the processor is used for calling the program codes stored in the memory to execute the method for processing the image provided by one or more embodiments of the disclosure.
Accordingly, according to one or more embodiments of the present disclosure, there is provided a non-transitory computer storage medium storing program code executable by a computer device to cause the computer device to perform a method of image processing provided according to one or more embodiments of the present disclosure.
Fig. 5 shows a schematic structural diagram of an image capture device 800 (e.g., a display device as shown in fig. 3) for implementing an embodiment of the disclosure. The image capturing device in the embodiments of the present disclosure may include a video camera, a still camera, a video camera, and an electronic device equipped with a video camera, such as a mobile phone, a notebook computer, a PAD (tablet computer), a PMP (portable multimedia player), a smart television, and the like. The image capturing apparatus shown in fig. 5 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.
As shown in fig. 5, the image capturing apparatus 800 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 801 that may perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)802 or a program loaded from a storage device 808 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data necessary for the operation of the image capturing apparatus 800 are also stored. The processing apparatus 801, the ROM 802, and the RAM 803 are connected to each other by a bus 804. An input/output (I/O) interface 805 is also connected to bus 804.
Generally, the following devices may be connected to the I/O interface 805: an input device 806 including a color image acquisition unit and, for example, a touch screen, a touch pad, a keyboard, a mouse, a microphone, an accelerometer, a gyroscope, or the like; output devices 807 including, for example, a Liquid Crystal Display (LCD), speakers, vibrators, and the like; storage 808 including, for example, magnetic tape, hard disk, etc.; and a communication device 809. For example, the storage 808 may store a first database and a second database, wherein the first database stores at least one first sub-program identifier of a first program; the second database stores at least one second sub-program identification of the first program. The communication means 809 may allow the image capturing apparatus 800 to perform wireless or wired communication with other apparatuses to exchange data. While fig. 5 illustrates the image capture device 800 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a non-transitory computer readable medium, the computer program comprising program code for executing an apparatus illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication means 809, or installed from the storage means 808, or installed from the ROM 802. The computer program, when executed by the processing apparatus 801, performs the above-described functions defined in the apparatus of the embodiments of the present disclosure.
It should be noted that the computer readable medium in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
In some embodiments, the clients, servers may communicate using any currently known or future developed network Protocol, such as HTTP (HyperText Transfer Protocol), and may interconnect with any form or medium of digital data communication (e.g., a communications network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed network.
The computer readable medium may be embodied in the image capturing apparatus; or may exist separately without being assembled into the image capturing device.
The computer readable medium carries one or more programs which, when executed by the image capture device, cause the image capture device to: acquiring color image information through a color image acquisition unit; determining depth information corresponding to the color image information according to the shooting height and the shooting angle of the color image acquisition unit; acquiring target key characteristics of a target object based on the depth information and the color image information; a perspective-based conversion is performed on the color image information according to depth information and target key features to produce corrected image information.
Computer program code for carrying out operations for the present disclosure may be written in any combination of one or more programming languages, including but not limited to an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, apparatuses, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present disclosure may be implemented by software or hardware. The name of a unit does not in some cases constitute a limitation on the unit itself, and for example, the color image acquisition unit may be described as "a unit for acquiring color image information by the color image acquisition unit".
The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), systems on a chip (SOCs), Complex Programmable Logic Devices (CPLDs), and the like.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
According to one or more embodiments of the present disclosure, there is provided a method of image processing, applied to an image capturing apparatus including a color image acquisition unit, the method including: acquiring color image information through the color image acquisition unit; determining depth information corresponding to the color image information according to the shooting height and the shooting angle of the color image acquisition unit; acquiring target key features of a target object based on the depth information and the color image information; performing perspective-based conversion on the color image information according to the depth information and the target key features to generate corrected image information.
According to one or more embodiments of the present disclosure, the image capturing apparatus includes a support portion for supporting the color image capturing unit; the support portion is configured to be capable of adjusting a support height and a support angle to adjust a photographing height and a photographing angle of the color image capturing unit.
According to one or more embodiments of the present disclosure, the determining depth information corresponding to the color image information according to the camera height and the camera angle of the color image capturing unit includes: acquiring the supporting height and the supporting angle; and determining depth information corresponding to the color image information according to the support height and the support angle.
According to one or more embodiments of the present disclosure, the target key feature comprises at least one vertex and/or at least one edge of the target object.
A method of image processing provided in accordance with one or more embodiments of the present disclosure further includes: acquiring the direction characteristic of the target object based on the color image information; the performing perspective-based conversion on the color image information according to the depth information and the target key features to generate corrected image information includes: performing perspective-based conversion on the color image information according to the depth information, the target key feature, and the direction feature to generate corrected image information.
The directional feature according to one or more embodiments of the present disclosure includes text on the target object, edges and/or corners of the target object covered by a user's writing arm.
According to one or more embodiments of the present disclosure, performing perspective-based conversion on the color image information according to the depth information and the target key feature to generate corrected image information includes: determining a perspective transformation matrix according to the depth information; and carrying out perspective transformation on the color image information according to the perspective transformation matrix to obtain corrected image information.
According to one or more embodiments of the present disclosure, the method further comprises: and performing character recognition on the corrected image information to obtain a character recognition result.
According to one or more embodiments of the present disclosure, the acquiring a target key feature of a target object based on the depth information and the color image information includes: acquiring at least one key feature of a target object based on the depth information and the color image information; and fitting the at least one key feature to obtain at least one target key feature.
According to one or more embodiments of the present disclosure, the method further comprises: outputting the color image information and the corrected image information.
According to one or more embodiments of the present disclosure, there is provided an apparatus for image processing, including: the color image acquisition unit is used for acquiring color image information through the color image acquisition unit; the depth information determining unit is used for determining depth information corresponding to the color image information according to the shooting height and the shooting angle of the color image acquisition unit; a feature acquisition unit configured to acquire a target key feature of a target object based on the depth information and the color image information; a perspective conversion unit for performing perspective-based conversion on the color image information according to the depth information and the target key feature to generate corrected image information.
According to one or more embodiments of the present disclosure, there is provided an image capturing apparatus including: at least one memory, at least one processor, and a color image acquisition unit; wherein the memory is configured to store program code, and the processor is configured to call the program code stored in the memory to perform a method of image processing provided according to one or more embodiments of the present disclosure.
According to one or more embodiments of the present disclosure, there is provided a non-transitory computer storage medium storing program code executable by a computer device to cause the computer device to perform a method of image processing provided according to one or more embodiments of the present disclosure.
The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the disclosure herein is not limited to the particular combination of features described above, but also encompasses other embodiments in which any combination of the features described above or their equivalents does not depart from the spirit of the disclosure. For example, the above features and (but not limited to) the features disclosed in this disclosure having similar functions are replaced with each other to form the technical solution.
Further, while operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are included in the above discussion, these should not be construed as limitations on the scope of the disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.
Although the subject matter has been described in language specific to structural features and/or logical acts of devices, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.
Claims (13)
1. A method of image processing for use in an image capture device, the image capture device comprising a color image capture unit, the method comprising:
acquiring color image information through the color image acquisition unit;
determining depth information corresponding to the color image information according to the shooting height and the shooting angle of the color image acquisition unit;
acquiring target key features of a target object based on the depth information and the color image information;
performing perspective-based conversion on the color image information according to the depth information and the target key features to generate corrected image information.
2. The method of image processing according to claim 1, wherein the image capturing apparatus includes a support for supporting the color image capturing unit;
the support portion is configured to be capable of adjusting a support height and a support angle to adjust a photographing height and a photographing angle of the color image capturing unit.
3. The method of image processing according to claim 2, wherein said determining depth information corresponding to the color image information from the imaging height and the imaging angle of the color image capturing unit comprises:
acquiring the supporting height and the supporting angle;
and determining depth information corresponding to the color image information according to the support height and the support angle.
4. The method of image processing according to claim 1, wherein the target key feature comprises at least one vertex and/or at least one edge of the target object.
5. The method of image processing according to claim 1, further comprising:
acquiring the direction characteristic of the target object based on the color image information;
the performing perspective-based conversion on the color image information according to the depth information and the target key features to generate corrected image information includes:
performing perspective-based conversion on the color image information according to the depth information, the target key feature, and the direction feature to generate corrected image information.
6. The method of image processing according to claim 5, wherein the directional feature comprises text on the target object, an edge and/or a corner of the target object covered by a user writing arm.
7. The method of image processing according to claim 1, wherein performing perspective-based conversion on the color image information based on the depth information and the target key feature to produce corrected image information comprises:
determining a perspective transformation matrix according to the depth information;
and carrying out perspective transformation on the color image information according to the perspective transformation matrix to obtain corrected image information.
8. The method of image processing according to claim 1, further comprising:
and performing character recognition on the corrected image information to obtain a character recognition result.
9. The method of image processing according to claim 1, wherein said obtaining target key features of a target object based on the depth information and the color image information comprises:
acquiring at least one key feature of a target object based on the depth information and the color image information;
and fitting the at least one key feature to obtain at least one target key feature.
10. The method of image processing according to claim 1, further comprising:
outputting the color image information and the corrected image information.
11. An apparatus for image processing, comprising:
the color image acquisition unit is used for acquiring color image information through the color image acquisition unit;
the depth information determining unit is used for determining depth information corresponding to the color image information according to the shooting height and the shooting angle of the color image acquisition unit;
a feature acquisition unit configured to acquire a target key feature of a target object based on the depth information and the color image information;
a perspective conversion unit for performing perspective-based conversion on the color image information according to the depth information and the target key feature to generate corrected image information.
12. An image capturing apparatus characterized by comprising:
at least one memory, at least one processor, and a color image acquisition unit;
wherein the memory is configured to store program code and the processor is configured to call the program code stored in the memory to perform the method of any of claims 1 to 10.
13. A non-transitory computer storage medium, characterized in that,
the non-transitory computer storage medium stores program code executable by a computer device to cause the computer device to perform the method of any of claims 1 to 10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011174740.9A CN112291445B (en) | 2020-10-28 | 2020-10-28 | Image processing method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011174740.9A CN112291445B (en) | 2020-10-28 | 2020-10-28 | Image processing method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112291445A true CN112291445A (en) | 2021-01-29 |
| CN112291445B CN112291445B (en) | 2023-04-25 |
Family
ID=74372838
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011174740.9A Active CN112291445B (en) | 2020-10-28 | 2020-10-28 | Image processing method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112291445B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113096194A (en) * | 2021-05-08 | 2021-07-09 | 北京字节跳动网络技术有限公司 | Method, device and terminal for determining time sequence and non-transitory storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2194725A1 (en) * | 2008-12-04 | 2010-06-09 | Samsung Electronics Co., Ltd. | Method and apparatus for correcting a depth image |
| CN107194958A (en) * | 2017-05-25 | 2017-09-22 | 深圳市金立通信设备有限公司 | A kind of method of image procossing, terminal and computer-readable recording medium |
| CN109101963A (en) * | 2018-08-10 | 2018-12-28 | 深圳市碧海扬帆科技有限公司 | Certificate image automatic positive method, image processing apparatus and readable storage medium storing program for executing |
| CN109996048A (en) * | 2017-12-29 | 2019-07-09 | 深圳市Tcl高新技术开发有限公司 | A kind of projection correction's method and its system based on structure light |
| CN110602344A (en) * | 2019-09-20 | 2019-12-20 | 北京维山科技有限公司 | High-speed shooting instrument and shooting method thereof |
| CN111339846A (en) * | 2020-02-12 | 2020-06-26 | 深圳市商汤科技有限公司 | Image recognition method and device, electronic equipment and storage medium |
-
2020
- 2020-10-28 CN CN202011174740.9A patent/CN112291445B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2194725A1 (en) * | 2008-12-04 | 2010-06-09 | Samsung Electronics Co., Ltd. | Method and apparatus for correcting a depth image |
| CN107194958A (en) * | 2017-05-25 | 2017-09-22 | 深圳市金立通信设备有限公司 | A kind of method of image procossing, terminal and computer-readable recording medium |
| CN109996048A (en) * | 2017-12-29 | 2019-07-09 | 深圳市Tcl高新技术开发有限公司 | A kind of projection correction's method and its system based on structure light |
| CN109101963A (en) * | 2018-08-10 | 2018-12-28 | 深圳市碧海扬帆科技有限公司 | Certificate image automatic positive method, image processing apparatus and readable storage medium storing program for executing |
| CN110602344A (en) * | 2019-09-20 | 2019-12-20 | 北京维山科技有限公司 | High-speed shooting instrument and shooting method thereof |
| CN111339846A (en) * | 2020-02-12 | 2020-06-26 | 深圳市商汤科技有限公司 | Image recognition method and device, electronic equipment and storage medium |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113096194A (en) * | 2021-05-08 | 2021-07-09 | 北京字节跳动网络技术有限公司 | Method, device and terminal for determining time sequence and non-transitory storage medium |
| CN113096194B (en) * | 2021-05-08 | 2024-03-26 | 北京字节跳动网络技术有限公司 | Method, device, terminal and non-transitory storage medium for determining time sequence |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112291445B (en) | 2023-04-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2018214365A1 (en) | Image correction method, apparatus, device, and system, camera device, and display device | |
| US10212337B2 (en) | Camera augmented reality based activity history tracking | |
| US8885963B2 (en) | Automatic image capture | |
| US9811910B1 (en) | Cloud-based image improvement | |
| US8818101B1 (en) | Apparatus and method for feature matching in distorted images | |
| CN108932051B (en) | Augmented reality image processing method, apparatus and storage medium | |
| US8947453B2 (en) | Methods and systems for mobile document acquisition and enhancement | |
| EP2991339A2 (en) | Photographing method and electronic device | |
| US20130044194A1 (en) | Automatic image capture | |
| CN112085775A (en) | Image processing method, device, terminal and storage medium | |
| CN109660723B (en) | Panoramic shooting method and device | |
| EP3940633B1 (en) | Image alignment method and apparatus, electronic device, and storage medium | |
| CN111064895B (en) | Virtual shooting method and electronic equipment | |
| CN110930329A (en) | Starry sky image processing method and device | |
| US8983227B2 (en) | Perspective correction using a reflection | |
| CN112291445B (en) | Image processing method, device, equipment and storage medium | |
| CN114125411A (en) | Projection equipment correction method and device, storage medium and projection equipment | |
| CN114022662A (en) | Image recognition method, device, equipment and medium | |
| CN109840059B (en) | Method and apparatus for displaying image | |
| JP2018125658A (en) | Portable information processing device having camera function, display control method thereof, and program | |
| CN113223185A (en) | Image processing method and device, electronic equipment and storage medium | |
| CN107087114B (en) | Shooting method and device | |
| CN111353929A (en) | Image processing method and device and electronic equipment | |
| CN112991147B (en) | Image processing method, device, electronic equipment and computer readable storage medium | |
| JP5804430B2 (en) | Book information processing apparatus, book information processing method, and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |