CN110297556B - Electronic projection drawing board system based on image recognition technology and processing method thereof - Google Patents
Electronic projection drawing board system based on image recognition technology and processing method thereof Download PDFInfo
- Publication number
- CN110297556B CN110297556B CN201910591343.2A CN201910591343A CN110297556B CN 110297556 B CN110297556 B CN 110297556B CN 201910591343 A CN201910591343 A CN 201910591343A CN 110297556 B CN110297556 B CN 110297556B
- Authority
- CN
- China
- Prior art keywords
- laser
- drawing board
- projection
- image
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
- G06F3/038—Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
- G06F3/0383—Signal control means within the pointing device
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
- G06F3/038—Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
- G06F3/0386—Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry for light pen
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/12—Picture reproducers
- H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
- H04N9/3141—Constructional details thereof
Abstract
An electronic projection drawing board system based on image recognition technology and a processing method thereof; the system comprises a laser pen, an electronic drawing board processing device, a camera module, a projector and a projection screen; the electronic drawing board processing device comprises a wireless signal receiving module, a Bluetooth receiving module, an image processing module and a display module; the laser pen comprises a pen point, a pressure sensing device, a laser emitter, a function switch, a function key and a Bluetooth module; the camera module comprises a camera and a wireless signal generation module; the laser pen has two modes, namely a drawing board mode and a mouse mode; the drawing board mode laser straight is connected with the projection screen for writing, the system identifies laser points to form drawing board tracks, and the drawing board tracks are projected on the projection screen; the mouse mode laser pen remotely emits laser, the system identifies a laser point to form a mouse track, and the mouse control is completed by matching with the function keys of the laser pen.
Description
Technical Field
The invention relates to the technical field of projection demonstration, in particular to an electronic projection drawing board system based on an image recognition technology and a processing method thereof.
Background
Many functions of the current projection systems are not fully developed. For example, when a teacher gives a lecture, the teacher needs to write on a blackboard with chalk, and cannot mark an important point on a slide, and some whiteboard systems are only matched with the teacher, and the whiteboard system is composed of a large number of sensors, the density and the precision of the sensors often determine the precision of a drawing board, so that the more precise whiteboard is more expensive, and the whiteboard needs to be connected with a projector in advance for correction.
The technical defects of the existing whiteboard and projector integrated system are as follows:
1. the correction is cumbersome. During correction, the pen is required to be used for correcting on the white board for multiple times, so that the writing track is approximately the same as the pen point track. If the projector is mistakenly touched, the correction needs to be carried out again.
2. The equipment is cumbersome. At present, the equipment can only be used by being connected with a specific white board, and the projector cannot be used for writing where the projector is projected.
3. The manufacturing cost is high. At present, the equipment needs a matched white board, which inevitably increases the cost of manufacturing and maintenance.
4. The application direction is limited. At present, when the equipment is used, related equipment needs to be built in advance, and the equipment is only applied to the field of education at present.
Disclosure of Invention
Aiming at the defects in the prior art, the invention aims to provide an electronic projection drawing board system based on an image recognition technology and a processing method thereof. The operation difficulty is greatly simplified, the cost is reduced, and convenient projection and writing operation is realized.
In order to achieve the purpose, the invention adopts the technical scheme that: constructing an electronic projection drawing board system based on an image recognition technology; the system comprises a laser pen, an electronic drawing board processing device, a camera module, a projector and a projection screen; the electronic drawing board processing device comprises a wireless signal receiving module, a Bluetooth receiving module, an image processing module and a display module; the laser pen comprises a pen point, a pressure sensing device, a laser emitter, a function selector switch, a function button and a Bluetooth module; the camera module includes establishes camera, wireless signal generation module, wherein:
the wireless signal receiving module is used for receiving the image signal sent by the wireless signal generating module in the image pickup module and transmitting the signal to the image processing module;
the Bluetooth receiving module is used for receiving a control signal sent by the Bluetooth module in the laser pen and transmitting the signal to the image processing module;
the image processing module is used for identifying the laser point coordinates in the received image signals, converting the laser point coordinates to drawing board coordinates or mouse coordinates, and sending the coordinates and control signals to the display module;
the display module is used for converting the coordinates of the image processing module into a drawing board track or a mouse track, and combining the track, a control signal and projection content to generate new projection content;
the laser pen is used for directly writing on the projection screen or remotely emitting laser points to irradiate on the projection screen, control operation is carried out, and control operation signals are transmitted to the Bluetooth receiver of the electronic drawing board processing device through the Bluetooth module;
and the camera module is used for capturing the projection screen image in real time and transmitting the data to a signal receiver of the electronic drawing board processing device through the wireless signal generation module.
According to the electronic projection drawing board system based on the image recognition technology, the pen point of the laser pen is a detachable cylinder made of transparent plastic, and the function of guiding laser is achieved.
According to the electronic projection drawing board system based on the image recognition technology, the tail part of the pen point of a laser pen is connected with a pressure sensing device, the pressure sensing device is connected with a laser emitter, and a laser emitting port of the laser emitter is aligned to the pen point; the function change-over switch in the laser pen is the drawing board function, and the pressure sensing device detects pressure when the pen point writes and controls the laser emitter to emit laser.
The invention relates to an electronic projection drawing board system based on an image recognition technology.A laser pen comprises a spring, a laser switch and a functional key; the laser switch and the function key are positioned on the shell of the laser pen; one end of the spring is connected with the shell of the laser pen, and the other end of the spring is connected with the Bluetooth module; the other end of the Bluetooth module is connected with the opposite end of the transmitting port of the laser transmitter; the emission port of the laser emitter is connected with the pressure sensing device, the other end of the pressure sensing device is connected with the tail of the pen point, and the head of the pen point directly writes on the projection screen; the pen point, the pressure sensing device, the laser emitter, the Bluetooth module and the spring are placed in the same horizontal direction; the spring can play a role in protecting the laser pen when falling; the laser switch can control the on and off of the laser pen; the laser pen function switch has a mouse function, and the function keys can be used as a left key and a right key of the mouse.
The invention relates to an electronic projection drawing board system based on an image recognition technology.A camera module comprises two cameras; the two cameras are mutually perpendicular and respectively arranged at two crossed sides of the projection screen, and the image pickup area monitored by the two cameras is larger than that of the projection screen; when shooting, the hand shields the laser point to a certain degree, and one laser point can be shot.
The invention also provides an electronic projection drawing board system processing method based on the image recognition technology, which specifically comprises the following steps:
s1, positioning a projection plane of a detection projector by a camera module through a positioning point;
s2, detecting whether a laser pen function switch is turned on, if so, enabling the function switch to be in a drawing board mode, and entering S3; if not, the function switch is in a mouse mode, and then the step S7 is entered;
s3, detecting whether the pen point of the laser pen has pressure or not, and if not, stopping the laser emitter in the laser pen from emitting laser; if yes, a laser emitter in the laser pen emits laser, and the step S4 is carried out;
s4, the electronic drawing board processing device receives the image signal transmitted by the camera module in real time, and the laser signal is converted into a position signal through the image processing module;
s5, converting the position signal into a drawing board movement track signal by a display module;
s6, the display module combines the drawing board track and the projection content to generate new projection content; entering step S10;
s7, receiving an image signal transmitted by the camera module in real time by the electronic drawing board processing device, and converting the laser signal into a position signal through the image processing module;
s8, converting the position signal into a mouse moving track signal by the display module;
s9, combining the mouse track, the mouse control signal and the projection content by the display module to generate new projection content;
and S10, projecting the new projection content to a projection screen by the projector.
The invention relates to a processing method of an electronic projection drawing board system based on an image recognition technology; the invention relates to a processing method of an electronic projection drawing board system based on an image recognition technology; s1, positioning and coordinate determination linear coordinate conversion algorithm, which specifically comprises the following steps:
s11, initializing a system, wherein (X, Y) is the abscissa and ordinate of a picture shot by a camera, and (X, Y) is the abscissa and ordinate of an actual picture on a projection screen, a m ,a n ,b m ,b n ,d x ,d y Constants of the correlation formula of the continuous coordinates; displaying the projector projection picture as black, and marking four points of 100 pixels on the upper, lower, left and right sides of the projection picture as red, wherein the coordinates of the four points in the captured image of the camera are (x 1, y 1), (x 2, y 2), (x 3.y 3), (x 4.y 4); the coordinates of the picture projected on the projection screen by the projector are (X1, Y1), (X2, Y2), (X3, Y3), (X4, Y4);
s12, combining every three points of the four points in the S11 into the following formula to express:
d x ＝X 1 -a m x 1 -a n y 1
d y ＝Y 1 -b m x 1 -b n y 1
can obtain a m ，a n ，b m ，b n ，d x ，d y The numerical value of (c). Taking the average value of the final calculation result as a final value; the positional relationship of the (X, Y) coordinates and the (X, Y) coordinates is obtained.
The invention relates to a processing method of an electronic projection drawing board system based on an image recognition technology, which is characterized in that an image processing module recognizes the coordinates of a laser point in a camera image; and substituting the position relation function into the step S1 to obtain the coordinates of the laser point on the projection screen.
The invention relates to a processing method of an electronic projection drawing board system based on an image recognition technology, which is characterized in that the tail part of a pen point of a laser pen is connected with a pressure sensing device, the pressure sensing device is connected with a laser emitter, and a laser emitting port of the laser emitter is aligned with the pen point; the function change-over switch in the laser pen is the drawing board function, and the pressure sensing device detects pressure when the pen point writes and controls the laser emitter to emit laser.
In the processing method of the electronic projection drawing board system based on the image recognition technology, the laser signal is converted into the position signal in the step S7, and the position signal can also be obtained by adopting an image perspective transformation method, and the method specifically comprises the following steps:
s13, setting four vertexes of the projected image as specific colors, and identifying coordinates of the four vertexes of the projected image in the picture shot by the camera 31 by the image processing module 22 to obtain a quadrangle;
s14, a quadrangle can be converted into a square through a general conversion formula of perspective conversion;
s15, performing perspective transformation again, and transforming the square into a new quadrangle with the same projection image size on the projection screen;
and S16, identifying the laser point coordinate in the new quadrangle, wherein the coordinate is the needed laser position coordinate.
The invention has the beneficial technical effects that:
1. the pen can be freely written on a large screen when any teaching task such as slides, videos and the like is carried out, and is simple, convenient and practical.
2. The device is simple and accurate in identification, the common laser pen is simply transformed, complete human-computer interaction can be completed, and cost is greatly reduced.
3. The method is flexible and convenient, has no requirement on a writing plane, and can be adapted to different planes only by initialization adjustment.
4. The multifunctional drawing board has the drawing board function and the mouse function, and meets all requirements in use of the projector.
Drawings
Fig. 1 is a schematic diagram of a system structural framework of a processing method of an electronic projection drawing board system based on an image recognition technology;
FIG. 2 is an installation diagram of a processing method of an electronic projection drawing board system based on an image recognition technology;
FIG. 3 is a control flow diagram of a processing method of an electronic projection drawing board system based on an image recognition technology;
FIG. 4 is a schematic structural diagram of a laser pointer;
FIG. 5 is a schematic view of a camera installation;
FIG. 6 is a schematic coordinate diagram;
wherein: 1 laser pen, 2 electronic drawing board processing device, 3 camera module, 4 projector, 5 projection screen, 11 laser pen nib, 12 pressure sensing device, 131 function keys, 132 function keys, 14 function change-over switch, 15 laser emitter, 16 bluetooth module, 17 laser pen switch, 18 spring, 21 wireless signal receiving module, 22 image processing module, 23 display module, 24 bluetooth receiving module, 31 camera, 32 wireless signal generating module
Detailed Description
In order to make the technical means, the creation characteristics, the achievement purposes and the effects of the invention easy to understand, the invention is further explained by combining the specific embodiments.
As shown in fig. 1, fig. 2 and fig. 5, an electronic projection drawing board system based on image recognition technology; the system comprises a laser pen 1, an electronic drawing board processing device 2, a camera module 3, a projector 4 and a projection screen 5; a camera module 3 is arranged above a common projection screen 5; the camera module 3 sends the shot image information to the electronic drawing board processing device 2 in a wireless mode; the electronic drawing board processing device 2 identifies laser points emitted by the laser pen 1 in the image through internal software and processes the laser points to generate a drawing board track or a mouse track, and saves the drawing board track or the mouse track and the original projection content to generate image information; the generated image information is projected on a projection screen by the projector 4; the laser pen 1 is internally provided with a built-in function switch 14, so that the two functions of a drawing board function and a mouse function can be switched; the laser pen switched to the drawing board function can directly write on the projection screen, and the laser emitted by the laser pen 1 is recognized by the system to project a drawing board track; the laser emitted by the laser pen 1 is switched to the mouse function and is recognized by the system, the mouse track is projected, and the mouse function can be realized by matching with the function keys of the laser pen 1.
The electronic drawing board processing device 2 is composed of a wireless signal receiving module 21, an image processing module 22, a display module 23 and a Bluetooth receiving module 24; the wireless signal receiving device 21 is used for receiving the image information sent by the camera module 3 and transmitting the image information to the image processing module 22; the bluetooth receiving module 24 is used for receiving the control information sent by the laser pen and transmitting the control information to the image processing module 22; the image processing module 22 identifies position information generated by a laser point in the image information, and generates drawing board coordinates or mouse coordinates and a control signal by combining with the control information of the bluetooth signal receiving module 24; the display module 23, in combination with the image processing module 22, generates a drawing board trace or a mouse trace and a control signal, and generates a new computer interface in combination with the original computer interface, and the new computer interface is projected onto the projection screen 5 by the projector 4. The wireless signal receiving module 21, the image processing module 22, the display module 23 and the bluetooth receiving module 24 in the electronic palette processing device 2 may be externally connected to a computer as an integral peripheral device. The wireless signal receiving module 21 and the Bluetooth receiving module 24 can be externally connected to a computer, the image processing module 22 and the display module 23 form drawing board software which is installed on the computer and continuously arranged at the forefront end of a computer desktop, when the drawing board is used, the software does not respond to a mouse and a keyboard at all and does not influence the use of the mouse and the keyboard for other software under the drawing board software, the mouse can directly penetrate through the software, the keyboard can normally input to the software under an interface, and the software is closed by specifying a special shortcut key; when the mouse mode is used, the software uses the obtained coordinate information to operate the mouse, and various operations of the mouse are completed.
The camera module 3 is composed of two cameras 31 and a wireless signal generation module 32, the two cameras 32 are perpendicular to each other and are respectively installed on two sides of the projection screen 5, when the two cameras 31 are used, the situation that a laser point is shielded by a hand when a user directly writes on the screen can be prevented, the camera area monitored by the cameras 31 is larger than the area of the projection screen, and pictures projected on the projection screen 5 by the whole projector 4 can be comprehensively shot; the graphic information shot by the camera 31 is transmitted to the wireless signal receiving module 21 in the electronic palette processing apparatus 2 in the form of a wireless signal through the wireless signal generating module 32 in the camera module 3. The wireless signal generation module 32 can adopt a WIFI module, and WIFI has the advantages of wide radio wave coverage range, high transmission speed, safety, health and the like; the WIFI wireless sensor network is mainly characterized in that an unprocessed module and an RF transceiver module are integrated, for example, a CS1010 chip is integrated in the WIFI wireless sensor network, one ARM chip is responsible for data processing, the other ARM chip is used for wireless transmission, and the WIFI wireless sensor network can be used for reference in the project.
As shown in fig. 1, fig. 2 and fig. 4, the laser pen 1 includes a pen point 11, a pressure sensing device 12, a laser emitter 15, a function switch 14, a function button 131, a function button 132, a bluetooth module 16, and a laser pen switch 17; the Bluetooth module 16 is used for transmitting a control signal in the laser pen 1 to the electronic palette processing device 2; the function switch 14 can switch a drawing board mode and a mouse mode, when the drawing board mode is adopted, the hand-held drawing board writes on the projection screen, the pressure sensing device 12 is started by the stress of the pen point 11, the laser emitter 15 is turned on to emit laser, the laser forms a laser point on the projection screen 5, and the laser point is recognized by the system to generate a drawing board track; when in a mouse mode, the laser pen 1 can control the emitted laser to generate a laser point through the laser pen switch 17, and the laser point is recognized by a system to generate a drawing board track; the function buttons 131 and 132 can be used as left and right buttons of the mouse to control the interface, thereby implementing all functions of the mouse. The tip 11 may be made of a removable cylinder made of transparent plastic for guiding the laser. The spring 18 can be added to the laser pen 1, so that the shock absorption effect is achieved when the laser pen falls, internal devices are protected, and the service life is prolonged; the connection relation of all parts of the laser pen 1; the laser switch 17 and the function key 13, the function switch 14 is positioned on the shell of the laser pen 1; one end of the spring 18 is connected with the shell of the laser pen 1, and the other end of the spring is connected with the Bluetooth module 24; the other end of the Bluetooth module 24 is connected with the opposite end of the transmitting port of the laser transmitter 15; the emission port of the laser emitter 15 is connected with the pressure sensing device 12, the other end of the pressure sensing device 12 is connected with the tail part of the pen point 11, and the head part of the pen point 11 directly writes on the projection screen 5; the pen point 11, the pressure sensing device 12, the laser emitter 15, the Bluetooth module 24 and the spring 18 are arranged in the same horizontal direction.
As shown in fig. 3 and fig. 6, the present invention further provides a processing method for an electronic projection drawing board system based on an image recognition technology, which specifically includes the following steps:
s1, positioning a projection plane of a detection projector 4 by a camera module 3 through a positioning point;
s2, detecting whether the function switch 14 of the laser pen 1 is turned on, if so, enabling the function switch 14 to be in a drawing board mode, and entering S3; if not, the function switch 14 is in the mouse mode, and then the step S7 is entered;
s3, detecting whether the pen point of the laser pen 1 has pressure, and if not, stopping emitting laser by the laser emitter 15 in the laser pen 1; if yes, the laser emitter 15 in the laser pen 1 emits laser, and the step S4 is carried out;
s4, the electronic drawing board processing device 2 receives the image signals transmitted by the camera module 3 in real time, and the laser signals are converted into position signals through the image processing module 22;
s5, the display module 23 converts the position signal into a drawing board movement track signal;
s6, the display module 23 combines the drawing board track and the projection content to generate new projection content; entering step S10;
s7, the electronic drawing board processing device 2 receives the image signal transmitted by the camera module 3 in real time, and the laser signal is converted into a position signal through the image processing module 22;
s8, the display module 23 converts the position signal into a mouse moving track signal;
s9, the display module 23 combines the mouse track, the mouse control signal and the projection content to generate new projection content;
s10, the projector 4 projects the new projection content onto the projection screen 5.
The invention relates to a processing method of an electronic projection drawing board system based on an image recognition technology; s1, a linear coordinate conversion algorithm for positioning and coordinate determination specifically comprises the following steps:
s11, initializing the system, (X, Y) is the abscissa and ordinate of the image photographed by the camera 31, (X, Y) is the abscissa and ordinate of the actual image on the projection screen 5, i.e. the coordinate of the computer display image, a m ，a n ，b m ，b n ，d x ，d y Constants of the correlation formula of the continuous coordinates; the projector 4 displays the projection screen in black, and four points of 100 pixels each on the upper, lower, left, and right sides of the projection screen are marked in red, and the coordinates of the captured image of these four points at the camera 31 are (x 1, y 1), (x 2, y 2), (x 3.y 3), (x 4.y 4); the coordinates of the screen projected on the screen 5 by the projector 4 are (X1, Y1), (X2, Y2), (X3, Y3), and (X4, Y4).
S12, combining every three points of the four points in the S11 into the following formula to express:
d x ＝X 1 -a m x 1 -a n y 1
d y ＝Y 1 -b m x 1 -b n y 1
can obtain a m ，a n ，b m ，b n ，d x ，d y The numerical value of (c). Taking the average value of the final calculation results as a final value; the positional relationship of the (X, Y) coordinates and the (X, Y) coordinates is obtained.
According to the electronic projection drawing board system processing method based on the image recognition technology, the image processing module 22 recognizes the coordinates of the laser point in the shot image; substituting the position relation function obtained in the step S1 to obtain the coordinates of the laser point on the projection screen 5.
A processing method of an electronic projection drawing board system based on an image recognition technology is disclosed, wherein a laser signal is converted into a position signal in S7, and the position signal can also be obtained by adopting an image perspective transformation method, and the processing method specifically comprises the following steps:
s13, setting four vertexes of the projected image as specific colors, and identifying coordinates of the four vertexes of the projected image in the picture shot by the camera 31 by the image processing module 22 to obtain a quadrangle;
s14, a quadrangle can be converted into a square through a general conversion formula of perspective conversion;
s15, performing perspective transformation again to transform the square into a new square with the same size of projection images of the projection screen;
and S16, identifying the laser point coordinate in the new quadrangle, wherein the coordinate is the needed laser position coordinate.
The linear coordinate conversion algorithm for determining the detection points and coordinates in step S11 and step S12 has the following specific principles.
Firstly, identifying a detection point:
the man-machine interaction system in design needs to accurately position the position of a laser point in a projection screen and transmit the position information to a computer for processing, so as to position a cursor position and respond to corresponding laser point operation. Therefore, the system firstly uses the camera module to capture the projection screen in real time to obtain the video data image. Because each frame of digital images in the memory is stored continuously, the sequence images captured by the camera module have two formats, RGB24 and YUV. The system thus sets the image format taken by the camera module to RGB24 by software when it is first run. The target to be identified in the system is an infrared laser point emitted to a projection screen by a laser pen, the RGB value of a pixel point of the laser point in a digital image with an RGB24 format is (255, 0), and the value is very obvious relative to a background color R value, and the position and the motion track of the laser point can be identified and tracked by filtering each frame of bitmap image and selecting a certain specific threshold value T by adopting a method combining interframe difference and background difference. In the laser point identification process, one byte is needed for each component of R, G and B, and the sequence of the image color table in the format is arranged according to the BGR sequence, so that the distribution of the RGB value of one pixel point in the image is that the first byte represents the B value, the second byte represents the G value, the third byte represents the R value, and the R value of the pixel point is mainly compared with the selected threshold value T when the interframe difference is carried out.
Second coordinate determination-linear coordinate conversion algorithm
The interactive system of this research requires the rectangular area shot by the camera module to be parallel to the upper, lower, left and right edges of the projection screen area as much as possible during system initialization, but the absolute strict parallel cannot be guaranteed by manual adjustment, and the image plane of the camera module and the plane of the projection screen must be inclined by a certain angle. In this way, coordinate axes between the imaging coordinate system of the sequence images captured by the camera module and the plane coordinate system of the projection screen are not parallel (assuming that the angular deviation is a), the coordinate origins are different, and the coordinate scales are different. It is assumed that the camera module captures a sequence of images of size 640 x 480 and the projection screen plane is of size 1024 x 768. The linear coordinate transformation algorithm assumes that no image distortion exists in the captured video sequence image, and we can consider that the image formed by the rectangular projection screen on the wide-angle USB camera module is also rectangular, so that the laser point coordinate transformation is a linear transformation process
As shown in fig. 6, the rectangular area represented by the xoy coordinate system is the area shot by the camera module, and the coordinate system is the video image imaging coordinate system. The rectangular area represented by the XOY coordinate system is a projection screen display area, the coordinate system is a logical coordinate system of the projection screen, and an angle deviation between the two coordinate systems is a. The coordinate origin of the image coordinate system of the camera module and the coordinate origin of the projection screen coordinate system are both in the upper left corner.
Let the unit values of the XOY coordinate system and the XOY coordinate system be X ', Y', and according to the previous assumption, there is a certain linear relationship between them, which is defined as follows:
X’＝x’X’＝y’Y’＝x’Y’＝y’
as shown in the figure, the coordinate value of the coordinate origin of the XOY coordinate system in the XOY coordinate system is assumed to be (x) 0 ，y 0 ) Then, the following relationship can be obtained according to the graph:
wherein order
Substituting the above formula can obtain:
from the equation, it can be seen that the binary quadratic equation set contains 6 unknown parameters, and to solve the six unknown parameters according to the mathematical knowledge, it is required to know the coordinates of three points on the image captured by the camera module, which are (X1, Y1), (X2, Y2), (X3, Y3), and the coordinates (X1, Y1) (X2, Y2) (X3, Y3) of the three points on the corresponding projection screen. Substituting these six coordinate points can create 6 equations, so that six unknown parameters can be solved as follows:
d x ＝X 1 -a m x 1 -a n y 1
d y ＝Y 1 -b m x 1 -b n y 1
the six parameters are solved, and the formula of the binary quadratic equation system is determined.
From the analysis, it can be seen that solving the six parameters of the equation requires knowing the coordinates of three known points on the projection screen in the camera module image, where we select four points (100 ), (924, 100), (100, 668), (924, 100) of 100 pixels on the projection screen above, below, left, and right from the edge of the distance. The system is initialized, the screen is displayed as black, the four points are red, and the coordinates of the four points in the image imaging plane are calculated by using an image recognition technology. And then, the four points are combined and calculated once for every three points, and the average value of the final calculation result is taken as a final value. When the system runs, only real-time scanning is needed, and the position coordinates (X, Y) of the laser point on the image of the camera module are calculated through an image recognition algorithm, and the corresponding logical coordinates (X, Y) of the laser point on the projection screen can be calculated through a formula, so that the accurate positioning and the related operation of the laser point emitted by the laser pen on software are realized.
In addition, the laser point coordinates can also be obtained by adopting an image perspective transformation method, and the perspective transformation step comprises the following steps:
1. setting four fixed points of a projected image as specific colors, and identifying coordinates of the four fixed points of the projected image in a picture shot by a camera by an image processing device to obtain a quadrangle;
2. the quadrangle can be converted into a square by a general transformation formula of perspective transformation.
3. And performing perspective transformation again to transform the square into a new square with the same projection image size on the projection screen.
4. And identifying the coordinates of the laser points in the new quadrangle, wherein the coordinates are the coordinates needing projection.
The foregoing shows and describes the general principles and features of the present invention, together with the advantages thereof. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, which are described in the specification and illustrated only to illustrate the principle of the present invention, but that various changes and modifications may be made therein without departing from the spirit and scope of the present invention, which fall within the scope of the invention as claimed. The scope of the invention is defined by the appended claims and equivalents thereof.
Claims (4)
1. A processing method of an electronic projection drawing board system based on an image recognition technology specifically comprises the following steps:
s1, positioning a projection plane of a detection projector (4) by a camera module (3) through a positioning point; in the positioning, the linear coordinate conversion algorithm for determining the coordinates specifically comprises the following steps:
s11, initializing the system, wherein (X, Y) is the abscissa and ordinate of the picture shot by the camera (31), and (X, Y) is the abscissa and ordinate of the actual picture on the projection screen (5), a m ,a n ,b m ,b n ,d x ,d y Constants of the correlation formula of the continuous coordinates; displaying the projection picture of the projector (4) in black, and marking four points of 100 pixels on the upper, lower, left and right sides of the projection picture as red, wherein the coordinates of the captured images of the four points at the camera (31) are (x 1, y 1), (x 2, y 2), (x 3. Y3), (x 4.y 4); the coordinates of the picture projected on the projection screen (5) by the projector (4) are (X1, Y1), (X2, Y2), (X3, Y3), (X4, Y4);
s12, combining every three points of the four points in the S11 into the following formula to express:
d x ＝X l -a m x 1 -a n y 1
d y ＝Y 1 -b m x 1 -b n y 1
can obtain a m ,a n ,b m ,b n ,d x ,d y The numerical value of (c). Taking the average value of the final calculation result as a final value; the positional relationship of the (X, Y) coordinates and the (X, Y) coordinates is obtained.
S2, detecting whether a function switch (14) of the laser pen (1) is turned on, if so, enabling the function switch (14) to be in a drawing board mode, and entering S3; if not, the function switch (14) is in a mouse mode, and the step S7 is entered;
s3, detecting whether the pen point of the laser pen (1) has pressure, and if not, stopping emitting laser by a laser emitter (15) in the laser pen (1); if yes, a laser emitter (15) in the laser pen (1) emits laser, and the step S4 is carried out;
s4, the electronic drawing board processing device (2) receives the image signals transmitted by the camera module (3) in real time, and the laser signals are converted into position signals through the image processing module (22);
s5, converting the position signal into a drawing board movement track signal by a display module (23);
s6, combining the drawing board track and the projection content by a display module (23) to generate new projection content; the step S10 is entered;
s7, the electronic drawing board processing device (2) receives the image signals transmitted by the camera module (3) in real time, and the laser signals are converted into position signals through the image processing module (22);
s8, converting the position signal into a mouse moving track signal by the display module (23);
s9, combining a mouse track, a mouse control signal and projection contents by a display module (23) to generate new projection contents;
s10, the projector (4) projects the new projection content to the projection screen (5).
2. The processing method of the electronic projection drawing board system based on the image recognition technology according to claim 1, wherein the image processing module (22) identifies the coordinates of the laser point in the shot image; and substituting the position relation function obtained in the step S1 to obtain the coordinates of the laser point on the projection screen (5).
3. The processing method of the electronic projection drawing board system based on the image recognition technology is characterized in that the tail part of the pen point (11) of the laser pen (1) is connected with a pressure sensing device (12), the pressure sensing device (12) is connected with a laser emitter (15), and the laser emitting port of the laser emitter (15) is aligned with the pen point (11); the function switch (14) in the laser pen (1) has a drawing board function, and the pressure sensing device (12) detects pressure when the pen point (11) writes to control the laser emitter (15) to emit laser.
4. The processing method of the electronic projection drawing board system based on the image recognition technology according to claim 1, wherein the laser signal converted into the position signal in S7 can be obtained by an image perspective transformation method, and the specific steps are as follows:
s13, setting four vertexes of the projected image as specific colors, and identifying coordinates of the four vertexes of the projected image in the picture shot by the camera (31) by the image processing module (22) to obtain a quadrangle;
s14, a quadrangle can be converted into a square through a general conversion formula of perspective conversion;
s15, performing perspective transformation again, and transforming the square into a new quadrangle with the same projection image size on the projection screen;
and S16, identifying the laser point coordinate in the new quadrangle, wherein the laser point coordinate is the required laser position coordinate.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910591343.2A CN110297556B (en) | 2019-07-02 | 2019-07-02 | Electronic projection drawing board system based on image recognition technology and processing method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910591343.2A CN110297556B (en) | 2019-07-02 | 2019-07-02 | Electronic projection drawing board system based on image recognition technology and processing method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110297556A CN110297556A (en) | 2019-10-01 |
| CN110297556B true CN110297556B (en) | 2023-03-31 |
Family
ID=68030036
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910591343.2A Active CN110297556B (en) | 2019-07-02 | 2019-07-02 | Electronic projection drawing board system based on image recognition technology and processing method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110297556B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112015286B (en) * | 2020-07-31 | 2023-06-09 | 青岛海尔科技有限公司 | Method, device and projection system for interactive projection |
| CN112297035B (en) * | 2020-11-24 | 2022-03-04 | 武汉大学苏州研究院 | Six-dimensional control device for B-ultrasonic robot |
| CN112738490A (en) * | 2020-12-28 | 2021-04-30 | 慧投科技（深圳）有限公司 | Projection method of electronic conference whiteboard system without calibration |
| CN112785883A (en) * | 2021-01-27 | 2021-05-11 | 吕瑞 | Point reading position marking method and system based on image recognition and intelligent learning table |
| CN115016691A (en) * | 2022-05-30 | 2022-09-06 | 海宁奕斯伟集成电路设计有限公司 | Electronic blackboard function realization server, client, system and method |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102880360A (en) * | 2012-09-29 | 2013-01-16 | 东北大学 | Infrared multipoint interactive electronic whiteboard system and whiteboard projection calibration method |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4283317B2 (en) * | 2007-03-08 | 2009-06-24 | Ｌｕｎａｓｃａｐｅ株式会社 | Projector system |
| CN103092432B (en) * | 2011-11-08 | 2016-08-03 | 深圳市中科睿成智能科技有限公司 | The trigger control method of man-machine interactive operation instruction and system and laser beam emitting device |
| CN102662498B (en) * | 2012-04-16 | 2015-09-30 | 中科华核电技术研究院有限公司 | A kind of wireless control method of projection demonstration and system |
| CN105912145A (en) * | 2016-04-15 | 2016-08-31 | 武汉邮电科学研究院 | Laser pen mouse system and image positioning method thereof |
| CN108227970A (en) * | 2018-03-08 | 2018-06-29 | 李慕浩 | A kind of remote mouse based on image identification |
-
2019
- 2019-07-02 CN CN201910591343.2A patent/CN110297556B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102880360A (en) * | 2012-09-29 | 2013-01-16 | 东北大学 | Infrared multipoint interactive electronic whiteboard system and whiteboard projection calibration method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110297556A (en) | 2019-10-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110297556B (en) | Electronic projection drawing board system based on image recognition technology and processing method thereof | |
| CN103092432B (en) | The trigger control method of man-machine interactive operation instruction and system and laser beam emitting device | |
| US7683881B2 (en) | Visual input pointing device for interactive display system | |
| JP3950837B2 (en) | Projector, electronic blackboard system using projector, and indication position acquisition method | |
| US6275214B1 (en) | Computer presentation system and method with optical tracking of wireless pointer | |
| CN102253737B (en) | Realizing method of screen-vision mouse system | |
| US8188973B2 (en) | Apparatus and method for tracking a light pointer | |
| CN106843602B (en) | Large-screen remote control interaction system and interaction method thereof | |
| KR101378318B1 (en) | Electronic board system using infrared camera | |
| CN101859210A (en) | Interactive projection system and implementation method thereof | |
| US20120293555A1 (en) | Information-processing device, method thereof and display device | |
| CN101730876A (en) | Pointing device using camera and outputting mark | |
| CN101896867A (en) | Apparatus and method for tracking a light pointer | |
| CN102194136A (en) | Information recognition system and its control method | |
| KR100820573B1 (en) | Computer input device utilizing a camera to recognize position and twinkling compare laser pointing image with computer display picture | |
| US11073949B2 (en) | Display method, display device, and interactive projector configured to receive an operation to an operation surface by a hand of a user | |
| KR100844129B1 (en) | A paratus for interfacing a mouse using a camera image, system and method using the paratus, computer readable record medium on which a program therefore is recorded | |
| US20080252737A1 (en) | Method and Apparatus for Providing an Interactive Control System | |
| US20060197742A1 (en) | Computer pointing input device | |
| US20170357336A1 (en) | Remote computer mouse by camera and laser pointer | |
| JP2010272078A (en) | System, and control unit of electronic information board, and cursor control method | |
| JP5413315B2 (en) | Information processing system and display processing program | |
| CN110716669A (en) | Image interface positioning system | |
| US20110285624A1 (en) | Screen positioning system and method based on light source type | |
| KR101687954B1 (en) | Handwriting recognition system using stereo infrared camera and method thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |