CN114938446A - Animal behavior reconstruction system, method, device and storage medium - Google Patents
Animal behavior reconstruction system, method, device and storage medium Download PDFInfo
- Publication number
- CN114938446A CN114938446A CN202210527842.7A CN202210527842A CN114938446A CN 114938446 A CN114938446 A CN 114938446A CN 202210527842 A CN202210527842 A CN 202210527842A CN 114938446 A CN114938446 A CN 114938446A
- Authority
- CN
- China
- Prior art keywords
- animal behavior
- data
- behavior data
- camera
- cameras
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/296—Synchronisation thereof; Control thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
- Closed-Circuit Television Systems (AREA)
Abstract
The invention discloses a system, a method, a device and a storage medium for reconstructing animal behaviors, wherein the system comprises: the system comprises at least two cameras, a fixing frame, a timer and terminal equipment; the timer is in communication connection with the terminal equipment and the cameras respectively, the terminal equipment is in communication connection with the cameras respectively, and the cameras are arranged at different mounting positions of the fixing frame respectively; the camera is used for receiving the waveform signal sent by the timer, acquiring animal behavior data based on the waveform signal and sending the animal behavior data to the terminal equipment; and the terminal equipment is used for storing the received at least two animal behavior data and determining three-dimensional reconstruction data based on each animal behavior data. The embodiment of the invention solves the problem that the existing animal behavior reconstruction system can not meet the animal behavior acquisition requirement, not only can ensure the synchronization of animal behavior data acquired by a plurality of cameras respectively, but also can improve the acquisition frequency of the cameras in the animal behavior reconstruction system.
Description
Technical Field
The invention relates to the technical field of computer vision, in particular to an animal behavior reconstruction system, method, device and storage medium.
Background
Animal behaviors play an important role in scientific research, and at present, an animal behavior reconstruction system for reconstructing three-dimensional behaviors of animals mainly acquires through a single camera or a plurality of cameras.
However, the animal behavior reconstruction system with a single camera loses information relative to the three-dimensional scene, while the animal behavior reconstruction system with multiple cameras usually needs multiple cameras with the same acquisition frequency, and the acquisition frequency is not high enough, usually 30 frames/second. Because the animal behavior reconstruction system of a plurality of cameras relies on the acquisition performance of each camera, the phenomenon that animal behavior data acquired by a plurality of cameras are asynchronous easily appears in the too high acquisition frequency in the long-time acquisition process, and great difficulty is brought to the subsequent three-dimensional reconstruction task. And the animal behaviors under fine or high-speed motion cannot be captured by the lower acquisition frequency.
Disclosure of Invention
The embodiment of the invention provides an animal behavior reconstruction system, method, device and storage medium, which aims to solve the problem that the existing animal behavior reconstruction system cannot meet the animal behavior acquisition requirement, ensure the synchronization of animal behavior data acquired by a plurality of cameras and improve the acquisition frequency of the cameras in the animal behavior reconstruction system.
According to an embodiment of the present invention, there is provided an animal behavior reconstruction system including: the system comprises at least two cameras, a fixing frame, a timer and terminal equipment;
the timer is in communication connection with the terminal equipment and the cameras respectively, the terminal equipment is in communication connection with the cameras respectively, and the cameras are arranged at different mounting positions of the fixing frame respectively;
the camera is used for receiving the waveform signal sent by the timer, acquiring animal behavior data based on the waveform signal and sending the animal behavior data to the terminal equipment;
the terminal device is used for storing the received at least two animal behavior data and determining three-dimensional reconstruction data based on the animal behavior data.
According to another embodiment of the present invention, there is provided an animal behavior reconstruction method including:
responding to the received animal behavior data respectively sent by at least two cameras, and respectively executing storage operation on the animal behavior data; the animal behavior data are acquired by the camera based on received waveform data sent by the timer;
three-dimensional reconstruction data is determined based on each of the animal behavior data.
According to another embodiment of the invention, there is provided a computer readable storage medium having stored thereon computer instructions for causing a processor to execute a method of animal behavior reconstruction according to any one of the embodiments of the invention.
According to the technical scheme of the embodiment of the invention, the timer is additionally arranged in the animal behavior reconstruction system, each camera is used for receiving the waveform signal sent by the timer, acquiring the animal behavior data based on the waveform signal and sending the animal behavior data to the terminal equipment, the terminal equipment is used for storing the received at least two animal behavior data and determining the three-dimensional reconstruction data based on the animal behavior data, the problem that the existing animal behavior reconstruction system cannot meet the animal behavior acquisition requirement is solved, the synchronization of the animal behavior data acquired by the multiple cameras respectively can be ensured, and the acquisition frequency of the cameras in the animal behavior reconstruction system can be improved.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present invention, nor do they necessarily limit the scope of the invention. Other features of the present invention will become apparent from the following description.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the description below are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic structural diagram of an animal behavior reconstruction system according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of an animal behavior reconstruction system according to a second embodiment of the present invention;
fig. 3 is a flowchart of an animal behavior reconstruction method according to a third embodiment of the present invention;
fig. 4 is a schematic structural diagram of an animal behavior reconstruction device according to a fourth embodiment of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "comprises" and "comprising," and any variations thereof, in the description and claims of the present invention and the above-described drawings, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Example one
Fig. 1 is a schematic structural diagram of an animal behavior reconstruction system according to an embodiment of the present invention, which is applicable to the case of collecting and reconstructing animal behaviors, and the present embodiment can provide services for an animal behavior reconstruction method provided in the following embodiments, and can configure an animal behavior reconstruction device provided in the following embodiments. As shown in fig. 1, the system includes: at least two cameras 110, a fixed mount 120, a timer 130 and a terminal device 140.
The timer 130 is in communication connection with the terminal device 140 and each camera 110, the terminal device 140 is in communication connection with each camera 110, and each camera 110 is arranged at different mounting positions of the fixing frame 120; the camera 110 is configured to receive the waveform signal sent by the timer 130, collect animal behavior data based on the waveform signal, and send the animal behavior data to the terminal device 140; and the terminal equipment 140 is used for storing the received at least two animal behavior data and determining three-dimensional reconstruction data based on the animal behavior data.
Specifically, the timer 130 may be disposed in a single chip, and for example, the type of the single chip may be 51 single chips. The type of the single chip is not limited herein. The timer 130 may generate a waveform signal with a preset collection frequency, for example, the preset collection frequency may be 160Hz, and the type of the waveform signal includes, but is not limited to, a square wave signal, a triangular wave signal, or a sine wave signal.
Specifically, the fixing frame 120 may be disposed on a ceiling or a supporting device, or may be directly placed on the ground, and the shape of the fixing frame 120 may be rectangular, conical, or trapezoidal, and the size of the fixing frame 120 is not limited in this embodiment. Fig. 1 illustrates an example in which the shape of the fixing frame 120 is rectangular.
In one embodiment, optionally, the fixing frame 120 includes a predetermined number of fixing pillars, each camera 110 is respectively disposed at a predetermined mounting position on each fixing pillar, and the direction of the camera 110 is inclined downward; wherein the preset number is greater than or equal to the number of the cameras 110.
Specifically, each camera 110 is installed at a predetermined installation position of one of the fixed pillars, for example, the predetermined installation position may be a top position, a middle position, or a predetermined ratio position, for example, the predetermined ratio may be 1/3 or 1/4. Fig. 1 is an exemplary illustration taking a preset installation position as a middle position. Specifically, the preset installation positions corresponding to the cameras 110 are the same, so that the spatial consistency of the collected animal behavior data can be ensured.
The number of the cameras 110 may be 2, 4, or 5, where the number of the cameras 110 is not limited, and the entire shooting range formed by the multiple cameras 110 may be guaranteed to cover the complete shooting plane 101 or the three-dimensional shooting space. Fig. 1 illustrates an example of a shooting plane 101, in which an animal can freely move in the shooting plane 101. Of course, the application scene of the animal behavior reconstruction system of the embodiment may also be a three-dimensional cage, the three-dimensional cage corresponds to the three-dimensional shooting space, and the animal can move freely in the three-dimensional cage.
The models of the cameras 110 may be the same or different, and for example, the interface of each camera 110 is a USB3.0 interface.
For example, when the waveform signal is a square wave signal, the camera 110 collects animal behavior data when the received waveform signal is at a rising edge, and when the waveform signal is a triangular wave signal or a sine wave signal, the camera 110 collects animal behavior data when the received waveform signal is at a peak point. Wherein, the data type of the animal behavior data is video data or image data.
On the basis of the foregoing embodiment, optionally, the terminal device 140 is specifically configured to: aiming at each received animal behavior data, writing the animal behavior data into a queue of a memory by adopting a thread reading mechanism; and acquiring animal behavior data in the queue by adopting a thread writing mechanism, and storing the animal behavior data into a local storage space.
The multi-thread mechanism is started by using Python language, for example, and the language used for starting the multi-thread mechanism is not limited here.
The advantage of such a configuration is that, influenced by the performance of the central processing unit or the storage performance of the terminal device 140, the terminal device 140 in the conventional behavior acquisition system is very likely to lose frames when storing animal behavior data. The multi-thread mechanism can solve the problem of frame loss of the traditional behavior acquisition system, thereby improving the efficiency and quality of the three-dimensional reconstruction data generated by the animal behavior reconstruction system.
Specifically, in response to detecting the system initialization signal, the terminal device 140 establishes communication connections with the timer 130 and the cameras 110, respectively, and starts the cameras 110, so that the cameras 110 wait for receiving the waveform signal sent by the timer 130. For example, the system initialization command may be a power-on signal. In this embodiment, the terminal device 140 is further configured to, in response to detecting the acquisition start instruction input by the user, send the acquisition start instruction to the timer 130 through the serial port, so that the timer 130 starts generating the waveform signal. When the terminal device 140 stores enough animal behavior data, an acquisition ending instruction is sent to the timer 130 through the serial port, so that the locator stops generating the waveform signal. Sufficient animal behavior data can be determined by the number of stored data frames and/or the storage duration, for example, when the number of stored data frames is equal to the preset frame number threshold, an acquisition ending instruction is sent to the timer 130 through the serial port, and/or when the storage duration is equal to the preset duration threshold, an acquisition ending instruction is sent to the timer 130 through the serial port. For example, the preset frame number threshold may be 16000 frames, and the preset duration threshold may be 100 seconds.
According to the technical scheme, the timer is additionally arranged in the animal behavior reconstruction system, each camera is used for receiving the waveform signal sent by the timer, animal behavior data are collected based on the waveform signal and sent to the terminal equipment, the terminal equipment is used for storing the received at least two animal behavior data, and the three-dimensional reconstruction data are determined based on the animal behavior data, so that the problem that the existing animal behavior reconstruction system cannot meet the animal behavior collection requirement is solved, the synchronization of the animal behavior data collected by the cameras respectively can be ensured, and the collection frequency of the cameras in the animal behavior reconstruction system can be improved.
Example two
Fig. 2 is a schematic structural diagram of an animal behavior reconstruction system according to a second embodiment of the present invention, and this embodiment further optimizes "the terminal device 140" in the foregoing embodiment, as shown in fig. 2, the system includes: at least two cameras 110, a fixed mount 120, a timer 130, a router 150, a master device 141, and at least one slave device 142; the router 150 is configured to establish a communication connection between the master device 141 and each slave device 142.
The communication connection type between the host device 141 and each of the slave devices 142 is, for example, a TCP (Transmission Control Protocol) connection. The type of communication connection is not limited herein.
In one embodiment, optionally, each slave device 142 is in communication connection with at least one camera 110, and accordingly, the camera 110 is specifically configured to transmit the animal behavior data to the slave device 142 in communication connection with the camera 110.
For example, the behavior collection system includes 5 cameras 110, which are respectively a camera 1101, a camera 1102, a camera 1103, a camera 1104, and a camera 1105, and it is assumed that the slave device 142A is communicatively connected to the camera 1101, the camera 1102, and the camera 1103, respectively, the slave device 142B is communicatively connected to the camera 1104 and the camera 1105, respectively, the camera 1101, the camera 1102 and the camera 1103 transmit the respectively collected animal behavior data 1, animal behavior data 2 and animal behavior data 3 to the slave device 142A, so that the slave device 142A stores animal behavior data 1, animal behavior data 2, and animal behavior data 3, and the camera 1104 and the camera 1105 transmit respectively collected animal behavior data 4 and animal behavior data 5 to the slave device 142B, so that the slave device 142B stores the animal behavior data 4 and the animal behavior data 5.
In this embodiment, the slave device 142 may perform the storage operation of the animal behavior data by using the multithreading mechanism mentioned in the above embodiments.
In another embodiment, optionally, the host device 141 is in communication connection with at least one camera 110, and each of the slave devices 142 is in communication connection with at least one camera 110, and accordingly, the camera 110 is specifically configured to send animal behavior data to a target device in communication connection with the camera 110; the target device is the master device 141 or the slave device 142.
For example, the behavior collection system includes 5 cameras 110, which are respectively a camera 1101, a camera 1102, a camera 1103, a camera 1104, and a camera 1105, and assuming that the host device 141 is in communication connection with the camera 1101, the slave device 142A is in communication connection with the camera 1102 and the camera 1103, and the slave device 142B is in communication connection with the camera 1104 and the camera 1105, respectively, the camera 1101 sends collected animal behavior data 1 to the host device 141, so that the host device 141 stores the animal behavior data 1, the camera 1102 and the camera 1103 send collected animal behavior data 2 and animal behavior data 3 to the slave device 142A, so that the slave device 142A stores the animal behavior data 2 and animal behavior data 3, and the camera 1104 and the camera 1105 send collected animal behavior data 4 and animal behavior data 5 to the slave device 142B, so that the slave device 142B stores the animal behavior data 4 and the animal behavior data 5.
In this embodiment, the master device 141 is configured to execute the control timer 130, store part of the animal behavior data collected by the camera 110, and determine an operation of three-dimensional reconstruction data based on the locally stored animal behavior data and the animal behavior data stored in each slave device 142, where an attribute of the master device 141 is a master-slave machine, and the master device is used as the master device 141 to execute the operation of controlling the timer 130 and determining the three-dimensional reconstruction data, and is also used as the slave device 142 to execute the operation of storing the animal behavior data.
In this embodiment, the host device 141 or the slave device 142 may perform the storage operation of the animal behavior data by using the multithreading mechanism mentioned in the above embodiments.
The advantage of this setting is that can make full use of terminal device 150's storage performance in the animal behavior rebuilds system, increased the storable space that the animal behavior rebuilds system provided, improved the utilization ratio of terminal device 150's storable space of animal behavior rebuild system.
In an embodiment, optionally, the host device 141 is configured to generate an acquisition end instruction based on a preset storage condition and storage information corresponding to the preset storage condition, and send the acquisition end instruction to the timer 130, so that the timer 130 stops generating the model signal.
The preset storage condition includes that the number of stored data frames is equal to a preset frame number threshold and/or the storage duration is equal to a preset duration threshold, correspondingly, when the number of stored data frames is equal to the preset frame number threshold, an acquisition ending instruction is sent to the timer 130 through the serial port, and/or when the storage duration is equal to the preset duration threshold, the acquisition ending instruction is sent to the timer 130 through the serial port.
In another embodiment, optionally, the master device 141 is configured to send a preset storage condition to each slave device 142, and send an acquisition end instruction to the timer 130 in response to receiving an acquisition end instruction sent by each slave device 142, so that the timer 130 stops generating the waveform signal; the slave device 142 is configured to generate an acquisition end instruction based on the preset storage condition and the storage information corresponding to the preset storage condition, and send the acquisition end instruction to the host device 141.
The preset storage condition comprises that the number of stored data frames is equal to a preset frame number threshold value and/or the storage duration is equal to a preset duration threshold value.
Specifically, after the current animal behavior data is written in the read thread by the slave device 142, it is determined whether the storage information corresponding to the preset storage condition meets the preset storage condition, and if so, an acquisition ending instruction is generated, or after the current animal behavior data is stored in the write thread by the slave device 142, it is determined whether the storage information corresponding to the preset storage condition meets the preset storage condition, and if so, an acquisition ending instruction is generated.
The method of controlling the collection end by the host device 141 or the timer 130 may cause the slave device 142 not to store enough animal behavior data, or cause the slave device 142 to store redundant animal behavior data due to a communication delay problem between the host device 141 and the timer 130. In this embodiment, each slave device 142 sends a collection end instruction to the master device 141 when sufficient animal behavior data is stored, and at this time, the slave device 142 does not perform a writing operation even if it still receives the animal behavior data sent by the camera 110. The master device 141 also transmits the collection completion instruction to the timer 130 when receiving the collection completion instruction transmitted from each slave device 142. Therefore, the present embodiment can avoid the situation that the master device 141 stops the timer 130 from generating the waveform signal when the slave device 142 does not store enough animal behavior data, and can also avoid the situation that the slave device 142 stores redundant animal behavior data due to a communication delay problem between the master device 141 and the timer 130.
Specifically, in response to detecting the system initialization signal, the host device 141 establishes communication connections with the timer 130 and the cameras 110, respectively, and starts the cameras 110, so that the cameras 110 wait for receiving the waveform signal sent by the timer 130. And sending preset storage information to each slave device 142, wherein the preset storage information includes preset storage conditions, preset storage locations, preset storage names and the like.
When the resolution of a plurality of cameras in the acquisition and reconstruction system is high and the acquisition frequency is high, the data transmission bandwidth and the storage space of a single terminal device cannot easily meet the storage requirements of a large amount of animal behavior data, in the technical scheme of the embodiment, a host device, a router and at least one slave device are arranged in the animal behavior reconstruction system, the host device is in communication connection with the at least one camera, each slave device is in communication connection with the at least one camera, the router is used for establishing communication connection between the host device and each slave device, and the camera is used for sending the animal behavior data to a target device in communication connection with the camera; the target equipment is host equipment or slave equipment, the problem that the storage space of a traditional animal behavior reconstruction system is insufficient is solved, and the requirements of a behavior acquisition system on bandwidth and storage space are met.
The technical scheme of the embodiment can meet the acquisition performance requirements of 160Hz acquisition frequency and 720P resolution. And when a camera needs to be added, only one slave device needs to be added in the animal behavior reconstruction system.
EXAMPLE III
Fig. 3 is a flowchart of an animal behavior reconstruction method according to a third embodiment of the present invention, where the present embodiment is applicable to collecting and reconstructing animal behaviors, and the method may be executed by an animal behavior reconstruction device, and the animal behavior reconstruction device may be implemented in a hardware and/or software form, and may be configured in a terminal device in an animal behavior reconstruction system. As shown in fig. 3, the method includes:
and S310, responding to the received animal behavior data respectively sent by the at least two cameras, and respectively executing storage operation on the animal behavior data.
In this embodiment, the animal behavior data is collected by the camera based on the received waveform data sent by the timer.
And S320, determining three-dimensional reconstruction data based on the animal behavior data.
In one embodiment, optionally, the determining three-dimensional reconstruction data based on the individual animal behavior data comprises: respectively inputting at least two animal behavior data into a posture estimation model which is trained in advance to obtain output two-dimensional animal posture estimation data; and generating three-dimensional reconstruction data by adopting a triangular algorithm based on the camera parameter data and the two-dimensional animal posture estimation data.
In one embodiment, optionally, the method comprises: acquiring at least two calibration plate images respectively acquired by at least two cameras; the postures of the calibration plates corresponding to the images of the calibration plates acquired by the same camera are different; determining camera parameter data based on each calibration board image; the camera parameters comprise camera internal reference data and camera external reference data.
The pattern of the calibration plate may be a checkerboard or a solid circular array, for example. Taking a checkerboard as an example, the number of checkerboard dots may be 7 × 4, and the size of the checkerboard dots may be determined according to the shooting size of the shooting plane or the three-dimensional shooting space, where the larger the shooting size, the larger the size of the checkerboard dots, and conversely, the smaller the shooting size.
Specifically, the position and/or angle of the calibration plate corresponding to each calibration plate image collected by the same camera are different, illustratively, each camera respectively shoots 108 calibration plate images in different postures, and if the number of the cameras is 4, 432 calibration plate images are obtained altogether. And uploading the images of all the calibration plates to an MATLAB working space, and performing calibration operation by using a StereoCameraCalibrator GUI tool box provided by the MATLAB working space to obtain camera parameters. The camera internal reference data includes, but is not limited to, focal length and distortion parameters of the camera, and the like, and the camera internal reference data includes, but is not limited to, a rotation matrix and a translation vector.
The model type of the attitude estimation model may be, for example, a deep learning network model of deplab cut, and the model type of the attitude estimation model is not limited herein. Specifically, the two-dimensional animal pose estimation data can be used to characterize a two-dimensional skeleton sequence of the animal, with each two-dimensional callus data containing 16 body key point data.
According to the technical scheme, the timer is additionally arranged in the animal behavior reconstruction system, each camera is used for receiving the waveform signal sent by the timer, animal behavior data are collected based on the waveform signal and sent to the terminal equipment, the terminal equipment is used for storing the received at least two animal behavior data, and the three-dimensional reconstruction data are determined based on the animal behavior data, so that the problem that the existing animal behavior reconstruction system cannot meet the animal behavior collection requirement is solved, the synchronization of the animal behavior data collected by the cameras respectively can be ensured, and the collection frequency of the cameras in the animal behavior reconstruction system can be improved.
Example four
Fig. 4 is a schematic structural diagram of an animal behavior reconstruction device according to a fourth embodiment of the present invention. As shown in fig. 4, the apparatus includes: an animal behavior data storage module 410 and a three-dimensional reconstruction data generation module 420.
The animal behavior data storage module 410 is configured to perform storage operation on each animal behavior data in response to receiving animal behavior data sent by at least two cameras respectively; the animal behavior data are acquired by the camera based on received waveform data sent by the timer;
and a three-dimensional reconstruction data generation module 420 for determining three-dimensional reconstruction data based on the animal behavior data.
According to the technical scheme, the timer is additionally arranged in the animal behavior reconstruction system, each camera is used for receiving the waveform signal sent by the timer, animal behavior data are collected based on the waveform signal and sent to the terminal equipment, the terminal equipment is used for storing the received at least two animal behavior data, and the three-dimensional reconstruction data are determined based on the animal behavior data, so that the problem that the existing animal behavior reconstruction system cannot meet the animal behavior collection requirement is solved, the synchronization of the animal behavior data collected by the cameras respectively can be ensured, and the collection frequency of the cameras in the animal behavior reconstruction system can be improved.
On the basis of the foregoing embodiment, optionally, the three-dimensional reconstruction data generating module 420 is specifically configured to:
respectively inputting at least two animal behavior data into a posture estimation model which is trained in advance to obtain output two-dimensional animal posture estimation data;
and generating three-dimensional reconstruction data based on the camera parameter data and the two-dimensional animal posture estimation data by adopting a trigonometric algorithm.
On the basis of the above embodiment, optionally, the apparatus further includes:
the camera parameter data determining module is used for acquiring at least two calibration plate images respectively acquired by at least two cameras; the postures of the calibration plates corresponding to the images of the calibration plates acquired by the same camera are different;
determining camera parameter data based on each calibration plate image; the camera parameters comprise camera internal reference data and camera external reference data.
The animal behavior reconstruction device provided by the embodiment of the invention can execute the animal behavior reconstruction method provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method.
EXAMPLE five
An embodiment of the present invention further provides a computer-readable storage medium, in which computer instructions are stored, and the computer instructions are used to enable a processor to execute a method for reconstructing animal behaviors, where the method includes:
responding to the received animal behavior data respectively sent by the at least two cameras, and respectively executing storage operation on the animal behavior data; the animal behavior data are acquired by the camera based on received waveform data sent by the timer;
and determining three-dimensional reconstruction data based on the behavior data of each animal.
In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), blockchain networks, and the internet.
The computing system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service are overcome.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present invention may be executed in parallel, sequentially, or in different orders, and are not limited herein as long as the desired results of the technical solution of the present invention can be achieved.
The above-described embodiments should not be construed as limiting the scope of the invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. An animal behavior reconstruction system, comprising: the system comprises at least two cameras, a fixing frame, a timer and terminal equipment;
the timer is in communication connection with the terminal equipment and the cameras respectively, the terminal equipment is in communication connection with the cameras respectively, and the cameras are arranged at different mounting positions of the fixing frame respectively;
the camera is used for receiving the waveform signal sent by the timer, acquiring animal behavior data based on the waveform signal and sending the animal behavior data to the terminal equipment;
the terminal device is used for storing the received at least two animal behavior data and determining three-dimensional reconstruction data based on the animal behavior data.
2. The system of claim 1, wherein the fixed frame comprises a predetermined number of fixed pillars, and each of the cameras is respectively disposed at a predetermined mounting position on each of the fixed pillars, and the cameras are oriented obliquely downward; the preset number is larger than or equal to the number of the cameras.
3. The system according to claim 1, wherein the terminal device is specifically configured to:
aiming at each received animal behavior data, writing the animal behavior data into a queue of a memory by adopting a thread reading mechanism;
and adopting a thread writing mechanism to obtain the animal behavior data in the queue and storing the animal behavior data into a local storage space.
4. The system according to claim 1, wherein the terminal device includes a master device and at least one slave device, the master device is communicatively connected to at least one camera, each slave device is communicatively connected to at least one camera, and accordingly, the system further includes a router, the router is configured to establish a communication connection between the master device and each slave device, and the cameras are specifically configured to: sending the animal behavior data to target equipment in communication connection with the camera; wherein the target device is a master device or a slave device.
5. The system according to claim 4, wherein the master device is configured to send a preset storage condition to each slave device, and in response to receiving an acquisition end instruction sent by each slave device, send the acquisition end instruction to the timer, so that the timer stops generating the waveform signal;
and the slave device is used for generating an acquisition ending instruction based on the preset storage condition and the storage information corresponding to the preset storage condition and sending the acquisition ending instruction to the host device.
6. A method of animal behavior reconstruction, comprising:
responding to the received animal behavior data respectively sent by at least two cameras, and respectively executing storage operation on each animal behavior data; the animal behavior data are acquired by the camera based on received waveform data sent by the timer;
three-dimensional reconstruction data is determined based on each of the animal behavior data.
7. The method of claim 6, wherein determining three-dimensional reconstruction data based on each of the animal behavior data comprises:
respectively inputting at least two animal behavior data into a pre-trained attitude estimation model to obtain output two-dimensional animal attitude estimation data;
and generating three-dimensional reconstruction data by adopting a triangular algorithm based on the camera parameter data and the two-dimensional animal posture estimation data.
8. The system of claim 7, wherein the method further comprises:
acquiring at least two calibration plate images respectively acquired by at least two cameras; the postures of the calibration plates corresponding to the images of the calibration plates acquired by the same camera are different;
determining camera parameter data based on each calibration plate image; the camera parameters comprise camera internal reference data and camera external reference data.
9. An animal behavior reconstruction device, comprising:
the animal behavior data storage module is used for responding to the animal behavior data respectively sent by the at least two cameras and respectively executing storage operation on the animal behavior data; the animal behavior data are acquired by the camera based on received waveform data sent by the timer;
and the three-dimensional reconstruction data generation module is used for determining three-dimensional reconstruction data based on the animal behavior data.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores computer instructions for causing a processor to carry out the animal behavior reconstruction method of any one of claims 6-8 when executed.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210527842.7A CN114938446A (en) | 2022-05-16 | 2022-05-16 | Animal behavior reconstruction system, method, device and storage medium |
| PCT/CN2022/095626 WO2023221163A1 (en) | 2022-05-16 | 2022-05-27 | Animal behavior reconstruction system and method, and apparatus and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210527842.7A CN114938446A (en) | 2022-05-16 | 2022-05-16 | Animal behavior reconstruction system, method, device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114938446A true CN114938446A (en) | 2022-08-23 |
Family
ID=82865383
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210527842.7A Pending CN114938446A (en) | 2022-05-16 | 2022-05-16 | Animal behavior reconstruction system, method, device and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN114938446A (en) |
| WO (1) | WO2023221163A1 (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108537879A (en) * | 2018-03-29 | 2018-09-14 | 东华智业（北京）科技发展有限公司 | Reconstructing three-dimensional model system and method |
| CN208705935U (en) * | 2018-03-28 | 2019-04-05 | 东华智业（北京）科技发展有限公司 | Reconstructing three-dimensional model system and annular image acquisition device for the system |
| CN112101146A (en) * | 2020-08-28 | 2020-12-18 | 中国科学院深圳先进技术研究院 | Animal behavior record three-dimensional capturing device, method, system and application |
| CN113724368A (en) * | 2021-07-23 | 2021-11-30 | 北京百度网讯科技有限公司 | Image acquisition system, three-dimensional reconstruction method, device, equipment and storage medium |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101571492B (en) * | 2009-06-11 | 2012-01-25 | 长安大学 | Three-dimensional reconstruction based explosive fluidity measuring method |
| WO2018043225A1 (en) * | 2016-09-01 | 2018-03-08 | パナソニックＩｐマネジメント株式会社 | Multiple viewpoint image capturing system, three-dimensional space reconstructing system, and three-dimensional space recognition system |
| WO2019225682A1 (en) * | 2018-05-23 | 2019-11-28 | パナソニックＩｐマネジメント株式会社 | Three-dimensional reconstruction method and three-dimensional reconstruction device |
| CN111105486A (en) * | 2019-12-07 | 2020-05-05 | 东南大学 | Multi-view-angle dynamic three-dimensional reconstruction method for mice |
| CN112288853B (en) * | 2020-10-29 | 2023-06-20 | 字节跳动有限公司 | Three-dimensional reconstruction method, three-dimensional reconstruction device, and storage medium |
| CN114445562A (en) * | 2022-02-17 | 2022-05-06 | 北京大甜绵白糖科技有限公司 | Three-dimensional reconstruction method and device, electronic device and storage medium |
-
2022
- 2022-05-16 CN CN202210527842.7A patent/CN114938446A/en active Pending
- 2022-05-27 WO PCT/CN2022/095626 patent/WO2023221163A1/en unknown
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN208705935U (en) * | 2018-03-28 | 2019-04-05 | 东华智业（北京）科技发展有限公司 | Reconstructing three-dimensional model system and annular image acquisition device for the system |
| CN108537879A (en) * | 2018-03-29 | 2018-09-14 | 东华智业（北京）科技发展有限公司 | Reconstructing three-dimensional model system and method |
| CN112101146A (en) * | 2020-08-28 | 2020-12-18 | 中国科学院深圳先进技术研究院 | Animal behavior record three-dimensional capturing device, method, system and application |
| CN113724368A (en) * | 2021-07-23 | 2021-11-30 | 北京百度网讯科技有限公司 | Image acquisition system, three-dimensional reconstruction method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2023221163A1 (en) | 2023-11-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114527872B (en) | Virtual reality interaction system, method and computer storage medium | |
| CN110310326B (en) | Visual positioning data processing method and device, terminal and computer readable storage medium | |
| CN104378586B (en) | The method and system that vision facilities based on desktop virtualization is redirected | |
| US9460351B2 (en) | Image processing apparatus and method using smart glass | |
| CN104869203B (en) | Method of testing, device and the test equipment of interim card | |
| CN108776917B (en) | Synchronous processing method and device for virtual three-dimensional space | |
| CN108960889B (en) | Method and device for controlling voice speaking room progress in virtual three-dimensional space of house | |
| CN107919987B (en) | Method for realizing micro-service cloud deployment | |
| CN108765084B (en) | Synchronous processing method and device for virtual three-dimensional space | |
| JP2019194902A (en) | Information processing method, device, system, and storage medium | |
| CN111179309A (en) | Tracking method and device | |
| CN114938446A (en) | Animal behavior reconstruction system, method, device and storage medium | |
| CN111669647B (en) | Real-time video processing method, device and equipment and storage medium | |
| Ruffaldi et al. | Data collection and processing for a multimodal learning analytic system | |
| CN111213088B (en) | Network controlled 3D video capture | |
| CN109121194B (en) | Method and apparatus for state transition of electronic device | |
| CN113496176A (en) | Motion recognition method and device and electronic equipment | |
| CN111787221B (en) | Image acquisition method, device, system, equipment and computer storage medium | |
| CN113436318A (en) | Scene reconstruction method and device, electronic equipment and computer storage medium | |
| CN113780045A (en) | Method and apparatus for training distance prediction model | |
| CN114219109A (en) | Conference room patrol record management method and device and electronic equipment | |
| CN113033483B (en) | Method, device, electronic equipment and storage medium for detecting target object | |
| Gesslein et al. | Bodydigitizer: an open source photogrammetry-based 3d body scanner | |
| CN109873945A (en) | A kind of method, photo taking, photo processing method, electronic equipment and storage medium | |
| CN102523434A (en) | Image monitoring system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |