CN111873911A - Method, device, medium, and electronic apparatus for adjusting rearview mirror - Google Patents

Method, device, medium, and electronic apparatus for adjusting rearview mirror Download PDF

Info

Publication number
CN111873911A
CN111873911A CN202010770978.1A CN202010770978A CN111873911A CN 111873911 A CN111873911 A CN 111873911A CN 202010770978 A CN202010770978 A CN 202010770978A CN 111873911 A CN111873911 A CN 111873911A
Authority
CN
China
Prior art keywords
rearview mirror
adjusted
target object
sight
vehicle
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010770978.1A
Other languages
Chinese (zh)
Other versions
CN111873911B (en
Inventor
格润洲
刘强
刘先明
邵文昕
杨哲宁
李源
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Horizon Robotics Science and Technology Co Ltd
Original Assignee
Shenzhen Horizon Robotics Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Horizon Robotics Science and Technology Co Ltd filed Critical Shenzhen Horizon Robotics Science and Technology Co Ltd
Priority to CN202010770978.1A priority Critical patent/CN111873911B/en
Publication of CN111873911A publication Critical patent/CN111873911A/en
Application granted granted Critical
Publication of CN111873911B publication Critical patent/CN111873911B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/02—Rear-view mirror arrangements
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/02—Rear-view mirror arrangements
    • B60R1/06—Rear-view mirror arrangements mounted on vehicle exterior
    • B60R1/062—Rear-view mirror arrangements mounted on vehicle exterior with remote control for adjusting position
    • B60R1/07—Rear-view mirror arrangements mounted on vehicle exterior with remote control for adjusting position by electrically powered actuators

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Mechanical Engineering (AREA)
  • Image Analysis (AREA)

Abstract

A method, an apparatus, a medium, and an electronic device for adjusting a rearview mirror are disclosed, wherein the method for adjusting the rearview mirror includes: detecting a gaze direction of a target object; determining the target position of the rearview mirror to be adjusted according to the preset rear view angle corresponding to the rearview mirror to be adjusted and the sight direction; and generating a motor control command for driving the rearview mirror to be adjusted to move from the current position to the target position according to the current position of the rearview mirror to be adjusted and the target position, and outputting the motor control command. The technical scheme provided by the disclosure can realize the automatic adjustment of the position of the rearview mirror of the vehicle under the condition of avoiding installing additional equipment such as an eyeball positioner in the vehicle.

Description

Method, device, medium, and electronic apparatus for adjusting rearview mirror
Technical Field
The present disclosure relates to computer vision technologies, and in particular, to a method for adjusting a rearview mirror, an apparatus for adjusting a rearview mirror, a storage medium, and an electronic device.
Background
Whether the angle of the vehicle rearview mirror is suitable for the current driver or not is concerned about the driving safety of the vehicle. The manual adjustment of the rearview mirror sometimes brings inconvenience to the driver.
How to conveniently and accurately adjust the angle of the rearview mirror is a technical problem worthy of attention.
Disclosure of Invention
The present disclosure is proposed to solve the above technical problems. The embodiment of the disclosure provides a method and a device for adjusting a rearview mirror, a storage medium and an electronic device.
According to an aspect of an embodiment of the present disclosure, there is provided a method of adjusting a rearview mirror, including: detecting a gaze direction of a target object; determining the target position of the rearview mirror to be adjusted according to the preset rear view angle corresponding to the rearview mirror to be adjusted and the sight direction; and generating a motor control command for driving the rearview mirror to be adjusted to move from the current position to the target position according to the current position of the rearview mirror to be adjusted and the target position, and outputting the motor control command.
According to still another aspect of the embodiments of the present disclosure, there is provided an apparatus for adjusting a rearview mirror, including: the sight direction detection module is used for detecting the sight direction of the target object; the target position determining module is used for determining the target position of the rearview mirror to be adjusted according to the preset rear view angle corresponding to the rearview mirror to be adjusted and the sight direction detected by the sight direction detecting module; and the control module is used for generating a motor control command for driving the rearview mirror to be adjusted to move from the current position to the target position according to the current position of the rearview mirror to be adjusted and the target position determined by the target position determining module, and outputting the motor control command.
According to still another aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium storing a computer program for implementing the above method.
According to still another aspect of an embodiment of the present disclosure, there is provided an electronic apparatus including: a processor; a memory for storing the processor-executable instructions; the processor is used for reading the executable instructions from the memory and executing the instructions to realize the method.
According to the method and the device for adjusting the rearview mirror provided by the above embodiments of the present disclosure, the sight line direction of the target object is detected, and the motor control command is generated by using the sight line direction, so that the motor can be driven to adjust the position of the rearview mirror based on the command. In addition, with the improvement of the sight line direction detection accuracy, the accuracy of the rearview mirror adjustment of the disclosure can be improved accordingly. Therefore, the technical scheme provided by the disclosure is favorable for reducing the realization cost of adjusting the rearview mirror and improving the convenience and accuracy of adjusting the rearview mirror.
The technical solution of the present disclosure is further described in detail by the accompanying drawings and examples.
Drawings
The above and other objects, features and advantages of the present disclosure will become more apparent by describing in more detail embodiments of the present disclosure with reference to the attached drawings. The accompanying drawings are included to provide a further understanding of the embodiments of the disclosure, and are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the description serve to explain the principles of the disclosure and not to limit the disclosure. In the drawings, like reference numbers generally represent like parts or steps.
FIG. 1 is a schematic diagram of a scenario in which the present disclosure is applicable;
FIG. 2 is a flow chart of one embodiment of a method of adjusting a rearview mirror of the present disclosure;
FIG. 3 is a flow diagram of one embodiment of the present disclosure for obtaining a gaze direction of a target object;
FIG. 4 is a schematic diagram of one embodiment of a neural network for multitask learning according to the present disclosure;
FIG. 5 is a schematic view of one embodiment of an XOY plane of a first predetermined coordinate system of the present disclosure;
FIG. 6 is a schematic view of one embodiment of an XOZ plane of a first predetermined coordinate system of the present disclosure;
FIG. 7 is a schematic diagram illustrating one embodiment of calculating a first target angle according to the present disclosure;
FIG. 8 is a schematic view of an embodiment of the apparatus for adjusting a rearview mirror of the present disclosure;
fig. 9 is a block diagram of an electronic device provided in an exemplary embodiment of the present application.
Detailed Description
Example embodiments according to the present disclosure will be described in detail below with reference to the accompanying drawings. It is to be understood that the described embodiments are merely a subset of the embodiments of the present disclosure and not all embodiments of the present disclosure, with the understanding that the present disclosure is not limited to the example embodiments described herein.
It should be noted that: the relative arrangement of the components and steps, the numerical expressions, and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.
It will be understood by those of skill in the art that the terms "first," "second," and the like in the embodiments of the present disclosure are used merely to distinguish one element from another, and are not intended to imply any particular technical meaning, nor is the necessary logical order between them.
It is also understood that in embodiments of the present disclosure, "a plurality" may refer to two or more and "at least one" may refer to one, two or more.
It is also to be understood that any reference to any component, data, or structure in the embodiments of the disclosure, may be generally understood as one or more, unless explicitly defined otherwise or stated otherwise.
In addition, the term "and/or" in the present disclosure is only one kind of association relationship describing the associated object, and means that there may be three kinds of relationships, such as a and/or B, and may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" in the present disclosure generally indicates that the former and latter associated objects are in an "or" relationship.
It should also be understood that the description of the various embodiments of the present disclosure emphasizes the differences between the various embodiments, and the same or similar parts may be referred to each other, so that the descriptions thereof are omitted for brevity.
Meanwhile, it should be understood that the sizes of the respective portions shown in the drawings are not drawn in an actual proportional relationship for the convenience of description.
The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the disclosure, its application, or uses.
Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail but are intended to be part of the specification where appropriate.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, further discussion thereof is not required in subsequent figures.
Embodiments of the present disclosure may be implemented in electronic devices such as terminal devices, computer systems, servers, etc., which are operational with numerous other general purpose or special purpose computing system environments or configurations. Examples of well known terminal devices, computing systems, environments, and/or configurations that may be suitable for use with an electronic device, such as a terminal device, computer system, or server, include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, hand-held or laptop devices, microprocessor-based systems, set top boxes, programmable consumer electronics, network pcs, minicomputer systems, mainframe computer systems, distributed cloud computing environments that include any of the above, and the like.
Electronic devices such as terminal devices, computer systems, servers, etc. may be described in the general context of computer system-executable instructions, such as program modules, being executed by a computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, etc. that perform particular tasks or implement particular abstract data types. The computer system/server may be implemented in a distributed cloud computing environment. In a distributed cloud computing environment, tasks may be performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules may be located in both local and remote computer system storage media including memory storage devices.
Summary of the disclosure
In carrying out the present disclosure, the inventors have found that it is often necessary to adjust the angle of a rearview mirror in the event that a replacement driver, the rearview mirror of a vehicle are changed from a stowed position to a deployed position, the rearview mirror of the vehicle is crashed, or the like. Since the manual adjustment of the angle of the rearview mirror usually brings inconvenience to the driver, an implementation scheme for automatically adjusting the rearview mirror has appeared, and specifically, specialized sensors such as an eyeball detector are installed on the vehicle to detect the position of the eyeball of the driver and adjust the angle of the rearview mirror according to the position of the eyeball of the driver.
However, the installation of additional sensors on the vehicle not only occupies the limited space of the vehicle, but also adds a certain cost. If the adjustment of the rearview mirror can be realized by utilizing the existing facilities of the vehicle, better driving experience can be brought to a driver on the basis of not installing additional equipment and fully utilizing the existing facilities of the vehicle.
Brief description of the drawings
The technical scheme for adjusting the rearview mirror can be suitable for various scenes. One example is shown in figure 1.
In fig. 1, a vehicle 100 has two rear view mirrors, a left rear view mirror (may also be referred to as a left side rear view mirror) and a right rear view mirror (may also be referred to as a right side rear view mirror), located on the left and right sides of the vehicle. The vehicle 100 is equipped with a DMS (Driver monitor system), and a photograph or a video taken by a camera 101 installed in the vehicle 100 can be provided to the DMS in real time.
When the driver is at the driving position of the vehicle 100, if the driver finds that the position of the left rear-view mirror or the right rear-view mirror of the vehicle 100 needs to be adjusted, the rear-view mirror adjustment function in the DMS can be started at any time, and the DMS knows whether the current rear-view mirror to be adjusted is the left rear-view mirror or the right rear-view mirror of the vehicle 100 through the interaction based on the touch screen or the voice interaction.
After the DMS knows the current rearview mirror to be adjusted, the DMS may prompt the driver to look directly at the current rearview mirror to be adjusted for a certain time (e.g. look directly at the current rearview mirror to be adjusted for 2 seconds) by voice or text, so as to detect the current viewing angle of the driver, after detecting the current viewing angle of the driver, the DMS may determine the target position of the current rearview mirror to be adjusted based on the pre-stored predetermined rear viewing angle corresponding to the rearview mirror and the current viewing angle of the driver, further, the DMS may generate a corresponding motor control command for the motor corresponding to the current rearview mirror to be adjusted based on the current position of the current rearview mirror to be adjusted and the target position thereof, the motor corresponding to the current rearview mirror to be adjusted is driven by the motor control command to drive the current rearview mirror to be adjusted to rotate in the corresponding direction, therefore, the rearview mirror to be adjusted is moved to the determined target position from the current position.
The DMS may ask the driver by voice or text: whether the position of the rearview mirror to be adjusted is adjusted to a proper position or not. If the DMS receives the feedback information that the position of the current rearview mirror to be adjusted is adjusted to a proper position, the rearview mirror adjusting process can be finished; if the DMS receives the feedback information that the position of the current rearview mirror to be adjusted is not adjusted to a proper position, the DMS can execute the process of adjusting the rearview mirror again.
Exemplary method
FIG. 2 is a flow chart of one embodiment of a method of adjusting a rearview mirror of the present disclosure. The method shown in fig. 2 comprises: s200, S201, and S202. The following describes each step.
S200, detecting the sight line direction of the target object.
The target object in the present disclosure is generally an object that observes the situation behind the vehicle with a rearview mirror. For example, the target object may be a driver of the vehicle or the like. The gaze direction of the target object in the present disclosure may specifically be a gaze direction of the target object when gazing at the predetermined location, e.g. a gaze direction of the target object when gazing at the rear view mirror to be adjusted. The line of sight direction in the present disclosure may be a line of sight direction based on a first predetermined coordinate system. The first predetermined coordinate system may be a three-dimensional coordinate system, and the first predetermined coordinate system may be a coordinate system of a vehicle in which the rearview mirror to be adjusted is located, or a coordinate system of an imaging device mounted in the vehicle in which the rearview mirror to be adjusted is located. In one example, the coordinate system of the vehicle in which the rearview mirror to be adjusted is located may be the same coordinate system as the coordinate system of the imaging device mounted in the vehicle in which the rearview mirror to be adjusted is located. In another example, the coordinate system of the vehicle in which the rearview mirror to be adjusted is located may be different from the coordinate system of the imaging device mounted in the vehicle in which the rearview mirror to be adjusted is located.
The gaze direction of the target object in the present disclosure may include: at least one of a first line-of-sight direction of an XOY plane formed based on the X-coordinate axis and the Y-coordinate axis of the first predetermined coordinate system, a second line-of-sight direction of a YOZ plane formed based on the Y-coordinate axis and the Z-coordinate axis of the first predetermined coordinate system, and a third line-of-sight direction of an XOZ plane formed based on the X-coordinate axis and the Z-coordinate axis of the first predetermined coordinate system.
The present disclosure may perform a line-of-sight direction detection process on a planar image (i.e., a 2D image) captured by an image capturing apparatus to obtain a line-of-sight direction of a target object, and may detect the line-of-sight direction of the target object using a neural network or the like. In one example, in a case where the DMS in the vehicle has a sight-line direction detection function, the present disclosure may obtain a sight-line direction of the target object based on the sight-line direction detection function provided by the DMS.
S201, determining the target position of the rearview mirror to be adjusted according to the preset rear view angle corresponding to the rearview mirror to be adjusted and the sight direction.
The rear view mirror to be adjusted of the present disclosure may be a left rear view mirror or a right rear view mirror of a vehicle or a center rear view mirror located inside the vehicle. The predetermined rear view angle in the present disclosure may refer to a view angle in a view range based on a rear situation that the rear view mirror presents to the target object, such as a maximum view angle. The predetermined back viewing angle may also be referred to as a standard back viewing angle or an ideal back viewing angle or a target back viewing angle, etc. In addition, the predetermined rear view angle in the present disclosure may be a predetermined rear view angle based on a second predetermined coordinate system, where the second predetermined coordinate system may be a three-dimensional coordinate system, and the second predetermined coordinate system may be a coordinate system of a vehicle in which the rear view mirror to be adjusted is located, a coordinate system of an imaging device mounted in the vehicle in which the rear view mirror to be adjusted is located, or the like. In one example, the second predetermined coordinate system is the same coordinate system as the first predetermined coordinate system. In another example, the second predetermined coordinate system is a different coordinate system than the first predetermined coordinate system. Under the condition that the second preset coordinate system is different from the first preset coordinate system, the preset rear view angle corresponding to the rearview mirror to be adjusted and the sight direction of the target object can be adjusted to be in the same coordinate system, and then the target position of the rearview mirror to be adjusted is determined by utilizing the preset rear view angle and the sight direction of the target object in the same coordinate system.
The predetermined rear view angle in the present disclosure may include: at least one of a first predetermined rear-view angle of the XOY plane formed based on the X-coordinate axis and the Y-coordinate axis of the second predetermined coordinate system, a second predetermined rear-view angle of the YOZ plane formed based on the Y-coordinate axis and the Z-coordinate axis of the second predetermined coordinate system, and a third predetermined rear-view angle of the XOZ plane formed based on the X-coordinate axis and the Z-coordinate axis of the second predetermined coordinate system.
When the first predetermined coordinate system and the second predetermined coordinate system are the same coordinate system, if the line of sight direction of the target object includes the first line of sight direction, the predetermined rear view angle should include the first predetermined rear view angle. The predetermined back viewing angle should comprise a second predetermined back viewing angle if the gaze direction at the target object comprises a second gaze direction. The predetermined rear-view angle should include a third predetermined rear-view angle if the gaze direction at the target object includes a third gaze direction.
The predetermined rear view angle corresponding to the rearview mirror to be adjusted in the present disclosure generally does not change with different target objects. That is, for a vehicle, the predetermined rear view angle for each of the rear view mirrors in the vehicle is generally fixed.
The target position of the rearview mirror to be adjusted in the present disclosure may refer to: the rearview mirror to be adjusted can provide a position with a preset rearview angle for a target object. For example, when the target object observes the rear situation from the to-be-adjusted rearview mirror, assuming that the maximum width angle of view of the rear situation that the to-be-adjusted rearview mirror can currently present for the target object is a predetermined rear angle of view, then the current position of the to-be-adjusted rearview mirror is the target position of the to-be-adjusted rearview mirror. The method and the device can calculate the preset rear view angle corresponding to the rearview mirror to be adjusted and the sight direction of the target object based on the reflection principle of light rays and the like, so that the target position of the rearview mirror to be adjusted is obtained.
It should be noted that, because the heights, thicknesses, sitting postures and the like of different target objects of the present disclosure often have differences, the sight line directions of different target objects often are different, and thus although the predetermined rear view angles corresponding to the same rearview mirror to be adjusted are the same, the target positions of the rearview mirror to be adjusted determined based on different sight line directions often are different.
S202, generating a motor control command for driving the rearview mirror to be adjusted to move from the current position to the target position according to the current position of the rearview mirror to be adjusted and the target position, and outputting the motor control command.
The motor control command in the present disclosure may refer to a command for driving the motor to rotate, and the command may control the rotation amplitude of the motor, so as to achieve the purpose of causing the rearview mirror to be adjusted to move from the current position to the target position. The motor control command in the present disclosure may include a parameter for controlling the rotation amplitude of the motor, and the value of the parameter is usually determined by the current position of the rearview mirror to be adjusted and the target position of the rearview mirror to be adjusted.
In one example, the multiple rearview mirrors in a vehicle of the present disclosure may correspond to the same motor, that is, one motor is connected to each of the multiple rearview mirrors in the vehicle, and the purpose of adjusting the positions of the multiple rearview mirrors in the vehicle respectively may be achieved by using one motor. In another example, the multiple rear view mirrors in a vehicle of the present disclosure may correspond to multiple motors, for example, one rear view mirror corresponds to one motor (i.e., one motor is connected to one rear view mirror in the vehicle), and different rear view mirrors correspond to different motors (i.e., different rear view mirrors are connected to different motors), in which case, the motor control command of the present disclosure is a command for the motor corresponding to the rear view mirror to be adjusted, i.e., the motor control command should be issued to the corresponding motor. The motor can change the position of the rearview mirror connected with the motor through the rotation of the rotating shaft of the motor. The present disclosure does not limit the connection relationship between the motor and the mirror.
The motor control command in the present disclosure may be directly transmitted to the motor corresponding to the rearview mirror to be adjusted, i.e., the motor control command in the present disclosure is a command that can be correctly recognized by the motor control element. The motor control command can be transmitted to the motor corresponding to the rearview mirror to be adjusted after being subjected to command conversion. The purpose of command conversion of the motor control command comprises the following steps: the motor control commands are converted into commands that can be correctly recognized by the motor control components.
The rearview mirror can be automatically adjusted by detecting the sight direction of the target object and generating the motor control command by utilizing the detected sight direction and driving the motor to adjust the position of the rearview mirror, so that the rearview mirror can be automatically adjusted by watching a preset position on the target object to obtain the sight direction of the target object without installing additional equipment such as an eyeball positioner and the like in a vehicle. In addition, with the improvement of the sight line direction detection accuracy, the accuracy of the rearview mirror adjustment of the disclosure can be improved accordingly. Therefore, the technical scheme provided by the disclosure is favorable for reducing the realization cost of adjusting the rearview mirror and improving the convenience and accuracy of adjusting the rearview mirror.
In an alternative example, an embodiment of the present disclosure for detecting a gaze direction of a target object may include the following two steps:
and step A, outputting information prompting a driver to watch the rearview mirror to be adjusted, and acquiring at least one image through a vehicle-mounted camera device.
Optionally, this disclosure can adopt the voice mode, and the suggestion driver is watched and is waited to adjust rear-view mirror for a certain duration (if 2 seconds etc. at least), and this disclosure also can adopt screen display characters mode, and the suggestion driver is watched and is waited to adjust rear-view mirror for a certain duration. The purpose of enabling the driver to look at the rear-view mirror to be adjusted for a certain period of time includes: in the process that a driver gazes at the rearview mirror to be adjusted, the vehicle-mounted camera device is used for shooting so as to obtain at least one image containing the face of the driver gazing at the rearview mirror to be adjusted. The in-vehicle image pickup device in the present disclosure may be a monocular image pickup device, and the image in the present disclosure may be a 2D image.
And B, performing line-of-sight direction detection processing on the obtained at least one image to obtain the line-of-sight direction of the target object.
Optionally, the present disclosure may filter all the acquired images, and perform the sight direction detection processing on the filtered images. For example, the present disclosure may screen one or more images for which the image clarity meets a requirement, and further, for example, the present disclosure may screen one or more images for which the eye openness of the driver meets a requirement. According to the method and the device, the neural network can be used for respectively carrying out the sight direction detection processing on each image, so that the sight direction detection processing result corresponding to each image is obtained. When a plurality of sight line direction detection processing results are obtained, one sight line direction detection processing result can be selected from the plurality of sight line direction detection processing results, the selected sight line direction detection processing result is used as the sight line direction of the target object, the plurality of sight line direction detection processing results can be subjected to processing such as mean value calculation, and the processing results such as the mean value calculation are used as the sight line direction of the target object.
According to the method and the device, the information prompting the driver to watch the rearview mirror to be adjusted is output, so that the vehicle-mounted camera device can be used for shooting under the condition that the driver watches the rearview mirror to be adjusted, and the head posture of the driver in the shot image is ensured to be the posture watching the rearview mirror to be adjusted; carrying out sight direction detection processing by using the image, wherein the obtained sight direction is the sight direction of a driver watching the rearview mirror to be adjusted; when the target position of the rearview mirror to be adjusted is determined by utilizing the sight line direction of the rearview mirror to be adjusted watched by the driver and the preset rear view angle corresponding to the rearview mirror to be adjusted, the calculation process for determining the target position of the rearview mirror to be adjusted is facilitated to be simplified, and the accuracy of the target position is facilitated to be improved.
In an alternative example, the present disclosure obtains an example of the gaze direction of the target object by performing gaze direction detection processing on at least one image, as shown in fig. 3.
In fig. 3, a head image block and an eye image block of a target object in an image are acquired S300.
Optionally, the present disclosure may utilize a head/face region positioning technology to perform positioning processing on a head region of a target object in an image, so that a head image block of the target object may be cut out from the image. The eye region positioning method and the eye region positioning device can perform positioning processing on the eye region of the target object in the image by utilizing an eye region positioning technology, so that the eye image block of the target object can be cut out from the image. In addition, the present disclosure may also obtain an eye image block of the target object from a head image block of the target object. For example, the present disclosure may determine an eye region in a head image block according to a predetermined positional relationship between the head and the eyes, and cut out an eye image block of the target object from the head image block. The predetermined position relationship may include: the position relation between the highest position of the eyes and the whole height of the head image block, the position relation between the lowest position of the eyes and the whole height of the head image block, the position relation between the height of the eyes and the whole height of the head image block and the like.
Optionally, the eye image blocks in the present disclosure may be image blocks including both eyes. The eye image blocks in the present disclosure may also be image blocks including a single eye. For example, an image block containing the left eye or an image block containing the right eye.
Optionally, the spatial resolution of the head image block and the spatial resolution of the eye image block in the present disclosure may be the same, for example, the present disclosure may adjust the spatial resolution of the clipped head region and the spatial resolution of the eye region to predetermined spatial resolutions, respectively, so as to obtain the head image block and the eye image block having the same spatial resolutions.
S301, extracting the features of the head image block and the eye image block, and obtaining a first feature map of the head image block and a second feature map of the eye image block.
Optionally, the present disclosure may use the head image block and the eye image block as inputs respectively, provide the inputs to a neural network for extracting image features, and obtain the first feature map of the head image block and the second feature map of the eye image block based on an output of the neural network. In one example, the neural network for extracting the image features may be a multi-task learning neural network, that is, the present disclosure may obtain the first feature map of the head image block and the second feature map of the eye image block using a feature extraction operation of the multi-task learning based neural network.
Optionally, the neural network for multi-task learning in the present disclosure generally includes a trunk portion (i.e., a trunk unit) and a plurality of branch portions (i.e., a plurality of branch units), and the trunk portion and all the branch portions are combined together to complete a plurality of tasks at the same time. For example, a head posture detection task and a line-of-sight direction detection task are simultaneously completed.
Alternatively, one example of a neural network for multitask learning in the present disclosure is shown in fig. 4. The neural network for multitask learning in fig. 4 includes: a trunk unit 400 and at least two branch units. Branching unit 401 and branching unit 402 are shown schematically in fig. 4.
Optionally, the present disclosure may respectively use the head image block and the eye image block as inputs of the trunk unit 400 of the neural network for multitask learning, and provide the inputs to the neural network, so that the head image block and the eye image block may be respectively subjected to feature extraction operation processing based on feature extraction operation of the trunk unit 400 of the neural network, and further the present disclosure may obtain the first feature map of the head image block and the second feature map of the eye image block according to output of the trunk unit 400. The trunk unit 400 in the present disclosure may adopt a structure such as CNN (Convolutional Neural Networks). In one example, the trunk unit 400 may include: multilayer convolutional layers, multilayer active layers, and the like. In another example, the trunk unit 400 may include: a multilayer convolution layer, a multilayer activation layer, at least one pooling layer, and at least one normalization layer. The present disclosure does not limit the specific structure of the trunk unit 400.
Alternatively, the present disclosure may stitch together the head patch and the eye patch as inputs to the backbone unit 400, which is provided to the neural network. The present disclosure may perform a slicing operation on the output of the trunk unit 400, thereby obtaining a first feature map of the head image block and a second feature map of the eye image block.
In the training process of the neural network for multi-task learning, the losses corresponding to different branches are reversely propagated to the trunk unit for multi-task learning, so that the features extracted from the successfully trained trunk unit are more accurate, and the method and the device for extracting the features of the trunk unit for the neural network are favorable for obtaining more accurate first feature maps and second feature maps.
S302, acquiring the head posture of the target object according to the first characteristic diagram.
Optionally, the present disclosure may perform Head posture detection processing on the first feature map, and obtain a Head posture (Head position) of the target object according to a result of the Head posture detection processing. The head pose of the target object in the present disclosure may refer to information for describing a rotation condition of the head of the target object in at least one of the X, Y and Z coordinate axes. The head pose may also be referred to as the pose angle of the head. For example, the head pose of the present disclosure may include three euler angles, namely, a pitch angle, a yaw angle, and a roll angle of the head. The pitch angle is a rotation angle pitch around an X coordinate axis of a head coordinate system of the target object, and the pitch angle can indicate the head-up condition of the target object. Yaw, i.e. the rotation angle yaw around the Y coordinate axis of the head coordinate system of the target object, may indicate the shaking of the head of the target object. The roll angle, i.e. the rotation angle roll around the Z coordinate axis of the target object head coordinate system, may represent the turning situation of the target object.
Alternatively, the present disclosure may provide the first feature map as an input to a neural network for detecting a head posture, and obtain a head posture of the target object, for example, pitch (pitch angle), yaw (yaw angle), and roll (roll angle), based on an output of the neural network. In one example, the neural network for detecting head pose may be a multi-task learning neural network, i.e., the present disclosure may obtain the head pose of the target object using a head pose detection operation of the multi-task learning based neural network. In a more specific example, the present disclosure may utilize one of the branch units of a neural network for multitask learning to perform head pose detection operations. For example, the present disclosure may use the first feature map output by the trunk unit 400 as an input of a first branch unit (e.g., the branch unit 401 in fig. 4) of a neural network for multitask learning, so that the head posture detection processing may be performed on the first feature map based on the head posture detection operation of the first branch unit, and further, the present disclosure may obtain the head posture of the target object from the output of the first branch unit. The first branching unit in the present disclosure may adopt a CNN or the like structure. In one example, the first branching unit may include: at least one winding layer and at least one active layer. The present disclosure does not limit the specific structure of the first branching unit.
According to the method, the head posture detection operation is completed by utilizing one of the branch units of the multi-task learning neural network, and the first characteristic diagram extracted from the main unit of the multi-task learning neural network is more accurate, so that the more accurate first characteristic diagram is beneficial to optimizing the network parameters of the first branch unit in the training process of the multi-task learning neural network, and the accuracy of obtaining the head posture is improved.
And S303, acquiring the sight line direction of the target object in the image according to the second feature map and the head posture.
Optionally, the present disclosure may perform gaze direction detection processing on the second feature map and the head pose, and obtain a gaze direction of the target object according to a result of the gaze direction detection processing. The present disclosure may provide the second feature map and the head pose as inputs to a neural network for detecting a line-of-sight direction, and obtain the line-of-sight direction of the target object based on an output of the neural network. In one example, the neural network for detecting the direction of the line of sight may be a neural network for multitask learning, i.e., the present disclosure may obtain the line of sight of the target object using a line of sight direction detecting operation of the neural network based on the multitask learning. More specifically, the present disclosure may utilize one of the branch units of the neural network for multitask learning to perform the gaze direction detection operation. For example, the present disclosure may use the second feature map output by the trunk unit and the head pose output by the first branch unit (e.g., the branch unit 401 in fig. 4) described above as inputs to a second branch unit (e.g., the branch unit 402 in fig. 4) of the neural network for multitask learning, so that the line-of-sight direction detection processing may be performed on the second feature map and the head pose based on the line-of-sight direction detection operation by the second branch unit, and the present disclosure may obtain the line-of-sight direction of the target object from the output by the second branch unit. The second branch unit in the present disclosure may adopt a network structure such as CNN. In one example, the second branching unit may include: at least one winding layer and at least one active layer. The present disclosure does not limit the specific structure of the second branching unit.
Optionally, in a case that the eye image block is an image block including two eyes, the second feature map provided by the present disclosure for the second branch unit is a second feature map based on two eyes, the gaze direction detection operation performed by the second branch unit may be an eyeball gaze direction detection operation based on a left eye and an eyeball gaze direction detection operation based on a right eye, the present disclosure may obtain a left-eye eyeball gaze direction and a right-eye eyeball gaze direction respectively according to an output of the second branch unit, the present disclosure may perform weighted calculation on the left-eye eyeball gaze direction and the right-eye eyeball gaze direction, so as to obtain a weighted average, and the present disclosure may use the weighted average as the gaze direction of the target object.
It should be noted that, in the present disclosure, when performing weighting calculation, the weights corresponding to the left eye eyeball sight line direction and the right eye eyeball sight line direction may be the same or different. When the weights corresponding to the left eye eyeball sight line direction and the right eye eyeball sight line direction are different, the weight corresponding to the left eye eyeball sight line direction and the right eye eyeball sight line direction can be determined according to the position of the rearview mirror to be adjusted, for example, when the rearview mirror to be adjusted is a left side rearview mirror, the weight corresponding to the left eye eyeball sight line direction can be larger than the weight corresponding to the right eye eyeball sight line direction. For another example, when the rearview mirror to be adjusted is a right rearview mirror, the weight corresponding to the eye sight direction of the right eye may be greater than the weight corresponding to the eye sight direction of the left eye.
Optionally, in a case that the eye image block is an image block including a single eye, the second feature map provided by the present disclosure for the second branch unit is a second feature map based on the single eye, and the gaze direction detection operation performed by the second branch unit may be an eyeball gaze direction detection operation based on the single eye, for example, an eyeball gaze direction detection operation based on a left eye or an eyeball gaze direction detection operation based on a right eye. The present disclosure may obtain the left eye eyeball sight line direction or the right eye eyeball sight line direction according to the output of the second branching unit, and the present disclosure may directly use the left eye eyeball sight line direction or the right eye eyeball sight line direction as the sight line direction of the target object.
According to the method, the sight direction detection operation is completed by utilizing one of the branch units of the neural network for multi-task learning, the second characteristic diagram extracted by the main unit of the neural network for multi-task learning is more accurate, and the head posture is associated with the sight direction to a certain extent, so that in the training process of the neural network for multi-task learning, the more accurate second characteristic diagram and the head posture are favorable for optimizing the network parameters of the second branch unit, the sight direction detection operation is realized by utilizing the second branch unit of the neural network, and the sight direction accuracy is favorably improved. In addition, the visual line direction of the target object can be obtained by utilizing the left eye visual line direction and/or the right eye visual line direction of the target object, and the technical scheme is favorably improved in flexibility.
According to the method and the device, the sight line direction of the target object is obtained by utilizing the second characteristic diagram of the eye image block and the head posture of the target object, so that the error of the determined sight line direction is favorably reduced, and the accuracy of the sight line direction is favorably improved.
In an alternative example, according to the present disclosure, an example of determining the target position of the rearview mirror to be adjusted according to the predetermined rearview angle and the sight line direction corresponding to the rearview mirror to be adjusted may be: and determining a first target included angle between the rearview mirror to be adjusted and the vehicle width direction according to a first preset rearview angle based on the view width and corresponding to the rearview mirror to be adjusted and the sight direction of the target object. That is, the first target included angle may be obtained by calculating the first predetermined rear view angle based on the view width and the view direction of the target object corresponding to the rear view mirror to be adjusted, for example, the first predetermined rear view angle and the view direction of the target object may be calculated based on a plane mirror reflection imaging principle, a triangle inner angle and principle, and the like. The rearview mirror to be adjusted can be located at the target position of the rearview mirror to be adjusted after being correspondingly rotated based on the first target included angle.
According to the method and the device, the first target included angle is obtained, so that when the rearview mirror to be adjusted moves to the target position based on the first target included angle, a better view field width is provided for the target object, namely, a better view field range is provided for the target object in the image width presenting direction.
Alternatively, the field of view width in the present disclosure may refer to a field of view range in the vehicle width direction that the rearview mirror can present to the target object. The first predetermined rear view angle based on the field of view width in the present disclosure may refer to a field of view angle in a field of view range in the vehicle width direction that the rearview mirror can provide to the target object. The vehicle width direction in the present disclosure may refer to a direction of a line connecting the left side mirror and the right side mirror of the vehicle. The vehicle width direction may also be referred to as a vehicle right-left direction.
Alternatively, as shown in fig. 5, it is assumed that the XOY plane formed by the X coordinate axis and the Y coordinate axis of the first predetermined coordinate system is: a top plan view of the vehicle; the XOY plane may also be regarded as a plane parallel to the ground or the vehicle chassis, and the vehicle width direction in this disclosure is a direction parallel to the X coordinate axis direction in fig. 5.
In an alternative example, according to the present disclosure, another example of determining the target position of the rearview mirror to be adjusted according to the predetermined rearview angle and the sight line direction corresponding to the rearview mirror to be adjusted may be: and determining a second target included angle between the rearview mirror to be adjusted and the vehicle height direction according to a second preset rearview angle degree based on the visual field height and the sight direction of the target object, which correspond to the rearview mirror to be adjusted. That is, the second target included angle may be obtained by calculating the second predetermined rear-view angle based on the height of the field of view and the direction of the line of sight of the target object corresponding to the rear-view mirror to be adjusted. And the rearview mirror to be adjusted can be located at the target position of the rearview mirror to be adjusted after correspondingly rotating based on the second target included angle. The second target included angle in the present disclosure may determine the position of the rearview mirror to be adjusted in the pitch direction.
According to the method and the device, the second target included angle is obtained, so that when the rearview mirror to be adjusted moves to the target position based on the second target included angle, a better view field height is provided for the target object, namely, a better view field range is provided for the target object in the direction of the image height.
Alternatively, the height of field of view in the present disclosure may refer to a range of field of view in the vehicle height direction that the rear view mirror can present to the target object. The second predetermined rear view angle based on the view field height in the present disclosure may refer to a view field angle in a view field range in the vehicle height direction that the rear view mirror can provide to the target object. The vehicle height direction in the present disclosure may refer to a direction of a line connecting a lowest position of any one wheel of the vehicle and a highest position of the wheel. The vehicle height direction may also be referred to as the up-down direction of the vehicle.
Alternatively, as shown in fig. 6, it is assumed that the XOZ plane formed by the X coordinate axis and the Z coordinate axis of the first predetermined coordinate system is: a rear view plane of the vehicle; the XOZ plane can also be regarded as a plane parallel to a vertical plane of the ground, and the vehicle width direction in the present disclosure is a direction parallel to the Z coordinate axis direction in fig. 6.
In an alternative example, according to a first predetermined rear view angle based on the view width corresponding to the rearview mirror to be adjusted and the sight line direction of the target object, an example of determining a first target included angle between the rearview mirror to be adjusted and the vehicle width direction is as follows: and determining a first target included angle between the rearview mirror to be adjusted and the vehicle width direction according to the included angle between the first angular bisector and the vehicle length direction and the included angle between the sight of the target object and the vehicle length direction. Wherein the first angular bisector is: and an angle bisector of an included angle between the maximum view field boundary line based on the view field width and the vehicle length direction corresponding to the rearview mirror to be adjusted.
Alternatively, as shown in fig. 7, it is assumed that an angle between a line of sight of a target object and a vehicle length direction 700 (i.e., a straight line 700 is parallel to two dotted lines in fig. 7), an angle bisector (i.e., a first angle bisector) of an angle formed between a maximum view range boundary line corresponding to a mirror 701 to be adjusted based on a view range width and the vehicle length direction is assumed to be a straight line 702, an angle between a first angle bisector (i.e., the straight line 702) corresponding to the mirror 701 to be adjusted and the vehicle length direction (i.e., a direction parallel to a vehicle body, i.e., the straight line 700) is assumed to be r, an angle between the mirror 701 to be adjusted and a first target in the vehicle width direction is assumed to be m, and the straight line 700 and the straight line 702 are assumed to form a predetermined rear view angle corresponding to the.
Under the above assumption, the following formula (1) holds according to the principle of plane mirror reflection imaging:
g+r+2θ1pi formula (1)
From the principle that the sum of the internal angles of the triangle is pi, the following equations (2) and (3) hold:
g+θ1+θ2pi formula (2)
m+θ2Pi/2 formula (3)
As is apparent from the above equations (1) to (3), the following equation (4) holds:
formula (4) where m is (g-r)/2
Since g and r are both known values, the present disclosure may calculate to obtain the first target included angle m using equation (4).
Under the condition that the visual field of the rearview mirror 701 to be adjusted presented to the target object is regarded as a conical visual field, the position area formed by the angle bisector of the conical taper angle is presented to the visual field of the target object, and the visual field range is usually comfortable for the target object to observe, so that the included angle r can reflect the comfortable visual field width presented to the target object when the rearview mirror 701 to be adjusted moves to the target position.
In an alternative example, according to a second predetermined rear view angle based on the height of the field of view corresponding to the rear view mirror to be adjusted and the sight line direction of the target object, an example of determining a second target included angle between the rear view mirror to be adjusted and the height direction of the vehicle is as follows: and determining a second target included angle between the rearview mirror to be adjusted and the vehicle height direction according to the included angle between the second angular bisector and the vehicle height direction and the included angle between the sight of the target object and the vehicle height direction. The second angular bisector is: and an angle bisector of an included angle between the maximum view field boundary line based on the view field height and the vehicle height direction corresponding to the rearview mirror to be adjusted. One specific example of calculating the second target angle is similar to the example shown in fig. 7 and will not be described in detail herein.
Similarly, when the field of view presented to the target object by the to-be-adjusted rearview mirror 701 is considered to be a conical field of view, a position region formed by an angular bisector of a conical angle is presented to the field of view of the target object, which is often a comfortable field of view range observed by the target object, so that an included angle between the second angular bisector and the vehicle height direction can reflect a comfortable field of view height that can be presented to the target object when the to-be-adjusted rearview mirror 701 moves to the target position.
Exemplary devices
Fig. 8 is a schematic structural diagram of an embodiment of a service discovery apparatus according to the present disclosure. The device of the embodiment can be used for realizing the corresponding method embodiment of the disclosure. The apparatus shown in fig. 8 comprises: a detect gaze direction module 800, a determine target location module 801, and a control module 802.
The detect gaze direction module 800 is used to detect the gaze direction of the target object.
The target position determining module 801 is configured to determine a target position of the rearview mirror to be adjusted according to a predetermined rear view angle corresponding to the rearview mirror to be adjusted and the sight direction detected by the sight direction detecting module 800.
The control module 802 is configured to generate a motor control command for driving the to-be-adjusted rearview mirror to move from the current position to the target position according to the current position of the to-be-adjusted rearview mirror and the target position determined by the target position determining module 801, and output the motor control command.
Optionally, the module 800 for detecting a direction of sight may be further configured to output information prompting the driver to watch the rearview mirror to be adjusted, acquire at least one image through the vehicle-mounted camera, and then perform, by the module 800 for detecting a direction of sight, a direction of sight detection on the at least one image acquired by the module for detecting a direction of sight, so as to acquire a direction of sight of the target object.
Optionally, the module 800 for detecting a gaze direction performs gaze direction detection processing on at least one image, and an example of obtaining a gaze direction of the target object may be: the module 800 for detecting the direction of the sight line acquires a head image block and an eye image block of a target object in an image, and then the module 800 for detecting the direction of the sight line extracts the features of the head image block and the eye image block to obtain a first feature map of the head image block and a second feature map of the eye image block; then, the module 800 for detecting the direction of sight acquires the head pose of the target object according to the first feature map; finally, the module for detecting the gaze direction 800 acquires the gaze direction of the target object in the image according to the second feature map and the head pose.
Optionally, the module 800 for detecting a gaze direction may extract features of the head image block and the eye image block to obtain a first feature map of the head image block and a second feature map of the eye image block: the module 800 for detecting the direction of sight uses the head image block and the eye image block as input of a trunk unit of the neural network, performs feature extraction processing on the head image block and the eye image block respectively based on feature extraction operation of the trunk unit, and the module 800 for detecting the direction of sight obtains a first feature map of the head image block and a second feature map of the eye image block according to output of the trunk unit.
Optionally, the module for detecting gaze direction 800 may obtain, according to the first feature map, an example of the head pose of the target object by: the gaze direction detection module 800 uses the first feature map as an input of a first branch unit of the neural network, performs head posture detection processing on the first feature map based on a head posture detection operation of the first branch unit, and the gaze direction detection module 800 obtains a head posture of the target object based on an output of the first branch unit.
Optionally, the module for detecting a gaze direction 800 may acquire a gaze direction of the target object in the image according to the second feature map and the head pose, for example, by: the module for detecting a gaze direction 800 uses the second feature map and the head pose as inputs of a second branch unit of the neural network, performs gaze direction detection processing on the second feature map and the head pose based on a gaze direction detection operation of the second branch unit, and the module for detecting a gaze direction 800 may obtain a gaze direction of the target object in the image according to an output of the second branch unit.
Alternatively, an example of the module 800 for detecting the gaze direction obtaining the gaze direction of the target object in the image according to the output of the second branch unit may be: eye gaze direction detection processing is performed on the second feature map and the head pose based on the gaze direction detection operation of the second branching unit, and the eye gaze direction detection module 800 obtains the eye gaze direction of the target object in the image according to the output of the second branching unit. Wherein the gaze direction of the target object in the image comprises: the eye gaze direction of the target object in the image, or the weighted average of the gaze directions of both eyes of the target object in the image.
Optionally, in the present disclosure, the target position determining module 801 may determine the target position of the to-be-adjusted rearview mirror according to the predetermined rear view angle corresponding to the to-be-adjusted rearview mirror and the sight line direction of the target object, for example: the target position determining module 801 determines a first target included angle between the rearview mirror to be adjusted and the vehicle width direction according to a first predetermined rear view angle based on the view width and the sight direction of the target object, which correspond to the rearview mirror to be adjusted.
Optionally, another example of determining the target position of the rearview mirror to be adjusted by the target position determining module 801 according to the predetermined rear view angle corresponding to the rearview mirror to be adjusted and the sight line direction of the target object in the disclosure may be: the target position determining module 801 determines a second target included angle between the rearview mirror to be adjusted and the vehicle height direction according to a second predetermined rear view angle based on the view height corresponding to the rearview mirror to be adjusted and the sight direction.
Optionally, the target position determining module 801 may specifically determine, according to the first predetermined rear view angle based on the view width and the sight line direction of the target object, a first target included angle between the to-be-adjusted rearview mirror and the vehicle width direction, where the first predetermined rear view angle is corresponding to the to-be-adjusted rearview mirror and is based on the view width: the target position determining module 801 determines a first target included angle between the rearview mirror to be adjusted and the vehicle width direction according to an included angle between the first angular bisector and the vehicle length direction and an included angle between the sight of the target object and the vehicle length direction. Wherein the first angular bisector is: and an angle bisector of an included angle between the maximum view field boundary line based on the view field width and the vehicle length direction corresponding to the rearview mirror to be adjusted.
Optionally, the target position determining module 801 may specifically determine, according to a second predetermined rear view angle based on the height of the field of view corresponding to the to-be-adjusted rearview mirror and the sight line direction of the target object, a second target included angle between the to-be-adjusted rearview mirror and the vehicle height direction, as follows: the target position determining module 801 determines a second target included angle between the rearview mirror to be adjusted and the vehicle height direction according to the included angle between the second angular bisector and the vehicle height direction and the included angle between the sight of the target object and the vehicle height direction. The second angular bisector is: and an angle bisector of an included angle between the maximum view field boundary line based on the view field height and the vehicle height direction corresponding to the rearview mirror to be adjusted.
Exemplary electronic device
An electronic device according to an embodiment of the present disclosure is described below with reference to fig. 9. FIG. 9 shows a block diagram of an electronic device in accordance with an embodiment of the disclosure. As shown in fig. 9, the electronic device 91 includes one or more processors 911 and memory 912.
The processor 911 may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device 91 to perform desired functions.
Memory 912 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory, for example, may include: random Access Memory (RAM) and/or cache memory (cache), etc. The nonvolatile memory, for example, may include: read Only Memory (ROM), hard disk, flash memory, and the like. One or more computer program instructions may be stored on the computer-readable storage medium and executed by processor 911 to implement the above-described method of adjusting a rearview mirror of the various embodiments of the present disclosure and/or other desired functions. Various contents such as an input signal, a signal component, a noise component, etc. may also be stored in the computer-readable storage medium.
In one example, the electronic device 91 may further include: an input device 913, and an output device 914, among others, which are interconnected by a bus system and/or other form of connection mechanism (not shown). The input device 913 may include, for example, a keyboard, a mouse, or the like. The output device 914 may output various information to the outside. The output devices 914 can include, for example, a display, speakers, a printer, and a communication network and remote output devices connected thereto, among others.
Of course, for simplicity, only some of the components of the electronic device 91 relevant to the present disclosure are shown in fig. 9, omitting components such as buses, input/output interfaces, and the like. In addition, the electronic device 91 may include any other suitable components depending on the particular application.
Exemplary computer program product and computer-readable storage Medium
In addition to the above-described methods and apparatus, embodiments of the present disclosure may also be a computer program product comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in the method of adjusting a rearview mirror according to various embodiments of the present disclosure described in the "exemplary methods" section of this specification above.
The computer program product may write program code for carrying out operations for embodiments of the present disclosure in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server.
Furthermore, embodiments of the present disclosure may also be a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, cause the processor to perform steps in a method of adjusting a rearview mirror according to various embodiments of the present disclosure described in the "exemplary methods" section above in this specification.
The computer-readable storage medium may take any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium may include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The foregoing describes the general principles of the present disclosure in conjunction with specific embodiments, however, it is noted that the advantages, effects, etc. mentioned in the present disclosure are merely examples and are not limiting, and they should not be considered essential to the various embodiments of the present disclosure. Furthermore, the foregoing disclosure of specific details is for the purpose of illustration and description and is not intended to be limiting, since the disclosure is not intended to be limited to the specific details so described.
In the present specification, the embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts in the embodiments are referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
The block diagrams of devices, apparatuses, systems referred to in this disclosure are only given as illustrative examples and are not intended to require or imply that the connections, arrangements, configurations, etc. must be made in the manner shown in the block diagrams. These devices, apparatuses, devices, and systems may be connected, arranged, configured in any manner, as will be appreciated by those skilled in the art. Words such as "including," comprising, "having," and the like are open-ended words that mean "including, but not limited to," and are used interchangeably therewith. The words "or" and "as used herein mean, and are used interchangeably with, the word" and/or, "unless the context clearly dictates otherwise. The word "such as" is used herein to mean, and is used interchangeably with, the phrase "such as but not limited to".
The methods and apparatus of the present disclosure may be implemented in a number of ways. For example, the methods and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order for the steps of the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above unless specifically stated otherwise. Further, in some embodiments, the present disclosure may also be embodied as programs recorded in a recording medium, the programs including machine-readable instructions for implementing the methods according to the present disclosure. Thus, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.
It is also noted that in the devices, apparatuses, and methods of the present disclosure, each component or step can be decomposed and/or recombined. These decompositions and/or recombinations are to be considered equivalents of the present disclosure.
The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects, and the like, will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the disclosure. Thus, the present disclosure is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
The foregoing description has been presented for purposes of illustration and description. Furthermore, the description is not intended to limit embodiments of the disclosure to the form disclosed herein. While a number of example aspects and embodiments have been discussed above, those of skill in the art will recognize certain variations, modifications, alterations, additions and sub-combinations thereof.

Claims (10)

1. A method of adjusting a rearview mirror, comprising:
detecting a gaze direction of a target object;
determining the target position of the rearview mirror to be adjusted according to the preset rear view angle corresponding to the rearview mirror to be adjusted and the sight direction;
and generating a motor control command for driving the rearview mirror to be adjusted to move from the current position to the target position according to the current position of the rearview mirror to be adjusted and the target position, and outputting the motor control command.
2. The method of claim 1, wherein the detecting a gaze direction of a target object comprises:
outputting information prompting a driver to watch a rearview mirror to be adjusted, and acquiring at least one image through a vehicle-mounted camera device;
and performing sight direction detection processing on the at least one image to obtain the sight direction of the target object.
3. The method according to claim 2, wherein the performing line-of-sight direction detection processing on the at least one image to obtain the line-of-sight direction of the target object comprises:
acquiring a head image block and an eye image block of a target object in an image;
extracting the features of the head image block and the eye image block to obtain a first feature map of the head image block and a second feature map of the eye image block;
acquiring the head posture of the target object according to the first feature map;
and acquiring the sight line direction of the target object in the image according to the second feature map and the head posture.
4. The method according to claim 3, wherein the extracting features of the head image block and the eye image block to obtain a first feature map of the head image block and a second feature map of the eye image block comprises:
respectively taking the head image blocks and the eye image blocks as input of a trunk unit of a neural network, and respectively performing feature extraction processing on the head image blocks and the eye image blocks based on feature extraction operation of the trunk unit to obtain a first feature map of the head image blocks and a second feature map of the eye image blocks;
the acquiring the head pose of the target object according to the first feature map comprises:
taking the first feature map as an input of a first branch unit of the neural network, and performing head posture detection processing on the first feature map based on head posture detection operation of the first branch unit to obtain a head posture of the target object;
the acquiring the sight line direction of the target object in the image according to the second feature map and the head pose comprises:
and taking the second feature map and the head pose as the input of a second branch unit of the neural network, and performing line-of-sight direction detection processing on the second feature map and the head pose based on line-of-sight direction detection operation of the second branch unit to obtain the line-of-sight direction of the target object in the image.
5. The method according to claim 4, wherein the performing line-of-sight direction detection processing on the second feature map and the head pose based on a line-of-sight direction detection operation of the second branch unit to obtain a line-of-sight direction of a target object in the image comprises:
performing eyeball sight direction detection processing on the second feature map and the head posture based on sight direction detection operation of the second branch unit to obtain eyeball sight direction of a target object in the image;
the gaze direction of the target object in the image comprises: an eyeball sight direction of a target object in the image, or a weighted average of sight directions of both eyes of the target object in the image.
6. The method according to any one of claims 1 to 5, wherein the determining the target position of the rearview mirror to be adjusted according to the predetermined rear view angle corresponding to the rearview mirror to be adjusted and the sight line direction comprises:
determining a first target included angle between the rearview mirror to be adjusted and the width direction of the vehicle according to a first preset rear view angle based on the view width and corresponding to the rearview mirror to be adjusted and the sight direction; and/or
And determining a second target included angle between the rearview mirror to be adjusted and the vehicle height direction according to a second preset rear view angle based on the view height and corresponding to the rearview mirror to be adjusted and the sight direction.
7. The method according to claim 6, wherein the determining a first target included angle between the rearview mirror to be adjusted and the vehicle width direction according to the first predetermined rear view angle based on the view width and the sight line direction corresponding to the rearview mirror to be adjusted comprises:
determining a first target included angle between the rearview mirror to be adjusted and the width direction of the vehicle according to an included angle between a first angular bisector and the length direction of the vehicle and an included angle between the sight line and the length direction of the vehicle; wherein the first angular bisector is: the rearview mirror to be adjusted is an angular bisector of an included angle between a maximum view boundary line based on the view width and the length direction of the vehicle;
the determining a second target included angle between the rearview mirror to be adjusted and the vehicle height direction according to a second preset rear view angle based on the view height and the sight direction corresponding to the rearview mirror to be adjusted comprises:
determining a second target included angle between the rearview mirror to be adjusted and the height direction of the vehicle according to an included angle between a second angular bisector and the height direction of the vehicle and an included angle between the sight line and the height direction of the vehicle; wherein the second angular bisector is: and the rearview mirror to be adjusted is an angular bisector of an included angle between the maximum view boundary line based on the view height and the vehicle height direction.
8. An apparatus for adjusting a rearview mirror, comprising:
the sight direction detection module is used for detecting the sight direction of the target object;
the target position determining module is used for determining the target position of the rearview mirror to be adjusted according to the preset rear view angle corresponding to the rearview mirror to be adjusted and the sight direction detected by the sight direction detecting module;
and the control module is used for generating a motor control command for driving the rearview mirror to be adjusted to move from the current position to the target position according to the current position of the rearview mirror to be adjusted and the target position determined by the target position determining module, and outputting the motor control command.
9. A computer-readable storage medium, the storage medium storing a computer program for performing the method of any of the preceding claims 1-7.
10. An electronic device, the electronic device comprising:
a processor;
a memory for storing the processor-executable instructions;
the processor is configured to read the executable instructions from the memory and execute the instructions to implement the method of any one of claims 1-7.
CN202010770978.1A 2020-08-04 2020-08-04 Method, device, medium, and electronic apparatus for adjusting rearview mirror Active CN111873911B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010770978.1A CN111873911B (en) 2020-08-04 2020-08-04 Method, device, medium, and electronic apparatus for adjusting rearview mirror

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010770978.1A CN111873911B (en) 2020-08-04 2020-08-04 Method, device, medium, and electronic apparatus for adjusting rearview mirror

Publications (2)

Publication Number Publication Date
CN111873911A true CN111873911A (en) 2020-11-03
CN111873911B CN111873911B (en) 2022-06-07

Family

ID=73211689

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010770978.1A Active CN111873911B (en) 2020-08-04 2020-08-04 Method, device, medium, and electronic apparatus for adjusting rearview mirror

Country Status (1)

Country Link
CN (1) CN111873911B (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112270304A (en) * 2020-11-17 2021-01-26 北京百度网讯科技有限公司 Data processing method, device, equipment and storage medium
CN112622765A (en) * 2020-12-28 2021-04-09 江苏合泰飞梵科技有限公司 Full-time vision auxiliary rearview mirror system
CN112714720A (en) * 2020-12-16 2021-04-27 华为技术有限公司 Rearview mirror control method and related equipment
CN114132259A (en) * 2021-12-10 2022-03-04 东南大学 Automobile exterior rearview mirror adjusting method and device and automobile
CN114212026A (en) * 2021-12-27 2022-03-22 东软睿驰汽车技术（沈阳）有限公司 Vehicle rearview mirror adjusting method and device, vehicle and storage medium
CN115158164A (en) * 2022-06-27 2022-10-11 重庆长安汽车股份有限公司 Method and device for controlling vehicle rearview mirror, vehicle and storage medium
DE102022106797B3 (en) 2022-03-23 2023-04-27 Dr. Ing. H.C. F. Porsche Aktiengesellschaft Method for automatically adjusting at least one rear-view mirror of a motor vehicle

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102582523A (en) * 2012-03-09 2012-07-18 深圳市领华卫通数码科技有限公司 Interior rearview mirror with voice recognition function and voice recognition method
CN205131100U (en) * 2015-11-27 2016-04-06 厦门智洋电子科技有限公司 Pronunciation remote control automobile rearview mirror
CN106945607A (en) * 2017-03-30 2017-07-14 京东方科技集团股份有限公司 The control method and device of vehicle
CN108515912A (en) * 2018-08-06 2018-09-11 江苏中科朗恩斯车辆科技有限公司 Vehicle, vehicle mirrors and its mirror driver and minute surface driving method
CN109017642A (en) * 2017-06-09 2018-12-18 意法半导体(大西部)公司 The method and system of the adjust automatically of at least one equipment for motor vehicles
CN111027506A (en) * 2019-12-19 2020-04-17 斑马网络技术有限公司 Method and device for determining sight direction, electronic equipment and storage medium

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102582523A (en) * 2012-03-09 2012-07-18 深圳市领华卫通数码科技有限公司 Interior rearview mirror with voice recognition function and voice recognition method
CN205131100U (en) * 2015-11-27 2016-04-06 厦门智洋电子科技有限公司 Pronunciation remote control automobile rearview mirror
CN106945607A (en) * 2017-03-30 2017-07-14 京东方科技集团股份有限公司 The control method and device of vehicle
CN109017642A (en) * 2017-06-09 2018-12-18 意法半导体(大西部)公司 The method and system of the adjust automatically of at least one equipment for motor vehicles
CN108515912A (en) * 2018-08-06 2018-09-11 江苏中科朗恩斯车辆科技有限公司 Vehicle, vehicle mirrors and its mirror driver and minute surface driving method
CN111027506A (en) * 2019-12-19 2020-04-17 斑马网络技术有限公司 Method and device for determining sight direction, electronic equipment and storage medium

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112270304A (en) * 2020-11-17 2021-01-26 北京百度网讯科技有限公司 Data processing method, device, equipment and storage medium
CN112714720A (en) * 2020-12-16 2021-04-27 华为技术有限公司 Rearview mirror control method and related equipment
WO2022126419A1 (en) * 2020-12-16 2022-06-23 华为技术有限公司 Rear-view mirror control method and related device
CN112714720B (en) * 2020-12-16 2022-09-16 华为技术有限公司 Rearview mirror control method and related equipment
CN112622765A (en) * 2020-12-28 2021-04-09 江苏合泰飞梵科技有限公司 Full-time vision auxiliary rearview mirror system
CN112622765B (en) * 2020-12-28 2021-10-26 江苏合泰飞梵科技有限公司 Full-time vision auxiliary rearview mirror system
CN114132259A (en) * 2021-12-10 2022-03-04 东南大学 Automobile exterior rearview mirror adjusting method and device and automobile
CN114212026A (en) * 2021-12-27 2022-03-22 东软睿驰汽车技术（沈阳）有限公司 Vehicle rearview mirror adjusting method and device, vehicle and storage medium
CN114212026B (en) * 2021-12-27 2024-01-23 东软睿驰汽车技术（沈阳）有限公司 Vehicle rearview mirror adjusting method and device, vehicle and storage medium
DE102022106797B3 (en) 2022-03-23 2023-04-27 Dr. Ing. H.C. F. Porsche Aktiengesellschaft Method for automatically adjusting at least one rear-view mirror of a motor vehicle
CN115158164A (en) * 2022-06-27 2022-10-11 重庆长安汽车股份有限公司 Method and device for controlling vehicle rearview mirror, vehicle and storage medium

Also Published As

Publication number Publication date
CN111873911B (en) 2022-06-07

Similar Documents

Publication Publication Date Title
CN111873911B (en) Method, device, medium, and electronic apparatus for adjusting rearview mirror
CN105916730B (en) Vehicle mirror is adjusted
JP5093611B2 (en) Vehicle periphery confirmation device
US20140002357A1 (en) Enabling and Disabling Features of a Headset Computer Based on Real-Time Image Analysis
US9606623B2 (en) Gaze detecting apparatus and method
EP2860063B1 (en) Method and apparatus for acquiring image for vehicle
JP5092776B2 (en) Gaze direction detection device and gaze direction detection method
JP2010127779A (en) Information serving apparatus, information serving method and program
CN109968979A (en) Vehicle-mounted projection processing method, device, mobile unit and storage medium
WO2024002303A1 (en) Robotic arm control method and apparatus for vehicle-mounted screen, device, and vehicle
JP7138175B2 (en) Method of operating head-mounted electronic display device for displaying virtual content and display system for displaying virtual content
WO2019187283A1 (en) Image processing device, image display system, and image processing method
CN111252074A (en) Multi-modal control method, device, computer-readable storage medium and vehicle
US20190171296A1 (en) Gesture determination apparatus and program
JP7296374B2 (en) Image processing device, mobile device, method, and program
JP2017091013A (en) Driving support device
CN112083795A (en) Object control method and device, storage medium and electronic equipment
US11276378B2 (en) Vehicle operation system and computer readable non-transitory storage medium
CN113840757A (en) Display screen adjusting method and device
CN112052770A (en) Method, apparatus, medium, and electronic device for fatigue detection
US20200282909A1 (en) Vehicle imaging system and method for a parking solution
CN112172671B (en) Method and device for displaying rear view image of commercial vehicle, computer equipment and storage medium
CN115690251A (en) Method and device for displaying image on front windshield of vehicle
CN114979615A (en) Method and device for displaying picture on head-mounted display device and electronic device
CN111985384A (en) Method and device for acquiring 3D coordinates of face key points and 3D face model

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant