WO2019119290A1 - 提示信息确定方法、装置、电子设备和计算机程序产品 - Google Patents

提示信息确定方法、装置、电子设备和计算机程序产品 Download PDF

Info

Publication number
WO2019119290A1
WO2019119290A1 PCT/CN2017/117424 CN2017117424W WO2019119290A1 WO 2019119290 A1 WO2019119290 A1 WO 2019119290A1 CN 2017117424 W CN2017117424 W CN 2017117424W WO 2019119290 A1 WO2019119290 A1 WO 2019119290A1
Authority
WO
WIPO (PCT)
Prior art keywords
user
image
prompt information
image area
environment
Prior art date
Application number
PCT/CN2017/117424
Other languages
English (en)
French (fr)
Inventor
王恺
廉士国
Original Assignee
深圳前海达闼云端智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳前海达闼云端智能科技有限公司 filed Critical 深圳前海达闼云端智能科技有限公司
Priority to PCT/CN2017/117424 priority Critical patent/WO2019119290A1/zh
Priority to CN201780002906.9A priority patent/CN108140124B/zh
Publication of WO2019119290A1 publication Critical patent/WO2019119290A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/35—Categorising the entire scene, e.g. birthday party or wedding scene
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
    • A61H3/00—Appliances for aiding patients or disabled persons to walk about
    • A61H3/06—Walking aids for blind persons

Definitions

  • the present application relates to the field of guiding blind technology, and in particular, to a method, device, electronic device and computer program product for determining prompt information.
  • Some wearable glasses devices can take images of the surrounding environment of the user, identify a specific kind of object or obstacle, and prompt the user to display the information of the object or the obstacle by means of voice broadcast.
  • the embodiment of the present application provides a method, a device, a device, and a computer program product for determining prompt information, which are mainly used to assist a user to obtain environmental information that is truly concerned by the user in an accurate and timely manner.
  • the embodiment of the present application provides a method for determining prompt information, which includes: acquiring an image region division result of an environment image of a user and prompt information corresponding to each image region; determining the user The selected image area; determining prompt information corresponding to the image area selected by the user.
  • the embodiment of the present application provides a prompt information determining apparatus, where the apparatus includes: an image acquiring module, configured to acquire an image of an environment image of a user The area division result and the prompt information corresponding to each image area; the selection determining module is configured to determine the image area selected by the user; and the prompt information determining module is configured to determine the prompt information corresponding to the image area selected by the user.
  • an embodiment of the present application provides an electronic device, including: a communication device, a memory, one or more processors; and one or more modules, the one or more The modules are stored in the memory and are configured to be executed by the one or more processors, the one or more modules including instructions for performing the various steps of the above methods.
  • embodiments of the present application provide a computer program product for use in conjunction with an electronic device, the computer program product comprising a computer program embedded in a computer readable storage medium, the computer program comprising An instruction to cause the electronic device to perform the various steps in the above methods.
  • the environment image is divided into multiple regions and the prompt information corresponding to the image region selected by the user is played, and the user's active information selection can promptly and accurately determine the prompt information of the most concerned regional environment, so as to give the user Better tips.
  • FIG. 1 is a schematic flowchart diagram of a method for determining prompt information in Embodiment 1 of the present application
  • FIG. 2 is a schematic flowchart diagram of a method for determining prompt information in Embodiment 2 of the present application
  • FIG. 3 is a schematic diagram of a third implementation scenario of the third embodiment of the present application.
  • FIG. 4 is a schematic diagram of a fourth implementation scenario of the embodiment of the present application.
  • FIG. 5 is a schematic structural diagram of a prompt information determining apparatus in Embodiment 6 of the present application.
  • FIG. 6 is a schematic structural diagram of an electronic device in Embodiment 7 of the present application.
  • the present application provides a method for determining prompt information. After obtaining an image region segmentation result of an environment image of a user and prompt information corresponding to each image region, the corresponding prompt information is determined according to the image region selected by the user.
  • the environment image is divided into multiple regions and the prompt information corresponding to the image region selected by the user is played, and the user's active information selection can promptly and accurately determine the prompt information of the most concerned regional environment, so as to give the user Better tips.
  • Embodiment 1 is a diagrammatic representation of Embodiment 1:
  • the blinding system includes one or more portable devices of the user and a cloud server having a communication connection with the guiding devices.
  • FIG. 1 is a schematic flowchart of a method for determining prompt information in the first embodiment of the present application. As shown in FIG. 1 , the method for determining prompt information includes:
  • Step 101 Collect an environment image of the user
  • Step 102 Send the environment image of the user to the cloud server; the cloud server performs semantic segmentation on the environment image, divides the environment image into at least one image region, and determines prompt information corresponding to each image region;
  • Step 103 Obtain image segmentation result of the environment image of the user where the cloud server is sent, and prompt information corresponding to each image region;
  • Step 104 Determine an image area selected by the user
  • Step 105 Determine prompt information corresponding to the image area selected by the user.
  • an image of the environment in which the user is located is collected, and the environment image of the user is collected by the user's portable device, and the user portable device may be a blind guide helmet, a guide blind glasses, a guide vest, and a guide cane.
  • the user portable device may be a blind guide helmet, a guide blind glasses, a guide vest, and a guide cane.
  • Such as a variety of wearable devices it can also be mobile phones, tablets and other devices that users usually carry with them.
  • the image of the environment in which the user is collected by the user's portable device may be a photo, and the photo may include: a photo in front of the user, a photo of the user's portable device such as a guide helmet or a guide lens, or a panoramic view of the user's direction.
  • the photo for example, a photo synthesized by a 180° image directly in front of the user, or a photo synthesized by 360° images around the user; the image of the environment in which the user is collected by the user's portable device may also be a screenshot of the video corresponding to the aforementioned photo collection mode.
  • the environment image of the user is collected when the environment image of the user, or the location of the user, or the change of one or more of the user gestures exceeds a preset threshold.
  • the change of the environment image of the user exceeds the preset threshold, that is, when the user's portable device captures the change of the environment image is too large, it is determined that the environment image needs to be re-identified and the user is prompted to information, and the environment where the user is located is collected. image.
  • the user portable device can capture and recognize the video of the environment in which the user is located.
  • the video clip of a certain frame is found to be 30% smaller than the pixel image of the last acquired environment image, the frame image is collected as the environment image of the current user.
  • the user portable device determines that the distance moved by the user exceeds a preset threshold according to the positioning device, it is determined that the environment image needs to be re-recognized and the user is prompted with the information.
  • Collect images of the environment in which the user is located For example, when the guide helmet worn by the user determines that the current user's position changes by 5 m compared with the last time the environment image was collected according to the GPS positioning device, the environment image of the current user is collected.
  • the user's portable device determines that the posture change of the user exceeds the preset threshold according to the posture sensing device, it is determined that the environment image needs to be re-recognized and the user is prompted to collect information.
  • the user portable device sends the environment image of the user to the cloud server, and the user portable device and the cloud server can be connected through various wireless network forms, and real-time data transmission can be realized.
  • the cloud server After receiving the environment image sent by the user's portable device, the cloud server performs semantic segmentation on the environment image, divides the environment image into at least one image region, and determines prompt information corresponding to each image region.
  • the semantic segmentation here can be implemented according to various algorithms.
  • the prompt information corresponding to each image region may directly be a semantic vocabulary corresponding to the semantic segmentation of the image region; or may be a prompt information for assisting the blind person according to the semantic matching of the image region, for example, “may advance” and “ Need to avoid” or “dangerous, please be careful”, etc.; may also be information for further image recognition acquisition of the image area, such as a car or bus that may be a certain color divided into the "vehicle” image area,
  • the corresponding prompt information may be “red vehicle” or “bus”, and the image area classified as “person” may be further identified to obtain a “person of red clothes and black pants” or a “ woman wearing a hat”. information.
  • the cloud server feeds back the image region division result of the environment image of the user and the prompt information corresponding to each image region to the portable device of the user.
  • the user portable device in the feedback may be the device for collecting the image of the environment in which the user is located in step 101, or may be other supporting portable devices of the user, that is, the guide helmet, the guide glasses, and the guide blind.
  • Various wearable devices such as vests, guide canes, and bracelets, or devices that users usually carry with them, such as mobile phones and tablet computers.
  • the user's guide helmet collects the image of the front of the user and sends it to the cloud server. After the cloud server finishes processing, it can send back to the guide helmet, or can send back to the mobile phone terminal of the user corresponding to the guide helmet.
  • step 103 the portable device of the user acquires an image region division result of the environment image of the user where the cloud server is sent, and prompt information corresponding to each image region.
  • the method further includes: when determining that the image region division result of the environment image meets a preset condition, notifying the user; One or more of the following: the image area division result is compared with the previous image area division result, a new image area appears, and the area of the new image area is greater than a preset threshold; or the image The area division result is compared with the previous image area division result, and the area growth of the original image area is greater than a preset threshold.
  • the user's portable device obtains the image region division result of the environment image of the user's environment sent by the cloud server, and can compare with the segmentation result sent by the last cloud server.
  • the segmentation result of the image region changes greatly, the user needs to be notified to additionally note. Specifically, when one or more of the following conditions occur, the determination of the division result meets the preset condition:
  • the user is notified at this time, and the content of the notification includes one or more of the following:
  • the notification form may be a vibration of the user's portable device or an alarm sound.
  • the notification form may be a voice notification or a vibration prompt of the user's portable device, such as voice playback. "Please pay attention to the right front", or the vibration warning on the right side of the guide helmet worn by the user. In combination with the subsequent steps, it is up to the user to decide whether to actively select the area and obtain the prompt information of the area.
  • the prompt information corresponding to the new large-area image area is directly played for the user, or the prompt information of the original image area with a large area change occurs, and the prompt information can be directly played by the current user portable device, or Sending the notification information of the notification to the other portable device of the user for playing, for example, the user guides the helmet voice play “Please pay attention to the building” or sends the voice information “Please pay attention to the red vehicle” to the user's earphone for playing.
  • the user's portable device determines the image area selected by the user, and determines The method of selecting the image area by the user may be receiving an instruction of the user on the touch panel or the touch screen, or receiving an action instruction of the user in a specific operation space, or receiving a voice instruction of the user.
  • the manner in which the user's portable device determines the image area selected by the user is: acquiring an operation position of the user on the touch panel or the touch screen; determining according to the position mapping relationship between the image areas and the touch panel or the touch screen.
  • the image area corresponding to the user operation position is an image area selected by the user.
  • the touch panel or the touch screen is usually installed in the user's portable device, and each position of the touch panel or the touch screen has a mapping relationship with the environment image, so that the location may correspond to the environment according to a touch operation of the user at a touch panel or a touch screen.
  • the position of the image, and the image area corresponding to the operation position is determined, thereby obtaining an image area selected by the user.
  • the touch operation may be a click, a long touch or a slide, or the like.
  • the manner in which the user's portable device determines the image area selected by the user is: acquiring gesture information of the user in the operation space; determining, according to a mapping relationship between each image area and the operation space, The image area corresponding to the gesture information in the user operation space is an image area selected by the user.
  • the user's portable device is installed with a camera capable of detecting a user's gesture image in a specific operation space.
  • the user's portable device obtains the image region division result of the environment image, the image region is mapped with the interaction space supported by the gesture detection camera.
  • a specific gesture appears in a spatial position corresponding to an image area, it is determined that the image area corresponding to the position is an image area selected by the user.
  • the camera that detects the user's gesture is usually independent of the camera that captures the image of the environment in which the user is located, wherein the camera that detects the user's gesture can be mounted on the user's headgear such as a blinded helmet or a guide blind with a large downtilt angle.
  • the user In order to make the gesture complete the interaction, the user only needs to raise the hand slightly and complete the gesture in the air without raising the hand to the front, which can alleviate the fatigue of the user's long-handed operation, and the other
  • the aspect can prevent the gesture image from being collected into the environment image of the user, and affect the semantic segmentation result of the environment image.
  • step 105 determining prompt information corresponding to the image area selected by the user, that is, determining, according to the selected image area of the user, the prompt information corresponding to the selected image area is the user's most Messages of interest.
  • the prompt information is further played.
  • the user's portable device determines the corresponding prompt information according to the image area selected by the user, the user can directly play the current portable device, and the playing mode can be voice.
  • the prompt information is further sent to the other portable device of the user, so that the user's portable device plays the prompt information.
  • the user's portable device determines its corresponding prompt information according to the image area selected by the user, it can be sent to other portable devices of the same user for playing, and the other portable device can be the user's wearing device, such as a helmet, glasses or earphone. It is closer to the user's ear and enables the user to hear the prompt information more clearly.
  • the environment image is divided into multiple regions and the prompt information corresponding to the image region selected by the user is played, and the user's active information selection can promptly and accurately determine the prompt information of the most concerned regional environment, so as to give Better user prompts;
  • the processing of the environment image is done in the cloud server, the image processing speed is faster and more accurate, and is not limited by the image processing capability of the user's portable device; can be in the environment image, location or user posture of the user
  • the environment image is collected and processed in time, which avoids a large amount of image data processing, and can also prompt the user to update the prompt information corresponding to the environment image; and can prompt the user to respond to the user when the environment changes suddenly and the image area changes greatly.
  • the user can complete the corresponding operation through various forms such as touchpad, touch screen or space gesture, which is in line with the user's habits; the prompt information can be played by the current portable device, or can be completed by other portable devices that are more convenient for the user, and the user experience is better. .
  • Embodiment 2 is a diagrammatic representation of Embodiment 1:
  • This embodiment is implemented in a blind guide system, and the facing user is usually a visually impaired person.
  • the guiding system includes one or more portable devices of the user, and a communication connection is established between the guiding devices.
  • FIG. 2 is a schematic flowchart of a method for determining prompt information in Embodiment 2 of the present application. As shown in FIG. 2, the method for determining prompt information includes:
  • Step 201 Collect an environment image of the user
  • Step 202 Perform semantic segmentation on the environment image, and divide the environment image into at least An image area, determining prompt information corresponding to each image area;
  • Step 203 Acquire an image region division result of the environment image of the user and prompt information corresponding to each image region.
  • Step 204 Determine an image area selected by the user
  • Step 205 Determine prompt information corresponding to the image area selected by the user.
  • the user's portable device collects an image of the environment in which the user is located.
  • the environment image of the user is collected when the environment image of the user, or the location of the user, or the change of one or more of the user gestures exceeds a preset threshold.
  • the portable device locally performs semantic segmentation on the environment image, divides the environment image into at least one image region, and determines prompt information corresponding to each image region.
  • the semantic segmentation may be implemented according to various algorithms, and the prompt information corresponding to each image region may directly correspond to the semantic vocabulary when the image region is semantically segmented, or may be an auxiliary blind person obtained according to the semantic matching of the image region. Prompt information, such as “can go forward”, “need to avoid” or “dangerous, please be careful”.
  • step 203 the portable device acquires the image region division result of the environment image of the user and the prompt information corresponding to each image region after the local processing is completed.
  • the method further includes: when determining that the image region division result of the environment image meets a preset condition, notifying the user; the preset condition includes one or more of the following: The image area division result is compared with the previous image area division result, a new image area appears, and the area of the new image area is greater than a preset threshold; or the image area division result and the previous image area Compared with the division result, the area growth of an original image area is greater than a preset threshold.
  • the real-time of this step can be explained by referring to the steps after step 103 in the first embodiment.
  • step 204 the user's portable device determines the image area selected by the user, and the method of determining the image area selected by the user may be receiving an instruction of the user on the touchpad or the touch screen, or receiving an action instruction of the user in a specific operation space. , or receive the user's voice instructions and so on.
  • the manner in which the user's portable device determines the image area selected by the user is: acquiring an operation position of the user on the touch panel or the touch screen; determining according to the position mapping relationship between the image areas and the touch panel or the touch screen.
  • the image area corresponding to the user operation position is an image area selected by the user.
  • the manner in which the user's portable device determines the image area selected by the user is: acquiring gesture information of the user in the operation space; determining, according to a mapping relationship between each image area and the operation space, The image area corresponding to the gesture information in the user operation space is an image area selected by the user.
  • step 205 the user portable device determines the prompt information corresponding to the image area selected by the user, that is, determines the prompt information corresponding to the selected image area according to the selected image area of the user as the prompt information that the user is most concerned about.
  • the method further includes: playing the prompt information.
  • the user portable device performs semantic segmentation on the environment image to obtain a plurality of image regions, and after determining the corresponding prompt information according to the image region selected by the user, the device can be directly played on the current portable device, and the playing mode can be voice.
  • the method further includes: sending the prompt information to the other portable device of the user, so that the user's portable device plays the prompt information.
  • the user portable device performs semantic segmentation on the environment image to obtain a plurality of image regions, and after determining the corresponding prompt information according to the image region selected by the user, the prompt information may be sent to other portable devices of the same user for playing, the other portable device
  • the device can be a user's head-mounted device, such as a helmet, glasses, or earphone, which is closer to the user's ear, enabling the user to more clearly hear the prompt message being played.
  • the environment image is divided into multiple regions and the prompt information corresponding to the image region selected by the user is played, and the user's active information selection can promptly and accurately determine the prompt information of the most concerned regional environment, so as to give
  • the user has better prompts; the processing of the environment image is done locally in the user's portable device, and is not restricted by the communication function of the user's portable device, and is not affected by the data transmission delay; can be in the environment image, location or user posture of the user
  • the environment image is collected and processed in time, which avoids a large amount of image data processing, and can also prompt the user to update the prompt information corresponding to the environment image; the user can complete the corresponding operation through various forms such as a touch panel, a touch screen or a space gesture. Meet the user's habits; the prompt information can be played It can be completed by the current portable device, or it can be completed by other portable devices that are more convenient for users, and the user experience is better.
  • Embodiment 3 is a diagrammatic representation of Embodiment 3
  • the guide blind system includes at least a user's guide glasses, a smart phone, and a cloud server, and a communication connection relationship is established between the three.
  • FIG. 3 is a schematic diagram of a third implementation scenario of the third embodiment of the present application.
  • the user's guide glasses periodically collect an environment image of the user's head in a direction, and upload the cloud server in real time; the cloud server guides the user in real time.
  • the environment image uploaded by the glasses is semantically segmented, for example, four regions of "sky”, “architecture”, “road” and “vehicle” are obtained, and the semantics are used as hint information of each region; the cloud server divides the region of the environment image
  • the result and the prompt information corresponding to each area are sent to the smart phone of the user of the guide glasses; after receiving the scene segmentation map sent by the cloud, the user's smart phone can display the image area segmentation result on the touch screen; the user clicks After sliding or touching different areas of the screen, the smart phone plays the prompt information corresponding to the image area, or sends the prompt information to the user's guide glasses, and the glasses are played, for example, the user touches the bottom of the mobile phone screen, and the guide glasses are The prompt information "road” corresponding to the image area is played by voice.
  • Embodiment 4 is a diagrammatic representation of Embodiment 4:
  • the guide blind system includes at least a user's guide helmet and a cloud server, and a communication connection relationship is established therebetween.
  • the guide blind helmet In addition to the environmental image capture camera with the horizontal front facing the front of the head, the guide blind helmet also has a gesture detection camera that faces the horizontal front angle of 60° toward the lower front of the user, which supports gesture recognition in a specific operation space in front of the user.
  • FIG. 4 is a schematic diagram showing the implementation of the fourth embodiment of the present application.
  • the user's guide helmet monitors the user's head and turns the camera to the right, and collects the environment image of the user's head in the direction of the head.
  • the server performs semantic segmentation on the environment image uploaded by the user's guide blind helmet in real time, for example, splitting into four regions of “sky”, “architecture”, “road” and “vehicle”, and the semantic vocabulary and the corresponding pair of users
  • the auxiliary information is used as the prompt information of each area
  • the cloud server sends the area segmentation result of the environment image and the prompt information corresponding to each area back to the location
  • the blind helmet is obtained; after the guide helmet obtains the region segmentation result of the environment image, the image is mapped with the operation space supported by the gesture detection camera, that is, each position of the gesture operation space corresponds to the environment image in which the image region is divided. One location.
  • the gesture detection camera detects the position of the user's finger and maps to the environment image in which the image area is divided, and the guide helmet broadcasts the prompt information corresponding to the image area pointed by the user through the voice.
  • the user lifts his finger to the lower right of the operation space, and the guide blind helmet plays the voice "vehicle, please avoid”.
  • Embodiment 5 is a diagrammatic representation of Embodiment 5:
  • the embodiment is implemented by a multifunctional guide vest
  • the guide vest has an environment image camera pointing to the front of the user, and is used for collecting an environment image of the user, and is mounted on the lower side of the guide vest near the user's hand.
  • the guide vest detects that the environment image in front of the change is too large, the current image is captured, and the environment image is semantically segmented and its semantics is used as prompt information of each region; the segmentation result and touch of the environment image are The area of the board establishes a mapping relationship.
  • the guide vest After detecting the touch operation of the user on the touch panel, determining the divided image area corresponding to the position where the user touches the touch panel to determine the image area selected by the user; the guide vest is a user voice playing user.
  • the prompt information corresponding to the selected image area.
  • the present application further provides a prompt information determining apparatus. Since the principle of solving the problem is similar to the prompt information determining method, the implementation of the apparatus can refer to the implementation of the method, and the repetition is no longer Narration. As shown in FIG. 5, the prompt information determining apparatus 500 includes:
  • the image acquisition module 501 is configured to acquire an image region division result of the environment image where the user is located and prompt information corresponding to each image region;
  • a selection determining module 502 configured to determine an image area selected by the user
  • the prompt information determining module 503 is configured to determine prompt information corresponding to the image area selected by the user.
  • the apparatus 500 further includes:
  • the image acquisition module 504 is configured to collect an image of the environment where the user is located;
  • a data sending module 505, configured to send an environment image of the user to the cloud server;
  • the image obtaining module 501 is configured to acquire an image region dividing result of the environment image of the user where the cloud server is sent, and prompt information corresponding to each image region.
  • the apparatus 500 further includes:
  • the image acquisition module 504 is configured to collect an image of the environment where the user is located;
  • the image processing module 506 is configured to perform semantic segmentation on the environment image, divide the environment image into at least one image region, and determine prompt information corresponding to each image region.
  • the image acquisition module 504 is specifically configured to:
  • the environment image of the user is collected when the environment image of the user, or the location of the user, or the change of one or more of the user gestures exceeds a preset threshold.
  • the apparatus 500 further includes:
  • the change prompting module 507 is configured to notify the user when determining that the image region dividing result of the environment image meets a preset condition
  • the preset condition includes one or more of the following:
  • the image area division result is compared with the previous image area division result, a new image area appears, and the area of the new image area is greater than a preset threshold; or
  • the image area division result is compared with the previous image area division result, and the area growth of the original image area is greater than a preset threshold.
  • the selection determining module 502 is specifically configured to:
  • the selection determining module 502 is specifically configured to:
  • the apparatus 500 further includes a playing module 508, configured to play the prompt information after the prompt information determining module 503 determines the prompt information corresponding to the image area selected by the user.
  • the data sending module 505 is further configured to: after the prompt information determining module 503 determines the prompt information corresponding to the image area selected by the user, send the prompt information to the user's portable a device to cause the user's portable device to play the prompt information.
  • the electronic device 600 includes: a communication device 601, a memory 602, one or more processors 603; and one or more modules, the one or more modules being stored in the memory, And being configured to be executed by the one or more processors, the one or more modules comprising instructions for performing the various steps of any of the above methods.
  • an embodiment of the present application further provides a computer program product for use in combination with an electronic device, the computer program product comprising a computer program embedded in a computer readable storage medium, the computer program comprising An instruction to cause the electronic device to perform each of the steps of any of the above methods.
  • embodiments of the present application can be provided as a method, system, or computer program product.
  • the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment in combination of software and hardware.
  • the application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) including computer usable program code.
  • the computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device.
  • the apparatus implements the functions specified in one or more blocks of a flow or a flow and/or block diagram of the flowchart.
  • These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device.
  • the instructions provide steps for implementing the functions specified in one or more of the flow or in a block or blocks of a flow diagram.

Landscapes

  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Physical Education & Sports Medicine (AREA)
  • Rehabilitation Therapy (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Epidemiology (AREA)
  • Pain & Pain Management (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Animal Behavior & Ethology (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

提示信息确定方法、装置、电子设备和计算机程序产品，所述方法包括：获取用户所处环境图像的图像区域划分结果和各图像区域对应的提示信息；确定所述用户选择的图像区域；确定所述用户选择的图像区域对应的提示信息。本申请中，将环境图像划分为多个区域并播放用户所选图像区域对应的提示信息，通过用户主动的信息选择，能够更及时、准确的确定其最关注区域环境的提示信息，以便给用户更好的提示。

Description

提示信息确定方法、装置、电子设备和计算机程序产品 技术领域
本申请涉及导盲技术领域，特别涉及提示信息确定方法、装置、电子设备和计算机程序产品。
背景技术
由于存在视力障碍，盲人无法通过视觉感知其周边环境，包括周围的人、物体、环境的分布和行为，这给其认识世界和出行带来了很大困难。现在已有一些装置和技术辅助盲人感知周围环境，帮助盲人日常室内生活和室外出行。例如一些穿戴式眼镜设备可以拍摄用户周围环境图像，识别出特定种类的物体或障碍后，将物体或者障碍的信息通过语音播报等形式对用户进行提示。
现有技术的不足在于：
由于用户周边环境通常非常复杂，用户会获得大量其并不关注的环境信息，同时也不能准确、及时的获得其真正关注的环境信息。
发明内容
本申请实施例提出了提示信息确定方法、装置、设备和计算机程序产品，主要用以协助用户准确、及时的获得其真正关注的环境信息。
在一个方面，本申请实施例提供了一种提示信息确定方法，其特征在于，所述方法包括：获取用户所处环境图像的图像区域划分结果和各图像区域对应的提示信息；确定所述用户选择的图像区域；确定所述用户选择的图像区域对应的提示信息。
在另一个方面，本申请实施例提供了一种提示信息确定装置，其特征在于，所述装置包括：图像获取模块，用于获取用户所处环境图像的图像 区域划分结果和各图像区域对应的提示信息；选择确定模块，用于确定所述用户选择的图像区域；提示信息确定模块，用于确定所述用户选择的图像区域对应的提示信息。
在另一个方面，本申请实施例提供了一种电子设备，其特征在于，所述电子设备包括：通信设备，存储器，一个或多个处理器；以及一个或多个模块，所述一个或多个模块被存储在所述存储器中，并被配置成由所述一个或多个处理器执行，所述一个或多个模块包括用于执行上述方法中各个步骤的指令。
在另一个方面，本申请实施例提供了一种与电子设备结合使用的计算机程序产品，所述计算机程序产品包括内嵌于计算机可读的存储介质中的计算机程序，所述计算机程序包括用于使所述电子设备执行上述方法中的各个步骤的指令。
本申请实施例的有益效果如下：
本申请中，将环境图像划分为多个区域并播放用户所选图像区域对应的提示信息，通过用户主动的信息选择，能够更及时、准确的确定其最关注区域环境的提示信息，以便给用户更好的提示。
附图说明
下面将参照附图描述本申请的具体实施例，其中：
图1示出了本申请实施例一中提示信息确定方法的流程示意图；
图2示出了本申请实施例二中提示信息确定方法的流程示意图；
图3示出了本申请实施例三实现场景示意图；
图4示出了本申请实施例四实现场景示意图；
图5示出了本申请实施例六中提示信息确定装置的结构示意图；
图6示出了本申请实施例七中电子设备的结构示意图。
具体实施方式
为了使本申请的技术方案及优点更加清楚明白，以下结合附图对本申请的示例性实施例进行进一步详细的说明，显然，所描述的实施例仅是本申请的一部分实施例，而不是所有实施例的穷举。并且在不冲突的情况下，本说明中的实施例及实施例中的特征可以互相结合。
发明人在发明过程中注意到：现有的盲人辅助方案中，由于用户周边环境通常非常复杂，用户会获得大量其并不关注的环境信息，同时也不能准确、及时的获得其真正关注的环境信息。
针对上述不足，本申请提供了一种提示信息确定方法，在获取用户所处环境图像的图像区域划分结果和各图像区域对应的提示信息后，根据用户选择的图像区域确定对应的提示信息。本申请中，将环境图像划分为多个区域并播放用户所选图像区域对应的提示信息，通过用户主动的信息选择，能够更及时、准确的确定其最关注区域环境的提示信息，以便给用户更好的提示。
以下通过具体示例，进一步阐明本发明实施例技术方案的实质。
实施例一：
本实施例在导盲系统中实现，面对的用户通常是视力障碍人士。所述导盲系统包括用户的一个或者多个随身设备以及与这些导盲设备具有通信连接的云端服务器。
图1示出了本申请实施例一中提示信息确定方法流程示意图，如图1所示，所述提示信息确定方法包括：
步骤101，采集用户所处环境图像；
步骤102，将所述用户所处环境图像发送至云端服务器；云端服务器对所述环境图像进行语义分割，将所述环境图像划分为至少一个图像区域，确定各图像区域对应的提示信息；
步骤103，获取云端服务器发送的用户所处环境图像的图像区域划分结果和各图像区域对应的提示信息；
步骤104，确定所述用户选择的图像区域；
步骤105，确定所述用户选择的图像区域对应的提示信息。
在步骤101中，采集用户所处环境的图像，所述用户所处环境图像是通过用户随身设备采集的，所述用户随身设备可以是导盲头盔、导盲眼镜、导盲背心、导盲手杖等各种可穿戴设备，也可以是手机、平板电脑等用户通常随身携带的设备。用户随身设备采集的用户所处环境的图像可以是照片，所述照片可以包括：用户前方的照片，用户随身设备如导盲头盔或者导盲眼镜朝向方向的照片，或者用户某个方向范围的全景照片，例如用户正前方180°图像合成的照片，或者用户周围360°图像合成的照片；用户随身设备采集的用户所处环境的图像也可以是与前述照片采集方式对应的视频的截图。
在一些实施方式中，当用户所处环境图像，或者用户所处位置，或者用户姿态中的一种或者几种发生的变化超过预设阈值时，采集用户所处环境图像。
当用户所处环境图像的变化超过预设阈值时，即用户随身设备捕捉到环境图像的变化过大时，判断为需要重新对环境图像进行识别以及时的给用户提示信息，采集用户所处环境图像。例如用户随身设备可以捕捉并识别用户所处环境的视频，当发现某帧视频截图较上次采集的环境图像像素变化量达到30％时，采集该帧图像作为当前用户所处环境图像。
当用户所处位置的变化超过预设阈值时，即用户随身设备根据自身定位装置确定用户移动的距离超过预设的阈值时，判断为需要重新对环境图像进行识别以及时的给用户提示信息，采集用户所处环境图像。例如用户佩戴的导盲头盔根据自身GPS定位装置判断当前用户的位置较上次采集环境图像时的位置变化达到5m时，采集当前用户所处环境图像。
当用户姿态的变化超过预设阈值时，即用户随身设备根据自身姿态感应装置确定用户的姿态变化超过预设的阈值时，判断为需要重新对环境图像进行识别以及时的给用户提示信息，采集用户所处环境图像。例如用户佩戴的导盲头盔根据自身陀螺仪检测到，当前用户头部较上次采集环境图像时旋转了超过10°时，采集当前用户所处环境图像。
当然也可以同时根据用所处环境图像，用户所处位置，或者用户姿态 中的几种发生的变化是否超过各自的预设阈值判断是否需要采集当前用户所处环境图像。
在步骤102中，用户随身设备将所述用户所处环境图像发送至云端服务器，用户随身设备与云端服务器可通过各种无线网络形式连接，能够实现实时数据传输。云端服务器接收到用户随身设备发送的环境图像后，对所述环境图像进行语义分割，将所述环境图像划分为至少一个图像区域，确定各图像区域对应的提示信息。
这里的语义分割可根据各种算法实现。所述各图像区域对应的提示信息可以直接为图像区域进行语义分割时对应的语义词汇；也可以是根据图像区域的语义进一步匹配处理得到的辅助盲人的提示信息，例如“可以前行”、“需要避让”或者“危险，请小心”等；还可以是对所述图像区域进行进一步图像识别获取的信息，例如被划分入“车辆”图像区域的可以是某种颜色的轿车或者公交车，此时其对应的提示信息可以为“红色车辆”或者“公交车”，被划为“人”的图像区域经进一步识别可得到“红衣服黑裤子的人”或者“戴帽子的女人”等的提示信息。
因云端服务器的图像识别和数据处理能力非常强，因此可以为各图像区域生成各种丰富的提示信息。
处理完成后，云端服务器将用户所处环境图像的图像区域划分结果和各图像区域对应的提示信息反馈给所述用户的随身设备。
需要说明的是，这里的反馈的用户随身设备可以是步骤101中采集用户所处环境的图像的随时设备，也可以是该用户的其他配套随身设备，即导盲头盔、导盲眼镜、导盲背心、导盲手杖、手环等各种可穿戴设备，或者手机、平板电脑等用户通常随身携带的设备。例如，用户的导盲头盔采集用户前方图像发送给云端服务器，云端服务器处理完成后可以发送回所述导盲头盔，也可以发送回该导盲头盔对应用户的手机终端等。
在步骤103中，用户的随身设备获取云端服务器发送的用户所处环境图像的图像区域划分结果和各图像区域对应的提示信息。
在一些实施方式中，在所述步骤103之后，还包括当判断所述环境图像的图像区域划分结果符合预设条件时，通知所述用户；所述预设条件包 括以下一种或者几种：所述图像区域划分结果与之前的图像区域划分结果相比较，出现了新的图像区域，并且所述新的图像区域的面积大于预设阈值；或者，所述图像区域划分结果与之前的图像区域划分结果相比较，原有的某个图像区域的面积增长大于预设阈值。
用户的随身设备获取云端服务器发送的用户所处环境图像的图像区域划分结果后可与上一次云端服务器发送的划分结果进行比较，当图像区域的划分结果出现了大幅的变化时，需要通知用户额外注意。具体的，当出现以下情况之中的一种或者几种时，判断划分结果符合预设条件：
1)出现了新的图像区域，并且所述新的图像区域的面积大于预设阈值；
对应用户前方较近距离突然出现了人或物体的情况等。
2)原有的某个图像区域的面积增长大于预设阈值；
对应距某个物体过近的情况，例如快撞墙或者有车辆行驶到附近等。
此时通知所述用户，通知的内容包括以下一种或者几种：
1)图像区域划分结果符合预设条件；
即发生了环境图像大幅变化时，通知用户这种情况的发生，请用户注意，其通知形式可以是用户某一随身设备的震动或者提示音报警等。
2)发生变化的图像区域的位置；
即通知用户新的较大面积的图像区域的位置或者发生了较大面积变化的原有的图像区域的位置，其通知形式可以为语音告知或者用户某一随身设备的震动提示等，例如语音播放“请注意右前方”，或者用户头戴的导盲头盔右侧震动提醒。结合后续步骤，可由用户自行决定是否主动选择该区域以及时获得该区域的提示信息。
3)发生变化的图像区域的提示信息；
即直接为用户播放新的较大面积的图像区域对应的提示信息，或者发生了较大面积变化的原有的图像区域的提示信息，该提示信息可以由当前的用户随身设备直接播放，也可以将所述通知的提示信息发送至所述用户其他的随身设备进行播放，例如由用户导盲头盔语音播放“请注意建筑”或者将语音信息“请注意红色车辆”发送至用户耳机进行播放。
在步骤104中，用户的随身设备确定所述用户选择的图像区域，确定 用户选择的图像区域的方法可以为接收用户在触摸板或者触摸屏的指令，或者接收用户在特定的操作空间中的动作指令，或者接收用户的语音指令等。
在一些实施方式中，用户的随身设备确定所述用户选择的图像区域的方式为：获取用户在触摸板或者触摸屏的操作位置；根据所述各图像区域与触摸板或者触摸屏的位置映射关系，确定所述用户操作位置对应的图像区域为用户选择的图像区域。
所述触摸板或者触摸屏通常安装在用户随身设备中，并且触摸板或者触摸屏的各位置与环境图像存在映射关系，因此可以根据用户在触摸板或者触摸屏某个位置的触摸操作确定该位置对应在环境图像的位置，并确定该操作位置对应的图像区域，进而获得用户选择的图像区域。所述触摸操作可以为点击、长时间触摸或者滑动等。
在一些实施方式中，用户的随身设备确定所述用户选择的图像区域的方式为：获取所述用户在操作空间中的手势信息；根据各图像区域与所述操作空间的映射关系，确定所述用户操作空间中手势信息对应的图像区域为用户选择的图像区域。
用户随身设备中安装能够检测用户在特定的操作空间手势图像的摄像头，当用户随身设备获得环境图像的图像区域划分结果后，会将该图像区域与手势检测摄像头支持的交互空间进行映射，当用户的特定手势出现在某图像区域对应的空间位置时，确定该位置对应的图像区域为用户选择的图像区域。
检测用户手势的摄像头通常与采集用户所处环境图像的摄像头是相互独立的，其中检测用户手势的摄像头可以安装在用户头戴设备如导盲头盔或者导盲眼镜上，具有较大的下倾角度，使用户在做出手势完成交互时只需将手稍稍抬起，在空中完成手势动作，而不需要将手抬高至正前方，一方面可以缓解用户长时间举手操作的疲劳，另一方面可以避免手势图像被采集入用户所处环境图像，影响环境图像语义分割结果。
在步骤105中，确定所述用户选择的图像区域对应的提示信息，即根据用户的选择的图像区域确定被选择的图像区域对应的提示信息为用户最 关注的提示信息。
在一些实施方式中，在步骤105后还包括，播放所述提示信息。用户随身设备根据用户选择的图像区域确定其对应的提示信息后，可以在当前随身设备直接播放，播放方式可以为语音。
在一些实施方式中，在步骤105后还包括，将所述提示信息发送至所述用户的其他随身设备，以使所述用户的随身设备播放所述提示信息。用户随身设备根据用户选择的图像区域确定其对应的提示信息后，可以将其发送至同一用户的其他随身设备进行播放，所述其他随身设备可以为用户的头戴设备，例如头盔、眼镜或者耳机，其距离用户耳朵更近，能够使用户更清晰的听到播放的提示信息。
本实施例中，将环境图像划分为多个区域并播放用户所选图像区域对应的提示信息，通过用户主动的信息选择，能够更及时、准确的确定其最关注区域环境的提示信息，以便给用户更好的提示；对环境图像的处理在云端服务器完成，图像处理速度更快、更精确，不受用户随身设备图像处理能力的限制；能够在用户所处环境图像、所处位置或者用户姿态发生变化时及时采集环境图像进行处理，既避免了大量的图像数据处理，也能够及时为用户更新环境图像对应的提示信息；能够在环境突变，图像区域出现大幅变化时，及时提示用户以便用户应对；用户能够通过触摸板、触摸屏或者空间手势等多种形式完成相应操作，符合用户习惯；提示信息的播放可以由当前随身设备完成，也可以由更方便用户的其他随身设备完成，用户体验更好。
实施例二：
本实施例在导盲系统中实现，面对的用户通常是视力障碍人士。所述导盲系统包括用户的一个或者多个随身设备，各导盲设备间建立有通信连接。
图2示出了本申请实施例二中提示信息确定方法流程示意图，如图2所示，所述提示信息确定方法包括：
步骤201：采集用户所处环境图像；
步骤202：对所述环境图像进行语义分割，将所述环境图像划分为至少 一个图像区域，确定各图像区域对应的提示信息；
步骤203：获取用户所处环境图像的图像区域划分结果和各图像区域对应的提示信息；
步骤204：确定所述用户选择的图像区域；
步骤205：确定所述用户选择的图像区域对应的提示信息。
步骤201的实施可参照上述实施例一中对步骤101的说明。在步骤201中，用户随身设备采集用户所处环境的图像。在一些实施方式中，当用户所处环境图像，或者用户所处位置，或者用户姿态中的一种或者几种发生的变化超过预设阈值时，采集用户所处环境图像。
在步骤202中，所述随身设备在本地对所述环境图像进行语义分割，将所述环境图像划分为至少一个图像区域，确定各图像区域对应的提示信息。
这里的语义分割可根据各种算法实现，所述各图像区域对应的提示信息可以直接为图像区域进行语义分割时对应的语义词汇，也可以是根据图像区域的语义进一步匹配处理得到的辅助盲人的提示信息，例如“可以前行”、“需要避让”或者“危险，请小心”等。
在步骤203中，随身设备获取本地处理完成后的，用户所处环境图像的图像区域划分结果和各图像区域对应的提示信息。
在一些实施方式中，在所述步骤203之后，还包括当判断所述环境图像的图像区域划分结果符合预设条件时，通知所述用户；所述预设条件包括以下一种或者几种：所述图像区域划分结果与之前的图像区域划分结果相比较，出现了新的图像区域，并且所述新的图像区域的面积大于预设阈值；或者，所述图像区域划分结果与之前的图像区域划分结果相比较，原有的某个图像区域的面积增长大于预设阈值。该步骤的实时可参照上述实施例一中步骤103后的步骤说明。
步骤204的实施可参照上述实施例一中对步骤104的说明。在步骤204中，用户的随身设备确定所述用户选择的图像区域，确定用户选择的图像区域的方法可以为接收用户在触摸板或者触摸屏的指令，或者接收用户在特定的操作空间中的动作指令，或者接收用户的语音指令等。
在一些实施方式中，用户的随身设备确定所述用户选择的图像区域的方式为：获取用户在触摸板或者触摸屏的操作位置；根据所述各图像区域与触摸板或者触摸屏的位置映射关系，确定所述用户操作位置对应的图像区域为用户选择的图像区域。在一些实施方式中，用户的随身设备确定所述用户选择的图像区域的方式为：获取所述用户在操作空间中的手势信息；根据各图像区域与所述操作空间的映射关系，确定所述用户操作空间中手势信息对应的图像区域为用户选择的图像区域。
在步骤205中，所述用户随身设备确定所述用户选择的图像区域对应的提示信息，即根据用户的选择的图像区域确定被选择的图像区域对应的提示信息为用户最关注的提示信息。
在一些实施方式中，在步骤205后还包括，播放所述提示信息。用户随身设备在对环境图像进行语义分割得到多个图像区域，并根据用户选择的图像区域确定其对应的提示信息后，可以在当前随身设备直接播放，播放方式可以为语音。
在一些实施方式中，在步骤205后还包括，将所述提示信息发送至所述用户的其他随身设备，以使所述用户的随身设备播放所述提示信息。用户随身设备在对环境图像进行语义分割得到多个图像区域，并根据用户选择的图像区域确定其对应的提示信息后，可以将提示信息发送至同一用户的其他随身设备进行播放，所述其他随身设备可以为用户的头戴设备，例如头盔、眼镜或者耳机，其距离用户耳朵更近，能够使用户更清晰的听到播放的提示信息。
本实施例中，将环境图像划分为多个区域并播放用户所选图像区域对应的提示信息，通过用户主动的信息选择，能够更及时、准确的确定其最关注区域环境的提示信息，以便给用户更好的提示；对环境图像的处理在用户随身设备本地完成，不受用户随身设备通信功能的限制，不受数据传输延时影响；能够在用户所处环境图像、所处位置或者用户姿态发生变化时及时采集环境图像进行处理，既避免了大量的图像数据处理，也能够及时为用户更新环境图像对应的提示信息；用户能够通过触摸板、触摸屏或者空间手势等多种形式完成相应操作，符合用户习惯；提示信息的播放可 以由当前随身设备完成，也可以由更方便用户的其他随身设备完成，用户体验更好。
实施例三：
本实施例在导盲系统中实现，面对的用户通常是视力障碍人士。所述导盲系统至少包括用户的导盲眼镜、智能手机以及云端服务器，并且三者间建立了通信连接关系。
图3示出了本申请实施例三实现场景示意图，如图3所示，用户的导盲眼镜周期性的采集用户头部朝向方向的环境图像，实时上传云端服务器；云端服务器实时对用户导盲眼镜上传的环境图像进行语义分割，例如分割得到“天空”，“建筑”，“道路”和“车辆”四个区域，并将其语义作为各区域的提示信息；云端服务器将环境图像的区域分割结果和各区域对应的提示信息发送至所述导盲眼镜的用户的智能手机；该用户的智能手机接收到云端发送的场景分割图后，可将图像区域分割结果显示在触摸屏上；用户以点击或滑动的方式触碰屏幕的不同区域后，智能手机播放该图像区域对应的提示信息，或者将该提示信息发送至用户的导盲眼镜，由眼镜播放，例如用户触摸手机屏幕底部，导盲眼镜语音播放该图像区域对应的提示信息“道路”。
实施例四：
本实施例在导盲系统中实现，面对的用户通常是视力障碍人士。所述导盲系统至少包括用户的导盲头盔和云端服务器，并且二者间建立了通信连接关系。导盲头盔除具有水平朝向头部前部的环境图像采集摄像头外，还具有与水平夹角60°朝向用户前方下部的手势检测摄像头，其支持用户前方下部特定的操作空间中的手势识别。
图4示出了本申请实施例四实现场景示意图，如图4所示，用户的导盲头盔监测到用户头部右转了10°后，采集用户头部朝向方向的环境图像，实时上传云端服务器；云端服务器实时对用户导盲头盔上传的环境图像进行语义分割，例如分割得到“天空”，“建筑”，“道路”和“车辆”四个区域，并将其语义词汇以及对应的对用户的辅助信息作为各区域的提示信息；云端服务器将环境图像的区域分割结果和各区域对应的提示信息发送回所 述导盲头盔；导盲头盔获得环境图像的区域分割结果后，会将该图像与手势检测摄像头支持的操作空间做映射，即手势操作空间的每一个位置都对应划分了图像区域的环境图像的一个位置。当用户使用手指隔空移动时，手势检测摄像头会检测到用户手指的位置，并映射至划分了图像区域的环境图像中，导盲头盔会将用户指向的图像区域对应的提示信息通过语音播报出来。例如用户抬手指向操作空间的右下部，导盲头盔播放语音“车辆，请避让”。
实施例五：
本实施例通过多功能导盲背心实现，所述导盲背心胸前具有指向用户前方的环境图像摄像头，用于采集用户所处环境图像，在所述导盲背心下侧靠近用户手的部分装有一个触摸板，能够收集用的触摸操控信息。当导盲背心检测到用正前方的环境图像变化过大时，采集当前图像，并对所述环境图像进行语义分割并将其语义作为各区域的提示信息；将环境图像的区域分割结果与触摸板的区域建立映射关系，在检测到用户在触摸板的触摸操作后，确定用户触摸触摸板的位置对应的分割后的图像区域，以确定用户选择的图像区域；导盲背心为用户语音播放用户选择的图像区域对应的提示信息。
实施例六：
基于同一发明构思，本申请实施例中还提供了一种提示信息确定装置，由于这些装置解决问题的原理与提示信息确定方法相似，因此这些装置的实施可以参见方法的实施，重复之处不再赘述。如图5所示，所述提示信息确定装置500包括：
图像获取模块501，用于获取用户所处环境图像的图像区域划分结果和各图像区域对应的提示信息；
选择确定模块502，用于确定所述用户选择的图像区域；
提示信息确定模块503，用于确定所述用户选择的图像区域对应的提示信息。
在一些实施方式中，所述装置500还包括：
图像采集模块504，用于采集用户所处环境图像；
数据发送模块505，用于将所述用户所处环境图像发送至云端服务器；
所述图像获取模块501，用于获取云端服务器发送的用户所处环境图像的图像区域划分结果和各图像区域对应的提示信息。
在一些实施方式中，所述装置500还包括：
图像采集模块504，用于采集用户所处环境图像；
图像处理模块506，用于对所述环境图像进行语义分割，将所述环境图像划分为至少一个图像区域，确定各图像区域对应的提示信息。
在一些实施方式中，所述图像采集模块504具体用于：
当用户所处环境图像，或者用户所处位置，或者用户姿态中的一种或者几种发生的变化超过预设阈值时，采集用户所处环境图像。
在一些实施方式中，所述装置500还包括：
变化提示模块507，用于当判断所述环境图像的图像区域划分结果符合预设条件时，通知所述用户；
所述预设条件包括以下一种或者几种：
所述图像区域划分结果与之前的图像区域划分结果相比较，出现了新的图像区域，并且所述新的图像区域的面积大于预设阈值；或者，
所述图像区域划分结果与之前的图像区域划分结果相比较，原有的某个图像区域的面积增长大于预设阈值。
在一些实施方式中，所述选择确定模块502具体用于：
获取用户在触摸板或者触摸屏的操作位置；
根据所述各图像区域与所述触摸板或者触摸屏的位置映射关系，确定所述用户操作位置对应的图像区域为用户选择的图像区域。
在一些实施方式中，所述选择确定模块502具体用于：
获取所述用户在操作空间中的手势信息；
根据各图像区域与所述操作空间的映射关系，确定所述用户操作空间中手势信息对应的图像区域为用户选择的图像区域。
在一些实施方式中，所述装置500还包括播放模块508，用于在所述提示信息确定模块503确定所述用户选择的图像区域对应的提示信息之后，播放所述提示信息。
在一些实施方式中，所述数据发送模块505还用于，在所述提示信息确定模块503确定所述用户选择的图像区域对应的提示信息之后，将所述提示信息发送至所述用户的随身设备，以使所述用户的随身设备播放所述提示信息。
实施例七：
基于同一发明构思，本申请实施例中还提供了一种电子设备，由于其原理与提示信息确定方法相似，因此其实施可以参见方法的实施，重复之处不再赘述。如图6所示，所述电子设备600包括：通信设备601，存储器602，一个或多个处理器603；以及一个或多个模块，所述一个或多个模块被存储在所述存储器中，并被配置成由所述一个或多个处理器执行，所述一个或多个模块包括用于执行任一上述方法中各个步骤的指令。
实施例八：
基于同一发明构思，本申请实施例还提供了一种与电子设备结合使用的计算机程序产品，所述计算机程序产品包括内嵌于计算机可读的存储介质中的计算机程序，所述计算机程序包括用于使所述电子设备执行任一上述方法中的各个步骤的指令。
为了描述的方便，以上所述装置的各部分以功能分为各种模块分别描述。当然，在实施本申请时可以把各模块或单元的功能在同一个或多个软件或硬件中实现。
本领域内的技术人员应明白，本申请的实施例可提供为方法、系统、或计算机程序产品。因此，本申请可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本申请可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
本申请是参照根据本申请实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专 用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
尽管已描述了本申请的优选实施例，但本领域内的技术人员一旦得知了基本创造性概念，则可对这些实施例作出另外的变更和修改。所以，所附权利要求意欲解释为包括优选实施例以及落入本申请范围的所有变更和修改。

Claims (18)

  1. 一种提示信息确定方法，其特征在于，所述方法包括：
    获取用户所处环境图像的图像区域划分结果和各图像区域对应的提示信息；
    确定所述用户选择的图像区域；
    确定所述用户选择的图像区域对应的提示信息。
  2. 如权利要求1所述的方法，其特征在于，在所述获取用户所处环境图像的图像区域划分结果和各图像区域对应的提示信息之前，还包括：
    采集用户所处环境图像；
    将所述用户所处环境图像发送至云端服务器；
    所述获取用户所处环境图像的图像区域划分结果和各图像区域对应的提示信息包括：
    获取云端服务器发送的用户所处环境图像的图像区域划分结果和各图像区域对应的提示信息。
  3. 如权利要求1所述的方法，其特征在于，在所述获取用户所处环境图像的图像区域划分结果和各图像区域对应的提示信息之前，还包括：
    采集用户所处环境图像；
    对所述环境图像进行语义分割，将所述环境图像划分为至少一个图像区域，确定各图像区域对应的提示信息。
  4. 如权利要求2或3中任一所述的方法，其特征在于，所述采集用户所处环境图像，包括：
    当用户所处环境图像，或者用户所处位置，或者用户姿态中的一种或者几种发生的变化超过预设阈值时，采集用户所处环境图像。
  5. 如权利要求1至4中任一所述的方法，其特征在于，在所述获取用户所处环境图像的图像区域划分结果和各图像区域对应的提示信息之后，还包括：
    当判断所述环境图像的图像区域划分结果符合预设条件时，通知所述用户；
    所述预设条件包括以下一种或者几种：
    所述图像区域划分结果与之前的图像区域划分结果相比较，出现了新的图像区域，并且所述新的图像区域的面积大于预设阈值；或者，
    所述图像区域划分结果与之前的图像区域划分结果相比较，原有的某个图像区域的面积增长大于预设阈值。
  6. 如权利要求1至5中任一所述的方法，其特征在于，所述确定所述用户选择的图像区域，包括：
    获取用户在触摸板或者触摸屏的操作位置；
    根据所述各图像区域与所述触摸板或者触摸屏的位置映射关系，确定所述用户操作位置对应的图像区域为用户选择的图像区域。
  7. 如权利要求1至5中任一所述的方法，其特征在于，所述确定所述用户选择的图像区域，包括：
    获取所述用户在操作空间中的手势信息；
    根据各图像区域与所述操作空间的映射关系，确定所述用户操作空间中手势信息对应的图像区域为用户选择的图像区域。
  8. 如权利要求1至7中任一所述的方法，其特征在于，在所述确定所述用户选择的图像区域对应的提示信息之后，还包括：
    播放所述提示信息；或者，
    将所述提示信息发送至所述用户的随身设备，以使所述用户的随身设备播放所述提示信息。
  9. 一种提示信息确定装置，其特征在于，所述装置包括：
    图像获取模块，用于获取用户所处环境图像的图像区域划分结果和各图像区域对应的提示信息；
    选择确定模块，用于确定所述用户选择的图像区域；
    提示信息确定模块，用于确定所述用户选择的图像区域对应的提示信息。
  10. 如权利要求9所述的装置，其特征在于，所述装置还包括：
    图像采集模块，用于采集用户所处环境图像；
    数据发送模块，用于将所述用户所处环境图像发送至云端服务器；
    所述图像获取模块，用于获取云端服务器发送的用户所处环境图像的图像区域划分结果和各图像区域对应的提示信息。
  11. 如权利要求9所述的装置，其特征在于，所述装置还包括：
    图像采集模块，用于采集用户所处环境图像；
    图像处理模块，用于对所述环境图像进行语义分割，将所述环境图像划分为至少一个图像区域，确定各图像区域对应的提示信息。
  12. 如权利要求10或11中任一所述的装置，其特征在于，所述图像采集模块具体用于：
    当用户所处环境图像，或者用户所处位置，或者用户姿态中的一种或者几种发生的变化超过预设阈值时，采集用户所处环境图像。
  13. 如权利要求9至12中任一所述的装置，其特征在于，所述装置还包括：
    变化提示模块，用于当判断所述环境图像的图像区域划分结果符合预设条件时，通知所述用户；
    所述预设条件包括以下一种或者几种：
    所述图像区域划分结果与之前的图像区域划分结果相比较，出现了新的图像区域，并且所述新的图像区域的面积大于预设阈值；或者，
    所述图像区域划分结果与之前的图像区域划分结果相比较，原有的某个图像区域的面积增长大于预设阈值。
  14. 如权利要求9至13中任一所述的装置，其特征在于，所述选择确定模块具体用于：
    获取用户在触摸板或者触摸屏的操作位置；
    根据所述各图像区域与所述触摸板或者触摸屏的位置映射关系，确定所述用户操作位置对应的图像区域为用户选择的图像区域。
  15. 如权利要求9至13中任一所述的装置，其特征在于，所述选择确定模块具体用于：
    获取所述用户在操作空间中的手势信息；
    根据各图像区域与所述操作空间的映射关系，确定所述用户操作空间中手势信息对应的图像区域为用户选择的图像区域。
  16. 如权利要求9至15中任一所述的装置，其特征在于，
    所述装置还包括播放模块，用于在所述提示信息确定模块确定所述用户选择的图像区域对应的提示信息之后，播放所述提示信息；
    或者，
    所述数据发送模块还用于，在所述提示信息确定模块确定所述用户选择的图像区域对应的提示信息之后，将所述提示信息发送至所述用户的随身设备，以使所述用户的随身设备播放所述提示信息。
  17. 一种电子设备，其特征在于，所述电子设备包括：
    通信设备，存储器，一个或多个处理器；以及一个或多个模块，所述一个或多个模块被存储在所述存储器中，并被配置成由所述一个或多个处理器执行，所述一个或多个模块包括用于执行权利要求1至8中任一所述方法中各个步骤的指令。
  18. 一种与电子设备结合使用的计算机程序产品，所述计算机程序产品包括内嵌于计算机可读的存储介质中的计算机程序，所述计算机程序包括用于使所述电子设备执行权利要求1至8中任一所述方法中的各个步骤的指令。
PCT/CN2017/117424 2017-12-20 2017-12-20 提示信息确定方法、装置、电子设备和计算机程序产品 WO2019119290A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2017/117424 WO2019119290A1 (zh) 2017-12-20 2017-12-20 提示信息确定方法、装置、电子设备和计算机程序产品
CN201780002906.9A CN108140124B (zh) 2017-12-20 2017-12-20 提示信息确定方法、装置、电子设备

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2017/117424 WO2019119290A1 (zh) 2017-12-20 2017-12-20 提示信息确定方法、装置、电子设备和计算机程序产品

Publications (1)

Publication Number Publication Date
WO2019119290A1 true WO2019119290A1 (zh) 2019-06-27

Family

ID=62400864

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/117424 WO2019119290A1 (zh) 2017-12-20 2017-12-20 提示信息确定方法、装置、电子设备和计算机程序产品

Country Status (2)

Country Link
CN (1) CN108140124B (zh)
WO (1) WO2019119290A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111783557A (zh) * 2020-06-11 2020-10-16 北京科技大学 一种基于深度视觉的可穿戴导盲设备及服务器

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109376581B (zh) * 2018-09-03 2022-02-22 腾讯科技（武汉）有限公司 对象关系识别方法和装置、存储介质及电子装置
CN112699884A (zh) * 2021-01-29 2021-04-23 深圳市慧鲤科技有限公司 定位方法、装置、电子设备及存储介质

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6198395B1 (en) * 1998-02-09 2001-03-06 Gary E. Sussman Sensor for sight impaired individuals
CN101986673A (zh) * 2010-09-03 2011-03-16 浙江大学 一种智能手机导盲器及导盲方法
CN104739622A (zh) * 2015-02-04 2015-07-01 覃泳睿 一种新型可穿戴导盲设备及其工作方法
CN105342816A (zh) * 2015-11-24 2016-02-24 上海斐讯数据通信技术有限公司 一种用于盲人导航的实时避障系统
CN105686935A (zh) * 2016-01-08 2016-06-22 中国石油大学(华东) 一种智能导盲方法
CN106265004A (zh) * 2016-10-08 2017-01-04 西安电子科技大学 多传感器智能盲人指路方法与装置
CN107223224A (zh) * 2016-12-19 2017-09-29 深圳前海达闼云端智能科技有限公司 一种弱视辅助方法和装置

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6198395B1 (en) * 1998-02-09 2001-03-06 Gary E. Sussman Sensor for sight impaired individuals
CN101986673A (zh) * 2010-09-03 2011-03-16 浙江大学 一种智能手机导盲器及导盲方法
CN104739622A (zh) * 2015-02-04 2015-07-01 覃泳睿 一种新型可穿戴导盲设备及其工作方法
CN105342816A (zh) * 2015-11-24 2016-02-24 上海斐讯数据通信技术有限公司 一种用于盲人导航的实时避障系统
CN105686935A (zh) * 2016-01-08 2016-06-22 中国石油大学(华东) 一种智能导盲方法
CN106265004A (zh) * 2016-10-08 2017-01-04 西安电子科技大学 多传感器智能盲人指路方法与装置
CN107223224A (zh) * 2016-12-19 2017-09-29 深圳前海达闼云端智能科技有限公司 一种弱视辅助方法和装置

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111783557A (zh) * 2020-06-11 2020-10-16 北京科技大学 一种基于深度视觉的可穿戴导盲设备及服务器
CN111783557B (zh) * 2020-06-11 2023-08-15 北京科技大学 一种基于深度视觉的可穿戴导盲设备及服务器

Also Published As

Publication number Publication date
CN108140124A (zh) 2018-06-08
CN108140124B (zh) 2020-12-08

Similar Documents

Publication Publication Date Title
US11366516B2 (en) Visibility improvement method based on eye tracking, machine-readable storage medium and electronic device
US10832448B2 (en) Display control device, display control method, and program
US11170580B2 (en) Information processing device, information processing method, and recording medium
US9563272B2 (en) Gaze assisted object recognition
US10126823B2 (en) In-vehicle gesture interactive spatial audio system
US10971188B2 (en) Apparatus and method for editing content
JP6574937B2 (ja) 通信システム、制御方法、および記憶媒体
EP2509070B1 (en) Apparatus and method for determining relevance of input speech
US9529428B1 (en) Using head movement to adjust focus on content of a display
CN111182205B (zh) 拍摄方法、电子设备及介质
CN111432245B (zh) 多媒体信息的播放控制方法、装置、设备及存储介质
WO2019119290A1 (zh) 提示信息确定方法、装置、电子设备和计算机程序产品
CN108958587B (zh) 分屏处理方法、装置、存储介质和电子设备
CN108881544B (zh) 一种拍照的方法及移动终端
CN109040427B (zh) 分屏处理方法、装置、存储介质和电子设备
US11756302B1 (en) Managing presentation of subject-based segmented video feed on a receiving device
CN113613053A (zh) 视频推荐方法、装置、电子设备及存储介质
US10860847B2 (en) Visual perception assistant
US12028645B2 (en) Subject-based smart segmentation of video feed on a transmitting device
CN113849142B (zh) 图像展示方法、装置、电子设备及计算机可读存储介质
CN116997886A (zh) 扩展现实中的数字助理交互

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17935103

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 22/10/2020)

122 Ep: pct application non-entry in european phase

Ref document number: 17935103

Country of ref document: EP

Kind code of ref document: A1