Be My Eyes

Overview

Be My Eyes addresses daily independence challenges encountered by blind and low-vision individuals worldwide. Originally established on a crowdsourced community network of sighted volunteers connected through live one-way video calls, the application expanded its capabilities by integrating Be My AI, a digital assistant powered by OpenAI multimodal vision models. This development enables users to obtain immediate, self-directed visual explanations for everyday questions without waiting for a volunteer connection.

The core workflow unites computer vision with interactive dialogue. When pointing a smartphone camera at an object, environment, or printed text, the system processes the capture and delivers a comprehensive descriptive summary. If more detail is required, users can maintain an ongoing conversational exchange, posing follow-up questions regarding item placement, cooking instructions, or specific visual characteristics within the scene.

Whenever automated descriptions prove inconclusive or sensitive tasks require nuanced human judgment, such as reading private financial statements or navigating complex physical environments, the interface provides immediate escalation. Users can transfer their session directly to a sighted human volunteer or to specialized enterprise support representatives in the Service Directory.

This dual architecture combining automated multimodal intelligence with dedicated human assistance sets a practical standard for digital accessibility. The service supports personal, educational, and professional autonomy while maintaining strict privacy boundaries regarding visual media processing.

Features and functionality

• Be My AI conversational vision: Interprets photos of scenes, objects, diagrams, and printed text, supporting natural follow-up questions about visual context.
• On-demand volunteer video calls: Connects users instantly to a global network of sighted volunteers for live visual assistance across dozens of languages.
• Specialized Service Directory: Provides direct accessibility-focused customer support from major technology, healthcare, and retail organizations.
• Document and display reading: Recognizes and structures text from packaged goods, physical correspondence, computer screens, and appliance panels.
• Smart glasses compatibility: Supports hands-free AI queries and visual interaction via Ray-Ban Meta smart glasses integration.
• Screen reader navigation: Implements full optimization for VoiceOver, TalkBack, and keyboard navigation with auditory and haptic feedback.

Use cases

• Identifying household items and groceries: Distinguishing between similar pantry containers, reading expiration dates, and verifying clothing colors.
• Navigating public transit and venues: Reading signage in transit stations, locating room numbers in office buildings, and checking flight status boards.
• Workplace and academic assistance: Reviewing charts in slide presentations, inspecting printed memos, and parsing software interfaces.
• Troubleshooting electronic appliances: Reading error codes on appliance digital displays, thermostats, and television settings menus.