EarSight: Hear where you’re go

Download the pro version for full functionality: https://play.google.com/store/apps/details?id=com.maxloh.earsight.pro


This app translates the camera's distance map (depth image) to audio tones, as an aid for the blind. It will be confusing to use at first, but will get easier with practice.

Disclaimer: This app uses Android's ARCore for depth detection, which is slow and unreliable unless your device has a dedicated depth sensor (TOF camera).

Each pitch corresponds to a particular position or "pixel" of the depth image. The amplitude (loudness) of that pitch represents how close it is to the device.

Scan modes:

Horizontal line: A line through the middle; low to high notes go left to right.

Grid: Each octave is a row; each note a column.

Sweep: All notes are a horizontal line, which constantly sweeps top to bottom. Higher spatial resolution at the expense of "frame rate".

Diatonic: 4 octaves, 7 notes per octave, aeolian minor scale.

Chromatic: 5 octaves, 12 notes per octave.

For example: The diatonic options have 28 notes (pitches) in total, in the key of A minor. In diatonic grid mode, the spatial resolution is 7x4 and temporal resolution is tens of frames per second. In diatonic sweep mode, the spatial resolution is 28x14 and temporal resolution is about 1 frame per second. Horizontally from left to right, the notes are A, B, C, D, E, F, G. That means if you're hearing a loud D across many octaves (among other frequencies), you're probably walking into a wall. But if you only hear a very high D, there's probably something hanging from above in front.