There are encouraging advances in prosthetic vision for the blind, including retinal and cortical implants, and other “sensory substitution devices” that use tactile or electrical stimulation. However, they all have low resolution, limited visual field, and can display only few gray levels (limited dynamic range), severely restricting their utility. To overcome these limitations, image processing or the imaging system could emphasize objects of interest and suppress the background clutter. We propose an active confocal imaging system based on light-field technology that will enable a blind user of any visual prosthesis to efficiently scan, focus on, and “see” only an object of interest while suppressing interference from background clutter. The system captures three-dimensional scene information using a light-field sensor and displays only an in-focused plane with objects in it. After capturing a confocal image, a de-cluttering process removes the clutter based on blur difference. In preliminary experiments we verified the positive impact of confocal-based background clutter removal on recognition of objects in low resolution and limited dynamic range simulated phosphene images. Using a custom-made multiple-camera system, we confirmed that the concept of a confocal de-cluttered image can be realized effectively using light field imaging.


01/2016 – 01/2021, Active confocal imaging for visual prostheses, US Department of Defense, Vision Research Program, W81XWH-16-1-0033 (Co-PI)

02/2015-10/2015, Active confocal imaging for visual prostheses, Grant from the Promobilia Foundation, Stockholm, Sweden #14222 (PI)


Pamir, Z., Jung, J. – H., & Peli, E. (2022). Preparing participants for the use of the tongue visual sensory substitution deviceDisability and Rehabilitation: Assistive Technology , 17(8), 888-896. Publisher’s Version

Peli, E., Jung, J. – H., & Qui, C. (2022). Motion parallax in object recognition. US Patent No. US11270451B . US patent.

Avraham, D., Jung, J. – H., Yitzhaky, Y., & Peli, E. (2021). Retinal prosthetic vision simulation: temporal aspectsJournal of Neural Engineering , 18 (4), 0460d9. Publisher’s Version

Avraham, D., Jung, J. – H., Yitzhaky, Y., & Peli, E. (2021). Simulating the effects of persistence and perceptual fading in retinal prosthetic vision12th The Eye and the Chip World Research Congress 2021.

Peli, E., & Jung, J. – H. (2020). Active confocal imaging systems and methods for visual prostheses . US Patent No. US10857036B2 . US patent.

Pamir, Z., Canoluk, M. U., Jung, J. – H., & Peli, E. (2020). Poor resolution at the back of the tongue is the bottleneck for spatial pattern recognitionScientific Reports , 10 (1), 2435. Publisher’s Version  pamir-etal-2020-scirep-tracingstudy-data_3.xlsx

Jung, J. – H., & Peli, E. (2018). Light-field background de-cluttering for visual prostheses. In Imaging and Applied Optics 2018 (3D, AO, AIO, COSI, DH, IS, LACSEA, LS&C, MATH, pcAOP) (pp. 3Tu3E.3) . Optical Society of America. Publisher’s Version

Han, S. ‘E., Qiu, C., Lee, K. R., Jung, J. – H., & Peli, E. (2018). Word recognition: re-thinking prosthetic vision evaluationJournal of Neural Engineering , 15 (5), 055003 . IOP Publishing. Publisher’s VersionAbstract wordrecognition2018datasharing_1.xlsx

Qiu, C., Lee, K. R., Jung, J. – H., Goldstein, R., & Peli, E. (2018). Motion Parallax Improves Object Recognition in the Presence of Clutter in Simulated Prosthetic VisionTranslational Vision Science & Technology , 7 (5), 29-29. Publisher’s Version motion_parallax_data_sharing_2.xlsx [PDF]

Peli, E., & Jung, J. – H. (2018). Active confocal imaging systems and methods for visual prostheses . US Patent No. US10130513B2 . US Patent.

Jung, J. – H., Qiu, C., & Peli, E. (2017). Background de-cluttering for object recognition in video-based visual prostheses. 10th The Eye and The Chip World Research Congress . Detroit, MI.

Jung, J. – H., Pu, T., & Peli, E. (2016). Comparing object recognition from binary and bipolar edge images for visual prosthesesJournal of Electronic Imaging , 25 (6), 061619. Publisher’s Version

Jung, J. – H., Pu, T., & Peli, E. (2016). Comparing object recognition from binary and bipolar edge featuresIS&T Human Vision and Electronic Imaging XXI. Publisher’s Version

Jung, J. – H., Aloni, D., Yitzhaky, Y., & Peli, E. (2015). Active Confocal Imaging for Visual ProsthesesVision Research , 111 (June), 182-196. Publisher’s Version activeconfocalimaging_datasharing_2.xlsx (Best paper of the year by a trainee in SERI, Harvard Medical School)