Authors
Christina M. Funke; Judy Borowski; Karolina Stosio; Wieland Brendel; Thomas S. A. Wallis; Matthias Bethge

Five points to check when comparing visual perception in humans and machines

publication date
March 2021
Category
see more

Abstract/Introduction

With the rise of machines to human-level performance in complex recognition tasks, a growing amount of work is directed toward comparing information processing in humans and machines. These studies are an exciting chance to learn about one system by studying the other. Here, we propose ideas on how to design, conduct, and interpret experiments such that they adequately support the investigation of mechanisms when comparing human and machine perception. We demonstrate and apply these ideas through three case studies. The first case study shows how human bias can affect the interpretation of results and that several analytic tools can help to overcome this human reference point. In the second case study, we highlight the difference between necessary and sufficient mechanisms in visual reasoning tasks. Thereby, we show that contrary to previous suggestions, feedback mechanisms might not be necessary for the tasks in question. The third case study highlights the importance of aligning experimental conditions. We find that a previously observed difference in object recognition does not hold when adapting the experiment to make conditions more equitable between humans and machines. In presenting a checklist for comparative studies of visual reasoning in humans and machines, we hope to highlight how to overcome potential pitfalls in design and inference.


Conclusion/Results

Comparing human and machine visual perception can be challenging. In this work, we presented a checklist on how to perform such comparison studies in a meaningful and robust way. For one, isolating a single mechanism requires us to minimize or exclude the effect of other differences between biological and artificial and to align experimental conditions for both systems. We further have to differentiate between necessary and sufficient mechanisms and to circumscribe in which tasks they are actually deployed. Finally, an overarching challenge in comparison studies between humans and machines is our strong internal human interpretation bias. 

Using three case studies, we illustrated the application of the checklist. The first case study on closed contour detection showed that human bias can impede the objective interpretation of results and that investigating which mechanisms could or could not be at work may require several analytic tools. The second case study highlighted the difficulty of drawing robust conclusions about mechanisms from experiments. While previous studies suggested that feedback mechanisms might be important for visual reasoning tasks, our experiments showed that they are not necessarily required. The third case study clarified that aligning experimental conditions for both systems is essential. When adapting the experimental settings, we found that, unlike the differences reported in a previous study, DNNs and humans indeed show similar behavior on an object recognition task. 

Our checklist complements other recent proposals about how to compare visual inference strategies between humans and machines (Buckner, 2019; Chollet, 2019; Ma & Peters, 2020; Geirhos et al., 2020) and helps to create more nuanced and robust insights into both systems.


Testimonials


  • Vision therapy is well worth the response, time and effort. Our Son had a hard time focusing and writing neatly. After vision therapy Seth could complete a task in half the time if previously took. His abilities to focus improved greatly and so did his handwriting.

    Also, he was better at listening. As a parent, we wanted learning to be fun for our Son, and vision therapy made this possible.


    Amanda T.

  • Vision Therapy is well worth the expense, time and effort. Our son had a hard time focusing handwriting neatly. After vision therapy Seth could complete a task in half the time it previously took his abilities to focus and improved greatly and so did his handwriting.

    Also, he was better at listening. As a parent you wanted learning to be fun for our son and vision therapy made this possible.


    Alexa H.

  • Vision Therapy has given or son the tools he needs to be able to scan and read the written word more effectively and efficiently.

    He love working with John and these working sessions give him the motivation to gladly work on his homework assignments.

    It amazed us to see the difference in the tracking of his eyes and along a line of it's from the beginning to the end of the treatment.

    Dr. McBryar , Kristen and John  are all marvelous and we would recommend them to anyone I only wish that we would have found them sooner!


    Charlotte M.

  • Prior to coming to the institute for vision development my son complained of daily headaches. Therapy has eliminated his headaches completely. I love knowing my son is able to learn pain-free for the rest of his life because of the work that has been done over just a few weeks in this office. He he absolutely loved coming that didn't even feel like going to a doctor or therapy. We are grateful for the relief he was able to find by coming here.

    Thank you!


    Olivia C.

  • Seems much less frustrated with life

    Reads non-stop and fast

    Spelling abilities have been hugely improved

    Seems much more confident

    Thank You!!!


    Kelly O.

Blog

group-of-small-nursery-school-children-with-teache-W2UF5RU

Vision Among Us

One of the challenges with visual impairments and visual deficits is that it's difficult to detect them just by looking […]

Read More
frustrated-millennial-businessman-having-strong-headache-tired-from-laptop-work

Vision Therapy for Neurological Conditions

Each year an estimated 1.5 million Americans sustain a traumatic brain injury (TBI). According to the Neuro Optometric Rehabilitation Association, […]

Read More
DSC_2077

Vision Therapy for Adults

Adults generally accept changes in themselves as part of aging, and think that there's nothing they can do about it. […]

Read More
see all blogs

Contact Us To Amplify Your EyeCare

Working Hours

Monday - Thursday
8:00 am - 5:15pm
friday-sunday
Closed
Location
1043 Executive Drive Suite #101 Hixson, TN 37343
Website Accessibility Policy
Safety protocols page
phone-handsetarrow-uparrow-right linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram