The Edge browser’s learning tools are a set of tools that help with reading and comprehension. Learning tools are aimed primarily at children and adults who can access the internet but have a difficult time comprehending textual content.
“We are not all the same; we do not all have the same kinds of minds; education works most efficiently for most individuals if these differences …are taken into account rather than denied or ignored.”
‘Reflections on Multiple Intelligences’ (Gardner, 1995)
Reading and comprehending content
Learning tools were classified into three broad categories; accessibility, dictionary and comprehension. For the first version of learning tools, the team decided to focus on accessibility and comprehension since dictionary was already a widely available feature across ecosystems.
Primary and secondary research showed that giving users the power to choose the way content appeared by changing fonts, text sizes as well as the color contrast between the foreground (text) and background would greatly aid in improving accessibility. The Edge browser by then already had a feature called reading view where the team piloted the first version with smart defaults for color contrasts. This version showed great promise and the team decided to extend it to the EPUB format which is widely used for text books and other educational content. I worked on the design of the toolbar to change the appearance of the content.
As the name suggests, ‘Read out loud’ reads out textual content on the browser while highlighting what is being read along with its context. OneNote’s learning tools had proved that a multi-sensory approach of a visual + audio feedback showed great results, thereby helping english language learners and students with learning disabilities to consume textual content. We decided to use their learnings and extend it to the browser. This was especially challenging since unlike OneNote which had reflowing content, the browser enabled consumption of both re-flowable (websites, EPUB) and fixed (PDF, fixed EPUB, etc) content. A TTS (Text to speech) engine was used to read out the text. Although it was straightforward to read out EPUB and re-flowable PDF content due to linear structure of the content, it was especially challenging when reading out HTML websites. The TTS engine would in ideal conditions read the DOM structure from top to bottom, however HTML websites could have elements that appeared to be in the beginning of the webpage, but the content could be anywhere in the structure. To mitigate this, a separate machine learning based approach of extracting the DOM and placing it linearly was used when reading out content.
Various types of color contrast ratios were tested that would work across file formats, however we soon realized that a one-fits-all approach would be counter to our goal of enabling learners to focus on the content. Thereafter, different contrast ratios were designed and tested.