Accessibility Evaluation for Programiz Pro— a Code Learning Platform
Programiz Pro
Programiz Pro is a paid online platform available in web-based environments that provides interactive coding lessons and exercises in a variety of programming languages like C, C++, Java, Python, and SQL. Programiz also provides free-to-use platforms like the web-based platform programiz.com and a few mobile applications which feature similar learning environments with ads.
Users have the opportunity to learn at their own pace with hands-on practice using quizzes, coding challenges, and projects in an in-browser IDE environment. Most of the learning resources are text-based, except for projects which include instructions with images. The platform does not utilize audio or video, but Programiz has a YouTube channel that provides additional help through video tutorials and courses.
Evaluation of Accessibility
The platform lacks an accessibility statement or a Voluntary Product Accessibility Template (VPAT). VPAT is a template that contains information regarding how a product or service conforms with Section 508 of the Rehabilitation Act of 1973.
On doing a quick accessibility audit of the website programiz.pro using an audit tool accessiBe that evaluates the adherence of the platform to the Web Content Accessibility Guidelines (WCAG) 2.1 level AA success criteria, the website was found to be semi-compliant. WCAG is a set of guidelines often considered to be the benchmark for website accessibility created by the World Wide Web Consortium (W3C). The audit features tests focusing on three main categories: screen-reader adjustment (for blind users), keyboard navigation adjustment (for motor-impaired users), and UI, design, and readability adjustments (for visually impaired users). You can read the full report of the audit here.
Furthermore, axe DevTools chrome extension was used to do an exploratory analysis of the accessibility of a few pages of the website (programiz.pro). This dev tool also audits the accessibility of the platform based on adherence to the WCAG.
Additionally, Google’s Mobile-friendly test and PageSpeed Insights were also used to assess the accessibility factors of the Programiz website. The website scored a high 93/100 score on mobile accessibility and a moderate 77/100 score on web accessibility.
Update: Google’s Mobile-friendly test was retired by Google in December 2023. The accessibility check was again performed on Google Lighthouse in September 2024 when the homepage for Programiz.pro scored 94 out of 100 for mobile and 86 out of 100 for desktop.
Some common accessibility problems found through these automated accessibility tests are:
- Failure to meet minimum color contrast ratio thresholds
- Not all scrollable regions have keyboard access
- Failure to inform screen-readers that links open in new tabs or on the same tab
- Has elements that visually appear as titles but actually do not have heading HTML tags
- Active pop-up banner about the Black Friday sale does not have an appropriate attribute, so screen readers do not know how to navigate through it
- Not all images have alternative text
- The font of some text elements is extremely small and non-readable
- Buttons do not have an accessible name, so screen readers only read “button” and won’t tell what the button does
- Form elements do not have associated labels, so form controls won’t be properly communicated by screen readers
The accessibility of the platform has also been manually assessed using the POUR (Perceivable, Operable, Understandable, and Robust) framework. The findings of the POUR evaluation are elaborated below:
Perceivable
All learners on Programiz should have access to and be able to understand all information provided by the platform. The platform is compatible with popular browsers like Google Chrome, Safari, and Edge on my Macbook Air 2020 and iPhone 14 Pro. The Programiz platform features a simple user interface that prevents cognitive overload. Additionally, the content is presented in simple language and sans-serif font with appropriate spacing and breaks, making it easy to read and understand. Since the platform is completely text-based, it is fairly accessible for learners who lack sensory disabilities or who are deaf and hard of hearing.
Ensuring a perceivable platform also involves providing options for learners to adjust the way information is presented to suit their personal needs and preferences. However, the platform lacks options to adjust preferences for font size, font spacing, color, etc. It also lacks built-in read-aloud features. The platform works poorly with the VoiceOver function on Mac, as it does not always read the description content. Additionally, I failed to figure out a way to use the in-browser compiler using voice control.
Operable
All learners on Programiz should have the ability to interact with the content using a mouse, keyboard, and voice controls. The platform features structured modules and a navigation bar on the left for easy navigation. It has been tested on Safari and Google Chrome browsers on both iPhone and Samsung mobile devices, and functions well on mobile browsers, including the in-browser IDE. This provides unique accessibility for low-income learners who may not have access to a laptop or computer.
On the other hand, the heavily marketed AI feature, SensAI, is difficult to operate as it can only be accessed by selecting a piece of code inside the compiler, and the platform lacks clear instructions for accessing this feature. The platform is also found to be fairly inaccessible for learners with dexterity impairments who cannot use touch screens, touchpads, or a mouse. Automated tests have shown that the platform is not completely accessible for keyboards. Additionally, the platform is extremely difficult to navigate using voice control, making it highly inaccessible to learners who are blind.
Understandable
All learners on Programiz should be able to understand and practice the content to develop their programming skills. In general, Programiz provides all the content in simple language without unnecessary difficult jargon. The instructions for coding challenges are clear and concise. Additionally, the comments and scaffolding in the in-browser IDE for attempting the coding challenges are easy to understand.
The platform provides easy-to-understand explanations and error messages for incorrect solutions in quizzes and coding challenges. Also, the SensAI feature provides extra help with descriptions to understand the code, detect mistakes, and correct the mistakes. However, the SensAI explanations are not always correct as it is generated by OpenAI’s GPT4 model and are susceptible to the limitations of the model’s programming knowledge.
Robust
Learning platforms should be supported in a variety of web browsers and devices including tablets and smartphones. The platform works fairly well on all kinds of devices: desktops, tablets, and smartphones. Also, the platform seems fairly robust across different browsers ( Safari, Chrome, and Edge) and different operating systems (MacOS, iOS, Windows, and Android). For Chrome browser, developer tools were used to check the screens for the homepage on different devices which showed decent results. However, due to insufficient resources and time, the robustness of the platform could not be tested with more screen sizes, browsers, and systems.
No statements or information were found about the compatibility of the platform with specific browsers, devices, or versions on the website. The platform was found to be inaccessible for voice control or screen readers.
Accessibility Recommendations
Perceivable
- Appropriate text descriptions should be added to the images
- Sufficient color contrast should be maintained for all UI components
- The font size should remain readable consistently
- There should be options for users to adjust the font size and spacing
- Keyboard access should be provided to all the scrollable regions
- All the platform content should be accessible to screen readers (including pop-ups, buttons, forms, etc.)
Operable
- Keyboard access should be provided to all the scrollable regions
- All the platform content should be accessible to screen readers (including pop-ups, buttons, forms, etc.)
- Clear instructions should be provided to use the SensAI feature
Understandable
- Learners would be prone to believe all the descriptions and suggestions of the SensAI feature to be correct. So, an appropriate disclaimer about the shortcomings of the AI’s capability should be provided
Robust
- FAQs should include information about supported devices, browsers, and systems
Note: The evaluation was done in Nov, 2023.
