Helping people take a step towards a tech career.
Addressing low conversion rates with CodeYourFuture’s relaunched Intro to Digital course.
Using User Research to Improve Conversion Rates in CodeYourFuture's Updated Introductory Coding Course
Project: Enhancing the effectiveness and accessibility of CodeYourFuture's foundational "Intro to Digital" (ITD) course.
My Role: Lead UX Researcher. Hands on work: strategy, creating research plan, supervising research, data analysis (qual and quant), reporting, leading ideation sessions.
Timeline: October 2024 - April 2025
Context: CodeYourFuture (CYF) is a non-profit organisation dedicated to training individuals from low-income backgrounds, helping them launch careers in the tech industry.
A New Course Meets Hurdles
In late 2023, facing a shrinking job market for junior developers, CodeYourFuture paused its programs from early to mid-2024 to undergo a significant curriculum redesign. Drawing on 7+ years of experience and our volunteer community expertise, CYF launched a new vocational pathway in September 2024, starting with the ‘Intro to Digital’ (ITD) course.
About the Intro to Digital (ITD) Course:
Course Goal: Equip beginners (no prior experience needed) with foundational tech skills (GenAI, data analysis, coding, cloud deployment) and serve as the entry point to CYF's advanced training programs.
Format: A flexible, self-paced online course (approx. 20-40 hours) supported by a welcoming community and optional workshops led by industry professionals.
However, the launch revealed a critical problem: disappointingly low conversion rates.
The Metric: Conversion Rate = (Number of Trainees Completing ITD) / (Number of Trainees Registered).
September 2023: The first cohort achieved only 10-12% conversion, falling short of the ~15% benchmark set by previous introductory courses.
November 2023: Despite initial adjustments after retrospectives, the rate remained stagnant. Out of 685 registered trainees, only 85 completed the course, yielding a 12.4% conversion rate.
While the Intro to Digital course was the least affected by the curriculum overhaul, we needed to understand why our Intro to Digital course was performing below expectation.
Launching an Investigation
I proposed and led a dedicated user research project. Our goal was to diagnose the drop-off points and identify actionable improvements for the next cohort.
Our Research Strategy:
Qualitative Deep Dive: I designed a research plan to understand the reasons behind the low progression rates.
AI Check: I used AI to review my research plan, generating suggestions for potential biases, additional question areas, and structural improvements.
User Interviews: Using the questions from our research plan, CYF's dedicated Learner Support team conducted phone interviews with 56 trainees from the November cohort who had registered but made minimal progress.
Data Analysis (AI-Enhanced): To complement the qualitative data, I leveraged AI tools to:
Clean and analyse recent referral source data, looking for patterns between how trainees joined CYF and their subsequent progress.
Analyse historical referral data to identify previously successful channels.
Note: All data put into AI was anonymised and handled according to privacy best practices.
Key Research Topics: Barriers, knowledge of available support, accessibility issues we've overlooked.
Initial Findings:
The interviews and data analysis quickly highlighted key solvable issues:
Information Gap: A significant 39% of interviewed trainees were unaware of the optional workshops and drop-in support sessions designed to help them.
Language Barrier: Our analysis, combined with qualitative feedback, indicated that some recruitment channels, particularly those reaching asylum seekers and refugees, were bringing in applicants who lacked the intermediate English skills necessary for the course material.
Laptop Policy: We confirmed our recent decision not to loan laptops for ITD (due to scale and increased availability of public/alternative access) wasn't the primary driver of drop-off at this stage, though it remained a factor for some. (Action: A backlog item was created to investigate mapping local library computer access points).
Targeted Interventions
Armed with the insights from our first findings, we moved quickly to implement changes before the January 2024 ITD cohort:
Proactive Onboarding: We introduced and ran online pre-course information sessions in December. Registered trainees were invited to live demonstrations of the platform, syllabus overviews, and Q&A sessions about available support.
Amplified Communications: We updated our communication strategy regarding workshops and support. Communications became more frequent, clearer, and we reintroduced SMS messaging. We implemented better comms tracking to gauge effectiveness.
Empowering Volunteers: We shared the research findings with our volunteer trainers and mentors, asking them to proactively reinforce key information and signpost support resources during their interactions with trainees.
Informed Outreach: Based on historical data analysis suggesting potential, we reintroduced Eventbrite as a recruitment channel, testing Eventbrite’s advertising platform too.
Measurable Improvement
The interventions yielded positive results for the January 2024 cohort:
January 2024: 594 registered, 105 completed.
Conversion Rate: 17.7%
This represented a ~5 percentage point increase from the previous cohort and surpassed our historical benchmark. While fewer people registered overall compared to November, significantly more completed the course, validating our research-driven approach. It also highlighted a separate challenge: increasing the overall volume of suitable applicants.
Iterating and Digging Deeper
Encouraged by the success, we prepared for the April 2024 cohort by:
Reinforcing Success: Repeating the successful interventions (pre-course sessions, enhanced comms, volunteer briefings).
Expanding Research: Conducting another round of qualitative interviews, this time speaking with 78 trainees who didn't progress in the January cohort.
AI-Powered Synthesis: The volume of notes from 78 interviews presented a significant analysis challenge. I used an AI tool to synthesise the interview transcripts and notes. This rapidly surfaced key themes, patterns, and correlations that would have taken much longer to identify manually.
Key Finding - The Laptop Nuance:
This round of research brought the lack of computer access to the forefront: 45% cited it as a major blocker.
However, the AI-assisted synthesis revealed a crucial nuance. Lack of a personal computer often correlated strongly with other significant challenges:
Lower existing digital literacy levels.
Language barriers.
Lower intrinsic motivation or significant competing life priorities (e.g., childcare).
Strategic Decision on Laptops: This deeper understanding, enabled by AI synthesis, led to an informed decision: We decided not to prioritise providing direct computer access support specifically for the ITD course at this time. The data suggested that individuals facing this barrier often had multiple, interconnected challenges.
Simply providing a laptop might not address the root causes and could set them up for frustration if they lacked the other necessary skills or bandwidth to succeed in the course. Our focus remained on ensuring those who were ready had the best possible support.
Scaling Outreach & Facing New Dynamics
Simultaneously, our outreach team acted on earlier referral data insights. Identifying influencers as a strong potential channel, they connected with a popular Arabic-speaking influencer relevant to our target audience.
Action: A promotional video collaboration was launched just before the April ITD intake.
Impact: This single partnership drove nearly 5 times the typical number of registrations.
However, this massive influx created a new dynamic: many registered too late to benefit fully from our pre-course onboarding interventions.
Volume vs. Efficiency
The April 2024 results reflected a new reality:
April 2024: 2,494 registered, 208 completed.
Conversion Rate: 8.34%
While the percentage conversion rate dropped significantly due to the massive, late-joining denominator, the absolute number of graduates (208) nearly doubled from January. This represented the highest number of successful ITD completions to date, feeding a larger cohort into the next stage of the CYF pathway.
Human-Centred Design With AI
This project directly addresses how we approached improving a service (the ITD course support structure) that teaches AI technologies along with other digital skills, using human-centred principles, while also integrating AI into our process:
AI as a Research Amplifier: AI wasn't a trainee-facing feature in this phase; it was a tool we used to enhance our human-centred research:
Sharpening Methodology: AI helped refine our research plan, making our qualitative inquiry more effective.
Uncovering Data Patterns: AI accelerated the analysis of referral data, guiding outreach efforts.
Deepening Qualitative Insights: AI synthesis of 78 interviews was crucial. It moved us beyond the surface-level "lack of laptop" issue to understand the complex, correlated factors, enabling a more nuanced and ethical strategic decision.
Accessibility Focus: The laptop decision, while seemingly counterintuitive, was rooted in ensuring trainees had the holistic readiness (skills, language, time) for the course, making it truly accessible to those prepared to succeed.
Ethical Considerations: We maintained data privacy throughout the use of AI. By not directing resources at those who appeared less ready for our course, we improved our services to those with the best chance of progressing.
Responsiveness & Iteration: The process was inherently responsive. We identified a problem (data), investigated (qualitative research), acted (interventions), measured (results), and then iterated with deeper research and further action.
By strategically integrating AI tools within our human-centred design process, we gained richer insights faster, allowing us to make more informed, ethical, and responsive improvements to the support structure surrounding our ITD course.
Conclusion: Learning and Looking Ahead
This iterative research journey demonstrates how to combine qualitative human insight with the analytical capabilities of AI.
We successfully diagnosed the initial conversion problem, implemented effective solutions, and uncovered deeper complexities related to trainee readiness.
Next Steps:
Continue refining onboarding and communication strategies based on ongoing feedback (in progress).
Explore methods for assessing digital literacy and language proficiency earlier in the application process (in progress).
Implement the planned mapping of community computer access resources for signposting (not yet started).
Develop more targeted outreach and potentially segmented onboarding flows to better support trainees with varying levels of readiness (in progress).
Investigate RAG project - to input learner conversations, questions and research findings into an AI-driven course chatbot (in progress).