Indeed uses OpenAI's ChatGPT Chatbot
Since its launch, Indeed has leveraged AI to facilitate millions of connections between job seekers and employers. Features like ‘Invite to Apply’ use AI-driven job recommendations tailored to a candidate’s resume, Indeed Profile, and qualifications. With advancements in generative AI, Indeed is now using OpenAI's GPT models to enhance these recommendations further. By fine-tuning the personalized language in the ‘Invite to Apply’ feature, Indeed more effectively highlights why a candidate’s background or experience aligns with specific job opportunities, making the platform even more effective at matching job seekers with the right roles.
Early stage Of AI driven Job Matching
In the early stages of AI-driven job matching, Indeed's models successfully connected job seekers with relevant job postings, offering concise explanations for these matches. However, the product and engineering teams at Indeed recognized the potential for even greater advancements by incorporating OpenAI’s GPT models. These models, with their ability to process natural language, could provide deeper insights into why a candidate receives a particular job recommendation.
The goal for Indeed was to enhance the "why" behind job recommendations, giving candidates a clearer understanding of the match. To achieve this, Indeed’s engineering team began collaborating with OpenAI in 2023. By leveraging the Chat Completions API and refining its few-shot prompting techniques, the team aimed to improve the contextual relevance of job matches. This collaboration involved extensive A/B testing and ongoing evaluation of each change, both during the development process and after the launch, ensuring that the AI-powered job recommendations were not only accurate but also meaningful for job seekers.
Fine-Tuning OpenAI's API for Enhanced Job Match Explanations
Indeed initially found success with few-shot prompting to enrich the explanations behind job matches, but the approach led to significant token consumption due to the platform’s large scale. To address this, Indeed collaborated with OpenAI to fine-tune a more efficient GPT model. This optimized model achieved similar performance while reducing token usage by 60%.
The fine-tuning process included leveraging GPT-4 for data augmentation and developing specific content guidelines. Indeed's team also established an annotation system to label LLM-generated outputs, create ground truths, and add context, ensuring the model could grasp nuanced details.
Extensive testing confirmed that personalized job recommendations increased the pool of qualified candidates. To scale this personalization, Indeed partnered with OpenAI to deploy dedicated instances in January 2024. The fine-tuned GPT model was successfully implemented, allowing Indeed to deliver personalized job opportunities to millions more job seekers.
Chris Hyams, CEO of Indeed, emphasizes, “Our investment in AI matching technology over the decades has been crucial in connecting job seekers with employers. While our matching technology is robust, the key to successful recommendations lies in explainability. By integrating OpenAI's GPT explanations with our proprietary AI and vast marketplace data, we’re able to connect more people to jobs faster—a significant win for job seekers, employers, and society."
GPT Personalization Drives Significant Growth
The power of GPT-enhanced personalized messaging was clearly demonstrated through a multi-stage experiment, which saw the volume of messages grow from 1 million to nearly 20 million per day over several months.
Horatio Lun, Platform PM at Indeed, notes, "We're constantly testing ways to improve the 'Invite to Apply' experience and connect job seekers with the most relevant opportunities. Fine-tuning OpenAI's GPT models has enabled us to deliver highly personalized recommendations, even as we scaled our messaging efforts to 20 million messages daily."
The Indeed team conducted a comparison between the traditional job matching version of 'Invite to Apply' and the GPT-powered version, which included more tailored context for each applicant. The results were striking:
- A 20% increase in started job applications
- A 13% uplift in downstream success, indicating that not only were more candidates applying, but employers were also finding these applicants to be a better fit, leading to more hires
By integrating GPT-powered context into its 'Invite to Apply' feature, Indeed saw a significant boost in both the number of qualified candidates and successful hires, leading to a positive impact on revenue during the testing phase.
Chris Hyams, CEO of Indeed, highlights, “Crucially, we've been able to utilize OpenAI in a way that's ROI-positive, which opens up tremendous opportunities to further invest in this new infrastructure and drive revenue growth.”
The success of 'Invite to Apply' is just one example of how Indeed is harnessing OpenAI technology. Nearly a dozen products across the platform now utilize OpenAI to deliver more personalized and engaging experiences, helping job seekers uncover new opportunities and enabling employers to hire more efficiently.
