Big tech company fine-tunes GenAI with TrainAI experts
Our client wanted to fine-tune its GenAI open-source LLM to increase its accuracy, safety and robustness. Realizing those goals would be hard to achieve with a conventional crowdsourcing approach to data annotation, the company reached out to RWS, who leveraged its TrainAI team to quickly recruit, train and manage a scalable team of qualified subject-matter experts as data specialists to complete the work.
How do you differentiate a GenAI open-source large language model (LLM) from others on the market? By fine tuning it using feedback from data specialists who are qualified experts in their field.
Many of today’s generative AI (GenAI) open-source LLMs have been trained on similar AI training data or content. Our client wanted to improve the usability, robustness and safety of its LLM so that user groups could rely on it more confidently to support innovation and collaboration. In particular, the client wanted the LLM to achieve a standard that would make it a resource for professionals in their own fields globally.
The client’s goals
Maximize the model’s accuracy by training it on specific topic areas
Improve the model’s safety and security by mitigating the risk of it generating hallucinations (nonsensical or false output) or potentially harmful content
Enhance the multilingual capabilities of the model
The company knew it couldn’t achieve those objectives using a conventional crowdsourcing approach that involves farming out data annotation tasks to freelancers or gig workers.
To deliver the differentiated results it was looking for, it needed access to qualified subject-matter and language experts to work as data specialists. Without the in-house resources to quickly recruit, train, manage and scale up such a team, it reached out to RWS – an existing approved vendor for localization and data services – for help.
Challenges
Maximize LLM accuracy by training it on specific topic areas
Improve safety and security by mitigating the risk of generating hallucinations or harmful content
Achieve a standard that makes the LLM a resource for professionals
Recruiting the right team of domain and language experts
RWS’s dedicated AI practice, TrainAI®, created a comprehensive AI training and fine tuning data services solution to meet the client’s objectives. Seasoned data services.
The first step was to recruit experts with university degrees in the client’s required fields – general knowledge, business, humanities and STEM (science, technology, engineering and mathematics) – to work as data specialists. Their role was to produce robust, domain-specific content to train and fine-tune the LLM.
To meet initial project needs, TrainAI proposed hiring 100 data specialists, based in locations specified by the client, to work 20 hours a week on the project. When the project scope was expanded from English only to include nine additional languages (French, German, Hindi, Indonesian, Italian, Portuguese, Spanish, Thai and Vietnamese), TrainAI quickly pivoted to incorporate the required languages into its hiring and onboarding plan.
To maintain exclusivity and data confidentiality, RWS employed the specialists as regular part-time employees, not as contractors or freelancers. The TrainAI team worked with RWS’s Vendor Resource Management and Talent Acquisition teams to develop a recruitment plan. To hire the right people in the right locations with the right expertise, they leveraged:
RWS’s TrainAI community of AI data specialists across the globe
RWS Article One Partners (AOP) Connect community (specializing in IP research)
External recruitment marketing strategies
4-week project ramp-up
10,000–13,000 hours of work per month
285 domain and language experts onboarded
Trained and ready to work in four weeks
TrainAI had 100 domain experts hired, tested, onboarded, trained and ready to start work in a short turnaround time. It continued to ramp up recruiting and onboarding efforts to meet client needs, bringing the total number of domain and language experts working on the project to 285.
AI experience wasn’t a pre-requisite, so TrainAI trained them on performing LLM fine tuning tasks to meet the client’s needs by:
Converting hundreds of pages of client-provided guidelines, instructions and examples into digestible training courses
Running multiple live training sessions on project-specific technology tools and tasks
Virtual desktops for data security
To safeguard the client’s data, TrainAI implemented secure infrastructure to minimize the risk of data breach or loss caused by, for example, device damage or theft.
Delivering LLM fine-tuning services
TrainAI provides the following GenAI services to fine-tune the client’s LLM:
Domain and language expertise
Recruiting and managing domain and language experts, and triaging tasks to the right experts with the appropriate topic knowledge, educational level and language expertise
Content creation
Prompt engineering (or prompt design), which involves the data specialists writing detailed, informative prompt-response pairs on topics in their specialist domains and languages.
Model fine-tuning: reinforcement learning from human feedback (RLHF)
Prompt-response quality assessment (QA) including response rating, evaluation, editing and enhancement. A complete qualitative error trend analysis and collection of low-quality examples to improve the model were provided. TrainAI was also responsible for performing quality audits on other third-party vendor work, making RWS the source of truth on quality for the client.
Fact extraction and verification including reviewing existing prompt-response pairs, identifying purported facts in the responses, and verifying their authenticity.
Risk mitigation
Red teaming to uncover vulnerabilities in the LLM that cause it to generate inaccurate, hallucinatory or potentially harmful responses.
Adversarial testing, a subset of red teaming, which involves the data specialists using creative or ambiguous prompts to test model robustness and assess response reliability.
Monitoring and reporting
TrainAI monitors team performance and provides additional training as needed. Any potential issues are proactively identified and resolved. Key metrics, including staffing, completed tasks, average handling time and quality criteria, are tracked against project objectives; and the client receives regular detailed reporting.
Blending technological understanding and human intelligence, TrainAI provides data collection, annotation and validation services for all types of AI data, in any language, at any scale, based on the principles of responsible AI.
Responsible AI: how it’s done
TrainAI’s project approach follows the principles of responsible AI to ensure delivery of dependable LLM training and fine-tuning data with the following characteristics.
Ethically sourced. Instead of crowdsourcing for the project and hoping for the best, TrainAI smartsourced a team of skilled, qualified and vetted experts to work as data specialists on the project and deliver the required quality output.
Fair. Specialists join RWS as regular part-time employees on W-2 contracts. They receive paid training and are compensated fairly for the time they spend working on the project.
Accurate and reliable. TrainAI matches domain-specific tasks to the right experts with the right qualifications and expertise to ensure delivery of trustworthy data the client can depend on.
Transparent and explainable. The client has visibility into project sourcing and compensation, as well as processes and workflows, for a full understanding of the data and its potential impact on LLM training.
Private and secure. TrainAI ensures the privacy and security of project data through a combination of HR, legal and IT best practices.
Accelerating LLM training and rollout at scale
TrainAI supported the training and roll-out of the latest generation of the client’s GenAI LLM by:
Ramping up the project within a tight 4-week time frame
Recruiting and training 285 (to date) qualified domain and language experts as data specialists, working as part-time RWS employees
Completing 10,000–13,000 hours of work per month at the project’s peak
As a result, the client’s LLM is well on its way to becoming more accurate, safe and robust, differentiating it from other models on the market.
Satisfied with the project outcomes, the client awarded four additional AI data services projects to TrainAI.
