PandaDoc
Senior Data Scientist - AI Team
Job Summary
This role involves designing, evaluating, and optimizing AI models, datasets, and pipelines, with a focus on large documents and advanced AI techniques such as RAG, knowledge graphs, and conversational search. The candidate will work closely with cross-functional teams to develop AI solutions that provide tangible value to small and medium-sized businesses using large language models and related technologies. Key responsibilities include model fine-tuning, deployment, dataset creation, and ensure responsible AI practices. The position requires extensive expertise in Python, GenAI, and a customer-centric, independent, product-driven approach.
Required Skills
Job Description
The world of AI and GenAI is evolving rapidly, and we’re on a mission to build a powerful AI platform that drives innovation across all PandaDoc products. Our goal is to develop scalable, impactful AI solutions that empower small and medium-sized businesses (SMBs) to thrive.
This is a unique opportunity to design, evaluate, and optimize AI models, datasets, and pipelines—helping us turn cutting-edge AI into practical, scalable solutions for tens of thousands of businesses.
What makes this role special?
- AI-First Products – Develop models and systems that enable AI-first experiences, enhancing the value we provide to SMBs.
- High Impact, High Ownership: You’ll have the freedom to build and lead, working cross-functionally with engineers, designers, and product managers and bring the best practices from the industry.
- Real-World AI Challenges: You’ll tackle the complexities of productizing AI and Gen-AI applications—from scalability and observability to security and fine-tuning.
- Innovation at Scale: Your work will enable fast, iterative experimentation for customer-facing AI solutions, helping SMBs grow, compete, and thrive.
In this role, you will:
- Model Evaluation & Testing – Create evaluation frameworks to assess LLMs and GenAI applications, measuring recall, effectiveness, and accuracy.
- Dataset Creation & Data Preparation – Develop high-quality datasets for model training, fine-tuning, and inference.
- Prompt Engineering & Optimization – Establish evaluation pipelines for prompt performance, ensuring reliability and efficiency.
- Model Fine-Tuning – Optimize LLMs for specific business use cases to improve their effectiveness.
- LLM Deployment & Guardrails – Deploy LLMs in production and implement systems to maintain model constraints and safe usage.
- Multi-Model Support – Work with various LLMs (e.g., GPT-4, Claude, fine-tuned models) for efficient routing and task handling.
- AI-Driven Automation – Contribute to AI-powered automation systems that enhance customer workflows and streamline routine tasks.
- Collaboration Across Teams – Work closely with PMs, designers, and engineers to define AI requirements and bring solutions to life.
About you:
- Metadata extraction – experience building pipelines that extract structured information from documents.
- Retrieval-Augmented Generation (RAG) – designing and optimizing RAG systems, including chunking strategies, retrieval performance, and integration with LLMs.
- Knowledge graphs – understanding of how to represent and query document relationships to support better retrieval and reasoning.
- Semantic and conversational search – familiarity with search systems, especially Elasticsearch or similar tools, and how to build intelligent search experiences, including conversational interfaces.
On top of:
- Python Expertise – 5+ years of experience in Python using in DS.
- GenAI Experience – Hands-on experience with LLMs, RAG, knowledge graphs and fine-tuning.
- LLM Evaluation & Optimization – Proven ability to measure and improve model performance.
- Dataset & Data Preparation Skills – Expertise in building and curating datasets for AI training.
- Production Deployment – Experience deploying LLMs and implementing guardrails for responsible AI usage.
- Trend Awareness – Stays updated on the latest AI research, tools, and frameworks, identifying ways to apply them effectively.
- Customer-Centric Mindset – Passion for building AI solutions that provide real value to end users.
- Fast-Paced & Independent – Thrives in a high-speed environment with autonomy to solve problems.
- Product-Driven Approach – Focuses on building AI solutions that drive tangible business impact.
Why PandaDoc?
At PandaDoc, we’re committed to helping SMBs succeed by providing AI-powered tools that simplify their workflows and drive efficiency. With over 60,000 businesses trusting our platform, we’re now investing in AI to take our impact even further.
You’ll be part of a collaborative, high-energy team that values creativity, autonomy, and real-world impact. If you’re excited about solving complex AI challenges and building tools that matter, let’s talk!
PandaDoc
PandaDoc streamlines proposals, quotes & other business document workflows - Trusted by 50K+ Clients - SOC 2, HIPAA & GDPR Compliant!
See more jobsSafe Remote Job Search Tips
Verify Employer Thoroughly
Research the company's identity thoroughly before applying. Check for a professional website with contacts, active social media, and LinkedIn profiles. Verify details across platforms and look for reviews on Glassdoor or Trustpilot to confirm legitimacy.
Never Pay to Get a Job
Legitimate employers never require payment for applications, training, background checks, or equipment. Always reject upfront payment requests or demands for bank details, even if they claim it's for purchasing necessary work gear on your behalf.
Safeguard Your Personal Information
Protect sensitive data like SSN, bank details, or ID copies. Share this only after accepting a formal, written job offer. Ensure it's submitted via a secure company system or portal, never through insecure channels like standard email attachments.
Scrutinize Communication & Interviews
Watch for communication red flags: poor grammar, generic emails (@gmail), vague details, or undue pressure. Be highly suspicious of interviews held only via text or chat apps; legitimate companies typically use video or phone calls.
Beware of Unrealistic Offers
If an offer's salary or benefits seem unrealistically high for the work involved, be cautious. Research standard pay for similar roles. Offers that appear 'too good to be true' are often scams designed to lure you into providing information or payment.
Insist on a Formal Contract
Always secure and review a formal, written job offer or employment contract before starting work or sharing final personal details. Ensure it clearly defines your role, compensation, key terms, and conditions to avoid misunderstandings or scams.