Software Engineer – AI Response Evaluation (Remote)

About the role

This contract position focuses on evaluating AI‑generated technical responses. You will assess clarity, correctness, tone, and usefulness, providing detailed rationales for each score.

Key responsibilities

• Evaluate and compare multiple AI‑generated responses to technical prompts, assigning analytical quality scores and documenting detailed rationales.
• Identify responses that are technically accurate but poorly communicated, or fluent yet factually incorrect, and record observations with precision.
• Spot small errors, inconsistencies, or ambiguities in technical communication and articulate them clearly.
• Collaborate asynchronously with project managers and reviewers to ensure alignment on evaluation standards.

Required profile

• Native‑level English fluency with exceptional written communication skills.
• Professional experience as a software engineer working with any modern technology stack.
• Proven track record authoring or reviewing technical documentation, design docs, API references, or specifications.
• Experience conducting code or design reviews and providing feedback on technical writing.
• Strong analytical skills and meticulous attention to detail.