Model response evaluation
Reviewing AI answers for correctness, completeness, instruction following, and clarity.
LLM evaluation · Scientific reasoning QA · Multilingual annotation
LLM evaluator for STEM reasoning, multilingual AI data, and model-output QA.
Physics-trained reviewer with experience evaluating model answers for numeric accuracy, conceptual correctness, assumption handling, and Turkish-English-Spanish language quality.
Evaluation systems for models that need more than fluent answers.
01
Gold-answer benchmark design
02
Failure-mode taxonomy
03
Human-reviewed model QA
04
Turkish / English / Spanish evaluation
Hire me for
AI model response evaluation
STEM and physics answer review
Turkish-English-Spanish AI data QA
Rubric-based annotation and labeling
Gold-answer comparison
Hallucination and reasoning-error detection
Dataset review and cleanup
Freelance services
Practical evaluation services
I support AI training and evaluation projects with careful model-output review, rubric-based labeling, STEM reasoning checks, and multilingual QA.
STEM and physics answer review
Checking calculations, formulas, assumptions, and conceptual explanations in physics/math-style tasks.
Multilingual AI data QA
Reviewing Turkish, English, and Spanish outputs for meaning, terminology, fluency, and reasoning consistency.
Rubric-based annotation
Applying client-provided labels, grading criteria, and quality rules consistently across model outputs.
Gold-answer comparison
Comparing model responses against reference answers and identifying missing, incorrect, or unsupported claims.
Error and hallucination tagging
Marking numerical mistakes, conceptual errors, unsupported statements, prompt misreadings, and formatting failures.
Dataset cleanup and review
Checking prompt/answer pairs, improving clarity, flagging unusable data, and preparing reviewable outputs.
Work style
- Comfortable with detailed annotation guidelines
- Careful with edge cases and ambiguous outputs
- Able to explain labels with short reviewer notes
- Experienced with human review after automatic evaluation
- Works in English, Turkish, and Spanish
Selected work
Project portfolio
A compact portfolio of published and incoming evaluation projects. The focus is evidence: what was tested, what failed, and how the results were reviewed.
Case study: testing whether LLMs can solve scientific problems correctly, not just produce fluent explanations.
In this project, I built and reviewed a two-stage benchmark for quantum-search reasoning. The goal was to check whether model answers preserved the correct math, assumptions, and reasoning structure under initial prompts and targeted follow-up questions.
The same evaluation workflow can be adapted to STEM answer review, multilingual AI data QA, rubric-based annotation, and model-output quality checks.
Scientific reasoning evaluation
Numeric and conceptual error detection
Assumption-handling review
Human review after automatic checks
Follow-up robustness testing
Partial Quantum Search LLM Evaluation Pipeline
A local-first evaluation pipeline for testing whether language models can reason through Partial Quantum Search problems beyond plausible final answers.
Stage 1 evaluates first-answer correctness. Stage 2 tests whether the same reasoning survives targeted follow-up pressure.
120Stage 1 model runs
189Stage 2 follow-ups
2Evaluation stages
Scientific reasoning
LLM evaluation
Deterministic checks
Human review
Robustness testing
Stage 1 — One-shot evaluation
120Total model runs
112Successful outputs
8Discarded (no result)
54Passed review
58Failed review
19Critical failures
Of 112 successful outputs, 58 failed review (52%) and 19 of those were critical. Figures match the published Stage 1 report (original 120-run set).
189Follow-up runs
63Parent runs
3Trials each
7Models
67%Pass rate
21%Fail rate
127PASS
40FAIL
22PARTIAL
By branch — Robustness 153 (114 P / 20 partial / 19 F) · Conceptual correction 24 (9 P / 15 F, 37.5% pass) · Numeric audit 12 (4 P / 2 partial / 6 F).
LLM Scientific Reasoning Metrics Dashboard
Compared deterministic scientific grading against BERTScore, ROUGE-L, chrF, and ROSCOE across 252 structured LLM outputs. The dashboard shows that text/reasoning metrics are useful diagnostics but weak correctness proxies, and that large numeric failures require deterministic gates.
252structured outputs
32scientific scenarios
8model identities
LLM Evaluation
Scientific Reasoning
Metrics
Data QA
Python
Dashboard
Five-step evaluation pipeline
Benchmark Design
Designed Partial Quantum Search scenarios to test scientific reasoning, assumption tracking, numerical consistency, and cost-model interpretation.
Stage 1: One-Shot Evaluation
Collected model answers to controlled PQS prompts and checked whether models produced correct reasoning, not only plausible final numbers.
Deterministic Checks + Human Review
Used deterministic extraction, numeric checks, concept checklists, and human review to classify answers into pass, numeric error, critical error, artifact, or unusable cases.
Stage 2: Targeted Follow-Ups
Generated follow-up prompts from Stage 1 outcomes: robustness checks for passes, numeric audits for arithmetic errors, and conceptual correction prompts for critical failures.
Evidence Export + Portfolio Reporting
Exported reviewed evidence, aggregate results, and report-ready artifacts for a transparent portfolio case study.
Clients and work contexts
Relevant evaluation and annotation work
Client names are included where appropriate. Private platform work is described by task type rather than exposing confidential details.
Datamundi
Multilingual annotation and quality review involving linguistic judgment and task-specific evaluation.
Mindrift
Physics and natural-science model evaluation context, including domain-specific reasoning checks.
Translation and terminology
Professional Turkish-English-Spanish language work, terminology review, and technical communication.
LLM annotation clients
Human-review workflows for correctness, rubric labels, response quality, and failure-mode documentation.
Services
Evaluation work I can support
Benchmark design
Controlled prompts, perturbation variants, answer keys, and scoring rubrics.
STEM answer QA
Physics, math, and technical reasoning review for model outputs.
Failure taxonomy
Error-label systems that separate conceptual failures, numeric mistakes, and artifact issues.
Multilingual evaluation
Turkish-English-Spanish review, terminology consistency, and translation-sensitive evaluation.
Human review workflows
Annotation guidelines, review queues, disagreement handling, and final cleaned outputs.
Dataset packaging
CSV, JSONL, Excel, Markdown, GitHub README, and report-ready deliverables.
Background
Physics-trained evaluator focused on quantum reasoning, STEM QA, and multilingual model evaluation.
My academic background is in physics, with a current focus on quantum-computation reasoning, scientific problem evaluation, and the failure modes of language models in technical domains. I work on evaluation projects where the important question is not only whether a model gives the right final answer, but whether it preserves the correct assumptions, cost model, notation, and reasoning structure.
My current portfolio connects physics reasoning with LLM evaluation: partial quantum search, query-complexity accounting, hallucination stress tests, Turkish scientific reasoning, and trilingual benchmark design. The goal is to turn model failures into inspectable evidence: gold answers, error labels, human-reviewed notes, and reusable datasets.
- Core area
- LLM evaluation for quantum computation, physics, and STEM reasoning
- Languages
- Turkish, English, Spanish
- Location
- Contact
- contact@ustunpasaigdir.com
- linkedin.com/in/ustunpasaigdir
Contact
Need a benchmark, dataset, or evaluation workflow?
Send the domain, model type, language, and what you need to measure.