Code Generation Models • Compared on your task

Understand AI-generated codeCompare. Diagnose. Improve.

Give several code generation models the same programming or data-science task. CGM Comparator runs each result in a sandbox, scores it, annotates it, and lets you refine and compare.

Start for free

The app opens on its own address. You'll sign in with the same Google account in the app.

  • Sandboxed execution
  • Shared test cases
  • Up to 5 models per run

AI ships code fast.
Verification is still manual.

The generation gap is solved. The verification gap is not.

Multiple models, no clear comparison

Different models produce different implementations. Existing tools provide outputs — not evidence.

Manual verification is expensive

AI rewrites large code blocks, but developers still review line by line. Verification becomes the new bottleneck.

Passing tests ≠ quality code

Readability, efficiency, and trust remain invisible. Current evaluation focuses only on functional correctness.

Core Capabilities

A platform for understanding, comparing, diagnosing, and improving AI-generated code.

Understand the Code

Turn raw generations into structured functional blocks to reason about behavior fast.

TRANSFORMTRAIN

Compare Across Models

See structural differences across models without reading full code line-by-line.

Model AModel BModel C

Multi-dimensional Evaluation

Shared test results plus efficiency, reusability and editability scores.

0.82 SCORE

Guided Refinement

Apply targeted fixes with guided prompts powered by diagnostics.

REFINE

Experiment History

Finished runs stay in your history with their code, scores and results.

WORKFLOW

Easy to start.

Follow a guided flow from prompt to results —
with annotation and diagnosis you can toggle anytime.

Screenshot of the CGM Comparator start screen: prompt box, model selection, analysis toggles and view mode

Enter a prompt

Describe your programming or data-science task. Every selected model gets the same requirement.

FEATURE DEEP DIVE

See how CGM Comparator
works in practice

Explore how the system analyzes, compares, and improves AI-generated workflows.

01 // ANALYSIS

Compare implementations at a functional level

Understand how different models solve the same task — not just what they output.

  • Structural comparison across models
  • Functional block alignment
  • Multi-dimensional evaluation signals
Compare View — 3 Candidates
Example
Model A
Example data
Model BBEST
Example data
Model C
Example data
Efficiency0
Reusability0
Editability0
Efficiency0
Reusability0
Editability0
Efficiency0
Reusability0
Editability0
PASSED
6/6
1.2s
PASSED
6/6
0.9s
FAILED
4/6
1.8s
02 // INTELLIGENCE

Reveal workflow structure and intent

Automatically segment generated code into functional units and expose execution logic.

  • Functional abstraction beyond syntax
  • Structured annotation pipeline
  • Clear execution flow
Annotation View — Example5 blocks detected
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
INPUT PARSING
def process_pipeline(filepath: str,
target: str = 'label'):
df = pd.read_csv(filepath)
df = df.dropna(subset=[target])
DATA TRANSFORM
scaler = StandardScaler()
features = [c for c in df.columns if c != target]
X = scaler.fit_transform(df[features])
y = df[target].values
CORE LOGIC
clf = RandomForestClassifier(
n_estimators=100, max_depth=5,
random_state=42)
clf.fit(X_train, y_train)
VALIDATION
score = clf.score(X_test, y_test)
assert score > THRESHOLD, f'Low: {score:.2f}'
OUTPUT
return {'model': clf, 'score': score,
'scaler': scaler, 'features': features}
INPUT
DATA
CORE
VALIDATION
OUTPUT
Illustration
03 // ITERATION

Improve results with structured guidance

Turn diagnosis into actionable steps and refine code iteratively.

  • Evidence-based diagnosis
  • Guided improvement steps
  • Iterative refinement loop
Refinement History — Example
3 versions
v1
v1.0Initial Generation
Model Abaseline
0

Functional but high cyclomatic complexity. Nested loops, unclear naming.

correctness
86
readability
61
efficiency
69
v2
v1.1Readability Refinement
Model Bguided
0
+11 pts

Logic restructured into named functions. Depth reduced from 5→2.

Readability: +23pts
correctness
88
readability
84
efficiency
76
v3
v1.2Efficiency OptimizationBEST
Model Bsurgical edit
0
+10 pts

Vectorized inner loop. Memory cut 40%. All 50 test cases pass.

Efficiency: +18pts
correctness
98
readability
86
efficiency
94
PLATFORM ARCHITECTURE

How the platform works end-to-end

A full-stack loop from prompt to evidence, diagnosis, and continuous improvement.

Interaction Layer

Where developers interact with generated code and insights.

Prompt interface
Compare view
Annotation panel
Refinement controls

Evaluation Engine

Multi-dimensional analysis and functional understanding of generated code.

Functional abstraction
Semantic analysis
Performance evaluation
Cross-model comparison

Execution + Storage Infrastructure

Sandbox execution, result persistence, and history tracking.

Sandbox execution
Database storage
Experiment history
Model adapters
USE CASES

Real scenarios. Real decisions.

From vendor selection to production review — CGM Comparator fits into the moment when AI code quality actually matters.

For engineers choosing a model

Know which model to use before you commit.

  • Run the same prompt across up to five models at once
  • See shared test results and efficiency, reusability and editability scores side by side
  • Choose with evidence from your own task
compare — 3 models (example)Example
Model A
Model B
Highest
Model C
Tests passed %
84
Tests passed %
95
Tests passed %
71
Efficiency
78
Efficiency
91
Efficiency
68
Reusability
72
Reusability
88
Reusability
74
Editability
80
Editability
94
Editability
70
Overall
79
Overall
92
Overall
71

How results are produced

What a score on this platform does and does not mean.

Tasks without a dataset

Function mode

Before generation, one evaluation spec is derived from your requirement: an entry point and a set of test cases. Every model implements that entry point and every candidate is checked against the same cases.

Expected values and the comparison stay outside the sandbox.

Tasks with a dataset

Script mode

Each candidate script runs with your files in its working directory. The result says whether the script ran to completion. It is not a check that the analysis is right, and the app says so.

Shown as “ran”, never as “passed”.
Up to 5
Models per run

From OpenAI, Anthropic, Google, Cohere, DeepSeek.

3
Quality scores

Efficiency, reusability, editability.

1
Credit per run

One multi-model run or one refinement.

7 days
Dataset retention

Uploaded datasets are deleted after at most 7 days.

Shared test cases
Sandboxed execution
Time and memory limits
Function-level annotations
Guided refinement
Run history
PRICING

Plans differ by credits,
nothing else.

One credit = one multi-model run or one refinement. Every plan has the same features.

Free

Free

Try the product on your own tasks.

  • 10 credits, one-time
  • Credits do not refill.
  • One credit = one multi-model run or one refinement.
Start free in the app

You'll sign in with the same Google account in the app.

Pro

$49/ month

For regular individual use.

  • 200 credits per billing period
  • Balance is set to 200 each time a billing period is paid. Unused credits do not carry over.
  • One credit = one multi-model run or one refinement.

You'll sign in with Google first, then pay through Stripe. No free trial.

Team

$149/ month

A larger credit allowance on one account. Seats and shared workspaces are not available.

  • 1,000 credits per billing period
  • Balance is set to 1,000 each time a billing period is paid. Unused credits do not carry over.
  • One credit = one multi-model run or one refinement.

You'll sign in with Google first, then pay through Stripe. No free trial.

Plan comparison
What you getFreeProTeam
PriceFree$49 per month$149 per month
Credits10 credits, one-time200 credits per billing period1,000 credits per billing period
RefillCredits do not refill.Balance is set to 200 each time a billing period is paid. Unused credits do not carry over.Balance is set to 1,000 each time a billing period is paid. Unused credits do not carry over.
Questions about plans or billing? Contact us
Sandboxed
Execution
Generated code runs in an isolated container with time, memory and output limits
Same tests
For every model
Function tasks are checked against one shared set of test cases
3 scores
Beyond pass / fail
Efficiency, reusability and editability for each candidate
Up to 5
Models per run
From OpenAI, Anthropic, Google, Cohere, DeepSeek
GET STARTED

Ready to verify
what AI writes?

The free plan includes 10 credits and needs no credit card. You'll sign in with the same Google account in the app.

Sandboxed execution
Google sign-in
Grew out of University of Waterloo research