RESEARCH v2.0 · REVIEW v1.1

Methodology

Research establishes facts. Review makes evidence-bounded editorial judgments. Commercial terms are kept outside the rating engine.

Research protocol

AiToolMap Research Methodology v2.0

Eligibility

AI materiality

AI must be a material component of the product value proposition, not incidental marketing.

Output: ELIGIBLE / HOLD_AI_RELEVANCE
Cadence: At discovery
Sources: Primary preferred
Review impact: None until verified

Eligibility

Usable product

Product must be usable autonomously by the public or by a clearly defined customer class.

Output: ELIGIBLE / HOLD
Cadence: At discovery
Sources: Primary preferred
Review impact: None until verified

Eligibility

Stable destination

Require a stable official destination or official repository/store record sufficient to identify the product.

Output: ELIGIBLE / HOLD_PRIMARY_UNREACHABLE
Cadence: At discovery
Sources: Primary
Review impact: None until verified

Exclusion

AI-adjacent only

Exclude ordinary products with a minor AI feature unless the AI functionality has a materially autonomous product proposition.

Output: REJECT_AI_ADJACENT / HOLD_AI_ADJACENT
Cadence: At verification
Sources: Primary + context
Review impact: Not reviewable

Exclusion

Unavailable / announcement only

Do not publish as an active tool if only announced, waitlisted without usable product, or represented by a landing page only.

Output: HOLD / COMING_SOON
Cadence: At verification
Sources: Primary
Review impact: Not rankable

Identity

Canonicalization

Rebrands, relaunches and versions must update the canonical tool record rather than create duplicates unless they are genuinely separate products.

Output: EXISTING_TOOL_UPDATE / VERIFIED_UPDATE
Cadence: At verification
Sources: Primary
Review impact: May trigger review update

Sources

Primary sources

Official site, pricing, documentation, changelog, release notes, official GitHub, app stores, security/privacy, affiliate pages and official announcements establish factual claims.

Output: PRIMARY
Cadence: Every verification/update
Sources: Highest
Review impact: Supports desk review

Sources

Independent sources

Credible editorials/tests, Reddit, Hacker News, professional communities, user reviews and credible demonstrations can corroborate performance and market experience.

Output: INDEPENDENT
Cadence: As needed
Sources: Secondary
Review impact: Can raise evidence confidence

Sources

Discovery sources

Product Hunt, AI directories, newsletters, social and launch sites are lead sources only; factual claims require independent verification.

Output: DISCOVERY
Cadence: Daily
Sources: Lead only
Review impact: No direct rating input

Daily scan

New products

Scan launches, newly public products, new products from known companies, relevant open-source projects and emerging vertical AI.

Output: UNVERIFIED_LEAD
Cadence: Daily
Sources: Discovery + primary verification
Review impact: New-entry review when verified

Daily scan

Product updates

Track meaningful features, model changes, integrations, modalities, APIs, rebrands, acquisitions and product pivots.

Output: UPDATE_SIGNAL
Cadence: Daily
Sources: Primary preferred
Review impact: Severity determines trigger

Daily scan

Commercial updates

Track pricing, free tier, trial, plans, API pricing and affiliate/referral program terms.

Output: COMMERCIAL_UPDATE
Cadence: Daily discovery; verify on change/new entry
Sources: Official pricing/affiliate pages
Review impact: Affiliate data never affects rating

Daily scan

Status

Track shutdowns, archived projects, inaccessible sites, acquisitions, rebrands and material privacy/security events.

Output: STATUS_UPDATE
Cadence: Daily
Sources: Primary + reputable reporting
Review impact: Critical events trigger review

Daily scan

Market signals

Track fast-growing tools, new task clusters and category emergence to inform taxonomy and coverage priorities.

Output: MARKET_SIGNAL
Cadence: Daily
Sources: Competitive/discovery sources
Review impact: No direct score effect

Update severity

CRITICAL

Shutdown, major pricing/free-tier removal, acquisition/rebrand, radical product pivot, or serious privacy/security event.

Output: CRITICAL
Cadence: Immediate
Sources: Primary + corroboration as needed
Review impact: NEEDS_REVIEW_UPDATE immediately

Update severity

MAJOR

Important capability, model, integration, API or material workflow/performance change.

Output: MAJOR
Cadence: On discovery
Sources: Primary preferred
Review impact: Flag for next review

Update severity

MINOR

Small feature, UI update, template or copy change with no material effect on user proposition.

Output: MINOR
Cadence: On discovery
Sources: Primary preferred
Review impact: Update data only

Cadence

Discovery

Run discovery and update scan at least once every day.

Output: DAILY_RUN
Cadence: Daily
Sources: All source tiers
Review impact: Feeds review triggers

Cadence

Data hygiene

Check duplicate identity, stale URLs, missing key fields and taxonomy consistency.

Output: HYGIENE_RUN
Cadence: Monthly
Sources: Database + primary sources
Review impact: May flag review

Cadence

Category audit

Deep review of taxonomy, category boundaries and competitive coverage.

Output: CATEGORY_AUDIT
Cadence: Quarterly
Sources: Database + market signals
Review impact: May require ranking recalibration

Governance

Commercial separation

Collect affiliate/referral and sponsorship data for monetization, but never use commercial terms as an editorial rating or ranking input.

Output: MANDATORY
Cadence: Always
Sources: Official program terms
Review impact: Excluded from Review Engine

Desk Review criteria

Only dimensions supportable from public evidence enter the standard Desk score.

25% WEIGHT

Capabilities & task coverage

Breadth and completeness of documented capabilities for the core task and target workflow.

Evidence: Official product surface, documentation, feature lists, supported workflows and credible independent corroboration.
Missing evidence: Do not infer unobserved capability or output quality; score only documented coverage.

20% WEIGHT

Value for money

Documented benefits and usable capacity relative to price, limits, credits, free tier and relevant alternatives.

Evidence: Official pricing, plan limits, credits, trial/free tier and comparable market context.
Missing evidence: If pricing or practical limits are unclear, reduce confidence and avoid strong value claims.

15% WEIGHT

Accessibility / usability evidence

Observed or clearly documented onboarding friction, availability, interface accessibility and technical requirements.

Evidence: Signup path, visible interface/demo, platform support, trial access, setup requirements and credible public walkthroughs.
Missing evidence: Do not claim hands-on ease of use without direct testing; phrase desk conclusions as evidence-based accessibility.

15% WEIGHT

Maturity & reliability evidence

Evidence of operational maturity, support, documentation, update cadence and organizational continuity.

Evidence: Docs, changelog, support, company/team signals, release cadence, public incidents and credible independent evidence.
Missing evidence: Do not convert absence of incident reports into a claim of high reliability.

15% WEIGHT

Workflow & integrations

How well documented integrations, APIs, export/import and collaboration features fit real workflows.

Evidence: Official integrations, API docs, supported platforms, export/import, automation and collaboration features.
Missing evidence: Score only integrations/workflows actually documented or independently verified.

10% WEIGHT

Transparency & trust

Clarity of pricing, limits, terms, privacy, data handling, ownership and security information.

Evidence: Official pricing, docs, terms, privacy/security pages, company identity and credible reporting.
Missing evidence: Unknown facts remain unknown; do not infer secure/private behavior from marketing language.

Editorial rules

RULE

Protocol

Definition

RULE

Review type

DESK uses public evidence only. HANDS_ON is allowed only when direct product testing was actually performed.

Never imply direct testing in a DESK review.

RULE

Output quality evidence

Values: NOT_ASSESSED, LIMITED_EVIDENCE, STRONG_EXTERNAL_EVIDENCE, HANDS_ON_VERIFIED.

Not a default weighted Desk criterion.

RULE

Insufficient evidence

A tool may be listed but unrated when evidence is insufficient. Never fill missing evidence with inferred pseudo-precision.

LISTED ≠ RATED ≠ RANKED

RULE

Overall internal score

Weighted average of the six v1.1 criteria only when enough evidence exists to support the component scores.

0–10 internal scale

RULE

Public rating

Public-facing rating is rounded/assigned in 0.5 increments to avoid false precision. Internal score may remain granular for ordering.

Example: 8.0 / 8.5 / 9.0

RULE

Rank eligibility

Requires a valid v1.1 rating and sufficient comparative evidence. Mere listing does not imply ranking eligibility.

YES / NO

RULE

Confidence — HIGH

Direct testing and/or unusually strong multi-source evidence supports the scored dimensions.

HIGH

RULE

Confidence — MEDIUM

Strong official/public evidence exists, but no sufficiently deep direct product test.

MEDIUM

RULE

Confidence — LOW

Evidence is sparse, partial, secondary-led or materially uncertain.

LOW / UNRATED if too weak

RULE

Update policy

Run rating recalibration at least weekly and at each new entry. MAJOR/CRITICAL research updates set NEEDS_REVIEW_UPDATE; MINOR updates do not force full review.

Operational rule

RULE

Commercial separation

Affiliate commission, referral economics, sponsorship and vendor promotion never enter rating or editorial ranking.

Mandatory

RULE

No-fake-testing rule

Never state or imply hands-on testing unless it was actually performed.

Mandatory