All research
Methodology··Updated ·6 min read·VPN·Evidence

How We Score VPN Evidence: TheVPNMatrix Methodology Explained

How TheVPNMatrix rates the evidence behind its VPN assessments: the eight-component Evidence Quality Score and the 28 criteria it is applied to.

Revised November 2026. The component values, score range and criteria groups below were corrected to match the scoring code and the methodology page on TheVPNMatrix. The earlier version of this article gave a different range and grouping.

TheVPNMatrix compares VPN providers. Like many comparison sites it earns affiliate commission from some of the providers it covers. Its disclosure page says so and names the providers it earns nothing from. That is a conflict of interest, so the site records the evidence behind each assessment and scores how good that evidence is. This article explains the evidence score.

Five principles

  1. Transparency. Sources are public and linked.
  2. Verifiability. A reader can check each claim against its source.
  3. Reproducibility. The formula and component values are published here.
  4. Independence. The site states that its scores are set by its methodology and not by commission rates. A conflict of interest in a source lowers that source’s score.
  5. Timeliness. Evidence is dated, and old evidence is marked down on a fixed schedule.

The Evidence Quality Score (EQS)

Each claim on the site, for example “Mullvad runs RAM-only servers”, is backed by an evidence record. A record’s EQS is the sum of eight components:

EQS = Type + Grade + SQS + Ind + Scp + Rep + Age + COI

Six components add to the score and two (Age and COI) subtract. The values below are the ones in the scoring code as of November 2026.

Evidence Type (1 to 3)

  • Primary (P), 3. Official audits, court records, technical whitepapers, provider documents.
  • Community-Replicable (CR), 2.5. Reproducible tests and open datasets.
  • Secondary (S), 2. Investigative journalism and expert analysis.
  • Tertiary (T), 1. Forums, unverified blogs, marketing.

Evidence Grade (1 to 5)

  • A, 5. Formal, complete and structured, with methods disclosed.
  • B, 4. Clear methods and scope, from a credible outlet.
  • C, 3. Average detail, with partial methods.
  • D, 2. Thin or unclear, with key details missing.
  • E, 1. Very weak, unverifiable or hearsay.

Source Quality Score (SQS, 1 to 5)

  • 5. Academic, government, or verified independent.
  • 4. Reputable technology publication or established expert.
  • 3. Established technology blog.
  • 2. General technology website.
  • 1. Unverified.

Independence (Ind, 0 to 2)

  • 2. Fully independent of the provider.
  • 1. Operationally independent.
  • 0. Affiliated with the provider.

Scope (Scp, 0 to 2)

  • 2. System-wide.
  • 1. Moderate.
  • 0. Narrow.

Reproducibility (Rep, 0 to 2)

  • 2. Replicable by a third party.
  • 1. Retestable.
  • 0. Opaque.

Age (0 to −3)

  • 0. Up to 24 months old.
  • −1. More than 24 and up to 36 months.
  • −2. More than 36 and up to 60 months.
  • −3. More than 60 months.

Conflict of Interest (COI, 0, −1 or −3)

  • 0. None.
  • −1. Soft conflict, for example a provider describing its own service.
  • −3. Hard conflict.

There is no −2 value for COI.

The range

The highest possible EQS is 19 (3 + 5 + 5 + 2 + 2 + 2 + 0 + 0). The lowest is −3 (1 + 1 + 1 + 0 + 0 + 0 − 3 − 3).

One limit applies today. The records in the public dataset carry four of the eight components: Type, Grade, SQS and COI. For those records the site sets Independence, Scope, Reproducibility and Age to zero when it calculates EQS, so a record currently scores between 0 and 13. For example, the record for Mullvad’s RAM-only servers is Primary (3), Grade A (5), SQS 5, with a soft conflict of interest (−1) because the source is Mullvad’s own announcement. Its EQS is 12.

EQS rates evidence. It is separate from the 0 to 5 assessment each provider receives on each criterion, and from the weighted headline rating, both of which are described on the site’s methodology page.

The 28 criteria

TheVPNMatrix assesses each provider against 28 criteria in six weighted categories. The weights below are from its methodology page in November 2026:

CategoryCriteriaWeight
Core Privacy & Security548%
Infrastructure210%
Transparency & Trust419%
Business & Usability612%
Ethics & Additional Features36%
Specialized Technical Assessments85%

The methodology page lists every criterion with its minimum requirement and best case. Examples are jurisdiction, audited no-logs policy, leak protection, RAM-only servers, ownership transparency and incident response.

The database

When this article was first published in April 2026 the evidence database held 3,469 records. The site’s About page gives the current count. One of the evidence files is public as JSON. Each record in it has:

  • provider: provider name
  • criterion: which of the 28 criteria it supports
  • summary: the claim in one sentence
  • source: title and URL
  • type, grade, sqs, coi: the four scored components
  • lastCaptured: the date the source was captured

TheVPNMatrix’s evidence policy says every provider gets a structured evidence review at least twice a year. That page summarises the approach in broader terms. Where it differs from this article, this article describes the scoring code.

What this does and does not show

EQS tells a reader how much weight a piece of evidence can bear. It does not remove the conflict of interest that comes with affiliate funding, and it does not make the headline ratings mechanical: the 0 to 5 assessments still involve judgement. If you think an assessment is wrong, the record behind it shows what it rests on, and you can send a better source.

The same approach could be applied to other products whose technical claims buyers cannot check for themselves, such as password managers, messaging apps and age-verification services: write the criteria down, link a source for every claim, and publish how the evidence is scored.


Competing interest: TheVPNMatrix is operated by Cosmo Codex and earns affiliate commission from some VPN providers. The scoring code is not public.

Sources

  1. TheVPNMatrix, Methodology: the 28 criteria, six categories and weights. Accessed 16 November 2026.
  2. TheVPNMatrix, Affiliate disclosure and editorial independence. Accessed 16 November 2026.
  3. TheVPNMatrix, Evidence policy. Accessed 16 November 2026.
  4. TheVPNMatrix, evidence dataset (JSON): source of the Mullvad example record. Accessed 16 November 2026.
  5. Mullvad, migration to RAM-only VPN infrastructure: the source cited by that record.

Interested in working with us?

Get in touch to discuss our products, partnerships, or research collaborations.