AI Vendor Checklist: the POPIA questions to ask before you sign

An AI vendor checklist is the set of questions you ask before signing, covering what the vendor does with your data, where that data goes, and what the arrangement costs as you scale. Under POPIA the written contract is not optional, and the responsible party stays accountable when it is missing.

Most South African businesses evaluating an AI tool are handed a demo and a price. Neither answers the questions that decide whether the thing is safe to run. This is the checklist, with the statutory wording that makes each question a requirement rather than a preference, and a note on the one number every vendor will quote at you.

The number every AI vendor quotes, and what it actually measured

Somewhere in the pitch you will be told that AI has become dramatically cheaper, usually as "280 times cheaper". That figure is real, it has a source, and it does not mean what it is used to mean.

It comes from Stanford HAI's 2025 AI Index Report. The sentence reads:

"The cost of querying an AI model that scores the equivalent of GPT-3.5 (64.8) on MMLU, a popular benchmark for assessing language model performance, dropped from $20.00 per million tokens in November 2022 to just $0.07 per million tokens by October 2024 (Gemini-1.5-Flash-8B), a more than 280-fold reduction in approximately 18 months."

Read what is being held constant. The measurement fixes one capability level, GPT-3.5's score of 64.8 on one benchmark, and asks what it costs to buy that level over time. It is a real and important finding about commoditisation at the bottom of the market. It is not a statement that your bill will fall 280 times, and the same report says so directly in the next breath:

"Depending on the task, LLM inference prices have fallen anywhere from 9 to 900 times per year."

A range of 9 to 900 is not a forecast you can put in a budget. It is a warning that the answer depends entirely on which task you are running. So the checklist question is not "is AI getting cheaper". It is "what does my specific workload cost at my specific volume, and what happens to that when the vendor's own costs move".

Why the checking got harder, not easier

You would expect that as these tools became normal business software, the disclosure around them would mature. The measurement says the opposite happened.

The 2025 Foundation Model Transparency Index, the third annual edition from Stanford's Center for Research on Foundation Models, scored the major model developers out of 100. The average fell to 40, down from 58 the year before, which puts it back near the 37 recorded when the index first ran in 2023. A year of progress was given back.

The detail matters more than the average. The report finds developers are least forthcoming about "training data and training compute as well as the post-deployment usage and impact of their flagship models". Those are not incidental categories. Training data and post-deployment usage are precisely the two things a South African responsible party has to be able to describe in order to meet its own obligations.

The spread is enormous, which is the useful part: IBM scored 95, while xAI and Midjourney each scored 14. Transparency is not a property of the industry. It is a property of the vendor, and it is therefore something you can select for.

Against that, Stanford's 2026 AI Index records global corporate AI investment reaching $581.7 billion in 2025, up 130 percent on the prior year. Money is arriving far faster than disclosure is. That gap is the entire reason this checklist has to be yours rather than theirs.

What POPIA actually requires you to ask

Most vendor conversations treat data protection as a box the vendor ticks. The Act puts the duty on you. Four sections do the work.

Section 21: there must be a written contract, and it must cover security

The wording of section 21(1) is short and leaves no discretion:

"A responsible party must, in terms of a written contract between the responsible party and the operator, ensure that the operator which processes personal information for the responsible party establishes and maintains the security measures referred to in section 19."

Note the structure. The obligation is on you to ensure it, and the instrument is a written contract. Clicking accept on terms of service is not obviously that instrument, and a vendor telling you their standard terms are sufficient is stating a legal conclusion about your business that they are not the ones accountable for.

Section 21(2) adds the notification duty: the operator "must notify the responsible party immediately where there are reasonable grounds to believe that the personal information of a data subject has been accessed or acquired by any unauthorised person." Ask what "immediately" means in their contract. If the number is in days, that is the answer to the question.

Section 19: you have to verify, repeatedly

Section 19(2) is the one that quietly makes this an ongoing job rather than a signing-day job. It requires the responsible party to take reasonable measures to:

1. "identify all reasonably foreseeable internal and external risks to personal information in its possession or under its control"
2. "establish and maintain appropriate safeguards against the risks identified"
3. "regularly verify that the safeguards are effectively implemented"
4. "ensure that the safeguards are continually updated in response to new risks or deficiencies in previously implemented safeguards"

Put point three next to the transparency finding above and the problem is obvious. You are required to regularly verify, and the average model developer became measurably less forthcoming about exactly the categories you would verify against. That is not a reason to avoid these tools. It is a reason to make disclosure a selection criterion, and to write the verification into a schedule rather than a memory.

Section 20: what the vendor may do without asking you

Section 20 requires an operator to "process such information only with the knowledge or authorisation of the responsible party" and to "treat personal information which comes to their knowledge as confidential".

The live question for AI vendors is training. If your inputs can be used to improve the vendor's model, that is processing beyond the purpose you engaged them for unless you have authorised it. Ask whether training on your data is off by default or off by request, and get the answer in the contract rather than in a support article the vendor can edit.

Section 72: where the processing physically happens

Almost every serious AI tool processes outside South Africa. Section 72(1) permits that on defined grounds, the first being that the recipient "is subject to a law, binding corporate rules or binding agreement which provide an adequate level of protection". Others include the data subject's consent and necessity for performing a contract with them.

So cross-border processing is lawful and normal. What is not acceptable is not knowing. Ask which country, which entity, and which of the section 72 grounds the vendor believes applies. A vendor who has thought about South African deployment will answer in one sentence.

The questions that are actually about money

Four, and none of them are the headline rate.

1. What is the unit, and what makes it move? Per user, per message, per token, per document. The unit determines whether your bill tracks headcount, activity or document length, and those grow at completely different speeds.
2. What happens at ten times the volume? Not the list price at that tier. Whether the tier exists, and whether the contract lets them reprice you into it mid-term.
3. What is the cost of leaving? Export format, whether history comes with you, and how long they keep it after termination. A tool you cannot leave has a price that is not on the invoice.
4. What is priced in rands and what is priced in dollars? A dollar-denominated subscription is a currency position you did not intend to take. It is not a reason to decline, it is a reason to know.

We have written separately about what these tools tend to cost in practice, in our guide to AI automation costs in South Africa. The reason it is a separate article is that price is the last question on this list, not the first.

How this stops being a manual job

A checklist that lives in somebody's head is a checklist that gets skipped on the third vendor, and section 19(2) asks for regular verification rather than a single good afternoon. The work reduces when four things get captured once and reused.

The register is captured once. Every tool that touches personal information gets one record: what it processes, which country, which section 72 ground, whether training is disabled, what the breach notification window says, and where the signed operator agreement is filed. Built once at the first vendor, it costs minutes at the next one because the questions are already written down.

The questionnaire stops being retyped. The same set of questions goes out to every vendor as a standing document. What changes between vendors is the answers, not the asking, and the asking is the part currently being rebuilt from memory each time.

The verification date is a scheduled reminder, not a good intention. Section 19(2)(c) says regularly. Something has to know that a given vendor was last checked eleven months ago and say so without being asked. That is the single highest value automation in this entire article, because it is the requirement most commonly met once and then quietly abandoned.

Renewal and review are the same event. The contract date and the verification date get held together, so the moment you have actual leverage is the moment the file in front of you is current.

None of that needs an AI tool to do it. It needs the record to exist in one place and the reminder to fire on its own. That is ordinary process work, and it is the difference between a business that can answer a regulator and one that starts a search.

What a bad answer sounds like

Three patterns, all of which are recoverable if the vendor is willing.

"We are fully POPIA compliant." POPIA compliance is a property of a responsible party's whole processing operation, not a certificate a supplier holds. The useful follow-up is simple: which sections do you mean, and will you sign an operator agreement that says so.

"Enterprise-grade security." This phrase carries no definition. Replace it with the specific questions: encryption in transit and at rest, who internally can read the data, retention period, deletion on termination, and what the breach notification window actually is in hours.

"Your data is never used for training." Good, if true and if it survives contact with the contract. Ask whether that applies to sub-processors as well, since most AI products are built on somebody else's model, and the sub-processor's terms are the ones that will govern in practice.

A vendor who answers these plainly is demonstrating exactly the disclosure the transparency index found to be scarce, which is itself a strong signal. A vendor who is annoyed by the questions has told you something useful about what the support relationship will feel like in month nine.

Frequently Asked Questions

Do I need a signed operator agreement for every AI tool?

If the tool processes personal information on your behalf, section 21(1) requires a written contract ensuring the operator maintains section 19 security measures. Where that contract is missing and the operator acts outside your instructions, you remain accountable for the non-compliance.

Is it legal to use AI tools that process data outside South Africa?

Yes, on the grounds in section 72(1). The most common is that the recipient is subject to a law, binding corporate rules or binding agreement providing adequate protection. The failure is not the transfer itself, it is being unable to name which ground applies.

Has AI really become 280 times cheaper?

That figure measures one thing: the cost of buying GPT-3.5 level performance on the MMLU benchmark, which fell from $20.00 to $0.07 per million tokens between November 2022 and October 2024. The same report puts task-dependent declines anywhere from 9 to 900 times per year.

Are AI vendors becoming more transparent as the market matures?

Measurably not. The 2025 Foundation Model Transparency Index average fell to 40 out of 100 from 58 the previous year. Individual vendors differ enormously though, from IBM at 95 to xAI and Midjourney at 14, so disclosure is a selectable criterion.

What is the single most important question to ask an AI vendor?

What happens to our data, stated in the contract rather than in marketing copy. It covers training use, retention, sub-processors, location and deletion, and a vendor's willingness to put the answer in writing predicts the rest of the relationship.

Where to start

Take the tools you already use rather than the one you are considering. Most businesses find they are further along than they thought, because the AI arrived inside software they already had. List them, and for each one answer four things: what personal information it sees, which country it processes in, whether an operator agreement exists, and when it was last checked.

The tools with no answer to question three are the priority. The tools with no answer to question four are the ones that will quietly become the problem, because section 19(2)(c) asks for regular verification and nothing in a busy month is going to raise its hand.

If you want that mapped across the whole business rather than tool by tool, that is what a Business Autopsy does: it traces what is actually running and what it touches before anybody recommends software. It is also worth reading why AI projects fail in South Africa, since vendor selection is where a surprising number of them are decided, and what to look for in an AI consultant in South Africa if you would rather not run this alone. Or start a discovery conversation about the tools you are already running.