Before It Goes Live: 10 Questions for Any AI-Built App

Before an AI-built app, chatbot or agent touches real data, someone should be able to answer ten questions about it. Most of them are the questions security teams have always asked about software. They get skipped because the app never went through the process that asks them. This page is the template: each question, why it matters, what a good answer looks like, the red flag, and which kind of testing settles it.

What should an AI risk assessment template cover?

An AI risk assessment template should cover the whole app around the model. Ask who owns it, how it was built, how it is audited, whether it fits your logging and identity stack, what it can access, what regulated data it holds, which policies apply, what a compromise would cost, what it connects to, and, for agents, whose credentials it uses. Each answer needs a named person and evidence: a repository link, a log query, a test report. Ethics and bias questions belong in AI governance, but they won’t tell you whether a stranger can pull your customer list out of the chatbot. If the builder can’t answer three or more of these, the app isn’t ready for production, however well it works.

We wrote this after the cases in The AI Risk We Keep Finding Is the App You Built: a client portal with no logs, a chatbot built from tutorials, a plumbing company’s agent with access to every customer account, and an innovation team that opened network paths through its own guardrails. Every one of them would have failed several of the questions below. None of them failed because AI wrote the code.

How to use this template

Use it two ways.

• If you built the app, answer each question in a sentence or two and attach the evidence. You know how it works better than anyone, so you’re the right person to start. You don’t need a security background for most of it.
• If you run security or IT, hand this page to the builder and review what comes back. An answer with no evidence is a guess. A question nobody can answer is your finding.

The same set works for a vibe-coded internal tool, an AI chat assistant on your website, a no-code automation wired to a model, or an agent with access to email and files. Questions 1 to 9 apply to all of them. Question 10 is for agents.

The ten questions

1. Who owns it, and who will maintain it?

Why it matters. Software without an owner doesn’t get patched. When the next framework flaw drops, someone has to know the app exists, know what it runs on, and have the access and the time to update it. In the cornerstone cases, the answer to “who owns it” was usually the person who built it, who had moved on to building the next thing.

A good answer names a person and a team, plus a backup, and says who gets the call when it breaks at 2 AM. It also says how long the app is expected to live and who decides when it’s retired.

Red flag: “Whoever built it,” a shared mailbox, or a vendor nobody has a contract with.

What answers it: this is an inventory question before it’s a testing question. The discovery work in an AI Security Assessment puts an owner on every app it finds. To find the apps in the first place, see finding the AI apps your teams already built.

2. How was it developed?

Why it matters. The development process decides which mistakes get caught before production. A model can write working code fast. It doesn’t set up the repository, the review step, the dependency checks or the secrets handling around that code. NIST’s Secure Software Development Framework (SP 800-218, version 1.1) describes those basics, and its community profile for generative AI (SP 800-218A) extends them to AI systems. You don’t need to adopt either in full. You do need the parts that would have caught the obvious problems.

A good answer says where the code lives (a managed repository, not a laptop or a chat history), who reviews changes before they ship, how dependencies are pinned and scanned, and where secrets are kept. If a no-code platform built it, say which one and who has admin rights to it.

Red flag: the code only exists inside the AI tool’s project, API keys are in the source, or “review” means the same person clicking deploy. Deleting a key from the code later doesn’t remove it from history either; see why deleting secrets from Git doesn’t work.

What answers it: a code and architecture review, which is part of our AI Application Security Review. For teams that need a standard to build against, a short secure SDLC policy is core security program development work.

3. How is it audited?

Why it matters. If something goes wrong, the logs are the only way to know what happened and to whom. Without them, you can’t scope an incident, and some regulations then treat every record in the system as exposed. Audit also means someone looks at the app on a schedule, before anything goes wrong.

A good answer says what the app logs (sign-ins, data access, admin actions, and for AI features the prompts and the tool calls), where those logs go, how long they’re kept, and who reviews them. It also names the last time anyone looked at the app’s security, and what they checked.

Red flag: “We reviewed the code with AI.” That’s useful, and it’s still one model checking another model’s work. It reads the code it’s given and nothing else. It won’t notice that nothing is being logged.

What answers it: a log review during testing. We check whether the attacks we run during a test show up in your logs, which tells you more about your audit trail than any configuration screenshot.

4. Does it fit into the existing security stack?

Why it matters. Your security program already does a lot of work: single sign-on, multi-factor authentication, central logging, endpoint protection, vulnerability scanning, the patch list. An app that sits outside all of that gets none of the protection, and your team gets none of the visibility.

A good answer says the app signs users in through your identity provider, sends its logs to your central logging or SIEM, runs on infrastructure your vulnerability scanner covers, and appears on whatever list your team uses when a critical patch drops.

Red flag: the app has its own username and password table, runs on a personal cloud account, or lives on a hosting platform that IT has never heard of.

What answers it: an architecture review that maps the app against your existing controls, plus a quick check of your asset and patch lists. If it isn’t on those lists, the next React2Shell-style flaw will find it before you do.

5. What controls limit what it can access?

Why it matters. Whatever the app can reach, an attacker who controls the app can reach too. For AI features this matters even more, because the model can be talked into misusing its access. The OWASP Top 10 for LLM Applications 2025 calls this Excessive Agency (LLM06) and names three root causes: excessive functionality, excessive permissions and excessive autonomy.

A good answer lists every data source, API and tool the app can use, the account it uses for each, and why it needs that level of access. Read access where read is enough. One customer’s session can’t reach another customer’s records. Actions with real consequences, like sending money, deleting data or emailing customers, need a person to approve them.

Red flag: the app uses an admin or service account “to make it work,” the chatbot can query the whole customer database, or nobody can list what tools the agent has.

What answers it: adversarial testing. We try to make the app or the model do things its role shouldn’t allow: one user reading another’s data, a chatbot revealing records it can see, an agent calling a tool it shouldn’t. That’s AI and LLM penetration testing for the AI parts and web application penetration testing for the rest.

6. Does it hold regulated or protected information?

Why it matters. Regulators don’t care how an app was built. If it holds patient records, customer financial data, student records or personal information covered by a state privacy law, the obligations that apply to your other systems apply to this one too. Data sent to a model provider counts as well: it’s leaving your environment for a third party.

A good answer classifies the data the app stores, the data it sends to any AI service, and the regulations that apply. It also says whether the AI provider’s terms allow that data, whether the provider keeps it, and whether you have the agreement your regulator expects, such as a business associate agreement for health data.

Red flag: nobody has checked what the app sends to the model, or the answer is “it’s only internal data,” when the internal data includes customer records.

What answers it: a data flow review that traces what goes in, what goes out and where it’s stored. Our compliance guides cover the specific obligations, for example the SEC cybersecurity rules for financial firms and OWASP AI security for the AI-specific risks.

7. Does it match the organization’s other policies?

Why it matters. You already have policies for acceptable use, vendor management, data retention, access reviews and change control. An AI-built app is subject to all of them. The model provider is a vendor. The no-code platform is a vendor. The app is a change to production.

A good answer lists the policies that apply and confirms the app follows them: the provider went through vendor review, the app’s data is kept and deleted on your retention schedule, and access to it is part of your periodic access review.

Red flag: the app was bought on a corporate card, or a free tier, and never went through procurement or vendor review. Or there’s an AI policy, and nobody who builds apps has read it.

What answers it: a governance review, which is part of the AI Security Assessment. A policy only describes what should happen, though. We made the case for testing what actually does in Your AI Governance Policy Won’t Stop a Prompt Injection.

8. What would happen to us if it were compromised?

Why it matters. This is the question that sets priorities. A compromised lunch-order bot is an annoyance. A compromised client portal can mean breach notices, regulator letters, lost clients and a ransom demand. The answer tells you how much testing and monitoring the app deserves.

A good answer describes the worst realistic outcome in plain terms: what data would be exposed, what an attacker could change or do, who you’d have to notify, and what it would cost to run the business without the app for a week. It also says whether your incident response plan covers this app.

Red flag: “Nothing, it’s just a prototype,” when the prototype is in daily use with real data. Or nobody has thought about it.

What answers it: a threat-model conversation during an assessment, and a tabletop exercise if the app is business-critical. The tabletop shows whether your team could scope and contain an incident in it.

9. What does it connect to, and what if those connections were compromised?

Why it matters. Most of the serious problems we find in AI-built tools sit in their connections: a firewall rule opened so the tool could reach production, an API key with full access to a SaaS platform, a model with read access to a shared drive. Every connection is a path in both directions. If the app is compromised, the attacker inherits its connections. If a connected system or vendor is compromised, it can reach back into the app.

A good answer maps every connection: which systems and data sources, which accounts and keys, which direction data flows, and what network paths had to be opened. For each one, it says what you’d do if that connection or that system were compromised, including how to cut it off quickly. It also covers the software supply chain: which packages, plugins and model providers the app depends on, and how you’d know if one were compromised.

Red flag: connections nobody wrote down, credentials shared between apps, or network paths opened “temporarily” months ago. If the app has AI tool connections such as MCP servers or plugins, and nobody can list them, that’s a red flag too.

What answers it: an architecture and data flow review, followed by testing the connections themselves. Our AI Application Security Review covers integration points and trust boundaries; AI agent security testing covers MCP servers and agent tool connections. For how an attacker abuses those connections, see how attackers hijack an AI code review agent.

10. For agents: who is acting, and what credentials does the agent hold?

Why it matters. This is the part that’s newer with AI. An agent acts on someone’s behalf. It signs in, reads mail, files tickets, merges code or places orders, often with that person’s own credentials. To every system it touches, the agent looks like that person. When something goes wrong, your logs may not be able to tell you whether the person did it or their agent did.

We use agents ourselves, and the productivity gain is real. So do clients: we see agents in development environments with the autonomy to review, build and push code, and prospects trying commercial and open-source agents. Using them is fine. The goal is to know which keys each one holds.

A good answer says whose identity the agent uses, which credentials and tokens it holds, what scope each one has, how long they last, and how to revoke them. Ideally, the agent has its own identity with its own limited permissions, so its actions are logged as the agent’s. Actions with real consequences need a person to approve them.

Red flag: the agent runs on a person’s full account, holds long-lived tokens with broad scopes, or nobody can say how to shut it off.

What answers it: AI agent security testing, where we test what the agent can reach, whether instructions planted in a document, issue or web page can redirect it, and whether its actions are traceable.

The template

Copy this into a document or ticket for each app. Fill one row per question. An empty row is a finding.

Revisit the answers when the app changes in a meaningful way: new data, new connections, new users, or a new AI feature. A yearly check is the floor.

When the answers run out

Most teams can answer the first two or three questions from memory. Questions 5, 8 and 9 are where it usually stops, because answering them takes testing, not knowledge of the app. That’s normal. The builder knows what the app is supposed to do. A test shows what it can be made to do.

If you’ve run this template against your AI-built apps and found more blanks than answers, talk to us. We’ll help you fill them in: find the apps, put an owner on each one, and test the ones that matter before an attacker does. For the full range of what we test, see our AI security services.