How panopticode works
Two kinds of programs. A frontend knows one language and its libraries: it reads one repository and writes down facts about its code. The engine knows no language at all: it reads the facts of all your repositories at once and finds the routes untrusted data can take to a dangerous call.
-
your repositories
webapp
federation
backend
-
one per language
panoptife-ts TypeScript, Svelte, GraphQL clients
panoptife-go Go, gRPC, gqlgen
-
CGF files
functions, how data moves in them, their calls
-
panopticode
+ the catalog: which calls are sources, sinks, sanitizers
-
routes
?q= → … → SQL, hop by hop
The example
One search term, three repositories.
Everything below runs on the small system that ships with panopticode's frontends: a SvelteKit web app (a page in the browser, and its server, a backend-for-frontend or BFF), a GraphQL gateway in Go, and a gRPC service in Go with a SQL store. We follow one value: the search term a user puts in the page's URL, all the way to a SQL query two services away.
-
usertypes a search term
?q=' OR 1=1 -- - the page reads it from its URL
-
browserwebapp: the Svelte page
POST /api/searchHTTP, but the page and the BFF live in one repository, so it is a plain call-
BFFwebapp: the SvelteKit server
- ◆
graphql:Query.searchByTokenGraphQL: a call into another repository -
federationthe GraphQL gateway · Go
- ◆
pb.Account/GetAccountgRPC: another repository again -
backendthe Account service · Go
webapp, runs in the browser · src/routes/search/+page.sveltesource ↗
1<script lang="ts">2 import { goto } from '$app/navigation';3 import { page } from '$app/stores';4 import { searchClient } from '$service/gateway/endpoints/search';56 let res = '';7 let typed = '';89 async function run() {10 const q = $page.url.searchParams.get('q') ?? '';source: ?q= enters as q11 res = await searchClient.call({ token: q });q goes to the BFF as { token: q }: POST /api/search12 await goto(q);13 }14</script>1516<input bind:value={typed} />17<button on:click={run}>search</button>18{@html res}
webapp, runs in the BFF · src/service/gateway/endpoints/search/index.tssource ↗
1import { browser, GatewayEndpoint, gql } from '$service/gateway/core/client';23export const $gateway = GatewayEndpoint.create({4 name: 'Search',5 gqlNode:6 !browser &&7 gql`$op, the GraphQL operation the BFF sends8 query Search($token: String!) {token arrives as $token9 searchByToken(token: $token)and leaves in searchByToken: graphql:Query.searchByToken10 }11 `,12 mapper,13});1415function mapper(value: { searchByToken: string }) {16 return value.searchByToken;17}1819export const searchClient = $gateway.getClientHandler('search');searchClient: the handler behind POST /api/search, called with { token: q }
federation · graph/graph.go · lines 90-98source ↗
90type queryResolver struct{ client pb.AccountClient }9192func (r *queryResolver) SearchByToken(ctx context.Context, token string) (string, error) {token arrives as the resolver's argument93 resp, err := r.client.GetAccount(ctx, &pb.GetAccountRequest{Number: token})token → req.Number, sent over pb.Account/GetAccount94 if err != nil {95 return "", err96 }97 return resp.Account, nil98}
backend · app/app.go · lines 18-25source ↗
18// GetAccount: req.GetNumber() (SOURCE) → FindAccountByNumber → Selectx (SINK).19func (i *Implementation) GetAccount(ctx context.Context, req *pb.GetAccountRequest) (*pb.AccountResponse, error) {req arrives, Number inside20 acc, err := i.storage.FindAccountByNumber(req.GetNumber())req.Number → FindAccountByNumber21 if err != nil {22 return nil, err23 }24 return &pb.AccountResponse{Account: acc}, nil25}
backend · store/store.gosource ↗
1package store23type Storage struct{}45// Selectx models a pgx-wrapper SQL exec sink: `query` is the tainted-carrying arg.6func (s *Storage) Selectx(query string) (string, error) {7 return "row:" + query, nil8}910// findBy takes a variadic arg (mirrors squirrel Where(pred, args...)).11func (s *Storage) findBy(pred string, args ...string) (string, error) {number arrives in args12 q := pred + args[0]q = pred + args[0]13 return s.Selectx(q) // SINKsink: Selectx(q) runs the SQL14}1516// FindAccountByNumber flows `number` through a VARIADIC call into the SQL sink.17func (s *Storage) FindAccountByNumber(number string) (string, error) {number arrives18 return s.findBy("number = ?", number)number → findBy19}
Part 1 · the frontends
Code becomes facts.
One frontend per language. Each one knows its language and the libraries that move data between services, such as gRPC, GraphQL and SvelteKit. It writes down what the code does. The only security fact it records is the one only it can see: which parameters receive requests, such as a handler's input or a resolver's arguments. Every other rule, what counts as a source, a sink or a sanitizer, comes from the catalog, which only the engine reads (step 06).
01 · frontends
Turn everything into functions and calls
Everything the engine analyses is a function or a call between functions. Most code already is that. Some code moves data without a call you can see in the text, and the frontend knows these patterns: it rewrites each one into a function or a call. The route has three, all in the web app.
-
1A Svelte template · +page.svelte
<input bind:value={typed} /> <button on:click={run}>search</button> {@html res}// in the page's function, +page.svelte:$script __pc_tpl(run); typed = __pc_bind(); __pc_html(res);The page becomes one function, and each place where the template moves data becomes a call. The button calls
run, which reads?q=(see the example's code above), so that read is now inside the page's function.bind:and{@html}become calls too, so a catalog rule (step 06) can name{@html res}as an XSS sink. -
2A handler that a library builds · BFF, endpoints/search/index.ts
export const searchClient = $gateway.getClientHandler('search');// the handler behind POST /api/search function searchClient(variables) { // untrusted return $op(variables); }To the compiler this is a constant made by a library it cannot see into. An adapter, a few lines of data per library, tells the frontend what the library does:
# adapters/bff-gateway.toml [[handler]] factories = ["getClientHandler", …] methods = ["call", …] route = "POST /api/{path}"So
searchClientis a request handler, and a request is untrusted input. -
3A GraphQL query in a string · the same file
gql` query Search($token: String!) { searchByToken(token: $token) } `function $op(token) { const answer = ◆ graphql:Query.searchByToken(token); return mapper(answer); }The frontend parses the query against the GraphQL schema: it asks for the field
searchByTokenof the typeQuery, passingtoken. That is a call into another service, and step 04 explains its name.
Go needs less of this. The Go frontend loads a module the way the Go compiler does, with every name and type resolved, and reads each function in SSA form, where every value is assigned exactly once.
02 · frontends
Map how data moves inside each function
For every function, the frontend follows each value from where it comes in (a parameter, the receiver, what a call returns) to where it goes out (a return, an argument of a call). The result is a small map. One rule matters more than any other: no line crosses a call. What the callee does with its inputs is not known yet, so the map stops at the call's inputs and starts again at its outputs. The engine fills that gap later, and a call to another service is just one more gap.
federation · graph/graph.go:92-98
func (r *queryResolver) SearchByToken(ctx context.Context, token string) (string, error) {
resp, err := r.client.GetAccount(ctx, &pb.GetAccountRequest{Number: token})
if err != nil {
return "", err
}
return resp.Account, nil
}
comes ingoes into the call to GetAccount
no line crosses the call: what GetAccount does is not known here
comes out of the callgoes out
token is an argument of a GraphQL resolver, so the frontend marks it untrusted: it is where requests come in. The map is simplified by one line: resp, as the whole answer, also reaches return 1. Step 05 shows the map exactly as it is written down.03 · frontends
Find out where each call goes
A map ends at its calls, so the frontend must also say where each call lands. The text does not always tell. A name imported from another file could be anything a library made. A method on an interface could belong to any type behind it. Three calls on the route, one of each kind:
browser · +page.svelte:11
searchClient.call({ token: q })
- Where does
searchClientcome from?imported from the BFF's endpoint file - What is it there?
$gateway.getClientHandler('search'): made by a library - What does the adapter say?
getClientHandlermakes a handler,.callcalls it
→ searchClientthe handler function from step 01
federation · graph/graph.go:93
r.client.GetAccount(ctx, …)
- What is
r.client?apb.AccountClient: an interface - A generated gRPC client?named
<Service>Client, in packagepb: yes
→ ◆ pb.Account/GetAccounta call into another service: step 04
backend · app/app.go:20
i.storage.FindAccountByNumber(…)
- What is
i.storage?a*store.Storage: a concrete type - Its method
FindAccountByNumber?exactly one
→ FindAccountByNumberone target, certain
04 · frontends
Give both ends of a network call one name
The two ends of a network call live in different repositories, read by separate runs of a frontend that never see each other. But both ends are built from the same schema, so each side can work out the same name on its own: the type or service and the field or method. Each side hashes its name into an id, and the engine links equal ids.
webapp · the client
gql`
query Search($token: String!) {
searchByToken(token: $token)
}
`
A query asks for fields of the schema's root type, Query.
graphql:Query.searchByToken
fbfd406e63d5ef84…
federation · the server
func (ec *executionContext) _Query_searchByToken(…) (string, error) {
…
return ec.Resolvers.Query().SearchByToken(ctx, args["token"].(string))
}
gqlgen generates one dispatcher per field, named after its type and field. The resolver it calls serves the field.
graphql:Query.searchByToken
fbfd406e63d5ef84…
federation · the client
type queryResolver struct{ client pb.AccountClient }
…
r.client.GetAccount(ctx, &pb.GetAccountRequest{Number: token})
A call on AccountClient, the client gRPC generated in package pb.
pb.Account/GetAccount
2f3f5d36df6f8880…
backend · the server
type Implementation struct {
pb.UnimplementedAccountServer
storage *store.Storage
}
func (i *Implementation) GetAccount(ctx, req) …
gRPC makes every server embed UnimplementedAccountServer; its method GetAccount serves the call.
pb.Account/GetAccount
2f3f5d36df6f8880…
05 · frontends
Write it all down as facts
Everything a frontend learned goes to disk as CGF, code graph facts: protobuf files, one per Go package or TypeScript directory. From here on nothing reads source code again. That is why the engine needs no change for a new language: a new frontend that writes the same facts is enough. Here is all the engine will ever know about SearchByToken:
function (*queryResolver).SearchByToken
id 208c2cf2d9c4 # a hash of repo, package, name, signature
untrusted parameter 1
serves ◆ graphql:Query.searchByToken
call 0 ◆ pb.Account/GetAccount, 3 arguments
map
receiver .client → call 0, argument 0
parameter 0 → call 0, argument 1
parameter 1 → call 0, argument 2 .Number
call 0, result 0 .Account → return 0
call 0, result 0 → return 1
call 0, result 1 → return 1
cgf/
├ webapp/ 3 files · 9 functions
├ federation/ 5 files · 24 functions
└ backend/ 3 files · 23 functions
Part 2 · the engine
Facts become summaries.
The engine is one Rust program. It loads the facts of any number of repositories and one policy file, and works out what every function does with untrusted data.
06 · engine
Load every repository as one program
The engine loads the facts of all three repositories into one program. A call into another service is still a loose end: it names a contract, and nothing connects it to the handler on the other side yet. Next to the facts, the engine reads the catalog: the security policy, as data. It names the calls that bring untrusted data in (sources), the dangerous ones (sinks), and the ones that make data safe (sanitizers).
webapp9 functions
federation24 functions
backend23 functions
one program: 56 functions, and 7 contracts not yet connected
- ◆ graphql:Query.searchByToken
- ◆ graphql:Account.details
- ◆ graphql:Account.archive
- ◆ pb.Account/GetAccount
- ◆ pb.Account/Archive
- ◆ pb.Feed/Download
- ◆ pb.Feed/Upload
catalog.example.toml: the two rules this route meets
[[sources]]
kind = "ts_url_query"
selector_regex = '^(URLSearchParams|URL)\.(get|getAll)$'
[[sinks]]
class = "sqli"
selector_regex = '\.(Getx|Selectx|Execx|Queryx|Getxx)$'
arg = "any"
URLSearchParams.get in the page and ends at Selectx in the backend. Most of the example catalog's rules match nothing in these three repositories; the engine lists those, so you can see what it was not looking for.07 · engine
Summarize every function, callees first
For every function, the engine writes a summary: if untrusted data comes in here, it goes out there, or it reaches this sink. A caller does not read a callee's map; it reads the callee's summary. So callees come first: the engine orders the functions so that each one is summarized after everything it calls.
Four backend functions on the route, in the order the engine summarizes them
-
1
Selectx(query)query→ resultA function like any other, so it gets a summary. What makes it dangerous is the catalog rule that marks a call to it as a sink.
-
2
findBy(pred, args...)pred,args→ ■ sqliCalls
Selectx(pred + args[0]): both parameters reach the sink. -
3
FindAccountByNumber(number)number→ ■ sqliPasses
numbertofindBy, and reads findBy's summary, not its code. -
4
GetAccount(ctx, req)req.Number→ ■ sqliThe gRPC handler. Its summary is what the other services will see in step 08.
When untrusted data reaches a call, the engine asks, in order
- Is it a sanitizer? Stop here.
- Is it a sink? Record a hit. The data still goes on through the call.
- Does the callee have a summary? Apply it.
- No summary, such as a library call? Assume that everything it returns is tainted, plus what a catalog rule adds, such as "
json.Unmarshalfills its second argument".
That last assumption keeps taint flowing through libraries the engine cannot see. It is also where most false positives come from.
Functions that call each other in a circle have no first one. The engine finds such groups and summarizes their functions together, round after round, until no summary changes.
08 · engine
Connect the services
After step 07, every repository is summarized on its own, and each call into another service is still a gap. The engine fills a gap by copying the handler's summary onto the contract, renumbered to match the caller's arguments. The caller then reads it like any other summary: no new algorithm. Whoever learns something is summarized again, which can change a contract it serves in turn, so the engine repeats until nothing changes. On this route, "reaches SQL" travels back one repository per round:
-
BeforeEach repository was summarized alone. Only the backend knows that its handler,
GetAccount, reaches SQL. Both contracts are gaps.webapppage, searchClient, $op◆ GraphQLgap federationSearchByToken◆ gRPCgap backendGetAccount■ reaches SQL -
Round 0Every contract gets a copy of its handler's summary. The gRPC contract
pb.Account/GetAccountnow says: the request reaches SQL.webapppage, searchClient, $op◆ GraphQLno SQL yet federationSearchByToken◆ gRPC■ reaches SQL backendGetAccount■ reaches SQL -
Round 1The federation is summarized again, reading that contract:
SearchByTokenreaches SQL. So the GraphQL contract it serves,graphql:Query.searchByToken, gets its new summary.webapppage, searchClient, $op◆ GraphQL■ reaches SQL federationSearchByToken■ reaches SQL ◆ gRPC■ reaches SQL backendGetAccount■ reaches SQL -
Round 2The web app is summarized again:
$op,searchClientand the page reach SQL. Nothing else changes, so the engine stops.webapppage, searchClient, $op■ reaches SQL ◆ GraphQL■ reaches SQL federationSearchByToken■ reaches SQL ◆ gRPC■ reaches SQL backendGetAccount■ reaches SQL
Part 3 · the findings
Summaries become routes.
A summary says that data gets through, not how. The last steps rebuild the how, hop by hop, and can check it a second time from the other end.
09 · findings
Rebuild the route
The summaries say that the search page reaches SQL, not through which calls. So the engine walks down from the source: in each function it finds the call whose summary produced the hit, steps into that callee, and repeats, across contracts into their handlers, until it stands at the sink. Each step is a hop.
-
● source
webapp
+page.svelte:$script -
call
webapp
+page.svelte:$script→URLSearchParams.get+page.svelte:10 -
call
webapp
+page.svelte:$script→searchClient· browser to BFF, POST /api/search +page.svelte:11 -
call
webapp
searchClient→$opendpoints/search/index.ts:19 -
◆ contract
webapp
$op→graphql:Query.searchByTokenendpoints/search/index.ts:7 -
◆ contract
federation
SearchByToken→pb.Account/GetAccount· field Number graph/graph.go:93 -
call
backend
GetAccount→FindAccountByNumberapp/app.go:20 -
call
backend
FindAccountByNumber→findBystore/store.go:18 -
■ sink
backend
findBy→Selectx· sqli store/store.go:13
10 · findings · optional
Check it backwards
With --backward, the engine checks every finding a second time, from the other end. It starts at the sink and follows the data back through the real maps of each function, not the summaries, keeping track of exactly which value and field it needs. If it reaches a source, the finding is confirmed. If it runs out of places to look, it is refuted. If it meets something it cannot model, it stays undecided.
- ■
findBySelectx(q), whereq = pred + args[0]look forargs[0] FindAccountByNumberfindBy("number = ?", number)look fornumberGetAccountFindAccountByNumber(req.GetNumber())look forreq.Number◆ pb.Account/GetAccountthe same request on both sideslook forreq.NumberSearchByTokenGetAccountRequest{Number: token}look fortoken◆ graphql:Query.searchByTokenthe client's argumenttokenlook fortoken$op·searchClientpassed straight throughlook for the handler's input+page.sveltesearchClient.call({ token: q })look forq- ●
+page.svelteq = $page.url.searchParams.get('q')a source: confirmed
--backward-unview, which maps the resolver's parameters back onto the client's arguments (the resolver takes ctx first); plain --backward stops there and leaves this finding undecided. With it, 13 of the example's 15 findings are confirmed and 2 stay undecided, both through gRPC streams; none is refuted. Refuted findings are dropped only if you ask (--backward-prune).11 · findings
What you get
One JSON array, one entry per finding, each with its whole route. On the example's three repositories that is 15 findings. The route this page followed is one of the first three.
$ panopticode taint --catalog catalog.example.toml \
--cgf testdata/cgf/webapp --cgf testdata/cgf/federation --cgf testdata/cgf/backend \
> chains.json
| starts in | crosses | ends at | findings |
|---|---|---|---|
webapp: the page's ?q=, the server load, the BFF endpoint | ◆ GraphQL, ◆ gRPC | backend: SQL | 3 |
| webapp: the page | nothing | the page: goto(q), open redirect; {@html res}, XSS | 2 |
| federation: its resolvers | ◆ gRPC, plain and streaming | backend: SQL | 6 |
| federation: a resolver that streams | ◆ gRPC stream, there and back | federation: SQL | 1 |
| backend: its gRPC handlers | nothing | backend: SQL | 3 |
- A finding is a route, not an exploit.
sqlimeans untrusted data reaches a call that runs SQL. Passed as a parameter of a prepared query, that is harmless, and it is still reported. panopticode never runs your code. - The catalog bounds recall. What it does not name is not looked for. Every run also prints the calls it could not see into; read that list on every new codebase.
- Scope matters. Code the frontend did not read has no map. A sink behind it is invisible.
- gRPC and GraphQL only, for now. Plain HTTP between services, queues, caches and database round trips are not followed.
panopticode impact re-extracts one repository at a merge request's head, reuses the cached summaries of everything the change did not touch, and reports the routes it did. Summaries are cached by what the callees do, not by their code, so an edit that keeps a function's behaviour invalidates nothing above it. With --pg, the call graph and the contract summaries go to Postgres, and panopticode query answers questions over them: which services can reach these changed files, and by which calls.