How panopticode works

Two kinds of programs. A frontend knows one language and its libraries: it reads one repository and writes down facts about its code. The engine knows no language at all: it reads the facts of all your repositories at once and finds the routes untrusted data can take to a dangerous call.

  1. your repositories

    webapp

    federation

    backend

  2. frontends

    one per language

    panoptife-ts TypeScript, Svelte, GraphQL clients

    panoptife-go Go, gRPC, gqlgen

  3. facts

    CGF files

    functions, how data moves in them, their calls

  4. engine

    panopticode

    + the catalog: which calls are sources, sinks, sanitizers

  5. findings

    routes

    ?q= → … → SQL, hop by hop

The example

One search term, three repositories.

Everything below runs on the small system that ships with panopticode's frontends: a SvelteKit web app (a page in the browser, and its server, a backend-for-frontend or BFF), a GraphQL gateway in Go, and a gRPC service in Go with a SQL store. We follow one value: the search term a user puts in the page's URL, all the way to a SQL query two services away.

  1. usertypes a search term

    ?q=' OR 1=1 --
  2. the page reads it from its URL
  3. browserwebapp: the Svelte page

  4. POST /api/searchHTTP, but the page and the BFF live in one repository, so it is a plain call
  5. BFFwebapp: the SvelteKit server

  6. ◆ graphql:Query.searchByTokenGraphQL: a call into another repository
  7. federationthe GraphQL gateway · Go

  8. ◆ pb.Account/GetAccountgRPC: another repository again
  9. backendthe Account service · Go

source: untrusted data comes in◆ a call to another repository sink: a dangerous callred: the value we followSelect a box to see its code.

webapp, runs in the browser · src/routes/search/+page.sveltesource ↗

1<script lang="ts">2	import { goto } from '$app/navigation';3	import { page } from '$app/stores';4	import { searchClient } from '$service/gateway/endpoints/search';56	let res = '';7	let typed = '';89	async function run() {10		const q = $page.url.searchParams.get('q') ?? '';source: ?q= enters as q11		res = await searchClient.call({ token: q });q goes to the BFF as { token: q }: POST /api/search12		await goto(q);13	}14</script>1516<input bind:value={typed} />17<button on:click={run}>search</button>18{@html res}

webapp, runs in the BFF · src/service/gateway/endpoints/search/index.tssource ↗

1import { browser, GatewayEndpoint, gql } from '$service/gateway/core/client';23export const $gateway = GatewayEndpoint.create({4	name: 'Search',5	gqlNode:6		!browser &&7		gql`$op, the GraphQL operation the BFF sends8			query Search($token: String!) {token arrives as $token9				searchByToken(token: $token)and leaves in searchByToken: graphql:Query.searchByToken10			}11		`,12	mapper,13});1415function mapper(value: { searchByToken: string }) {16	return value.searchByToken;17}1819export const searchClient = $gateway.getClientHandler('search');searchClient: the handler behind POST /api/search, called with { token: q }

federation · graph/graph.go · lines 90-98source ↗

90type queryResolver struct{ client pb.AccountClient }9192func (r *queryResolver) SearchByToken(ctx context.Context, token string) (string, error) {token arrives as the resolver's argument93	resp, err := r.client.GetAccount(ctx, &pb.GetAccountRequest{Number: token})token → req.Number, sent over pb.Account/GetAccount94	if err != nil {95		return "", err96	}97	return resp.Account, nil98}

backend · app/app.go · lines 18-25source ↗

18// GetAccount: req.GetNumber() (SOURCE) → FindAccountByNumber → Selectx (SINK).19func (i *Implementation) GetAccount(ctx context.Context, req *pb.GetAccountRequest) (*pb.AccountResponse, error) {req arrives, Number inside20	acc, err := i.storage.FindAccountByNumber(req.GetNumber())req.Number → FindAccountByNumber21	if err != nil {22		return nil, err23	}24	return &pb.AccountResponse{Account: acc}, nil25}

backend · store/store.gosource ↗

1package store23type Storage struct{}45// Selectx models a pgx-wrapper SQL exec sink: `query` is the tainted-carrying arg.6func (s *Storage) Selectx(query string) (string, error) {7	return "row:" + query, nil8}910// findBy takes a variadic arg (mirrors squirrel Where(pred, args...)).11func (s *Storage) findBy(pred string, args ...string) (string, error) {number arrives in args12	q := pred + args[0]q = pred + args[0]13	return s.Selectx(q) // SINKsink: Selectx(q) runs the SQL14}1516// FindAccountByNumber flows `number` through a VARIADIC call into the SQL sink.17func (s *Storage) FindAccountByNumber(number string) (string, error) {number arrives18	return s.findBy("number = ?", number)number → findBy19}

Part 1 · the frontends

Code becomes facts.

One frontend per language. Each one knows its language and the libraries that move data between services, such as gRPC, GraphQL and SvelteKit. It writes down what the code does. The only security fact it records is the one only it can see: which parameters receive requests, such as a handler's input or a resolver's arguments. Every other rule, what counts as a source, a sink or a sanitizer, comes from the catalog, which only the engine reads (step 06).

01 · frontends

Turn everything into functions and calls

Everything the engine analyses is a function or a call between functions. Most code already is that. Some code moves data without a call you can see in the text, and the frontend knows these patterns: it rewrites each one into a function or a call. The route has three, all in the web app.

  1. 1A Svelte template · +page.svelte

    <input bind:value={typed} />
    <button on:click={run}>search</button>
    {@html res}
    // in the page's function, +page.svelte:$script
    __pc_tpl(run);
    typed = __pc_bind();
    __pc_html(res);

    The page becomes one function, and each place where the template moves data becomes a call. The button calls run, which reads ?q= (see the example's code above), so that read is now inside the page's function. bind: and {@html} become calls too, so a catalog rule (step 06) can name {@html res} as an XSS sink.

  2. 2A handler that a library builds · BFF, endpoints/search/index.ts

    export const searchClient =
    	$gateway.getClientHandler('search');
    // the handler behind POST /api/search
    function searchClient(variables) { // untrusted
    	return $op(variables);
    }

    To the compiler this is a constant made by a library it cannot see into. An adapter, a few lines of data per library, tells the frontend what the library does:

    # adapters/bff-gateway.toml
    [[handler]]
    factories = ["getClientHandler", …]
    methods   = ["call", …]
    route     = "POST /api/{path}"

    So searchClient is a request handler, and a request is untrusted input.

  3. 3A GraphQL query in a string · the same file

    gql`
    	query Search($token: String!) {
    		searchByToken(token: $token)
    	}
    `
    function $op(token) {
    	const answer =
    		◆ graphql:Query.searchByToken(token);
    	return mapper(answer);
    }

    The frontend parses the query against the GraphQL schema: it asks for the field searchByToken of the type Query, passing token. That is a call into another service, and step 04 explains its name.

Go needs less of this. The Go frontend loads a module the way the Go compiler does, with every name and type resolved, and reads each function in SSA form, where every value is assigned exactly once.

02 · frontends

Map how data moves inside each function

For every function, the frontend follows each value from where it comes in (a parameter, the receiver, what a call returns) to where it goes out (a return, an argument of a call). The result is a small map. One rule matters more than any other: no line crosses a call. What the callee does with its inputs is not known yet, so the map stops at the call's inputs and starts again at its outputs. The engine fills that gap later, and a call to another service is just one more gap.

federation · graph/graph.go:92-98

func (r *queryResolver) SearchByToken(ctx context.Context, token string) (string, error) {
	resp, err := r.client.GetAccount(ctx, &pb.GetAccountRequest{Number: token})
	if err != nil {
		return "", err
	}
	return resp.Account, nil
}

comes ingoes into the call to GetAccount

r.clientreceiver r.clientargument 0
ctxparameter 0 ctxargument 1
tokenparameter 1 · untrusted request.Numberargument 2, field Number

no line crosses the call: what GetAccount does is not known here

comes out of the callgoes out

respresult 0 resp.Accountreturn 0
errresult 1 errreturn 1
● token is an argument of a GraphQL resolver, so the frontend marks it untrusted: it is where requests come in. The map is simplified by one line: resp, as the whole answer, also reaches return 1. Step 05 shows the map exactly as it is written down.

03 · frontends

Find out where each call goes

A map ends at its calls, so the frontend must also say where each call lands. The text does not always tell. A name imported from another file could be anything a library made. A method on an interface could belong to any type behind it. Three calls on the route, one of each kind:

browser · +page.svelte:11

searchClient.call({ token: q })
  1. Where does searchClient come from?imported from the BFF's endpoint file
  2. What is it there?$gateway.getClientHandler('search'): made by a library
  3. What does the adapter say?getClientHandler makes a handler, .call calls it

→ searchClientthe handler function from step 01

federation · graph/graph.go:93

r.client.GetAccount(ctx, …)
  1. What is r.client?a pb.AccountClient: an interface
  2. A generated gRPC client?named <Service>Client, in package pb: yes

→ ◆ pb.Account/GetAccounta call into another service: step 04

backend · app/app.go:20

i.storage.FindAccountByNumber(…)
  1. What is i.storage?a *store.Storage: a concrete type
  2. Its method FindAccountByNumber?exactly one

→ FindAccountByNumberone target, certain

Most calls are like the last one. When an interface could hide several types, the Go frontend keeps the ones that can actually reach the variable and splits the confidence between them, 1/n each. With more than ten, it gives up, and the engine treats the call like a library call it cannot see into (step 07).

04 · frontends

Give both ends of a network call one name

The two ends of a network call live in different repositories, read by separate runs of a frontend that never see each other. But both ends are built from the same schema, so each side can work out the same name on its own: the type or service and the field or method. Each side hashes its name into an id, and the engine links equal ids.

webapp · the client

gql`
	query Search($token: String!) {
		searchByToken(token: $token)
	}
`

A query asks for fields of the schema's root type, Query.

graphql:Query.searchByToken fbfd406e63d5ef84…
same id: linkeddifferent id: no link

federation · the server

func (ec *executionContext) _Query_searchByToken(…) (string, error) {
	…
	return ec.Resolvers.Query().SearchByToken(ctx, args["token"].(string))
}

gqlgen generates one dispatcher per field, named after its type and field. The resolver it calls serves the field.

graphql:Query.searchByToken fbfd406e63d5ef84…

federation · the client

type queryResolver struct{ client pb.AccountClient }
…
r.client.GetAccount(ctx, &pb.GetAccountRequest{Number: token})

A call on AccountClient, the client gRPC generated in package pb.

pb.Account/GetAccount 2f3f5d36df6f8880…
same id: linkeddifferent id: no link

backend · the server

type Implementation struct {
	pb.UnimplementedAccountServer
	storage *store.Storage
}

func (i *Implementation) GetAccount(ctx, req) …

gRPC makes every server embed UnimplementedAccountServer; its method GetAccount serves the call.

pb.Account/GetAccount 2f3f5d36df6f8880…

05 · frontends

Write it all down as facts

Everything a frontend learned goes to disk as CGF, code graph facts: protobuf files, one per Go package or TypeScript directory. From here on nothing reads source code again. That is why the engine needs no change for a new language: a new frontend that writes the same facts is enough. Here is all the engine will ever know about SearchByToken:

function   (*queryResolver).SearchByToken
id         208c2cf2d9c4 # a hash of repo, package, name, signature
untrusted  parameter 1
serves     ◆ graphql:Query.searchByToken
call 0     ◆ pb.Account/GetAccount, 3 arguments

map
  receiver .client          → call 0, argument 0
  parameter 0               → call 0, argument 1
  parameter 1               → call 0, argument 2 .Number
  call 0, result 0 .Account → return 0
  call 0, result 0          → return 1
  call 0, result 1          → return 1
No source text, no variable names: positions, fields and calls. "untrusted" is the one security fact a frontend writes. The line from result 0 to return 1 is the one step 02 left out.
cgf/
├ webapp/      3 files ·  9 functions
├ federation/  5 files · 24 functions
└ backend/     3 files · 23 functions
The whole example: 11 files, 56 functions, 49 KB.

Part 2 · the engine

Facts become summaries.

The engine is one Rust program. It loads the facts of any number of repositories and one policy file, and works out what every function does with untrusted data.

06 · engine

Load every repository as one program

The engine loads the facts of all three repositories into one program. A call into another service is still a loose end: it names a contract, and nothing connects it to the handler on the other side yet. Next to the facts, the engine reads the catalog: the security policy, as data. It names the calls that bring untrusted data in (sources), the dangerous ones (sinks), and the ones that make data safe (sanitizers).

webapp9 functions

federation24 functions

backend23 functions

one program: 56 functions, and 7 contracts not yet connected

  • ◆ graphql:Query.searchByToken
  • ◆ graphql:Account.details
  • ◆ graphql:Account.archive
  • ◆ pb.Account/GetAccount
  • ◆ pb.Account/Archive
  • ◆ pb.Feed/Download
  • ◆ pb.Feed/Upload

catalog.example.toml: the two rules this route meets

[[sources]]
kind           = "ts_url_query"
selector_regex = '^(URLSearchParams|URL)\.(get|getAll)$'

[[sinks]]
class          = "sqli"
selector_regex = '\.(Getx|Selectx|Execx|Queryx|Getxx)$'
arg            = "any"
A rule matches the name of the function a call lands in, as the frontend wrote it. The value enters at URLSearchParams.get in the page and ends at Selectx in the backend. Most of the example catalog's rules match nothing in these three repositories; the engine lists those, so you can see what it was not looking for.

07 · engine

Summarize every function, callees first

For every function, the engine writes a summary: if untrusted data comes in here, it goes out there, or it reaches this sink. A caller does not read a callee's map; it reads the callee's summary. So callees come first: the engine orders the functions so that each one is summarized after everything it calls.

Four backend functions on the route, in the order the engine summarizes them

  1. 1Selectx(query)

    query → result

    A function like any other, so it gets a summary. What makes it dangerous is the catalog rule that marks a call to it as a sink.

  2. 2findBy(pred, args...)

    pred, args → ■ sqli

    Calls Selectx(pred + args[0]): both parameters reach the sink.

  3. 3FindAccountByNumber(number)

    number → ■ sqli

    Passes number to findBy, and reads findBy's summary, not its code.

  4. 4GetAccount(ctx, req)

    req.Number → ■ sqli

    The gRPC handler. Its summary is what the other services will see in step 08.

When untrusted data reaches a call, the engine asks, in order

  1. Is it a sanitizer? Stop here.
  2. Is it a sink? Record a hit. The data still goes on through the call.
  3. Does the callee have a summary? Apply it.
  4. No summary, such as a library call? Assume that everything it returns is tainted, plus what a catalog rule adds, such as "json.Unmarshal fills its second argument".

That last assumption keeps taint flowing through libraries the engine cannot see. It is also where most false positives come from.

Functions that call each other in a circle have no first one. The engine finds such groups and summarizes their functions together, round after round, until no summary changes.

08 · engine

Connect the services

After step 07, every repository is summarized on its own, and each call into another service is still a gap. The engine fills a gap by copying the handler's summary onto the contract, renumbered to match the caller's arguments. The caller then reads it like any other summary: no new algorithm. Whoever learns something is summarized again, which can change a contract it serves in turn, so the engine repeats until nothing changes. On this route, "reaches SQL" travels back one repository per round:

  1. BeforeEach repository was summarized alone. Only the backend knows that its handler, GetAccount, reaches SQL. Both contracts are gaps.

    webapppage, searchClient, $op ◆ GraphQLgap federationSearchByToken ◆ gRPCgap backendGetAccount■ reaches SQL
  2. Round 0Every contract gets a copy of its handler's summary. The gRPC contract pb.Account/GetAccount now says: the request reaches SQL.

    webapppage, searchClient, $op ◆ GraphQLno SQL yet federationSearchByToken ◆ gRPC■ reaches SQL backendGetAccount■ reaches SQL
  3. Round 1The federation is summarized again, reading that contract: SearchByToken reaches SQL. So the GraphQL contract it serves, graphql:Query.searchByToken, gets its new summary.

    webapppage, searchClient, $op ◆ GraphQL■ reaches SQL federationSearchByToken■ reaches SQL ◆ gRPC■ reaches SQL backendGetAccount■ reaches SQL
  4. Round 2The web app is summarized again: $op, searchClient and the page reach SQL. Nothing else changes, so the engine stops.

    webapppage, searchClient, $op■ reaches SQL ◆ GraphQL■ reaches SQL federationSearchByToken■ reaches SQL ◆ gRPC■ reaches SQL backendGetAccount■ reaches SQL
Filled: what changed in that round; the arrow marks a summary copied onto a contract. Without these rounds there is no route: before them, nothing outside the backend reaches SQL.

Part 3 · the findings

Summaries become routes.

A summary says that data gets through, not how. The last steps rebuild the how, hop by hop, and can check it a second time from the other end.

09 · findings

Rebuild the route

The summaries say that the search page reaches SQL, not through which calls. So the engine walks down from the source: in each function it finds the call whose summary produced the hit, steps into that callee, and repeats, across contracts into their handlers, until it stands at the sink. Each step is a hop.

  1. ● source webapp +page.svelte:$script
  2. call webapp +page.svelte:$script → URLSearchParams.get +page.svelte:10
  3. call webapp +page.svelte:$script → searchClient · browser to BFF, POST /api/search +page.svelte:11
  4. call webapp searchClient → $op endpoints/search/index.ts:19
  5. ◆ contract webapp $op → graphql:Query.searchByToken endpoints/search/index.ts:7
  6. ◆ contract federation SearchByToken → pb.Account/GetAccount · field Number graph/graph.go:93
  7. call backend GetAccount → FindAccountByNumber app/app.go:20
  8. call backend FindAccountByNumber → findBy store/store.go:18
  9. ■ sink backend findBy → Selectx · sqli store/store.go:13
9 hops, 3 repositories, 2 contracts crossed. This is how the engine prints a finding's route.

10 · findings · optional

Check it backwards

With --backward, the engine checks every finding a second time, from the other end. It starts at the sink and follows the data back through the real maps of each function, not the summaries, keeping track of exactly which value and field it needs. If it reaches a source, the finding is confirmed. If it runs out of places to look, it is refuted. If it meets something it cannot model, it stays undecided.

  1. ■ findBySelectx(q), where q = pred + args[0]look for args[0]
  2. FindAccountByNumberfindBy("number = ?", number)look for number
  3. GetAccountFindAccountByNumber(req.GetNumber())look for req.Number
  4. ◆ pb.Account/GetAccountthe same request on both sideslook for req.Number
  5. SearchByTokenGetAccountRequest{Number: token}look for token
  6. ◆ graphql:Query.searchByTokenthe client's argument tokenlook for token
  7. $op · searchClientpassed straight throughlook for the handler's input
  8. +page.sveltesearchClient.call({ token: q })look for q
  9. ● +page.svelteq = $page.url.searchParams.get('q')a source: confirmed
The same route as step 09, read from the sink back. Crossing the GraphQL contract needs --backward-unview, which maps the resolver's parameters back onto the client's arguments (the resolver takes ctx first); plain --backward stops there and leaves this finding undecided. With it, 13 of the example's 15 findings are confirmed and 2 stay undecided, both through gRPC streams; none is refuted. Refuted findings are dropped only if you ask (--backward-prune).

11 · findings

What you get

One JSON array, one entry per finding, each with its whole route. On the example's three repositories that is 15 findings. The route this page followed is one of the first three.

$ panopticode taint --catalog catalog.example.toml \
    --cgf testdata/cgf/webapp --cgf testdata/cgf/federation --cgf testdata/cgf/backend \
    > chains.json
The 15 findings of the example
starts incrossesends atfindings
webapp: the page's ?q=, the server load, the BFF endpoint◆ GraphQL, ◆ gRPCbackend: SQL3
webapp: the pagenothingthe page: goto(q), open redirect; {@html res}, XSS2
federation: its resolvers◆ gRPC, plain and streamingbackend: SQL6
federation: a resolver that streams◆ gRPC stream, there and backfederation: SQL1
backend: its gRPC handlersnothingbackend: SQL3
Read a finding carefully
  • A finding is a route, not an exploit. sqli means untrusted data reaches a call that runs SQL. Passed as a parameter of a prepared query, that is harmless, and it is still reported. panopticode never runs your code.
  • The catalog bounds recall. What it does not name is not looked for. Every run also prints the calls it could not see into; read that list on every new codebase.
  • Scope matters. Code the frontend did not read has no map. A sink behind it is invisible.
  • gRPC and GraphQL only, for now. Plain HTTP between services, queues, caches and database round trips are not followed.
→ docs/limitations.md
Beyond findings panopticode impact re-extracts one repository at a merge request's head, reuses the cached summaries of everything the change did not touch, and reports the routes it did. Summaries are cached by what the callees do, not by their code, so an edit that keeps a function's behaviour invalidates nothing above it. With --pg, the call graph and the contract summaries go to Postgres, and panopticode query answers questions over them: which services can reach these changed files, and by which calls.