AI product engineering for legal technology

Authority model

A source is not authority without context

Legal retrieval needs more structure than semantic similarity. A court decision, statute, regulation, contract clause, commentary and client memorandum have different authority. Jurisdiction, court level, enactment or decision date, effective period, subsequent treatment and matter context can determine whether a passage is usable. Store these attributes as first-class metadata and make unresolved status visible to the user.

A citation is also not proof merely because the document exists. The product should verify that the cited passage supports the particular proposition, preserve a pinpoint reference and let the reviewer open the original. For generated drafting, distinguish quotations from paraphrases and inferences. If the retrieved record is incomplete, conflicting or outside the approved source boundary, abstention is a product capability rather than a defect.

Information boundaries

Matter isolation must survive every product path

Authentication is only the beginning. The system must decide which client, matter, document, annotation and derived artefact a user may access for every retrieval and tool call. That includes embeddings, caches, evaluation traces, support logs and generated exports. A shared vector index with a late filter can be unsafe if ranking, logging or diagnostics expose cross-matter information before the filter applies.

Model-provider terms, retention, training use, subprocessors and deployment location should be evaluated against the organization’s obligations and client instructions. These are procurement and architecture inputs, not marketing badges. Test the actual configuration and data flow. A product should support deletion, access revocation and legal hold requirements without claiming that any technical setting by itself guarantees confidentiality or legal professional privilege.

• Apply authorization before search, retrieval, tool execution and export.
• Keep matter identifiers and classifications on derived artefacts.
• Exclude sensitive content from unnecessary prompts, logs and analytics.
• Test ethical walls, changed team membership and external sharing.
• Make retention, deletion and incident paths operationally verifiable.

Evaluation

Evaluate the work lawyers actually need to verify

A single accuracy score hides the errors that matter. Build evaluation sets from representative, permissioned materials and define expected evidence with practitioners. Segment results by task, jurisdiction, authority type, document quality and consequence. A clause comparison can be assessed against labeled spans and playbook rules. Research support requires citation existence, proposition support, authority status and completeness. Draft quality still requires professional judgment.

Measure the burden transferred to reviewers. If a tool produces more polished text but requires line-by-line reconstruction of every source, it may move effort rather than reduce it. Record material correction categories, unsupported propositions, missed exceptions, false confidence and appropriate abstentions. Release thresholds should reflect the task’s risk, with a narrow initial scope and a tested rollback when model or source changes degrade behavior.

• Use reviewed cases that represent real task variation and hard edge conditions.
• Keep development, release and regression sets separate where practical.
• Test citation support, not only citation existence.
• Measure reviewer correction effort and consequential misses.
• Revalidate when authorities, prompts, models, tools or permissions change.

Professional control

The product supports responsibility instead of obscuring it

Professional duties differ by jurisdiction and role, so product design should make policy configurable and review explicit. The American Bar Association’s Formal Opinion 512 discusses competence, confidentiality, communication, supervision, candor and fees for lawyers using generative AI. It is not a universal rulebook, but it illustrates why a generic acceptance checkbox is inadequate for consequential legal use.

Show the status of each artefact: machine suggestion, reviewed extract, approved clause, internal draft or filed document. Preserve who approved a change and which source state they saw. The Courts and Tribunals Judiciary guidance similarly stresses personal responsibility, inaccuracy risk and care with private information. The engineering implication is direct: do not let automation, interface polish or a high aggregate score erase the accountable professional at the point of reliance.