Format-Preserving Encryption in Practice: FF1, Trade-offs, and Failure Modes

Most encryption assumes the rest of the system can tolerate ciphertext. A surprising amount of enterprise software cannot.

Format-preserving encryption exists for systems where the representation of a value — its length, alphabet, and shape — is part of the interface. This article explains how FF1 provides that compatibility, what it exposes in return, and what deploying it correctly requires.

---

The Compatibility Problem

The format constraint is the point. If you could just change the schema — update the column type, retrain the downstream validator, refactor every consumer — you probably wouldn't be reading this. You'd use AES-GCM and move on.

FPE exists because you often can't.

Standard encryption takes structured input and produces binary ciphertext. Once the ciphertext, authentication tag, and nonce are packaged for storage or transmission, a 16-digit PAN becomes a much longer textual value — character-set-incompatible, structurally unrecognizable. Your VARCHAR(16) column rejects it. Your card validation logic rejects it. The third-party payment processor that expects 16 digits doesn't know what to do with it.

In a greenfield system, fix the schema and move on. In financial services, healthcare, and large enterprise, you're often not in greenfield: COBOL programs that have been running since the 1980s. Fixed-width record formats. EBCDIC character encoding on mainframes. Validation logic embedded in copybooks that nobody alive fully understands. Downstream systems you don't own — third-party payment processors, core banking platforms, vendor integrations locked by contract. "Change the schema" is not a weekend project. Sometimes it's a multi-year initiative. Sometimes it's genuinely not possible.

FPE was designed for this constraint: protect the data without changing its format.

A 16-digit PAN goes in. A 16-digit protected value comes out. Same format. Same length. Same character set. The field still fits. Systems that enforce length and character set can continue accepting it. Semantic constraints such as Luhn check digits require separate handling. The mainframe copybook has no idea anything changed.

Determinism is what enables the rest of the value proposition. Within the same FPE profile, the same plaintext produces the same ciphertext every time. That means FPE-protected values can serve as join keys across systems — analytics, deduplication, grouping by customer ID, cross-system reporting — all on protected data, without a decryption step.

The same properties that make FPE compatible with existing systems — fixed domains, deterministic output, preserved length, stable relationships — also define its security limitations. The rest of this article is about that trade.

That unusual requirement produced several competing designs. NIST eventually standardized two, but today only one remains a sensible choice for new systems.

---

How FF1 Became the Standard That Survived

Today, the practical choice for standardized FPE is FF1. It was not the only construction NIST considered, and it was not the only one NIST initially standardized.

The history starts with FFSEM (Feistel Finite Set Encryption Mode), an early NIST submission for encrypting values from finite sets using an AES-based Feistel construction^[1] — useful but limited in generalizability.

In 2010, Bellare, Rogaway, and Spies published FFX — a generalized framework that addressed those gaps^[2]. FF1 was submitted to NIST as FFX[Radix], an instantiation of FFX parameterized over an arbitrary character alphabet. FF1 is the direct descendant of that lineage.

FF3 has a different origin entirely. It came from the BPS construction (Brier, Peyrin, Stern, 2010^[6]) — a separate balanced Feistel scheme over a number ring.

The lineage:

FFSEM
  │
  ▼
FFX framework ────► FF1 (FFX[Radix])

BPS construction ──► FF3  (separate branch)

NIST SP 800-38G (2016)^[3] standardized both — FF1 from the FFX lineage, FF3 from BPS. An FF2 was proposed and never approved.

FF3 did not survive scrutiny. Durak and Vaudenay demonstrated in 2017 that it failed to provide its intended security level over practical domains^[4]. NIST responded by proposing FF3-1 in its 2019 draft revision. Beyne later identified a weakness in the tweak schedule affecting both FF3 and FF3-1, which NIST cites as the reason the current revision draft removes the FF3 family entirely^[5]. The February 2025 second public draft retains only FF1 and raises the minimum domain size from 100 to one million distinct values. That document is still a draft; the 2016 edition remains the currently published standard. But the direction is unambiguous: new implementations use FF1.

If your library still defaults to FF3, change it.

Why FF1 survived becomes clearer once you understand the parameters it exposes and how they affect the cipher.

---

How It Works

FF1 is a ten-round, AES-based Feistel construction operating over strings drawn from a defined alphabet. Before it can encrypt a value, the implementation must define the value's alphabet, length, and tweak. Those parameters determine the domain it operates over — and whether separate systems will produce compatible ciphertext.

Domain, Alphabet, and Radix

FF1 has to turn characters into numbers before it can do arithmetic. The implementation must first define: which characters are permitted in this field, and in what order. That ordered list is the alphabet. The number of characters in it is the radix.

Alphabet: 0123456789
Radix: 10
Length: 16
Domain: 10^16 possible strings

Radix determines the number space for the arithmetic inside every Feistel round. During each round, FF1 converts one half into an integer and performs arithmetic modulo radix^m, where m is the length of the half being updated. That modulus keeps the resulting numeral string at the required length and within the configured alphabet — the mechanism that makes output format-compatible with input.

The consequence that matters:

domain size = radix ^ length

The cipher is mathematically correct in every row. The security is not equivalent. Not even close. That problem gets its own section.

radix^length is the configured FF1 domain: every numeral string the cipher can accept. The effective plaintext population is often considerably smaller. A 16-digit PAN field accepts 10¹⁶ possible numeral strings, but real PANs are constrained by issuer identification ranges, Luhn validity, and reserved patterns. A 2-letter state field has 676 configured values but only 50 populated ones. Those semantic constraints don't change FF1's configured domain, but they make plaintext guessing easier in practice. Evaluate the realistic plaintext population, not just the mathematical container.

Radix is only the size of the alphabet. Interoperability also depends on a separate, canonical mapping from each character to a numeral. FF1 operates on numeral strings — every system must use the same symbol-to-numeral mapping. ASCII and EBCDIC do not create different ciphertext merely because their encoded bytes differ; both can map the character 0 to numeral 0. Problems arise when one implementation feeds raw encoded values into the cipher, normalizes characters differently, or orders the alphabet differently. [0-9, a-z, A-Z] and [0-9, A-Z, a-z] are both radix-62, but in one mapping numeral 10 represents a; in the other it represents A — different modular arithmetic, different output. Two correct implementations of the same spec, silently producing different ciphertext, no error. Define the alphabet and the character-to-numeral mapping explicitly and test them across implementations.

Feistel Rounds

FF1 splits the input string X into two halves — A (left, length u) and B (right, length v), where u = floor(n/2) and v = n−u. These don't have to be equal; the unbalanced split is built into the spec. Run 10 rounds:

For round i in 0..9:
    m = u if i is even, else v          # active half-length alternates each round
    y = PRF(K, T, i, B)                # AES-based PRF over key K, tweak T, round i, and current B
    c = (NUMradix(A) + y) mod radix^m  # convert A to integer, add round output, take mod
    A, B = B, STRradix(c, m)           # swap halves; new A is old B, new B is result
Output: A || B

This pseudocode is conceptual — the actual FF1 spec includes formatting, padding, and expansion steps not captured here. What it shows: each round mixes left and right halves using AES as the cryptographic engine, operating in the number space defined by your radix. Modular arithmetic mod radix^m keeps the result in the same character space. That's the format-preservation mechanism.

A 9-digit SSN might split as u=4, v=5. A 16-digit PAN as u=8, v=8. The construction handles any supported input length directly, including odd lengths.

The Round Function

The round function invokes AES under key K over a formatted encoding of the FF1 parameters, tweak, round number, and current half-value. The key is not part of the input block — it keys AES, which then processes the formatted parameters. The tweak feeds into every round as part of that constructed input block — it's incorporated into what AES processes, not a separate XOR against the key schedule.

Tweak

A tweak is a public input that changes which permutation FF1 uses without requiring a separate key.

It is not secret. It is not an IV. It does not need to be random. A tweak can be a static string like "pan-prod-v1" — the requirements are that it is consistently encoded, documented, and within your implementation's configured maximum tweak length. The exact byte encoding and length of the tweak are both part of the interoperability surface. NIST permits fixed tweaks but strongly recommends variable tweaks as a security enhancement. Change the string, the encoding, or the length, and the cipher produces different output silently, with no error. Whatever you encrypt with, you must use the exact same tweak to decrypt.

What tweaks are for: domain separation. Same key, different tweaks per field, completely different permutations:

Key K + tweak "pan:prod:acme"   → permutation A
Key K + tweak "taxid:prod:acme" → permutation B

The same plaintext value in both fields now produces different ciphertext. An observer can no longer correlate those fields through direct ciphertext equality alone.

Of these parameters, radix/domain design and tweak design are where nearly every practical FPE failure begins.

Those parameters are not implementation trivia. The domain, tweak, and deterministic behavior define both what FPE enables and what it exposes.

---

What Preserving the Format Costs

Every decision on one axis constrains you on another. Most teams pick up an FPE library, call fpe_encrypt(), and don't encounter the decision surface until they hit it — usually during an incident, an audit, or a key rotation.

Small Domains Remain Small

A customer once asked whether we could use FPE to protect a gender field.

The answer was: technically, we could build something that accepts it. But what would be the point?

A field containing M or F has two meaningful plaintext values. Padding it, embedding it inside a larger numeral string, or running ten AES-backed Feistel rounds does not change that fact. An attacker who can verify guesses still has two guesses. Cryptography can transform the representation. It cannot manufacture uncertainty that does not exist in the data.

Birthdates create the same problem in a less obvious form. An eight-digit YYYYMMDD field has a configured decimal domain of 10⁸, but the realistic plaintext population is only the set of plausible human birthdates — roughly tens of thousands of values, not one hundred million. Knowledge about the population, such as approximate age, employment history, or even the application context, reduces that set further. The domain table looks safe. The actual exposure is not.

This is the distinction that matters throughout FPE:

• The configured domain is the full set of numeral strings accepted by FF1.
• The realistic plaintext population is the much smaller set of values the application will actually encrypt.

Security analysis has to consider both. A large syntactic field does not automatically imply a large guessing space.

Small domains sharply limit the protection FPE can provide — given the right attacker position. With an encryption oracle — or any other mechanism that verifies plaintext guesses — small realistic populations can be enumerated directly. A decryption oracle is a more complete failure: the attacker can submit the target ciphertext itself. Without an oracle, the attacker still needs auxiliary information to connect ciphertexts to plaintexts, but a small population gives them far fewer possibilities to work through.

ZIP codes: 100,000 configured values. US state codes: 676 configured, 50 realistic. 4-digit PIN: 10,000 configured values. For these fields, security depends heavily on preventing attackers from verifying guesses or accumulating plaintext-ciphertext pairs.

NIST's minimum is 100, with a recommended floor of one million (the current draft makes one million mandatory). Either way, it's a floor, not a target. A 10⁹ SSN domain is above it, but evaluate the actual attacker model for each field.

Variable-length data introduces a related problem: ciphertext length leaks plaintext length. FPE preserves length, so if your account numbers range from 8 to 16 digits depending on issuer, an observer who can see ciphertext lengths already knows something about the original values. The fix — padding to a canonical maximum length — introduces coordination cost: every system that touches the field must agree on padded length, padding character, and stripping behavior on decrypt.

Split field modeling is another failure mode. Some schemas store a PAN across two separate columns. Encrypting two eight-digit halves independently does not reduce the number of possible combined plaintexts, but it decomposes the problem into two independently enumerable 10⁸ domains. With a verification oracle, an attacker can attack each half separately and combine the results — roughly two 10⁸ searches instead of reasoning over one joint 10¹⁶ mapping.

Beyond Strict FF1: Schema-Compatible Transformations

Strict format-preserving encryption: output is in exactly the same alphabet and length as the input. A 9-digit SSN encrypts to a 9-digit value. A 16-digit PAN stays 16 digits. The output is syntactically indistinguishable from an unprotected value to systems that validate only length and alphabet. Check digits, reserved ranges, and other semantic constraints are separate.

One caveat: Luhn check digits are almost never preserved. A real PAN satisfies the Luhn algorithm. An FPE-protected PAN almost certainly doesn't — Luhn is a structural constraint the Feistel construction doesn't maintain without extra machinery. If your downstream validates Luhn, that's additional work.

Expanded-alphabet FF1: FF1 can also be configured with an alphabet larger than the application's plaintext alphabet. For example, FF1 configured as radix-62 (digits + letters) while the application supplies only nine-digit SSNs as input. The plaintext is still a valid radix-62 numeral string — it occupies a restricted digit-only subset of the configured domain. FF1 may then produce any nine-character radix-62 ciphertext. This is still vanilla FF1; the cipher doesn't know or care that the application restricts which inputs it sends.

The distinction matters:

Configured FF1 domain:            62^9 ≈ 13.5 quadrillion possible strings
Application plaintext set:        10^9 = 1 billion digit-only values
Ciphertext image of plaintexts:   10^9 values scattered across the 13.5 quadrillion-value domain

Roughly one in every 13.5 million radix-62 strings corresponds to an encryption of a valid nine-digit SSN. That sparse embedding has practical consequences:

• Protected values will almost always contain letters, making them visibly distinct from genuine SSNs. Any auditor, DBA, or developer looking at the field knows it's protected.
• A randomly modified ciphertext or a value decrypted under the wrong key is unlikely to land back inside the digit-only plaintext subset. The sparse embedding can help detect accidental corruption, uninformed ciphertext modification, and wrong-key decryption because most resulting plaintexts will fall outside the application's valid digit-only subset. This is useful redundancy, not cryptographic authentication: replaying or substituting a known-valid ciphertext still succeeds unless integrity is enforced separately.
• At the FF1 primitive level, the cipher genuinely operates over 62⁹, not 10⁹. For FF1's parameter checks, the configured domain is 62⁹, so this is not a small-domain FF1 instance. Plaintext-guessing attacks, however, still operate against the application's 10⁹ possible SSNs.

What it does not give you: more plaintext entropy. An attacker who knows the original value must be a nine-digit SSN and can verify guesses still has only 10⁹ plaintexts to test — not 62⁹. Equality leakage and frequency patterns among observed ciphertexts also remain.

A separate wrapper or encoding layer is required only when the output must guarantee additional structure — a reserved prefix, key-version marker, checksum, or particular character placement. Raw FF1 can produce any symbol from its configured alphabet in any position; it does not reserve metadata fields by itself.

Expanded-alphabet FF1 — and any wrapper that introduces characters or structure outside the original field format — breaks compatibility with systems that enforce the original character set. Strict same-alphabet FF1 does not. If downstream validation of the original character set is what you can't change, strict FPE is your option. If you have flexibility, expanded-alphabet FF1 often provides better auditability and more obvious protection signals.

Determinism Leaks Equality

Within a shared FPE profile, the same plaintext always produces the same ciphertext. That's what enables joins.

It's also what makes frequency analysis possible. In any field with low cardinality — job titles, risk categories, credit grades, state codes, any field where the same values repeat frequently — an observer watching the ciphertext column learns something about the distribution of the underlying plaintext. The mapping from ciphertext frequency to plaintext frequency requires knowing or estimating the plaintext distribution and having enough samples. It's not automatic. But the patterns are there. This is the ECB penguin problem, reissued for column-level data.

Even in large, high-cardinality domains (16-digit PANs, SSNs), where simple frequency inference becomes impractical, determinism still leaks equality and repetition. The same customer_id appearing in 10,000 records looks the same in ciphertext as in plaintext — the same ciphertext value in the same column, 10,000 times. Repeated service accounts, test records, known public identifiers, or any value that appears far more often than chance are visible. Domain size reduces classical frequency inference risk; it does not eliminate equality leakage.

If you're protecting a field with low cardinality and high sensitivity, evaluate whether FPE is the right tool. Deterministic masking with a purpose-built suppression table, or synthetic data substitution, may be a better fit.

Tweaks: Security vs. Referential Integrity

The recommendation: use meaningful tweaks, vary them by context, separate your domains. More contextual separation limits how observations from one use can be reused against another — but it also reduces joinability. That's the same trade-off throughout this section.

Here's the constraint. Cross-system analytics require identical ciphertext for the same plaintext. Identical ciphertext requires identical tweaks. The moment you vary tweaks by system, environment, or deployment context, your joins break. Two tables that should join on a protected customer ID won't match because they used different tweaks — silently. The values pass every format check. They just never equal each other.

If the encrypted value is the key — a foreign key, a cross-system lookup identifier, a join column in a data warehouse — the ciphertext must be identical everywhere that value appears. All systems using that field as a join key must agree on the same key, the same tweak, the same algorithm, the same radix, the same canonical length. Every parameter must match, or the cipher produces different output with no indication anything is wrong.

If referential integrity doesn't matter for a field — it's protected but never used for joins — you can vary the tweak aggressively. Use tenant ID, field name, environment, record type. Greater separation, provided the field does not need to join elsewhere.

Common mistakes: empty tweak (valid per spec, provides no domain separation in multi-field or multi-tenant deployments); reused tweaks across fields that shouldn't be correlated; different tweaks on systems that need to join on the same field. The last failure is the worst — completely silent, values look valid.

Treat your tweak scheme as schema. Write it down. Version it. Don't leave it as an "obvious" convention nobody documented, because conventions drift across implementations, library upgrades, and years.

Configuration Is Part of the Cipher

Key management gets most of the attention in FPE deployments. Configuration management is the recurring operational tax.

With authenticated encryption such as AES-GCM, saying decryption requires "the ciphertext and the key" is shorthand. The recipient also needs the nonce, authentication tag, algorithm parameters, and any associated data — values that can usually be packaged alongside the ciphertext in a self-describing envelope.

Strict FPE often has no room for such an envelope. Preserving the existing field format is the reason it was selected in the first place.

Reproducing an FF1 result requires more than possession of the key. Systems must agree on the algorithm, key version, exact tweak bytes or derivation rule, radix, ordered character-to-numeral mapping, normalization rules, input-length policy, and any treatment of punctuation, prefixes, padding, or check digits. A mismatch in any one of those inputs produces different ciphertext.

The dangerous part: most mismatches do not fail loudly. Both systems produce values of the correct length using characters from the configured alphabet. The values simply do not match.

That becomes an enterprise-scale problem when multiple systems send protected identifiers into a shared data lake. If one producer orders uppercase characters before lowercase, another uses a different tweak encoding, and a third still uses the previous key version, one customer may arrive under three unrelated ciphertext values. Joins fail, counts fragment, and analytics quietly become wrong while every pipeline remains green.

Treat the complete FPE configuration as a versioned cryptographic schema:

• algorithm and underlying cipher
• key identifier and version
• tweak construction and byte encoding
• radix and ordered character-to-numeral mapping
• canonicalization and normalization rules
• field segmentation and padding policy
• check-digit and structural-validity handling

Fields that must preserve referential integrity must share the same profile everywhere they appear. Fields that should not be correlated should deliberately use different profiles. The profile itself must be discoverable through trusted external metadata — a schema registry, dataset manifest, sidecar field, or centralized transformation service — because the ciphertext generally cannot tell you which profile produced it.

Key rotation is the hardest individual lifecycle event. Configuration management is what you're paying every day in between.

Key Rotation: The Hardest Part

Rotating the FPE key changes every ciphertext. That 16-digit protected PAN becomes a different 16-digit value under the new key. Everywhere that value exists — your database, report outputs, indexes built on protected values, downstream API consumers who cached the value, vendor integrations, historical data exports — now contains the old ciphertext. None of it matches the new ciphertext.

This is not a simple re-encryption job. It's a referential integrity problem wearing a key management problem's clothes. Rotation scope is determined not by what you own but by everywhere that protected value has traveled.

Under strict same-alphabet FF1, decrypting with the wrong key still produces another syntactically valid plaintext, so trying several keys and choosing the result that "looks valid" cannot identify the correct version. Expanded-alphabet profiles may reject many wrong-key results through application-level validation, but that is probabilistic redundancy, not authoritative key-version detection. Either way, the version authority must be external: a sidecar column, record envelope, partition epoch, table-level migration state, or trusted application context. During migration, systems may accept multiple key versions, but they select the correct key from that external metadata. Background re-encryption under the new key, controlled cutovers during low-traffic windows, and explicit deprecation schedules for old key versions all work — when the version authority is external.

Not rotating is also a documented choice some organizations make explicitly, with compensating controls and risk sign-off. The failure mode is treating rotation as a forgotten task rather than a deliberate decision.

One migration trap worth naming: during migration from plaintext to FPE, teams routinely create intermediate artifacts where the plaintext and the protected value sit next to each other — migration logs, staging tables, comparison exports, validation reports, backups taken "just in case." FPE is deterministic. That artifact is a ready-made plaintext-ciphertext codebook. Anyone who obtains it can resolve every protected value represented in the table without attacking FF1. The fix: never materialize plaintext-ciphertext pairs in any persistent store. Encrypt in-flight. Perform validation in memory and persist only aggregate results or error records that do not contain both values.

FPE Is Not Authenticated Encryption

AES-GCM provides confidentiality and integrity — if someone modifies the ciphertext, decryption fails with an authentication error. FF1 provides confidentiality only. There is no MAC. There is no authentication tag. A modified FPE ciphertext decrypts to a different plaintext, and the cipher won't tell you it was tampered with.

FPE alone does not protect against ciphertext substitution. If an attacker can modify a protected value in your database, the modified value decrypts to a different-but-format-valid plaintext — and depending on your validation logic, may pass every downstream check.

Some deployments accept this trade-off because integrity is enforced at another layer. For cases where modification detection matters, FPE needs to pair with integrity controls at a higher layer — application-level MACs, cryptographic signatures, or a containing authenticated structure. Database audit logs help with attribution after the fact, but they don't cryptographically authenticate the protected value itself.

NIST addresses part of this risk surface — especially small domains, tweak use, and implementation interoperability. The remaining operational problems are yours to solve.

---

FPE, Tokenization, and Deterministic Masking

Not every "format-compatible data protection" problem requires FPE. Before committing to FF1, consider what you're actually buying:

Tokenization vaults are not cryptographic FPE. A vault stores the real value, issues a format-valid token, maintains a lookup table. No Feistel construction or domain permutation is required; the vault mapping and its access controls become the primary security boundary. Vaults can be configured to issue stable tokens (same plaintext → same token), which preserves joins exactly as FPE does — without requiring a cryptographic permutation over the field's domain.

Non-reversible deterministic masking produces the same replacement for the same input without retaining a reverse mapping. It may use a keyed hash, PRF, or deterministic synthetic substitution. Joins remain possible, but the original value cannot be recovered. For use cases where you never need the original back — analytics, ML training, inference over aggregate patterns — this approach can work as well as FPE or better, with less infrastructure and often simpler auditability. Some products called "masking" retain a mapping table; once a reverse mapping exists, the design is operationally closer to tokenization and inherits the security and availability requirements of that mapping store.

AES-GCM alone doesn't preserve equality or enable joins. However, AEAD ciphertext is commonly paired with a separate deterministic index or token for lookup and joining — storing authenticated ciphertext for recovery alongside a keyed deterministic index for relational access. That's more infrastructure and may require schema changes, but it combines authentication guarantees with the ability to join on protected values.

Before picking any of these, work through the actual requirements:

1. Must the protected value retain the same length and alphabet?
2. Must identical plaintexts remain joinable across systems?
3. Must the original value be recoverable?
4. Must tampering be detectable?
5. Can the schema or surrounding application change?
6. Can a centralized vault be tolerated operationally?
7. What oracle or query access could a realistic attacker obtain?

"Do you need the original value back?" is one input to this decision, not the whole thing.

---

What NIST Says — and What the Revision Draft Changes

Published SP 800-38G (2016)

The 2016 standard is the currently published, normative reference. It specifies both FF1 and FF3. Key constraints:

• Domain minimum: at least 100 values (absolute floor)
• Domain recommendation: at least one million values
• Tweak: empty or fixed tweaks are permitted, but variable tweaks are strongly recommended as a security enhancement

Current Revision 1 Second Public Draft (February 2025)

The second public draft is not yet final. The direction it establishes:

• FF1 only. FF3 and FF3-1 removed entirely.
• One-million-value domain becomes mandatory, not merely recommended.
• Forward AES only. The draft no longer permits the inverse AES transformation as FF1's designated cipher function.
• No floating-point arithmetic. The revised specification explicitly prohibits floating-point implementations of FF1 following real interoperability failures caused by precision loss. The relevant length calculations must use integer arithmetic.

What FF1 does and doesn't guarantee. FF1 is designed to provide confidentiality over a configured numeral-string domain while preserving its length and alphabet. It does not automatically preserve structural validity: Luhn check digits on PANs, IBAN check digits, ISBN check digits — none of these survive FPE intact without additional machinery. It does not provide integrity guarantees.

Production Checklist

The following combines NIST requirements, draft direction, and deployment recommendations based on implementation experience. Before deploying FPE on any field:

1. Domain is large enough. The published 2016 standard requires a minimum of 100 values and recommends at least one million; the current Revision 1 draft would make one million mandatory. Think about your actual attacker — is enumeration feasible given their access level and time? Evaluate the realistic plaintext population, not just radix^length.
2. Alphabet mapping defined explicitly. Record the radix and the exact character-to-numeral mapping, including normalization and case handling. [0-9, a-z, A-Z] and [0-9, A-Z, a-z] are both radix-62 but produce different output from the same inputs.
3. Tweak strategy is written down. Different tweaks for different fields where appropriate. Fields used as join keys must use the same tweak consistently across every system that touches them.
4. FF1 only. Not FF3. Not FF3-1. If your library supports FF3-1, use it strictly for migrating existing legacy ciphertext — never for new data.
5. Key-separation boundaries are deliberate. Define whether keys are separated by environment, tenant, application, and purpose, and document where tweaks are being used for domain separation instead.
6. Rotation path exists before you deploy. Where does this protected value live beyond your primary database? Every system that caches or exports it is in scope for rotation. Know where the key version metadata lives — strict same-domain FF1 usually provides no spare capacity for it. Embedding version information requires deliberately changing the domain or adding a wrapper.
7. Cross-system round-trip tested. If two systems both touch the same protected field, write the integration test that verifies they produce matching ciphertext and round-trip correctly. Actually tested, not assumed.
8. No production plaintext-ciphertext mappings persisted. Check staging tables, migration logs, comparison exports, and backups. Test vectors and in-memory validation are different from retaining a production codebook.
9. Structural validity handled. If downstream validates Luhn, IBAN, or other check digits, that's additional machinery, not an assumption.
10. Implementation uses forward AES and integer arithmetic. If your library is old or unaudited, verify it is not using the inverse AES function or floating-point math for length calculations.

---

Implementations

For most of FPE's history, if you wanted it in production, you bought it from a vendor. Open-source FF1 implementations are now available in several languages.

Note on OpenSSL: mainline OpenSSL does not expose a documented, production-stable FF1 interface through EVP. There is an open proposal (GitHub #5131) but no finalized API in the released documentation. Do not assume FF1 availability in OpenSSL without verifying against your specific build and version.

Before using any library, verify output against NIST SP 800-38G test vectors. Test vectors establish a minimum interoperability check, not a security audit. Implementation bugs have appeared in published FPE specifications and early libraries. "Implements FF1" is a starting point, not a guarantee of correctness. Cross-system interoperability requires verifying alphabet ordering, encoding, and tweak serialization match exactly — not just that both sides produce valid-looking output.

---

Why This Matters Again for AI

FPE originally solved a compatibility problem for databases, payment systems, and legacy interfaces. LLM pipelines have created a new version of the same problem: protect sensitive values in structured data without destroying the structure the model needs to work with it.

The standard alternatives make different trade-offs for structured model inputs.

AES-GCM / AEAD: output is binary. Packaged with a nonce and authentication tag, the representation is longer and structurally unrecognizable. You have removed the value-level structure: its original format, equality relationship, and direct usefulness as a join key. AEAD ciphertext can be paired with a separate deterministic index for joining, but that requires additional infrastructure and doesn't solve the token cost problem for the model's context. (Exact savings depend on your tokenizer — FPE's standard numeral strings generally map to fewer tokens than the Base64 or hex encoding required by AEAD ciphertext, but measure against your specific model rather than assuming.)

Suppression / generic redaction: the model sees the field shape but can't do anything useful with the value. No analytics, no counting, no joining.

Deterministic masking: consistent, repeatable one-way replacement. Joins still work and format can be preserved. The original value is not recoverable without a retained reverse mapping. For use cases where you never need the original back, this is often simpler and lower-risk than FPE — and if that's your situation, don't default to FPE.

FPE: output stays in the same character space, same length. A model receiving FPE-protected 16-digit values sees values structurally consistent with the field type — though preserving the appearance of real data also means protected values may be indistinguishable from genuine card data to downstream systems, which is part of the exact-format trade-off. Within a shared profile, the protected representation retains equality relationships the pipeline can use for counting, grouping, deduplication, and joins across tables on shared protected identifiers.

The context window efficiency case is real, though the exact savings depend on your tokenizer. FPE-protected fields are generally shorter representations than AEAD ciphertext packaged with nonce, authentication tag, and version metadata. At sufficient scale — many records, many protected fields per record — that difference affects whether your dataset fits in context or gets truncated. Measure against your specific model and field types rather than assuming a fixed character-to-token ratio.

More importantly: if records share a foreign key — customer_id, account_id, any join column — FPE ciphertext is the same value wherever that key appears. FPE can preserve equality relationships and shared identifiers directly in the model input, reducing the preprocessing needed to reconstruct joins. With AEAD or random tokenization, that structure is invisible and has to be re-narrated in the prompt or reconstructed through pre-processing.

The caveat still applies: small domain problems don't disappear because the use case is AI. Encrypting a gender field before sending it to an LLM still gives you a field with two possible values. The domain math doesn't care what's on the other end.

---

Closing

FPE is a specific tool with a specific job. When that job is your job — large structured domains, format constraints you can't remove, referential integrity you need to preserve across protected data — it's the right answer. When it isn't, it's an expensive way to get false confidence.

The algorithm is standardized. FF1 implementations are accessible without a vendor contract. The hard part is the engineering judgment: knowing when to use it, how to configure it, and what you're accepting when you do.

Every property that makes FPE useful carries an obligation. Determinism enables joins — and makes frequency analysis on ciphertext possible. Format preservation makes ciphertext schema-compatible — and preserves enough structure to make small or constrained plaintext populations enumerable when guesses can be verified. Referential integrity makes analytics work — and makes a known plaintext-ciphertext pair valid everywhere the same profile is used.

None of these are bugs. They're the design. Using FPE responsibly means understanding what you're trading.

---

References

1. Terence Spies, "Feistel Finite Set Encryption Mode," NIST submission. csrc.nist.gov
2. M. Bellare, P. Rogaway, T. Spies, "The FFX Mode of Operation for Format-Preserving Encryption," Draft 1.1, 2010. csrc.nist.gov
3. NIST Special Publication 800-38G, "Recommendation for Block Cipher Modes of Operation: Methods for Format-Preserving Encryption," 2016. nvlpubs.nist.gov
4. F. B. Durak, S. Vaudenay, "Breaking the FF3 Format-Preserving Encryption Standard Over Small Domains," CRYPTO 2017. eprint.iacr.org
5. NIST SP 800-38G Rev. 1 Second Public Draft, February 2025. csrc.nist.gov
6. E. Brier, T. Peyrin, J. Stern, "BPS: a Format-Preserving Encryption Proposal," 2010.