How to answer a data migration question with unknowns visible

Answer boundary

State what is known before describing how data will move

Begin with the procurement record. Capture the current question, evaluation criterion, specification, data schedules, bidder assumptions, clarification answers, contract terms and any supplied samples. Then define the offered target service and migration responsibility. A question about “all legacy data” may hide separate obligations for active transactions, closed cases, audit history, documents, images, user accounts, permissions and records held only by an incumbent. The response should name those categories rather than let one broad noun become an unlimited promise.

Distinguish four evidence states. A buyer-stated fact is traceable to a current tender source. An observed fact comes from a sample or approved discovery evidence and carries its observation date. An assumption is a priced or designed basis that still needs confirmation. An unknown has no defensible value yet. The answer may promise a profiling and decision process for an unknown; it must not silently turn the unknown into a clean-data claim. FAR 15.305 and the UK guidance on assessing competitive tenders apply only in their own regimes, but both illustrate why a technical response should address the stated evaluation factors and the bidder’s ability to perform.

The fictional East Fen Housing Cooperative asks bidders to migrate repairs and asset information from two maintenance systems and a shared file store. Its schedule estimates 142,000 repair records and 31,000 assets, but provides no schema, duplicate rate or attachment inventory. The answer treats those figures as buyer estimates at the tender date. It commits to source profiling, a field and relationship map, two rehearsals and agreed acceptance evidence. It does not guarantee the final run time or exception count before access exists.

Source inventory

Count business populations, not only tables and bytes

A database inventory is necessary but insufficient. Build populations around the objects the service must preserve: properties, assets, tenants, contractors, repair orders, visits, charges, notes, documents and status history in the housing example. Record physical source, logical owner, primary and alternate identifiers, volume, growth, date range, change rate, file formats, encoding, relationships, security class, retention treatment and extraction route. Include records that exist only in reports, scheduled jobs, interface queues or document stores. A source-system owner should confirm the inventory, but confirmation does not prove data quality.

Take special care with identifiers and relationships. A property reference with a leading zero may be text in one system and a number in another. A repair can point to an asset that has been retired, or an attachment can be linked through a filename rather than a database key. Record orphan rates and many-to-one relationships before designing deduplication. The National Archives guidance on formats and DROID is specific to digital preservation, not a universal migration mandate, yet it demonstrates why file identity depends on format evidence rather than filename extensions alone.

Inventory the change path as well as the baseline. State whether the source stays writable during rehearsal and production, which timestamps are reliable, whether deletes are logged and whether late attachments can arrive outside the main database. If no change-data mechanism is known, the answer should offer bounded options such as a controlled freeze, repeated delta extraction or reconciliation of a defined high-risk population. “Near-zero downtime” is not a plan until the source can support the required capture semantics.

• Keep active, closed, archived and legally held populations separate.
• Record both logical business keys and physical source keys.
• Include files, notes, histories, permissions and interface state.
• Name the owner who can interpret each ambiguous population.
• Mark every count with its extraction time and filter.

Mapping register

Make every transformation explainable and reversible in review

The mapping register is the core work product. Each row connects a source element to a target element and records the source type, target type, business meaning, transformation, lookup version, default rule, null treatment, rejection behavior, lineage field, test and approving owner. One source field may feed several targets; several sources may converge into one target. Preserve those relationships instead of forcing a one-cell-to-one-cell spreadsheet. If a target has no source, state whether it is intentionally blank, derived, defaulted or populated after migration.

Transformations need decision classes. Format conversion may be mechanically testable. A code crosswalk needs a business owner because two legacy statuses may not mean the same thing in the new workflow. Deduplication needs match rules, collision treatment and an audit trail of kept and merged identifiers. Derived values require a named formula and effective period. Destructive choices, including truncation, excluded history and discarded attachments, need explicit buyer authority. The migration team can expose consequences and options; it should not make a records, privacy or legal decision through code.

For East Fen, legacy priority “2” means attendance within five working days in one system and a tenant-vulnerability override in the other. Mapping both to the same target priority would be valid at the type level and wrong in operation. The response therefore commits to a source-specific crosswalk, sample replay with service managers and a recorded decision on conflicting cases. That detail proves understanding without pretending the final crosswalk already exists.

Quality rules

Replace “clean data” with measures tied to use

Data quality has several dimensions. The UK Government Data Quality Framework distinguishes completeness, uniqueness, consistency, timeliness, validity and accuracy, while its current action-plan guidance asks teams to select rules according to purpose. Apply that reasoning to each critical population. A complete postcode column can still be inaccurate. A unique tenant record can still point to the wrong property. A valid closed date can precede the repair request. The answer should name the dimension, the calculation, the denominator, the tolerance, the evidence source and the consequence of failure.

Use exhaustive checks where the rule is cheap and deterministic, such as required keys, row counts, referential integrity, allowed formats and duplicate identifiers. Sampling is appropriate when accuracy requires human inspection, but the sample frame, selection method, size, strata and escalation rule must be visible. A clean sample does not prove the full population. For financially or operationally critical aggregates, reconcile control totals and trace selected records back to authoritative source evidence. Preserve exceptions rather than editing the source extract until the numbers pass.

Set defect treatment before rehearsal. A source defect might be corrected by the buyer, accepted with a documented operational workaround or excluded by authority. A mapping defect returns to the bidder. A target constraint may require design change. An unresolved interpretation becomes a decision, not a guessed transformation. The response should explain these routes and avoid a blanket obligation to cleanse every historical defect, especially when the tender has not defined acceptable quality or ownership.

• Define the population and denominator for every percentage.
• Distinguish missing records from missing values inside present records.
• Keep technical validity separate from factual accuracy.
• Name the tolerance owner and the action when it is exceeded.
• Retain source evidence for corrections and accepted exceptions.

Rehearsal

A rehearsal must expose defects, timing and manual work

A useful rehearsal starts from a controlled extract and ends with a decision. Record the source snapshot, filters, mapping version, target build, tool version, environment, compute and network conditions, start and finish time, throughput, rejected rows, warnings, manual interventions, reconciliation result and business tests. Use representative distributions, not only average records. Long notes, old encodings, large attachments, duplicate keys, invalid dates, missing parents and peak-volume deltas are often the records that determine whether the production window is credible.

Repeatability matters more than a polished demonstration. Re-running the same input with the same rules should produce the same target result and exception set. Re-running after a corrected rule should show the intended difference and no unrelated changes. AWS DMS documents row-level comparison and failure thresholds; Microsoft recommends checksums, counts and end-to-end validation; Google’s migration guidance calls for source-to-copy consistency checks. These are provider-specific examples. The bid should describe the verification needed for the proposed source and target, not imply that one vendor feature settles business meaning.

East Fen’s first fictional rehearsal finds that 3.8 percent of attachment names are duplicated and 611 open repairs lack the property key expected by the target. Those results do not mean the bidder failed. They trigger the planned routes: hash and relationship checks for files, a buyer decision on orphan repair handling, a revised extraction query and another rehearsal. A response that allows learning without changing the promised control is more credible than one that predicts a defect-free first run.

Cutover

Define what happens to every write during failure

Choose a cutover pattern from the service constraints. A full freeze can simplify consistency but creates downtime. Initial load plus deltas shortens the final window but depends on reliable change capture and ordering. Phased migration reduces one release step but can introduce coexistence, duplicate work and reconciliation across two systems. State the proposed pattern, the evidence needed to confirm it and the conditions that would force another option. The plan must cover jobs, interfaces, users and downstream reporting, not only the database connection.

Write checkpoints and authority into the runbook. Before migration, verify backup restoration, source snapshot, environment readiness, access, mapping version, support coverage and business blackout. During the run, compare progress and defect measures to thresholds. Before traffic moves, complete technical and business smoke tests. Name the person who may continue, pause, fix forward or recover. NIST contingency guidance, the current NIST OT backup guide and BSI backup guidance all emphasize prepared and tested recovery in their own contexts. They support the control principle but do not determine the buyer’s migration design.

Rollback becomes difficult once the target accepts new writes. Returning application traffic to a stale source can lose or duplicate repairs created after cutover. The response must state whether writes are paused, journaled, replicated back, replayed later or protected by a forward-recovery plan. Define the last reversible checkpoint and the evidence for it. AWS cutover guidance makes this distinction explicit. “We can roll back at any time” should be removed unless the proposed architecture and rehearsal prove how post-cutover changes survive.

Acceptance

Technical equality and business usability need separate proof

Reconciliation should operate at several levels. Structural checks compare expected objects, columns, types and constraints. Population checks compare counts by meaningful category and period, not only one grand total. Value checks use hashes, sums and selected record comparisons where transformations allow them. Relationship checks find orphans and broken cardinality. File checks compare counts, sizes, signatures and links. Business checks run high-consequence scenarios with migrated records, such as opening an emergency repair, viewing its visit history, finding its attachment and issuing the correct contractor instruction.

Define acceptance as a set of decisions. For each population and scenario, record the test, expected result, tolerance, actual result, exception, owner and status. Use states such as accepted, accepted_with_exception, correction_required, retest_required, excluded_by_authority and not_evaluated. A waiver needs the named authority and consequence. The bidder should not convert an unresolved buyer decision into acceptance simply because the overall count is high.

Decommissioning is later than technical cutover. Retain the source, extracts, mappings, logs, reconciliation evidence and approved exception record for the agreed period and access purpose. Confirm operational stabilization, audit and records obligations before removal. The EU Data Act contains switching and export duties for certain data-processing services, while privacy, records and contract rules may add other constraints. Their applicability is a legal and contractual decision. The RFP answer should identify the decision path without claiming a universal retention or deletion rule.

Response design

Let the method carry confidence while uncertainty stays visible

Organize the scored answer around the evaluator’s decision. Open with the migration outcome and scope. Follow with the answer basis, source and mapping method, quality controls, rehearsal sequence, cutover and recovery, reconciliation, acceptance, responsibilities and evidence. Use one compact assumption register where the buyer has not supplied facts. For each assumption, state why it matters, the current design basis, how it will be confirmed, who decides and what changes if it is wrong. This is more useful than scattering caveats through the prose.

Separate current capability from future work. Existing tooling, tested migration components and comparable experience can support the proposed method when their scope is stated. A past project does not prove this source quality or duration. A proposed profiling activity is not evidence that the data already passed. Do not name a specific tool as the answer unless the tender mandates it or the source and target assessment justifies it. Commit to outputs and controls that remain valid if the implementation technology changes.

Before release, trace every claim to the current tender source or approved bidder evidence. Reconcile volumes, dates, roles, environments, security measures, personal-data handling, downtime, staffing, price, acceptance and source-retirement language with the rest of the submission. AN-170 owns the data-specific response. The implementation article owns the wider delivery chain, the transition article owns service mobilization, the data-residency article owns geographic claims, and the security and privacy mapping articles own their respective control conclusions.