Answer an RFP scalability question with measured limits

Demand model

Turn the buyer number into work the service must finish

Copy the scalability question and every number attached to it. Then find the definitions, service scope, transaction descriptions, forecast periods and performance criteria elsewhere in the procurement pack. Northmere uses 120,000 concurrent users as a headline, but the useful question is what those candidates do at the same time. Viewing a cached result, submitting an appeal with documents and paying a fee place different demands on the service. A single virtual-user count cannot represent all three.

Build a workload from completed business operations. Give each operation a start and successful end, its share of demand, expected arrival rate, active concurrency, think time, payload range, data access pattern and dependency calls. Add background work such as virus scanning, document conversion, notifications, reconciliation and reporting. Keep registered accounts, logged-in sessions, open connections, requests and successful outcomes as separate units. The relationship between them depends on user behavior and system design.

Demand also has a shape. Results publication may create a steep opening burst, while appeals arrive over several hours and attachments continue processing after the browser confirms receipt. State the ramp, peak duration, geographic distribution and expected growth horizon. If the buyer has supplied only a user count, mark the missing arrival and operation mix as buyer facts to clarify or approved sizing assumptions. Do not hide them inside a test script.

Evidence state

Keep a measured result separate from a forecast or design claim

A proposal usually draws on several kinds of capacity information. A completed test is an observation about one release, environment, workload and time. A calculation derives another value from stated inputs. A forecast estimates later demand. A design statement explains how capacity can change. A supplier quota or buyer interface may condition the result. Give each statement a state before using it. A measured 18,000-session run cannot become proof of 120,000 sessions merely because the architecture allows more instances.

Derived evidence can still be useful when its relationship is honest. If a component was tested separately, show which complete service path it represents and what remains untested. If a smaller environment is used, document the dimensions that scale proportionally and those that do not. Stateful data, shared queues, rate limits, coordination work and fixed dependencies often break a simple multiplication. A model should expose those terms and a validation point rather than turn them into an exact-looking total.

ISO/IEC 25030 provides a framework for defining and governing quality requirements, including their use in a call for tender. ISO/IEC 25040 provides an evaluation framework but does not prescribe one test method. That distinction matters here. A standard can support the structure of the evidence, yet it does not certify Northmere capacity or choose the buyer target. The current product and test records must do that work.

Test boundary

Describe test equivalence without publishing the system map

Freeze the software release, enabled functions, data model, dependency versions and configuration class before the run. Record how the test environment relates to the offered service in capacity, network path, regional placement, storage state and scaling policy. The buyer does not need hostnames, addresses, administrative endpoints or a component-by-component resource inventory. It does need to know whether the evidence came from the offered product and whether any difference could change the result.

Test data must exercise the same work. Use representative account relationships, authorization checks, search cardinality, document counts, file-size distribution, validation failures and historical records. Preserve privacy by using approved synthetic or de-identified data and by testing the properties that affect performance rather than copying a customer database. Warm caches and empty stores can both be valid scenarios, but neither should be passed off as the only production state.

Check the load generator separately. Confirm that it can produce the planned arrival curve without exhausting its own processors, network or connection pools. Synchronised test clients can also create an artificial burst unlike the buyer pattern. Retain client-side timings, service-side observations and completed business counts. Disagreement between them is evidence to investigate, not a reason to publish the faster graph.

• Name the release and offered functions under test.
• Record material environment similarities and differences.
• Use representative data shape, volume, permissions and state.
• Include external calls and background work that affect completion.
• Prove that the test client can generate and observe the intended load.
• Remove credentials, addresses, defensive thresholds and exploitable topology from the release copy.

Measurements

Count finished work and show what happened to the slow requests

Throughput should count the useful outcome named in the workload model. Ten thousand accepted upload requests are not ten thousand stored, scanned and retrievable appeal files. Define completion and verify it with a downstream record. Separate success, business rejection, technical failure, timeout, abandonment, retry and duplicate. If a client retries after a slow response, raw request rate rises while useful throughput may remain flat or fall.

Report a distribution rather than one average. At each load step, retain at least the chosen median and tail percentiles for every critical operation, together with errors and correctness failures. Averages can improve when the service rejects difficult work early. A percentile can also look healthy if failed operations disappear from the timing population. State the eligible population for each measure and reconcile submitted, accepted, completed and failed totals.

Observe queues and constrained resources alongside user results. Queue depth alone is incomplete, because a stable count can hide increasingly old work. Track oldest-item age, processing delay and drain time. Record resource pressure at a safe level of abstraction, such as compute, memory, connection, storage, external quota or serialized work. OpenTelemetry defines a stable HTTP server request-duration metric; its active-request metric is opt-in and still marked Development. Both describe protocol activity. The proposal still needs business completion measures for the buyer journey.

Operating envelope

Stop calling the last green point a maximum

Increase load in controlled steps and hold each step long enough to reach the relevant state. Plot demand, useful completions, latency, errors, queue age and resource pressure on the same timeline. The important point is often the knee where additional demand produces less useful throughput, longer tails or accumulating work. If the test ends while every curve remains stable, it has shown a tested lower bound, not a maximum capacity.

Define the offered operating envelope below the observed instability. The margin should account for forecast uncertainty, failover or maintenance states, noisy dependencies, uneven partitioning, scaling delay and normal change. It is a decision supported by evidence, not a universal percentage. Record the first limiting condition and whether it belongs to the service, a supplier, the buyer, the test harness or an unresolved interaction.

Go beyond the intended range under controlled conditions. Observe whether the service queues, rejects, slows, drops optional work or corrupts state. Verify retry treatment and recovery after demand falls. Google SRE guidance recommends testing capacity limits and overload behavior because resource exhaustion can create cascading failure. That is useful engineering guidance, not proof of a particular product. Northmere evidence must show what this offered service did.

Expansion path

Measure the delay between a scale trigger and useful capacity

Automatic scaling is a sequence, not a binary feature. Record the signal, observation window, decision rule, requested change, provisioning delay, readiness condition, traffic distribution and point at which useful throughput increases. A new worker that is running but still loading state or receiving no work has not added usable capacity. Measure scale-in as well, because an aggressive return can recreate pressure or interrupt queued work.

Test the fixed parts. Shared stores, ordered queues, external APIs, licenses, regional quotas and buyer networks may limit the service after stateless processing expands. State movement and rebalancing can consume capacity during the change. The Azure scale-out guidance describes scalability as throughput gain relative to added resources and warns that bottlenecks and synchronization limit proportional growth. Use that relationship as one bounded method, not as a claim that every layer doubles.

Give demand beyond the current envelope an owned route. The route might require a tested configuration change, quota increase, additional region, data partition, supplier agreement or planned engineering work. Name the trigger, evidence, lead time, cost treatment, acceptance test and decision owner. If the action is only on a roadmap, label it proposed and keep it out of the current-capacity claim.

Evidence quality

Preserve enough context to repeat the result after a change

Retain the test plan, workload version, data recipe, environment record, release identifier, start and end times, load-controller logs, service observations, result calculations, incidents and deviations. Record the number of repetitions and variation between them. IETF RFC 8239 is limited to data-centre network benchmarking, but its insistence on reported iterations and repeatability is a useful bounded example. Do not apply its network-specific thresholds as a universal application rule.

Link every claim to the exact evidence snapshot and calculation. Another reviewer should be able to reproduce the published number from retained result rows, including the population of failures. Keep excluded intervals and test defects visible. If a client fault invalidates one stage, rerun that stage or report the limitation. Selecting the best run without a predefined rule turns normal variation into unsupported confidence.

Set expiry triggers. Reopen the schedule after a material release, data-model change, altered workload, infrastructure-class change, dependency revision, quota change, scaling-policy change or monitoring gap. Recheck it when the buyer supplies a different peak or operation mix. A dated test can remain relevant, but relevance must be decided against the current offer rather than inferred from a familiar product name.

Proposal release

Give the evaluator the envelope, evidence and next capacity action

Answer the requirement directly. Name the workload basis, tested release, representative boundary, useful outcome measures and observed envelope. State the test date or evidence period. Explain how capacity responds inside that range, how the service behaves under excess demand and which monitoring signal starts an expansion decision. Put material assumptions and buyer dependencies in the location required by the tender.

A bounded Northmere answer could say that the offered release was tested against the stated mix of result views, appeal submissions and evidence processing, using a representative data population and dependency stubs or live test services as identified. It would report the sustained useful completion range, selected tail-latency and error criteria, observed scale-response time and recovery result. It would then describe the approved path for the buyer forecast that remains outside direct evidence. It would not claim unlimited scale.

Reconcile the wording with the service-level schedule, price, monitoring, support and security disclosure. A test result does not authorize a contractual target or remedy. A service-level target does not prove capacity. Technical reviewers confirm the method and result; the service owner accepts the operating conditions; security controls the detail released; finance and commercial owners confirm cost; the relevant delegate approves the exact offer. Any missing authority remains visible rather than being replaced by confident prose.