Guides

Running in production

Service lifetime, connections, transactions, cancellation and timeouts, retries, logging and security: what an application needs to decide before it runs ARX for real.

Startup checks

At startup, CheckServerAsync() (from 0.1.0-preview.2) asks the server once what it allows: its version, the capabilities it removes with the reason (a standalone MongoDB runs no transactions), and problems such as a DynamoDB table of the model that doesn't exist. ValidateSchemaAsync() checks the schema catalog against the tables.

One service per database, shared

Create the service once at startup and share it (in ASP.NET Core, register it as a singleton; from 0.1.0-preview.2, as IArxDataService where the code shouldn't depend on the provider). It holds the connection string, the schema catalog and the provider's settings, and no connection: each call takes a connection from the driver's pool (PostgreSQL, SQL Server, MySQL), or uses the client the service was given (MongoDB, DynamoDB), so concurrent calls from many requests are fine.

Give it a schema catalog, declared or preloaded at startup. A service created with only the connection string works (the SQL providers read each table from the database before the call), but every call then pays one more round trip: much slower, and meant for simple CRUD in small applications. If you do run one, set RememberResolvedTables = true so each table is read once per service.

Set the service's properties (Strict, PagingCookieKey, RememberResolvedTables) when you create it, before the first call, and don't change them afterwards.

Connections, transactions and batch scopes are the opposite: one per unit of work, never shared between threads or requests.

Connections and pooling

The connection string comes from configuration or a secret store, never from code. Pool sizes, timeouts to connect and TLS settings are the driver's and go in the connection string (Npgsql, Microsoft.Data.SqlClient, MySqlConnector, the MongoDB driver). For DynamoDB, configure the AmazonDynamoDBClient you pass in: region, credentials, retries.

Pools. On PostgreSQL, ARX opens its own connections with Timezone=UTC added to the connection string (unless it names a zone), and Npgsql pools by connection string: if the application also opens Npgsql connections with the same string, they use a second pool beside ARX's, so size Maximum Pool Size for both. ARX doesn't take an NpgsqlDataSource.

MongoDB's pool doesn't queue for long. When every connection is busy, the MongoDB driver puts a call in a wait queue and fails it once the queue is full or waitQueueTimeoutMS passes: ARX7005 ConnectionFailure (transient), with the driver's MongoWaitQueueFullException inside. The SQL drivers wait for a free connection up to their connect timeout instead. With many concurrent calls on one MongoDB service, size maxPoolSize for the peak, and retry ARX7005 with a backoff. To share one connection and transaction with your own Npgsql or Dapper code, pass them to the interop overloads.

Driver versions. Each provider is built on its lowest supported driver version: Npgsql 8.0.3, Microsoft.Data.SqlClient 7.1.1, MySqlConnector 2.6.2, MongoDB.Driver 3.13.0, AWSSDK.DynamoDBv2 4.0.110. The PostgreSQL provider's whole test suite also runs on Npgsql 10.0.3; the other providers are tested on their lowest version only (Versions and compatibility, "Supported driver versions"). NuGet takes a higher version when another package needs one (EF Core 10's provider lifts Npgsql to 10.x). ARX doesn't forbid that: run your own tests after a driver upgrade to a version not listed as tested.

Transactions

For several writes that must succeed or fail together, open the provider's connection and begin a transaction, or use a batch scope:

await service.RunBatchAsync(async batch =>
{
    ArxRecordId orderId = await batch.CreateAsync(order, cancellationToken);
    await batch.CreateMultipleAsync(lines, cancellationToken);
}, cancellationToken: cancellationToken);

A write by a filter (UpdateMultipleAsync(values, filter), DeleteMultipleAsync(table, filter)) is all or nothing by itself. It refuses an empty filter, so a missing condition doesn't change the whole table; pass new ArxFilterWriteOptions { AllRows = true } only where every row is meant, and check the count it returns against what you expect. For a large load whose ids you don't need, new ArxCreateMultipleOptions { ReturnIds = false } takes the database's bulk path on PostgreSQL (5,000 rows or more of one shape) and SQL Server (1,000 or more); below those sizes the ordinary statement was faster on the benchmarks.

RunBatchAsync commits when the body returns and rolls back when it throws; a scope from BeginBatchAsync commits only when you call CommitAsync, and disposing it without that rolls back. After a database error inside a transaction, the transaction is aborted: later calls and the commit throw ARX1015 TransactionAborted, and you start again.

A commit can also fail after it was sent: the connection breaks or times out before the database answers. ARX then can't know whether the work was saved, so the transaction's State (or the scope's) is OutcomeUnknown and the error (ARX7005 or ARX7003) has IsTransient false, so a retry policy doesn't save the work twice. Check what is in the database (the order number you wrote, a version) before doing the work again. A commit the database refuses (a deadlock, a serialization failure) is rolled back and stays transient: retrying the whole unit of work is safe. On MongoDB the same holds for a single write outside a transaction: one that timed out or lost its connection after it was sent is ARX7005 or ARX7003 with IsTransient false (its outcome is unknown), not a transient error.

Cancellation and timeouts

Pass the request's CancellationToken. A cancelled call throws OperationCanceledException (or TaskCanceledException), not an ArxException.

Every call takes a token, the interop overloads of the SQL providers too (the ones that take your own IDbConnection and IDbTransaction): from 1.0.0-rc.1 their parameters are named connection and transaction, and they end with a CancellationToken.

Long queries take a timeout in their options (CommandTimeout, in seconds); without one the driver's default applies (30 s for the SQL drivers). A timeout is ARX7003, transient.

MongoDB writes once sent. MongoDB has no way to stop a write the server is already running, so ARX honours a token only until a write outside a transaction (or a commit) is sent, then waits for the outcome: a caller is never told "cancelled" about a write that was applied. A write blocked behind another transaction therefore waits until that transaction ends, past the token. Set a server-side limit where that matters (maxTimeMS, or maxTransactionLockRequestTimeoutMillis on the server), or write inside an ArxTransaction: there a cancelled write fails the call and nothing of the transaction is kept.

Version columns protect only writers that use them. .Version(table, column) checks every write that goes through ARX with that catalog. Another service, a script or a hand-written UPDATE that doesn't change the version can still overwrite a row unseen: give every writer the same rule (or a trigger that bumps the version).

Retries

ARX doesn't retry your calls or your transactions. Retry the whole unit of work when an error's IsTransient is true, with a backoff, and keep in mind that the AWS SDK and the MongoDB driver already retry some failures underneath, and that on MongoDB the transaction ARX opens itself for a call over several documents is run once more on a TransientTransactionError label (and committed once more on UnknownTransactionCommitResult), as MongoDB recommends (Errors in practice, "Retrying"). A write that failed after it was sent may or may not have happened: supplied ids or a business key let you check before writing again. The Errors in practice guide has the details.

Wrapping the service

A decorator around IArxDataService (for auditing, tenancy checks, retries) sees the calls made on the service. Calls made inside a batch scope go to the scope (IArxBatchScope), not to the service: wrap IArxBatchDataService.BeginBatchAsync too, and return a wrapped scope, to see them. Every call, in a scope or not, is traced (ARX.SDK ActivitySource), which covers auditing that only needs to observe.

Tracing, metrics and diagnostics

From 0.1.0-preview.2, every operation is an activity of the ARX.SDK ActivitySource and a measurement of the ARX.SDK Meter. With OpenTelemetry:

builder.Services.AddOpenTelemetry()
    .WithTracing(tracing => tracing.AddSource(ArxDiagnostics.SourceName))
    .WithMetrics(metrics => metrics.AddMeter(ArxDiagnostics.SourceName));

Each activity is named after the operation and the table (Update account), kind Client, with db.system.name, db.operation.name, db.collection.name and arx.provider; a failure sets the status Error, error.type and arx.error.code. The metrics are arx.operation.duration (seconds) and arx.operation.errors, with the same tags. Values are never recorded, only names. Without a listener, each call costs a check. An ActivityListener or a MeterListener reads them without any package (the "Set the service up for production" samples). To log each call through ILogger, see Logging and tracing.

Besides those:

  • The exception: Code, Category, IsTransient, the table and column where known, the database's own code (SqlState or NativeCode) and the driver's exception as InnerException. The message is written not to repeat the values sent, so it can go to logs.
  • The driver's own logging and diagnostics, configured as its documentation says.
  • For DynamoDB, the AmazonDynamoDBClient you configure: its logging, metrics and retry settings are yours.

Security

  • Values are always sent as parameters, never written into SQL or query text.
  • Names (tables, attributes, match columns) are checked: a name carrying SQL or query syntax is refused before the database as ARX1017 SuspectedInjection. Build names from your own code or an allow list, not from requests; log ARX1017, since it is an attempt rather than a mistake (the GuardAgainstAttacks samples show one way).
  • MongoDB and DynamoDB paging cookies carry the last item's key values (an email, a tenant id). From 0.1.0-preview.2 they are sealed by default: encrypted and authenticated, so a client can neither read nor change one. Without a key, ARX seals with a random key of the process: right for one process. With several instances, or cookies that must survive a restart, give every instance the same PagingCookieKey (32 bytes or more, from your secret store); a cookie that reaches an instance with another key is ARX1001. PagingCookieProtection = Signed keeps cookies readable but unchangeable; Plain leaves both open.
  • Run under a least-privilege database role. The schema catalog says which tables ARX writes and reads by id; it doesn't stop a query (RetrieveMultipleAsync) reading any table or view the database user can see, system views included, or ExecuteAsync calling any function it may run. service.Strict = true (with a preloaded catalog) limits queries to the preloaded tables and columns, but not functions or procedures. What the database user may read and call is the boundary: grant it only the application's tables and routines, never a superuser or db_owner. ARX needs no administrative rights for its operations.