SQL Server · UpdateMultiple

Reprice 20,000 products from a feed

A supplier's price feed reprices the catalogue, in two scenarios: 20,000 products by SKU in one call (one set-based statement), and a feed with a SKU the catalogue doesn't have, which changes nothing.

Set up for this page

The scenarios build on the sample application: SampleContext.cs (the dbContext every sample is given); SampleOutput.cs (Step and Show, which print); SampleContext.Application.cs (dbContext.Service, the application's ARX service); SampleCatalog.cs (the tables' ids, keys, lookups and display names).

Before scenario 1 the sample runs this, so the service and the rows the scenarios use are there:

Set up
var products = Enumerable.Range(1, PRODUCTS).Select(i =>
{
    var product = new ArxEntity("product");
    product["sku"] = $"SKU-U8{i:000000}";
    product["name"] = $"Feed product {i}";
    product["unit_price"] = 10.00m;
    return product;
}).ToList();
await service.CreateMultipleAsync(products);

1 The whole feed, by SKU

Every line of the feed is one entity, found by its SKU (the declared key). All of them go in one statement.

Already in scope: service (ArxMsSqlDbService)

C#
var feed = Enumerable.Range(1, PRODUCTS).Select(i => (Sku: $"SKU-U8{i:000000}", Price: 10.00m + i % 100 / 100m));

var prices = feed.Select(line =>
{
    var price = new ArxEntity("product");
    price["sku"] = line.Sku;
    price["unit_price"] = line.Price;
    return price;
}).ToList();

var started = DateTime.UtcNow;
var changed = await service.UpdateMultipleAsync(prices, new ArxUpdateOptions { Match = ArxMatch.Key("sku") });
Console.WriteLine($"{changed} prices changed in {(DateTime.UtcNow - started).TotalMilliseconds:0} ms");

Check it in the database

SQL

Run it in any SQL tool connected to the database, after the code has run.

SELECT count(*) AS products, min(unit_price), max(unit_price)
FROM product
WHERE sku LIKE 'SKU-U8%';

Expect: 20000, 10.00, 10.99.

2 A feed with an unknown SKU

One SKU in the feed isn't in the catalogue: the call refuses the whole feed, so no price is half-applied.

Already in scope: service (ArxMsSqlDbService)

C#
var prices = new[] { ("SKU-U8000001", 99.00m), ("SKU-U8000002", 99.00m), ("SKU-NOT-IN-CATALOG", 99.00m) }.Select(line =>
{
    var price = new ArxEntity("product");
    price["sku"] = line.Item1;
    price["unit_price"] = line.Item2;
    return price;
}).ToList();

try
{
    await service.UpdateMultipleAsync(prices, new ArxUpdateOptions { Match = ArxMatch.Key("sku") });
}
catch (ArxException error) when (error.Code == ArxErrorCode.RecordNotFound)
{
    Console.WriteLine(error.Message);
}

Check it in the database

SQL

Run it in any SQL tool connected to the database, after the code has run.

SELECT count(*)
FROM product
WHERE sku LIKE 'SKU-U8%' AND unit_price = 99.00;

Expect: 0: the two known SKUs weren't repriced either.

Run the complete sample

The code above comes from one program, built in a fresh project and run against the database. To run it yourself, here is every file and step.

The program, its files and how to run it

Run it locally

  1. .NET SDK: .NET SDK 10.0.100 or later 10.0.x (target net10.0).
  2. Create the project:
    Terminal
    dotnet new console -n ArxSample -f net10.0
    cd ArxSample
  3. Install the packages:
    Terminal
    dotnet add package ARX.SDK.SQL --version 1.0.0-rc.1
  4. Database: a SQL Server 2022 (tested with 16.0.4225.2) database with the sample schema.
  5. Configuration: set ARX_SAMPLES_SQLSERVER to its connection string. Nothing is hard-coded.
  6. Source code: add the files of the sample application this sample uses (SampleContext.cs, SampleOutput.cs, SampleContext.Application.cs, SampleCatalog.cs), then replace Program.cs and add the other file below, with the file names shown.
  7. Run it:
    Terminal
    dotnet run
  8. Expected output: below.

The files

Program.cs

Runs the sample. Generated for this page; it reads the connection string from the environment.

using ArxSamples.MSSQL;
using ArxSamples.MSSQL.Operations.UpdateMultiple.RealCases;

// The connection string comes from the environment, never from the code (see "Configure the connection").
var connectionString = Environment.GetEnvironmentVariable("ARX_SAMPLES_SQLSERVER")
    ?? throw new InvalidOperationException("Set the ARX_SAMPLES_SQLSERVER environment variable to your SQL Server connection string.");

await new RealCase2_RepriceTheCatalog().RunAsync(new SampleContext(connectionString));
RealCase2_RepriceTheCatalog.cs

The sample, as it is in the repository.

using ARX.SDK;
using ARX.SDK.Services;
using static ArxSamples.MSSQL.SampleOutput;

namespace ArxSamples.MSSQL.Operations.UpdateMultiple.RealCases;

/// <summary>
/// A supplier's price feed reprices the catalogue, in two scenarios: 20,000 products by SKU in one call (one set-based
/// statement), and a feed with a SKU the catalogue doesn't have, which changes nothing.
/// </summary>
public sealed class RealCase2_RepriceTheCatalog : ISample
{
    private const int PRODUCTS = 20_000;

    public string Name => "update-multiple-real-2-reprice-the-catalog";

    public string Title => "Reprice 20,000 products from a feed";

    public async Task RunAsync(SampleContext dbContext)
    {
        var products = Enumerable.Range(1, PRODUCTS).Select(i =>
        {
            var product = new ArxEntity("product");
            product["sku"] = $"SKU-U8{i:000000}";
            product["name"] = $"Feed product {i}";
            product["unit_price"] = 10.00m;
            return product;
        }).ToList();
        await dbContext.Service.CreateMultipleAsync(products);

        await Scenario1_TheWholeFeedAsync(dbContext);

        await Scenario2_AFeedWithAnUnknownSkuAsync(dbContext);
    }

    /// <summary>1. Every line of the feed is one entity, found by its SKU (the declared key). All of them go in one statement.</summary>
    private static async Task Scenario1_TheWholeFeedAsync(SampleContext dbContext)
    {
        Step("1. The whole feed, by SKU");

        var feed = Enumerable.Range(1, PRODUCTS).Select(i => (Sku: $"SKU-U8{i:000000}", Price: 10.00m + i % 100 / 100m));

        var prices = feed.Select(line =>
        {
            var price = new ArxEntity("product");
            price["sku"] = line.Sku;
            price["unit_price"] = line.Price;
            return price;
        }).ToList();

        var started = DateTime.UtcNow;
        var changed = await dbContext.Service.UpdateMultipleAsync(prices, new ArxUpdateOptions { Match = ArxMatch.Key("sku") });
        Show($"{changed} prices changed in {(DateTime.UtcNow - started).TotalMilliseconds:0} ms");

        /* ===========================================================
         * [DIRECT QUERY CHECK]
         * -----------------------------------------------------------
         * SELECT count(*) AS products, min(unit_price), max(unit_price)
         * FROM product
         * WHERE sku LIKE 'SKU-U8%';
         * -- Expect: 20000, 10.00, 10.99.
         * ===========================================================
         */
    }

    /// <summary>2. One SKU in the feed isn't in the catalogue: the call refuses the whole feed, so no price is half-applied.</summary>
    private static async Task Scenario2_AFeedWithAnUnknownSkuAsync(SampleContext dbContext)
    {
        Step("2. A feed with an unknown SKU");

        var prices = new[] { ("SKU-U8000001", 99.00m), ("SKU-U8000002", 99.00m), ("SKU-NOT-IN-CATALOG", 99.00m) }.Select(line =>
        {
            var price = new ArxEntity("product");
            price["sku"] = line.Item1;
            price["unit_price"] = line.Item2;
            return price;
        }).ToList();

        try
        {
            await dbContext.Service.UpdateMultipleAsync(prices, new ArxUpdateOptions { Match = ArxMatch.Key("sku") });
        }
        catch (ArxException error) when (error.Code == ArxErrorCode.RecordNotFound)
        {
            ShowError(error);
        }

        /* ===========================================================
         * [DIRECT QUERY CHECK]
         * -----------------------------------------------------------
         * SELECT count(*)
         * FROM product
         * WHERE sku LIKE 'SKU-U8%' AND unit_price = 99.00;
         * -- Expect: 0: the two known SKUs weren't repriced either.
         * ===========================================================
         */
    }
}

Expected output

What dotnet run printed when the sample was verified, on a freshly created sample schema. Ids differ when the tables already hold rows.

Output
  > 1. The whole feed, by SKU
      20000 prices changed in 544 ms
  > 2. A feed with an unknown SKU
      caught ArxException: ARX2002 RecordNotFound: UpdateMultiple on 'product': 1 of 3 entities matched no row. Nothing was changed.