SQL Server · Execute

Page through a procedure's result: the procedure pages itself, ARX pages a function, ARX pages a result set

Paging what a procedure or function returns, the three ways, in four scenarios. Each sets PageInfo, as a query does, and gets back a result set with CurrentPage, HasNextPage and, when asked, the total. The total is opt-in: ask for it once (page 1), as for RetrieveMultiple.

  1. The routine pages itself (best for large data: the database produces only the page). ARX fills its page arguments from PageInfo and reads its total from an OUT parameter.
  2. ARX pages a function's rows: OFFSET/FETCH NEXT and the total in one round trip. Cheap for an inline table-valued function (SQL Server expands it, so the order and the limit reach the index); a multi-statement function fills its whole table first.
  3. ARX pages a table the procedure returns (a result set): SQL Server has no cursor to move, so ARX reads past the earlier rows, keeps one page and one row more, and counts the rest. Every row crosses the network.
  4. Which to choose.

Set up for this page

The scenarios build on the sample application: SampleContext.cs (the dbContext every sample is given); SampleOutput.cs (Step and Show, which print); SampleContext.Application.cs (dbContext.Service, the application's ARX service); SampleCatalog.cs (the tables' ids, keys, lookups and display names).

Before scenario 1 the sample runs this, so the service and the rows the scenarios use are there:

Set up
var products = Enumerable.Range(1, 120).Select(i =>
{
    var product = new ArxEntity("product");
    product["sku"] = $"{PREFIX}{i:D3}";
    product["name"] = $"Paged product {i}";
    product["unit_price"] = 1m + i % 10;
    return product;
}).ToList();
await service.CreateMultipleAsync(products);

1 The procedure pages itself

sp_products_page(p_prefix, p_page, p_page_size, INOUT p_rows, INOUT p_total) pages itself. ARX fills p_page and p_page_size, reads the table from p_rows and the total from p_total.

Already in scope: service (ArxMsSqlDbService)

C#
foreach (var pageNumber in new[] { 1, 3 })
{
    var procedure = new ArxProcedureExpression("sp_products_page");
    procedure.AddParameter("p_prefix", PREFIX);
    procedure.PageInfo = new ArxPagingInfo { Count = 50, PageNumber = pageNumber, ReturnTotalRecordCount = pageNumber == 1 };
    procedure.PagingParameters = ArxPagingParameters.PageNumber("p_page", "p_page_size", totalRecordCount: "p_total");

    Console.WriteLine(Describe((await service.ExecuteAsync(procedure)).ResultSets[0]));
}
Used above

Defined once in this sample.

private static string Describe(ArxEntityCollection page)
    => $"page {page.CurrentPage}: {page.Entities.Count} rows ({page.Entities.FirstOrDefault()?["sku"]} to {page.Entities.LastOrDefault()?["sku"]}), next page: {page.HasNextPage}"
       + (page.TotalRecordCount > 0 ? $", {page.TotalRecordCount} in all, {page.TotalPageCount} pages" : string.Empty);

Check it in the database

SQL

Run it in any SQL tool connected to the database, after the code has run.

SELECT count(*), min(sku), max(sku)
FROM (SELECT sku FROM product WHERE sku LIKE 'PG-E7%' ORDER BY sku OFFSET 100 ROWS FETCH NEXT 50 ROWS ONLY) AS page_3;

Expect: 20, PG-E7101, PG-E7120 (page 1 is PG-E7001 to PG-E7050; 120 in all, 3 pages).

2 ARX pages a function's rows

fn_products_like(p_prefix) returns every matching product; ARX orders and pages its rows, with the total in the same round trip.

Already in scope: service (ArxMsSqlDbService)

C#
var function = new ArxProcedureExpression("fn_products_like") { ProcedureType = ArxProcedureType.Function };
function.AddParameter(PREFIX);
function.AddOrder("unit_price", ArxOrderType.Descending);
function.AddOrder("sku", ArxOrderType.Ascending);
function.PageInfo = new ArxPagingInfo { Count = 25, PageNumber = 2, ReturnTotalRecordCount = true };

var page = (await service.ExecuteAsync(function)).ResultSets[0];
Console.WriteLine(Describe(page) + $"; prices {page.Entities[0]["unit_price"]:N2} to {page.Entities[^1]["unit_price"]:N2}");
Used above

Defined once in this sample.

private static string Describe(ArxEntityCollection page)
    => $"page {page.CurrentPage}: {page.Entities.Count} rows ({page.Entities.FirstOrDefault()?["sku"]} to {page.Entities.LastOrDefault()?["sku"]}), next page: {page.HasNextPage}"
       + (page.TotalRecordCount > 0 ? $", {page.TotalRecordCount} in all, {page.TotalPageCount} pages" : string.Empty);

Check it in the database

SQL

Run it in any SQL tool connected to the database, after the code has run.

SELECT sku, unit_price
FROM fn_products_like('PG-E7')
ORDER BY unit_price DESC, sku OFFSET 25 ROWS FETCH NEXT 25 ROWS ONLY;

Expect: 25 rows, PG-E7017 (8.00) first and PG-E7015 (6.00) last (120 in all, 5 pages).

3 ARX pages a table the procedure returns

sp_products returns every matching product as a result set; ARX pages it.

Already in scope: service (ArxMsSqlDbService)

C#
var procedure = new ArxProcedureExpression("sp_products");
procedure.AddParameter("p_prefix", PREFIX);
procedure.PageInfo = new ArxPagingInfo { Count = 40, PageNumber = 3, ReturnTotalRecordCount = true };

Console.WriteLine(Describe((await service.ExecuteAsync(procedure)).ResultSets[0]));
Used above

Defined once in this sample.

private static string Describe(ArxEntityCollection page)
    => $"page {page.CurrentPage}: {page.Entities.Count} rows ({page.Entities.FirstOrDefault()?["sku"]} to {page.Entities.LastOrDefault()?["sku"]}), next page: {page.HasNextPage}"
       + (page.TotalRecordCount > 0 ? $", {page.TotalRecordCount} in all, {page.TotalPageCount} pages" : string.Empty);

Check it in the database

SQL

Run it in any SQL tool connected to the database, after the code has run.

SELECT count(*), min(sku), max(sku)
FROM (SELECT sku FROM product WHERE sku LIKE 'PG-E7%' ORDER BY sku OFFSET 80 ROWS FETCH NEXT 40 ROWS ONLY) AS page_3;

Expect: 40, PG-E7081, PG-E7120: the last page (no next page; 120 in all, 3 pages).

4 Which to choose

The choice, with the numbers of ExecutePagingBenchmarks.

C#
Console.WriteLine("large data, deep pages: let the routine page itself (1); the database produces only the page.");
Console.WriteLine("an inline table-valued function: let ARX page it (2); SQL Server expands it, so the limit reaches the index.");
Console.WriteLine("a multi-statement function or a procedure's result set: ARX pages it (2, 3), but every skipped row is still produced (and, for 3, sent).");
Console.WriteLine("the total costs a count of every row: ask for it on page 1 only, or not at all for \"load more\".");

Check it in the database

Run the complete sample

The code above comes from one program, built in a fresh project and run against the database. To run it yourself, here is every file and step.

The program, its files and how to run it

Run it locally

  1. .NET SDK: .NET SDK 10.0.100 or later 10.0.x (target net10.0).
  2. Create the project:
    Terminal
    dotnet new console -n ArxSample -f net10.0
    cd ArxSample
  3. Install the packages:
    Terminal
    dotnet add package ARX.SDK.SQL --version 1.0.0-rc.1
  4. Database: a SQL Server 2022 (tested with 16.0.4225.2) database with the sample schema.
  5. Configuration: set ARX_SAMPLES_SQLSERVER to its connection string. Nothing is hard-coded.
  6. Source code: add the files of the sample application this sample uses (SampleContext.cs, SampleOutput.cs, SampleContext.Application.cs, SampleCatalog.cs), then replace Program.cs and add the other file below, with the file names shown.
  7. Run it:
    Terminal
    dotnet run
  8. Expected output: below.

The files

Program.cs

Runs the sample. Generated for this page; it reads the connection string from the environment.

using ArxSamples.MSSQL;
using ArxSamples.MSSQL.Operations.Execute.RealCases;

// The connection string comes from the environment, never from the code (see "Configure the connection").
var connectionString = Environment.GetEnvironmentVariable("ARX_SAMPLES_SQLSERVER")
    ?? throw new InvalidOperationException("Set the ARX_SAMPLES_SQLSERVER environment variable to your SQL Server connection string.");

await new RealCase3_PageThroughAProcedure().RunAsync(new SampleContext(connectionString));
RealCase3_PageThroughAProcedure.cs

The sample, as it is in the repository.

using ARX.SDK.Query;
using ARX.SDK.Services;
using static ArxSamples.MSSQL.SampleOutput;

namespace ArxSamples.MSSQL.Operations.Execute.RealCases;

/// <summary>
/// Paging what a procedure or function returns, the three ways, in four scenarios. Each sets
/// <see cref="ArxProcedureExpression.PageInfo"/>, as a query does, and gets back a result set with <c>CurrentPage</c>,
/// <c>HasNextPage</c> and, when asked, the total. The total is opt-in: ask for it once (page 1), as for RetrieveMultiple.
/// <list type="number">
/// <item><b>The routine pages itself</b> (best for large data: the database produces only the page). ARX fills its page
/// arguments from <c>PageInfo</c> and reads its total from an OUT parameter.</item>
/// <item><b>ARX pages a function's rows</b>: <c>OFFSET</c>/<c>FETCH NEXT</c> and the total in one round trip. Cheap
/// for an inline table-valued function (SQL Server expands it, so the order and the limit reach the index); a
/// multi-statement function fills its whole table first.</item>
/// <item><b>ARX pages a table the procedure returns</b> (a result set): SQL Server has no cursor to move, so ARX reads
/// past the earlier rows, keeps one page and one row more, and counts the rest. Every row crosses the network.</item>
/// <item>Which to choose.</item>
/// </list>
/// </summary>
public sealed class RealCase3_PageThroughAProcedure : ISample
{
    public string Name => "execute-real-3-page-through-a-procedure";

    public string Title => "Page through a procedure's result: the procedure pages itself, ARX pages a function, ARX pages a result set";

    private const string PREFIX = "PG-E7";

    public async Task RunAsync(SampleContext dbContext)
    {
        var products = Enumerable.Range(1, 120).Select(i =>
        {
            var product = new ArxEntity("product");
            product["sku"] = $"{PREFIX}{i:D3}";
            product["name"] = $"Paged product {i}";
            product["unit_price"] = 1m + i % 10;
            return product;
        }).ToList();

        await dbContext.Service.CreateMultipleAsync(products);

        await Scenario1_TheProcedurePagesItselfAsync(dbContext);

        await Scenario2_ARXPagesAFunctionAsync(dbContext);

        await Scenario3_ARXPagesAProceduresTableAsync(dbContext);

        Scenario4_WhichToChoose();
    }

    private static string Describe(ArxEntityCollection page)
        => $"page {page.CurrentPage}: {page.Entities.Count} rows ({page.Entities.FirstOrDefault()?["sku"]} to {page.Entities.LastOrDefault()?["sku"]}), next page: {page.HasNextPage}"
           + (page.TotalRecordCount > 0 ? $", {page.TotalRecordCount} in all, {page.TotalPageCount} pages" : string.Empty);

    /// <summary>
    /// 1. <c>sp_products_page(p_prefix, p_page, p_page_size, INOUT p_rows, INOUT p_total)</c> pages itself. ARX fills
    /// <c>p_page</c> and <c>p_page_size</c>, reads the table from <c>p_rows</c> and the total from <c>p_total</c>.
    /// </summary>
    private static async Task Scenario1_TheProcedurePagesItselfAsync(SampleContext dbContext)
    {
        Step("1. The procedure pages itself");

        foreach (var pageNumber in new[] { 1, 3 })
        {
            var procedure = new ArxProcedureExpression("sp_products_page");
            procedure.AddParameter("p_prefix", PREFIX);
            procedure.PageInfo = new ArxPagingInfo { Count = 50, PageNumber = pageNumber, ReturnTotalRecordCount = pageNumber == 1 };
            procedure.PagingParameters = ArxPagingParameters.PageNumber("p_page", "p_page_size", totalRecordCount: "p_total");

            Show(Describe((await dbContext.Service.ExecuteAsync(procedure)).ResultSets[0]));
        }

        /* ===========================================================
         * [DIRECT QUERY CHECK]
         * -----------------------------------------------------------
         * SELECT count(*), min(sku), max(sku)
         * FROM (SELECT sku FROM product WHERE sku LIKE 'PG-E7%' ORDER BY sku OFFSET 100 ROWS FETCH NEXT 50 ROWS ONLY) AS page_3;
         * -- Expect: 20, PG-E7101, PG-E7120 (page 1 is PG-E7001 to PG-E7050; 120 in all, 3 pages).
         * ===========================================================
         */
    }

    /// <summary>
    /// 2. <c>fn_products_like(p_prefix)</c> returns every matching product; ARX orders and pages its rows, with the total
    /// in the same round trip.
    /// </summary>
    private static async Task Scenario2_ARXPagesAFunctionAsync(SampleContext dbContext)
    {
        Step("2. ARX pages a function's rows");

        var function = new ArxProcedureExpression("fn_products_like") { ProcedureType = ArxProcedureType.Function };
        function.AddParameter(PREFIX);
        function.AddOrder("unit_price", ArxOrderType.Descending);
        function.AddOrder("sku", ArxOrderType.Ascending);
        function.PageInfo = new ArxPagingInfo { Count = 25, PageNumber = 2, ReturnTotalRecordCount = true };

        var page = (await dbContext.Service.ExecuteAsync(function)).ResultSets[0];
        Show(Describe(page) + $"; prices {page.Entities[0]["unit_price"]:N2} to {page.Entities[^1]["unit_price"]:N2}");

        /* ===========================================================
         * [DIRECT QUERY CHECK]
         * -----------------------------------------------------------
         * SELECT sku, unit_price
         * FROM fn_products_like('PG-E7')
         * ORDER BY unit_price DESC, sku OFFSET 25 ROWS FETCH NEXT 25 ROWS ONLY;
         * -- Expect: 25 rows, PG-E7017 (8.00) first and PG-E7015 (6.00) last (120 in all, 5 pages).
         * ===========================================================
         */
    }

    /// <summary>3. <c>sp_products</c> returns every matching product as a result set; ARX pages it.</summary>
    private static async Task Scenario3_ARXPagesAProceduresTableAsync(SampleContext dbContext)
    {
        Step("3. ARX pages a table the procedure returns");

        var procedure = new ArxProcedureExpression("sp_products");
        procedure.AddParameter("p_prefix", PREFIX);
        procedure.PageInfo = new ArxPagingInfo { Count = 40, PageNumber = 3, ReturnTotalRecordCount = true };

        Show(Describe((await dbContext.Service.ExecuteAsync(procedure)).ResultSets[0]));

        /* ===========================================================
         * [DIRECT QUERY CHECK]
         * -----------------------------------------------------------
         * SELECT count(*), min(sku), max(sku)
         * FROM (SELECT sku FROM product WHERE sku LIKE 'PG-E7%' ORDER BY sku OFFSET 80 ROWS FETCH NEXT 40 ROWS ONLY) AS page_3;
         * -- Expect: 40, PG-E7081, PG-E7120: the last page (no next page; 120 in all, 3 pages).
         * ===========================================================
         */
    }

    /// <summary>4. The choice, with the numbers of <c>ExecutePagingBenchmarks</c>.</summary>
    private static void Scenario4_WhichToChoose()
    {
        Step("4. Which to choose");

        Show("large data, deep pages: let the routine page itself (1); the database produces only the page.");
        Show("an inline table-valued function: let ARX page it (2); SQL Server expands it, so the limit reaches the index.");
        Show("a multi-statement function or a procedure's result set: ARX pages it (2, 3), but every skipped row is still produced (and, for 3, sent).");
        Show("the total costs a count of every row: ask for it on page 1 only, or not at all for \"load more\".");
    }
}

Expected output

What dotnet run printed when the sample was verified, on a freshly created sample schema. Ids differ when the tables already hold rows.

Output
  > 1. The procedure pages itself
      page 1: 50 rows (PG-E7001 to PG-E7050), next page: True, 120 in all, 3 pages
      page 3: 20 rows (PG-E7101 to PG-E7120), next page: False
  > 2. ARX pages a function's rows
      page 2: 25 rows (PG-E7017 to PG-E7015), next page: True, 120 in all, 5 pages; prices 8.00 to 6.00
  > 3. ARX pages a table the procedure returns
      page 3: 40 rows (PG-E7081 to PG-E7120), next page: False, 120 in all, 3 pages
  > 4. Which to choose
      large data, deep pages: let the routine page itself (1); the database produces only the page.
      an inline table-valued function: let ARX page it (2); SQL Server expands it, so the limit reaches the index.
      a multi-statement function or a procedure's result set: ARX pages it (2, 3), but every skipped row is still produced (and, for 3, sent).
      the total costs a count of every row: ask for it on page 1 only, or not at all for "load more".