Category: Azure

Paging Azure Cosmos DB Query Results from .NET

Derek Comartin 3 Comments

I received a comment on my Azure Cosmos DB Transactions from .NET post from Onur:

The thing I don’t like about Azure Cosmos DB is that it doesn’t support aggregations nor paging.

This was actually some pretty good timing of this comment as I just ran into a situation of a side project that required to page through  a result set.

Skip & Take

It does not appear that Azure Cosmos DB supports (yet) SKIP and TAKE, which you would expect to use in order to do paging.  Because of this, you can’t implement all the same functionality you might expect.

But there is a a way to limit the number of records returned in a query and subsequently having the next query return the records were the previous query left off.  This means you can basically continue through your result set.

Continuations

Azure Cosmos DB has the concept of a continuationToken which is returned from a query which limits the number of records to return.  When using the CreateDocumentQuery<T> from the .NET SDK, you can specify an additional parameter for FeedOptions.

FeedOptions

Here’s an example of using FeedOptions to limit the number of results:

RequestContinuation

FeedOptions also property for specifying the RequestContinuation.  If you original query had a MaxItemCount and your query result exceeds the specified count, then your results will contain a ResponseContinuation property.

Combine these two to basically do forward paging.  Here is an xUnit test to outline:

 

 

Demo Source Code

I’ve put together a small .NET Core sample with an XUnit test from above. All the source code for this series is available on GitHub.

Are you using Azure Cosmos DB? I’d love to hear your experiences so far along. Let me know on twitter or in the comments.


 

Azure Cosmos DB Transactions from .NET

Derek Comartin 3 Comments

I received a comment to my Optimistic Concurrency in Azure Cosmos DB  a couple weeks ago from Jerry Goyal:

Can we somehow handle the concurrency among multiple documents (transactions)?

Since ETags are defined on each document, you must build your concurrency around them.  However, this made me start to wonder how to update multiple documents at the same time using their respective ETags.  Meaning you would want both documents to update together only if both of their ETags were current.  If one was valid and the other was out of date, none of the documents would update.

Transactions

It’s pretty obvious that I’m looking for transactions within Azure Cosmos DB.

Azure Cosmos DB supports language-integrated transactions via JavaScript stored procedures and triggers. All database operations inside scripts are executed under snapshot isolation scoped to the collection if it is a single-partition collection, or documents with the same partition key value within a collection, if the collection is partitioned.

There is no client transaction scope you can use with the .NET SDK.   Meaning from the client (.NET) you cannot create a transaction scope.  It has to be done on the Azure Cosmos DB server via  stored procedure or trigger.

Azure Cosmos DB Emulator

There are several ways you can create a stored procedure.  One of which is via the Azure Cosmos DB Emulator web UI.

You can create a store procedure under any given collection.

You can also create stored procedures via the .NET SDK using the CreateStoredProcedureAsync.  However for this demo, I’m going to be creating it via the emulator.

Stored Procedure

For demo purposes I wanted a way to do a bulk insert.  If any of the customers failed to be added to the collection, due to the Name being falsey, I don’t want any to be created.

Here is the stored procedure I created called sp_bulkinsert.  Notice the getContext() method which is available to.  Check out the JavaScript docs for more on the Context/Request/Response.

Client

Here are a couple tests using the .NET SDK which call the stored procedure passing in an array of new customers.

The first test passes returning the number of created customer documents.  The second test fails because the customer name is empty/blank.

 

Demo Source Code

I’ve put together a small .NET Core sample with an XUnit test from above. All the source code for this series is available on GitHub.

Are you using Azure Cosmos DB? I’d love to hear your experiences so far along. Let me know on twitter or in the comments.


 

Optimistic Concurrency in Azure Cosmos DB

Derek Comartin 9 Comments

I’ve started working on new side project using ASP.NET Core.  I wanted to try a new datastore and decided to give Azure Cosmos DB a try.

This project isn’t doing anything complicated but does contain enough real world use cases that can give me an idea of how the API works.

Concurrency

The first thing I needed to implement was how to handle concurrency.  Specifically optimistic concurrency.

In an optimistic concurrency model, a violation is considered to have occurred if, after a user receives a value from the database, another user modifies the value before the first user has attempted to modify it.

This is pretty typical when dealing with multi user environments like a web application.  Specifically in my case is:

  • Fetching out a document from Azure Cosmos DB
  • Mutating the data of the document
  • Sending the document back to to Azure Cosmos DB to be replaced

In my web application, the concurrency issue arises if the same document is being modified by multiple users at the same time.

Without having any type of concurrency, we are what is called a “Last Wins” mode.  Meaning, the last process/user that sends the document back to Azure Cosmos DB is what will be persisted.

https://www.youtube.com/watch?v=dU0TWWdQgZ0&

ETags

Each document within Azure Cosmos DB has an ETag Property.

The ETag or entity tag is part of HTTP, the protocol for the World Wide Web. It is one of several mechanisms that HTTP provides for web cache validation, which allows a client to make conditional requests.

You may be familiar with ETag’s related caching.  A typical scenario is a user makes an HTTP request to the server for a specific resource.  The server will return the response along with an ETag in the response header.  The client then caches the response along with the associated ETag.

ETag: "686897696a7c876b7e"

If they client then makes another request to the same resource, it will pass a If-Non-Match header with the ETag it received.

If-None-Match: "686897696a7c876b7e"

If the resource has not changed and the ETag represents the current version, then the server will return a 304 Not modified status.  If the resource has been modified, it will return the appropriate 2XX status code along with the content new ETag header.

AccessCondition

Azure Cosmos DB uses ETags for handling optimistic concurrency.  When we retrieve a document from Azure Cosmos DB, it always contains an ETag property as apart of our document.

When we then want to send our request to replace a document, we can specify an AccessCondition with the ETag we received when we fetched out our document.

If the ETag we send is not current, the server will return a 412 Precondition Failed status code.  In our .NET SDK, this is wrapped up in a DocumentClientException.

Here is a full an example.

Demo Source Code

I’ve put together a small .NET Core sample with an XUnit test from above. All the source code for this series is available on GitHub.

Are you using Azure Cosmos DB? I’d love to hear your experiences so far along. Let me know on twitter or in the comments.