{"id":368,"date":"2015-12-06T20:00:36","date_gmt":"2015-12-06T19:00:36","guid":{"rendered":"http:\/\/www.cymbeline.ch\/?p=368"},"modified":"2017-11-13T12:30:53","modified_gmt":"2017-11-13T11:30:53","slug":"azure-queue-agent-introduction","status":"publish","type":"post","link":"https:\/\/cymbeline.ch\/2015\/12\/06\/azure-queue-agent-introduction\/","title":{"rendered":"Azure Queue Agent &#8211; Introduction"},"content":{"rendered":"<p>For a small side project I&#8217;ve been working on I needed a way to schedule background tasks and handle them on a pool of workers. Since I was planning to run this on Azure, the Queues offered with <a href=\"https:\/\/azure.microsoft.com\/en-us\/services\/storage\/\" title=\"Azure Storage\" target=\"_blank\">Azure Storage<\/a> seemed to be a no-brainer. Even more so since the <a href=\"https:\/\/azure.microsoft.com\/en-us\/services\/scheduler\/\" title=\"Azure Scheduler\" target=\"_blank\">Azure Scheduler<\/a>, which can be used to periodically execute some action, can also be used to add messages to Queues. I figured that I wasn&#8217;t the only one needing something to handle such tasks, so I decided to build a lightweight open-source library for this.<\/p>\n<h2>Enter Azure Queue Agent (AQuA)<\/h2>\n<p>AQuA comes with two main components: a <em>Producer<\/em> and a <em>Consumer<\/em>. As the names suggest, the <em>Producer<\/em> can be used to produce (i.e. enqueue) new jobs, while the <em>Consumer<\/em> can be used to consume (i.e. dequeue and then handle) jobs from the queue. Job Descriptors are used to define which job should be executed and what parameters should be used for the execution. They are encoded as simple JSON objects like the one below, i.e. they can also easily be written manually (e.g. when used with the Azure Scheduler). That said, with AQuA it is very simple to build a scalable and robust collection of workers which take care of all your background processing jobs.<\/p>\n<pre class=\"brush: jscript; title: ; notranslate\" title=\"\">\r\n{ &quot;Job&quot;: &quot;HelloWho&quot;, &quot;Properties&quot;: { &quot;Who&quot;: &quot;World&quot; } }\r\n<\/pre>\n<p>The above example for instance would queue the <em>HelloWho<\/em> job, which does nothing more but print the value of the Who parameter on stdout like this: \u00e2\u20ac\u0153Hello, &lt;Who&gt;!\u00e2\u20ac\u009d. In addition, the Azure Queue Agent Consumer can be configured to either delete or requeue messages which were badly formatted, using unknown jobs or which could not be executed successfully, such that you can even use a single queue for multiple different pools of workers, should you ever find yourself in that situation.<\/p>\n<h2>Getting Started<\/h2>\n<p>The Azure Queue Agent is available as a <a href=\"https:\/\/www.nuget.org\/packages\/aqua.lib\/\" title=\"Azure Queue Agent Library on NuGet\" target=\"_blank\">NuGet package<\/a>, currently however only in pre-release. You can get it like this:<\/p>\n<pre class=\"brush: plain; title: ; notranslate\" title=\"\">Install-Package aqua.lib -Pre<\/pre>\n<p>Once this is done, you need to create an instance of <em>Producer<\/em> (if you want to create job requests from your code), and an instance of <em>Consumer<\/em> (for when you want to handle job requests).<\/p>\n<pre class=\"brush: csharp; title: ; notranslate\" title=\"\">\r\n\/\/ Setup and initialization\r\nJobFactory factory = new JobFactory();\r\n\r\n\/\/ Register all the jobs you want your consumer to be able to handle.\r\nfactory.RegisterJobType(typeof(HelloWho));\r\nfactory.RegisterJobType(typeof(MyBackgroundJob));\r\n\r\n\/\/ Use the storage account from the emulator with queue &quot;jobs&quot;.\r\nConnectionSettings connection = new ConnectionSettings(&quot;jobs&quot;);\r\n\r\nProducer producer = new Producer(connection, factory);\r\nConsumer consumer = new Consumer(connection, factory);\r\n\r\n\/\/ Produce (i.e. enqueue) a HelloWho job request.\r\nHelloWho job = new HelloWho() { Who = &quot;Azure Queue Agent Example&quot; };\r\nproducer.One(job);\r\n\r\n\/\/ Consume (i.e. dequeue and handle) a job request.\r\nconsumer.One();\r\n<\/pre>\n<p>This should get you going for now. I&#8217;ll follow up with more later. Oh, and you can read the sources on <a href=\"https:\/\/github.com\/rokeller\/aqua\" title=\"Azure Queue Agent (AQuA) on GitHub\" target=\"_blank\">GitHub<\/a>.<\/p>\n<p><\/p>","protected":false},"excerpt":{"rendered":"<p>For a small side project I&#8217;ve been working on I needed a way to schedule background tasks and handle them on a pool of workers. Since I was planning to run this on Azure, the Queues offered with Azure Storage seemed to be a no-brainer. Even more so since the Azure Scheduler, which can be &hellip; <a href=\"https:\/\/cymbeline.ch\/2015\/12\/06\/azure-queue-agent-introduction\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Azure Queue Agent &#8211; Introduction&#8221;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[61],"tags":[81,83,84,82,85],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v19.13 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Introduction to Azure Queue Agent<\/title>\n<meta name=\"description\" content=\"Introduction to Azure Queue Agent (AQuA), a lightweight open-source library to assist you enqueueing, dequeueing and handling background tasks in Azure.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/cymbeline.ch\/2015\/12\/06\/azure-queue-agent-introduction\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Introduction to Azure Queue Agent\" \/>\n<meta property=\"og:description\" content=\"Introduction to Azure Queue Agent (AQuA), a lightweight open-source library to assist you enqueueing, dequeueing and handling background tasks in Azure.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/cymbeline.ch\/2015\/12\/06\/azure-queue-agent-introduction\/\" \/>\n<meta property=\"og:site_name\" content=\"Tales of a Code Monkey\" \/>\n<meta property=\"article:published_time\" content=\"2015-12-06T19:00:36+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2017-11-13T11:30:53+00:00\" \/>\n<meta name=\"author\" content=\"roger\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"roger\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/cymbeline.ch\/2015\/12\/06\/azure-queue-agent-introduction\/\",\"url\":\"https:\/\/cymbeline.ch\/2015\/12\/06\/azure-queue-agent-introduction\/\",\"name\":\"Introduction to Azure Queue Agent\",\"isPartOf\":{\"@id\":\"https:\/\/cymbeline.ch\/#website\"},\"datePublished\":\"2015-12-06T19:00:36+00:00\",\"dateModified\":\"2017-11-13T11:30:53+00:00\",\"author\":{\"@id\":\"https:\/\/cymbeline.ch\/#\/schema\/person\/cf1f3b3205f4266bf8b68a80fa6b0916\"},\"description\":\"Introduction to Azure Queue Agent (AQuA), a lightweight open-source library to assist you enqueueing, dequeueing and handling background tasks in Azure.\",\"breadcrumb\":{\"@id\":\"https:\/\/cymbeline.ch\/2015\/12\/06\/azure-queue-agent-introduction\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/cymbeline.ch\/2015\/12\/06\/azure-queue-agent-introduction\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/cymbeline.ch\/2015\/12\/06\/azure-queue-agent-introduction\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/cymbeline.ch\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Azure Queue Agent &#8211; Introduction\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/cymbeline.ch\/#website\",\"url\":\"https:\/\/cymbeline.ch\/\",\"name\":\"Tales of a Code Monkey\",\"description\":\"... the adventures of a guy making software.\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/cymbeline.ch\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/cymbeline.ch\/#\/schema\/person\/cf1f3b3205f4266bf8b68a80fa6b0916\",\"name\":\"roger\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/cymbeline.ch\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/33cc08db99ae52beb26ab6fda46e0eb7?s=96&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/33cc08db99ae52beb26ab6fda46e0eb7?s=96&r=g\",\"caption\":\"roger\"},\"sameAs\":[\"https:\/\/www.cymbeline.ch\"],\"url\":\"https:\/\/cymbeline.ch\/author\/roger\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Introduction to Azure Queue Agent","description":"Introduction to Azure Queue Agent (AQuA), a lightweight open-source library to assist you enqueueing, dequeueing and handling background tasks in Azure.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/cymbeline.ch\/2015\/12\/06\/azure-queue-agent-introduction\/","og_locale":"en_US","og_type":"article","og_title":"Introduction to Azure Queue Agent","og_description":"Introduction to Azure Queue Agent (AQuA), a lightweight open-source library to assist you enqueueing, dequeueing and handling background tasks in Azure.","og_url":"https:\/\/cymbeline.ch\/2015\/12\/06\/azure-queue-agent-introduction\/","og_site_name":"Tales of a Code Monkey","article_published_time":"2015-12-06T19:00:36+00:00","article_modified_time":"2017-11-13T11:30:53+00:00","author":"roger","twitter_misc":{"Written by":"roger","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/cymbeline.ch\/2015\/12\/06\/azure-queue-agent-introduction\/","url":"https:\/\/cymbeline.ch\/2015\/12\/06\/azure-queue-agent-introduction\/","name":"Introduction to Azure Queue Agent","isPartOf":{"@id":"https:\/\/cymbeline.ch\/#website"},"datePublished":"2015-12-06T19:00:36+00:00","dateModified":"2017-11-13T11:30:53+00:00","author":{"@id":"https:\/\/cymbeline.ch\/#\/schema\/person\/cf1f3b3205f4266bf8b68a80fa6b0916"},"description":"Introduction to Azure Queue Agent (AQuA), a lightweight open-source library to assist you enqueueing, dequeueing and handling background tasks in Azure.","breadcrumb":{"@id":"https:\/\/cymbeline.ch\/2015\/12\/06\/azure-queue-agent-introduction\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/cymbeline.ch\/2015\/12\/06\/azure-queue-agent-introduction\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/cymbeline.ch\/2015\/12\/06\/azure-queue-agent-introduction\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/cymbeline.ch\/"},{"@type":"ListItem","position":2,"name":"Azure Queue Agent &#8211; Introduction"}]},{"@type":"WebSite","@id":"https:\/\/cymbeline.ch\/#website","url":"https:\/\/cymbeline.ch\/","name":"Tales of a Code Monkey","description":"... the adventures of a guy making software.","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/cymbeline.ch\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/cymbeline.ch\/#\/schema\/person\/cf1f3b3205f4266bf8b68a80fa6b0916","name":"roger","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/cymbeline.ch\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/33cc08db99ae52beb26ab6fda46e0eb7?s=96&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/33cc08db99ae52beb26ab6fda46e0eb7?s=96&r=g","caption":"roger"},"sameAs":["https:\/\/www.cymbeline.ch"],"url":"https:\/\/cymbeline.ch\/author\/roger\/"}]}},"_links":{"self":[{"href":"https:\/\/cymbeline.ch\/wp-json\/wp\/v2\/posts\/368"}],"collection":[{"href":"https:\/\/cymbeline.ch\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cymbeline.ch\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/cymbeline.ch\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/cymbeline.ch\/wp-json\/wp\/v2\/comments?post=368"}],"version-history":[{"count":3,"href":"https:\/\/cymbeline.ch\/wp-json\/wp\/v2\/posts\/368\/revisions"}],"predecessor-version":[{"id":377,"href":"https:\/\/cymbeline.ch\/wp-json\/wp\/v2\/posts\/368\/revisions\/377"}],"wp:attachment":[{"href":"https:\/\/cymbeline.ch\/wp-json\/wp\/v2\/media?parent=368"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cymbeline.ch\/wp-json\/wp\/v2\/categories?post=368"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cymbeline.ch\/wp-json\/wp\/v2\/tags?post=368"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}