{"id":1432,"date":"2022-04-06T08:55:00","date_gmt":"2022-04-06T08:55:00","guid":{"rendered":"https:\/\/nag.com\/?post_type=insights&#038;p=1062"},"modified":"2023-07-05T10:20:32","modified_gmt":"2023-07-05T10:20:32","slug":"ad-myths-debunked-it-wont-work-on-my-code","status":"publish","type":"insights","link":"https:\/\/nag.com\/insights\/ad-myths-debunked-it-wont-work-on-my-code\/","title":{"rendered":"AD Myths Debunked: It Won\u2019t Work on My Code"},"content":{"rendered":"<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <p>This post is part of the\u00a0AD Myths Debunked series.<\/p>\n<p>You&#8217;re curious about automatic differentiation (AD) and its benefits, but you&#8217;ve heard that it would be impossible to use AD on your code, because of numerous technical and mathematical difficulties&#8230; so, you don&#8217;t pursue the idea much further.<\/p>\n<p>We have heard this story many times over the years. After all, what does one do about external libraries, non-differentiability, parallelization, or heterogeneous architectures, not to mention polymorphism and modern C++ language features? Those are showstoppers, right? Or is that just a myth?<\/p>\n<p>At <span class=\"nag-n-override\" style=\"margin-left: 0 !important;\"><i>n<\/i><\/span>AG, we have NEVER encountered a code to which we could not apply our tool, dco\/c++. This is thanks to the flexibility and the features of dco\/c++ (which form the basis for resolving all challenges), and the huge experience AD specialists at <span class=\"nag-n-override\" style=\"margin-left: 0 !important;\"><i>n<\/i><\/span>AG have with large-scale mathematical simulation codes.<\/p>\n<p>Below we have listed the various concerns we have previously encountered in regard to AD working on a specific code. We felt it was time to explain these common misconceptions, or myths, away!<\/p>\n<p><span class=\"nag-n-override\" style=\"margin-left: 0 !important;\"><i>n<\/i><\/span>AG&#8217;s AD toolset has been developed over the last 12 years and it builds upon a further 10 years of AD R&amp;D experience. We have collected a big stock of experience on how to apply AD and how to resolve a multitude of issues.<\/p>\n<p>Myths are narratives, which might sound like truths, and by talking through these in some detail and sharing our experiences, we hope to help businesses navigate these issues. Results matter, myths should not.<\/p>\n<p>Take a look at our other posts\u00a0on Myths in AD, including\u00a0<a href=\"https:\/\/nag.com\/insights\/ad-myths-debunked-ill-always-run-out-of-memory-using-adjoint-ad\/\">memory use<\/a>\u00a0and\u00a0<a href=\"https:\/\/nag.com\/insights\/ad-myths-debunked-ill-have-to-re-write-my-libraries\/\">rewriting libraries<\/a>!<\/p>\n<ul>\n<li>&#8220;The code that I need to differentiate does not have inputs and outputs all collected neatly together in one top-level routine such as I&#8217;ve seen in all examples. They&#8217;re scattered all over the place.&#8221;<br \/>\u00a0<br \/>Though the examples usually use such a top-level routine, it is not a requirement at all. Inputs and outputs can be spread across various routines, compilation units, or libraries. The dco\/c++ API does not rely on collecting all these at one location in your code.<br \/>\u00a0<\/li>\n<li>&#8220;I&#8217;m using modern C++ features like polymorphism, lambdas, patterns (factory, visitor, CRTP, adaptor, \u2026) as well as Standard Library containers and algorithms.&#8221;<br \/>\u00a0<br \/>This is just the kind of code base that dco\/c++ was designed to handle. We&#8217;ve spent a lot of time ensuring that dco\/c++ supports modern C++, whereas many other AD tools will run into problems at some point.<br \/>\u00a0<\/li>\n<li>&#8220;I am using external numerical libraries, which I cannot call with any data types other than &#8216;doubles&#8217;.&#8221;<br \/>\u00a0<br \/>First, you need to check whether this specific external library actually needs to be differentiated; does it sit between the inputs and outputs you need derivatives of? If that is the case, indeed there is no miraculous solution for calling the external library with dco\/c++ data types. There are viable alternatives, though. The obvious one is to use a numerical library, which does support dco\/c++ types. If you need functionality not supported by the <span class=\"nag-n-override\" style=\"margin-left: 0 !important;\"><i>n<\/i><\/span>AG AD Library, you can couple the overall AD solution with local finite difference approximation or hand-written derivatives.<br \/>\u00a0<\/li>\n<li>&#8220;My code is parallel.&#8221;<br \/>\u00a0<br \/>That&#8217;s great! dco\/c++ is thread safe and has simple APIs for dealing with parallelism. Many other AD tools have neither of these.<br \/>\u00a0<\/li>\n<li>&#8220;I am using multiple programming languages in my code.&#8221;<br \/>\u00a0<br \/>AD is a technique, which is conceptually not constrained by the language you use. Whenever language interoperability is required, AD can work in these instances as well. dco\/c++ has production-ready front ends for C++ and Fortran, and alpha releases for C# and Python. Custom solutions for other languages exist; this includes scripting languages based on source transformation, or hand-coding for rather static codes.<br \/>\u00a0<\/li>\n<li>&#8220;The problem I&#8217;m solving is nondifferentiable.&#8221;<br \/>\u00a0<br \/>AD requires the implementation to be differentiable, not the mathematical problem. In many circumstances, this fact circumvents the non-differentiability right away. In addition, AD by default computes a sub gradient in case of an evaluation at a non-differentiable point (e.g., left or right derivative). If the non-differentiability turns out to be an issue, there are many ways of approaching this. Potential solutions include local manual implementation (e.g., symbolic differentiation), local application of finite differences, or local smoothing. All these approaches can be integrated into an overall AD solution with dco\/c++.<br \/>\u00a0<\/li>\n<li>&#8220;I&#8217;m doing heterogeneous programming.&#8221;<br \/>\u00a0<br \/>There are various approaches to handle distributed and shared-memory parallelism with dco\/c++. The tool is thread-safe and applying it correctly preserves the parallelism. For GPU programming, we offer the general-purpose tool dco\/map, but have also implemented custom solutions.<\/li>\n<\/ul>\n        <\/div>\n    <\/div>\n<\/div>","protected":false},"excerpt":{"rendered":"<p>At <span class=\"nag-n-override\" style=\"margin-left: 0 !important;\"><i>n<\/i><\/span>AG, we have NEVER encountered a code to which we could not apply our tool, dco\/c++. This is thanks to the flexibility and the features of dco\/c++ (which form the basis for resolving all challenges), and the huge experience AD specialists at <span class=\"nag-n-override\" style=\"margin-left: 0 !important;\"><i>n<\/i><\/span>AG have with large-scale mathematical simulation codes.<\/p>\n","protected":false},"author":5,"featured_media":988,"parent":0,"menu_order":0,"template":"","meta":{"content-type":"","footnotes":""},"post-tag":[16,25],"class_list":["post-1432","insights","type-insights","status-publish","has-post-thumbnail","hentry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v25.8 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>AD Myths Debunked: It Won\u2019t Work on My Code - nAG<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/nag.com\/insights\/ad-myths-debunked-it-wont-work-on-my-code\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"AD Myths Debunked: It Won\u2019t Work on My Code - nAG\" \/>\n<meta property=\"og:description\" content=\"At NAG, we have NEVER encountered a code to which we could not apply our tool, dco\/c++. This is thanks to the flexibility and the features of dco\/c++ (which form the basis for resolving all challenges), and the huge experience AD specialists at NAG have with large-scale mathematical simulation codes.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/nag.com\/insights\/ad-myths-debunked-it-wont-work-on-my-code\/\" \/>\n<meta property=\"og:site_name\" content=\"nAG\" \/>\n<meta property=\"article:modified_time\" content=\"2023-07-05T10:20:32+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/nag.com\/wp-content\/uploads\/2023\/05\/Blog_Post-Myth-1-1024x576.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1024\" \/>\n\t<meta property=\"og:image:height\" content=\"576\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:site\" content=\"@NAGTalk\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/nag.com\/insights\/ad-myths-debunked-it-wont-work-on-my-code\/\",\"url\":\"https:\/\/nag.com\/insights\/ad-myths-debunked-it-wont-work-on-my-code\/\",\"name\":\"AD Myths Debunked: It Won\u2019t Work on My Code - nAG\",\"isPartOf\":{\"@id\":\"https:\/\/nag.com\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/nag.com\/insights\/ad-myths-debunked-it-wont-work-on-my-code\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/nag.com\/insights\/ad-myths-debunked-it-wont-work-on-my-code\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/nag.com\/wp-content\/uploads\/2023\/05\/Blog_Post-Myth-1.png\",\"datePublished\":\"2022-04-06T08:55:00+00:00\",\"dateModified\":\"2023-07-05T10:20:32+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/nag.com\/insights\/ad-myths-debunked-it-wont-work-on-my-code\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/nag.com\/insights\/ad-myths-debunked-it-wont-work-on-my-code\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/nag.com\/insights\/ad-myths-debunked-it-wont-work-on-my-code\/#primaryimage\",\"url\":\"https:\/\/nag.com\/wp-content\/uploads\/2023\/05\/Blog_Post-Myth-1.png\",\"contentUrl\":\"https:\/\/nag.com\/wp-content\/uploads\/2023\/05\/Blog_Post-Myth-1.png\",\"width\":5333,\"height\":3000,\"caption\":\"Automatic Differentiation\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/nag.com\/insights\/ad-myths-debunked-it-wont-work-on-my-code\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/nag.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Insights\",\"item\":\"https:\/\/nag.com\/insights\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"AD Myths Debunked: It Won\u2019t Work on My Code\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/nag.com\/#website\",\"url\":\"https:\/\/nag.com\/\",\"name\":\"NAG\",\"description\":\"Robust, trusted numerical software and computational expertise.\",\"publisher\":{\"@id\":\"https:\/\/nag.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/nag.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/nag.com\/#organization\",\"name\":\"Numerical Algorithms Group\",\"alternateName\":\"NAG\",\"url\":\"https:\/\/nag.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/nag.com\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/nag.com\/wp-content\/uploads\/2023\/11\/NAG-Logo.png\",\"contentUrl\":\"https:\/\/nag.com\/wp-content\/uploads\/2023\/11\/NAG-Logo.png\",\"width\":1244,\"height\":397,\"caption\":\"Numerical Algorithms Group\"},\"image\":{\"@id\":\"https:\/\/nag.com\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/x.com\/NAGTalk\",\"https:\/\/www.linkedin.com\/company\/nag\/\",\"https:\/\/www.youtube.com\/user\/NumericalAlgorithms\",\"https:\/\/github.com\/numericalalgorithmsgroup\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"AD Myths Debunked: It Won\u2019t Work on My Code - nAG","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/nag.com\/insights\/ad-myths-debunked-it-wont-work-on-my-code\/","og_locale":"en_US","og_type":"article","og_title":"AD Myths Debunked: It Won\u2019t Work on My Code - nAG","og_description":"At NAG, we have NEVER encountered a code to which we could not apply our tool, dco\/c++. This is thanks to the flexibility and the features of dco\/c++ (which form the basis for resolving all challenges), and the huge experience AD specialists at NAG have with large-scale mathematical simulation codes.","og_url":"https:\/\/nag.com\/insights\/ad-myths-debunked-it-wont-work-on-my-code\/","og_site_name":"nAG","article_modified_time":"2023-07-05T10:20:32+00:00","og_image":[{"width":1024,"height":576,"url":"https:\/\/nag.com\/wp-content\/uploads\/2023\/05\/Blog_Post-Myth-1-1024x576.png","type":"image\/png"}],"twitter_card":"summary_large_image","twitter_site":"@NAGTalk","schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/nag.com\/insights\/ad-myths-debunked-it-wont-work-on-my-code\/","url":"https:\/\/nag.com\/insights\/ad-myths-debunked-it-wont-work-on-my-code\/","name":"AD Myths Debunked: It Won\u2019t Work on My Code - nAG","isPartOf":{"@id":"https:\/\/nag.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/nag.com\/insights\/ad-myths-debunked-it-wont-work-on-my-code\/#primaryimage"},"image":{"@id":"https:\/\/nag.com\/insights\/ad-myths-debunked-it-wont-work-on-my-code\/#primaryimage"},"thumbnailUrl":"https:\/\/nag.com\/wp-content\/uploads\/2023\/05\/Blog_Post-Myth-1.png","datePublished":"2022-04-06T08:55:00+00:00","dateModified":"2023-07-05T10:20:32+00:00","breadcrumb":{"@id":"https:\/\/nag.com\/insights\/ad-myths-debunked-it-wont-work-on-my-code\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/nag.com\/insights\/ad-myths-debunked-it-wont-work-on-my-code\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/nag.com\/insights\/ad-myths-debunked-it-wont-work-on-my-code\/#primaryimage","url":"https:\/\/nag.com\/wp-content\/uploads\/2023\/05\/Blog_Post-Myth-1.png","contentUrl":"https:\/\/nag.com\/wp-content\/uploads\/2023\/05\/Blog_Post-Myth-1.png","width":5333,"height":3000,"caption":"Automatic Differentiation"},{"@type":"BreadcrumbList","@id":"https:\/\/nag.com\/insights\/ad-myths-debunked-it-wont-work-on-my-code\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/nag.com\/"},{"@type":"ListItem","position":2,"name":"Insights","item":"https:\/\/nag.com\/insights\/"},{"@type":"ListItem","position":3,"name":"AD Myths Debunked: It Won\u2019t Work on My Code"}]},{"@type":"WebSite","@id":"https:\/\/nag.com\/#website","url":"https:\/\/nag.com\/","name":"NAG","description":"Robust, trusted numerical software and computational expertise.","publisher":{"@id":"https:\/\/nag.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/nag.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/nag.com\/#organization","name":"Numerical Algorithms Group","alternateName":"NAG","url":"https:\/\/nag.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/nag.com\/#\/schema\/logo\/image\/","url":"https:\/\/nag.com\/wp-content\/uploads\/2023\/11\/NAG-Logo.png","contentUrl":"https:\/\/nag.com\/wp-content\/uploads\/2023\/11\/NAG-Logo.png","width":1244,"height":397,"caption":"Numerical Algorithms Group"},"image":{"@id":"https:\/\/nag.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/x.com\/NAGTalk","https:\/\/www.linkedin.com\/company\/nag\/","https:\/\/www.youtube.com\/user\/NumericalAlgorithms","https:\/\/github.com\/numericalalgorithmsgroup"]}]}},"_links":{"self":[{"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/insights\/1432","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/insights"}],"about":[{"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/types\/insights"}],"author":[{"embeddable":true,"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/users\/5"}],"version-history":[{"count":3,"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/insights\/1432\/revisions"}],"predecessor-version":[{"id":3002,"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/insights\/1432\/revisions\/3002"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/media\/988"}],"wp:attachment":[{"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/media?parent=1432"}],"wp:term":[{"taxonomy":"post-tag","embeddable":true,"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/post-tag?post=1432"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}