{"id":1460,"date":"2022-06-09T10:12:00","date_gmt":"2022-06-09T10:12:00","guid":{"rendered":"https:\/\/nag.com\/?post_type=insights&#038;p=1009"},"modified":"2025-05-29T09:59:22","modified_gmt":"2025-05-29T09:59:22","slug":"ad-myths-debunked-its-hard-to-maintain-code-with-ad-embedded","status":"publish","type":"insights","link":"https:\/\/nag.com\/insights\/ad-myths-debunked-its-hard-to-maintain-code-with-ad-embedded\/","title":{"rendered":"AD Myths Debunked: It\u2019s Hard to Maintain Code With AD Embedded"},"content":{"rendered":"<div class=\"container content-area-default \">\n    <div class=\"row justify-content--center\">\n        <div class=\"col-12 col-md-10 col-lg-8 col-xl-6\">\n            <p>This post is part of the\u00a0AD Myths Debunked series.<\/p>\n<p>You\u2019ve decided\u00a0you want\u00a0your C++ library or application to benefit from the advantages of\u00a0AD. So, let\u2019s further assume you successfully integrated AD. At <span class=\"nag-n-override\" style=\"margin-left: 0 !important;\"><i>n<\/i><\/span>AG, we understand things\u00a0change and new code is always being developed. This presents\u00a0you and your development team with another challenge: maintaining the AD in\u00a0your source code. Is maintaining AD code an expensive process that requires the whole development team to learn and understand AD in detail? Or is that a myth?\u00a0 In this article, we look at the potential maintenance costs and pitfalls, depending on which approach to AD you take.<\/p>\n<ul>\n<li>Writing derivative code by hand, not using any AD tools.<\/li>\n<\/ul>\n<p>Writing derivative code by hand is not only difficult and error-prone, you also create two separate codebases: one is the primal code, i.e., the original model, and the other one is the derivative code, the adjoint. There are a few important rules\u00a0to remember when working in this way! First, whenever a change is made in the primal code, the derivative code needs to reflect that change. Every developer in your team needs to be aware of this crucial\u00a0fact and must be able to write the corresponding derivative code. If the primal and\u00a0derivative code run out of sync, the effects can be costly. Furthermore, it is important to be aware that a seemingly tiny change in the primal code may require global changes in the derivative code. Additionally, if, for example, you need Hessians as well as first-order\u00a0sensitivities, the required workload increases. In summary, maintaining a hand-written derivative code usually comes with huge costs and time delays.\u00a0<\/p>\n<ul>\n<li>Using automatic source transformation.\u00a0<\/li>\n<\/ul>\n<p>Assuming you\u2019ve managed to successfully use a source transformation approach on your code, you end up having two separate codebases for the primal and the sensitivities (similar to writing it by hand). You are, though, in a much better position since you can rerun the source-to-source compiler whenever you make a change in your primal. Smartly designed, this can be part of your build system. However, experience shows that today\u2019s source-to-source compilers are not as robust as you\u2019d like them to be. This usually means that the primal code needs some massage to be digestible and the generated code also needs some manual postprocessing. These steps are not easily automatable. Furthermore, depending on the user-friendliness of the tool, cryptic compilation errors can be expected. In summary, this is a much better approach than writing AD code by hand, but it still\u00a0comes with many uncertainties and pitfalls.\u00a0<\/p>\n<ul>\n<li>Using operator overloading techniques.<\/li>\n<\/ul>\n<p>As described in\u00a0previous Myth Busting posts, integrating AD with operator overloading techniques using a robust tool like dco\/c++ is the easiest approach. This holds true for the maintenance of the code. The main benefit is that you have a single code base. Every change in the primal code directly results in the corresponding change for the derivative code. When using templates in C++, all incarnations of your problem, primal, first-, and higher-order derivative codes coincide in the same source. Although compiler messages are likely to be more complex in case of an error, it is standard C++ and developers will be familiar with the types of errors reported. In summary, the approach with the lowest maintenance costs is operator overloading.\u00a0<\/p>\n<p>Maintaining hand-written or source-to-source compiled derivative code is by no means impossible. However, if you rely on the additional performance you might achieve through these approaches, you should\u00a0try to make the generation and build process as robust as possible. Targeted testing and smart coupling with overloading tools can help with this but there will be considerably more effort involved.\u00a0<\/p>\n<p><span class=\"nag-n-override\" style=\"margin-left: 0 !important;\"><i>n<\/i><\/span>AG&#8217;s AD toolset has been developed and enhanced over the last 12 years and it builds upon a further 10 years of AD R&amp;D experience. We know that details matter. Myths are narratives which might sound like truths but by talking through these in some detail and sharing our experiences we hope to help businesses navigate these issues. Results matter, myths should not.<\/p>\n        <\/div>\n    <\/div>\n<\/div>\n\n\n<div class=\"gbc-title-banner tac tac-lg tac-xl\" style='border-radius: 0px; '>\n    <div class=\"container\" style='border-radius: 0px; '>\n        <div class=\"row justify-content--center\" >\n            <div class=\"col-12\"  >\n                <div class=\"wrap pv-4 \" style=\"0pxbackground-color: \">\n                                <div class=\"col-12 col-md-10 col-lg-8 col-xl-6  banner-content\"  >\n    \n                    \n                    <div class=\"mt-1 mb-1 content\"><\/div>\n\n                    \n                    <a href='https:\/\/nag.com\/insights\/' style='background-color: #ff7d21ff; color: #ffffffff; border-radius: 30px; font-weight: 600; ' class='btn mr-1  ' >More Insights <i class='fas fa-angle-right'><\/i><\/a>                <\/div>\n                <\/div>\n            <\/div>\n        <\/div>\n    <\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>Writing derivative code by hand is not only difficult and error-prone, you also create two separate codebases: one is the primal code, i.e., the original model, and the other one is the derivative code, the adjoint. There are a few important rules to remember when working in this way!<\/p>\n","protected":false},"author":5,"featured_media":988,"parent":0,"menu_order":0,"template":"","meta":{"content-type":"","footnotes":""},"post-tag":[16,25],"class_list":["post-1460","insights","type-insights","status-publish","has-post-thumbnail","hentry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v25.8 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>AD Myths Debunked: It\u2019s Hard to Maintain Code With AD Embedded - nAG<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/nag.com\/insights\/ad-myths-debunked-its-hard-to-maintain-code-with-ad-embedded\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"AD Myths Debunked: It\u2019s Hard to Maintain Code With AD Embedded - nAG\" \/>\n<meta property=\"og:description\" content=\"Writing derivative code by hand is not only difficult and error-prone, you also create two separate codebases: one is the primal code, i.e., the original model, and the other one is the derivative code, the adjoint. There are a few important rules to remember when working in this way!\" \/>\n<meta property=\"og:url\" content=\"https:\/\/nag.com\/insights\/ad-myths-debunked-its-hard-to-maintain-code-with-ad-embedded\/\" \/>\n<meta property=\"og:site_name\" content=\"nAG\" \/>\n<meta property=\"article:modified_time\" content=\"2025-05-29T09:59:22+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/nag.com\/wp-content\/uploads\/2023\/05\/Blog_Post-Myth-1-1024x576.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1024\" \/>\n\t<meta property=\"og:image:height\" content=\"576\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:site\" content=\"@NAGTalk\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/nag.com\/insights\/ad-myths-debunked-its-hard-to-maintain-code-with-ad-embedded\/\",\"url\":\"https:\/\/nag.com\/insights\/ad-myths-debunked-its-hard-to-maintain-code-with-ad-embedded\/\",\"name\":\"AD Myths Debunked: It\u2019s Hard to Maintain Code With AD Embedded - nAG\",\"isPartOf\":{\"@id\":\"https:\/\/nag.com\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/nag.com\/insights\/ad-myths-debunked-its-hard-to-maintain-code-with-ad-embedded\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/nag.com\/insights\/ad-myths-debunked-its-hard-to-maintain-code-with-ad-embedded\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/nag.com\/wp-content\/uploads\/2023\/05\/Blog_Post-Myth-1.png\",\"datePublished\":\"2022-06-09T10:12:00+00:00\",\"dateModified\":\"2025-05-29T09:59:22+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/nag.com\/insights\/ad-myths-debunked-its-hard-to-maintain-code-with-ad-embedded\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/nag.com\/insights\/ad-myths-debunked-its-hard-to-maintain-code-with-ad-embedded\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/nag.com\/insights\/ad-myths-debunked-its-hard-to-maintain-code-with-ad-embedded\/#primaryimage\",\"url\":\"https:\/\/nag.com\/wp-content\/uploads\/2023\/05\/Blog_Post-Myth-1.png\",\"contentUrl\":\"https:\/\/nag.com\/wp-content\/uploads\/2023\/05\/Blog_Post-Myth-1.png\",\"width\":5333,\"height\":3000,\"caption\":\"Automatic Differentiation\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/nag.com\/insights\/ad-myths-debunked-its-hard-to-maintain-code-with-ad-embedded\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/nag.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Insights\",\"item\":\"https:\/\/nag.com\/insights\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"AD Myths Debunked: It\u2019s Hard to Maintain Code With AD Embedded\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/nag.com\/#website\",\"url\":\"https:\/\/nag.com\/\",\"name\":\"NAG\",\"description\":\"Robust, trusted numerical software and computational expertise.\",\"publisher\":{\"@id\":\"https:\/\/nag.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/nag.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/nag.com\/#organization\",\"name\":\"Numerical Algorithms Group\",\"alternateName\":\"NAG\",\"url\":\"https:\/\/nag.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/nag.com\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/nag.com\/wp-content\/uploads\/2023\/11\/NAG-Logo.png\",\"contentUrl\":\"https:\/\/nag.com\/wp-content\/uploads\/2023\/11\/NAG-Logo.png\",\"width\":1244,\"height\":397,\"caption\":\"Numerical Algorithms Group\"},\"image\":{\"@id\":\"https:\/\/nag.com\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/x.com\/NAGTalk\",\"https:\/\/www.linkedin.com\/company\/nag\/\",\"https:\/\/www.youtube.com\/user\/NumericalAlgorithms\",\"https:\/\/github.com\/numericalalgorithmsgroup\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"AD Myths Debunked: It\u2019s Hard to Maintain Code With AD Embedded - nAG","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/nag.com\/insights\/ad-myths-debunked-its-hard-to-maintain-code-with-ad-embedded\/","og_locale":"en_US","og_type":"article","og_title":"AD Myths Debunked: It\u2019s Hard to Maintain Code With AD Embedded - nAG","og_description":"Writing derivative code by hand is not only difficult and error-prone, you also create two separate codebases: one is the primal code, i.e., the original model, and the other one is the derivative code, the adjoint. There are a few important rules to remember when working in this way!","og_url":"https:\/\/nag.com\/insights\/ad-myths-debunked-its-hard-to-maintain-code-with-ad-embedded\/","og_site_name":"nAG","article_modified_time":"2025-05-29T09:59:22+00:00","og_image":[{"width":1024,"height":576,"url":"https:\/\/nag.com\/wp-content\/uploads\/2023\/05\/Blog_Post-Myth-1-1024x576.png","type":"image\/png"}],"twitter_card":"summary_large_image","twitter_site":"@NAGTalk","twitter_misc":{"Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/nag.com\/insights\/ad-myths-debunked-its-hard-to-maintain-code-with-ad-embedded\/","url":"https:\/\/nag.com\/insights\/ad-myths-debunked-its-hard-to-maintain-code-with-ad-embedded\/","name":"AD Myths Debunked: It\u2019s Hard to Maintain Code With AD Embedded - nAG","isPartOf":{"@id":"https:\/\/nag.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/nag.com\/insights\/ad-myths-debunked-its-hard-to-maintain-code-with-ad-embedded\/#primaryimage"},"image":{"@id":"https:\/\/nag.com\/insights\/ad-myths-debunked-its-hard-to-maintain-code-with-ad-embedded\/#primaryimage"},"thumbnailUrl":"https:\/\/nag.com\/wp-content\/uploads\/2023\/05\/Blog_Post-Myth-1.png","datePublished":"2022-06-09T10:12:00+00:00","dateModified":"2025-05-29T09:59:22+00:00","breadcrumb":{"@id":"https:\/\/nag.com\/insights\/ad-myths-debunked-its-hard-to-maintain-code-with-ad-embedded\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/nag.com\/insights\/ad-myths-debunked-its-hard-to-maintain-code-with-ad-embedded\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/nag.com\/insights\/ad-myths-debunked-its-hard-to-maintain-code-with-ad-embedded\/#primaryimage","url":"https:\/\/nag.com\/wp-content\/uploads\/2023\/05\/Blog_Post-Myth-1.png","contentUrl":"https:\/\/nag.com\/wp-content\/uploads\/2023\/05\/Blog_Post-Myth-1.png","width":5333,"height":3000,"caption":"Automatic Differentiation"},{"@type":"BreadcrumbList","@id":"https:\/\/nag.com\/insights\/ad-myths-debunked-its-hard-to-maintain-code-with-ad-embedded\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/nag.com\/"},{"@type":"ListItem","position":2,"name":"Insights","item":"https:\/\/nag.com\/insights\/"},{"@type":"ListItem","position":3,"name":"AD Myths Debunked: It\u2019s Hard to Maintain Code With AD Embedded"}]},{"@type":"WebSite","@id":"https:\/\/nag.com\/#website","url":"https:\/\/nag.com\/","name":"NAG","description":"Robust, trusted numerical software and computational expertise.","publisher":{"@id":"https:\/\/nag.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/nag.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/nag.com\/#organization","name":"Numerical Algorithms Group","alternateName":"NAG","url":"https:\/\/nag.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/nag.com\/#\/schema\/logo\/image\/","url":"https:\/\/nag.com\/wp-content\/uploads\/2023\/11\/NAG-Logo.png","contentUrl":"https:\/\/nag.com\/wp-content\/uploads\/2023\/11\/NAG-Logo.png","width":1244,"height":397,"caption":"Numerical Algorithms Group"},"image":{"@id":"https:\/\/nag.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/x.com\/NAGTalk","https:\/\/www.linkedin.com\/company\/nag\/","https:\/\/www.youtube.com\/user\/NumericalAlgorithms","https:\/\/github.com\/numericalalgorithmsgroup"]}]}},"_links":{"self":[{"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/insights\/1460","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/insights"}],"about":[{"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/types\/insights"}],"author":[{"embeddable":true,"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/users\/5"}],"version-history":[{"count":8,"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/insights\/1460\/revisions"}],"predecessor-version":[{"id":63613,"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/insights\/1460\/revisions\/63613"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/media\/988"}],"wp:attachment":[{"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/media?parent=1460"}],"wp:term":[{"taxonomy":"post-tag","embeddable":true,"href":"https:\/\/nag.com\/wp-json\/wp\/v2\/post-tag?post=1460"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}