{"id":1434,"date":"2022-03-16T09:24:00","date_gmt":"2022-03-16T09:24:00","guid":{"rendered":"https:\/\/nag.com\/?post_type=insights&p=1077"},"modified":"2023-07-05T10:29:52","modified_gmt":"2023-07-05T10:29:52","slug":"ad-myths-debunked-ill-have-to-re-write-my-libraries","status":"publish","type":"insights","link":"https:\/\/nag.com\/insights\/ad-myths-debunked-ill-have-to-re-write-my-libraries\/","title":{"rendered":"AD Myths Debunked: I\u2019ll Have to Re-write My Libraries"},"content":{"rendered":"
\n
\n
\n

This post is a part of the\u00a0AD Myths Debunked series.<\/p>\n

You\u2019re curious about AD and its benefits, but you\u2019ve heard that you may need to re-write your libraries in order to integrate it. That\u2019s far too much work, so you don\u2019t pursue the idea much further.<\/p>\n

We’ve encountered this scenario many times over the years. When we ask people why the integration cost would be so large, the response is typically \u201cwell, we have a large legacy code\u201d, and they usually go on to explain just how poorly built the legacy code is.<\/p>\n

So, we ask which aspects of this legacy code would make it difficult to implement AD? This question is never met with an answer.<\/p>\n

How hard is it to integrate AD into C++ code? What makes it difficult? Why do some people say that it\u2019s a huge job? Is this really a myth? n<\/i><\/span>AG has helped dozens of clients to integrate our AD solution, dco\/c++ into very large and complex quant libraries, and never once has this led to a re-write of the code \u2013 not even close.<\/p>\n

The ease of integration depends on the sophistication of your AD tool. A poorly designed tool will make the job most unpleasant indeed! An enterprise-class product like dco\/c++ avoids many of the pitfalls that plague these other tools (for example, dco\/c++ doesn\u2019t assume that your code has any particular shape or form).<\/p>\n

When we talk about production C++ libraries, the only AD approach which consistently proves viable is the operator overloading approach. At a high level, the AD tool presents itself as a \u201cnumeric datatype\u201d and the idea is that instead of computing with doubles or floats, you compute with this datatype and the tool takes care of the rest.<\/p>\n

A very efficient AD tool (like dco\/c++) will make heavy use of expression templates to get the compiler to do as much of the heavy lifting as possible. Those of us who bear the scars of working with expression template libraries know that mixing such libraries is a recipe for pain. So, if your library uses Boost or Eigen (or some home-grown expression library), then you must proceed with caution. Luckily, dco\/c++ supports both Boost and Eigen, and is robust in the face of other expression template engines.<\/p>\n

But let\u2019s talk through a real example to put some numbers on the page. We recently (2021) applied dco\/c++ to the master branch of QuantLib. QuantLib is a large code with all the standard C++-isms one would expect: templates (although it is not templated on the arithmetic type), typedefs, polymorphism and virtual inheritance, design patterns, logging (via streams), and it also uses boost and a good portion of C++11 classes and algorithms. Code statistics of the project show the following:<\/p>\n\n\n\n\n\n
Language<\/td>\nfiles<\/td>\nblank lines<\/td>\ncomments<\/td>\ncode lines<\/td>\n<\/tr>\n
C++<\/td>\n915<\/td>\n23793<\/td>\n21250<\/td>\n220934<\/td>\n<\/tr>\n
C\/C++ Header<\/td>\n1374<\/td>\n27319<\/td>\n43601<\/td>\n83622<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

\u00a0
One of the engineers on n<\/i><\/span>AG\u2019s AD team integrated AD into the whole of QuantLib in slightly less than a day. In doing so, a total number of 200 files changed with overall 625 insertions and 462 deletions (that’s characters), 90% of which could safely be done by a tool (indeed, n<\/i><\/span>AG has prototypes of such tools).<\/p>\n

This simple integration of dco\/c++ also achieved some impressive results:<\/p>\n

The performance of AD tools is measured by their \u201cadjoint factor\u201d, the ratio (adjoint runtime) \/ (original runtime). Speedup over finite differences is then roughly (# sensitivities) \/ (adjoint factor).<\/p>\n