[{"data":1,"prerenderedAt":420},["ShallowReactive",2],{"navigation":3,"search":43,"\u002Finsights\u002Fwhat-does-this-test-actually-prove":278,"site-footer":397,"\u002Finsights\u002Fwhat-does-this-test-actually-prove-authors":406,"\u002Finsights\u002Fwhat-does-this-test-actually-prove-surround":417},[4,26],{"title":5,"path":6,"stem":7,"children":8,"page":25},"Insights","\u002Finsights\u002F","insights",[9,13,17,21],{"title":10,"path":11,"stem":12,"children":-1},"Design Patterns in the Age of AI","\u002Finsights\u002Fdesign-patterns-in-the-age-of-ai\u002F","insights\u002F20260814-design-patterns-in-the-age-of-ai",{"title":14,"path":15,"stem":16,"children":-1},"Complexity Has to Earn Its Place","\u002Finsights\u002Fcomplexity-has-to-earn-its-place\u002F","insights\u002F20260819-complexity-has-to-earn-its-place",{"title":18,"path":19,"stem":20,"children":-1},"Performance is an Architectural Property","\u002Finsights\u002Fperformance-is-an-architectural-property\u002F","insights\u002F20260828-performance-is-an-architectural-property",{"title":22,"path":23,"stem":24,"children":-1},"What Does This Test Actually Prove?","\u002Finsights\u002Fwhat-does-this-test-actually-prove\u002F","insights\u002F20260901-what-does-this-test-actually-prove",false,{"title":27,"path":28,"stem":29,"children":30,"page":25},"Work","\u002Fwork\u002F","work",[31,35,39],{"title":32,"path":33,"stem":34,"children":-1},"Multi-Blockchain, Cross-Language SDKs","\u002Fwork\u002Fmulti-blockchain-sdk\u002F","work\u002F20230930-multi-blockchain-sdk",{"title":36,"path":37,"stem":38,"children":-1},"Coinbase Mesh API Adapter","\u002Fwork\u002Fcoinbase-mesh-api-adapter\u002F","work\u002F20240830-coinbase-mesh-api-adapter",{"title":40,"path":41,"stem":42,"children":-1},"Multi-Blockchain Token Bridge","\u002Fwork\u002Fmulti-blockchain-token-bridge\u002F","work\u002F20251015-multi-blockchain-token-bridge",[44,48,54,59,64,69,74,77,82,87,92,97,102,105,110,115,120,124,127,132,137,142,146,149,154,159,165,170,175,180,185,190,195,198,202,206,211,216,221,224,229,234,239,242,246,250,255,260,263,268,273],{"id":11,"title":10,"titles":45,"content":46,"level":47},[],"Explore how design patterns remain relevant in AI-assisted software development as a shared vocabulary for architecture,\ntradeoffs, and engineering intent. Software developers of a certain age are intimately aware of the famous Design Patterns: Elements of Reusable Object-Oriented Software book.\nThe fundamental premise of the book is that while few software problems are identical, many rhyme.\nEach recipe in the book is essentially a tool in the software developer's toolbox.\nThe art of software development is in the analysis of the problem and the application of the right tool.\nEveryone knows the anecdote that \"if your only tool is a hammer then every problem looks like a nail\".\nAs more and more software development moves to LLMs, the danger becomes treating the LLM as the ultimate hammer.\nIs this a wise approach?\nWhat about classical design patterns?\nDo they still matter at all?",1,{"id":49,"title":50,"titles":51,"content":52,"level":53},"\u002Finsights\u002Fdesign-patterns-in-the-age-of-ai\u002F#the-obvious-solution","The Obvious Solution",[10],"LLMs can generate workable code given a prompt, but they are limited in understanding the broader context of a problem.\nHumans have a lot of implicit assumptions and often use short-circuit reasoning without realizing it.\nIt is hard, if not impossible, for a human to convey all this knowledge to an LLM, so an LLM will often be operating with imperfect information.\nIn order to reach a solution, an LLM will need to make its own assumptions, which might differ from the human's.\nThis can lead it to build a suboptimal solution. As a trivial example, consider asking an LLM to build a simple calculator supporting addition and subtraction: Write a Python application that accepts three parameters where the first is a string and the latter two are decimal numbers.\nIf the string is \"add\", add the two numbers and print the result.\nIf the string is \"sub\", subtract the two numbers and print the result.\nOtherwise, raise an error. The LLM will likely come up with something usable, where the main logic (the arithmetic) looks something like this: def try_execute(operation_name, a, b):\n    if 'add' == operation_name:\n        print(a + b)\n    elif 'sub' == operation_name:\n        print(a - b)\n    else:\n        raise ValueError(f'Unknown operation \"{operation_name}\". Expected \"add\" or \"sub\"') In certain situations, this code might be fine.\nIt is certainly functional and satisfies all the requests in the prompt.\nIf the code is intended to be a quick proof of concept, this approach is perfectly acceptable.",2,{"id":55,"title":56,"titles":57,"content":58,"level":53},"\u002Finsights\u002Fdesign-patterns-in-the-age-of-ai\u002F#when-requirements-grow","When Requirements Grow",[10],"What if the code is actually only the start of a larger project?\nAdditional requirements start streaming in: multiplication, division, and operation-specific validation. We can ask the LLM to do this for us too: Please add support for multiplication (mul) and division (div). The LLM will again come up with something usable: def try_execute(operation_name, a, b):\n    if 'add' == operation_name:\n        print(a + b)\n    elif 'sub' == operation_name:\n        print(a - b)\n    elif 'mul' == operation_name:\n        print(a * b)\n    elif 'div' == operation_name:\n        if b == 0:\n            raise ValueError('Division by zero is not allowed')\n        print(a \u002F b)\n    else:\n        raise ValueError(f'Unknown operation \"{operation_name}\". Expected \"add\", \"sub\", \"mul\" or \"div\"') Quickly, the try_execute function is becoming very busy.\nThe structure is beginning to work against the open-closed principle because adding additional operations requires modifying the main if\u002Felse control flow.\nThe don't repeat yourself (DRY) principle is also being stretched because the supported operations are repeated in the if\u002Felse control flow and the available operations list.\ntry_execute is beginning to have too many responsibilities.\nIt now contains dispatch, validation, calculation, and presentation logic.\nThese crosscurrents of responsibility are beginning to make maintenance difficult.\nFor example, if we wanted to change the output format, we'd need to change all four print statements.",{"id":60,"title":61,"titles":62,"content":63,"level":53},"\u002Finsights\u002Fdesign-patterns-in-the-age-of-ai\u002F#applying-an-appropriate-pattern","Applying an Appropriate Pattern",[10],"Luckily, we know some design patterns that can help improve this situation.\nWe can take inspiration from the chain of responsibility and strategy patterns discussed in Design Patterns and adapt them to better fit our problem.\nThis customization is essential because design patterns are only guidelines.\nThey should not be thought of as overly strict and all or nothing.\nAll problems are different and might require slightly different applications.\nThe art of software development comes from knowing how to adapt and apply these patterns to the problem at hand. First, we define an Operation object that is composed of the following members: name - identifies the operation that the object handlescheck_parameters - inspects both parameters for validityexecute - performs the arithmetic operation given the two parameters Next, we create a list of supported operations and search it in try_execute.\nOnce a matching operation is found, we use it to check parameters and perform the desired arithmetic operation. The updated code will look something like this: from collections import namedtuple\n\nOperation = namedtuple('Operation', ('name', 'check_parameters', 'execute'))\n\ndef require_nonzero(description, value):\n    if 0 == value:\n        raise ValueError(f'{description} by zero is not allowed')\n\noperations = [\n    Operation('add', lambda _a, _b: None, lambda a, b: a + b),\n    Operation('sub', lambda _a, _b: None, lambda a, b: a - b),\n    Operation('mul', lambda _a, _b: None, lambda a, b: a * b),\n    Operation('div', lambda _a, b: require_nonzero('Division', b), lambda a, b: a \u002F b)\n]\n\ndef try_execute(operation_name, a, b):\n    operation = next((operation for operation in operations if operation.name == operation_name), None)\n    if not operation:\n        raise ValueError(f'Unknown operation \"{operation_name}\". Expected {\" or \".join(f\"\\\"{operation.name}\\\"\" for operation in operations)}')\n\n    operation.check_parameters(a, b)\n    print(operation.execute(a, b)) Notice the differences compared to the obvious solution.\nThe try_execute function is now closed for modification when adding new operations, in accordance with the open-closed principle.\nAdding additional operations only involves updating the list of operations.\nWhile this trivial example uses lambdas to encapsulate different operation logic, in a more complex example, the logic differences would likely be encapsulated in separate classes with a shared interface.\nValidation (check_parameters) and computation (execute) are clearly separated in alignment with the single responsibility principle.\nFinally, the list of supported operations is generated dynamically.\nEach operation's name only appears once in the code, better conforming with the DRY principle. Suppose we get a new requirement to add support for modulo.\nWe only have to add one additional entry to operations.\nNothing else needs to change. Operation('mod', lambda _a, b: require_nonzero('Modulo', b), lambda a, b: a % b)",{"id":65,"title":66,"titles":67,"content":68,"level":53},"\u002Finsights\u002Fdesign-patterns-in-the-age-of-ai\u002F#design-patterns-as-a-shared-vocabulary","Design Patterns as a Shared Vocabulary",[10],"Design patterns have always been used as a lingua franca among developers.\nA software developer could suggest using a design pattern - the flyweight pattern, for example - and another would understand the suggestion immediately without it needing to be explained from first principles.\nIn the age of AI, the other developer is an LLM instead of a human.\nThe shared vocabulary is more, not less, important now.\nSuggesting a design pattern communicates the developer's intent clearly and effectively to the LLM.\nSome of the architectural intent that led the developer to choose that pattern is encoded in the choice itself and implicitly communicated to the LLM. We believe it is best to think of an LLM as a collaborator.\nLet the human do what the human is good at, and let the LLM do what the LLM is good at.\nDon't treat it like a universal hammer and expect it to do everything.\nTeams that do this will likely end up with suboptimal solutions. With an AI collaborator, knowing how to implement a design pattern becomes less important, while knowing when to use one becomes more important.",{"id":70,"title":71,"titles":72,"content":73,"level":53},"\u002Finsights\u002Fdesign-patterns-in-the-age-of-ai\u002F#takeaways","Takeaways",[10],"One of our core beliefs is that software development is as much an art as a science.\nWhile the limitations of the obvious solution may be clear, the pattern-based solution is not a panacea either.\nLayers of abstraction can add complexity and maintenance overhead of their own, so they should be created with careful thought.\nThe appropriateness of the solution depends on the context - both technical and business - for which it is built.\nIf many operations are likely to be added in the future, that is a point in favor of the abstractions.\nThe additional flexibility and maintainability will be very desirable.\nIf the code is expected to have a short lifecycle before being discarded, that is a point against the abstractions.\nThe additional complexity might make the code more difficult to understand, which is especially undesirable if it is intended to be used as an example.\nIn software development, like everything else, there are tradeoffs everywhere. An experienced human developer will typically have a better understanding of the overall context - both explicit and implicit - of a project, some of which will be difficult to express to an LLM.\nThis puts the developer in a better position to design the higher-level architecture and weigh the interests of various stakeholders.\nIn contrast, LLMs shine at fast prototyping and can produce code very rapidly.\nUsed without sufficient context or architectural guidance, that speed can just as easily accelerate the accumulation of technical debt.\nThey can be given instructions to use specific design patterns in certain situations, which might lead to better outcomes. The human can relay goals, constraints, and relevant context to the LLM collaborator.\nThis can include specific design patterns to use or even a template for the LLM to complete.\nLLMs can quickly prototype solutions based on these strictures and discover alternatives.\nBoth can iterate over architecture and implementation improvements, eventually reaching a good endpoint.\nUltimately, the human is responsible for determining whether the result fits the actual problem. html pre.shiki code .spNyl, html code.shiki .spNyl{--shiki-light:#9C3EDA;--shiki-default:#C792EA;--shiki-dark:#C792EA}html pre.shiki code .s2Zo4, html code.shiki .s2Zo4{--shiki-light:#6182B8;--shiki-default:#82AAFF;--shiki-dark:#82AAFF}html pre.shiki code .sMK4o, html code.shiki .sMK4o{--shiki-light:#39ADB5;--shiki-default:#89DDFF;--shiki-dark:#89DDFF}html pre.shiki code .sHdIc, html code.shiki .sHdIc{--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#EEFFFF;--shiki-default-font-style:italic;--shiki-dark:#BABED8;--shiki-dark-font-style:italic}html pre.shiki code .s7zQu, html code.shiki .s7zQu{--shiki-light:#39ADB5;--shiki-light-font-style:italic;--shiki-default:#89DDFF;--shiki-default-font-style:italic;--shiki-dark:#89DDFF;--shiki-dark-font-style:italic}html pre.shiki code .sfazB, html code.shiki .sfazB{--shiki-light:#91B859;--shiki-default:#C3E88D;--shiki-dark:#C3E88D}html pre.shiki code .sTEyZ, html code.shiki .sTEyZ{--shiki-light:#90A4AE;--shiki-default:#EEFFFF;--shiki-dark:#BABED8}html pre.shiki code .sBMFI, html code.shiki .sBMFI{--shiki-light:#E2931D;--shiki-default:#FFCB6B;--shiki-dark:#FFCB6B}html pre.shiki code .sbssI, html code.shiki .sbssI{--shiki-light:#F76D47;--shiki-default:#F78C6C;--shiki-dark:#F78C6C}html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html pre.shiki code .swJcz, html code.shiki .swJcz{--shiki-light:#E53935;--shiki-default:#F07178;--shiki-dark:#F07178}",{"id":15,"title":14,"titles":75,"content":76,"level":47},[],"The right level of software complexity is a Goldilocks problem.\nToo little and too much can both create technical debt.\nJudgment matters even more with AI-assisted coding. Discretion plays a larger role in software development than most people believe.\nTrade-offs are everywhere, and choosing the right ones can be the difference between elegance and crud.\nGood developers are knowledgeable about design patterns and the strengths and weaknesses of their programming languages.\nGreat developers need to understand the broader context they're building within.\nHow will this software be deployed in practice?\nWhat are the broader financial and business constraints?\nSometimes a highly flexible, abstracted design is absolutely the correct choice.\nOther times, it is a poor fit for the actual requirements, bringing along unnecessary maintenance overhead.",{"id":78,"title":79,"titles":80,"content":81,"level":53},"\u002Finsights\u002Fcomplexity-has-to-earn-its-place\u002F#anecdote-too-little-complexity","Anecdote: Too Little Complexity",[14],"In one of my first jobs, I worked on a suite of enterprise anti-virus products.\nThe products hooked into Exchange and SharePoint and dispatched content to a separate process to perform the anti-virus scanning.\nBy the time I joined the team, the external process code had evolved piecemeal over time and was exhibiting the god object anti-pattern. Unfortunately, the reality was even worse.\nThe product suite supported three scanning hooks - Exchange email receipt, SharePoint document upload and manual Exchange mailbox scan.\nThere was a separate scanning process for each of those, each with its own god object.\nThree separate files, each over ten thousand lines of code, with over 90% similarity.\nThe reasons for most of the differences were lost to the passage of time and the turnover of developers.\nWhat remained was difficult to maintain.\nMany changes needed to be synchronized across all three god-object files. It's unlikely that anyone would set out to reach this end state, but it's valuable to analyze the decisions that led to this point.\nThe company started by offering only an anti-virus solution for Exchange email receipt.\nEventually, they decided to expand their product portfolio by migrating their core value proposition to other workloads.\nThey identified the potential reputational damage that could come if they broke their existing product offering.\nThey were not confident in their testing system to catch problems, so they decided to mitigate the risk by making a separate copy of the god-object workflow for each integration.\nThis decision may have made sense at the time, but it led to innumerable future maintenance headaches.\nOften, a bug fixed in one workflow was not correctly ported to all the others.\nPreviously diagnosed and fixed bugs could and would linger in less popular workflows. In hindsight, the \"simple\" choice of making multiple copies of the god-object workflow was clearly sub-optimal.\nAll workflows were essentially pipelines with the same stages in the same order.\nNearly all of the differences were in adapting the incoming data from the integration points prior to forwarding them to the main pipeline.\nExtracting the original pipeline into a separate component would have been the better long-term choice.\nWell-defined extensibility points would allow real, intentional differences if necessary. We don't know what led to the initial decision to make the first copy.\nIt's possible that the business was operating under intense time or other external pressures, which would make it more understandable.\nNonetheless, the cost of that decision became more and more expensive as more copies were made and they diverged.\nWhile a more thoughtful alternative was readily apparent in hindsight, the cost and difficulty of implementing it increased over time.\nArchitecture needs to be reconsidered when the assumptions that produced it change.",{"id":83,"title":84,"titles":85,"content":86,"level":53},"\u002Finsights\u002Fcomplexity-has-to-earn-its-place\u002F#anecdote-too-much-complexity","Anecdote: Too Much Complexity",[14],"Later on, I was working on a product that provided basic Data Loss Prevention (DLP) by extracting text from files and searching it for potentially prohibited content (e.g., personally identifiable information, like SSNs, or other private information).\nThe product was highly flexible and modeled as a state machine.\nAs a consequence of the state machine logic, multiple processing nodes could be reordered dynamically.\nWhile this might seem useful, this solution was a little too clever for its own good. After spending time debugging issues with the state machine and data flows through it, the appropriateness of this abstraction came into question.\nEvery data packet flowed through the same stages in the same order: inspect the type of file, parse it (e.g., extract files from ZIP archives), extract text content, inspect text content.\nThe argument for the state machine was that any stage could result in a terminal error condition, so processing could proceed either to the next stage or the error stage.\nIn reality, this could be better modeled as a pipeline with short-circuiting on error conditions. The state machine added unnecessary complexity and maintenance costs for no benefit.\nIt added the overhead of an orchestrator for dispatching work from stage to stage.\nIt made debugging issues a bit more difficult since it allowed dynamic control flow.\nNone of this was needed.\nThe wrong abstraction was applied.\nBy making a slight change to the adapter on top of each stage, this entire subsystem could be replaced by a for loop with an error condition check! A very senior software architect at the company was responsible for the state machine design.\nWe don't know if he wanted to support other scenarios where a state machine may have made more sense, but there was never any strong argument for why it was needed or might be needed in the future.\nRegardless, in its actual and envisioned production use, the abstraction was misapplied.\nIt led to added costs without any benefits.\nArchitecture needs to be kept grounded in realistic business requirements.",{"id":88,"title":89,"titles":90,"content":91,"level":53},"\u002Finsights\u002Fcomplexity-has-to-earn-its-place\u002F#anecdote-applications-to-ai-assisted-coding","Anecdote: Applications to AI-Assisted Coding",[14],"Recently, I was using AI to do a light refactoring of a Nuxt Content blog site.\nOriginally, the site was structured with all the author details embedded in each blog page.\nThis led to quite a bit of duplication and made it quite cumbersome to change an author's avatar.\nAs an improvement, I instructed the AI to refactor the author content into a separate Nuxt collection. When I reviewed the AI's work, I noticed one strange change.\nIt modified the homepage to load the new authors collection and merge it with the blog posts.\nThis is not necessary because the author information is not displayed as part of the simple blog listing on the homepage.\nThe AI decided to create a collection structurally identical to the original collection, with the embedded authors.\nThis retains the flexibility to eventually display the author images.\nHowever, for my use case, where I have no desire to ever display the author images, it was sub-optimal.\nIt added unnecessary coupling between the homepage and the authors collection, along with a (very) slight performance penalty, for no actual benefit. If I had not reviewed the code, I likely wouldn't have noticed anything.\nAI dramatically reduces the cost of creating complexity, but it doesn't eliminate the cost of owning it.\nUnchecked, decisions that add a little complexity here and there will eventually snowball into an overly complex system.\nThe developer is still ultimately responsible for the system delivered and needs to use discretion when using AI.",{"id":93,"title":94,"titles":95,"content":96,"level":53},"\u002Finsights\u002Fcomplexity-has-to-earn-its-place\u002F#too-little-too-much-just-right","Too Little, Too Much, Just Right",[14],"Complexity isn't something we should strive to maximize or minimize.\nSomething too complex for one project might be too simple for another.\nWe need to use our discretion as developers to pick the right level of complexity for the problem at hand.\nOften there are trade-offs, but we should always be able to justify the level picked. Eschewing abstraction where it is needed can be just as harmful as adding abstractions to handle phantom requirements.\nThe code might work, but it is harder to understand and maintain. Abstractions often emerge naturally during development.\nIn the \"Too Little Complexity\" discussion, an abstraction wasn't necessary when there was only a single integration.\nThe requirement emerged when a second integration was added.\nIt's generally better to start with a simple solution, and add complexity only when the requirements justify it.\nIn the \"Too Much Complexity\" discussion, this advice was not followed.\nInstead, a complex state machine subsystem was built when a pipeline would have been sufficient.",{"id":98,"title":99,"titles":100,"content":101,"level":53},"\u002Finsights\u002Fcomplexity-has-to-earn-its-place\u002F#conclusion","Conclusion",[14],"We have reviewed a handful of situations where abstractions have been omitted or misapplied.\nIn the \"Too Little Complexity\" discussion, we see the danger in simplicity.\nIn the \"Too Much Complexity\" discussion, we see the danger in complexity.\nIn the \"Applications to AI-Assisted Coding\" discussion, we see that these architectural considerations and principles remain applicable today. The developer needs to make decisions that will enable both present and future requirements.\nThese requirements will guide the appropriate level of abstraction and extensibility. In all of these examples, the solutions all worked.\nNevertheless, the systems were loaded with latent technical debt.\nAn under-abstracted system can lead to unnecessary duplication and coupling.\nAn over-abstracted system can be harder to understand and debug.\nIn both cases, the costs of maintenance and future enhancements are larger than necessary. Simply \"working\" is necessary, but it is not sufficient for a well-designed system.\nFor long-lived software, we also need to consider the lifetime cost of the decisions we make.\nComplexity can either increase or decrease that cost.\nWhen we introduce it, we need to be able to explain what requirement justifies it.\nComplexity has to earn its place.",{"id":19,"title":18,"titles":103,"content":104,"level":47},[],"Measurement identifies where resources are being consumed.\nUnderstanding why those costs exist determines where to intervene. In a traditional min\u002Fmax problem, there is always a danger of settling into a local optimum instead of a global one.\nTraditional performance tools can make it easy to optimize this way.\nUsing these tools, the optimization process tends to follow these steps: Run a simulated but representative load through the software.Identify a hot path, drill down into it and identify the most expensive operations.Review each expensive operation and attempt to reduce its cost. After going through this performance improvement process a few times, everything will look much better.\nThere won't be any obvious low-hanging fruit to optimize next, and everything will look reasonably lean.\nDespite this, the code might be far away from optimal and be stuck at a local optimum.\nWhy? Because performance tools only give a partial view.\nThey point to where to investigate, not necessarily where to optimize.\nThey can't identify code that shouldn't exist or code that bakes in incorrect assumptions.\nA profiler can identify where resources are currently being spent, but it can't identify whether that cost is warranted.\nGetting beyond that local optimum requires a more holistic view.",{"id":106,"title":107,"titles":108,"content":109,"level":53},"\u002Finsights\u002Fperformance-is-an-architectural-property\u002F#measure-first","Measure First",[18],"A good carpenter measures twice and cuts once.\nSimilarly, a good performance engineer measures before optimizing.\nSoftware developers can spend a lot of time optimizing a code path that is nowhere near a hot path and will have minimal impact on overall system performance.\nThis is not an excuse to intentionally write slow code, but a warning to avoid expending a lot of effort optimizing before measuring. Traditional performance tools allow a developer to find where resources are being consumed.\nThey can measure CPU usage, memory consumption, synchronization contention and other resource costs.\nFor best results, it's important to use these tools to analyze a real-world scenario or a closely simulated one.\nAnalyzing an unrealistic scenario can point to bottlenecks that aren't significant under real-world conditions. When I was analyzing the performance of a file processing pipeline, I had access to statistics about the actual mix of file types seen in production.\nI built a representative corpus composed of this same mix and used it in my performance testing.\nWhile profiling this pipeline, I used a CPU profiler and noticed that there was a lot of time being spent in an XML library.\nThat result was unexpected, so I measured again and saw the same thing.\nNow, I was confident it was not a spurious result and started investigating further. Eventually, I found where the XML processing was happening.\nThis pipeline received files from another process and then reported results about them.\nThe results were serialized in XML by the pipeline process and then immediately deserialized by the other process prior to inspecting or storing them.\nBoth processes were part of the same product.\nThere was no requirement to use XML.\nIt was just chosen as a convenience.\nIn fact, there was no requirement to use a standard structured format at all. After identifying the problem and the true requirements, I proposed an internal binary format to eliminate textual parsing entirely.\nSince the interprocess communication was only a product implementation detail, it had no client-facing impact.\nNonetheless, it improved throughput by approximately 54% compared to using XML. The performance tools pointed to the problem but not the solution.\nA narrow reading of the performance results would have suggested optimizing the XML processing.\nThat would have treated XML as a requirement rather than questioning why it was being used at this boundary.\nWithout going back to the requirements, the option to replace XML would not have been discovered at all.",{"id":111,"title":112,"titles":113,"content":114,"level":53},"\u002Finsights\u002Fperformance-is-an-architectural-property\u002F#question-the-machinery","Question the Machinery",[18],"Sometimes performance problems are spread out throughout a system.\nDegradation comes from the death of a thousand cuts.\nLayers on top of layers of abstractions add up and cost an aggregate performance penalty.\nThis is true even when each layer, in isolation, is performance-tuned and optimized.\nPerformance tools might not find any obvious optimization candidates because the cost is distributed across the system.\nThe next step is to review the architectural decisions that might be imposing distributed costs. The file processing pipeline discussed in the previous section had the ability to load third-party processors and forward them content to scan.\nBased on the trust levels of these processors, they could either be loaded in the main process or hosted in a child process.\nEach processor also needed to be able to be updated independently of the product cadence, so they needed to have a clean module boundary.\nIn order to facilitate these requirements, the original developers created a custom cross-platform C++ ABI framework.\nThis necessitated a lot of conversions to and from custom ABI types, which added overhead even to the in-process calls.\nThe individual conversions and dispatches were cheap in isolation, but in aggregate, they were significant.\nThere were also other latent costs related to the use and orchestration of the ABI objects. Importantly, the product only ever needed to run on Windows operating systems.\nWhile possible, C++ does not have a standard way to define classes and objects that are portable across modules built by different compiler toolchains.\nThe choice to develop a solution from scratch might have made sense if the product needed to run on non-Windows operating systems.\nSince only Windows needed to be supported, Microsoft COM provided an existing solution to these requirements. In order to test the performance impact of this hypothesis, I created a small COM wrapper around one of the processors and built a simple test harness to compare the overhead of the bespoke ABI version and the COM version.\nInitial testing showed a marginal performance improvement, so I went ahead with the migration to COM.\nIn the end, the custom ABI solution and the orchestration code of the ABI objects, amounting to roughly 30,000 lines of code, were deprecated and removed.\nRetesting the new solution found that it was roughly 9% faster than the old one.\nThe initial testing underestimated the impact because it left many of the existing layers in place for a fairer comparison.\nArchitectural overhead does not have to appear as a single expensive operation.\nSmall costs imposed by an architecture can be broadly dispersed and make the architecture itself the optimization target.",{"id":116,"title":117,"titles":118,"content":119,"level":53},"\u002Finsights\u002Fperformance-is-an-architectural-property\u002F#question-the-optimization","Question the Optimization",[18],"Other times, a hot path might have been properly identified and an optimization put in place.\nStill, it's important to review the applicability of the optimization.\nThere are many ways to optimize a hot path.\nThe goal isn't always to find and implement the fastest possible solution.\nIt's to find a solution that meets the performance requirements at a reasonable engineering cost.\nA marginal performance improvement might not justify the future maintenance cost of tens of thousands of lines of code. I was working on a product that had a C# process send data to a C++ process for processing.\nThe interprocess communication (IPC) had already been identified as a potential bottleneck.\nA custom IPC solution had been built that used various techniques depending on file size.\nFor example, it would use memory-mapped files for small files and socket communication for large files.\nThis solution worked well and met the performance requirements, but was never benchmarked against alternative approaches. I began benchmarking the custom IPC solution against alternatives, expecting to prove its performance advantages.\nMy first attempt was to compare it to a simple .NET\u002FCOM interop that sent every file one at a time.\nThe custom solution outperformed it easily.\nNext, I grouped the files in the test set by size and retested.\nThese results revealed something different.\nThe custom solution performed even better for small files, but had almost no advantage over medium and large files.\nThis was an unexpected result and piqued my interest.\nI hypothesized that the .NET\u002FCOM approach had a fixed overhead cost for each dispatch call independent of file size.\nThis would be significant relative to the size-dependent transport costs for small files.\nI decided to buffer multiple small files together until either a specific size was reached or time elapsed.\nThis would allow the per-dispatch costs to be amortized across multiple files.\nAfter implementing this and rerunning the tests, the performance matched that of the custom IPC solution. It's not enough to identify a performance problem and a good solution.\nAlternative solutions need to be considered before undertaking a big effort.\nSometimes, identifying the problem too broadly will not lead to the best solution.\nIn this case, the problem was originally framed as IPC being the bottleneck that needed optimization.\nIn reality, the performance problem was isolated to small files, which had a relatively high fixed per-dispatch overhead.\nIn the first case, the goal was to optimize each individual IPC call.\nIn the second, the goal was to reduce the number of IPC calls.\nThis reframing led to a different solution and the reduction of roughly 20,000 lines of custom IPC code without any measurable performance degradation.",{"id":121,"title":99,"titles":122,"content":123,"level":53},"\u002Finsights\u002Fperformance-is-an-architectural-property\u002F#conclusion",[18],"Many times, performance engineering is viewed as mechanistic.\nFind a slow function and make it faster.\nWhile this is a valid approach, it is much too narrow.\nConstraining a performance investigation to existing code and structures will leave a lot of potential optimizations out of scope. It's important to measure first.\nFind things that are really expensive instead of focusing on things that might be expensive.\nOnce something is identified, analyze it further.\nDetermine why the operation is expensive and whether the expense is a result of satisfying a requirement or just an implementation detail.\nIf the expensive work isn't required, replace or remove it. Examine the architectural decisions for any undesirable costs.\nLook for small costs imposed by the architecture that are individually insignificant but add up in aggregate.\nFind ways to perform expensive work less frequently.\nBenchmark any optimizations already in place against alternatives. In the end, performance tools can only identify where resources are being used.\nEngineering judgment determines what to do with that information.\nIs there something to optimize, eliminate, replace or call less frequently?",{"id":23,"title":22,"titles":125,"content":126,"level":47},[],"Different testing strategies reduce different sources of uncertainty.\nProduction evidence can further improve both the product and how it is tested. A software engineer's job is not done when the code is written.\nEqually important is verifying that the code performs as expected.\nThere are various strategies for testing software.\nSeparately, each can find different gaps in the implementation.\nTogether, they can provide even greater confidence in the system.\nFurthermore, applying insights from real-world deployments can improve the effectiveness of some of these strategies. Illustrating how different testing strategies are used in the context of a real system can help highlight what each contributes.\nConsider a pipeline that accepts an arbitrary stream of binary data.\nIt attempts to determine the file format that's encoded in the stream, such as ZIP or PNG, and parse it into a structured form.\nThe data crosses a trust boundary, so it should be treated as untrusted by the parsers.\nThis article will examine the testing of a ZIP parser, from verifying its behavior in isolation through post-deployment monitoring.",{"id":128,"title":129,"titles":130,"content":131,"level":53},"\u002Finsights\u002Fwhat-does-this-test-actually-prove\u002F#verify-the-functionality","Verify the Functionality",[22],"It's important to verify that the ZIP parser behaves as expected.\nUnit tests allow the targeting of individual behaviors directly.\nIn the case of a ZIP archive, it would be prudent to test each of the compression algorithms (e.g., stored, deflate) individually on a data stream. Writing tests might give insight into parts of the code that are too complex or coupled.\nFor example, while adding tests to the ZIP archive, each compression algorithm was originally coupled within the parsing logic.\nThis made it difficult to test compression edge cases because they required embedding within a valid ZIP archive.\nRefactoring each compression algorithm into its own class simplified testing these edge cases and made adding support for additional algorithms easier.\nAfter this refactoring, a truncated deflate stream or invalid deflate headers could be tested directly instead of also requiring embedding within a full ZIP archive. Having implementation knowledge will help identify edge cases that should be exercised with targeted tests.\nA useful but imperfect proxy is code coverage.\nEach test is only as valuable as what it is asserting.\nCode coverage cannot measure this.\nFor example, if a corrupt compressed byte stream is used in a test, the error branches will be covered in the coverage report, but the behavior could still be wrong if the test isn't asserting an error.\nWhile code coverage is a good guide, it is not sufficient. After gaining confidence that the ZIP parser behaves according to expectations, the next thing to do is to verify it behaves correctly in the broader system.\nEnd-to-end tests should send ZIP archives through the file processing pipeline and validate the processing outputs.\nThe main task is to identify representative ZIP payloads and add them to these tests.\nSince the unit tests already give confidence that the ZIP parser behaves as expected, it's unnecessary to exhaustively test different permutations of ZIP archives at this level too.\nA small representative set (e.g., a ZIP archive containing a single file, a ZIP archive containing multiple files, and a corrupt ZIP archive) will provide a substantial amount of confidence at this level.\nIf there are any other unusual integrations, there should also be individual tests for those.\nThese tests are primarily concerned with the correct integration of the ZIP parser and should be written independently of the specific implementation. At this point, there is a high level of confidence that the code performs as expected both individually and as part of the larger system.\nBoth unit tests and end-to-end tests are types of functional tests that verify behavior.\nThe important qualifier is that this only holds for scenarios envisioned by the developer and tester.\nThe real world might introduce other scenarios that lead to real bugs or crashes.",{"id":133,"title":134,"titles":135,"content":136,"level":53},"\u002Finsights\u002Fwhat-does-this-test-actually-prove\u002F#verify-the-resilience","Verify the Resilience",[22],"Fuzz testing (fuzzing) increases confidence that a system behaves safely when given unexpected data.\nIt's unlikely that every potential edge case was imagined and tested during functional testing.\nFor example, the functional tests should validate an error is raised when the ZIP parser is passed a ZIP archive with an invalid compression method.\nFrom a functional perspective, any invalid compression method seems like it should be treated like any other, so one test seems like enough.\nIf there is a specific invalid compression method that causes a different class of error due to a bug, it's unlikely to be the invalid compression method used in the functional tests. Fuzzing stresses a system by sending it lots of variations of data.\nThere are three fuzzing approaches to consider for the ZIP parser: Completely unstructured: Sends a randomized unstructured byte stream to the pipeline.\nThis is useful for testing how the pipeline handles random inputs, which is important since it is required to accept untrusted data that could be anything.\nIn most cases, this stream will not match any parsers, so it is not very useful in stressing them.Partially unstructured: Sends a known byte stream, randomly modified, to the pipeline.\nThe known byte stream can be one of the ZIP archives used in functional testing.\nAs long as the ZIP archive signature remains, the stream will be passed to the ZIP parser and exercise it.\nSince the generated data is based on a finite input set, it will only be able to produce data similar to that initial set.Structured: Sends a randomized structured byte stream to the pipeline.\nThe fuzzing tool is given a template of a ZIP archive.\nDepending on the tool, this template could be built declaratively or inferred from a set of representative files.\nThis allows it to generate randomized byte streams that conform to the ZIP archive format.\nThese will be passed to the ZIP parser and exercise a broader and deeper set of code branches. The list is ordered from easiest to hardest to set up.\nCode coverage can approximate the thoroughness of each fuzzing approach.\nStructured fuzzing will touch a larger percentage of the ZIP parser's implementation than the other types.\nFor example, structured fuzzing identified a handful of severe issues in the ZIP parser.\nThese included integer overflow from adding manipulated relative offsets and buffer overruns from offsets pointing past the end of the ZIP archive. Fuzzing typically runs for hours or days.\nWhenever unexpected failures or crashes are detected, the byte stream causing the violation is archived for later diagnosis.\nConfidence increases as fuzzing exercises a broader range of inputs without discovering new failures.\nWhile production can always introduce its own wrinkles, confidence at this stage allows the system to be deployed into production.",{"id":138,"title":139,"titles":140,"content":141,"level":53},"\u002Finsights\u002Fwhat-does-this-test-actually-prove\u002F#verify-the-deployment","Verify the Deployment",[22],"After deployment, it's important to monitor real usage.\nProduction alerting and crash analysis are outside the scope of this article.\nInstead, the focus will be on collecting real-world usage data and using that to improve the product. The file processing pipeline was instrumented with counters to obtain real-world usage statistics.\nSince the main purpose of the pipeline is to identify and parse binary data streams, it was important to know what types of files were being seen in the real world.\nThis data helped prioritize the list of additional file parsers to add. The performance monitoring system being used only allowed 64-bit integral key values.\nThe pipeline could identify a large number of file types, so it was easy to map an enum value to an integral key value.\nPerformance counters were added to count the occurrences of each file type.\nThis data was collected and uploaded regularly.\nUnfortunately, after receiving the initial data, it became clear that this was insufficient.\nA plurality of files were simply in the \"unknown\" bucket, which was not helpful. In order to combat this, a simple two-way encoding was used to map a file extension into an appropriate key.\nThe next deployment allowed collection of both file type (identified by byte stream inspection) and file extension (extracted from the filename).\nThe number of \"unknown\" files decreased substantially.\nOne interesting finding was a significant increase in the number of PLIST files for a few months, which was likely due to an Apple operating system update. With this real-world data about file types and extensions seen in the wild, the performance test corpus was updated.\nOriginally, it had contained a fairly equal distribution of file types.\nWith the real-world data, it was tuned to more closely match the real-world distribution.\nUsing the new corpus allowed performance testing that more closely aligned with workloads observed in production.\nThis led to a reordering of optimization priorities.\nMany of the hot paths that showed up when using the original corpus (e.g., ZIP deflate performance) were less significant when using the new data-driven corpus.\nChanging the corpus to reflect production workloads changed the conclusions produced by the performance tests.",{"id":143,"title":99,"titles":144,"content":145,"level":53},"\u002Finsights\u002Fwhat-does-this-test-actually-prove\u002F#conclusion",[22],"Testing is just as important to software development as writing code.\nA well-engineered system doesn't just run.\nIt satisfies its design expectations.\nThere are uncertainties around a code-complete software system: Does the system work as expected and designed?Does the system handle unexpected inputs resiliently?Does the system behave in production as modeled? At every stage, during and after development there's an opportunity to reduce uncertainty by applying one or more testing strategies.\nUnit testing and end-to-end testing reduce uncertainty around the system functioning correctly.\nFuzz testing reduces uncertainty around the system failing on unexpected inputs.\nProduction monitoring reduces uncertainty around real-world usage and workload assumptions. Importantly, each testing strategy will provide better results when paired with inputs targeted to the question being asked.\nFunctional testing is best paired with representative success and failure files, specially crafted to exercise known edge cases.\nFuzz testing benefits from unexpected and pathological inputs that exercise behavior outside the targeted functional tests.\nReal-world statistics obtained through performance monitoring can also be used to inform improvements to both the product and its testing.\nFor example, performance testing is most realistic when using a corpus that closely mirrors real-world customer workloads.\nApplying real-world prevalence data can help build and evolve this corpus. Each test has a purpose and helps reduce uncertainty.\nIt provides evidence about a particular question under particular conditions.\nTogether, a verification strategy composed of different types of tests can provide evidence about different questions and reduce multiple sources of uncertainty.\nConfidence comes with understanding what is and isn't out of scope of the strategy.",{"id":33,"title":32,"titles":147,"content":148,"level":47},[],"Replaced a fragmented collection of blockchain SDKs with consistent Python and JavaScript implementations backed by a shared serialization model. Re-architected a fragmented collection of blockchain SDKs into consistent Python and JavaScript implementations.\nDesigned a custom binary-layout DSL and code-generation system to eliminate hand-written serialization.\nIntroduced common protocol abstractions that simplify development across two related blockchains.",{"id":150,"title":151,"titles":152,"content":153,"level":53},"\u002Fwork\u002Fmulti-blockchain-sdk\u002F#context","Context",[32],"Our client maintained SDKs for two related blockchain protocols across multiple programming languages.\nOver time, the implementations had diverged, creating inconsistent APIs, duplicated serialization logic, and significant maintenance overhead.\nThe client requested a more maintainable and longer-term solution. Although the protocols shared many concepts, important differences prevented the SDKs from simply sharing a common implementation.\nEach SDK had been developed independently, with its own type system, making blockchain-agnostic application code difficult to write.\nIn addition, some of the SDKs had up to three representations of each transaction type.\nEven SDKs targeting the same blockchain shared few conventions across programming languages.\nSerialization was hand-coded across all SDKs, and there was no source of truth for the serialization format.",{"id":155,"title":156,"titles":157,"content":158,"level":53},"\u002Fwork\u002Fmulti-blockchain-sdk\u002F#engineering-approach","Engineering Approach",[32],"After reviewing the existing SDKs, we determined that their implementations had diverged too far to reconcile without significant breaking changes.\nRather than incrementally modifying the existing libraries, we recommended a complete rearchitecture that would address the underlying sources of inconsistency and maintenance overhead.",{"id":160,"title":161,"titles":162,"content":163,"level":164},"\u002Fwork\u002Fmulti-blockchain-sdk\u002F#serialization-dsl-and-code-generator","Serialization DSL and Code Generator",[32,156],"The client's blockchain protocols prescribe exact binary layouts that are optimized for minimal transaction size.\nWe reviewed a handful of general-purpose serialization libraries, including Protocol Buffers, but none could handle these layouts with complete fidelity.\nWe designed a custom DSL that supports the specification of all binary layouts required by the client's protocols.\nWe then wrote a code generator to create objects with serialization and deserialization support directly from the DSL.\nAs a result, the DSL definitions became the single source of truth for the binary format, and all hand-written serialization and deserialization logic was eliminated.",3,{"id":166,"title":167,"titles":168,"content":169,"level":164},"\u002Fwork\u002Fmulti-blockchain-sdk\u002F#common-protocol-abstraction","Common Protocol Abstraction",[32,156],"The client's two blockchains share many higher-level operations - such as creating, signing, and verifying transactions - but differ in the transaction types and cryptographic algorithms used.\nIn the new SDKs, we structured the blockchain-specific code as similarly as possible.\nFor example, there is a KeyPair for each blockchain that has the same methods and fields.\nThis enables application-level reuse via metaprogramming.\nEither KeyPair can be operated on through the same interface without knowing its associated blockchain. We also introduced a higher-level facade for each blockchain to expose multi-step operations and further hide protocol differences.\nThese allow developers to work with domain objects rather than low-level representations.\nFor example, they support signing a transaction object directly rather than a binary buffer.\nTogether, the abstractions allow application code to perform common operations like building, signing, and verifying transactions without being tightly coupled to either blockchain.\nRather than forcing protocol-specific behavior behind a leaky universal implementation, we standardized the shape of the APIs while keeping genuinely different behavior isolated within each blockchain's implementation.",{"id":171,"title":172,"titles":173,"content":174,"level":164},"\u002Fwork\u002Fmulti-blockchain-sdk\u002F#consistent-cross-language-sdks","Consistent Cross-Language SDKs",[32,156],"We delivered two SDKs - one in Python and one in JavaScript - to the client.\nThese SDKs expose the same concepts, organization, and behavior while remaining idiomatic to each language - for example, using snake_case in Python and lowerCamelCase in JavaScript.\nThis allows developers to easily move between the SDKs and transfer knowledge from one SDK directly to the other.",{"id":176,"title":177,"titles":178,"content":179,"level":53},"\u002Fwork\u002Fmulti-blockchain-sdk\u002F#results","Results",[32],"",{"id":181,"title":182,"titles":183,"content":184,"level":164},"\u002Fwork\u002Fmulti-blockchain-sdk\u002F#consolidated-sdk-portfolio","Consolidated SDK Portfolio",[32,177],"We consolidated multiple independent SDKs into one Python SDK and one JavaScript SDK, each of which included support for both of the client's blockchains.\nAfter a transition period, the client deprecated their legacy SDKs in favor of the new implementations.\nThis reduced the number of independent implementations that must be maintained.\nAdditional language support can follow the structure established by the delivered SDKs rather than introducing another bespoke and independently designed API.",{"id":186,"title":187,"titles":188,"content":189,"level":164},"\u002Fwork\u002Fmulti-blockchain-sdk\u002F#declarative-serialization","Declarative Serialization",[32,177],"We made serialization declarative by designing a custom DSL.\nThe client no longer needs to maintain serialization code in multiple places.\nAdding support for a new binary object at the serialization layer now requires only a corresponding definition in the DSL.\nRunning the code generators automatically propagates the corresponding models and serialization logic to the SDKs.",{"id":191,"title":192,"titles":193,"content":194,"level":164},"\u002Fwork\u002Fmulti-blockchain-sdk\u002F#consistent-developer-experience","Consistent Developer Experience",[32,177],"We made development more consistent across both blockchains and programming languages.\nApplication code can more easily support both of the client's blockchains due to the protocol abstractions in the SDK.\nThe consistent experience across Python and JavaScript allows developers to easily switch between the two and take advantage of their existing knowledge.",{"id":37,"title":36,"titles":196,"content":197,"level":47},[],"Built a lightweight server to adapt requests between Coinbase and native blockchain APIs, focused on the capabilities required for the client's exchange integration. Created a lightweight Node.js server that implements the Coinbase Mesh API on top of the native REST APIs of two blockchains.\nBoth blockchain implementations share a common server shell while keeping blockchain-specific translation logic separate.\nAt startup, configuration determines which blockchain implementation the server activates.",{"id":199,"title":151,"titles":200,"content":201,"level":53},"\u002Fwork\u002Fcoinbase-mesh-api-adapter\u002F#context",[36],"Our client wanted to satisfy the technical requirements for an exchange listing on Coinbase.\nThis included providing an implementation of Coinbase's Mesh API for each of their two blockchains. In 2020, Coinbase introduced Rosetta, which was subsequently renamed to Mesh.\nCoinbase claimed its motivation for introducing this specification was to make it easier to build blockchain-agnostic applications - like block explorers and network monitors.\nIn practice, Mesh also shifts much of the work traditionally done by Coinbase's engineering team when adding support for a new blockchain to the blockchain teams requesting integration. Mesh standardizes the API contract, but it cannot completely abstract away the semantics of the underlying blockchain, especially those with less common functionality.\nOne limitation of the Mesh API is that it expects the deconstruction of blocks and transactions into operations, but does not define a set of known operations that must be supported upstream, even for something as fundamental as an account balance change.\nAt a minimum, integrating any new blockchain requires the Mesh API caller (e.g., Coinbase) to manually add support for the custom set of operations a blockchain defines.\nAs a consequence, while the integration costs are substantially reduced, they are not zero.\nThis is especially true if a blockchain exposes custom functionality that few, if any, others do. Mesh was primarily designed to support blockchains like Bitcoin and Ethereum.\nOur client's blockchains did not perfectly mimic either.\nThey used an account-based model, like Ethereum, but supported only a deterministic, fixed set of transactions instead of allowing arbitrary code execution in a VM.",{"id":203,"title":156,"titles":204,"content":205,"level":53},"\u002Fwork\u002Fcoinbase-mesh-api-adapter\u002F#engineering-approach",[36],"After a careful review of the Mesh API and the REST APIs of our client's blockchains, we discussed our findings with them and clarified the implementation priorities.\nThe client's existing REST APIs already met their needs, and they didn't need the ability to create the full breadth of their transactions through the Mesh API.\nRather than implementing transaction construction for every operation supported by the native blockchains, we focused on implementing the capabilities required for the Coinbase integration. These included: Track native and custom-token balance changes while allowing Mesh to initiate native-currency transfers.Track multisignature account and cosignatory changes while allowing Mesh to initiate those changes.",{"id":207,"title":208,"titles":209,"content":210,"level":164},"\u002Fwork\u002Fcoinbase-mesh-api-adapter\u002F#lightweight-adapter-server","Lightweight Adapter Server",[36,156],"We created a lightweight Node.js server that exposes the Mesh Data and Construction APIs.\nThe common server handles transport, configuration, logging, and error handling in a blockchain-agnostic way.\nAt startup, the server reads a configuration file that determines which blockchain-specific Mesh implementation to activate.",{"id":212,"title":213,"titles":214,"content":215,"level":164},"\u002Fwork\u002Fcoinbase-mesh-api-adapter\u002F#native-api-proxy","Native API Proxy",[36,156],"We created a thin proxy around each blockchain's native REST API.\nThe proxy centralized request and error handling and provided caching for frequently requested data.\nMany of the simpler Mesh APIs could be implemented by making one or more calls through this proxy and mapping the responses to the corresponding Mesh API objects.",{"id":217,"title":218,"titles":219,"content":220,"level":164},"\u002Fwork\u002Fcoinbase-mesh-api-adapter\u002F#operation-processing","Operation Processing",[36,156],"The trickiest part of the integration was deconstructing blocks and transactions into operations.\nProcessing all of these operations must reproduce the same account balances as processing the underlying blocks and transactions according to the native blockchain protocol.\nThe resulting balance for every account must match exactly.\nWhile certain transactions that change account balances - like sending funds - are directly observable in the blockchain, others are not.\nFor example, the creation of a custom token involves paying a fee to a predefined account, in accordance with the rules of the protocol, but that balance change is not directly encoded in the transaction.\nAs a result, we needed to review every transaction type and identify all potential direct and indirect balance changes.\nEach balance change then needed to be represented as a corresponding credit or debit operation to an account.\nSimilarly, we needed to extract operations representing the creation and modification of multisignature accounts as well as the cosignatures of those. Alongside this deconstruction, we also added support for the Mesh Construction API, which performs the reverse transformation.\nGiven one or more operations, it will build a transaction, sign it with optional multisig support, and submit it to the native blockchain network.\nFor transaction construction, signing, and submission, we reused the Multi-Blockchain SDK developed in an earlier engagement rather than duplicating blockchain-specific transaction logic in the adapter.",{"id":222,"title":177,"titles":223,"content":179,"level":53},"\u002Fwork\u002Fcoinbase-mesh-api-adapter\u002F#results",[36],{"id":225,"title":226,"titles":227,"content":228,"level":164},"\u002Fwork\u002Fcoinbase-mesh-api-adapter\u002F#shared-adapter-server-shell","Shared Adapter Server Shell",[36,177],"We delivered a single Node.js server that can expose the implemented Mesh API for either blockchain.\nA common server shell allows our client to deploy and operate both implementations consistently.",{"id":230,"title":231,"titles":232,"content":233,"level":164},"\u002Fwork\u002Fcoinbase-mesh-api-adapter\u002F#coinbase-integration-capabilities","Coinbase Integration Capabilities",[36,177],"We implemented the Mesh Data and Construction capabilities required for the client's Coinbase listing effort.\nCallers can track native and custom-token balance changes, send native currency across accounts, and create and manage multisignature accounts without having to learn our client's custom SDKs or REST APIs.\nImplementations for both of our client's blockchains passed all standard Mesh Construction API tests included in Coinbase's Mesh CLI.",{"id":235,"title":236,"titles":237,"content":238,"level":164},"\u002Fwork\u002Fcoinbase-mesh-api-adapter\u002F#verified-data-fidelity","Verified Data Fidelity",[36,177],"Processing the full blockchain history through the Mesh Data API produced account balances that reconciled exactly with those returned by the native APIs.\nImplementations for both of our client's blockchains passed all standard Mesh Data API tests included in Coinbase's Mesh CLI.",{"id":41,"title":40,"titles":240,"content":241,"level":47},[],"Built a configurable bridge framework connecting Ethereum with two proprietary blockchains so their native currencies could be represented and used in Ethereum's broader DeFi ecosystem. Created a configurable token bridge framework for representing one blockchain's native currency as a token on another blockchain.\nThe framework supports converting a blockchain's native currency to and from a token representation on another blockchain, as well as directly swapping native currencies between blockchains.\nThe initial deliverable included support for Ethereum and our client's two blockchains.",{"id":243,"title":151,"titles":244,"content":245,"level":53},"\u002Fwork\u002Fmulti-blockchain-token-bridge\u002F#context",[40],"Our client develops and maintains two blockchains with relatively small ecosystems.\nThey wanted to allow their users to participate in the broader Decentralized Finance (DeFi) ecosystem, but DeFi liquidity is currently highly concentrated in a small number of ecosystems.\nBuilding a competitive DeFi ecosystem across their blockchains was not a practical alternative.\nInstead, they wanted to make their native currencies accessible within the Ethereum DeFi ecosystem by representing them as ERC-20 tokens.\nCoinbase provides a similar offering (cbBTC), which allows BTC held at Coinbase to participate in DeFi ecosystems.",{"id":247,"title":156,"titles":248,"content":249,"level":53},"\u002Fwork\u002Fmulti-blockchain-token-bridge\u002F#engineering-approach",[40],"We began this engagement by reviewing existing token bridges and identifying common requirements.\nOur review identified four distinct responsibilities: tracking bridge account state, discovering conversion requests, fulfilling those requests on the destination network, and independently confirming their fulfillment.\nRather than combining these responsibilities into a long-running bridge service, we implemented them as independent, single-purpose processes.\nThe framework assumes that bridge account private keys are managed through an appropriately secured operational environment.\nCustody and key-management infrastructure were outside the scope of this engagement.",{"id":251,"title":252,"titles":253,"content":254,"level":164},"\u002Fwork\u002Fmulti-blockchain-token-bridge\u002F#single-purpose-workflows","Single-Purpose Workflows",[40,156],"We created a set of single-purpose Python scripts to perform each of the identified actions: Track bridge account state (only required for Stake mode)\nRetrieves bridge account balance changes to calculate the exact token-to-native conversion rate.\nThis workflow must process all blocks after the last checkpoint to capture both directly observable and indirect balance changes.\nUpon completion, it advances the checkpoint to the last processed block.\nThis is the only workflow that requires block-by-block processing.Discover conversion requests\nDownloads conversion requests sent to the bridge account on the source network.\nValid requests are queued as unprocessed.\nInvalid requests are flagged as errors and not further processed.\nThis is the only workflow that downloads transactions sent to the bridge account.Fulfill conversion requests\nProcesses pending conversion requests and initiates corresponding payouts on the destination network.\nFor each unprocessed request, the converted amount is calculated net of fees.\nRequests that cannot cover the required fees are rejected and flagged as errors.\nOtherwise, the payout is submitted and the request is marked as sent.\nTransient failures, such as insufficient bridge funds or exceeding configurable throttling limits, stop processing without losing progress.\nAfter such a failure, processing resumes from the same point when restarted.Verify fulfillment\nMonitors destination-network payouts until they become finalized (irreversible).\nThe status of each pending payout is independently queried from the destination network.\nIts corresponding conversion request is marked as completed only after finalization. All of the scripts are implemented in a blockchain-agnostic way.\nThe shared initialization routine reads the source and destination blockchains from its configuration and instantiates a blockchain-specific facade for each.\nThis also simplifies the reverse operation of converting a token representation back to a native currency.\nBecause the same facade interface is implemented around the source and destination blockchains, the same processing pipeline can be used with the two facades swapped.",{"id":256,"title":257,"titles":258,"content":259,"level":164},"\u002Fwork\u002Fmulti-blockchain-token-bridge\u002F#three-operation-modes","Three Operation Modes",[40,156],"The general bridge framework supports three operating modes: Wrap: Convert a native currency to a token at a fixed 1:1 conversion rate.Stake: Convert a native currency to a token at a dynamic conversion rate.\nAs the bridge account receives additional native currency, such as staking rewards, the conversion rate increases so that each outstanding token represents a proportionally larger amount of native currency.Swap: Directly convert one blockchain's native currency to another blockchain's native currency using an external price oracle. The single set of scripts supports all of these modes as well as the corresponding unwrap and unstake operations, which are the inverses of wrap and stake.\nThe exact mode is specified in the configuration provided to the scripts.\nA blockchain-specific facade may only support a subset of these modes.",{"id":261,"title":177,"titles":262,"content":179,"level":53},"\u002Fwork\u002Fmulti-blockchain-token-bridge\u002F#results",[40],{"id":264,"title":265,"titles":266,"content":267,"level":164},"\u002Fwork\u002Fmulti-blockchain-token-bridge\u002F#one-framework-across-chains-and-modes","One Framework Across Chains and Modes",[40,177],"We delivered a configurable bridge framework composed of a single set of scripts that supports multiple blockchains through blockchain-specific facades and multiple operation modes.\nRunning these scripts as scheduled jobs provides an automated bridge between two blockchains.\nThe client can selectively deploy the blockchain and bridge-mode combinations they need without supporting every mode across every blockchain pair.",{"id":269,"title":270,"titles":271,"content":272,"level":164},"\u002Fwork\u002Fmulti-blockchain-token-bridge\u002F#well-defined-conversion-lifecycle","Well-Defined Conversion Lifecycle",[40,177],"Every conversion request progresses through a well-defined lifecycle: Unprocessed - The request is valid but no payout has been sent.Sent - The payout has been submitted but has not yet become irreversible.Completed (terminal) - The payout has become irreversible.Failed (terminal) - The request cannot be fulfilled and there will be no payout. This lifecycle distinguishes submitted payouts from finalized ones and makes the state of each request explicit.\nProcessing can resume after transient failures without losing progress.\nOperators can distinguish requests that are still in progress from those that have reached a terminal outcome.",{"id":274,"title":275,"titles":276,"content":277,"level":164},"\u002Fwork\u002Fmulti-blockchain-token-bridge\u002F#extensible-blockchain-support","Extensible Blockchain Support",[40,177],"The core scripts are blockchain-agnostic.\nAdding another blockchain requires implementing the shared facade interface for that blockchain and validating the desired bridge modes.\nThe core processing scripts do not need to change because they interact with each blockchain exclusively through that interface.",{"id":279,"title":22,"author":280,"body":281,"date":388,"description":389,"extension":390,"image":391,"meta":392,"navigation":393,"path":394,"seo":395,"stem":24,"__hash__":396},"insights\u002Finsights\u002F20260901-what-does-this-test-actually-prove.md","Libélula Software",{"type":282,"value":283,"toc":382},"minimark",[284,288,291,295,298,301,304,307,310,313,316,319,332,335,338,341,344,347,350,353,356,359,362,373,376,379],[285,286,287],"p",{},"A software engineer's job is not done when the code is written.\nEqually important is verifying that the code performs as expected.\nThere are various strategies for testing software.\nSeparately, each can find different gaps in the implementation.\nTogether, they can provide even greater confidence in the system.\nFurthermore, applying insights from real-world deployments can improve the effectiveness of some of these strategies.",[285,289,290],{},"Illustrating how different testing strategies are used in the context of a real system can help highlight what each contributes.\nConsider a pipeline that accepts an arbitrary stream of binary data.\nIt attempts to determine the file format that's encoded in the stream, such as ZIP or PNG, and parse it into a structured form.\nThe data crosses a trust boundary, so it should be treated as untrusted by the parsers.\nThis article will examine the testing of a ZIP parser, from verifying its behavior in isolation through post-deployment monitoring.",[292,293,129],"h2",{"id":294},"verify-the-functionality",[285,296,297],{},"It's important to verify that the ZIP parser behaves as expected.\nUnit tests allow the targeting of individual behaviors directly.\nIn the case of a ZIP archive, it would be prudent to test each of the compression algorithms (e.g., stored, deflate) individually on a data stream.",[285,299,300],{},"Writing tests might give insight into parts of the code that are too complex or coupled.\nFor example, while adding tests to the ZIP archive, each compression algorithm was originally coupled within the parsing logic.\nThis made it difficult to test compression edge cases because they required embedding within a valid ZIP archive.\nRefactoring each compression algorithm into its own class simplified testing these edge cases and made adding support for additional algorithms easier.\nAfter this refactoring, a truncated deflate stream or invalid deflate headers could be tested directly instead of also requiring embedding within a full ZIP archive.",[285,302,303],{},"Having implementation knowledge will help identify edge cases that should be exercised with targeted tests.\nA useful but imperfect proxy is code coverage.\nEach test is only as valuable as what it is asserting.\nCode coverage cannot measure this.\nFor example, if a corrupt compressed byte stream is used in a test, the error branches will be covered in the coverage report, but the behavior could still be wrong if the test isn't asserting an error.\nWhile code coverage is a good guide, it is not sufficient.",[285,305,306],{},"After gaining confidence that the ZIP parser behaves according to expectations, the next thing to do is to verify it behaves correctly in the broader system.\nEnd-to-end tests should send ZIP archives through the file processing pipeline and validate the processing outputs.\nThe main task is to identify representative ZIP payloads and add them to these tests.\nSince the unit tests already give confidence that the ZIP parser behaves as expected, it's unnecessary to exhaustively test different permutations of ZIP archives at this level too.\nA small representative set (e.g., a ZIP archive containing a single file, a ZIP archive containing multiple files, and a corrupt ZIP archive) will provide a substantial amount of confidence at this level.\nIf there are any other unusual integrations, there should also be individual tests for those.\nThese tests are primarily concerned with the correct integration of the ZIP parser and should be written independently of the specific implementation.",[285,308,309],{},"At this point, there is a high level of confidence that the code performs as expected both individually and as part of the larger system.\nBoth unit tests and end-to-end tests are types of functional tests that verify behavior.\nThe important qualifier is that this only holds for scenarios envisioned by the developer and tester.\nThe real world might introduce other scenarios that lead to real bugs or crashes.",[292,311,134],{"id":312},"verify-the-resilience",[285,314,315],{},"Fuzz testing (fuzzing) increases confidence that a system behaves safely when given unexpected data.\nIt's unlikely that every potential edge case was imagined and tested during functional testing.\nFor example, the functional tests should validate an error is raised when the ZIP parser is passed a ZIP archive with an invalid compression method.\nFrom a functional perspective, any invalid compression method seems like it should be treated like any other, so one test seems like enough.\nIf there is a specific invalid compression method that causes a different class of error due to a bug, it's unlikely to be the invalid compression method used in the functional tests.",[285,317,318],{},"Fuzzing stresses a system by sending it lots of variations of data.\nThere are three fuzzing approaches to consider for the ZIP parser:",[320,321,322,326,329],"ol",{},[323,324,325],"li",{},"Completely unstructured: Sends a randomized unstructured byte stream to the pipeline.\nThis is useful for testing how the pipeline handles random inputs, which is important since it is required to accept untrusted data that could be anything.\nIn most cases, this stream will not match any parsers, so it is not very useful in stressing them.",[323,327,328],{},"Partially unstructured: Sends a known byte stream, randomly modified, to the pipeline.\nThe known byte stream can be one of the ZIP archives used in functional testing.\nAs long as the ZIP archive signature remains, the stream will be passed to the ZIP parser and exercise it.\nSince the generated data is based on a finite input set, it will only be able to produce data similar to that initial set.",[323,330,331],{},"Structured: Sends a randomized structured byte stream to the pipeline.\nThe fuzzing tool is given a template of a ZIP archive.\nDepending on the tool, this template could be built declaratively or inferred from a set of representative files.\nThis allows it to generate randomized byte streams that conform to the ZIP archive format.\nThese will be passed to the ZIP parser and exercise a broader and deeper set of code branches.",[285,333,334],{},"The list is ordered from easiest to hardest to set up.\nCode coverage can approximate the thoroughness of each fuzzing approach.\nStructured fuzzing will touch a larger percentage of the ZIP parser's implementation than the other types.\nFor example, structured fuzzing identified a handful of severe issues in the ZIP parser.\nThese included integer overflow from adding manipulated relative offsets and buffer overruns from offsets pointing past the end of the ZIP archive.",[285,336,337],{},"Fuzzing typically runs for hours or days.\nWhenever unexpected failures or crashes are detected, the byte stream causing the violation is archived for later diagnosis.\nConfidence increases as fuzzing exercises a broader range of inputs without discovering new failures.\nWhile production can always introduce its own wrinkles, confidence at this stage allows the system to be deployed into production.",[292,339,139],{"id":340},"verify-the-deployment",[285,342,343],{},"After deployment, it's important to monitor real usage.\nProduction alerting and crash analysis are outside the scope of this article.\nInstead, the focus will be on collecting real-world usage data and using that to improve the product.",[285,345,346],{},"The file processing pipeline was instrumented with counters to obtain real-world usage statistics.\nSince the main purpose of the pipeline is to identify and parse binary data streams, it was important to know what types of files were being seen in the real world.\nThis data helped prioritize the list of additional file parsers to add.",[285,348,349],{},"The performance monitoring system being used only allowed 64-bit integral key values.\nThe pipeline could identify a large number of file types, so it was easy to map an enum value to an integral key value.\nPerformance counters were added to count the occurrences of each file type.\nThis data was collected and uploaded regularly.\nUnfortunately, after receiving the initial data, it became clear that this was insufficient.\nA plurality of files were simply in the \"unknown\" bucket, which was not helpful.",[285,351,352],{},"In order to combat this, a simple two-way encoding was used to map a file extension into an appropriate key.\nThe next deployment allowed collection of both file type (identified by byte stream inspection) and file extension (extracted from the filename).\nThe number of \"unknown\" files decreased substantially.\nOne interesting finding was a significant increase in the number of PLIST files for a few months, which was likely due to an Apple operating system update.",[285,354,355],{},"With this real-world data about file types and extensions seen in the wild, the performance test corpus was updated.\nOriginally, it had contained a fairly equal distribution of file types.\nWith the real-world data, it was tuned to more closely match the real-world distribution.\nUsing the new corpus allowed performance testing that more closely aligned with workloads observed in production.\nThis led to a reordering of optimization priorities.\nMany of the hot paths that showed up when using the original corpus (e.g., ZIP deflate performance) were less significant when using the new data-driven corpus.\nChanging the corpus to reflect production workloads changed the conclusions produced by the performance tests.",[292,357,99],{"id":358},"conclusion",[285,360,361],{},"Testing is just as important to software development as writing code.\nA well-engineered system doesn't just run.\nIt satisfies its design expectations.\nThere are uncertainties around a code-complete software system:",[320,363,364,367,370],{},[323,365,366],{},"Does the system work as expected and designed?",[323,368,369],{},"Does the system handle unexpected inputs resiliently?",[323,371,372],{},"Does the system behave in production as modeled?",[285,374,375],{},"At every stage, during and after development there's an opportunity to reduce uncertainty by applying one or more testing strategies.\nUnit testing and end-to-end testing reduce uncertainty around the system functioning correctly.\nFuzz testing reduces uncertainty around the system failing on unexpected inputs.\nProduction monitoring reduces uncertainty around real-world usage and workload assumptions.",[285,377,378],{},"Importantly, each testing strategy will provide better results when paired with inputs targeted to the question being asked.\nFunctional testing is best paired with representative success and failure files, specially crafted to exercise known edge cases.\nFuzz testing benefits from unexpected and pathological inputs that exercise behavior outside the targeted functional tests.\nReal-world statistics obtained through performance monitoring can also be used to inform improvements to both the product and its testing.\nFor example, performance testing is most realistic when using a corpus that closely mirrors real-world customer workloads.\nApplying real-world prevalence data can help build and evolve this corpus.",[285,380,381],{},"Each test has a purpose and helps reduce uncertainty.\nIt provides evidence about a particular question under particular conditions.\nTogether, a verification strategy composed of different types of tests can provide evidence about different questions and reduce multiple sources of uncertainty.\nConfidence comes with understanding what is and isn't out of scope of the strategy.",{"title":179,"searchDepth":53,"depth":53,"links":383},[384,385,386,387],{"id":294,"depth":53,"text":129},{"id":312,"depth":53,"text":134},{"id":340,"depth":53,"text":139},{"id":358,"depth":53,"text":99},"2026-09-01 04:00:00","Different testing strategies reduce different sources of uncertainty.\nProduction evidence can further improve both the product and how it is tested.\n","md","\u002Fhero\u002F20260901-what-does-this-test-actually-prove.png",{},true,"\u002Finsights\u002Fwhat-does-this-test-actually-prove",{"title":22,"description":389},"1IdKM9h-e5go2XXmUEVM1WOovo2Zuz8XTdV7wkGj0a8",{"id":398,"email":399,"extension":400,"logo":401,"meta":403,"stem":404,"__hash__":405},"site\u002Fsite.yml","hello@libelula.codes","yml",{"src":402,"alt":280},"\u002Fbrand\u002Flogo\u002Fprimary_logo_48.png",{},"site","1AH6vXdMiU9wDmpJw5an4z8LRDLBx5vmTEzUbIcGdGA",[407],{"id":408,"avatar":409,"description":412,"extension":400,"meta":413,"name":280,"stem":414,"to":415,"twitter":415,"username":415,"__hash__":416},"authors\u002Fauthors\u002Flibelula-software.yml",{"src":410,"alt":411},"\u002Fbrand\u002Flogo\u002Fprimary_logo.png","Libélula Software logo","Software architecture and engineering consultancy",{},"authors\u002Flibelula-software",null,"3O99_si1A7gvm48a56DMJ6wA2IvdReXyeMSqJfaudNM",[418,415],{"title":18,"path":19,"stem":20,"description":419},"Measurement identifies where resources are being consumed.\nUnderstanding why those costs exist determines where to intervene.\n",1788374777936]