Multi-Blockchain, Cross-Language SDKs

Re-architected a fragmented collection of blockchain SDKs into consistent Python and JavaScript implementations. Designed a custom binary-layout DSL and code-generation system to eliminate hand-written serialization. Introduced common protocol abstractions that simplify development across two related blockchains.

Context

Our client maintained SDKs for two related blockchain protocols across multiple programming languages. Over time, the implementations had diverged, creating inconsistent APIs, duplicated serialization logic, and significant maintenance overhead. The client requested a more maintainable and longer-term solution.

Although the protocols shared many concepts, important differences prevented the SDKs from simply sharing a common implementation. Each SDK had been developed independently, with its own type system, making blockchain-agnostic application code difficult to write. In addition, some of the SDKs had up to three representations of each transaction type. Even SDKs targeting the same blockchain shared few conventions across programming languages. Serialization was hand-coded across all SDKs, and there was no source of truth for the serialization format.

Engineering Approach

After reviewing the existing SDKs, we determined that their implementations had diverged too far to reconcile without significant breaking changes. Rather than incrementally modifying the existing libraries, we recommended a complete rearchitecture that would address the underlying sources of inconsistency and maintenance overhead.

Serialization DSL and Code Generator

The client's blockchain protocols prescribe exact binary layouts that are optimized for minimal transaction size. We reviewed a handful of general-purpose serialization libraries, including Protocol Buffers, but none could handle these layouts with complete fidelity. We designed a custom DSL that supports the specification of all binary layouts required by the client's protocols. We then wrote a code generator to create objects with serialization and deserialization support directly from the DSL. As a result, the DSL definitions became the single source of truth for the binary format, and all hand-written serialization and deserialization logic was eliminated.

Common Protocol Abstraction

The client's two blockchains share many higher-level operations - such as creating, signing, and verifying transactions - but differ in the transaction types and cryptographic algorithms used. In the new SDKs, we structured the blockchain-specific code as similarly as possible. For example, there is a KeyPair for each blockchain that has the same methods and fields. This enables application-level reuse via metaprogramming. Either KeyPair can be operated on through the same interface without knowing its associated blockchain.

We also introduced a higher-level facade for each blockchain to expose multi-step operations and further hide protocol differences. These allow developers to work with domain objects rather than low-level representations. For example, they support signing a transaction object directly rather than a binary buffer. Together, the abstractions allow application code to perform common operations like building, signing, and verifying transactions without being tightly coupled to either blockchain. Rather than forcing protocol-specific behavior behind a leaky universal implementation, we standardized the shape of the APIs while keeping genuinely different behavior isolated within each blockchain's implementation.

Consistent Cross-Language SDKs

We delivered two SDKs - one in Python and one in JavaScript - to the client. These SDKs expose the same concepts, organization, and behavior while remaining idiomatic to each language - for example, using snake_case in Python and lowerCamelCase in JavaScript. This allows developers to easily move between the SDKs and transfer knowledge from one SDK directly to the other.

Results

Consolidated SDK Portfolio

We consolidated multiple independent SDKs into one Python SDK and one JavaScript SDK, each of which included support for both of the client's blockchains. After a transition period, the client deprecated their legacy SDKs in favor of the new implementations. This reduced the number of independent implementations that must be maintained. Additional language support can follow the structure established by the delivered SDKs rather than introducing another bespoke and independently designed API.

Declarative Serialization

We made serialization declarative by designing a custom DSL. The client no longer needs to maintain serialization code in multiple places. Adding support for a new binary object at the serialization layer now requires only a corresponding definition in the DSL. Running the code generators automatically propagates the corresponding models and serialization logic to the SDKs.

Consistent Developer Experience

We made development more consistent across both blockchains and programming languages. Application code can more easily support both of the client's blockchains due to the protocol abstractions in the SDK. The consistent experience across Python and JavaScript allows developers to easily switch between the two and take advantage of their existing knowledge.