A Flexible and Sustainable Exchange Contract Architecture

0x protocol has had 3 major versions so far. Each of them introduced multiple feature updates at once and caused disruptions in the ecosystem during the upgrade process. We realized that we need to move to an exchange architecture that simplifies the migration process for makers and takers alike. Additionally, we want to be able to extend the feature set of the the exchange without having to deprecate a prior instance of the Exchange contract. As part of this process, we listed key pain points (below), then brainstormed on a potential solution.

What problems are we solving?

  1. Extending features native to the Exchange contract means redeploying the Exchange contract.
    1. Market makers need to point to a new address for cancellations and signing orders.
    2. Takers need to interact with a new address to fill orders.
  2. When a vulnerability in the Exchange contract is discovered, the entire pipeline must be redeployed and the protocol forked in order to address it promptly.
  3. Monolithic upgrades are cumbersome to develop, audit, and deploy.
    1. This limits our ability to quickly roll out features that address user demands.
  4. It is difficult for a governance model to propose and vote on the entire feature set of the Exchange contract.
  5. The current Exchange contract is bumping into the limit of what is deployable, and maintainable.
  6. Adding functionality via extension contracts can present UX/DX issues because they replace the taker.

What problems we are not solving

This new architecture should allow us patch vulnerabilities in-place without disrupting the ecosystem, but other changes should always require users to deliberately opt-in or intervene. Examples of these changes are:

  • API-breaking changes (new functions, parameters, order formats)
  • Behavioral changes (new settlement, fee schedule, etc)

Why is this important?

It is extremely difficult to migrate the ecosystem to a new protocol version, as we’re operating in a two-sided market where developers cannot afford to prioritize changes to their infra each time the protocol gets patched.

We cannot build an effective, decentralized governance system at the protocol level without giving the ability to make binding decisions on a per-feature basis.

Maintaining and auditing a monolithic model is difficult because the entire contract must be evaluated from scratch each time a new version is proposed.

Initial proposal - Exchange Proxy

This is a a per-function proxy pattern. Every function registered to the proxy contract can have a distinct implementation contract. There will no longer be a collective “version” of the API, opting for a rolling release model.

The Proxy Contract

The ZeroEx contract reroutes (delegate) calls to per-function implementation contracts, called “features.” The contract itself is very lean and only does two things:

  • Forward calls in its fallback function to the current implementation found in the registry (Storage.impls).
  • Track and expose msg.sender when being called from outside the contract, so features can call other features without losing the taker context.
contract ZeroEx {

    // Storage bucket for the proxy.
    struct Storage {
        // Mapping of function selector -> function implementation
        mapping(bytes4 => address) impls;
        // The most recent foreign caller.
        address payable msgSender;
    }

    // Construct and initialize this contract. `bootstrapper` will be
    // delegatecalled into and should set up the initial features.
    constructor(IBootstrapper bootstrapper) public {
        // Initialize msg.sender to this contract.
        _getStorage().msgSender = address(this);
        // Call the bootstrap contract.
        (bool success, bytes memory result) = address(bootstrapper)
            .delegatecall(abi.encodeWithSelector(
                IBootstrapper.bootstrap.selector,
                address(bootstrapper)
            ));
        if (!success) {
            revertData(result);
        }
    }

    fallback() external payable {
        address impl = getImplementation(msg.data.readBytes4());

        // Preserve msg.sender
        Storage storage st = _getStorage();
        address payable prevMsgSender = st.msgSender;
        if (msg.sender != prevMsgSender && msg.sender != address(this)) {
            st.msgSender = msg.sender;
        }

        // Forward the call.
        (bool success, bytes memory result) = impl.delegatecall(msg.data);
        if (!success) {
            revertData(result);
        }

        // Restore msg.sender
        if (msg.sender != prevMsgSender) {
            st.msgSender = prevMsgSender;
        }
        returnData(result);
    }

    receive() external payable {}

    // Get the preserved msg.sender.
    function getMsgSender()
        external
        view
        returns (address msgSender)
    {
       return _getStorage().msgSender;
    }
    
    // Get the implementation for a function.
    function getImplementation(bytes4 selector)
        public
        view
        returns (address impl)
    {
        address impl = _getStorage().impls[selector];
        require(impl != address(0));
        return impl;
    }

    // Get the storage bucket for this contract.
    function _getStorage() private view returns (Storage storage st) {
        bytes32 storageId = 0x1234....;
        assembly { st_slot := storageId }
    }
}

Registry Management

The ZeroEx contract does not implement any functions that write to the registry. This is because we want these registry management functions to also be features we can upgrade. This requires a bootstrapping pattern, which is executed in the proxy’s constructor.

So this brings us to the very first feature of the exchange proxy: the function registry feature:

contract SimpleFunctionRegistryFeature is
    IBootstrapper, // `bootstrap()` function
    FixinAuthorizable // `Authorizable`, but using storage IDs
{
    // Storage bucket for this feature.
    struct Storage {
        // Mapping of function selector -> implementation history.
        mapping(bytes4 => address[]) implHistory;
    }

    // Initialize the implementation registry.
    function bootstrap(address impl) external {
        // Register the registration function (inception vibes).
        extend(this.extend.selector, impl);
        // Register the rollback function.
        extend(this.rollback.selector, impl);
    }

    // Roll back to the last implementation of a function.
    function rollback(bytes4 selector)
        external
        onlyAuthorized
    {
        address[] storage history = _getFeatureStorage().implHistory[selector];
        require(history.length > 0);
        _getProxyStorage().impls[selector] = history[history.length - 1];
        history.pop();
    }

    // Register or replace a function.
    function extend(bytes4 selector, address impl)
        public
        onlyAuthorized
    {
        address[] storage history = _getFeatureStorage().implHistory[selector];
        history.push(_getProxyStorage().impls[selector]);
        _getProxyStorage().impls[selector] = impl;
    }

    // Get the storage bucket for this feature.
    function _getFeatureStorage() private view returns (Storage storage st) {
        // Use a storage ID unique to this contract.
        bytes32 storageId = 0xf00b4...; // 
        assembly { st_slot := storageId }
    }

    // Get the storage bucket for the proxy contract.
    function _getProxyStorage() private view returns (ZeroEx.Storage storage st) {
        // Use the storage ID of the proxy contract.
        bytes32 storageId = 0x1234....; // 
        assembly { st_slot := storageId }
    }
}

Features

Features are contracts that implement one or more (related) functions. For example, a 0x-API market fill feature called QuoteMarketFeature might implement the marketBuyQuote() and marketSellQuote() functions.

Execution Context

All feature functions are executed with delegatecall. This means that they all share the same storage context (see State Management). Despite this, the proxy will still track the msg.sender because there are scenarios where it can be lost, such as when one feature calls another feature through the proxy (this would result in a call then delegatecall).

Features can choose to perform the function lookup and delegatecall it themselves, which saves a hop and would preserve msg.sender, but this is pretty ugly so we should probably only do this when we need to be gas-efficient.

Extending

To add a new feature, we just call extend() for every function the feature exposes. Using an existing selector will clovver the existing implementation. However, extend() will maintain a history of prior implementations if we wish to roll back.

It’s up to the governor to establish and enforce rules around registration.

Deprecation

To remove a feature, we call extend() again for every function that was registered from that feature, but we pass a null implementation address. This will cause the fallback to revert when it looks up the selector.

This architecture has no opinion on how we sunset or tombstone a feature because the governor can enforce this logic itself. For example, if we want to sunset a function in 2 weeks, the governor can queue up a call to extend() and allow anyone to execute it after 2 weeks have passed.

Rollbacks

Every time a function is added by extend(), it keeps a history of previous implementations in a stack. By calling rollback(), the prior implementation of the function will immediately become the head.

Again, it’s up to the governor to enforce rules around rollbacks. For example, we’d likely have no timelock/governance on rollbacks.

Internal Features

Not all feature functions have to be for public consumption. Functions designed for internal use should apply some sort of onlySelf modifier that checks that msg.sender == address(this).

Versioning

Functions themselves don’t follow explicit versioning (e.g, semver). Instead, a function’s version is implied by its selector (major/minor) and implementation (patch).

Major/Minor changes

Every time we make a major or minor (interface or behavioral) change, we should use a new function name, and therefore a new function selector. For interface-breaking changes (like changes to call args or Order structure), the selector would already automatically change. We just need to make this outcome consistent for behavioral changes as well.

This makes it explicit for takers to opt-in to the new behavior by having to call a new function.

Patches

By changing the implementation but keeping the same function selector, we would automatically “upgrade,” takers into the new function behavior. This should only be done for bug fixes or benign patches (e.g., optimizations).

State Management

Because every feature executes under the context of a delegatecall, they all share the same storage. This has the upside of saving a lot of gas if we need to read state shared by multiple features (e.g., checking on order’s filled state) and also preserves state between upgrades. However this does mean we have to take some precautions to avoid accidentally overwriting the state of other features. Fortunately, modern solidity makes this somewhat easy. We can define our storage layout in structs and offset them by some unique ID to ensure they don’t overlap the slots of other features.

I propose a pattern like this:

// Some unique ID for this feature.
bytes32 constant internal STORAGE_ID = 0x1234....;

// Storage layout for this feature.
struct Storage {
    mapping(bytes32 => bytes) myData;
}

// Get the storage bucket for this feature.
function _getStorage() private view returns (Storage storage st) {
    bytes32 storageId = STORAGE_ID;
    assembly { st_slot := storageId }
}

With the above pattern, writing to storage is simply:

_getStorage().myData[...] = ...;

To access the state of another feature, we just use their STORAGE_ID when computing the storage slot.

Examples

V3 fillOrder() Bridge with automatic ETH unwrapping

As a more realistic example, this feature exposes a fillV3OrderWithETH() function that settles on V3 and automatically wraps/unwraps ETH.

// Helper for the `ExchangeHost`.
contract LibExchangeHost {
    // Unique storage ID for the ExchangeHost.
    bytes32 constant private STORAGE_ID = 0x1234....;

    // Storage bucket for the ExchangeHost.
    struct Storage {
        ...
        // The most recent foreign caller.
        address payable msgSender;
    }

    // Get the storage bucket for this feature.
    function getStorage() internal view returns (Storage storage st) {
        bytes32 storageId = STORAGE_ID;
        assembly { st_slot := storageId }
    }

    // Accesses state directly to get the `msgSender`.
    function getMsgSender() internal view returns (address) {
        return getStorage().msgSender;
    }
}

// Helper for the `V3Bridge` feature.
library LibV3BridgeFeature {
    // Some unique ID for this feature.
    bytes32 constant internal STORAGE_ID = 0xf00ba....;

    // Storage layout for this feature.
    struct Storage {
        // The V3 exchange address.
        IV3Exchange exchange;
    }

    // Get the storage bucket for this feature.
    function getStorage() internal view returns (Storage storage st) {
        bytes32 storageId = STORAGE_ID;
        assembly { st_slot := storageId }
    }
}

// Implementation for the `V3Bridge` feature.
contract V3BridgeFeature {

    // Fills a V3 order with automatic ETH conversion.
    function fillV3OrderWithETH(
        Order memory order,
        uint256 takerAssetAmount,
        bytes memory signature
    )
        public
        payable
        returns (FillResults memory fillResults)
    {
        address payable taker = LibExchangeHost.getMsgSender();
        // Transfer taker asset.
        _transferFromTaker(taker, order.takerAssetData, takerAssetAmount);
        // Fill on V3.
        FillResults memory fr = _getExchange().fillOrder
            { value: address(this).balance }
            (order, takerAssetAmount);
        // Transfer maker asset.
        _transferToTaker(
            taker,
            order.makerAssetData,
            fr.makerAssetFilledAmount
        );
        // Return taker asset.
        _transferToTaker(
            taker,
            order.takerAssetData,
            amount - fr.takerAssetFilledAmount
        );
        return fr;
    }

    function _transferFromTaker(
        address payable taker,
        bytes memory assetData,
        uint256 amount
    )
        private
        view
    {
        if (_isAssetWETH(assetData)) {
            _wrapWETH(amount);
        } else {
            _transferFromTaker(taker, assetData, amount);
        }
    }

    function _transferToTaker(
        address payable taker,
        bytes memory assetData,
        uint256 amount
    )
        private
        view
    {
        if (_isAssetWETH(assetData)) {
            taker.transfer(_unwrapWETH(amount));
        } else {
            _transferAsset(assetData, taker, amount);
        }
    }

    // Get the V3 exchange contract.
    function _getExchange() private view returns (IV3Exchange) {
        return LibV3BridgeFeature.getStorage().exchange;
    }
    
    // ... other private functions
}
5 Likes

Very exciting proposal @dorothy-zbornak. Would upgrading 0x V3 to this architecture be a non-breaking change for developers other then updating the contract address they point to?

To be clear, changing an address would be considered a breaking change. But to your point, it depends on whether we decide to namespace the V3 functions (e.g., fillOrder() -> fillOrderV3()). If we don’t, then the answer is yes. The new proxy contract would (eventually) have the same privileges across the asset proxies (so no new allowances needed) and, so long as we perfectly mirror them, the V3 functions will also be there. So, in an ideal world, people coming from V3 and looking for V3 would just need to update the address they’re interacting with.