External Signing Hashing Algorithm
Introduction
This document specifies the encoding algorithm used to produce a deterministic hash of acom.daml.ledger.api.v2.interactive.PreparedTransaction. The resulting hash is signed by the holder of the external party’s private key. The signature authorizes the ledger changes described by the transaction on behalf of the external party.
The specification can be implemented in any language, but certain encoding patterns are biased due to Canton being implemented in a JVM-based language and using the Java protobuf library. Those biases are made explicit in the specification.
Protobuf serialization is unsuitable for signing cryptographic hashes because it is not canonical. We must define a more precise encoding specification that can be re-implemented deterministically across languages and provide the required cryptographic guarantees. See https://protobuf.dev/programming-guides/serialization-not-canonical/ for more information on the topic.
Versioning
Hashing Scheme Version
The hashing algorithm as a whole is versioned. This enables updates to accommodate changes in the underlying Daml format, or, for instance, to the way the protocol verifies signatures. The implementation must respect the specification of the version it implements.| Protocol Version | Supported Hashing Schemes |
|---|---|
| v34 | V2 |
| v35 | V2, V3 |
| dev | V2, V3, V4 |
Transaction Nodes
Transaction nodes are additionally individually versioned with a Daml version (also called LF version). The encoding version is decoupled from the LF version and implementations should only focus on the hashing version. However, new LF versions may introduce new fields in nodes or new node types. For that reason, the protobuf representation of a node is versioned to accommodate those future changes. In practice, every new Daml language version results in a new hashing version.Specification
General approach
The hash of thePreparedTransaction is computed by encoding the fields specified by this section to byte arrays, and feeding those encoded values into a SHA-256 hash builder. The rest of this section details how to deterministically encode every proto message into a byte array. Sometimes during the process, partially encoded results are hashed with SHA-256, and the resulting hash value serves as the encoding in messages further up. This is explicit when necessary.
Big Endian notation is used for numeric values. Furthermore, protobuf numeric values are encoded according to their Java type representation. Refer to the official protobuf documentation for more information about protobuf to Java type mappings: https://protobuf.dev/programming-guides/proto3/#scalar In particular:
In Java, unsigned 32-bit and 64-bit integers are represented using their signed counterparts, with the top bit simply being stored in the sign bit
Summary of differences between V3 and V4
V4 is currently unstable and subject to changes. For this reason it targets Protocol Version
Dev only.- New fields: Exercise nodes with version
devinclude a newexternal_call_resultsfield. This binds the recorded external-call result payloads shown in prepared transaction review to the external party’s signature. - Final hash: The hashing scheme version byte changes from
0x03(V3) to0x04(V4).
Summary of differences between V2 and V3
The following is a summary of the differences between V2 and V3. The full specification for each version is provided in the sections below, where version-specific details are shown using tabs.- Removed prefixes: V3 removes the
0x01“Node encoding version” byte prefix from all node encodings, and the0x01“Metadata encoding version” prefix from the metadata encoding. - New fields:
key_opt(optional, composite) is added to Create, Exercise, and Fetch nodes.by_key(boolean) is added to Exercise and Fetch nodes. - New node type:
QueryByKeyis introduced in V3. - Metadata:
max_record_timeis added to the metadata encoding. - Final hash: The hashing scheme version byte changes from
0x02(V2) to0x03(V3).
Notation and Utility Functions
encode: Function that takes a protobuf message or primitive typeTand transforms it into an array of bytes:encode: T => byte[]
to_utf_8: Function converting a JavaStringto its UTF-8 encoded version:to_utf_8: string => byte[]
len: Function returning the size of a collection (array,listetc…) as a signed 4 bytes integer:len: Col => Int
split: Function converting a JavaStringto a list ofString, by splitting the input using the provided delimiter:split: (string, char) => byte[]
||: Symbol representing concatenation of byte arrays
[]: Empty byte array. Denotes that the value should not be encoded.from_hex_string: Function that takes a string in the hexadecimal format as input and decodes it as a byte array:from_hex_string: string => byte[]
int_to_string: Function that takes an int and converts it to a string :int_to_string: int => string
some: Value wrapped in a defined optional. Should be encoded as a defined optional value:some: T => optional T
encode_key_with_maintainers(from V3 only): Function encoding aGlobalKeyWithMaintainerscomposite value. This is used for thekeyfield in Create, Exercise, Fetch, and QueryByKey nodes.
encode_external_call_result: Function encoding anExternalCallResultcomposite value. This is used for theexternal_call_resultsfield in Exercise nodes.
Primitive Types
Unless otherwise specified, this is how primitive protobuf types should be encoded.Not all protobuf types are described here, only the ones necessary to encode a
PreparedTransaction message.google.protobuf.Empty
bool
int64 - uint64 - sint64 - sfixed64
int32 - uint32 - sint32 - sfixed32
bytes / byte[]
string
Collections / Wrappers
repeated
repeated protobuf fields represent an ordered collection of values of a specific message of type T. It is critical that the order of values in the list is not modified, both for the encoding process and in the protobuf itself when submitting the transaction for execution. Below is the pseudocode algorithm encoding a protobuf value repeated T list;
This encoding function also applies to lists generated from utility functions (e.g:
split).optional
is_set returns true if the value was set in the protobuf, false otherwise.
map
The ordering ofmap entries in protobuf serialization is not guaranteed, making it problematic for deterministic encoding. To address this, repeated values are used instead of map throughout the protobuf definitions.
gRPC Ledger API Value
Encoding for theValue message defined in com.daml.ledger.api.v2.value.proto For clarity, all value types are exhaustively listed here. Each value is prefixed by a tag unique to its type, which is explicitly specified for each value below.
Unit
Bool
Int64
Numeric
Timestamp
Date
Party
Text
Contract_id
Optional
optional protobuf modifier, with the addition of the type tag prefix.
List
TextMap
Record
Variant
Enum
GenMap
Identifier
Transaction
A transaction is a forest (list of trees). It is represented with a following protobuf message found here. The encoding function for a transaction isencode_node_ids(node_ids) encodes lists in the same way as described before, except the encoding of a node_id is NOT done by encoding it as a string, but instead uses the following encode(node_id) function:
Node
Each node’s encoding is prefixed with additional meta-information about the node, this is made explicit in the encoding of each node.
Exercise and Rollback nodes both have a children field that references other nodes by their NodeId.
The following find_seed: NodeId => optional bytes function is used in the encoding:
some represents a set optional field, none an empty optional field.
Create
- V4 (unstable)
- V3
- V2
Exercise
- V4 (unstable)
- V3
- V2
Fetch
- V4 (unstable)
- V3
- V2
Rollback
- V4 (unstable)
- V3
- V2
Rollback nodes do not have an lf version.
QueryByKey
This node type is only present in hashing scheme V3 and higher.- V4 (unstable)
- V3
Transaction Hash
Once the transaction is encoded, the hash is obtained by runningsha_256 over the encoded byte array, with a hash purpose prefix:
Metadata
The final part ofPreparedTransaction is metadata. Note that all fields of the metadata need to be signed. Only some fields contribute to the ledger change triggered by the transaction. The rest of the fields are required by the Canton protocol but either have no impact on the ledger change, or have already been signed indirectly by signing the transaction itself.
- V4 (unstable)
- V3
- V2
ProcessedDisclosedContract
Metadata Hash
Once the metadata is encoded, the hash is obtained by runningsha_256 over the encoded byte array, with a hash purpose prefix:
Final Hash
Finally, compute the hash that needs to be signed to commit to the ledger changes.- V4 (unstable)
- V3
- V2
PreparedTransaction must be sent to the API to submit the transaction to the ledger.