Eternity Wall Blog

sign-to-contract

Fri, 13 Apr 2018 00:00:00 +0200

Creating a timestamp involves writing some data on the blockchain, by including those data in a valid bitcoin transaction (e.g. with OP_RETURN). The timestamp then is the cryptographic path that goes from the timestamped object to a block header, passing through the created transaction. However, including those extra data causes your transaction size to increase, making the overall cost of the transaction higher.

Thanks to OpenTimestamps aggregators and calendars, this issue is almost completely solved by collecting several timestamp requests and fitting those inside a single transaction, funded by the calendar. Consequently, if you pass through a calendar, you can timestamp completely for free. Yet, if you want to timestamp by yourself, you have to pay some more fees.

Actually there is a way to avoid to pay those extra satoshis: a math trick that makes possible to include a cryptographic commitment inside an elliptic curve point. Thanks to this gimmick every purely financial transaction can be turned into a timestamping transaction, while still serving for its original purpose and without adding a single additional byte. This allows users not only to timestamp their data with no marginal cost, but, with no expense, they can also help the calendar timestamping its Merkle tip, leading to more frequent timestamps for calendar clients.

The post is structured as follows:

history of the name
the math trick: elliptic curve commitments
- pay-to-contract
- sign-to-contract
- some remarks
an actual implementation w/ Electrum
- how to run
- how to timestamp
- how to verify
conclusions

A bit of history

When hearing the name sign-to-contract, you may wonder from where the contract comes out. To find an answer we have to go back (in 2012) to one of the first applications that was developed: pay-to-contract (sign-to-contract elder brother) which basically allows to commit a contract to a receiver public key. When a customers performs a payment to a merchant, with pay-to-contract, he can commit the bill containing the details of the purchase to the receiving address.

Both pay-to-contract and sign-to-contract are based on the same operation that allows to commit a value to a public key (secp256k1 point) used in a bitcoin transaction. Such operation is called secp256k1 commitment and its functioning is detailed in the following section.

Although secp256k1 commitments could serve different scopes, we focus on timestamping purposes and we take as a reference the PR to OpenTimestamps (and the preceding issue) by Andrew Poelstra.

A bit of math: elliptic curve commitments

Now we need to get our hands into cryptography to unveil how this math trick works under the hood (don’t get scared, we provide a quick recap of the results at the end of the section).

Given an elliptic curve with generator G and a random oracle hash function h. The map m,P -> h(P||m)G + P, where || stands for concatenation, m is the value to commit and P is an elliptic curve point, is a valid commitment operation. The output of the map is C(m,P), which is an elliptic curve point.

The value m is arbitrary, in particular it can be a commitment to something else (e.g. the hash of a document), and P is a public key used for a certain (and possibly independent) purpose. The public key C(m,P) is a commitment to m and P in the sense that if the input is changed, so (m,P)!=(m',P'), then the output changes, C(m,P)!=C(m',P'). Thus we can use C as a commitment operation (like a hash function) inside a timestamp proof.

Now, to curb the abstraction, choose secp256k1 as elliptic curve and sha256 as hash function. The motivation is that we want to use Bitcoin as a notary, and Bitcoin itself grounds its working on the integrity of these two ingredients. We won’t use directly the corresponding C, but instead a slightly modified version: OpSecp256k1Commitment. It takes one input, P||m, and outputs the x-coord of the elliptic curve point C(m,P).

Recap: pick a value to commit m and a public key P (on the curve secp256k1), concatenate them (P||m), then compute:

C(m,P) = SHA256(P||m)G + P 
OpSecp256k1Commitment(P||m) = C(m,P).x

The value obtained is an elliptic curve point that serves two (independent) purposes: is a public key and is a commitment to (m,P) (or only to m, if P is given before).

To timestamp we need to write the x-coord of the elliptic curve point C(m,P) on the blockchain, there are two places where such points can be found, leading to two techniques:

Address/Public key: pay-to-contract
Signature: sign-to-contract

pay-to-contract

Alice has to receive some bitcoins from Bob. Alice has public key A but she also wants to timestamp a document with hash value d. Thus she tells Bob to send his bitcoins to Q = A + h(A||d)G. Bob, who eventually is not aware that he is timestamping Alice’s document, broadcasts the transaction, which after a while becomes part of the blockchain forever. Alice composes her OpenTimestamps proof for her document, by finding Q.x on the chain and conveniently arranging the proof.

However, by implementing this scheme Alice is exposing herself to a new risk. Her new key Q is a deterministic function of x (private key of A) and d, but, while x is derived from the seed (carefully stored), instead the value d is probably something new and unrelated to her wallet. If her computer catches fire, the seed alone won’t be enough to find the private key to redeem the coin locked by Q, since it is required also d, which needs an ad hoc backup that possibly has not taken place.

This technique may lead to a loss of funds, hence using pay-to-contract for timestamping purposes is not advisable.

sign-to-contract

Now Carl has to pay Diana. Carl wants also to timestamp a value c. With sign-to-contract he can fit a commitment to c inside the signature. To understand how, we need to recall how signatures work.

Signatures in Bitcoin are made with ECDSA which involves elliptic curve points, namely a signature is a couple of integers (r,s) in Z/nZ where n is the order of the curve. In a simplified way, a signature for a message m and from a user with private key x is done as follows:

def ECDSAsign(x,m):
    k = deterministic_nonce(x,m)
    R = k*G
    r = R.x mod n
    s = k^(-1) * (m + r*x) mod n
    return (r,s)

Carl wants to spend the coins locked in a given address whose private key is x. He composes an unsigned transaction uTX sending from that address to Diana’s. To sign the transaction he applies ECDSAsign(x,m) where m is the hash value of uTX conveniently serialized. Once produced the signature (r,s) Carl fill uTX with an encoded version of the signature SIG(r,s), obtaining the signed (and broadcastable) transaction TX. Then sends TX to the network and, later on, when confirmed, it becomes part of the blockchain forever.

We can see that inside ECDSAsig an elliptic curve point (R) is used. The idea of sign-to-contract is simply to tweak it with h(R||c), making it also a commitment to another value c.

def ECDSAsign2contract(x,m,c):
    k = deterministic_nonce(x,m)
    R = k*G
    e = k + h(R||c)
    Q = R + h(R||c)*G  # which is Q = e*G = C(c,R)
    q = Q.x mod n
    z = e^(-1) * (m + q*x) mod n
    return (q,z), R

Carl applies ECDSAsign2contract, which produces (q,z) and R. The former is the signature corresponding to the pubkey x*G for the message m, while the latter and c are the ingredients to prove the commitment to Q.x. Now uTX is filled with SIG(q,z) which contains q, which in turn is (almost always) Q.x. As before, the resulting TX then is committed to the blockchain.

Carl then looks at TX, in which he spots Q.x, with TX = TXp||Q.x||TXa. By retrieving the info to link TX to a block header we can complete the timestamp for c:

File sha256: c
Timestamp:
prepend R == R||c
secp256k1commitment == Q.x
prepend TXp == TXp||Q.x
append TXa == TXp||Q.x||TXa
# transaction id ...
sha256
...
sha256
verify BitcoinBlockHeaderAttestation(...)
# Bitcoin merkle root ...

Resuming, while doing a purely financial transaction to Diana, Carl timestamped a value, without adding any bytes, so with zero marginal cost. Diana could be unaware that Carl timestamped, from her perspective the coins she just received are indistinguishable from coins signed in the standard way. Such coins are locked in the pubkey (or address) that Diana told Carl, hence Diana’s new coins are as secured as normal coins.

Carl chose a particular nonce that allowed him to commit to a value. The resulting singature is indistinguishable from a standard one and has the same security. Moreover, if he loses the committed value c, then he loses the ability of proving the timestamp, but he does not compromise any coin: his coins have been already spent and Diana has hers in the desired destination, which is independent from c. For this reason sign-to-contract should be preferred when timestamping.

Remarks

The name is merely due to historical reasons and it came out of the desire to commit to contracts when paying someone. The value committed could be any arbitrary data: for practical timestamping purposes it will be a Merkle tip aggregating several single timestamps.
Each elliptic curve point can include a commitment, thus each pubkey, as well as each signature, could be used as anchoring point. Multiple commitments can be included in a single transaction, however, for timestamping, this is not essential: as pointed out above, a contract can be a commitment to an arbitrary high number of single timestamps.
These techniques (as everything in OTS) does not apply only to Bitcoin, with convenient adjustments, they can be extended to other similar systems, for instance Litecoin or MimbleWimble.
With segwit some issues arise. The signature is committed in the wtxid, but not in the txid: the latter is committed in the transaction Merkle tree, while the former is committed in another Merkle tree, whose tip is inserted in the coinbase transaction. As a result, the signature is committed in the block header, but the path has to traverse both trees and the coinbase. Thus proofs at least double in size, the miner has some control on what is inside the coinbase, so he may decide to include malicious data or make the coinbase so large (multiple KB) that it won’t be possible to create an OTS proof passing through that.

An actual implementation w/ Electrum

Implementing this stuff may seem simple, but in practice it involves many steps very unrelated between each other: extract private keys, perform elliptic curve math, fill the custom signature in the tx, correctly serialize in the tx, retrieve the link from tx to the block header and finally compose the OTS proof.

Anyhow we made it simple with an extension to the Electrum plugin we presented few weeks ago.

We now outline how to run this custom version and then how to create your first sign-to-contract OTS proof.

Remark: at this stage the code is not intended for a general public thus the procedure to make it works may be not super easy to put in place.

Getting started

Beware: what follows is experimental, make sure it is not messing up your Electrum or OpenTimestamps working libraries.

Assuming running on Linux.

You need to use the custom library by apoelstra that integrates OpSecp256k1Commitment in OpenTimestamps. You need to download Electrum from source and integrate it with the timestamp plugin.

git clone https://github.com/apoelstra/python-opentimestamps.git
git clone https://github.com/LeoComandini/electrum-timestamp-plugin.git
git clone git://github.com/spesmilo/electrum.git

Now go to the sign-to-contract (s2c) branch, copy the plugin file in the local Electrum directory, then copy a temporary version of setup.py in the custom version of python-opentimestamps just downloaded.

cd electrum-timestamp-plugin
git checkout s2c
cd ..
cp -r electrum-timestamp-plugin/timestamp electrum/plugins
cp electrum-timestamp-plugin/setup.py python-opentimestamps

Install the custom library.

pip3 install python-opentimestamps/

Follow an adapted version of Electrum README.

Electrum is a pure python application. If you want to use the Qt interface, install the Qt dependencies:

sudo apt-get install python3-pyqt5
sudo apt-get install python3-setuptools
cd electrum
python3 setup.py install

Compile the icons file for Qt:

sudo apt-get install pyqt5-dev-tools
pyrcc5 icons.qrc -o gui/qt/icons_rc.py

If you tried the version of the plugin without s2c remove the db storing the info for generating the proof.

rm db_file.json

Run on testnet

./electrum --testnet

Run on mainnet

./electrum

Create timestamps with your transactions

Enable the plugin:
- Tools -> Plugins -> Timestamp, tick the checkbox
- clicking on Settings you can switch from op_return to sign-to-contract, stay on the latter
- close and restart Electrum to activate the plugin
Visualize your timestamp history list:
- Tools -> Timestamps
Start tracking the file(s) to timestamp:
- click on Add New File and select the file
Create and broadcast a bitcoin transaction including the timestamp:
- on the Send Tab select outputs, amount, fee
- click on Preview
- click on S2C, insert the password, this will insert a commitment to the file you selected in the signature using sign-to-contract
- click on Broadcast
Check the timestamp history:
- Tools -> Timestamps, the file now is an pending state
Wait until the transaction is confirmed, you can now create the timestamp proof (file_name.ots):
- Tools -> Timestamps, click on Upgrade, the timestamp now is complete

You can find the timestamp proof next to the file(s):

/path/file_name.txt
/path/file_name.txt.ots

Try the proof

Note that the .ots contains a OpSecp256k1Commitment so the standard OTS library won’t recognize it. Print it using the python library just installed, for instance with python3 ots-info.py "/path/file_name.txt.ots". Then manually verify the correspondence with the hash of the file and the block header merkle root.

Conclusions

Elliptic curve commitments commit values in elliptic curve points. They can be used with public keys (pay-to-contract) or in signatures (sign-to-contract). Although they have several uses, we confined ourselves to timestamping. pay-to-contract drives you out of a BIP32 logic and may lead you to a loss of funds; sign-to-contract does not, thus is better for timestamping purposes.

Nevertheless, sign-to-contract has some issues that should be addressed. If using segwit, remind that the proof is longer, contains arbitrary data by the miner and could be too large to create the OTS proof.

As of writing, OpSecp256k1Commitment is not yet part of the standard library python-opentimestamps, thus proofs including it won’t be retained valid by the standard clients. If and when the PR will be merged, this new commitment operation can become part of valid timestamps.

sign-to-contract can be integrated in every Bitcoin signing software and can produce OTS proofs with zero marginal cost. We integrated it with Electrum as a plugin: your purely financial transaction can also timestamp stuff with no extra charge.

Since the cost reduction, users may start to consistently help the calendar timestamping its Merkle tip, leading to more frequent timestamps for calendar clients. However, even though this seems promising, it may expose clients to new risks, hence it requires some work to be implemented properly.

P.S. I would like to thank Riccardo Casatta, Peter Todd and Andrew Poelstra for reviewing this post.

P.P.S. This blog post sums up the the content of my master’s degree thesis. The work was done during an internship at Eternity Wall, which put me in the condition to properly understand the subject and deserves most credit. The complete research can be found here, contributions of any kind are more than welcome.

JS OpenTimestamps by example

Mon, 26 Mar 2018 00:00:00 +0200

OpenTimestamps is a standard format for blockchain timestamping.

The reference implementation is written in Python by Peter Todd, and there are libraries in different languages: javascript, java, rust.

Opentimestamps javascript library is written for nodejs and npm package is publicly available. OpenTimestamps library could be used also on the client side, for more details check browser compatibility.

Now let’s begin!

Get started with opentimestamps.js
Get the hash of a file
Timestamp hash and retrieve the proof
Get information about the proof
Upgrade a pending proof
Verify the ots proof on the blockchain

JS OpenTimestamps web-interface and web-tools are available at opentimestamps.org/tools

Get started with opentimestamps.js

In order to get the opentimestamps.js you can proceed in the following manner:

Download a ready file from the website opentimestamps.org (or directly include in your page)

<script src="https://opentimestamps.org/assets/javascripts/vendor/opentimestamps.js"></script>

Or build sources from javascript-opentimestamps

git clone https://github.com/opentimestamps/javascript-opentimestamps.git
npm install --dev
npm run dist

and add to html page:

<script src="./dist/opentimestamps.js"></script>

Get the hash of a file

The first step in order to create a timestamp of a file calculates the hash of the file, you can calculate this hash directly from the webpage using Javascript in the following manners:

using the library crypto-js from string object

<script type="text/javascript" src="/crypto-js.js"></script>
<script type="text/javascript">
    var encrypted = CryptoJS.SHA256("...")
</script>

using hash function available in javascript-opentimestamps from byte array

<script type="text/javascript" src="/opentimestamps.js"></script>
<script type="text/javascript">
    const op = new OpenTimestamps.Ops.OpSHA256()
    const hashByteArray = op.call([...])
</script>

OpenTimestamps supports only sha1, sha256 and ripemd160 hash functions.

Timestamp hash and retrieve the proof

After the calculation of the hash (in hexadecimal format) you can call the methods to generate the timestamps like show in the following code :

const hashData = "16193782f1d839a08f9fc9a94cec1675f1729db1abc15cf9b57f31aa1724a0ae"
const op = new OpenTimestamps.Ops.OpSHA256()
const detached = OpenTimestamps.DetachedTimestampFile.fromHash(op, hashData)
OpenTimestamps.stamp(detached).then( ()=>{
    const ots = detached.serializeToBytes()
    const hex = bytesToHex(ots)
    console.log(hex)
})

The first step is to create an OpenTimestamps.DetachedTimestampFile object which is the virtual representation of a proof in OpenTimestamps protocol. The proof is a merkle tree of data where the original hash is the head and attestations are the tips.

To create a new detached object use fromHash() constructor and pass as input parameters:

hash type instance of OpenTimestamps.Ops (supported: OpSha1, OpSha256, OpRimepd160)
hash data in hex format

Use fromBytes() constructor to create a new detached file from an array of bytes, instead fromHash()

Then call OpenTimestamps.stamp() to timestamp the hash inside detached object.

In stamp() function, detached is an input / output parameters, since OpenTimestamps.DetachedTimestampFile is the proof.

A detached proof could be serialized to bytes with serializeToBytes() and download as ots file; vice-versa, deserialize an byte array to an object with OpenTimestamps.DetachedTimestampFile.deserialize(ots).

Get information about the proof

The stamp command instantly provides an OpenTimestamps proof which contains the promised attestation of the stamped hash. You can use the info command in order to visualize the binary ots in a human-readable format.

const detached = OpenTimestamps.DetachedTimestampFile.deserialize(ots)
const output = OpenTimestamps.info(detached)
console.log(output)

The info output is a chain of crypto-operation that describe a path from the originally stamped hash to one or more attestations. The info of the previous example shows:

File sha256 hash: 16193782f1d839a08f9fc9a94cec1675f1729db1abc15cf9b57f31aa1724a0ae
Timestamp:
append c04158b183a378f3d051d55979a918d8
sha256
 -> append 89c144ad44156173d93cd470569c3087
    sha256
    prepend d37278872acc324f499d54dd488fbe29f76d892204793701e38129e0a2f224b0
    sha256
    prepend 333ea54d67017894b5e581b2bd4e197d3990c031bc78331b7e5c9ee89162c8fd
    sha256
    append bdef644af9388e148d198dec47e62fdaf9e264baf28f9e8115f0468354b2cc44
    sha256
    prepend 5ab7a0c9
    append 05765ac5abc32c3b
    verify PendingAttestation('https://alice.btc.calendar.opentimestamps.org')
 -> append 26ef6425656e1095c5a88badcdea280c
    sha256
    prepend 4e3e4ca04a74381180c6b4a648560e679b321b3e12d1932781b6398267151d70
    sha256
    prepend 61febb5707e8c9ae78c1248ca389d0dba9c00c156c2282020e27fcbae04d1810
    sha256
    prepend 5ab7a0c9
    append 6481affe61f15ea9
    verify PendingAttestation('https://bob.btc.calendar.opentimestamps.org')
 -> append 3dd70ea7ab4c979a7747ab0570d2ea27
    sha256
    prepend b47ab96db80690d1c436022e2869404d7423f7225eb47c81436e342062c70eba
    sha256
    append 7c8025312aa0b452457a1a33d1e12cf0237bc34cb359140dee74f45e6ef8ead7
    sha256
    prepend 5ab7a0c9
    append c302502f95a0f008
    verify PendingAttestation('https://finney.calendar.eternitywall.com')

The OpenTimestamps proof is a chain of unary (hashing function) and binary (append, prepend) operations from the original stamped hash to the attestation. A PendingAttestation is a promise, given from the ots calendar, to put this data in future transaction.

The OpenTimestamps library timestamp can use multiple calendars in order to allow to be more reliable and a robust from side attacks.

Uploading file uploading the proof-file at opentimestamps.org and press INFO button in order to show a web visualizer of the ots proofs

Upgrade a pending proof

The ots proof can be upgraded to resolve pending attestations and in order to obtain a complete timestamp. A PendingAttestation is a promise provided by an ots calendar that only this specific calendar can resolve. A resolved attestation is BitcoinBlockHeaderAttestation and it is verificable by block explorer. A timestamp is completed when there are at least one bitcoin attestation. The following code allows to upgrade a timestamp:

const detachedOts = OpenTimestamps.DetachedTimestampFile.deserialize(ots)
OpenTimestamps.upgrade(detachedOts).then( (changed) =>{
    if(changed){
        console.log("Timestamp upgraded")
        const upgradedOts = detachedOts.serializeToBytes()
        const upgradedHex = bytesToHex(upgradedOts)
        console.log(upgradedHex)
    } else {
        console.log("Timestamp not changed")
    }
})

After upgrade command, the example shows the output of info command:

File sha256 hash: 16193782f1d839a08f9fc9a94cec1675f1729db1abc15cf9b57f31aa1724a0ae
Timestamp:
append c04158b183a378f3d051d55979a918d8
sha256
 -> append 89c144ad44156173d93cd470569c3087
    sha256
    prepend d37278872acc324f499d54dd488fbe29f76d892204793701e38129e0a2f224b0
    sha256
    prepend 333ea54d67017894b5e581b2bd4e197d3990c031bc78331b7e5c9ee89162c8fd
    sha256
    append bdef644af9388e148d198dec47e62fdaf9e264baf28f9e8115f0468354b2cc44
    sha256
    prepend 5ab7a0c9
    append 05765ac5abc32c3b
    verify PendingAttestation('https://alice.btc.calendar.opentimestamps.org')
    sha256
    append 9392bf72d9060a7b50972524836e578e7e63dcdf14c3f5d4c1dca9d86756cb9b
    sha256
    append ce3d685da3581a2bb94b71cfaea03474ebe5fe40e4dca672afe7e1da742db297
    sha256
    prepend fce1e10cc3091a3846c93bf4aa84413fb250e5ee34b53dcad606573e3d78c4ac
    sha256
    append cce5fea18e0db1f32045a3eb9dfdcd4aa705fe3b180fcebf444d67c94c294c04
    sha256
    prepend eb3be78a0442c27bdd502552a256be442e85c47db7bf4e311cc831002b3f9b38
    sha256
    append 4b60e02f7497b5803e08daba5b71d48da32a173301915690e481b67f51970751
    sha256
    prepend b9b9f9fa9bdeb8a595bfa4eca6369506d2f1546e6b21b9c7bbd7fb2548a26ffe
    sha256
    append bef44e97f18402c26657532a8f73a7789105cd59f12f8d71515912470ff1fd6d
    sha256
    prepend 1bc0e9e84a56d090d0b3c3eb4f82e2d540f0ed7a814c3c8c408d2f578dfea53d
    sha256
    append 4e315507d6bfd306166e1020bd051d71e783ea1082d22013304626f7c5a279e6
    sha256
    append e5e9e4a6fe92bb4477cea41e2c15c2c16510bef0b45b269a9f0baba752ea2c8b
    sha256
    prepend 76242fc7615358d5cf2faa3ce7c0fea8eda591c128a7ab7f881bfc9da5dd96e2
    sha256
    append 59942d9922a624ddd4b4ce58de992436565db8cf682dbe98cfb1e71bd6a6a049
    sha256
    prepend 01000000012eac8b315182773e76d57edd23b0297f60c2b5cce26e8b1958609bf53f17c998000000001716001405b45b2e5fb44a6f875d69cb4df5d414f7ba1c34fdffffff02588665000000000017a9146f626afde13c6395faad1615fe281d5c0be123c0870000000000000000226a20
    append 22dc0700
    # transaction id fd02844474819a15ae726e2f7e75d0eeb7e5de497c918ee69758130e41e30cbc
    ...
    verify BitcoinBlockHeaderAttestation(515107)
    # Bitcoin block merkle root 7d7cd153b837a18ba32b21af103af1691b1ba3d4248e7fc9c290211e15de37d5
 -> append 26ef6425656e1095c5a88badcdea280c
    sha256
    prepend 4e3e4ca04a74381180c6b4a648560e679b321b3e12d1932781b6398267151d70
    sha256
    prepend 61febb5707e8c9ae78c1248ca389d0dba9c00c156c2282020e27fcbae04d1810
    sha256
    prepend 5ab7a0c9
    append 6481affe61f15ea9
    verify PendingAttestation('https://bob.btc.calendar.opentimestamps.org')
    ...
    # transaction id df7b8ba0739571e6508f812c82de25259059bd5129a4f96816567480d3718692
    ...
    verify BitcoinBlockHeaderAttestation(515104)
    # Bitcoin block merkle root 8331bed61eb2858361b16ac337da5fdacf9b3ee41f6c8f1e204a983b9fccdb36
 -> append 3dd70ea7ab4c979a7747ab0570d2ea27
    sha256
    prepend b47ab96db80690d1c436022e2869404d7423f7225eb47c81436e342062c70eba
    sha256
    append 7c8025312aa0b452457a1a33d1e12cf0237bc34cb359140dee74f45e6ef8ead7
    sha256
    prepend 5ab7a0c9
    append c302502f95a0f008
    verify PendingAttestation('https://finney.calendar.eternitywall.com')
    ...
    # transaction id 355d3a6da112e10a589b84eb3d45682c79b1a85dbd1a3720ccf7c7bf40b02969
    ...
    verify BitcoinBlockHeaderAttestation(515107)
    # Bitcoin block merkle root 7d7cd153b837a18ba32b21af103af1691b1ba3d4248e7fc9c290211e15de37d5

Now the timestamp is completed and each incomplete PendingAttestation has become a BitcoinBlockHeaderAttestation with the corresponding block height. The bitcoin block contains the timestamp, so the date, of the stamped operation. Follow the crypto-operations path from hash to attestation, to get the merkle root of the bitcoin block.

To verify manually the ots proof: open a block-explorer and check if the block height and merkle root inside the attestation match.

Verify the ots proof on the blockchain

OpenTimestamps library provides the verify command to check the integrity of a proof and verify all the completed attestations in order to get the date of the proof.

const hashData = "16193782f1d839a08f9fc9a94cec1675f1729db1abc15cf9b57f31aa1724a0ae"
const op = new OpenTimestamps.Ops.OpSHA256()
const detached = OpenTime

stamps.DetachedTimestampFile.fromHash(op, hashData)
const detachedOts = OpenTimestamps.DetachedTimestampFile.deserialize(ots)
OpenTimestamps.verify(detachedOts, detached).then( (results)=>{
    if(Object.keys(results).length === 0){
        console.log("Pending attestation");
    }else{
        Object.keys(results).forEach(key => {
            console.log ( key+" attests data existed as of " + (new Date(results[key] * 1000)) );
        });
    }
}).catch( err => {
    console.log("Bad attestation" + err);
});

The result is the unix timestamp of the first completed attestation, with lower block-height, in the example is {bitcoin: 1521990170}. The previous code formats the unix timestamp and show the following message: Bitcoin attests data existed as of Sun Mar 25 2018 17:02:50 GMT+0200

OpenTimestamps proof supports multiple chains, a multi-chain proof is verified in both the chains and providing different timestamps.

OpenTimestamps meets Electrum

Wed, 14 Mar 2018 00:00:00 +0100

Have you ever timestamped?

Being more precise, did you ever create an OpenTimestamps proof all by yourself? To do so you need to send a bitcoin transaction incluing some extra data which are a commitment to, let’s say, your files my_first_timestamp.txt and my_second_timestamp.txt. Then for each file you should create a .ots receipt containing the proof linking the file to the merkle root in the block header, using the OpenTimestamps standard.

This is not extremely easy to do, you should be familiar with the OpenTimestamps libraries and a software to create custom bitcoin transactions. While trying you may fail, wasting some bytes in your transaction or even worse some of your bitcoins. But now there is an easy way: you can timestamp your files with your transactions using this Electrum plugin.

Note: except for educational purposes or particular cases, creating a timestamp in this way is the dumb way to do a timestamp. The smart way is using a public calendar service: it’s cheaper, requires less efforts and burdens less on the network.

Getting started

In this launch phase the plugin is not super easy to install or download, we plan to make this easier after more reviews and tests. In addition we tried to make it the less invasive possible, we used the existing hooks and we even haven’t (yet) added any fancy OpenTimestamps picture. In the next phases we will make some steps to provide a nicer user experience.

To use the plugin you need to install a custom version of Electrum including the plugin.

The instructions are adapted from Electrum README and it is assumed to run on Linux.

(For Mac OS X use brew install instead of sudo apt-get install, instead of python-pyqt5 and pyqt5-dev-tools install pyqt5 and check that it is using python3. This should do the job.)

Electrum is a pure python application. If you want to use the Qt interface, install the Qt dependencies:

$ sudo apt-get install python3-pyqt5

Install plugin requirements:

$ pip3 install opentimestamps

Clone the Electrum source code, then include the plugin files:

$ git clone https://github.com/spesmilo/electrum.git
$ git clone https://github.com/LeoComandini/electrum-timestamp-plugin.git
$ cp -r electrum-timestamp-plugin/timestamp electrum/plugins
$ cd electrum
$ python3 setup.py install

Compile the icons file for Qt:

sudo apt-get install pyqt5-dev-tools
pyrcc5 icons.qrc -o gui/qt/icons_rc.py

To run on mainnet

$ ./electrum

To run on testnet

$ ./electrum --testnet

Timestamp your files with your transaction

Enable the plugin

Click on Tools -> Plugins and tick the Timestamp checkbox

Close and restart to Electrum to abilitate the plugin.

Visualize the timestamps history

Click on Tools -> Timestamps…

Add new files

Click on the Add New File button and select the file yo want to timestamp.

Create and broadcast a bitcoin transaction including the timestamp:

On the Send Tab select outputs, amount, fee (select a fee > 1 sat/byte)

Click on Preview

Click on Timestamp, this write a commitment to your file(s) in the transaction

Confirm the changes in the transaction (if fee is too low the transaction won’t be relayed or mined)

Sign and broadcast the transaction (click on Sign, then Broadcast)

Note: the hash value in the red box is the Merkle root resulting from the hashes of the files along with nonces to guarantee extra privacy.

Check the timestamps history

Click on Tools -> Timestamps…, the file now is an pending state.

If you try to click on upgrade nothing will happen, to create your timestamps you need to wait to have the transaction is confirmed (6 blocks) so reorgs won’t be an issue.

Upgrade

After 6 blocks clicking on upgrade will do something, your timestamps are now completed.

Your timestamp proofs are next to the file you timestamped.

Verify your proofs

Use the OpenTimestamps client (better) or go on opentimestamps.org (suboptimal, you are trusting the website) to verify your proofs.

Conclusions

With this plugin you can easily timestamp your files while doing bitcoin transactions with Electrum.

While it may be sometimes useful, it is better to use a calendar.

However this could become relevant if sign-to-contract or external timestamping becomes a thing. Stay tuned.

Integrate OpenTimestamps on S3 cloud storage

Wed, 28 Feb 2018 00:00:00 +0100

OpenTimestamps is a scalable, trustless, distributed protocol to timestamp files, documents and media on Bitcoin.

There are available clients, documentations, tools and libraries in different languages, for more info look at GitHub page. Moreover, OpenTimestamps website provides free and easy methods to stamp and verify documents.

In this tutorial, I want to allow lazy people, as me, to automatically stamp documents in the cloud without any human interaction, by using Amazon Lambda functions: when I transfer a file in my Amazon S3 cloud storage, the Lambda function stamps the document and stores the proof-file on the S3 itself.

Note: Run your own Amazon Lambda function, because you should not trust to any third-party service.

Why AWS Lambda?

With Amazon AWS Lambda, you pay only for the compute time you consume, there is no charge when your code is not running. With Lambda, you can run code for virtually any type of application or backend service, all with zero administration.

A full guide of AWS Lambda is available at https://docs.aws.amazon.com/lambda/latest/dg/lambda-dg.pdf .

Get started

To create an AWS Lambda Function go to the list of Services and select Lambda: Run Code without Thinking about Server.

Create function from scratch: select name, node.js as runtime, select the existing role lambda_basic_execution and press Create function.

Connect to S3 bucket

I build and configure my S3 bucket previously with 2 main folders:

uploads: store files
proof: store OTS proof-files

In the Designer, select S3 voice to import your S3 bucket

Configure S3 bucket to enable Triggers

Press the orange button Save in the top-right of the page to save your current setting.

Write lambda function

In Designer, select OpenTimestamps lambda function to view Function code panel.

You can write your own script to call OpenTimestamps calendar servers or use OpenTimestamps nodejs library available on npm. Otherwise, you can find my own script at https://github.com/lvaccaro/javascript-opentimestamp-awslambda. Download and build a .zip package to upload on your AWS lambda platform. Select Upload a .ZIP file` on function code panel.

Check the execution role and select lambda_basic_execution and increase the Timeout. Keep the other values as default.

Set access and roles

Go to Services IAM: Manage User Access and Encryption Keys. In Roles menu, click on role lambda_basic_execution and attach the following policies: AmazonS3FullAccess, AWSLambdaRole. Press Save to upload and save your current settings.

Publish

Click Actions and Publish: set a version name before publish a new version.

Run

Go in your S3 bucket. Upload your files on uploads folder and automatically our AWS lambda function build the OTS receipt on proof folder.

Go to upload folder to upload a file

And in the proof folder, it is automatically generated and stored the OTS file.

Create the shortest transaction

Tue, 12 Dec 2017 00:00:00 +0100

Disclaimer: what follows is a mere exercise in style.

Along with higher fees and block size debate a natural question arise: which is the shortest transaction?

There are some extremely short transactions (like this), but we are looking for something that could be sent by everyone and relayed by the network, so ~~nonstandard~~, ~~coinbase~~.

A sent short transaction

Extract the transaction with txid = 3419cbc51cec42d4f55d4147d0c1cef54fbdabddf4384270e6e93970b2496f74 from the blockchain and decode the raw transaction:

$ bitcoin-cli decoderawtransaction 0200000001017a28f3d62d98803174bd1ba3827c62e4ccb28b4aa23aa5c7bec74301005f1b0000000044433040022002aedaad7897022be857509bbf753d5c05d6c20092cf5088413f31d6ff661f0b021c57e00a637102d8223360b6797d0bb33c2dfc4504c30d0c4ced29ddb301ffffffff010000000000000000016a00000000
{
  "txid": "3419cbc51cec42d4f55d4147d0c1cef54fbdabddf4384270e6e93970b2496f74",
  "hash": "3419cbc51cec42d4f55d4147d0c1cef54fbdabddf4384270e6e93970b2496f74",
  "version": 2,
  "size": 129,
  "vsize": 129,
  "locktime": 0,
  "vin": [
    {
      "txid": "1b5f000143c7bec7a53aa24a8bb2cce4627c82a31bbd743180982dd6f3287a01",
      "vout": 0,
      "scriptSig": {
        "asm": "3040022002aedaad7897022be857509bbf753d5c05d6c20092cf5088413f31d6ff661f0b021c57e00a637102d8223360b6797d0bb33c2dfc4504c30d0c4ced29ddb3[ALL]",
        "hex": "433040022002aedaad7897022be857509bbf753d5c05d6c20092cf5088413f31d6ff661f0b021c57e00a637102d8223360b6797d0bb33c2dfc4504c30d0c4ced29ddb301"
      },
      "sequence": 4294967295
    }
  ],
  "vout": [
    {
      "value": 0.00000000,
      "n": 0,
      "scriptPubKey": {
        "asm": "OP_RETURN",
        "hex": "6a",
        "type": "nulldata"
      }
    }
  ]
}

It is only 129 bytes long:

4 bytes for version
1 byte for number of inputs
32 bytes for txid
4 bytes for vout
1 byte for length of unlocking script
1 byte for length of signature
67 bytes for DER encoded signature (including sighash)
4 bytes for sequence number
1 byte for number of outputs
8 bytes for amount
1 byte for length of locking script
1 bytes for locking script
4 bytes for nlocktime

How to create such a short transaction?

Work on its body minimizing inputs and outputs.

Squeeze the signature taking advantage of DER encoding.

Minimize inputs and outputs

A standard transaction must have at least one input and one output, so one input and one output.

The input is created ad hoc for this transaction. It must be spendable (~~nulldata~~ has no bitcoin locked in). It must have the least possible redundant data in the ScriptSig (~~P2PKH~~ has to specify the public key, ~~multisig~~ has to specify n-of-m). It must be signed (~~P2SH~~ without a signature can be stolen, with a signature has redundant data). Thus the input used is a P2PK.

The shortest possible standard output is a nulldata with nothing following the OP_RETURN.

Squeeze the signature

DER encoded signatures haven’t fixed length. Usually they are 73, 72 or 71 bytes long, however sometimes they could be shorter.

Reminding how a signature is encoded:

a ECDSA signature is a couple of integer in [1..order-1]: (r, s)
as specified in BIP66, a DER encoded signature is composed by:
- 0x30 DER encoded signature follows
- 1 byte for length of signature
- 0x02 int follows
- 1 byte for length of r
- r in bytes (33 or less bytes)
- 0x02 int follows
- 1 byte for length of s
- s in bytes (33 or less bytes)
- 1 byte for sighash
when encoding r and s in bytes if the most signficant byte is 0x80 or more then prepend 0x00

The idea is to try different nonces in order to find a shorter DER encoded signature. Generating multiple nonces doesn’t have to be expensive, one can generate the first in the a standard way and then derive the next ones deterministically. You can see the code here.

How much is likely to get a short signature?

In the following we will use an approximation: order = 2**256. It will make computations easier: with this apporximation P(r < 2**255) = 1/2 while the true value is slightly above (since r cannot assume values in [order..2**256-1]). However this values are extremely close to the true results (computed here).

Compute the cumulative distribution:

P(len(encoded int) <= 33 bytes) = 1
P(len(encoded int) <= 32 bytes) = 1/2
P(len(encoded int) <= 31 bytes) = 1/256 * 1/2
P(len(encoded int) <= 30 bytes) = 1/256**2 * 1/2
...
P(len(encoded int) <= (33-n) bytes) = | 1                    if n = 0
                                      | 1/2 * 1/256**(n-1)   if n = 1, 2, ..., 32

Compute the discrete distribution per difference:

P(len(encoded int) = 33 bytes) = P(33) = 1 - 1/2 = 1/2
P(len(encoded int) = 32 bytes) = P(32) = 1/2 - 1/256 * 1/2 = 1/2 * 255/256
P(len(encoded int) = 31 bytes) = P(31) = 1/256 * 1/2 - 1/256**2 * 1/2 = 1/2 * 1/256 * 255/256
P(len(encoded int) = 30 bytes) = P(30) = 1/256**2 * 1/2 - 1/256**3 * 1/2 = 1/2 * 1/256**2 * 255/256
...
P(33-n) = | 1/2                if n = 0
          | 255/2 * 1/256**n   if n = 1, 2, ..., 32

Remembering that a DER signature is composed by r and s plus 7 bytes (sighash is included), compute the probability of obtaining a DER signature with a certain length:

let a = 255/2, b = 1/256, recall P(33) = 1/2

P(len(encoded sig) = 73 bytes) = P(33) * P(33) = 1/4
P(len(encoded sig) = 72 bytes) = 2 * P(33) * P(32) = ab
P(len(encoded sig) = 71 bytes) = 2 * P(33) * P(31) + P(32) * P(32) = ab**2 + a**2*b**2 = b**2*(a + a**2)
P(len(encoded sig) = 70 bytes) = 2 * P(33) * P(30) + 2 * P(32) * P(31) =  ab**3 + 2*a**2*b**3 = b**3*(a + 2*a**2)
P(len(encoded sig) = 69 bytes) = 2 * P(33) * P(29) + 2 * P(32) * P(30) + P(31) * P(31) = ab**4 + 2*a**2*b**4 + a**2*b**4 = b**4*(a + 3*a**2)
P(len(encoded sig) = 68 bytes) = 2 * P(33) * P(28) + 2 * P(32) * P(29) + 2 * P(31) * P(30) = ab**5 + 2*a**2*b**5 + 2*a**2*b**5 = b**5*(a + 4*a**2)
...
P(len(encoded sig) = 73-n bytes) = | 1/4                   if n = 0
                                   | b**n*(a+(n-1)*a**2)   if n = 1,  ..., 32
                                   | b**n*(65-n)*a**2      if n = 33, ..., 65

Resuming:

Length of DER encoded signature	Probability	Avg number of tries to generate such length
73 bytes	0.25	4
72 bytes	0.498046875	2
71 bytes	0.24999618530273438	4
70 bytes	0.00194549560546875	514
69 bytes	0.00001138454535976	87838
68 bytes	0.00000005925585355	16875969
67 bytes	0.00000000028922197	3457551881
…	…	…

So if you have exceeding computational power you can sign several times, get a shorter signature and save some bytes.

Is this worthy?

Not really.

There are some issues:

Privacy: if few people squeeze their signatures then the squeezed ones become easily recognizable.
Security: reducing the first part of the signature (r) of n bytes is equivalent to choose the nonce from a set (approximately) 1/256**n times smaller than the stardard one.
SegWit: the signature is in the witness which is discounted.

Still the transaction is smaller and weights less on the network.

Conclusions

A very small UTXO could be insert in a transaction with 1 input and 1 nulldata minimal output. This could be used to timestamp using a sign-to-contract scheme.

For every signature one may sign several times to save some bytes. This will make the transaction smaller and hence cheaper, but in the end it is not worth the effort.

We haven’t found any signature smaller than 67 bytes, if you find one please comment down here!

Remark: Most signature algorithms use a deterministic nonce (with RFC6979) hence to generate new signatures you should modify how the signature is made. But this is very delicate and could lead to loss of funds, don’t expose others to this risk.

OpenTimestamps performances

Thu, 22 Jun 2017 00:00:00 +0200

Executive Summary

Is the Bitcoin Blockchain a good choiche for timestamping purposes? Some articles conclude that bitcoin can not be used for such applications due to the writing time needed for include a transaction in a valid block.

In this article I will analize the OpenTimestamps protocol perfomances (using public available servers) and I will show how timestamping can work in a efficient and cheap way on the Bitcoin Blockchain.

Introduction

Checking some articles online related with Bitcoin blockchain I found the Blockchain Inefficiency in the Bitcoin Peers Network from Giuseppe Pappalardo, T. Di Matteo, Guido Caldarelli, Tomaso Aste (you can find it on arxiv) that sound very pessimistic about the ability of the blockchain to support timestamping solutions.

The article start with the following abstract

We investigate Bitcoin network monitoring the dynamics of blocks and transactions. We unveil that 43% of the transactions are still not included in the Blockchain after 1h from the first time they were seen in the network and 20% of the transactions are still not included in the Blockchain after 30 days, revealing therefore great inefficiency in the Bitcoin system. However, we observe that most of these “forgotten” transactions have low values and in terms of transferred value the system is less inefficient with 93% of the transactions value being included into the Blockchain within 3h.

And continue with some conclusion about the use of the Blockchain for timestamping purposes

The fact that a sizeable fraction of transactions is not processed timely casts serious doubts on the usability of the Bitcoin Blockchain for reliable time-stamping purposes and calls for a debate about the right systems of incentives which a peer-to-peer unintermediated system should introduce to promote efficient transaction recording.

In my opinion this pessimistic conclusion on what you can do and what you can not do are totally wrong and the article miss an in-depth analysis of what cause that kind of delay.

Then I tried to recreate the same results by contacting the author to obtain the script used or the complete dataset, unfortunately sources and dataset are not available for any elaboration.

In the article the approach was bottom-up, from the raw data of the blockchain authors try to elaborate high level messages, in this post I will use a top-down analysis, I will analyze directly the data of some OpenTimestamps servers in order to check real performances of the system.

OpenTimestamps calendar is designed to spend approximately a certain amount of money per day; this amount can be chosen adjusting the granularity required from the calendar server and it’s unrelated to the amount of file timestamped in the interval.

For this analysis I will use the data available from public calendars server.

Get the data

First step was to generate system’s performance data, the idea is the following:

every 6 hours a new timestamp is generated using a script, informations about the digest and the creation date (timeCreated) is saved on a database,
every 5 minutes for each digest not confirmed with a proof on blockchain the script request an upgrade to the relative caledar server, if the digest appear in a confirmed transaction the actual time is recorded in the relative field (timeCompleted)

All this informations are exported in a csv file weekly. I decide to elaborate this data using the R language, the following code download the data from the website and convert in the internal data frame rappresentation.

library(readr)
library(ggplot2)
library(rvest)
library(plyr)
library(dplyr)
library(knitr)
library(kableExtra)
setwd("~/r-studio-workspace/MinMon")
today <- Sys.Date()

dataLog_plot <-  read_csv("http://wpc.uk.to/miner_img/OTS/2017-06-22-dataset.csv")
dataLog_plot$timeCompleted <- as.POSIXct(dataLog_plot$timeCompleted)
dataLog_plot$timeCreated <- as.POSIXct(dataLog_plot$timeCreated)
dataLog_plot$siteUrl <- as.factor(dataLog_plot$siteUrl)
dataLog_plot$blockchain <- as.factor(dataLog_plot$blockchain)

We can print the first rows of the dataset with the following code.

options(knitr.table.format = "html")
kable(head(dataLog_plot), caption="Raw dataset") %>%
   kable_styling(bootstrap_options = "striped", font_size = 7)

Raw dataset
X1	digest	datastoreId	callNumber	timeCompleted	timeCreated	siteUrl	delta	blockchain
1	94802b7f99680476e82915cb93336290b8652a5213b06b47168a6622d05be6e262c93f59576137f8e94757dbd9a987e4	5.668601e+15	6	2017-03-30 09:35:33	2017-03-29 17:25:34	https://alice.btc.calendar.opentimestamps.org	969.9709	BTC
2	94802b7f99680476e82915cb93336290b8652a5213b06b47168a6622d05be6e262c93f59576137f8e94757dbd9a987e4	5.707702e+15	6	2017-03-30 09:35:33	2017-03-29 17:25:34	https://finney.calendar.eternitywall.com	969.9815	BTC
3	4666d7a59b1791130cbd53b3e64bfe7e07e7d2c9aecfa117868fa24e1d3c8f00	5.629500e+15	57	2017-03-31 20:55:00	2017-03-31 16:15:36	https://finney.calendar.eternitywall.com	279.3964	BTC
4	70f9ff70692148b0bcb103510a4793cbb41ac8a7b71496a7087974d3704979c6	5.629500e+15	68	2017-03-31 20:55:01	2017-03-31 15:07:21	https://finney.calendar.eternitywall.com	347.6549	BTC
5	c8f90141c605bfa7ab5ac7da2195320c6b98c1ea8a4595d048c3dedf510ce4bf	5.707702e+15	68	2017-03-31 20:55:01	2017-03-31 15:07:17	https://finney.calendar.eternitywall.com	347.7321	BTC
6	480aaf138ff25cea952f24265899abf635bdded3429f986a09c41743c5c9abd1	5.629500e+15	53	2017-04-01 02:25:01	2017-03-31 22:01:01	https://alice.btc.calendar.opentimestamps.org	264.0046	BTC

The dataset contain the following columns:

X1 a progressive row identificator,
digest the value which is going to be timestamped, normally the hash of a document, in this monitoring case 32 random byte,
datastoreId the row id in the datastore,
callNumber number of times the monitoring process asked the calendar if the timestamp is upgraded, approximately (timeCreated - timeCompleted) ~= callNumber * 5 minutes,
timeCompleted time when the process monitor found a complete timestamp at the calendar server,
timeCreated timestamp of the creation,
siteUrl url of the public calendar server,
delta difference between timeCreated and timeCompleted in minutes,
blockchain kind of blockchain used.

You can also download the raw data from each server calling a special page, for example for the finney public server you can call https://finney.calendar.eternitywall.com/calendar/ and access to all the content of the local database.

Each commitment in the journal file has a seconds granularity UTC unix timestamp attached which records exactly when the commitment was generated, this timestamp is added only for debugging and monitoring purposes and it’s not part of OpenTimestamps protocol.

Using this datasets a reader can autonomously check the performances of the service for a specific server.

Note that even we can fake the time when a timestamp is requested, the time when a timestamp is completed is proved by a transaction written in the blockchain which cannot be tampered.

Analyzing the data

The first idea is to plot the delta (difference between timeCreated and timeCompleted in minutes) for each transaction.

ggplot(dataLog_plot[dataLog_plot$blockchain=="BTC", ], aes(x=timeCreated, y=delta)) +
   geom_line(aes(color=siteUrl), alpha=0.3) +
   geom_point(aes(color=siteUrl)) +
   geom_hline(yintercept=720, linetype="dashed") +
   geom_text(aes(x=min(dataLog_plot[dataLog_plot$blockchain=="BTC", ]$timeCreated), y=720, label="12h", vjust=-1)) +
   geom_hline(yintercept=1440, linetype="dashed") +
   geom_text(aes(x=min(dataLog_plot[dataLog_plot$blockchain=="BTC", ]$timeCreated), y=1440, label="24h", vjust=-1)) +
   geom_hline(yintercept=mean(dataLog_plot[dataLog_plot$siteUrl=="https://alice.btc.calendar.opentimestamps.org", ]$delta), color="red", linetype="dashed") +
   geom_text(aes(x=min(dataLog_plot[dataLog_plot$blockchain=="BTC", ]$timeCreated),
                 y=as.numeric(mean(dataLog_plot[dataLog_plot$siteUrl=="https://alice.btc.calendar.opentimestamps.org", ]$delta)),
                 label=paste(round(as.numeric(mean(dataLog_plot[dataLog_plot$siteUrl=="https://alice.btc.calendar.opentimestamps.org", ]$delta))/60, digits = 2),"h"), vjust=-1, color="https://alice.btc.calendar.opentimestamps.org")) +
   geom_hline(yintercept=mean(dataLog_plot[dataLog_plot$siteUrl=="https://bob.btc.calendar.opentimestamps.org", ]$delta), color="green", linetype="dashed") +
   geom_text(aes(x=min(dataLog_plot[dataLog_plot$blockchain=="BTC", ]$timeCreated),
                 y=as.numeric(mean(dataLog_plot[dataLog_plot$siteUrl=="https://bob.btc.calendar.opentimestamps.org", ]$delta)),
                 label=paste(round(as.numeric(mean(dataLog_plot[dataLog_plot$siteUrl=="https://bob.btc.calendar.opentimestamps.org", ]$delta))/60, digits = 2),"h"), vjust=-1, color="https://bob.btc.calendar.opentimestamps.org")) +
   geom_hline(yintercept=mean(dataLog_plot[dataLog_plot$siteUrl=="https://finney.calendar.eternitywall.com", ]$delta), color="blue", linetype="dashed") +
   geom_text(aes(x=min(dataLog_plot[dataLog_plot$blockchain=="BTC", ]$timeCreated),
                 y=as.numeric(mean(dataLog_plot[dataLog_plot$siteUrl=="https://finney.calendar.eternitywall.com", ]$delta)),
                 label=paste(round(as.numeric(mean(dataLog_plot[dataLog_plot$siteUrl=="https://finney.calendar.eternitywall.com", ]$delta))/60, digits = 2),"h"), vjust=-1, color="https://finney.calendar.eternitywall.com")) +
   ggtitle("Delay in the OTS servers") +
   xlab("Date") +
   ylab("Delay in minutes") +
   guides(color = guide_legend(title="OpenTimestamps server"))

In this chart unfortunately we can see how some public server went down for maintenance/debugging activities generating long delays. Thanks to the architecture of OpenTimestamps this is not a big deal! Every timestamp request from the reference client is made to the 3 servers, what you really care of is the reply of the fastest one to write in the blockchain

For this reason we can plot just the minimum time for each timestamp, this is a good approximation of the time needed to publish a message. This is how OpenTimestamps protocol work, imagine to ask two servers (Alice and Bot) the same timestamp, the protocol return the timestamp of the minimal height block so, if Alice publish transaction before Bob, the protocol will return the timestamp on the block containing Alice’s transaction.

We can plot it using a dataset summarized by digest like in the following chunk.

my_dataLog_plot <- dataLog_plot %>%
   group_by(digest, blockchain) %>%
   summarize(timeCreated = min(timeCreated, na.rm = TRUE),
             siteUrl = siteUrl[which.min(timeCreated)],
             delta = delta[which.min(timeCreated)]
             )  

ggplot(my_dataLog_plot[my_dataLog_plot$blockchain=="BTC", ], aes(x=timeCreated, y=delta)) +
   geom_line(alpha=0.3) +
   geom_point(aes(color=siteUrl)) +
   geom_hline(yintercept=720, linetype="dashed") +
   geom_text(aes(x=min(my_dataLog_plot[my_dataLog_plot$blockchain=="BTC", ]$timeCreated), y=720, label="12h", vjust=-1)) +
   geom_hline(yintercept=1440, linetype="dashed") +
   geom_text(aes(x=min(my_dataLog_plot[my_dataLog_plot$blockchain=="BTC", ]$timeCreated), y=1440, label="24h", vjust=-1)) +
   ggtitle("Minimum delay in the OTS servers") +
   xlab("Date") +
   ylab("Delay in minutes") +
   guides(color = guide_legend(title="OpenTimestamps server"))

Remember that every server can decide:

how many transaction do in a day,
the amount of the fee for publish a message (adjustable using Replace By Fee approach better explained later).

In this chart we can also add the average confirmation time that represent the load of the blockchain.

load <- read.csv("https://api.blockchain.info/charts/avg-confirmation-time?daysAverageString=1&format=csv&scale=0&timespan=30days", header = FALSE)

ggplot(my_dataLog_plot[my_dataLog_plot$blockchain=="BTC", ], aes(x=timeCreated, y=delta)) +
   geom_line(data=load, aes(x=as.POSIXct(load$V1), y=as.numeric(load$V2)*5), alpha=0.8, linetype="dotted") +
   geom_line(alpha=0.3) +
   geom_point(aes(color=siteUrl)) +
   geom_hline(yintercept=720, linetype="dashed") +
   geom_text(aes(x=min(my_dataLog_plot[my_dataLog_plot$blockchain=="BTC", ]$timeCreated), y=720, label="12h", vjust=-1)) +
   geom_hline(yintercept=1440, linetype="dashed") +
   geom_text(aes(x=min(my_dataLog_plot[my_dataLog_plot$blockchain=="BTC", ]$timeCreated), y=1440, label="24h", vjust=-1)) +
   ggtitle("Minimum delay in the OTS servers") +
   xlab("Date") +
   ylab("Delay in minutes") +
   guides(color = guide_legend(title="OpenTimestamps server"))

As we can see the average load has some effects on service speed but a good choice of fees can reduce the probability of an high delay.

Conclusions

We show how 3 OpenTimestamps free public servers perform in the real life and allow the publish of the transaction in less than 24 hours. With higher budget you can obviously obtain better performance, but even with infinite budget the maximum precision with bitcoin is +/- 2 hours.

If we represent the performance of the system using an histogram we can see how a large amount of timestamps are written in the chain in less than 12 hours and almost all timestamps are written and confirmed in the chain in less than 24 hours.

my_dataLog_plot$bar <- my_dataLog_plot$delta %/% 60
ggplot(my_dataLog_plot[my_dataLog_plot$blockchain=="BTC", ], aes(x=bar, fill=siteUrl)) +
   geom_histogram(bins = max(my_dataLog_plot$bar)+1) +
   ggtitle("") +
   xlab("Delay in hours") +
   ylab("Transactions") +
   guides(fill = guide_legend(title="OpenTimestamps server"))

Or if you prefer in band of 6 hours…

my_dataLog_plot$bar_2 <- my_dataLog_plot$bar %/% 6
my_dataLog_plot$bar_2 <- factor(my_dataLog_plot$bar_2)
levels(my_dataLog_plot$bar_2) <- c("<6","6-12","12-18",">18",">18")
ggplot(my_dataLog_plot[my_dataLog_plot$blockchain=="BTC", ], aes(x=bar_2, fill=siteUrl)) +
   geom_histogram(bins = max(as.numeric(my_dataLog_plot$bar_2))+1, stat="count") +
   ggtitle("") +
   xlab("Delay in hours") +
   ylab("Transactions") +
   guides(fill = guide_legend(title="OpenTimestamps server"))

Not bad for public servers that work 24/365 with low fees, a lot of timestamp are processed in less than 12 hours and many in less than 6 (hey hey remember that a public server make a transaction every 6 hours …). Every transaction is processed in a day!

What makes possible this performances?

OpenTimestamps calendar may take long to create proofs but they will certainly do with a predetermined budget. The algorithm to create bitcoin transactions leverage the Replace By Fee feature.

By default, every node enforce the first seen rule on tx, but if RBF flag is set a transaction in mempool could be replaced by one spending the same input if it has higher fee.

When a calendar server needs to create a timestamping tx A, it starts with a minimum fee amount F. When the calendar receive a newly mined block, it checks if the tx A it’s in the block, if not, it creates another tx B using the same input as A but with a fee equal to transaction_fee_of_A + F. When another block is mined the algo repeats, until the transaction will be eventually confirmed.

With this algorithm, independently from the network condition, the Calendar server running budget is the number of blocks in a day (144) multiplied by F. What change depending on network conditions is the frequency of timestamping.

As we saw timestamping is possible using transaction on blockchain using the OpenTimestamps protocol, but remember:

delay depends on the writing policy chosen (e.g. if you decide to write 4 times per day accept 12 hours granularity with acceptable fees) and
delay depends on the fee chosen, if very low you accept to have a less precise timestamp.

A special thanks to Peter Todd and Riccardo Casatta for feedbacks and everyone who works on OpenTimestamps.

EW position on the hard-fork

Thu, 30 Mar 2017 00:00:00 +0200

Eternity Wall works on the assumption of an immutable and decentralized blockchain. It’s by leveraging this properties that we can build useful new products for our users. Blockchains losing this kind of properties are no different from existing technologies and we are not interested in this field.

We didn’t even think to need to write this kind of statement but recently it looks a part of the network think can decide for all the partecipants and this is not the idea we have of Bitcoin. Eternity Wall will not consider the current Bitcoin Unlimited proposal to be the original Bitcoin cryptocurrency and even if there will be a BTU chain, we are not interested in building on top of it.

We consider hard-fork only remotely possible with super-majority of every actor of the network.

We support SegWit development and we encourage miners to signal adoption of this technology.

We recommend interesting readings from other companies or individuals sharing similar views:

Timestamping is the first non-monetary blockchain use case

Fri, 16 Sep 2016 00:00:00 +0200

ESMA (European Securities and Markets Authority) recently asked a Consultation on the distributed ledger technology applied to securities markets.

Ferdinando Ametrano ¹, Emilio Barucci, Daniele Marazzina and Stefano Zanero submitted an answer, suggesting that timestamping and notarization services might be the first non-monetary use case for blockchain technology. The relevant section of their response is the following:

Q2: Do you see any other potential benefits of the DLT for securities markets? If yes, please explain.

Notarization services are a very promising blockchain application [11]: the bitcoin blockchain (the most secure one, since the effort/cost for its manipulation is prohibitive) can be used for the trustless time-stamping of documents and the anchoring of arbitrarily large data set. A generic data file can be hashed to producing a short unique identifier, equivalent to its digital fingerprint. Such a fingerprint can be associated to a bitcoin transaction, the bitcoin amount being irrelevant, and hence registered on the blockchain: the blockchain immutability then provides robust non-repudiable time-stamping that can always prove without doubt the existence of that data file in that specific status at that precise moment in time. This generic process is even undergoing some standardization to achieve third party auditable verification [12]. Broker-dealers could use it to satisfy the regulatory prescriptions [13] for storing required records exclusively in non-rewriteable and non-erasable electronic storage media. WORM (write once read many) optical media has been used so far, but it is quite impractical, especially for large data set; instead, compliance could be easily achieved anchoring rewritable data sources to the blockchain, providing accurate and secure time-stamping resilient to manipulation.

…

[11] https://eternitywall.it/notarize/, https://stampery.com/, https://tierion.com/
[12] http://blog.eternitywall.it/2016/06/24/announcing-opentimestamps-support/, https://github.com/opentimestamps/python-opentimestamps
[13] Rule 17a-4 of the Securities Exchange Act (Broker Dealers). See also http://www.17a-4.com/regulations-summary/

Eternity Wall strongly agree with the content of the answer. We are happy to see our efforts to promote standardization has been noted, and acknowledged also with a direct reference to our article.

We also endorse this tweet by Ametrano.

“What jewelry is for gold, timestamping could be for bitcoin: not essential, but effective at leveraging its beauty”

After the response, another blog post about the new version of OpenTimestamps has been published by Peter Todd.

We are already working to provide WORM solutions in response to regulatory prescriptions in the Dodd Frank normative using the blockchain. Updates will be soon on this blog.

The ESMA response is a confirmation of our vision, and our commitment on the subject is stronger than ever.

F. Ametrano will be teaching “Bitcoin and Blockchain Technology” this semester at Politecnico, and Politecnico will host Scaling Bitcoin Milan in October ↩

Announcing OpenTimestamps support

Fri, 24 Jun 2016 00:00:00 +0200

OpenTimestamps

16 February update
A new version of OpenTimestamps has been developed, this post could be outdated.
Consider visiting the OpenTimestamps web site

The path towards third party auditable verification

Eternity Wall is making efforts towards allowing a fully independent timestamps verification.

We already provided our users instructions and open-source tools to verify our notarizations. We would like to improve the auditability of our service by having an externally developed independent stamp verifier.

OpenTimestamps discovery

Peter Todd showed us his OpenTimestamps format during his recent visit in Milan. We immediately realized this was what we needed to create a trustworthy timestamping ecosystem. One of the best part of the format is flexibility, without changing the way Eternity Wall is stamping documents we added in ours stamps the OpenTimestamps format, under the ots1 property as in this example.

Basic Idea

The basic idea of the format is that if you can create a path of generic operations from the document to the hash of a block you are proving that the document is committing to that block.

All the operations you need are hashing functions (like sha256 and ripemd), generic functions (reverse) and concatenations of binary data as a prefix or a suffix. We are using the bitcoin blockchain to prove the timestamping but the format is flexible enough to support also other blockchains.

How to verify

The following are the steps you need to verify an Eternity Wall notarized document using Peter’s python tool. The tool needs a bitcoin node with a reachable RPC endpoint (basing on the settings in your bitcoin.conf).

We are using the picture of this example as the document to prove. Get the stamp data and copy in the clipboard the content of the ots1 child in the json, you will need this data later.

git clone https://github.com/petertodd/python-opentimestamps
cd python-opentimestamps
wget https://blog.eternitywall.it/img/riccardo-casatta-profilo.jpg
nano riccardo-casatta-profilo.jpg.ots # paste the clipboard content in this file
./ots verify riccardo-casatta-profilo.jpg.ots

INFO:root:Success! riccardo-casatta-profilo.jpg was created on or before 2016-01-31 09:46:31

Getting this messages means the document has been timestamped in the bitcoin blockchain on or before 2016-01-31 09:46:31

Improvements

The actual verification is still cumbersome but we are hoping that a wider ecosystem support of the OpenTimestamps format will grant an easier verification process in the future.

We are already working in this direction to promote broader adoption of the standard. Stay tuned!

Happy first birthday Eternity Wall

Tue, 21 Jun 2016 00:00:00 +0200

We would like to eat the cake with everyone who posted on the wall and talked about it with friends!

This birthday has a special proof of authenticity, you can see this proof from the very first message on the wall.

Thanks everybody!