What's New in Eth2 - 29 March 2019

Edition 20. Archive. Trouble viewing? Load original.

My avatarBen Edgington (PegaSys, ConsenSys — but views expressed are all my own)


The Taking Stock Edition

It's nine months since the old Casper and Sharding designs were discarded and Ethereum 2.0 was conceived as a ground-up development, alongside the current Ethereum Mainnet. Let's take stock a little.

Nine months ago, the specification was a blank page on HackMD. Now it is a fully fledged GitHub repo with five versioned pre-releases. 1860 commits by 54 contributors have seen about 23,000 lines added and 19,000 lines deleted. The specification now spans 12 separate documents covering different aspects of the protocol, not including the validator management contract which sits in a different repo. And it continues to advance rapidly.

The design contains some brilliant innovations and applications of existing technologies, like the swap-or-not shuffle, the LMD GHOST fork choice rule, BLS signature aggregation, the hash tree root, the quadratic leak, the proof of custody game, and more.

In short, it is phenomenal. It is thrilling. I would never have imagined just nine months ago, in that basement in Berlin, that in such a brief time so much would be accomplished. And I haven't even mentioned the client implementations yet.

Three phases are planned before Ethereum 2.0 can be considered delivered and functional. How are they coming along?

Phase 0 is the beacon chain, the proof-of-stake coordination layer for the whole system. The phase 0 specification is converging on finished; the pace of change has slowed rapidly. The spec is actually executable and testable. Pretty much all that remains is choosing a serialisation algorithm (sorry! "In" joke 😂).

Beacon chain client implementations have been created by multiple teams in a range of languages, and test nets (see below) are being assembled. We have a long road ahead of debugging, networking implementation and compatibility testing, but the way forward is clear. Some believe that we could have a mainnet deployment with real Ether staked by the end of 2019. Certainly early 2020. This is perhaps the most challenging implementation phase.

Phase 1 is the sharded data layer: dividing transactions among over 1000 independent blockchains all anchored to the proof-of-stake beacon chain. The spec for phase 1 is coming along well, and will likely be finalised soon after that of Phase 0. That is, in the next couple of months. Implementation is primarily a peer-to-peer network engineering challenge. This work is underway.

Phase 2 is the sharded state and execution layer. Phase 2 has no specification document yet, and there remains a lot of R&D to do. The question is not so much how will we do these things, more which of the available solutions is the optimal choice, considering all stakeholders in the future system. People are beginning to tackle the hard problems: see below for more.

So that's where we are. It's been an astonishing nine months. I'd say we're perhaps a quarter to a third of the way there. At this rate, it's going to be delivered before Brexit...

Specification updates

There's an experimental PR to see what it would look like to generate RST/Sphinx, the ReadTheDocs format, from the all the spec documents.

Beacon chain specification (Phase 0)

There have been no major spec releases since our last edition two weeks ago, but there was one bugfix minor release, so we're now on v0.5.1 of the spec.

Looking ahead, Justin confirmed that we're moving to SHA256 as the standard hash function (previously it was Keccak256, and before that Blake2b, via a brief flirtation with STARK-friendliness—MiMC was in the frame for a while). The main driver for the choice of SHA256 is cross-chain standardisation and compatibility.

In a surprise move, Danny mentioned on the call that this silliness around not starting slot numbers at zero may be over! He didn't mention the real reason—that he reads and heeds What's New in Eth2 😀.

Other changes in the pipeline include splitting out and making executable the fork choice rule. Doing this allows test vectors to be generated directly from the spec, which, once again, is super-cool. Validator exiting has also been reworked and simplified.

Version 0.6 is expected within a couple of weeks, and will contain radically simplifying changes.

Shard chains specification (Phase 1)

The main update for phase 1 has been to replace the proof of custody game with JABS (JABS is Justin's Awesome Bit Sum). 500 lines have been removed, so, whatever it's called, it's definitely a major simplification. On the implementers' call, Justin mentioned that the phase 1 spec will be split into two, with the second part containing updates to the beacon chain to support the proof-of-custody game.

It should be possible to finalise the phase 1 spec not very long after the phase 0 spec is finalised.

State execution (Phase 2)

I was very heartened this week to come across Brooklyn Zelenka's work on the Actor Model (actually, Joe Lubin raised it up my backlog reading list for me). Brooke and Boris Mann (who seems to be everywhere these days) also convened a discussion group about what a cross-shard communication model might look like in Ethereum 2.0. One of the things I love so much about this space is that smart, knowledgeable people keep popping up at the right time to wrestle with the hard problems. This conversation deserves to gain momentum. Cross shard messaging is one of the critical components still in research.

BLS signatures

Just a clarification on the result of aggregating empty lists of signatures or keys: it's the point at infinity (the identity element) of the relevant group.

Simple Serialize

Piper Merriam has proposed an update to SSZ, based on Péter Szilágyi's SOS serialisation, that allows efficient indexing into serialised data.

Light client specification

Minor updates.

Networking

This is a new section and already contains the beginnings of a substantial document on the RPC interface by Matthew Slipper. Matt also convened a call to discuss wire protocol things. Somebody please let me know if there is a recording of the call somewhere, or notes.

Implementation stuff

Testing

Hot news from the implementers' call is that @protolambda (Diederik Loerakker) has been brought in to work on the testing and infrastructure for Ethereum 2.0 🎉. Among other things, he will be working on fuzz testing and randomised inputs.

Now that the spec is largely executable, a lot of apparatus for generating and running tests on the spec is being imported into the repo, even to the extent of having an automated CircleCI set up. There is a lengthy discussion about this for those who enjoy devopsy kinds of things.

In addition, the formal test generation repo is being moved over to the specs repo to keep them better aligned. This will help implementers a lot with keeping the test suite in-sync with the spec releases.

A few of the implementation teams have announced test nets recently: Prysmatic's ambitious long-lived open testnet on Görli is close; Lighthouse is planning a short-lived testnet within days, Parity's testnet v3 is available. Stop press: Nimbus has joined the party with its own public testnet.

As for cross-client testnets, Danny has some advice. A suggested approach is, (1) each team runs its own long-standing single client testnet, (2) clients pass all the consensus test vectors, (3) only then work 1-1 with another client team to target compatibility on a common release of the spec. Only once we're past this point does it make sense to build a long-standing, multi-client testnet. At that point we could consider something like Cosmos's Game of Stakes to gameify testnet participation and robustness testing.

Fortnightly call

There was a call on March 28th with this agenda, and video here. As usual, there were over 50 attendees for much of it. Today's Ethereum 1 All Core Devs call had 25. Just sayin'. Efficient chairing by Danny Ryan helped us to get done in about an hour for once.

Among the things discussed on the call was documenting the invariants of the specification so that implementers can be confident about making optimisations. They will be compiled in this document for now.

Most of the other news from the call can be found in the other sections of this update.

Research

A lot of work has been done on the so-called proof of custody game. This is at the heart of phase 1, the sharded data layer, and is a mechanism for holding validators accountable for having seen the shard block data that they are attesting to. @dankrad has a nice article on using a PRF (pseudo-random function) to implement this, in the form of the Legendre symbol, rather than the currently proposed hash function. This would be allow for the proofs-of-custody to be efficiently calculated in multi-party compute setting, such as resilient shared-staking implementations. The RANDAO reveal was previously changed to a BLS signature for this reason.

As a bonus, @dankrad notes that there are places in the spec (such as validator shuffling, a known performance bottleneck) which don't strictly require the properties of a (slow) cryptographic hash function, although we are currently using one: I wonder if there are more cases where we default to hash functions although a PRF would be enough.

Elsewhere, Vitalik presents some formulae for calculating the "safety" of a finalised block. That is, if a different finalised block were to show up that conflicts with the one you have, what is the minimum number of validators that must have been slashed, taking into account entries to and exits from the validator set?

In other news

Planning continues for an Ethereum 2.0 work day just ahead of EDCON in Sydney. We'll be meeting up on April 9th, during the pre-event hackathon. Info and sign-up sheet here. I'll likely go nocturnal and join remotely, myself, although a bunch of PegaSys folk will be there in person.

We (Artemis team) are thinking about hosting an Eth2 client implementers' workshop around NYC blockchain week (somewhere near May 12th). Watch this space, and let us know if you are interested in being there, leading a session, getting the doughnuts in...

Hsiao-Wei's beautiful slides on the Life of An Ethereum Beacon Chain Validator just missed inclusion in the last edition. It's a terrific overview — take a look.

Vitalik has been getting r/ethtrader excited. I've haven't ploughed through all of that, but here are his concerns about proof-of-stake, and an article off the back of this. Vitalik on the Into The Ether Podcast is pretty good listening.

From the Gitter, a nice discussion about using physical mail to transfer attestations 📨. Could be useful for resiliency. Maybe I need to put this edition out on April 1st 🤔

Finally, if you've made it this far, you deserve some light relief. We're ad-free here, but this from Mainframe is spectacularly good — possibly the best three and a half minutes I spent this week (OK, 7 mins - I watched it twice 🤣). I sincerely hope that this is the future of advertising.


[Twitter][Peepeth]

Follow me on Twitter and/or Peepeth to hear when the next edition is out 🙌.

[RSS] New: We have an RSS feed.


Main sources: