Registrars updates

Back in 2019, I submitted a proposal to add an identity registrar to the Kusama blockchain. This registrar known as Registrar #1 has been operating since.

I gave some details about the process in this article and more recently provided the steps to use Subkey with Registrar #1.

The Registrar #1 is now available on several chains:

and is constantly verifying identities.

When I initially launched Registrar #1, the entire process was very time consuming (for me!) and would not scale to a huge amount of simultaneous verification requests.

In the first version, I mainly used a script (Javascript) running inside the PolkadotJS Apps. Usually, users would contact me and provide their account’s address. I entered this address in my script and let it perform a bunch of checks a pre-verifications for me as well as generating the challenges for each communication channel (email, twitter, etc…​).

The next steps of the process were probably the most time consuming as I had to send those challenges per email, over the Matrix.org network (Riot) and over Twitter. That involved many — boring and error prone — copy/paste.

I had many nice and interesting chats with the people behind the accounts. While that was entertaining, it was hard to track down which verification 'case' was open, partly finished, or close. It was fine with a few verifications per day but that would not have worked for more and I started getting tired of copy/pasting all over the place!

One of the issues I noticed, was that some verifications were very simple and some overly complex and that mainly related to the time it took the registrant to answer and send back the challenges. Some were able to repond and send back signatures in a matter of seconds while it took days for others. I ended up with a queue of verification that was a mix of fresh and older requests.

It was time to implement a solution that could track down and take care of helping and reminding users. I decided for a BPM solution.

In this article, I will not disclose the specifics of the engine I am using mainly because I am not 100% satisfied with its ease of use and I would rather not for this article to sound like an endorsement or a promotion of the solution.

Such BPM engine can run one or several workflows that are usually described using a graphical environment translating the process into BPMN.

Having some experience on the topic helped me quickly design a solution that would be flexible enough to adapt the process without any inconvenience for the users while keeping it flexible enough to adapt to the dynamic aspects of Substrate based chains such as Polkadot and Kusama.

The solution is made of 3 processes, one for each chain. The process itself is the same but connects to different web services, granting access to different chains.

While BPM solutions are good at describing processes and connecting to mainstream solutions (a few bucks will open up the Sharepoint or SAP worlds you may like…​), they are not always great at connecting to new fancy technologies unless investing into developing some connectors in a programming language I don’t even dare to mention here :)

Any decent BPM engine will have connectors that will be able to talk to standard REST endpoints. My first endeavour has been to use the PolkadotJS API in order to expose a few simple endpoints allowing my BPM engine to query the chain for identity, check balances, generate challenges, verify signatures, etc…​

It is also much easier to massage a bit the raw data coming from the chain with a webservice and deliver it 'ready to use' to the workflow.

That was already a great improvement and way less manual work for me! This web service uses confmgr to handle configurations so it can have an instance connecting to a Polkadot node while another connects to Kusama.

The first version of the Workflow was already a great improvement. Users would still contact me with their account’s address and I would simple provide the information into basic a bootstrap based webform.

While simple and not especially beautiful, this form already gave me lots of insights to help me guide the registrants.

It allowed helping either setting their identity as we see below:

or request judgement to the registrar if that bit was missing.

I quickly tweaked the process to allow starting it even when none of the requirements we saw above would be fulfilled. In those cases, the process takes care of sending a nice customized email pointing the user towards the official documentation and providing all the required information, directly coming from the chain (ie: reg_index, max_fee, account’s free balance, etc…​).

The feedback has constantly improved as I brought more useful information and saved the registrant from having to go and fetch information onchain.

Just got my @polkadotnetwork identity verified with @chevdor 's automated identity registrar. He has truly set up a clean, concise, and clear process that anyone can follow. 👍

— Shawn Tabrizi (@shawntabrizi) June 21, 2020

The only requirements to start the workflow automatically after sending the requestJudgement extrinsic are:

The very first step of the process is a pre-check. I used to to it manually (!?!?!?) and tell every single registrant about the rules of Registrar #1. Most of those rules inherited from the history of the data already onchain on Kusama.

For instance, twitter handles to be formated as @handle and not http://twitter.com/handle (which by the way saves a few bytes, prevent bloating the chain and reduces the fees you have to pay to set the identity!)

Some of the branches of the process will trigger depending on the content of the identity. The workflow is now fully in charge of contacting registrants via twitter, email, and matrix.org to send them the challenges as well as all the explanation to successfully get their accounts verified.

I also got a very positive feedback regarding the Matrix.org automation but I let you discover that by yourself if you request a judgement from Registrar #1 :)

At that point, the registrants still have to contact me with their address so we can start the process.

That can be improved since nodes are emitting events that are also available through the PolkadotJS API. Those events contain all the information we need to start the workflow as long as the identity contains an email address.

More automation for Registrar #1 on @polkadotnetwork: The process now starts immediately after seeing your ‘requestJudgement’ transaction on-chain as long as your identity contains a valid email.https://t.co/zRnEc1uEbg

On #Kusama, #Westend, #Polkadot#Blockchain #Identity pic.twitter.com/x97NPessGA

— Chevdor (@chevdor) June 23, 2020

A big part of the process remains the judgement itself.

No serious registrar would:

The second point especially is important. Having the keys of a registrar allows a malicious actor to:

For those reasons, I refrained from automating that part of the process until a dedicated proxy account could be used. It was time for another contribution!

Fortunately Polkadot now supports generalized proxies and I could submit a Pull Request that was merged in Polkadot v0.8.13. This pull request introduces a new ProxyType called IdentityJudgement that allows the registrar’s main account to let one of more proxy accounts providing judgements.

In case an attacker would put his hands on the private key of a proxy, he could only set judgements until the registrar revokes the rights of the proxy and reverts any of the 'illegal' calls. The attacker cannot do anything else with this proxy account. That makes proxies rather unsexy targets.

Now having this new ProxyType, I finally could automate the judgement part of the process. I decided however against making it a fully automatic process, despite that it would now be totally possible and safe.

There are a few reasons for keeping Registrar #1 (It is actually Registrar #0 on Westend…​) semi-automatic (at least for now):

Twitter has shown to be hell to work with. First of all, it tries to be smart in parsing users messages to stuff some urls here and there but it is not smart at all and modifies the messages the process is sending to the users…​

Another twitter related issue is that the process can only send DMs to users if they follow a given account. This has been the source of many failures I had to more of less manually handle in the past. Those were due to users not following the account because they: - did not follow the account! (although it is written in red in the first email that is sent by the process…​) - did not have the time to follow the account: when some users contacted me, some of the processes started quicker than they could read the first email and take action

In order to avoid this issue, the process is now checking whether users are following the right twitter account. If not, the process keep on reminding the registrants at exponentially increasing intervals.

All of that together brings us currently to the following architecture:

I shared pictures of the Workflow diagram a few times already but it becomes rather complex to be really useful for the common reader so instead of sharing another snaphot of a huge and barely readable diagram, I extracted a part of it around the end of the process:

On this extract of the process, we see one human action (in blue on the left) that is still done manually by the Registrar #1 Operator. On the other hand, the activities marked as 'BYPASSED' are no longer required in 99% of the cases but have been left as fallback in case something goes south.

We see that, depending on the judgement, the user will receive a 'Positive' or 'Negative' notification (per email). After that, ignoring those balance related activities, the process is automatically setting the judgement, using a proxy account.

In the event it would fail (ie the proxy has no more funds to send the transaction), the task is then given to the Operator for manual handling.

In practise, this case should never happen^TM. As we saw, the process is reading the balances before and after judgement which allows me guestimating the number of judgements the proxy can still set before running out of funds for the fees. That allows funding the proxy account accordingly and in a timely manner.

Here are a few charts made of the answers from users who verified their identities and nicely spent a little more time to provide some feedback. I have used those answers to improve decide what area of the process to prioritize.

The new versions of the Registrar #1 process are now live and ready to handle your requests.