I’ve written quite a few articles about using Functional Programming (FP) with Magik on this site, and provided detailed code examples. However I haven’t yet put together an entire FP application.
So let’s do that now.
But rather than follow the same-old-same-old and write a standard GIS app, we’ll hook into one of the hottest technologies going right now and develop a Blockchain Magik GIS application.
Interested?
Of course you are, so let’s develop a distributed, immutable ledger of audit history records. That way we can distribute the history of VMDS changes (as well as insertions and deletions) to as many people as we like, independent of the VMDS, and they can be sure the audit history is accurate and hasn’t been modified in any way.
Before we begin, however, it’s necessary for us all to be on the same page. Blockchain is a buzzword and that means there is quite a bit of hype associated with it. Many don’t understand what it is and some even conflate it with cryptocurrencies .
So let’s clear up some misconceptions and then move on to the development. To do that, I’m going to make this a 3-part series with part one (you’re reading part one right now) introducing the Blockchain and developing it in Magik.
Part two will cover how to distribute the Blockchain over a network (private or Internet) and arrive at consensus.
Then in part three we’ll write our audit history application.
Onward to part one.
Bitcoin
Bitcoin brought Blockchain technology to the forefront and is now forever intertwined with it. However Bitcoin is not Blockchain in much the same way Electric Office (EO) is not VMDS.
EO is an application that uses the VMDS. EO requires the VMDS and an associated data model to work, but the VMDS does not require EO. It can be used to create a variety of applications outside of EO.
In the same way, Bitcoin uses the Blockchain as its database and cannot exist in its current form without the Blockchain. However the Blockchain does not need Bitcoin. It can be used to create a variety of applications outside of Bitcoin.
Hopefully that’s clear enough. Having said that, however, I will refer to Bitcoin from time to time in order to show how Bitcoin implements certain aspects on top of the Blockchain. As the most famous Blockchain application, there are some things we can learn by looking at Bitcoin.
With that out of the way, let’s dive in and look at the Blockchain starting from first principles.
What is a Blockchain?
As the name implies, a Blockchain is a chain of blocks linked together in a sequence. Each block contains data and metadata to form a ledger (or database, if you will) containing entries. The Blockchain can be public or private, but applications such as Bitcoin use public ledgers.
That’s really all there is to it. And if everyone in the world was 100% trustworthy and didn’t make mistakes, then we could simply implement our Blockchain as a text document with each block being a paragraph. Each subsequent paragraph would form the next block in the sequence and the entire document would be our Blockchain.
Unfortunately the world includes shady characters, malicious actors and even the most honest of us make mistakes. So we need a way to verify data to ensure no tampering has taken place.
And that is the brilliance of the Blockchain.
It’s public, so everyone can look at it. It’s immutable, so it can’t be changed without others knowing something was changed. It uses consensus, so no one entity can easily influence it and it’s distributed, so no central authority controls it — which also means it’s replicated in many different locations.
All these attributes are built upon the foundation of cryptography in general and the SHA256 digest in particular.
So before we can understand Blockchain, we need to understand SHA256.
SHA256 Digest
As its name suggests, the SHA256 Digest takes data of varying length and digests it into 256 bits. Give it an empty string and you’ll get back 256 bits. Give it the email you just sent to your buddy and you’ll get back 256 bits. Give it the complete works of William Shakespeare and you’ll get back, you guessed it, 256 bits.
It’s what we call, in FP terminology, a pure function. It maps inputs to one output and depends solely on its input. No matter how many times you provide that input, you’ll receive the same output. It doesn’t produce side-effects or use hidden inputs. Pure functions eliminate entire classes of errors and make concurrent programming a breeze. For more details, see my articles about the benefits of Functional Programming.
But let’s get back to those 256 bits, because they have some special properties. First, as mentioned, you’ll get the identical 256 bits back for identical input each and every time. Second, for all practical purposes, the output is unique. Third, although it’s easy to verify whether a given piece of text gives you a particular 256 bit value (just run the text through the SHA256 algorithm and compare the output), it’s absurdly impractical to reverse that and determine the text input if you only have the 256 bit value.
As it turns out, these properties make SHA256 ideal for use in the Blockchain.
When we turn input text into a 256 bit output, we call this process hashing (and we call the result a hash). Hashing forms the foundation upon which the Blockchain is built.
Smallworld has a built-in method named system.sha256_digest(). Pass in a text argument (or even a non-text argument, but for our purposes we’ll only be dealing with text) and you’ll receive a 256 bit output – the output is displayed as a 64 character hexadecimal string, but since each hex character is equivalent to 4 bits, the output is 256 bits.
The code below shows a few examples of hashing in action.
Magik> system.sha256_digest("")
"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
Magik> system.sha256_digest("abc")
"ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad"
Magik> system.sha256_digest("abcdefghijklmnopqrstuvwxyz")
"71c480df93d6ae2f1efad1447c66c9525e316218cf51fc8d9ed832f2daf18b73"
Magik> system.sha256_digest("abcdefghijklmnopqrstuvwxyZ")
"9597f3a8256752399e151f39d6dad267882c481852af6be95e00aaa936083bec"
Magik> system.sha256_digest("Hello World!")
"7f83b1657ff1fc53b92dc18148a1d65dfc2d4b1fa3d677284addd200126d9069"
Magik>
$
Notice how a small change in the input (lines 7 and 10) produces a large change in the output. A one character change makes a big difference in the result.
By hashing the data, as well as some other values, we generate a unique fingerprint for these values. If even one character is modified, inserted or deleted, our hash value changes. And that’s how we can ensure data hasn’t been tampered with.
Okay, enough with the theory. Let’s start writing code.
A Blockchain Factory
_global beeble_blockchain_factory <<
_proc @beeble_blockchain_factory()
# returns a blockchain object holding methods and data.
# PRIVATE values and methods.
_constant GENESIS_BLOCK_PREVIOUS_HASH << "0000000000000000000000000000000000000000000000000000000000000000"
_constant CALC_HASH << _proc @hash_block(p_block)
_return system.sha256_digest(write_string(p_block.block_num,
p_block.nonce,
p_block.data,
p_block.previous_hash,
p_block.timestamp))
_endproc
_constant BLOCKCHAIN << rope.new()
# PUBLIC methods.
_constant EXPORT << beeble_object.new()
EXPORT.create_block << _proc @create_block(p_data)
_import BLOCKCHAIN
_import CALC_HASH
_import EXPORT
_constant LAST_BLOCK << BLOCKCHAIN.last
_constant BLOCK << beeble_object.new({:block_num, LAST_BLOCK.block_num + 1,
:data, p_data,
:nonce, 0,
:timestamp, date_time.now().write_string,
:previous_hash, LAST_BLOCK.hash})
BLOCK.hash << CALC_HASH(BLOCK)
BLOCKCHAIN.add_last(BLOCK)
_endproc
# generate the Genesis block and add it as the first entry on the Blockchain.
_constant GENESIS_BLOCK << beeble_object.new()
GENESIS_BLOCK.block_num << 0
GENESIS_BLOCK.nonce << 0
GENESIS_BLOCK.data << "Genesis Block"
GENESIS_BLOCK.previous_hash << GENESIS_BLOCK_PREVIOUS_HASH
GENESIS_BLOCK.timestamp << date_time.now().write_string
GENESIS_BLOCK.hash << CALC_HASH(GENESIS_BLOCK)
BLOCKCHAIN.add(GENESIS_BLOCK)
_return EXPORT
_endproc
In the code above, we’ve created a Blockchain factory that will return a Blockchain instance for us. This is a standard FP Module pattern that’s analogous to creating objects from exemplars in OO programming.
I’ve described how this works in another article, so I won’t get into the details here. But feel free to read through that article if you want to understand what’s happening.
The factory uses a Closure to create private variables and methods; it also exports an object containing its public methods. Take note anything private is truly private, unlike supposedly private data and methods in Magik classes.
Right now our factory has three private data items (GENESIS_BLOCK_PREVIOUS_HASH in line 7, BLOCKCHAIN in line 19 and EXPORT in line 24), one private method (CALC_HASH in line 9) and one public method (CREATE_BLOCK in line 28).
I’ll also point out the fact I’ve used a beeble_object in line 24 (we’ll be making frequent use of this prototypal object, so it’s a good idea to become familiar with it).
Also notice the factory automatically creates the first block (lines 51 to 56) for us and adds it to the Blockchain (line 58). This block is called the Genesis Block.
The factory give us a good starting point and exposes methods that allow us to create new blocks on the Blockchain.
Let’s do that now.
Magik> bc << beeble_blockchain_factory()
a beeble_object
Magik> print(bc)
{"create_block":"proc create_block(p_data)"}
Magik> bc.create_block("Block 1")
Magik> print(bc)
{"create_block":"proc create_block(p_data)"}
Magik> bc.create_block("Block 2")
Magik> print(bc)
{"create_block":"proc create_block(p_data)"}
Magik>
We use CREATE_BLOCK to create blocks on the Blockchain. But… you might have realized, we can’t see the Blockchain. And that’s because our FP module pattern locks it away in a Closure.
Try anything you can think of in Magik (outside of modifying the source code) and you will not be able to access the Blockchain. Sys!slot() doesn’t work. Sys!perform() doesn’t work. Using the .private?(_false) method doesn’t work. You can’t see that private BLOCKCHAIN constant no matter what you do, however it is there – in memory, held in a Closure.
Let’s remedy that situation by creating another public method to expose its contents.
EXPORT.blocks <<
_proc @blocks()
_import BLOCKCHAIN
# return a copy of the blockchain. However the block elements are NOT copies.
# this allows us to change the blocks (for testing purposes), but keeps the
# blockchain private and inacessible except through the exported public methods.
_return BLOCKCHAIN.fp_map(_proc(p_block) print(p_block); _return p_block _endproc)
_endproc
There are two things I’d like you to notice about the code above.
First, it returns a copy of the private BLOCKCHAIN constant (this is standard behaviour for fp_map()). So anyone with a handle to that copy, can’t actually modify the private BLOCKCHAIN. However the block elements in the Blockchain aren’t copies (so we can modify these — which we’ll do later to simulate someone trying to change values on the chain).
Second, we use fp_map() to traverse the rope in a functional manner rather than using an imperative loop. I’ve written extensively about how map works in FP, so if you’re unclear, check it out here. This makes it easy for us to change functionality by simply providing different procedures to fp_map(). It decouples functionality from control structures and is a concise, elegant way of writing code.
Great. Now we can see our Blockchain. Let’s create some new blocks and look at them.
Magik> bc << beeble_blockchain_factory()
a beeble_object
Magik> print(bc)
{"create_block":"proc create_block(p_data)","blocks":"proc blocks"}
Magik> bc.create_block("Block 1")
Magik> bc.create_block("Block 2")
Magik> bc.create_block("Block 3")
Magik> bc.blocks()
{"previous_hash":"0000000000000000000000000000000000000000000000000000000000000000","hash":"90613b187c1c2330c0ff5cca1ba70a3114e0e5c4870afe7d5d50beb640f691cc","block_num":"0","timestamp":"09/06/2021 21:23:12","data":"Genesis Block","nonce":"0"}
{"previous_hash":"90613b187c1c2330c0ff5cca1ba70a3114e0e5c4870afe7d5d50beb640f691cc","hash":"1049f1cbef2e94f70e19f1ed40f7998cb23889e2ec807e5add2d345829f1bbca","block_num":"1","timestamp":"09/06/2021 21:23:19","data":"Block 1","nonce":"0"}
{"previous_hash":"1049f1cbef2e94f70e19f1ed40f7998cb23889e2ec807e5add2d345829f1bbca","hash":"506cc07f600b0a03907fc37d6380c05e6ce598242d66627ef12c21296ee124fd","block_num":"2","timestamp":"09/06/2021 21:23:20","data":"Block 2","nonce":"0"}
{"previous_hash":"506cc07f600b0a03907fc37d6380c05e6ce598242d66627ef12c21296ee124fd","hash":"04e62e9436d9f063c2d5bb2e4d45ac61a0594f3adf02e5366e78c1ec946c1a17","block_num":"3","timestamp":"09/06/2021 21:23:21","data":"Block 3","nonce":"0"}
sw:rope:[1-4]
Magik>
And just like that we have a Blockchain… almost.
Details of a Block
Before I get into what else is needed for a fully functioning Blockchain, let’s take a step back and look at CREATE_BLOCK.
EXPORT.create_block <<
_proc @create_block(p_data)
_import BLOCKCHAIN
_import CALC_HASH
_import EXPORT
_constant LAST_BLOCK << BLOCKCHAIN.last
_constant BLOCK << beeble_object.new({:block_num,
LAST_BLOCK.block_num + 1,
:data, p_data,
:nonce, 0,
:timestamp,
date_time.now().write_string,
:previous_hash, LAST_BLOCK.hash})
BLOCK.hash << CALC_HASH(BLOCK)
BLOCKCHAIN.add_last(BLOCK)
_endproc
$
You’ll notice it creates a beeble_object to represent the block (line 9) and adds properties named block_num, nonce, data, previous_hash and timestamp.
Let’s go through each one.
block_num is the number assigned to that particular block (which is simply the next integer added to the previous block’s block_num in the Blockchain). The Genesis Block has a block_num of 0.
nonce is a number we use once. I’ll explain more about this important property later.
data contains whatever data we want to store in the block. It can be anything, but for our application it will eventually hold an audit record.
previous_hash is the previous block’s hash. It’s used to link blocks in the Blockchain and helps to verify the chain is valid.
timestamp is self-explanatory and generated by Magik’s date_time.now() method.
Other applications may add additional properties, but for our purposes these ones will do.
(As an aside, Bitcoin adds properties for block size, number of transactions, version, a Merkle Root and more. If you’d like to see what Bitcoin uses and how they’re used, do a Google search for, “Bitcoin Header”.)
Now look at the CALC_HASH() proc and note how it uses all these properties, in the specific order listed, to create a hash via the system.sha256_digest() method.
_constant CALC_HASH <<
_proc @hash_block(p_block)
_return system.sha256_digest(write_string(p_block.block_num,
p_block.nonce,
p_block.data,
p_block.previous_hash,
p_block.timestamp))
_endproc
This hash forms the basis of the entire Blockchain.
(Incidentally, the Bitcoin protocol uses double-hashing, where the resulting hash is fed back into the SHA256 digest to get the final hash. The exact reason is unknown and some have speculated it is to protect against a length extension attack. However there’s really no need to do this, so for our purposes we’ll stick to single hashing.)
Now let’s jump back to CREATE_BLOCK which uses CALC_HASH, in line 17, to generate a unique hash for the block it just created. What’s important is to notice all the properties, except for nonce, are fixed. They can’t be changed once the block has been created.
The block_num and previous_hash can’t be changed because those are based on properties of the previous block, and data and timestamp can’t be changed because these values represent immutable properties.
After all, it wouldn’t be good to change the data or timestamp because that would not represent the actual data in the block or when it was created.
However, because the nonce can change, and because changing it also changes the hash value, we can produce different hashes for blocks simply by changing the nonce.
Finally CREATE_BLOCK adds the new block to the Blockchain in line 20.
After we’ve added some blocks, take a look at the Blockchain.
Magik> bc.blocks()
{"previous_hash":"0000000000000000000000000000000000000000000000000000000000000000","hash":"90613b187c1c2330c0ff5cca1ba70a3114e0e5c4870afe7d5d50beb640f691cc","block_num":"0","timestamp":"09/06/2021 21:23:12","data":"Genesis Block","nonce":"0"}
{"previous_hash":"90613b187c1c2330c0ff5cca1ba70a3114e0e5c4870afe7d5d50beb640f691cc","hash":"1049f1cbef2e94f70e19f1ed40f7998cb23889e2ec807e5add2d345829f1bbca","block_num":"1","timestamp":"09/06/2021 21:23:19","data":"Block 1","nonce":"0"}
{"previous_hash":"1049f1cbef2e94f70e19f1ed40f7998cb23889e2ec807e5add2d345829f1bbca","hash":"506cc07f600b0a03907fc37d6380c05e6ce598242d66627ef12c21296ee124fd","block_num":"2","timestamp":"09/06/2021 21:23:20","data":"Block 2","nonce":"0"}
{"previous_hash":"506cc07f600b0a03907fc37d6380c05e6ce598242d66627ef12c21296ee124fd","hash":"04e62e9436d9f063c2d5bb2e4d45ac61a0594f3adf02e5366e78c1ec946c1a17","block_num":"3","timestamp":"09/06/2021 21:23:21","data":"Block 3","nonce":"0"}
sw:rope:[1-4]
Magik>
Notice how each block holds the value of the previous block’s hash and how each subsequent block’s block_num is 1 greater than the previous block’s block_num in the Blockchain? That’s important because they not only link blocks together, but both are used to verify a Blockchain is valid.
And speaking of validity, let’s write some code to see if our newly minted Blockchain is valid.
Validating a Block
First we’ll define a private procedure to check if a block is valid.
# checks to see if a single block is valid. This means it has been mined and the difficulty target has been reached.
_constant BLOCK_VALID? <<
_proc @block_valid?(p_block)
_import l_difficulty_target
_import CALC_HASH
_constant HASH << CALC_HASH(p_block)
_return HASH.slice(1, l_difficulty_target.size) = l_difficulty_target
_endproc
$
In line 9 we check to see if the required number of leading zeros is present in the hash. If so, the block is valid, otherwise it’s not.
For a block to be valid, its hash must be less than the current difficulty target, which is just a numeric value. As the difficulty increases (i.e. the target number gets lower), the more computing power is required to find a valid hash – which is known as mining a block. And the more computing power required to mine a block means the Blockchain network is more secure.
In our implementation we’ll use a shortcut to compare our hash to the target because the target and hash numbers are very, very, very large (more than 67 base 10 digits).
So instead of checking to see if our hash (a very large number) is less than the difficulty target (another very large number), we’ll simply compare the first n digits of our hash and require them to be zeros. As n increases, the difficulty increases because our hashes have less probability of meeting the requirements.
Validating the Blockchain
Once we know how to validate a block, we then define a private procedure with four rules that need to be satisfied for a Blockchain to be considered valid.
As you’ll see, if you try to validate the blocks we created above, validation will fail because our blocks haven’t been mined so they won’t pass the difficulty target test that shows proof-of-work has been done.
“Say what?”
“Proof-of-work?”
“What is this proof-of-work of which you speak?”
Slow down bucko, we’ll get there in a moment. But before we do, let’s put on our conceptual hardhats and look at the Blockchain verification rules as well as the concept of mining.
Onward.
# verify all blocks on the blockchain are valid.
_constant VERIFY_BLOCKS <<
_proc @verify_block(p_old_block, p_block)
_import CALC_HASH
_import BLOCK_VALID?
# Rule 1:
_if BLOCK_VALID?(p_block) _isnt _true
_then
# this occurs if the block hasn't been mined with a nonce that hashes to the difficulty target.
print(p_block)
condition.raise(:error, :string, "new block is NOT valid.")
_endif
_if p_block = p_old_block
_then
# skip the Genesis block.
_return p_block
_endif
# Rule 2:
_if p_old_block.block_num + 1 <> p_block.block_num
_then
# blocks on the blockchain are out of order.
write("old block:")
print(p_old_block)
write(newline_char,"new block:")
print(p_block)
condition.raise(:error, :string, "old block number + 1 <> new block number.")
_endif
# Rule 3:
_if p_old_block.hash <> p_block.previous_hash
_then
# this is how we chain blocks together on the blockchain.
write("old block:")
print(p_old_block)
write(newline_char,"new block:")
print(p_block)
condition.raise(:error, :string, "old block hash <> new block previous hash.")
_endif
# Rule 4:
_if CALC_HASH(p_block) <> p_block.hash
_then
write(newline_char,"new block:")
print(p_block)
condition.raise(:error, :string, "calc(new block) <> new_block hash.")
_endif
_return p_block
_endproc
$
Lines 9 (which uses the private BLOCK_VALID?) , 23, 36 and 49 define the rules we need for now.
Finally we’ll define a public method (valid?), that uses VERIFY_BLOCKS, to check for Blockchain validity.
EXPORT.valid? <<
_proc @valid?()
_import BLOCKCHAIN
_import CALC_HASH
_import BLOCK_VALID?
_import VERIFY_BLOCKS
_try _with cond
_if BLOCKCHAIN.empty?
_then
_return _false
_endif
_return BLOCKCHAIN.fp_reduce(VERIFY_BLOCKS, BLOCKCHAIN.first).first = BLOCKCHAIN.last
_when error
write("Failed because ", cond.report_string)
_return _false
_endtry
_endproc
$
Line 16 uses fp_reduce() with the VERIFY_BLOCKS procedure to validate the Blockchain. This, once again, is the functional way of doing what would normally be done with imperative loops. See if you can understand how it works. If you need a refresher on what fp_reduce() does, check out this article.
Once we have our public valid? method, we can start testing our Blockchain.
Magik> bc << beeble_blockchain_factory()
a beeble_object
Magik> bc.create_block("Block 1")
Magik> bc.create_block("Block 2")
Magik> bc.create_block("Block 3")
Magik> bc.valid?()
{"block_num":"1","data":"Block 1","previous_hash":"0000441e632646be8759cffe45c080baa6df29e3d10459bf509741be52f7fcdb","timestamp":"10/06/2021 11:27:09","hash":"694570757b3498eac9aeceeb74edf7208986e85f7175df9cfb5634cc813b4543","nonce":"0"}
Failed because **** Error: new block is NOT valid.
False
Magik>
Whoops! See how our Blockchain is invalid as reported in line 12? I told you that would happen… and the reason it happened is because our block didn’t pass the rule requiring a block’s hash to start with the required number of leading zeros. The hash, in this case, is…
694570757b3498eac9aeceeb74edf7208986e85f7175df9cfb5634cc813b4543
Sooooo, we need to mine the blocks in order to get those elusive leading zeros.
Mining a Block
As previously mentioned, only the nonce in a block is available to be changed. Therefore, in order to mine a block and make it valid, we look for a nonce, that when hashed with the other properties, gives us a hash that satisfies the difficulty target.
For demonstration purposes we’ll set our difficulty target to be a hash with 4 leading zeros. This allows us to see some work being done but doesn’t take too long to come up with a result. Of course when we put our application into production, we’ll significantly increase the difficulty target.
Go back and add the difficulty target (as in line 6 below) to the top of the beeble_blockchain_factory procedure.
_global beeble_blockchain_factory <<
_proc @beeble_blockchain_factory()
# returns a blockchain object holding methods and data.
_local l_difficulty_target << "0000"
.
.
.
<all the other code>
.
.
.
_endproc
We’re now almost ready to mine… just as soon as we write the mining code.
Here’s that code.
EXPORT.mine <<
_proc @mine(p_block)
_import CALC_HASH
_import BLOCK_VALID?
_local l_nonce << 0
_loop
p_block.nonce << l_nonce
p_block.hash << CALC_HASH(p_block)
_if BLOCK_VALID?(p_block) _is _true
_then
_leave
_endif
l_nonce +<< 1
_endloop
p_block.nonce << l_nonce
_return p_block
_endproc
Note how we start with a nonce = 0, execute CALC_HASH() and then check to see if the block is valid by calling BLOCK_VALID?
If valid, we set the nonce for that block (line 22) and consider it mined. If not, we increment the nonce’s value (line 18) and repeat until we’ve found a suitable one.
The idea is that finding a hash with the required number of leading zeros takes some computing power. However once that hash is found, it’s easy to verify it’s correct.
Mathematically we can say:
y = H(x)
where y is the hash, H is the hashing function (i.e. SHA256) and x is the concatenated data and other properties we’re feeding to the SHA256 digest.
Calculating y is easy if we are only given x (we simply run it through H). But going the other way is not feasible on a practical level because of the one-way nature of H. So given only y, there is no function, H’, that makes it feasible to find x.
This leads to a significant asymmetry that ensures work must be done to find a suitable nonce that, when combined with the other relevant but unchanging properties, produces y.
However, once that nonce has been calculated, it is easy to verify the required work was done by simply hashing x (which includes the newly calculated nonce) and ensuring the result is y.
With that business out of the way, let’s see how changing the difficulty level correlates to required computing power and, by extension, time. The example below shows the time required to mine a block with increasing difficulty targets.
target 0 --> Duration: 0 milliseconds
target 00 --> Duration: 0 milliseconds
target 000 --> Duration: 0 milliseconds
target 0000 --> Duration: 3 seconds 0 milliseconds
target 00000 --> Duration: 1 minute 16 seconds
target 000000 --> Duration: 1 minute 25 seconds
target 0000000 --> Duration: 59 minutes 34 seconds
target 00000000 --> Duration: > 24 hours
As you can see, the higher the target, the more time it takes. By the time we get to 8 leading zeros, the time it takes to mine a block is orders of magnitude more than the earliest levels. And I ran this on an 8-core PC with 64GB of RAM.
It should be apparent that mining a block with a production-strength difficulty target is no easy feat. And that’s a major reason the Blockchain is so secure.
Now let’s get back to validating our Blockchain. We’ll create some blocks on the Blockchain and then test its validity.
Magik> bc << beeble_blockchain_factory()
a beeble_object
Magik> bc.create_block("Block 1")
Magik> bc.create_block("Block 2")
Magik> bc.create_block("Block 3")
Magik> bc.valid?()
{"block_num":"1","data":"Block 1","previous_hash":"9d8d8a68f49ca415639d1427e529b4321690d82168c447fe382c1e4e3e709605","timestamp":"10/06/2021 12:25:05","hash":"41114139abebc5c433097a7aea77add14d55b08fad79e2eb35a5b18c64fa25a8","nonce":"0"}
Failed because **** Error: new block is NOT valid.
False
Magik> blks << bc.blocks()
{"block_num":"0","data":"Genesis Block","previous_hash":"0000000000000000000000000000000000000000000000000000000000000000","timestamp":"10/06/2021 12:25:03","hash":"9d8d8a68f49ca415639d1427e529b4321690d82168c447fe382c1e4e3e709605","nonce":"0"}
{"block_num":"1","data":"Block 1","previous_hash":"9d8d8a68f49ca415639d1427e529b4321690d82168c447fe382c1e4e3e709605","timestamp":"10/06/2021 12:25:05","hash":"41114139abebc5c433097a7aea77add14d55b08fad79e2eb35a5b18c64fa25a8","nonce":"0"}
{"block_num":"2","data":"Block 2","previous_hash":"41114139abebc5c433097a7aea77add14d55b08fad79e2eb35a5b18c64fa25a8","timestamp":"10/06/2021 12:25:06","hash":"d7ad705c35ffc7f07eeb1ec57bdbf6d10c866fe5c21756c045ee4dc556c0d2a3","nonce":"0"}
{"block_num":"3","data":"Block 3","previous_hash":"d7ad705c35ffc7f07eeb1ec57bdbf6d10c866fe5c21756c045ee4dc556c0d2a3","timestamp":"10/06/2021 12:25:07","hash":"d3ae9f18bc5aac75c3d52121b6185edd45f3a1484983121a9b47feea4dcc0d6e","nonce":"0"}
sw:rope:[1-4]
Magik>
As expected, validation failed (line 12) because the blocks’ hashes (lines 17 to 20) don’t have the required 4 leading zeros (because they weren’t mined).
So let’s mine them…
Magik> bc.mine(blks[1])
a beeble_object
Magik> bc.mine(blks[2])
a beeble_object
Magik> bc.mine(blks[3])
a beeble_object
Magik> bc.mine(blks[4])
a beeble_object
Magik> blks << bc.blocks()
{"block_num":"0","data":"Genesis Block","previous_hash":"0000000000000000000000000000000000000000000000000000000000000000","timestamp":"10/06/2021 12:25:03","hash":"0000b1a91efea3aedb9bedd9da7158c4c8490cf3b16f8d5a2cde8d540bea0e38","nonce":"47798"}
{"block_num":"1","data":"Block 1","previous_hash":"9d8d8a68f49ca415639d1427e529b4321690d82168c447fe382c1e4e3e709605","timestamp":"10/06/2021 12:25:05","hash":"0000de3d374071b2778fde32c564873f001a3b1689f1a1763a0cc5f8d1baf8d9","nonce":"117698"}
{"block_num":"2","data":"Block 2","previous_hash":"41114139abebc5c433097a7aea77add14d55b08fad79e2eb35a5b18c64fa25a8","timestamp":"10/06/2021 12:25:06","hash":"000075ad12ce0193f4b17e1446a27912a99a799f6ce42ebea23866c5552f2e3b","nonce":"40794"}
{"block_num":"3","data":"Block 3","previous_hash":"d7ad705c35ffc7f07eeb1ec57bdbf6d10c866fe5c21756c045ee4dc556c0d2a3","timestamp":"10/06/2021 12:25:07","hash":"00008dfad89c15bb4003e8a987349bf5b9c2904171d6405134fc0cd5e4a107a5","nonce":"51794"}
sw:rope:[1-4]
Magik> bc.valid?()
old block:
{"block_num":"0","data":"Genesis Block","previous_hash":"0000000000000000000000000000000000000000000000000000000000000000","timestamp":"10/06/2021 12:25:03","hash":"0000b1a91efea3aedb9bedd9da7158c4c8490cf3b16f8d5a2cde8d540bea0e38","nonce":"47798"}
new block:
{"block_num":"1","data":"Block 1","previous_hash":"9d8d8a68f49ca415639d1427e529b4321690d82168c447fe382c1e4e3e709605","timestamp":"10/06/2021 12:25:05","hash":"0000de3d374071b2778fde32c564873f001a3b1689f1a1763a0cc5f8d1baf8d9","nonce":"117698"}
Failed because **** Error: old block hash <> new block previous hash.
False
Magik>
We’re getting closer. Notice our blocks now have the required 4 leading zeros in their hashes (lines 14 to 17) so Rule 1 passes, however we have another problem in line 27: Rule 3 fails. The previous_hash now doesn’t match the new hash calculated by mining the block.
We can fix that by copying the newly created hash into the previous_hash property for the next block (for each block) and try again, but that won’t work either. As an exercise, try doing that and see what happens. It will give you a good sense of why the Blockchain algorithm is so secure.
Proof of Work
See how each block’s hash, in the previous example, now starts with at least 4 leading zeros? This indicates the work was done in order to find the correct nonce, and thus we call it proof-of-work.
And that’s all there is to proof-of-work and mining.
So once we’ve mined a block and found a nonce that gives us a hash with 4 leading zeros, we say that block is valid.
If even the smallest bit of data changes in any of the properties we’ve hashed (or even the order in which we’ve concatenated them changes), the hash will be different and the entire mining process will have to be re-run in order to validate the block again.
With our easy difficulty target, re-mining only takes a few seconds. However as the difficulty target increases, more time will be required, even on very powerful hardware. So re-mining a block will take time but, as we shall see, because each block is linked to others via the block_num and previous_hash properties, when a block’s hash changes, all blocks that follow it also change and therefore they all become invalid.
Consequently anyone who changes a block in the Blockchain will have to not only re-mine that block, but also all subsequent blocks in the chain. If our proof-of-work is difficult, the time required will be enormous and therefore infeasible.
Also keep in mind that new blocks are constantly being mined and added to the Blockchain, so our re-mining bad actor will be continually playing catch up.
Next Steps
Yet even if the bad actor managed to catch up, there’s still another hurdle to overcome.
We’ll look at that in the next part of this series when we create a distributed Blockchain network.
For now, play around with the code. Create new blocks on the Blockchain. Change a property in a block and see how it becomes invalid. Try to fix it without re-mining the entire Blockchain. You can’t. It’s like herding cats. When you fix one thing, another thing breaks in the chain… and on and on it goes. But try anyways. It will teach you something valuable and it’s fun.
After that, see if you can fix the code so the Blockchain is valid when it’s created and when you add blocks. I’ll give you a hint… you need to mine the block when it’s created and you should also mine the Genesis Block.
Once you’ve done that, you’ll have a fully functional Blockchain written in a functional manner using functional programming. I’d highly recommend you study the code to understand how the FP pieces work. It will improve your coding skills even if you decide to stick with how you’re writing code now.
So that’s it for part one. In part two we’ll look at how to make our Blockchain super-secure by distributing it over a network.
See you in the next part.
