Last year, I spent some time in Jakarta visiting HARA, an AWS customer. They've created a way to connect small farms in developing nations to banks and distributers of goods, like seeds, fertilizer, and tools. Traditionally, rural farms have been ignored by the financial world, because they don't normally have the information required to open an account or apply for credit. With HARA, this hard-to-obtain data on small farms is collected and authenticated, giving these farmers access to resources they've never had before.

A major component to the system that HARA created is blockchain. This is a technology used to build applications where multiple parties can interact through a peer-to-peer-network and record immutable transactions with no central trusted authority. HARA has had to develop additional technologies to make their application work on Ethereum, a popular, open source, blockchain framework.

This is an issue many customers, such as Nestle and Singapore Exchange, have shared with us—building a scalable enterprise application on blockchain today is time-consuming and complex. Organizations that can't get past these challenges are left behind. Their ideas are left behind too, ideas that could change the world for the better, like what HARA is doing.

To remove the complexity associated with blockchain networks, we recently announced Amazon Managed Blockchain, a fully managed service that makes it easy to create and manage scalable blockchain networks.

We also announced Amazon Quantum Ledger Database (Amazon QLDB), a ledger database that provides some of the same features as blockchain for data integrity. It's designed for centralized systems, where there is a central trusted authority. Amazon QLDB provides a transparent, immutable, and cryptographically verifiable transaction log ‎owned by a central trusted authority.

In this post, I dive deeper into some of these concepts, the challenges people face building with blockchain, and how our two new services solve these issues.

A closer look at blockchain and ledger technology

First, let's take a closer look at some of the technology behind blockchain: Ledgers, decentralized networks, consensus algorithms, and smart contracts.

Blockchains are designed to maintain the integrity of data. They are immutable—committed data history cannot be altered or deleted, it can only be updated. Ledgers are the building blocks that help make blockchains immutable, so I'm going to explain this concept in detail.

Ledgers have been around for a long time and were typically used to record a history of economic and financial activity between two or more parties. Earlier civilizations, such as Mesopotamia and Ancient Egypt, used ledgers built with stones and papyrus plants. Today, a banking application that tracks credits and debits is one of the most common examples of a ledger.

Ledgers found in a blockchain (and now in Amazon QLDB) typically consist of the following:

• Current and historical state: A data structure that keeps the current and historical state values, allowing applications to easily access the data without needing to traverse the entire transactional log.
• A journal: A transactional log that keeps a complete record of the entire history of data changes. The transactional log is append-only, meaning that each new record is chained to the previous, allowing you to see the entire lineage of data's change history. Additionally, with the help of cryptographic hashing, a process that assigns a unique identifier (like a fingerprint) to each record, blocks are chained to one another. This allows ledgers to have a timekeeping property allowing anyone to look back in time and get proof that the data transaction occurred, making auditing simple.

Compare this to relational databases where customers have to engineer an auditing mechanism because the database is not inherently immutable. Such auditing mechanisms built with relational databases can be hard to scale. They put the onus on the application developer to ensure that all the right data is being recorded.

In addition to a distributed ledger, blockchains also include a consensus mechanism and smart contract execution environment.

• Consensus algorithms help ensure that the members in the network have an agreed-upon method to allow transactions and data to be committed to the ledger. If the consensus requirements aren't met, then the transaction is rolled back and not valid.
• Smart contracts are programs that have the rules and penalties of engagement for a contract defined into lines of code. The program continuously checks when the conditions for a contract are met, and then ensures that the contract gets automatically executed.

Together, these elements allow two or more parties to transact with decentralized trust, where each party consents to the transaction and records the transaction. Decentralized trust makes sense when multiple organizations must independently verify transaction history and have a single, up-to-date, accurate view of data. It also makes sense when there is no single party that wants to maintain an application but network members still want to transact with other parties efficiently.

Let's take the example of a trade finance application where decentralized trust is needed. Trading goods across international borders requires many orgs to work with one another, such as importers, exporters, multiple banks, shipping companies, insurance companies, and customs departments. With the number of stakeholders involved, there is no trusted central authority. Each stakeholder wants to independently verify the documentation related to the trade and doesn't want any single entity to own the record of activity.

The current process requires trade-related paperwork (for example, a letter of credit) to go back and forth between the stakeholders, which can take 5–10 business days to complete. This results in long processing time and high costs.

In this scenario, enabling the stakeholders to operate with decentralized trust improves efficiency and cuts down costs. A single participant does not own the infrastructure, and the system distributes a copy of the transaction ledger to each participant for independent verification. The business contract, such as a letter-of-credit, can be written as a smart contract in the blockchain application. It can automatically execute as soon as all parties provide consensus to record the transaction.

Customers look to blockchain as technology that enables them to transact with multiple parties when there is no single trusted authority and they need a system with decentralized trust.

Making blockchain easy with Managed Blockchain

It's difficult, expensive, and time-consuming to create and manage blockchain networks using existing frameworks. First, to create a blockchain network with permissions, each network member must manually provision hardware, install software, create and manage certificates for access control, and configure networking components. When the blockchain network is running, users have to continuously monitor the infrastructure. They must adapt to changes, such as an increase in transaction requests or new members joining or leaving the network.

To help overcome the barriers that people face trying to build using blockchain, AWS has created Managed Blockchain. Unlike a self-hosted blockchain network, Amazon Managed Blockchain eliminates the need for manually provisioning hardware, configuring software, and setting up networking and security components. This service allows users to set up and manage a scalable blockchain network with just a few clicks. It automatically scales to meet the demands of thousands of applications running millions of transactions.

Managed Blockchain manages certificates and track operational metrics, such as usage of compute, memory, and storage resources. With Managed Blockchain's voting API, network participants can add or remove members. When a network member requires additional capacity for creating and validating transactions,the member can quickly add a new node using Managed Blockchain's APIs.

In addition, Managed Blockchain provides a selection of instance types that comprise varying combinations of CPU and memo. This gives you the flexibility to choose the appropriate mix of resources to support your nodes. Users pay according to their usage and don't worry about any upfront costs for infrastructure.

Managed Blockchain supports two popular blockchain frameworks, Hyperledger Fabric and Ethereum. Hyperledger Fabric is well-suited for applications that require stringent privacy and permission controls and with a known set of members. For example, this might include a financial application where certain trade-related data is only shared between a subset of the network (only the banks that are part of the trade).

Ethereum is well suited for highly distributed blockchain networks where transparency of data for all members is important. Each transaction is visible to all the members of the network. For example, this might include a customer loyalty blockchain application that allows any network retailer to verify user activity by broadcasting the transaction to all members.

Managed Blockchain is now generally available for Hyperledger Fabric (Ethereum coming soon).

Managed Blockchain creates a network and manages its key components:

• Members
• Hyperledger Fabric's certificate authority (CA) for each member
• Peer nodes
• The ordering service

A blockchain network is a peer-to-peer network running a decentralized blockchain framework. Each network includes one or more members, which are unique identities in the network (for example, an organization in a consortium of banks). Each member on the network can easily create their own peer nodes that come with a variety of compute and storage options.

With each member, the Hyperledger Fabric CA also gets created. The Hyperledger Fabric CA provides a number of certificate services that relate to user enrollment, transactions invoked on the blockchain, and TLS-secured connections between users or components of the blockchain.

The peer nodes of each member interact to run smart contracts known as chaincode in Hyperledger Fabric. They create and endorse transactions proposed in the network, and store a local copy of the ledger.

Members define the rules in the endorsement process based on their application's business logic. For example, in a trade finance application, the bank for the exporting party wants to verify that the importing party has necessary funds before endorsing the transaction. To configure blockchain applications on peer nodes and to interact with other network resources, members use a client configured with the AWS CLI or SDK. Those network resources could include the certificate authority, ordering service, and peer nodes.

Managed Blockchain provides endpoints to access these services, which can be accessed via an AWS PrivateLink endpoint. Additionally, in Managed Blockchain, we have improved the reliability of the default ordering service, a component that ensures delivery of transactions across the blockchain network. We have rebuilt the ordering service using the same underlying technology as Amazon QLDB. There's even now an immutable change log that accurately maintains the complete history of all transactions in the blockchain network, ensuring that you durably save this data.

A ledger solution with centralized trust

We've spent a lot of time at AWS talking to customers about blockchain to better understand the problems that it solves. Something interesting that we've found is that many companies are interested in blockchain because they need a ledger that is transparent, immutable, and cryptographically verifiable. However, they don't have a use case that requires decentralized trust.

One example of this is a department of motor vehicles that tracks vehicle ownership and registration history in a way that is resilient against data manipulation. People have tried to solve this problem with a relational database, but it's not built to be immutable. Users have to do a bunch of work to create any complex auditing functionality. Also, with a traditional database there's no way to go back in time and verify the integrity of the change log.

Logically, the next thought is, "What about the ledger component of a blockchain?" This solves the problems around data integrity and audit functionality, in addition to providing decentralized trust for transactions involving multiple parties. For companies that don't need consensus on every transaction across all parties, it's not the right tool for the job either.

The solution for this problem is a centralized ledger that serves as a transparent, immutable, and cryptographically verifiable transaction log. That's why we created the first-ever, purpose-built ledger database—Amazon QLDB.

Amazon QLDB makes it easy to understand how application data has changed over time, eliminating the need to build complicated audit functionality within the application. An Amazon QLDB journal is an immutable log where transactions are appended as blocks of data. After a transaction gets written as a block into the journal, it cannot be changed or deleted—it becomes a permanent record.

These blocks are also hash-chained together using cryptography (SHA-256). This allows you to verify and show the proof of your data's integrity (for example, an ecommerce business may have to show proof of a winning bid). This transaction then gets updated in the Current State table, which always keeps the current value of the data. And, finally, the transaction gets indexed in the history table, which can be queried to track how the data has changed over time.

Additionally, because Amazon QLDB is a database, it provides better performance and scale than blockchain frameworks. It can easily scale up and execute two to three times as many transactions as common blockchain frameworks. Amazon QLDB is now available in limited preview.

Making modern ledgers available to everyone

Blockchain and ledger technology has the potential to dramatically improve many types of transactions. However, if it isn't accessible to everyone, many great ideas may never be realized.

With Managed Blockchain and Amazon QLDB, anyone can create a ledger that is transparent, immutable, and cryptographically verifiable. There are many use cases for blockchain and ledger databases. Here are just a few examples.

Summary

These two solutions—Managed Blockchain and Amazon QLDB—expand the AWS portfolio of databases that offers the most breadth and depth of any cloud provider. By creating more options, we help more people. And when a company has the freedom to choose the best tool for a job, it makes a big difference.