Dev Release 27

Greetings BlockDAG Community,

As mentioned in previous post, the next step is to start the syncing process between the nodes. Today's post is all about doing the syncing implementation in the BlockDAG project. 
Here's an introduction to what's the purpose of syncing is in our implementation.
Syncing algorithm is designed to make sure that the particular chain is in the latest state and synced with the rest of the network by downloaded and importing new data as soon as possible. The node then steps in, either through the one of the initial synchronization methods mentioned above, and then the keep-up sync is used to download new blocks. These two methods satisfy the requirements for a network to produce a cryptocurrency.
Syncing substreams involves the process of ensuring that a node's local representation of the BlockDAG is updated and aligned with the network's state, particularly focusing on specific substreams of interest to that node. This synchronization process is fundamental for maintaining consensus and coherence across a decentralized network where different nodes may have varying views of the DAG due to network latency and propagation delays.

Now let's understand few types of syncing processes that can be used in the implementation process:
 

1. Full and light sync: By default, this is auto-sync option for the first and follow-up patterns. This algorithm starts with the current block best and downloads progressively block data from several peers in the multiple peer-to-peer streams if available. This is done so that the request will hold there till the time when they are available. Heavy nodes will fully download and execute the whole blocks while light-weighted nodes import block headers. This will be repeated repeatedly until every block is investigated from the other student until everyone has no new block to reveal. In every sync, the node has to keep the number of the block both nodes have by every peer. When peer share with us a new block or we discover new best block. Thanks to this technique when known citizen suddenly announces new information peers that are interested in it can easily react and request it. In addition to regular monitoring of the best-case blocks, we mark all the announced peer children in all branches. The synch algorithm wants to consume as much data promised by freedom modes as possible.
    

Steps to implement the full sync process:

• Initial Block Download (IBD):
• We'll start by connecting to the network and downloading the entire blockchain starting from the genesis block.
• Fetch each block sequentially from network peers and store them locally.
• Transaction Verification:
• Validate each block and its transactions against the consensus rules (e.g., Proof of Work for Bitcoin, Proof of Stake for Ethereum) to ensure integrity.
• State Update:
• Process each transaction within the blocks to update the node's local state (e.g., account balances, contract states).
• Compute and store the state after processing each block.
• Continued Block Retrieval:
• Continue downloading blocks and validating transactions until reaching the current tip of the blockchain.
• Final Verification:
• Validate the entire downloaded blockchain up to the latest block to confirm consistency and correctness.
   

Steps to implement the light sync process:

• Initial Block Header Download:
• Connect to the network and retrieve block headers (containing essential block information but not full transactions) from network peers.
• Checkpoint Block Retrieval:
• Identify specific checkpoint blocks at regular intervals (e.g., every 1,000 blocks) in the blockchain's history.
• Request full blocks for these checkpoints from network peers to ensure data integrity.
• State Trie Pruning:
• Prune unnecessary historical state trie data to reduce storage requirements.
• Discard transaction details not needed for current state verification.
• Fast Sync State Retrieval:
• Download recent state trie data (e.g., current account balances, contract states) from network peers without processing all historical transactions.
• Validation and Verification:
• Validate received checkpoint blocks and state trie data against consensus rules and cryptographic proofs.
• Ensure the integrity and correctness of the blockchain up to the latest known state.

Fast sync: The download mode is the instant one of in this mode downloads the full header history. The process of transformation of authority set operations and comes up with a most recent header is possible by that. The node checks the header chain throughput spot and saves these records. Then importing the state, a short process, starts. Each StateResponse provides a storage key, which is void for the first one. StateResponse comprises proof of storage location thus indicating where the key and all the other keys starts (but not including the key that is in the request). After running the proof trie in the storage root that is currently on target header, the node sends the next request element of StateResponse with key range that will start from that last key from the previous response. This process repeatedly carries out until the shift in trie reaches its last point. The state is then shipped to the database and the normal full/lite sync starts in your way.

Steps to implement the fast sync process:

• Initialization:
• Initialize the node's database and networking components.
• Connect to peers in the BlockDAG network.
• Download Recent Block Headers:
• Initiate the synchronization process by requesting recent block headers from network peers.
• Start with the genesis block and progressively download block headers up to the current tip of the BlockDAG.
• Select Checkpoint Blocks:
• Identify specific checkpoint blocks at regular intervals in the BlockDAG's history (e.g., every 1,000 blocks).
• Request full blocks for these checkpoints from network peers to establish synchronization points.
• Prune Unnecessary Data:
• Prune unnecessary historical data from the node's local database to optimize storage usage.
• Discard transaction details and state trie data not required for current state validation.
• Retrieve Recent State Data:
• Download recent state trie data (e.g., account balances, contract states) corresponding to the latest checkpoint blocks.
• Fetch state data from network peers using efficient retrieval methods (e.g., Merkle proofs).
• Validate State Data:
• Validate the received state trie data against cryptographic proofs and consensus rules to ensure integrity.
• Verify the correctness and consistency of account states and other critical data.
• Finalize Synchronization:
• Complete the synchronization process by confirming that the node's local state matches the state derived from the received data.
• Ensure that the node has synchronized up to the latest state of the BlockDAG network.
• Enable Operational Mode:
• Transition the node into an operational mode, where it can participate in network activities such as transaction processing and block validation.

Warp sync: BFT sync also known as warp sync is a brand new technique utilized by blockchains such as Ethereum to speed up the process of logging a new node or syncing it with the latest state on the chain. Unlike the full synchronizing method of traditional thinking about every network node finishing downloading and verifying each block and transaction starting with the very first one, warp sync attempts to speed up the whole process. The process is started by the Initial Block Download (IBD) phase and it involves retrieving recent block headers from network peers. Contrary to the download of all historical raw data, the node regulates the checkpointing of those blocks in intervals whereby it only acquires specific checkpoint blocks—the historical blocks ranging at intervals. g. , they do mining every 10 minutes on the average (10 minutes is equivalent to about 100,000 blocks, 1 block in about every 10 minutes)—to form the bridge between the current state and the future one. These checkpoint points comprise a core element of the blockchain which have 'data sets' been verified by that particular points. Having been granted the checkpoint block nodes shave off the historical state trie's redundant information to ease on storage. Latterly "fast sync" mode is enabled in the the node where is it synchronize the recent state trie data (e. g. We will inherit the ability to validate and transfer funds, settle accounts, and get the state of contract without having so to much work on historical transactions.

Steps to implement the wrap sync process:

• Initial Block Download (IBD) Phase:
• The warp sync process begins with an Initial Block Download (IBD) phase where the node starts by downloading recent block headers from peers on the network.
• Instead of downloading and verifying each block and transaction sequentially from the genesis block, the node focuses on retrieving the most recent block headers.
• Checkpoint Synchronization:
• Once a sufficient number of block headers are downloaded, the node identifies and retrieves checkpoint blocks. Checkpoint blocks are specific historical blocks chosen at regular intervals (e.g., every 100,000 blocks) that serve as synchronization points.
• Checkpoint blocks are requested in full from network peers to ensure the integrity of the blockchain up to those points.
• State Trie Pruning and Fast Sync:
• After acquiring checkpoint blocks, the node prunes unnecessary historical state trie data, reducing storage requirements.
• The node then initiates a "fast sync" mode where it retrieves recent state trie data (account balances, contract states, etc.) from network peers without downloading all historical transactions.
• Validation and Verification:
• As the node receives checkpoint blocks and recent state data, it validates and verifies the integrity of the blockchain up to the latest known state.
• The node ensures that the received state data matches the consensus rules and cryptographic proofs of the blockchain.
• Finalization and Node Activation:
• Once the node has synchronized up to the latest state of the blockchain and validated the received data, it finalizes the warp sync process.
• The node transitions into an operational state, where it can participate in network activities like transaction processing, block validation, and smart contract execution.

After the successful implementation of the syncing progress we will shift to the implementation of mining progress. By this once the block is generated it will be sent out to other fellow miners to validate the produced blocks.