Architecture Diagrams #


Overview Diagram #

TODO
  • cleanup / reorganize
    • this diagram is accurate, and helps lots to navigate, but it’s still a bit confusing
    • the arrows and lines make it a bit hard to follow. We should have a much cleaner version (maybe based on C4)
  • reflect addition of Token system
    • move data_transfers into Token
Filecoincluster_blockchain_subgroupcluster_blockchainBlockchaincluster_vmFilecoin VMcluster_messagepoolMessage Poolcluster_storagepowerconsensusStorage Power Consensuscluster_marketscluster_storagemarketStorage Marketcluster_retrievalmarketRetrieval Marketcluster_filesdatacluster_piecesData Transferscluster_filesLocal File Storagecluster_storageminerprocesscluster_storageminingStorage Miningcluster_sectorindexerSector Indexercluster_filproofs_subsystemStorage Provingcluster_systemscluster_repositoryRepositorycluster_filproofs_libFilecoin Proofscluster_paymentsPaymentscluster_netcluster_clockClockcluster_networkNetwork Interfacecluster_ipfsIPFScluster_libp2plibp2pChainChainBlock ProducerBlock ProducerVM InterpreterVM InterpreterChain->VM InterpreterIPLD DagStoreIPLD DagStoreChain->IPLD DagStoreMessage StorageMessage StorageBlock Producer->Message StorageBlock SyncerBlock SyncerGossipsubGossipsubBlock Syncer->GossipsubMessagesMessagesInit ActorInitActorAccount ActorAccountActorCron ActorCronActorMessage SyncerMessage SyncerFilecoin Network NodeFilecoin Network NodeMessage Syncer->GossipsubExpected ConsensusExpected ConsensusStorage Power ActorStoragePowerActorStorage ProviderStorageProviderStorage Provider->ChainData TransfersData TransfersStorage Provider->Data TransfersStorage ClientStorageClientStorage Deal ProtocolStorageDealProtocolStorage Market ActorStorageMarketActorRetrieval ProviderRetrievalProviderRetrieval Provider->ChainRetrieval Provider->Data TransfersRetrieval ClientRetrievalClientRetrieval Deal ProtocolRetrievalDealProtocolPieceStorePieceStoreData Transfers->PieceStoreData Transfers SchedulerData Transfers SchedulerAuthSwapdatatransferprotocol(unnamed)GraphsyncGraphsyncAuthSwap->GraphsyncFileStoreFileStoreDisk2Data Storage System(Disks, etc)FileStore->Disk2PieceStore->FileStoreMining SchedulerMining SchedulerSector BuilderSector BuilderMining Scheduler->Sector BuilderPledge ProverPledge ProverMining Scheduler->Pledge ProverStorage Miner ActorStorageMinerActorSector StoreSector StoreSector Store->PieceStoreSeal ProverSeal ProverPoSt GeneratorPoSt GeneratorPoSt Generator->ChainPledge Prover->Storage Power ActorStacked-DRG-PorepStacked-DRG-PorepPledge Prover->Stacked-DRG-PorepConfigConfigLocal StorageLocal Storage(Disk)Config->Local StorageKeyStoreKeyStoreKeyStore->Local StorageIPLD DagStore->Local StorageRational PoStRational PoStPiece Inclusion ProofsPieceInclusionProofsVRFsVRFsVDFsVDFsWalletWalletWallet->KeyStorePayment Channel ActorPaymentChannelActorWall ClockWall ClockNTPNTPWall Clock->NTPSwitchSwitchNTP->Switchp2p.Nodep2p.NodeGraphsync->p2p.NodeBitswapBitswapBitswap->p2p.Nodep2p.Node->SwitchSwitch->KeyStoreConnConnSwitch->ConnIdentifyIdentifyIdentify->p2p.NodeGossipsub->p2p.NodeBootstrap ListBootstrapListBootstrap List->p2p.Node
Protocol Overview Diagram #

Protocol Flow Diagram #

RetrievalClientRetrievalProviderStorageMarketParticipant1StorageMarketParticipant2StorageMarketActorPaymentChannelActorPaymentSubsystemBlockchainSubsystemBlockSyncerBlockProducerStoragePowerConsensusSubsystemStoragePowerActorStorageMiningSubsystemStorageMinerActorSectorIndexerSubsystemStorageProvingSubsystemFilecoinProofsSubsystemClockSubsystemlibp2pStorageMarketParticipantSectorSealerWallClockBlockchainMessagePoolMiningSchedulerStorageMinerSubsystemRetrievalMarketSubsystemStorageMarketSubsystemBlockchainGroupMiningGroupNewRetrievalQuery(RetreivalQuery)RetrievalQueryResponseNewRetrievalDealProposal(RetrievalDealProposal)AcceptRetrievalDealProposal(RetrievalDealPropsal)NewPaymentChannelopt[ RetrievalMarket ]CreateMiner(ownerPubKey PubKey, workerPubKey PubKey, pledgeAmt TokenAmount)CreateStorageMiner(OwnerAddr, WorkerPubKey)StorageMinerActoropt[ RegisterStorageMiner ]RegisterParticipant(TokenAmount)AddBalance(TokenAmount)WithdrawBalance(TokenAmount)CheckLockedBalance(Address)TokenAmountNewStorageDealProposal(StorageDealProposal)VerifyBalance(StorageMarketParticipant1)SignStorageDealProposal(StorageDealProposal)NewStorageDeal(StorageDeal)PublishStorageDeal(StorageDeal)PublishStorageDeal(StorageDeal)opt[ StorageDealMake ]HandleStorageDeal(Deal)AddDealToSector(Deal, SectorID)IndexSectorByDealExpiration(SectorID, Deal)(SectorID, Deal)NotifyStorageDealStaged(Deal,SectorID)opt[ AddingDealToSector ]QueryStorageDealProposalStatus(StorageDealProposalQuery)StorageDealProposalQueryResponseQueryStorageDealStatus(StorageDealQuery)StorageDealQueryResponseopt[ ClientQuery ]GetSealSeed(Chain, Epoch)SeedSealSector(Seed, SectorID, ReplicaCfg)Seal(Seed, SectorID, ReplicaCfg)SealOutputsSealOutputsCommitSector(Seed, SectorID, SealCommitment, SealProof, [&Deal], [Deal])VerifySeal(SectorID, OnSectorInfo){1,0}AddDeal(SectorID, [&Deal], DealStatusOnChain)AddDeal(SectorID, [Deal], DealStatusPending)IncrementPower(StorageMiner.WorkerPubKey)CommitSectorErroralt[ 1 - success ][ 0 - failure ]opt[ CommitSector ]opt[ SealingSector ]Subscribe(OnNewBlock)Event(OnNewBlock, block)ValidateBlockSyntax(block)HandleBlock(block)ValidateBlock(block)ValidateBlock(block)ValidateBlock(block)StateTree ← TryGenerateStateTree(block)AssembleTipsets()[Tipset] ← AssembleTipsets()Tipset ← ChooseTipset([Tipset])ApplyStateTree(StateTree)opt[ Round Cutoff ]loop[ BlockReception ]OnNewTipset(Chain, Epoch)OnNewRound()epoch is incremented by 1alt[ New Tipset ][ Null block last round ]GetElectionArtifacts(Chain, Epoch)TK ← TicketAtEpoch(Chain, Epoch-k)T1 ← TicketAtEpoch(Chain, Epoch-1)TK, T1EP ← DrawElectionProof(TK.randomness(), StorageMiner.WorkerKey)T0 ← GenerateNextTicket(T1.randomness(), StorageMiner.WorkerKey)T1 ← GenerateNextTicket(T0.randomness(), StorageMiner.WorkerKey)Using tickets derived in failed election proof in last epochalt[ New Tipset ][ Null block last round ]TryLeaderElection(EP){1, 0}GetPoStChallenge(Chain, Epoch)challengeGeneratePoSt(challenge, [SectorID])PoStProofSubmitPoSt(PoStProof, DoneSet)VerifyPoSt(PoStProof){1,0}UpdateDoneSet()HandleStorageDealPayment()CloseExpiredStorageDeal()PoStErrorSlashStorageCollateral()alt[ 1 - success ][ 0 - failure ]opt[ SubmitPoSt ]GenerateBlock(EP, T0, Tipset, StorageMiner.Address)GetMostProfitableMessages(StorageMiner.Address)[Message]block ← AssembleBlock([Message], Tipset, EP, T0, StorageMiner.Address)PropagateBlock(block)opt[ 1- success ]loop[ forEach StorageMiningSubsystem.StorageMiner ]loop[ BlockProduction ]OnNewTipset(Chain, Epoch)[SectorID] ← LookupSectorByDealExpiry(Epoch)PurgeSectorsWithNoLiveDeals([SectorID])HandleExpiredDeals([SectorID])CloseExpiredStorageDeal(StorageDeal)opt[ Expired deals ]Schedule and resume PoStsSchedule and resume SEALsMaintain FaultSetMaintain DoneSetopt[ MiningScheduler ]PoSt challenge if a node has not mined a block in a ProvingPeriodGetPoStChallenge(Chain, Epoch)challengeGeneratePoSt(challenge, [SectorID])PoStProofSubmitPoSt(PoStProof, DoneSet)VerifyPoSt(PoStProof){1,0}UpdateDoneSet()HandleStorageDealPayment()CloseExpiredStorageDeal()PoStErrorSlashStorageCollateral()alt[ 1 - success ][ 0 - failure ]opt[ SubmitPoSt ]opt[ ClockSubsystem ]UpdateSectorStatus([FaultSet], SectorStateSets)RecomputeMinerPower()SlashStorageCollateral()SectorStateSets := (FaultSet, RecoverSet, ExpireSet)opt[ Declaration before receiving a challenge ]ProvingPeriodUpdate()computeProvingPeriodEndSectorState()FaultSet is all sectors if no post submittedsectors Faulted longer than threshold proving periods are destroyedUpdateSectorStatus(newSectorState)RecomputeMinerPower()HandleFailedDeals([newSectorState.DestroyedSet])opt[ if miner ProvingPeriod ends ]loop[ forEach StorageMinerActor in StoragePowerActor.Miners ]loop[ EveryBlock ]opt[ Storage Fault ]DeclareConsensusFault(ConsensusFaultProof)ValidateFault(ConsensusFaultProof)TerminateMiner(Address)SlashPledgeCollateral(Address)UpdateBalance(Reward)opt[ Valid Fault ]opt[ Consensus Fault ]RetrievalClientRetrievalProviderStorageMarketParticipant1StorageMarketParticipant2StorageMarketActorPaymentChannelActorPaymentSubsystemBlockchainSubsystemBlockSyncerBlockProducerStoragePowerConsensusSubsystemStoragePowerActorStorageMiningSubsystemStorageMinerActorSectorIndexerSubsystemStorageProvingSubsystemFilecoinProofsSubsystemClockSubsystemlibp2pStorageMarketParticipantSectorSealerWallClockBlockchainMessagePoolMiningSchedulerStorageMinerSubsystem
Deals on Chain #

Parameter Calculation Dependency Graph #

This is a diagram of the model for parameter calculation. This is made with orient, our tool for modeling and solving for constraints.

%369ACTORS-MESSAGES-FRACTION68ACTORS-MESSAGES-PER-BLOCK68->6962MESSAGES62->6962->6866TX-MESSAGES-PER-BLOCK62->6664RECEIPTS62->6463MESSAGES-SIZE62->6367TX-MESSAGES-FRACTION67->6666->6265RECEIPTS-SIZE64->6561PROOF-MESSAGES-FRACTION61->6260AVG-PROOFS-MESSAGES-PER-BLOCK60->6259AVG-POSTS-MESSAGES-PER-BLOCK59->6058AVG-SEALS-MESSAGES-PER-BLOCK58->6056ALL-POST-MESSAGES-PER-YEAR56->5957MINERS57->5655ALL-SEAL-SIZE-PER-YEAR54ALL-SEAL-MESSAGES-PER-YEAR54->5854->5553SECTORS-COUNT53->5653->5452STORAGE-NETWORK-CAPACITY52->5348PROVING-PERIOD-SECONDS51POSTS-PER-SECTOR-PER-YEAR48->5151->5650SEALS-PER-SECTOR-PER-YEAR50->5449RESEAL49->5047PROVING-PERIOD-HOURS47->4846MAX-TICKETS45ONE-BLOCK-IN-TEN-YEARS45->4644BLOCKS-IN-TEN-YEARS44->4541BLOCKS-IN-A-YEAR41->5941->5841->4443EXPECTED-WINNING-MINERS43->4142YEAR-IN-SECONDS42->5142->4140TICKETS39AVG-TICKETS39->4036POST-CHALLENGE-TIME38POST-CHALLENGE-BLOCKS36->3837BLOCK-TIME37->4137->3835POST-CHALLENGE-HOURS35->3633ENCODING-TIME34KDF-HASH-SIZE34->3332KDF-CONTENT32->3331NODES30NODE-SIZE30->3330->3129SECTOR-SIZE-GIB28SECTOR-SIZE29->2828->5328->3127DEGREE-EXPANDER26DEGREE27->2626->3225DEGREE-BASE25->2623MESSAGE-RECEIPT23->6524GAS-USED24->2322RETURN22->2321EXIT-CODE21->2320MESSAGE-RECEIPTS-CID12MESSAGE-SIZE12->6319ACTOR-METHOD19->1218GAS-LIMIT18->1217GAS-PRICE17->1216VALUE16->1215U6415->2415->1915->1815->1715->1614MESSAGE-NONCE14->1213VARINT13->2213->2113->1411FROM-ADDRESS11->1210TO-ADDRESS10->129MESSAGES-ROOT-CID8ADDRESS-SIZE8->118->107CID-SIZE7->207->97->86EIX6->525PIB5->64TIB4->53GIB3->283->42MIB2->31KIB1->2
Protocol Overview Diagram #