RFP: PB-level miner testing

Project Name: Large Miner Testing

Proposal category: mining

Proposer: @keren117

Deadline: April 15, 2020 23:59 PDT

Project Introduction

As the Filecoin testnet reaches petabyte scale and node implementations approach protocol completeness, we are interested in understanding how the Filecoin mining software performs on large-scale configurations.

If you are running a mining operation on the scale of 5 PB or larger, we are seeking advice on designing, implementing, and running tests of mining and storage hardware at scale.

Proposals can be emailed directly to [email protected], and the details of your action will be kept confidential.

The main objectives are to understand:

• Phase 2 v23 or v24 Proof Parameters PoRep Sealing Rate

• ePoSt Performance on 5 PB+ Sealed Data

• What other performance bottlenecks are there?

• Major events in mining

• How to improve mining software for large-scale environments

• Seal cost in US dollars (GB/h)

We are particularly interested in measuring sealing cost and performance per GB per hour, so we encourage suggestions that help us understand this metric. We also invite other suggestions for better configurations and other useful test plans.

Application Requirements

1. Verify your current capacity and throughput in the testnet storage power meter (e.g., a message signed using the method described in Lotus PR #1292)

2. General test plan description involving Lotus or go-filecoin

3. Description of the topology of the 5 PB+ mining operation under test (including geographic location)

Other test criteria that may be considered

The test plan may also optionally include some of the following benchmarks to benchmark your operations:

• Benchmarking Tools

• (Open source sourcing of any custom tools you use would be greatly appreciated and might make your proposal more competitive)

• Hardware Configuration

• Performance and usage over time

• Any cooling requirements and failure rates

• CPU, GPU, RAM, SSD and HDD, switch (if any)

• Performance-based BIOS changes (if any)

• Indicators:

• Network topology, bandwidth and throughput metrics

• Network topology description

• libp2p metrics, any relays, etc.

• Number of companions seen in a period of time

• Filecoin software configuration

• For example, block production, block propagation time, chain storage, etc.

• Proportional choice voting rights, e.g. 1-5 + PB

• Including any shared storage

• Sector Builder and Tested Sizes (We are particularly interested in 32GB sectors)

• Departmental loss monitoring

• daemon-miner-sealer configuration

• Chain checkpoints and backups

• Sealing and certification indicators

• Transaction status indicators

• Blockchain Indicators

• Fork inspection and related information

• Automated or integration testing

• Bootstrapping and data ingestion optimization

• Queuing strategy (if any)

• Cache policy (if any)

• Any software modifications or other logging tools made

• Support software configuration

• Redundancy strategy (if any)

• Workflow orchestration tools (if any)

• Network configuration and tools

• Storage layer management (we are particularly interested in this)

• For example, RAID, file systems, any software-defined storage, NAS, SAN

• Includes I/O per second metrics and HDD and SSD usage

• Your storage growth strategy

• (e.g. DDOS sentry, multi-node to multiaddr reverse proxy, etc.)

• Any other tools used

• Any other relevant metrics that might be useful

• For example, data center, location, comparison to any other network you might be running

• Finally, we are also interested in the following financial models:

• Mining costs, broken down by capital expenditure, electricity costs, data center costs, etc.

• Profitability assuming a fixed FIL price

• Drastic reduction

• The proportion of successful blocks on the main chain

• Any other information that may help us understand the financial issue

Based on the initial results, we may order additional tests.

Grant Amount

The purpose of this grant is to incentivize the sharing of testing knowledge and test results to benefit the mining community and understand how the protocol works at scale. We will maintain confidentiality of any proprietary or personal information, but aim to share common knowledge and results with the broader community.

We are unsure how many miners will apply for this grant, and the budget is uncapped, so once all proposals have been submitted and reviewed, a decision will be made on how to fairly distribute the funds.

Multiple teams may be accepted based on validation of the quality of the proposal and storage capacity. Please also include your geographic location of operation in your proposal.

The grant can be statutory, a future grant from FIL, or a hybrid grant.

Deliverables

Once the proposal is accepted, the following are the planned deliverables:

• A specific and detailed test plan involving Lotus or go-filecoin.

• Description of the mining operation topology under test.

• A summary of the test results, including all monitoring and metrics screenshots.

• A written record of your learning and recommendations.

Deliverables can be emailed to [email protected] or published in a Github repository.

Delivery requirements:

• Document a clear, understandable test plan and a summary of the results and recommendations.

• A list of all tools and configurations used with the document, with links to any custom tools

• Use current Filecoin nodes for large-scale hardware testing (lotus or go-filecoin if available)

Milestones and Funding

Milestones and Funding

If milestones cannot be met satisfactorily, we may not continue to fund your team for the project.

Have questions about this RFP? Email [email protected]