Valuable datasets stored via Filecoin Slingshot

Over the past two weeks, the Filecoin community has stored 170 TiB of valuable data through more than 200,000 storage transactions on the Filecoin network. This data was stored by participants in the Slingshot competition, a community competition for storage clients and developers to onboard valuable data to the Filecoin network.

Overall, we’ve seen teams create Dropbox-style storage apps, games, and UIs around all kinds of datasets (scientific, cultural, entertainment, software, and more)! Let’s take a look at some of the most interesting datasets we’ve seen so far in this competition.

Dataset: Fiber Optic Data

Data size: ~50-100 TiB

Smartcity is building a sensor-based network and data analytics system that uses data generated by previously laid fiber optic cables to monitor temperature, pressure, vibration, sound, and other information around the fibers in cities around the world. Researchers, municipal managers, and transportation facility supervisors can use this data to assist in urban planning and construction. It can also be used to build predictive monitoring systems and, in some cases, even help predict earthquakes. These cables act like a nervous system for the city. The Smartcity project is using this generated data to help cities run more efficiently.

Data from the fiber optic sensor network is stored on Filecoin and IPFS. Deep learning models will be trained on this data and will ultimately be used to analyze various signal types and perform pattern recognition to classify and predict events. The team created a web page to play the sounds collected from these sensors and a query interface to send these files from IPFS and Filecoin to a remote web server that performs the deep learning processing.

Dataset: Virtual Reality Data

Data size: ~50 TiB

Zangshell is a virtual reality (VR) and augmented reality (AR) system that has been collecting raw video from professional cameras, then processing and storing it. This data was originally used in the real estate and municipal engineering industries. Previously, this data has been stored in centralized data centers, but the team is migrating the data to the IPFS and Filecoin networks to save storage costs. The Zangshell team intends to migrate its storage and data processing pipelines directly onto IPFS and Filecoin.

Zangshell users will be able to watch these VR/AR images or videos directly in their web browsers. They can zoom in or out of the scene, and can also view the virtual reality scene from different angles by using a PC mouse or gestures on the mobile screen.

Dataset: Astronomical and meteorological data

Data size: ~200TiB

The Yunnan Starry Sky team is building a repository of astronomical and meteorological data, including charts, analysis, and photos. This data is generated, collected, and stored by Yunnan University in Kunming, China. The Yunnan Starry Sky team has recently worked with the university on several different projects and discussed storing this data on Filecoin. The release of Slingshot gave them a good reason to start this new project.

The team believes that this data will be useful to future generations and believes that Filecoin’s mission to store humanity’s most important information makes the Filecoin storage network a good storage solution for this data. They plan to onboard members from universities and other organizations that process astronomical and weather data as the first users.

In addition to datasets stored by Slingshot teams for specific use cases, Slingshot teams are encouraged to store a large number of other important and publicly available datasets. These datasets have the potential to provide important medical, scientific, and cultural information to users around the world. Some of these datasets include:

Space Data: The Sloan Digital Sky Survey (SDSS) has created a detailed 3D map of the universe, with deep multi-color images and spectra of more than three million celestial objects for one-third of the sky. The SDSS has been mapping the universe for more than 20 years and continues to add data. New discoveries have been made using this data, including the discovery of new types of quasars, the discovery of a new population of "ultra-faint" dwarf galaxies orbiting the Milky Way, and data on "hypervelocity stars" that are bound for the Milky Way's central black hole and are moving so fast that they will escape the Milky Way entirely.

COVID-19 Data: The U.S. government and a coalition of leading research groups have prepared the publicly available COVID-19 Open Research Dataset (CORD-19) – more than 200,000 scholarly articles on COVID-19, SARS-CoV-2, and related coronaviruses. This dataset will enable the global research community to gain new insights to support the fight against COVID-19 and prevent future pandemics.

Museum Data: To improve the quality of government data and its availability for educational, personal, and commercial use, the National Palace Museum in Taiwan established a public “Database Search” and “Image Download” resource repository in 2015 to make the museum’s images and research materials more accessible to the public. The data includes exhibition information as well as images of museum artifacts, including paintings, antiquities, costumes, inscriptions, and calligraphy.

Every day, our world generates billions of bytes of data. Much of this data has the potential to help answer humanity’s most critical questions. We believe the Filecoin network will play a major role in addressing these big questions by ensuring our species’ data is preserved and accessible for future generations. We’re excited to see Slingshot’s participants working together to help the network realize this potential.

——End——