Warp-CLI is tool designed to make interacting with Facebook’s Warp Speed Data Transfer (WDT) pain-free.
https://github.com/JustinTimperio/warp-cli This tool is still under development. Please feel free to submit any bugs or feature requests.
Abstract WDT is designed to provide the highest possible speed when transferring files(to be only hardware and network limited). WDT provides many advantages over most file transfer protocols including: native concurrency, end-to-end encryption, IPV6 support, and the ability to easily achieve +40Gbit speeds when supported.
This script is designed to download, convert, and sort GDELT source files automatically into a user-specified path. https://www.gdeltproject.org/data.html#rawdatafiles
*Note: This script is in BETA and may have bugs. https://github.com/JustinTimperio/gdelt-diff
Description To maintain consistency across my personal datasets, these files are converted upon download from .zip to .gz. Due to the large size of the GDELT source files extra care is taken to ensure that as much file processing as possible is done in ram.