Skip to content

Instantly share code, notes, and snippets.

@leostera

leostera/download_day.py

Last active December 7, 2025 20:58
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save leostera/07a75ddb6772f18777cf04b4222f2493 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save leostera/07a75ddb6772f18777cf04b4222f2493 to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. leostera revised this gist Dec 7, 2025. 1 changed file with 1 addition and 7 deletions.
    8 changes: 1 addition & 7 deletions download_day.py
    View file
    Original file line number Diff line number Diff line change
    @@ -8,16 +8,10 @@
    output_dir = Path(day)
    output_dir.mkdir(exist_ok=True)


    def download(file):
    data = requests.get(file["url"]).content
    (output_dir / file["name"]).write_bytes(data)
    print(f"✓ {file['name']} ({len(data) / 1e6:.1f} MB)")


    with ThreadPoolExecutor(max_workers=10) as executor:
    executor.map(download, index["files"])

    print(
    f"Downloaded {index['file_count']} files ({index['total_size_bytes'] / 1e9:.1f} GB)"
    )
    executor.map(download, index["files"])
  2. leostera created this gist Dec 7, 2025.
    23 changes: 23 additions & 0 deletions download_day.py
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,23 @@
    import requests
    from pathlib import Path
    from concurrent.futures import ThreadPoolExecutor

    day = "2025-11-01"
    index = requests.get(f"https://data.solarchive.org/txs/{day}/index.json").json()

    output_dir = Path(day)
    output_dir.mkdir(exist_ok=True)


    def download(file):
    data = requests.get(file["url"]).content
    (output_dir / file["name"]).write_bytes(data)
    print(f"✓ {file['name']} ({len(data) / 1e6:.1f} MB)")


    with ThreadPoolExecutor(max_workers=10) as executor:
    executor.map(download, index["files"])

    print(
    f"Downloaded {index['file_count']} files ({index['total_size_bytes'] / 1e9:.1f} GB)"
    )