Skip to content

Instantly share code, notes, and snippets.

@hiAndrewQuinn

hiAndrewQuinn/download_tweets.sh

Last active December 6, 2024 14:34
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save hiAndrewQuinn/e1776d1e524ab424f9489a0454408b9e to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save hiAndrewQuinn/e1776d1e524ab424f9489a0454408b9e to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. hiAndrewQuinn revised this gist Dec 6, 2024. 1 changed file with 5 additions and 1 deletion.
    6 changes: 5 additions & 1 deletion download_tweets.sh
    View file
    Original file line number Diff line number Diff line change
    @@ -1,10 +1,14 @@
    #!/usr/bin/env bash

    # List of usernames
    # List of usernames. MUST BE ALL LOWERCASE!
    users=(
    visakanv
    eigenrobot
    patio11
    daniellefong
    thezvi
    richdecibels
    qiaochuyuan
    )

    # ... Add anyone else you may want! I just grabbed the most followed accounts currently there.
  2. hiAndrewQuinn revised this gist Dec 6, 2024. 1 changed file with 6 additions and 9 deletions.
    15 changes: 6 additions & 9 deletions download_tweets.sh
    View file
    Original file line number Diff line number Diff line change
    @@ -2,19 +2,16 @@

    # List of usernames
    users=(
    patio11
    eshear
    visakanv
    eigenrobot
    DanielleFong
    m1guelpf
    QiaochuYuan
    RichDecibels
    TheZvi
    ) # ... Add anyone else you may want! I just grabbed the most followed accounts currently there.
    patio11
    )

    # ... Add anyone else you may want! I just grabbed the most followed accounts currently there.
    # N.B.: Some accounts will just return "Object Not Found", not sure why.

    # Use GNU parallel to download files in parallel.
    # {} will be replaced with each username.
    parallel -j4 curl -sS -o "tweets_{}.json" "https://fabxmporizzqflnftavs.supabase.co/storage/v1/object/public/archives/{}/archive.json" ::: "${users[@]}"

    # Output: tweets_patio11.json, tweets_eshear.json, ..., tweets_TheZvi.json.
    # Output: tweets_patio11.json, tweets_eshear.json, ..., tweets_TheZvi.json.
  3. hiAndrewQuinn created this gist Dec 6, 2024.
    20 changes: 20 additions & 0 deletions download_tweets.sh
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,20 @@
    #!/usr/bin/env bash

    # List of usernames
    users=(
    patio11
    eshear
    visakanv
    eigenrobot
    DanielleFong
    m1guelpf
    QiaochuYuan
    RichDecibels
    TheZvi
    ) # ... Add anyone else you may want! I just grabbed the most followed accounts currently there.

    # Use GNU parallel to download files in parallel.
    # {} will be replaced with each username.
    parallel -j4 curl -sS -o "tweets_{}.json" "https://fabxmporizzqflnftavs.supabase.co/storage/v1/object/public/archives/{}/archive.json" ::: "${users[@]}"

    # Output: tweets_patio11.json, tweets_eshear.json, ..., tweets_TheZvi.json.