Skip to content

Instantly share code, notes, and snippets.

@chezou

chezou/as_msgpack_stream.R

Last active October 26, 2019 13:05
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save chezou/27a975565fb83afb78c33cb1af2ff6a0 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save chezou/27a975565fb83afb78c33cb1af2ff6a0 to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. chezou revised this gist Oct 26, 2019. 1 changed file with 12 additions and 0 deletions.
    12 changes: 12 additions & 0 deletions read_msgpack_stream.py
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,12 @@
    import gzip

    import msgpack

    with gzip.GzipFile("test.msgpack.gz", "rb") as f:
    unpacker = msgpack.Unpacker(f, raw=False)
    for u in unpacker:
    print(u)

    # {'a': 1, 'b': 4, 'time': 1572094906}
    # {'a': 2, 'b': 5, 'time': 1572094906}
    # {'a': 3, 'b': 6, 'time': 1572094906}
  2. chezou created this gist Oct 26, 2019.
    10 changes: 10 additions & 0 deletions as_msgpack_stream.R
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,10 @@
    devtools::install_github("crowding/msgpack-r")

    library(msgpack)

    x <- data.frame(a=1:3, b=4:6)

    conn <- gzfile("test.msgpack.gz", open="w+b")
    apply(x, 1, function(x) {writeMsg(c(x, time=as.integer(Sys.time())), conn)})
    flush(conn)
    close(conn)