Skip to content

Instantly share code, notes, and snippets.

@gambitier

gambitier/s3download_promise.js

Forked from milesrichardson/s3download_promise.js
Created February 1, 2022 09:06
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save gambitier/df110550a9a057056d7019a0fb086e20 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save gambitier/df110550a9a057056d7019a0fb086e20 to your computer and use it in GitHub Desktop.
Download ZIP
S3 download promise: nodeJS promise to download file from amazon S3 to local destination
Raw
s3download_promise.js
const AWS = require('aws-sdk');
const fs = require('fs')
const s3download = (bucketName, keyName, localDest) => {
if (typeof localDest == 'undefined') {
localDest = keyName;
}
let params = {
Bucket: bucketName,
Key: keyName
}
let file = fs.createWriteStream(localDest)
return new Promise((resolve, reject) => {
s3.getObject(params).createReadStream()
.on('end', () => {
return resolve();
})
.on('error', (error) => {
return reject(error);
}).pipe(file)
});
};
@gambitier
Copy link
Copy Markdown
Author

gambitier commented Feb 1, 2022

one of comment

A late answer but the problem is what at first glance seems as only a semantic error.

The Promise resolves on the end event of the read stream. This event is triggered when no more data can be read from the read stream. Meaning that the end of the read stream has been reached.

However, this does not necessarily mean, that all the data has already been written to disk in the example.

So the Promise should actually only resolve() once the write stream sends a finish event.


const s3 = new AWS.S3();
const params = {
    Bucket: bucketName,
    Key: keyName
};
const readStream = s3.getObject(params).createReadStream();

// Error handling in read stream
readStream.on("error", (e) => {
    console.error(e);
    reject(e);
});

// Resolve only if we are done writing
writeStream.once('finish', () => {
    resolve(filename);
});

// pipe will automatically finish the write stream once done
readStream.pipe(writeStream);

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment