NodeJS upload files to AWS

Tram Ho

Introdution

  • This article I will write about how to upload files to Amazon S3.
  • S3 is AWS file storage service. Before using the service we need to create the Bucket (which will contain the files we upload).
  • FYI: When creating AWS account, some services will be free for 12 months: EC2, S3, RDS, CloudFront (remember to read carefully how free S3 wallet will be 5GB, 20k GET request and 2k PUT request – that’s all, than charging should be careful). However, some other services are always free (even after 12 months – should be spoiled): DynamoDB, Lamda ..

Before start

  • Before starting, we need to have an AWS account (if you do not have one, create one).
  • Create a new (if you do not have) an AWS Access Key .

Create Bucket

  • After having an account and Access Key we need to create a Bucket to test upload files.
  • There are several ways to create a Bucket:
    • You can create directly on the AWS page
    • Or you can create with AWS cli with the command: aws s3 mb s3://created-by-cli here created-by-cli is the name of the Bucket you want to set. ( mb turn off make bucket).
    • Created with the API
  • When creating Bucket, there are still many settings (try and see ? ) such as region, public, CORS .. I and ways 1 and 2 (created on AWS page, aws cli) should only perform tests to know. The rest should be created with API so the settings will be saved without having to remember.
  • Before we begin we set up the project a bit.
  • Create folders, create projects, initialize git with commands
  • Create config.json file with the content:
  • Create a makeBucket.js file (to coincide with the creation of the bucket command in aws-cli)
  • The current directory structure is ntn:
  • The makeBucket.js file will take configuration in config.json and create a new Bucket with the name created-by-api . Then adjust the bucket CORS settings with the editBucketCORS function. I tried setting up at the time of creating the bucket but it didn’t work. Search doc, there is only 1 edit phase after creating the bucket here ? .
  • Install the dependency package npm i aws-sdk , configure the key correctly, we run the file with: node makeBucket.js will have the following output as successfully created:
  • We can also check by web interface or cli, cli will have output like this:
  • Only run 1 makeBucket.js file, why configure the project, config things to do? Be familiar with how to manage the configuration files related to the Key of services like AWS or Google … one way is that you save to the config file and add that file to .gitignore. (If deployed on the cloud, these configuration files should be saved in encrypted areas and limited access so it is safer.) Then you can safely push your project to github or share the repo link to others.
  • File .gitignore ntn:

Upload the file

  • Bucket we have test file upload to S3
  • Depending on our service, we have different ways to upload files to S3.

Direct upload

  • First lesson: let’s say your server creates its own invoice - hóa đơn then we need to push the file directly to S3 and return the link to the customer to see the invoice.
  • Let the server directly push the file to S3. We create the file directUpload.js with the following content:
  • The directory now adds 2 files: directUpload.js and cat.jpg
  • Test run the program, delicious upload ok:

Upload with signedURL

  • Problem 2: Your application allows users to upload files directly to S3. At this time, it is not possible for users to upload to the server and then the server to upload to S3 again. Instead, the Client will call the server to get a signedURL and the client uses it to upload files to S3.
  • First on the server side we create a file: uploadWithSignedURL.js with the following content:
  • What does the server segment do? Initialize an api that receives a request with route sign-s3 and queries: file-name, file-type. Then request signedURL (with s3.getSignedUrl ) and return it to the client. That’s enough, the rest is for the client.
  • Here I will create an ugly-client index.html file with the following content:
  • Here, the folder has 2 new files:
  • Talk a little bit. The file only has 1 input tag for the user to select the file. Script will listen onchange to send the request to the server using getSignedRequest . Here I am fixing the local URL hard as you see: http://localhost:3000/sign-s3?file-name=${file.name}&file-type=${file.type} . After receiving the signedURL returned by the server, the client will upload the file to S3 using: uploadFile .
  • Ok, got it Test it out. First npm i express cors . Run the server first:
  • Then open index.html file with chrome and select cat.jpg file wait a while check on the console window of the browser to see the form as if the upload was successful (if failed any step will turn on popup):
  • Sure you can check back on the AWS site, or faster with aws-cli:
  • As you can see there are 2 files 1 file directUpload from server nodejs and 1 file uploaded from client side with signedURL.
  • Thank you for watching here! The full code in this article is here, so read through ReadMe.md if you can run it!
  • Happy using AWS service!
Share the news now

Source : Viblo