Serverless functions using Netlify

#Netlify #NetlifyHackathon

A serverless api to publish my media assets to cloudinary media storage is built part of the this project

Here i am using Netlify functions to demonstrate the capability.

Why Netlify for this project? 🌟

Netlify is an all-in-one platform for automating modern web projects. Replace your hosting infrastructure, continuous integration, and deployment pipeline with a single workflow.

Integrate dynamic functionality like serverless functions, user authentication, and form handling as your projects grow.

Millions of developers use Netlify to ship faster, simplify their workflows, and scale effortlessly.

Why publishing to cloudinary? ☁️

Besides acting us a bucket to store media assets on cloud, cloudinary also helps by transform images and videos to load faster with no visual degradation, automatically generate image and video variants and deliver high quality, responsive experience to increase conversions for our websites. Also it also supports AI and ML on the analysis over the media resource with in house and other well known solutions in it's market place. Besides that is also offers its web console to manage the media artefacts.

This can also be very much helpful for amateur project developers to have a scalable asset storage, transformation, content delivery solutions on top of their solutions.

You may try the below preview link to understand more on this 🎥 cloudinary-res.cloudinary.com/video/upload/..

ts-cloud-sync 🌱

I have only built this as potential capability which i very much require to have scale my amateur blogging solutions without any bottleneck, hopefully in future and yes would make this util open source once fully ready. Initially I built this as a express api which uses multer and streamifier to my publish my media to cloudinary. Then i decided to leverage netlify functions to help automate my ci/cd stack and also have a bridge api available always on demand like lamdba and going serverless. But quickly i realised that deploying to netlify or may be other solutions raise some warning in using multer since it has deprecated buffer feature, which causing a stopper in our working in the last hours of deployment. Hopefully multer will release it v2 soon with stream handling support. I refactored this into by completely replacing multipart upload feature into streaming upload endpoint, which gave significant advantages in terms of the speed of upload and the memory involved in every invocation. Because since i pipe the binary streaming data from client end directly into streamifier to further pipe to cloudinary v2 upload_stream function. Eventually i was able to achieve what i have planned for this day long project conceived to participate and share my solution in utilising netlify.

Don't try the endpoint yourself 🚧

since netlify function endpoint will be validating my authorization key only then cloudinary keys will be supplied to further publish this.
The cloudinary asset links are secured and signed based on the access level i may have set for the resource in my presets while i upload.

Netlify function lambda logs :running:

Mar 1, 12:06:38 PM: 7d9b959c INFO   /.netlify/functions/index/stream/NetlifyHackathonPoster
Mar 1, 12:06:38 PM: 7d9b959c INFO   Authorization Verified for the resource:  /.netlify/functions/index/stream/NetlifyHackathonPoster
Mar 1, 12:06:38 PM: 7d9b959c INFO   {
  asset_id: '8d7ba9d8201a69febf3f24dd49c29b24',
  mimetype: 'image/jpg',
  asset_size: '124.63 KB',
  public_id: 'private/NetlifyHackathonPoster',
  secure_url: 'https://res.cloudinary.com/dev-post/image/authenticated/s--RDz5Tfma--/v1646116598/private/NetlifyHackathonPoster.jpg',
  Overwritten: false,
  created_at: '2022-03-01T06:36:38Z',
  etag: '844e3ae734b21a11a07192458f1f803e',
  signature: '34139038bbd548fe24ed05fdf8b42d212c9e0426'
}
Mar 1, 12:06:38 PM: 7d9b959c Duration: 471.71 ms    Memory Usage: 73 MB

When I re-upload the same image this will be overwritten so that my resource name is going to be the same.

Mar 1, 12:07:14 PM: 6541e1ec INFO   /.netlify/functions/index/stream/NetlifyHackathonPoster
Mar 1, 12:07:14 PM: 6541e1ec INFO   Authorization Verified for the resource:  /.netlify/functions/index/stream/NetlifyHackathonPoster
Mar 1, 12:07:14 PM: 6541e1ec INFO   {
  asset_id: '8d7ba9d8201a69febf3f24dd49c29b24',
  mimetype: 'image/jpg',
  asset_size: '124.63 KB',
  public_id: 'private/NetlifyHackathonPoster',
  secure_url: 'https://res.cloudinary.com/dev-post/image/authenticated/s--RDz5Tfma--/v1646116598/private/NetlifyHackathonPoster.jpg',
  Overwritten: true,
  created_at: '2022-03-01T06:36:38Z',
  etag: '844e3ae734b21a11a07192458f1f803e',
  signature: '34139038bbd548fe24ed05fdf8b42d212c9e0426'
}
Mar 1, 12:07:14 PM: 6541e1ec Duration: 441.02 ms    Memory Usage: 76 MB

As a amateur tech blogger i feel this capability to stream uploads to my custom serverless function could be further vertically integrated to automate my publishing process and manage the data at scale and flexibily and securely without relying on other thirdparty hacks.

How I read the file stream inside the function 🚡


let file_id: string = publicName(req.params.asset_id);
    async function upload(req: any) {
      let result: any = await streamUpload(file_id, req);
      let logJSON:any = {
        "asset_id": req.params.asset_id,
        "mimetype": `${result.resource_type}/${result.format}`,
        "asset_size": formatBytes(result.bytes),
        "public_id": result.public_id
      }
      if (result.public_id?.length > 0) {
        logJSON = {
          ...logJSON,
          "secure_url": result.secure_url,
          "Overwritten": result.overwritten != undefined ? result.overwritten : false,
          "created_at": result.created_at,
          "etag": result.etag,
          "signature": result.signature,
          "asset_id": result.asset_id,
        }
        console.log(logJSON);
        res.status(200);
        res.json(result);
      } else {
        res.status(400);
        res.json(`{ "error": "Unknown error"}`);
      }
    }

Also find my writeable cloudinary v2 stream created 🚠


export const stream_upload = (public_id: string, resolve: any, reject: any) => {
  let stream = cloudinary.v2.uploader.upload_stream(
    {...stream_upload_options,
        public_id: public_id,
    }
    ,
    (error: any, result: any) => {
      if (result) {
        resolve(result);
      } else {
        reject(error);
      }
    }
  );
  return stream;
};

Uploaded screenshots

Uploaded image below

Thanks for supporting! 🙏

Would be really great if you like to ☕ Buy Me a Coffee, to help boost my efforts.

🔁 reposted at 🔗 dev to @aravindvcyber

❄️ Serverless functions Netlify 🚀