At my current company we rely heavily on Airflow for job orchestration and scheduling. From time to time we need to show these airflow task logs to our customers. The problem arises when the same task has multiple threads, each thread throws up its logs whenever it can which causes the logs to go into a frenzy. If there are more than 20-30 threads then the logs look like – With that established, in this blog post we will look …
Blog Posts
Want to lock your table but only on DDL operations? Use the below code to ensure no one is altering your tables. To achieve this we will create a trigger on all alter table commands, then filter it down by table name and finally throw an exception telling the user that they cannot alter the table. …
One of the tasks I recently came accross my job was ingest large files but with the following Do some processing ( like generate hash for each row ) Insert it into S3 for audit purposes Insert into postgres …
If your application allows your users to download files directly from s3, you are bound to get this error sometime in the future whenever you scale to other regions – The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256. …
Do you want need a quick solution without going into the hassle of setting up a Database? If your answer to any of those questions was a yes, then you’ve come to the right place. This post will show you how you can use Google sheets as your database. For the purposes of this blogpost I will be usiing this Google sheet. As you can see, we will be collecting the following data from the user – Name, Email …
Social Profiles