Blog Posts

When thinking atrophies

Are you using GPT to think for you?  In the past, humans would think and machines would do the task. Now machines are thinking and humans are doing the task. Are we gladly giving away a part of what makes us human for a little bit of convenience or time saved?

Continue Reading

Adding a custom filter to Airflow logs

Recently I tried to add a custom logger for our Airflow deployment in kubernetes. Since we allow our logs to be shown to our customers we cannot have our secrets and other kubernetes config be shown to the users. On pod failure we get a dump of the entire pod config and that is what we want to remove today Note: Our logs are sent to S3 and remote logging is enabled for our version of airflow 2.5.0, so whatever

Continue Reading

Clubbing multithreaded logs together in Python

At my current company we rely heavily on Airflow for job orchestration and scheduling. From time to time we need to show these airflow task logs to our customers. The problem arises when the same task has multiple threads, each thread throws up its logs whenever it can which causes the logs to go into a frenzy. If there are more than 20-30 threads then the logs look like – With that established, in this blog post we will look

Continue Reading

Lock your (postgres) tables from being altered

red and yellow stop sticker

Want to lock your table but only on DDL operations? Use the below code to ensure no one is altering your tables. To achieve this we will create a trigger on all alter table commands, then filter it down by table name and finally throw an exception telling the user that they cannot alter the table.

Continue Reading

Site Footer

Sliding Sidebar

About Me

About Me

Hey, I am Thomas Ashish Cherian. What I cannot create, I do not understand.

Social Profiles