Recently I tried to add a custom logger for our Airflow deployment in kubernetes. Since we allow our logs to be shown to our customers we cannot have our secrets and other kubernetes config be shown to the users. On pod failure we get a dump of the entire pod config and that is what we want to remove today Note: Our logs are sent to S3 and remote logging is enabled for our version of airflow 2.5.0, so whatever …
Category: data science
Henceforth I will be using the bluescreen blog to document the things I do and experiment with. One of the recent things that got me interested is datacamp. What is datacamp? Datacamp is a platform to learn Python, R and Data Science. The good thing about datacamp is that it provides most of the courses for free. Apart from that it also provides a certificate at the end of the course completion. Course The first course I started at datacamp …
Social Profiles