Recently I tried to add a custom logger for our Airflow deployment in kubernetes. Since we allow our logs to be shown to our customers we cannot have our secrets and other kubernetes config be shown to the users. On pod failure we get a dump of the entire pod config and that is what we want to remove today Note: Our logs are sent to S3 and remote logging is enabled for our version of airflow 2.5.0, so whatever …
Category: python
At my current company we rely heavily on Airflow for job orchestration and scheduling. From time to time we need to show these airflow task logs to our customers. The problem arises when the same task has multiple threads, each thread throws up its logs whenever it can which causes the logs to go into a frenzy. If there are more than 20-30 threads then the logs look like – With that established, in this blog post we will look …
Do you want to download your Emails to your local system? Does your email provider not allow you to download specific emails? Do you have a some knowledge of Python Scripting? If your answers to your above questions were “yes”, then you’ve come to the right place.If you want to get all your Emails data at one go, you can follow this video(for Gmail). For the purposes of this tutorial we will be using the imap protocol to access and …
One of the biggest problems bloggers, solopreneurs and startups face is to keep their users engaged. One of the best ways to do this is to send them useful mail.I have been using Mailchimp to send out mails for a little more than 6 years now. And I absolutely love and recommend their service. The free plan allows about 2000 subscribers. Fortunately(or unfortunately) my mailing lists will reach that number very soon. Once the number does exceed 2000 you have …
The second course i took at DataCamp was Intro to Python for Data Science. As you might know by now, the purpose of this website is to train you in data science. I recommend you to learn python first from- ThinkPython before taking up this course. Doing so will allow you to complete this course in under 30-40 mins. Anyways, the course in itself is sufficient for begineers. It takes you through the various data structures, fucntions and packages and …
Social Profiles