Data Engineering
Migrating existing OCI Kubernetes to VCN-Native Cluster with Terraform
Your OCI Kubernetes Cluster might have a little tool tip which states “migration required”. This is because, “in earlier releases (before March 16, 2021), Container Engine for Kubernetes provisioned clusters with Kubernetes API endpoints that were not integrated into your…
Using pushdataset in PowerBI to create near real time logging dashboard
Recently i participated in a hackerthon, in which the goal was to create a near real time monitoring dashboard using Microsoft PowerBI. The data was already generated and persisted in SQLServer and needed to be queried efficiently. Since i am…
Reminder to update statistics
The other day we had a moderate complex query which involved around 270000 rows but run for over an hour. After updating the statistics the query finished in only 4 seconds.
Kubernetes pod stuck in pending status. Nodes had no available volume
A deployed pod is stuck in pending status. Looking deeper into the pod a warning appears: 0/3 nodes are available: 3 node(s) had no available volume zone.
Airflow tasks do not run at specified time as scheduled
We observed a problem where dags did not run at the specified time at all but consistently started at a random time. Let’s dig into it.
pgadmin in Kubernetes – mount Permission denied
If you want to mount a PVC to your k8 cluster to persist data of your pgadmin deployment, you will likely get a mount permission denied error.
Null values and their pitfalls
Learning: Best to not permit NULL values in columns which are used for comparisons
Passing comma separated String to function / stored procedure to use with IN condition
[..] passing comma separated variables into a function does not seem to allow the use of the variable directly in the IN condition.
Add leading zeroes to time
When a time is saved as integer the leading zeroes will not be saved.
