Data Engineering

Airflow tasks do not run at specified time as scheduled

Airflow tasks do not run at specified time as scheduled

Hung ManhDec 9, 20213 min read

We observed a problem where dags did not run at the specified time at all but consistently started at a random time. Let’s dig into it. Expected Behavior: We have a job chain of three dags which are scheduled for…

dummy-img

How to attach block volume to VM instance in OCI

Hung ManhSep 22, 20213 min read

As a lot of cloud providers offer a free tier option of their products now, i wanted to run some tests on the Oracle Cloud Infrastructure (OCI) and encountered some actually trivial problems on how to increase the volume of…

pgadmin in Kubernetes – mount Permission denied

pgadmin in Kubernetes – mount Permission denied

Hung ManhJul 2, 20212 min read

If you are using Docker, or for this matter Kubernetes, you would need to mount a host folder into your container to store your data persistently. For pgadmin, you would like to persistently save the database, which holds all metadata…

Null values and their pitfalls

Null values and their pitfalls

Hung ManhAug 6, 20204 min read

Learning: Best to not permit NULL values in columns which are used for comparisons If you compare a NULL value with any other character the output will always be false which sometimes can lead to confusion since it is not…

Passing comma separated String to function / stored procedure to use with IN condition

Passing comma separated String to function / stored procedure to use with IN condition

Hung ManhApr 28, 20201 min read

where column = any (string_to_array(replace(delimSources,’ ‘,”),’,’)) — replace(source,old, new) => removes whitespace — string_to_array(source,separator) => puts the string separated by seperator in an array — any => equivalent to IN, BUT in contrast to IN it can take an array,…

Add leading zeroes to time

Add leading zeroes to time

Hung ManhApr 24, 20201 min read

SELECT RIGHT(CONCAT(‘000000’,13000),6) When a time is saved as integer the leading zeroes will not be saved. (E.g. 1:30 am => 013000 => 13000). To add the leading zeroes we can use the above snippet. Now we can cast it to…

hungsblog | Nguyen Hung Manh | Dresden
Scroll to Top