Usage of loops and conditions is unavoidable in both application development and data processing. However, for a data engineer, this becomes a nightmare if not handled correctly. I will not be deep diving into how loops are handled in Databricks. However, in short, the problem is that, the same block of code is processed one… Continue reading