Data in Vatico: Shift from ETL to ELT

Have you ever wondered how a company transforms raw data into beautiful dashboards? Behind the scenes, there was actually some data engineering involved to make raw information meaningful and interpretable. Today’s post will dive into the intricacies of data engineering in Vatico. Specifically, this essay will examine Vatico’s shift from an Extract-Transform-Load (ETL) approach to an Extract-Load-Transform (ELT) approach, and the reasons behind this shift.

Vatico in the Past: ETL approach

As can be seen in Figure 1, ETL is an approach in data engineering where data is extracted from a range of sources, subsequently transformed so that it is usable, and lastly loaded to a repository to analyze and visualize the data.

In the past, Vatico only had one business unit. As a small company, its data needs are relatively straightforward. It thus makes sense to clean and curate the data, and subsequently load them to a repository. The benefit is that whenever data needs to be used, it is ready and servable. 

Figure 1: Extract-Transform-Load Approach

However, there are two drawbacks of this approach.

Vatico in the Present: ELT approach 

Now that Vatico has expanded to three business units, its data needs are more varied. It makes more sense to load the extracted data into a data warehouse, and subsequently transform them, as can be seen in Figure 2.  

Figure 2: Extract-Load-Transform Approach

There are two strengths of this approach.

Data is the heart of Vatico and the ELT approach plays a crucial role in empowering data storytelling. In the future, there might be better data pipelines which are better able to support the data needs in Vatico. We will adapt accordingly depending on the tools available and the ever-changing business priorities.