May I ask if Data Warehouse is the place to store the data after passing the ETL process or does it contain the data during the ETL process too?
When I searched on the internet some sites said that Data Warehouse is the place to contain the Data after the ETL process (Data after processing, ready to Analysis), while some sites said that Data during the ETL process is also stored in Data Warehouse (Data is currently processing).
the way i choose to run my projects are that i have work databases that contains all the process and the resultant curated data, then the "data repository" is a separate data only database where the production ready data is copied after it has been certified production ready. it is a matter of choice, but i prefer the separation of work and production ready data.
Out on HF and heard nobody, but didn't call CQ? Nobody heard you either. 73 de N7GH
ETL stands for Extract, Transform, Load. Normally ETL acts as a pipeline process, where data is extracted from some source(s), may be validated, normalized or otherwise transformed (including lookup up table codes), and ultimately stored in a destination(s).
ETL can work with spreadsheets, databases, Amazon S3 storage or just about any other persistent data repository you can think of for both sources and destinations.
So, no, ETL would not normally consider a data warehouse to be an intermediate stage in an ETL process. In fact, it would more likely to be a source and/or destination. You might have a separate ETL process that later extracts from that warehouse for loading into another destination, but not simply as a mid-point in a single run.
The secret of how to be miserable is to constantly expect things are going to happen the way that they are "supposed" to happen.
You can have faith, which carries the understanding that you may be disappointed. Then there's being a willfully-blind idiot, which virtually guarantees it.
It is difficult to free fools from the chains they revere - Voltaire. tiny ad:
a bit of art, as a gift, the permaculture playing cards