In last tutorial ,we discuss about the data warehouse concept and their importance in data world. We also checked some of the key feature is Data warehousing . In below tutorial , we will talk about Data Warehouse Architecture in details .We will discuss about the various steps involves in a data warehouse life cycle.
Data warehouse Architecture
Data warehouse Architecture
1. Data Source
Data Source serve as the input for whole data warehouse. It can be in form of Relational Database , Flat files,XML or any other format. Lets talk about commonly used data sources.
1.1 Operational Data Store : Most common source of data warehouse is Operation database itself , which actually store the present data. It can be spread among different type of servers.It is also termed as OLTP.
1.2 ERP : Enterprise resource planning is a business management software is use to store and manage data from every stage of business, including:
- Product planning, cost and development
- Marketing and sales
- Inventory management
- Shipping and payment
Its purpose is to facilitate the flow of information between all business functions inside the boundaries of the organization and manage the connections to outside stakeholders.
1.3. CRM : Customer relationship management is a system to manage a company’s interactions with customers, clients and sales prospects. It holds the information of customer , like address ,contact , customer type and also store the hierarchy of customer itself.
1.4. Flat Files : Some data can be also given in form of flat files (delimited type or fixed length) .
Once data is given to the system , it is transformed , validated and encrypted in a temporary database . This process is commonly known as ETL process , which involve :-
- Extracting data from outside sources
- Transforming it to fit operational needs (which can include quality levels)
- Loading it into the end target (database or data warehouse)
3. Data warehouse
After ETL process , whole summarized data is stored at a common database. It can be any relational DBs
Check here for What is data warehouse
4. Data Mart
A data mart (DM) is the access layer of the data warehouse (DW) environment that is used to store data related to specific subject (say sales of a product in a origin) and this data is directly given to end user for further reporting.
4.1 OLAP : OLAP (Online Analytical Processing) is a process to provide end users with access to large amounts of data in an intuitive and rapid manner to assist with data based on user’s requirement.
5. Reporting :
Now data can be used for final reporting and it can be view via many reporting tools like Cognos , Business Objects,Qlikview. Final report can be accessed via mail or some web services also.
5.1 Data Mining: Data mining is the process of extracting patterns based on large data sets (over some time origin) by combining methods from statistics and artificial intelligence with database management.
So in above tutorial , we went through the basic architecture and component of Data warehouse project. We also see the life cycle of Data warehouse project.