In those day of huge focus on the Big Data mouvement, it seems that nobody needs a data warehouse anymore but a huge cloud of Hadoop instead. That’s far from the truth. The data warehouse is here to stay as the center of the decision system of any company. Nevertheless, it need to evolve and meet the new challenges. In this post I will introduce a new data warehouse methodology : the modular data warehouse.
Read more…
Usually we use sequence to generate unique identifier for surrogate keys. A sequence is simply a database object that return a number every time you call it. Sadly today I face a limit in the sequence implementation of Greenplum. In the process of finding a solution, I found a way to generate surrogate keys faster than sequence (another way is in this blog).
Read more…
For the end of 2011, many around the blogosphere are forecasting what will be on hype next year. I often read that big data and mobile BI are on the hype. In this article, I will discuss about those technologies. Don’t believe the ads, those technologies aren’t game changer at all.
My last post discuss about SQL queries. Nevertheless, sometimes data came from differents databases. In such cases, it is no longer possible to use SQL. ETL tools, which stands for Extract Transform Load are designed to easilly allow data transformations. I have currently used three tools : Talend, SAP Business Objects Data Integrator and Kettle. I will review them and explain one or two tips I’ve learned using ETL tools.
Read more…
Data manipulation is a big part of a data mining process. Some authors claims it could take 80% of a data mining project. I could only agree. If data comes from the data warehouse it could be a lot faster. If you have to dig (and understand) operational systems or adding some externals data the works takes even more time. Therefore it is of greatest importance to be efficient in data manipulation. Currently I use two way to do this task : big SQL queries or ETL depending on the situation.
Read more…
In those day of huge focus on the Big Data mouvement, it seems that nobody needs a data warehouse anymore but a huge cloud of Hadoop instead. That’s far from the truth. The data warehouse is here to stay as the center of the decision system of any company. Nevertheless, it need to evolve and meet the new challenges. In this post I will introduce a new data warehouse methodology : the modular data warehouse.