Comments for Data into results

Comment on Machine learning vs simulation by Sébastien Derivaux

Sébastien Derivaux — Fri, 30 Oct 2009 23:50:44 +0000

I agree on your comment but it is only when you choose the method before applying it and not trying other methods. Usually, I pick many methods and try all choosing the best or the best combination. Sometimes it is not the method I would choose by “domain knowledge” which is the best. For unsupervised classification, the choice rely more on knowledge than evaluation (usually not possible).

Thanks for your PS.

Comment on Machine learning vs simulation by Steffen

Steffen — Thu, 29 Oct 2009 17:14:35 +0000

Strictly speaking, the usage of a specific machine learning method already adds domain knowledge (e.g. in a specific area some methods are more suitable than others).

I think an example for the merging of both “tools” is the EM-Algorithm (read: expectation maximization) for e.g. estimating the parameters of a multi-gaussian-distribution.

kind regards,

Steffen

PS: I like your style of writing and the topics you are covering. I am looking forward to read more posts (more often ) in the future.

Comment on Using MySQL as a Data Warehouse by Sébastien Derivaux

Sébastien Derivaux — Wed, 28 Oct 2009 22:36:16 +0000

Yes for the index.

Not easy to managed so many engines when no one is perfect and still no column oriented storage engine in the main Mysql version.

Comment on Data mining tools by Sébastien Derivaux

Sébastien Derivaux — Wed, 28 Oct 2009 22:33:16 +0000

@Steffen I was not having the chance to use Rapidminer. Of course it could be interresting as it become most popular than Weka.

@dodo data preparation is of course a big topic. I’m using Talend to do this job, very efficient and easy (and you could add Java in it). Most ETL package will work.

Comment on Data mining tools by Steffen

Steffen — Thu, 24 Sep 2009 11:27:36 +0000

@dodo

I disagree for two reasons:
- I cannot count the algorithms I can give you a understandable description of in 1 minute, but when it comes to a real data analysis you will meet special cases where you have to know EXACTLY how this algorithm is implemented. That is the reason I could never work with non-open source programs
- If you are not able to write code (at least for changing the behavior of present algorithms or create new ones) you restrict yourself to use only what’s available. Are you sure your data mining environment is prepared for every possible data analysis problem ?

@tools: You forgot RapidMiner (former Yale) which does an excellent job in handling large datasets and data preparation (its key focus). It is free, it is open source and it is written in java.

@this blog: This is my first visit and I really like it. Design, content … sweet and technical, that is how I like it.

Comment on Using MySQL as a Data Warehouse by dodo

dodo — Wed, 23 Sep 2009 11:26:00 +0000

Nice article, thanks for it!
Looks like storage engine is an another dimension in MySQL DWH design
Have you created indexes for innodb in “innodb memory memory 28s” ?

Comment on Data mining tools by dodo

dodo — Wed, 23 Sep 2009 10:11:28 +0000

Hi,
what I see as the biggest challenge in DM is data-preparation. That’s not task for Weka, not for Excel and I’m not familiar with R.

SAS Base does the great job there. SPSS Modeler as well. SPSS Statistic trial is available at http://www.spss.com.

Java is cool. But you are wasting time with programming. Data-miner has more important task to do than generating tons of code.
Doesn’t (s)he?

Comment on How to : What to do when your model fails? by dodo

dodo — Wed, 23 Sep 2009 10:03:25 +0000

“… use genetic programming … You could look at GenIQ pages for a description. 99.9% of the time I think it will not work”
Good point! Just what would people behind the GenIQ say

Comment on About me by Sandro

Sandro — Fri, 11 Sep 2009 10:36:20 +0000

Do you have a RSS please?
Thanks and nice blog btw…
Regards.
Sandro.

Comment on How to : What to do when your model fails? by Sébastien Derivaux

Sébastien Derivaux — Thu, 30 Jul 2009 21:17:54 +0000

Thanks for your comment. You are of course right, but I don’t know any statistical test which tell you can’t do significantly better. Maybe thinking of an unexpected attribute could bring new insights on the business.
It’s not an easy decision to say stop. It will always be too soon or to late.
If you have many to problems to solve, a breadth first strategy should be better than a depth first one. Find a first solution for each, then look for something better.