About the book

Are you an owner of a company? Are you a Director, manager or anyone in your company that want to make changes in your company?

Or are you only following the Gartner Top 10 priorities of the CEOs for 2012, 2013, 2014 and 2015: you have to invest in Business Intelligence and Analytics?

If your answer is yes, you choose the right book.

In this book, we will talk about Pentaho and how to implement it on your company paying almost nothing. You will run a Ferrari paying for a regular bike.

That’s true. In 2007 I started my work with Pentaho and started helping companies, also individuals, to use Pentaho, the results were amazing and, every year, more companies begin to use it and trust on it.

Although the selection of the right software is relevant, BI requires more than this.

It has been a long journey helping a lot of people to use Pentaho and implement it. However sometimes the results are great, sometimes the results will be not so good since implement BI involves more than software: it is about people, hardware and software. They all need to work harmoniously.

Why do I love Pentaho?

Because it is open source and it is free!!! I remember when I went to Africa in order to to help a big ONG. They were doing an amazing work in Mozambique and they were starting to use Pentaho. I stayed there for some days teaching regular and advanced courses and always giving some consulting about how to use Pentaho and how to use it according to their needs. It was rewarding to see how powerful is an open source and free software, it has no boundaries and it changes the environment.

Afterwards this great experience in Africa, I talked in an event with Julian Hyde and also in another event with Matt Castters about the same subject… Do you have any idea about the impact you guys did in the world because you guys created Mondrian and Kettle?

For me Kettle/PDI and Mondrian are astonishing. They are the core of Pentaho and all the rest are complement for these two pieces that are all free and open source

During these years, since 2007, I have been implementing Pentaho everywhere and also givinga lot of training to all kind of students.The magic about the software is that everyone can download it and start creating incredible things.

Before I work with Pentaho I was working with Business Intelligence, but in the proprietary world, with huge, big companies as clients. The day I tried Pentaho for the first time it was the first version, the 1.2 version, and it was horrible, it was so hard to install and almost impossible to find documentation about it.

My first project in a production environment was with the Pentaho 1.6GA, it was the next release after the horrible 1.2 version, if you do not believe me, please, you still can download it from Sourceforge and try it.

I am very happy that in my first real project, for a real client, I found the Pentaho Community and made some international friends, specially Pedro Alves who in those days was working in a company in Lisboa, Portugal. He started helping me and we started changing e-mails and skype. Then, one day he said that he was opening a company to work alone. Afterwards we were working together, not as a community friends anymore, we were working together, in a real project, with real money and he was receiving to help us to implement Pentaho in a real and big company.

It was a remarkable experience, the software was so unfinished that even an admin panel to create users and roles we had to pay Pedro Alves to build it, from zero.

To create an OLAP Cube we had to use the Cube OLAP Wizard, it was discontinued but I think that the actual tool to create cubes direct in the PUC (Pentaho User Console) is based on this fantastic piece of software that unfortunately disappeared,. Now I cannot find it to download anywhere, but I also have a copy of it on my hard drive.

In the 1.6GA version you did not have a good way to manage your users and roles, it was very hard to install, to create a cube or a cube vision you had to create a xaction and there was no documentations. To create a Dashboard you had to create a JSP and I can list a lot of things that did not exist in those days.

Why Am I telling you all these things? Because I want you to know the kind of software I believed in 2007 and invested and dedicated a lot of years and hours of my life.

It says that to become a specialist in something you need at least 10.000 hours on it, I can say that I have much more than 10.000 hours on Pentaho but I am still learning a lot of things about it.

After all this history you probably want to give up and buy something, if you want to do so go ahead, or I mean, stop!!! First you need to try Pentaho, everyone that tried it will stay with Pentaho forever or at least with the Kettle.

For sure, Kettle or, as they call now, PDI (Pentaho Data Integration) will change your life and your company, so please try it.

The good thing is that today we have the Pentaho 5.4 version and we are almost in the 6.0 version, so a lot of things change for better.

Now it is so much easy to work with Pentaho that you will probably enjoy a lot.

In resume, yes you can implement the Pentaho CE or EE version and you will love it.

Who Am I?

My name is Caio, I am an expert in Decision Systems based on Open Source, whichmeans that I worked with Business Intelligence, Analytics, Data Warehouse, ETL, Data Preparation, Data Cleaning, Data Mining, Open Source, Visualization, Machine Learning, Predictive Models, Big Data and a I use a lot of open source tools to create real solutions able to help Executives on taking better decisions.

I live in Madrid, Spain but I was born in São Paulo, Brazil in October, 5, 1983. My dad is from Porto Alegre, Brazil and my mom from Madrid, Spain. Being a Brazilian and Spanish I had the opportunity to take advantage of both countries (Brazil and Spain) and both continents (Europe and South America).

I sold my first website to a friend when I was 14 years old but I started working with computers professionally when I was 16 years old, I was a web programmer in a start-up company that raised some millions of dollars to create an internet company. It happened during the bubble period when everybody was trying to create and invest in internet company. This experience helped me shape my vision of the internet industry.

When I was 18 years old I worked as an Intern at AOL (America Online) in the Loyalty/Marketing Department and I also worked direct with the AOL Brazil Vice President. I learned in 11 months things that I use in my daily day until now, what a blessing to be an intern in such a great company like AOL. It was like working at Google in our days. In fact, in that year Google did not exist yet, they founded Google on September, 4, 1998.

I had to leave AOL to serve as a missionary for The Church of Jesus Christ of Letter Day Saints in the South of Brazil during two years. During this sacred moment in my life I learned some lessons that I still carry with me everyday and which shaped my life forever. It was a hard decision to leave my job, family, friends, car, house, city and everything that I had to be able to serve as a missionary. But I am sure that I did the right thing and everyday I am thankful to had taken this decision when I was so young and immature, it was an act of faith and courage that changed my life and I am happy for this decision.

I was raised in São Paulo, the largest city in South America and a terrific IT place to be. I am the first son of three children, my parents are wonderful and they tried their best to help me understand that life is about attitude and we have to do our best always.

My parents always gave me the vision that I had to study and work for myself and try to be independent in the material and spiritual things. After my mission I worked in a Brazilian factory as the IT guy, I was responsible for managing all the IT of the factory, the internet, the infrastructure, the ERP, IP Cameras, etc. I decided to implement SugarCRM, an open source CRM Tool, in order to help the Sales Team manage their sales process in a better way.

Because of this implementation I worked with the first and principal SugarCRM partner company in Brazil and learned a lot about the CRM business, having the opportunity to implement SugarCRM in a bank and in some small, medium and big companies.

Because this CRM experience I started working with Business Intelligence as a consultant in large companies and I had the opportunity to work with solutions like Microstrategy, IBM Cognos, Info Seagate, Microsoft Business Intelligence and Pentaho.

Everything changed when I implemented Pentaho in a real and big company, after that, the rest is just history and I prove myself that it was possible to dream about BI for all.

In the beginning it was hard to believe, but today Pentaho is reality and everybody use it. The big moment of Pentaho happened this year when Hitachi HDS bought them for a lot of money.

IoT (Internet of Things) is reality, Big Data is reality, Analytics is reality, Open Source is reality, Predictive Models is reality, Data Preparation and Integration is reality and Pentaho is the correct solution for all of it.

Sorry, However I am a Pentaho Sales Man, I am also the kind of person that like to share with others what I believe: I truly believe that Pentaho is a terrific solution.

During my career I also studied to be a bachelor in Technology and Digital Media in PUC-SP, I have an specialization in Software Development with Java Technology, M.B.A in FGV-SP, Dimensional Modeling in Depth with Ralph Kimball and Margy Ross in Chicago, USA, Harvard, I.E and now I am attending a master in Data Mining and Business Intelligence at UCM (Madrid), an expert program in Data Science at U-TAD (Madrid) and a master in Artificial Intelligence at UPM (Madrid).

Moreover, lately I also had time to get married on July, 26, 2008,to have a daughter in March, 13, 2013, to travel to more than 15 countries and x continents,to live in cities like São Paulo, Curitiba, Vancouver, Vigo, Zaragoza, Madrid and to be a backpacker when I was 22 years old. My dream is to travel around the world teaching people about what I learned and impacting different organizations. .

I am sure that Open Source, Technology and Analytics can change the world and that is my goal and purpose when trying to impact the environment around me with my job.It not just about money, it is about contributing with a better world.

During these years, I feel very happy when people send me e-mails saying thank you for my work with Pentahon thank you for the posts in my blogs, tutorials, free online videos, Pentaho’s translation from English to Portuguese, plugins, events that I helped to organize, talks that I gave and other collaborations.

I know that I am far away from my goal of impacting the world, nevertheless I can say that until now I already fulfilled some of my dreams, since I had relevant opportunities to work with companies that definitely have the power of changing the world using the knowledge of Open Source Technology and creating Decision Systems able to influence their state, city or local community. Examples of these companies are: Ministério da Saúde (Healthy Ministry), ANS, Hamburg SUD, Petrobras, Vale, Caixa Economica Federal, Banco do Brasil (Brazil Bank), CREMESP, colocar todas aqui ...

Considering more than 8 years dedicated to Pentaho, I decided to write a book in English. Therefore people from everywhere, every nation can read it and take advantage of this powerful solution and, maybe, it can change your life.

My book is focused only in open source and free software, I mean, I will just talk about free options. Hence, people and companies can start spending almost nothing.

How to start a project?

Everything starts with a need, or a pain. Then it becomes a desire and after that you have a real demand.

Let’s talk about all of them.

Needs or Pain

You are the owner of your company but you do not have all the information you need to make decisions.

That’s a pain, you feel bad about it, and sometimes you are not sleeping because of this.

Maybe you are the Director of a company and you are totally dependent of an excel guy to have your reports on time.

Or during meeting different people present the same business process, using diverse numbers and you do not know which number is correct, actually, nobody knows it.

Otherwise, you have more than one system generating reports and you want to integrate them all in a Dashboard in order to save time and provide confidence in the results and processes.

In resume, you have a business question and you want to solve it.

Desire

After you discover that you have a pain or a need you, automatically, start looking for a solution or a medicine for you.

However the medicine sometimes already exists in the Decision System World, it is frequently very expensive. So you have the first perception: the price is not in accordance with your budget.

Even Big companies are trying to save money,doing more with less.

So, you start to realize that you have no option and this pain will kill you. On the other hand we all find a way to solve our problem and sometimes we will accept a solution as a temporary solution and go on.

Demand

After we have money to invest we will change our pain/need and desire in a demand. In this moment we will buy a medicine for our sickness.

Pain > Desire > Demand

The purpose of this book is to help you find a solution for your budget and business.

In the real world

Let’s try to bring it to the real world: a CEO calls the IT Department and asks them to find a solution for the “pain” of not having information for the company.

The IT Department call every vendor and start a survey about prices, services and a lot of information about the software you can buy in the market.

This is the first big mistake, because with all this effort you will just create an excel matrix with the companies and maybe the prices to buy their solutions, but it could take months to figure all this information out and, as time is money, why are you spending all this money with this?

I know companies that spent millions of dollars buying licenses of business intelligence tools, databases, servers and they did not use it.

So, why the Software is not so important? What is more important than the software?

The answer is the USER.

So, first step to start a project is to talk with the user about his pain or, in other words, you have to interview the user and say:

What is your business question? What is the one million dollar question? Or in some companies what is the one billion dollar question?

In what question do you need our help to make the company bigger and more profitable?

Unfortunately, usually who answers this question is not the right person, consequently tons of projects will fail.

The user is the secret.

If you want to do a real intelligence system you need an intelligent user.

Bad questions = Bad answers = Bad projects

Let me give some examples of business questions:

What client will leave me next month?

Who are my clients that only bought in my store once? twice? More than 5 times?

Where are my clients? How old are they?

What is my best product or service in terms of sales, profit, quantity?

Who are my clients?

Who is my best client?

What is my main supplier?

Can we detect fraud in our sales?

There are a lot of questions that business users ask themselves everyday and IT is probably not answering it. They are probably using excel and an intern to generate all this information and it is probably wrong.

I worked in some projects of migrating all the excel reports to Pentaho and in the end we discovered that some of the reports that they were using for years were wrong.

So, ask yourself.

How much money are you losing because you are not investing on Decision Systems? Is it really a luxury system or not?

Big companies already discovered that it is not luxury it is a basic need and that’s the reason why they are hiring people or companies to create BI Systems.

But remember, your system is as smart as you are. Your system will be as smart as your user, so sometimes you will have to change the user and not the system.

Talking about excellent users, in my life I had the privilege to work directly with Presidents, Vice Presidents, Directors, Managers and owners of all kind of companies and some of them are excellent in asking the right question. When the right question is asked from the right person the project can fly and everybody can get excellent results.

In my opinion, a good project is when my user gets a promotion because all the useful job he did with the company information, since he transformed raw data in information and then in knowledge and he acted and made the company better.

In the information age is all about having the right question, in the right time and the answer in the right moment.

So, in this part of the book you have to stop and do an exercise.

Exercise 1 - Documentation of the Pain

In a paper write the questions below and answer it:

a) Define your pain/need or, with the real user, define his pain/needs

b) Write all the business questions you want to answer

c) Describe the profile of the main user and try to write about the other possible users of this system

d) How much money do you have to invest?

e) How much time do you need? When do you need this solution ready?

f) What resources do you have? Do they have the skills for the project?

g) What are the results you expect to have? How are you going to measure the ROI of this project?

h) Is it your first BI project? Is it the first BI project of your company?

i) Can you fail? Is it a lab project?

j) Who is the project sponsor?

l) Why do you need it? Try to convince yourself or you company that you really need it

m) Do you have the raw data? Where is it? Try to map the source data and the people who are specialists in the source data and try to see if they will help you to get the data if you need it.

You can write more questions and answer, but try to focus and do not thing about the tool yet.

Maybe you will not be able to answer all these questions, but do your part and delegate to other person to finish it, but please use the best resources you have, as much intelligent is your team will be your system, try to use the best you can get. It is all about brains and not muscle.

Congratulations!!! You have created a small documentation about your project and now you are prepared to create your decision system.

If you do not have a business specialist in your team, I highly recommend to hire one . For example, if you are a retail company why not to hire a consultant to help you define your pains, your KPI’s and all the things you need to measure?

You have to always remember that "If you can not measure it, you can not improve it."

During my journey I worked with some business specialists who helped a lot., Usually they were CEOs or specialists in that particular field and they know what to measure and keep track of it.

I mentioned the term KPI and maybe you do not know it. It means Key Performance Indicator or Indicador Chave de Desempenho.

We will talk about it later, at this moment you just have to know it.

I mentioned the term KPI and maybe you do not know it. It means Key Performance Indicator.

We will talk about it later, at this moment you just have to know it.

With this book I hope to present the main characteristics of a BI project, helping to develop it using Pentaho, in a simple and easy way.