Big data in practice: Hadoop and Spark, data lakes, streaming and processing data at a scale where the usual tools stop working.
This blog post describes how to create a hosted Cloudera repository and use it in your Cloudera Manager deployment. The first step is to install a web server, which will host RPM packages and repodata. The common way, is to use an Apache web server. Installing Apache HTTPD service [cdhtest@edge ]$ sudo yum install httpd -y Starting Apache HTTPD…
Let’s assume we need to create an HDFS home directory for a user named “dbitest”. We need first to verify if the user exists on the local filesystem. It’s important to understand that HDFS is mapping users from the local filesystem. [cdhtest@master ~]$ cat /etc/passwd | grep dbitest Create a user on the local file system When the user is…
After the deployment step with Terraform and the configuration/installation with Ansible, we will continue the installation of our Cloudera cluster with Cloudera Manager. By following the below steps you will see how to install CDH on our hosts using Cloudera Manager. Connection First, Login to Cloudera manager URL. When you connect to C.M for the first time, you need to…
In this part of the blog posts series, we will show how ansible helps us to configure our cluster and install all pre-requisite needed for Cloudera Manager. Ansible is one of the most important automation tools currently. Ansible will help us to configure all nodes for a manual installation using Cloudera Manager. Our playbook will contain the following roles: cm_repo:…
Deploying a Cloudera distribution of Hadoop automatically is very interesting in terms of time-saving. Infrastructure as Code tools such as Ansible, Puppet, Chef, Terraform, allow now to provision, manage and deploy configuration for large clusters. In this blog posts series, we will see how to deploy and install a CDH cluster with Terraform and Ansible in the Azure cloud. The…
In this blog post, we will see how to manage Oracle Public Cloud Big Data service Compute Edition with REST API. Scheduling the start/stop/restart of a metered PaaS in the Oracle Cloud can be interesting for managing efficiently the consumption of your cloud credits. We should first have a look at the official documentation so as to understand what the API is composed…
Since few years, Oracle decided to move forward in the Big Data area, as their main competitor. The goal of this blog post is to explain you, how the Oracle Big Data offering is composed. As the Oracle Big Data offering is continuously improving, I’m always open to your feedback 🙂 Oracle Big Data offering is split in 2 parts:…
