Let’s assume we need to create an HDFS home directory for a user named “dbitest”.
We need first to verify if the user exists on the local filesystem. It’s important to understand that HDFS is mapping users from the local filesystem.
[[email protected] ~]$ cat /etc/passwd | grep dbitest
Create a user on the local file system
When the user is not created, we can easily create one with it associated group.
[[email protected] ~]$ sudo groupadd dbitest [[email protected] ~]$ sudo useradd -g dbitest -d/home/dbitest dbitest [[email protected] ~]$ cat /etc/passwd | grep dbitest dbitest:x:1002:1002::/home/dbitest:/bin/bash [[email protected] ~]$
Note that, the user dbitest should be created in all cluster hosts.
Create a directory in HDFS for a new user
Then we can create the directory under /user in HDFS for the new user dbitest. This directory needs to be created using hdfs user as hdfs user is the super user for admin commands.
[[email protected] ~]$ sudo -u hdfs hdfs dfs -mkdir /user/dbitest
Verify the owner for our new directory
[[email protected] ~]$ sudo -u hdfs hdfs dfs -ls /user Found 5 items drwxr-xr-x - hdfs supergroup 0 2018-07-10 10:10 /user/dbitest drwxrwxrwx - mapred hadoop 0 2018-07-10 07:54 /user/history drwxrwxr-t - hive hive 0 2018-07-10 07:55 /user/hive drwxrwxr-x - hue hue 0 2018-07-10 07:55 /user/hue drwxrwxr-x - oozie oozie 0 2018-07-10 07:56 /user/oozie
The new home directory has been created but it’s owned by hdfs user.
Change owner for /user/dbitest directory
Use the below command to change the owner of the new user home directory created.
[[email protected] ~]$ sudo -u hdfs hdfs dfs -chown dbitest:dbitest /user/dbitest
Let’s see if the owner has changed.
[[email protected] ~]$ sudo -u hdfs hdfs dfs -ls /user Found 5 items drwxr-xr-x - dbitest dbitest 0 2018-07-10 10:10 /user/dbitest drwxrwxrwx - mapred hadoop 0 2018-07-10 07:54 /user/history drwxrwxr-t - hive hive 0 2018-07-10 07:55 /user/hive drwxrwxr-x - hue hue 0 2018-07-10 07:55 /user/hue drwxrwxr-x - oozie oozie 0 2018-07-10 07:56 /user/oozie
Change permissions
Change the permissions of the newly created home directory so that no other users can have read, write and execute permissions except the owner.
[[email protected] ~]$ sudo -u hdfs hdfs dfs -chmod 700 /user/dbitest
[[email protected] ~]$ sudo -u hdfs hdfs dfs -ls /user Found 6 items drwxr-xr-x - admins cdhtest 0 2018-07-10 08:56 /user/cdhtest drwx------ - dbitest dbitest 0 2018-07-10 10:10 /user/dbitest drwxrwxrwx - mapred hadoop 0 2018-07-10 07:54 /user/history drwxrwxr-t - hive hive 0 2018-07-10 07:55 /user/hive drwxrwxr-x - hue hue 0 2018-07-10 07:55 /user/hue drwxrwxr-x - oozie oozie 0 2018-07-10 07:56 /user/oozie
Test the user dbitest home directory
We can now test the user home directory creation by uploading data into it without specifying the destination directory. The file will be automatically uploaded to the user’s home directory if no destination is specified.
[[email protected] ~]$ sudo su dbitest
[[email protected] ~]$ hdfs dfs -ls /user/dbitest
[[email protected] ~]$ hdfs dfs -put HelloWorld.txt
[[email protected] ~]$ hdfs dfs -ls /user/dbitest Found 1 items -rw-r--r-- 3 dbitest dbitest 39 2018-07-10 10:30 /user/dbitest/HelloWorld.txt
Your user home directory has been created successfully.