How to copy a file into HDFS with a different block size to that of existing block size configuration?
One can copy a file into HDFS with a different block size by using:
–Ddfs.blocksize=block_size, where block_size is in bytes.
So, let us explain it with an example:
Suppose, you want to copy a file called test.txt of size, say of 128 MB, into the hdfs. And for this file, you want the block size to be 32MB (33554432 Bytes) in place of the default (128 MB). So, you would issue the following command:
[php]Hadoop fs –Ddfs.blocksize=33554432 –copyFromlocal/home/dataflair/test.txt/sample_hdfs[/php]
Now, you can check the HDFS block size associated with this file by:
[php]hadoop fs –stat %o/sample_hdfs/test.txt[/php]
Else, you can also use the NameNode web UI for seeing the HDFS directory.