+1 vote
in Hadoop by
How to copy a file into HDFS with a different block size to that of existing block size configuration?

1 Answer

0 votes
by

How to copy a file into HDFS with a different block size to that of existing block size configuration?

One can copy a file into HDFS with a different block size by using:

–Ddfs.blocksize=block_size, where block_size is in bytes.

So, let us explain it with an example:

Suppose, you want to copy a file called test.txt of size, say of 128 MB, into the hdfs. And for this file, you want the block size to be 32MB (33554432 Bytes) in place of the default (128 MB). So, you would issue the following command:

[php]Hadoop fs –Ddfs.blocksize=33554432 –copyFromlocal/home/dataflair/test.txt/sample_hdfs[/php]

Now, you can check the HDFS block size associated with this file by:

[php]hadoop fs –stat %o/sample_hdfs/test.txt[/php]

Else, you can also use the NameNode web UI for seeing the HDFS directory.

Related questions

0 votes
asked Jun 7, 2020 in Hadoop by Robindeniel
0 votes
asked Nov 7, 2020 in Hadoop by SakshiSharma
...