0 votes
in HDFS by

What is the problem in having lots of small files in HDFS?

1 Answer

0 votes
by

As we know, the NameNode stores the metadata information regarding file system in the RAM. Therefore, the amount of memory produces a limit to the number of files in my HDFS file system. In other words, too much of files will lead to the generation of too much meta data and storing these meta data in the RAM will become a challenge. As a thumb rule, metadata for a file, block or directory takes 150 bytes.  

Related questions

+1 vote
asked Dec 3, 2020 in Hive by sharadyadav1986
0 votes
0 votes
asked Jun 8, 2020 in HDFS by Robindeniel
...