Question

What are the important steps when you are partitioning table?

Answer 1

Don’t over partition the data with too small partitions, it’s overhead to the namenode. if dynamic partition, atleast one static partition should exist and set to strict mode by using given commands.

SET hive.exec.dynamic.partition = true;

SET hive.exec.dynamic.partition.mode = nonstrict;

first load data into non­partitioned table, then load such data into partitioned table. It’s not possible to load data from local to partitioned table. insert overwrite table table_name partition(year) select * from non­partition­table;