hadoop - Different ways to import files into HDFS -


i want know different ways through can bring data hdfs.

i newbie hadoop , java web developer till time. want know if have web application creating log files, how can import log files hdfs.

there lot's of ways on how can ingest data hdfs, let me try illustrate them here:

  1. hdfs dfs -put - simple way insert files local file system hdfs
  2. hdfs java api
  3. sqoop - bringing data to/from databases
  4. flume - streaming files, logs
  5. kafka - distributed queue, near-real time stream processing
  6. nifi - incubating project @ apache moving data hdfs without making lots of changes

best solution bringing web application logs hdfs through flume.


Comments