Hadoop Distributed File System

Input—per 1M tokens
Output—per 1M tokens
Context—tokens
WeightsClosed

About

Hadoop Distributed File System is ranked #4 of 29 in distributed file system software on Inferse. It runs on API, Linux, macOS, Self-hosted, Windows. There is a free plan.

Compared on distributed file system software

Deployment model
self_hostedhadoop.apache.org
Data replication
Yeshadoop.apache.org
Multi-site support
Yeshadoop.apache.org
File access protocols
Java API, C API, REST API, HTTP browser, NFS gateway, FS shellhadoop.apache.org
Encryption at rest
Yeshadoop.apache.org

Facts

Purpose
HDFS is the primary distributed storage used by Hadoop applications.hadoop.apache.org · 30 Sept 2026
Architecture
An HDFS cluster primarily consists of a NameNode managing filesystem metadata and DataNodes storing data.hadoop.apache.org · 30 Sept 2026
Fault tolerance
HDFS is highly fault-tolerant and designed to run on low-cost commodity hardware.hadoop.apache.org · 30 Sept 2026
Workload fit
HDFS is designed for streaming access and batch processing with high throughput rather than low latency.hadoop.apache.org · 30 Sept 2026
Scale
HDFS is tuned for gigabyte-to-terabyte files, hundreds of nodes per cluster, and tens of millions of files in one instance.hadoop.apache.org · 30 Sept 2026
Language and platforms
Hadoop is written in Java and supported on all major platforms.hadoop.apache.org · 30 Sept 2026
Administration
HDFS provides shell-like commands, built-in NameNode and DataNode web servers, fsck diagnostics, safemode, and a cluster balancer.hadoop.apache.org · 30 Sept 2026
Permissions
HDFS implements a permissions model sharing much of POSIX and optionally supports POSIX access control lists.hadoop.apache.org · 30 Sept 2026
Authentication
Hadoop supports simple operating-system identity mode and Kerberos authentication mode.hadoop.apache.org · 30 Sept 2026
Encryption
Configured HDFS encryption provides transparent end-to-end encryption for data at rest and in transit using encryption zones and the Hadoop Key Management Server.hadoop.apache.org · 30 Sept 2026
REST integration
The WebHDFS HTTP REST API supports the complete FileSystem and FileContext interfaces for HDFS.hadoop.apache.org · 30 Sept 2026
Storage integrations
The documentation lists Amazon S3, Azure Blob Storage, Azure Data Lake Storage, Tencent COS, and other Hadoop-compatible file systems.hadoop.apache.org · 30 Sept 2026
Downloads and integrity
Apache distributes source and binary tarballs and recommends verifying releases with GPG signatures or SHA-512 checksums.hadoop.apache.org · 30 Sept 2026
Support
Apache Hadoop support is provided through project mailing lists maintained by non-paid volunteers.hadoop.apache.org · 30 Sept 2026
License
Apache Hadoop is licensed under the Apache License 2.0.hadoop.apache.org · 30 Sept 2026

Best Hadoop Distributed File System alternatives

See all 12