Apache Spark

Input—per 1M tokens

Output—per 1M tokens

Context—tokens

WeightsClosed

About

Apache Spark is ranked #8 of 21 in stream processing software on Inferse. It runs on API, Linux, macOS, Self-hosted, Windows. There is a free plan.

Compared on stream processing software

Free plan: Yesspark.apache.org
Deployment: self_hostedspark.apache.org
Transformation mode: mixedspark.apache.org
Incremental loading: Yesspark.apache.org
Change data capture: Yesspark.apache.org
Custom code transforms: Yesspark.apache.org

Facts

Purpose: Apache Spark is a unified engine for large-scale data analytics.spark.apache.org · 30 Sept 2026
Batch and streaming: Spark processes data in batches and real-time streams using Python, SQL, Scala, Java, or R.spark.apache.org · 30 Sept 2026
SQL: Spark SQL executes distributed ANSI SQL queries and supports structured tables and unstructured data such as JSON or images.spark.apache.org · 30 Sept 2026
Machine learning: MLlib provides machine-learning algorithms that can run locally and scale to clusters.spark.apache.org · 30 Sept 2026
Data science: The project says Spark supports exploratory data analysis on petabyte-scale data without downsampling.spark.apache.org · 30 Sept 2026
Integrations: The listed ecosystem includes pandas, TensorFlow, PyTorch, scikit-learn, Apache Kafka, Kubernetes, Delta Lake, and Apache Iceberg.spark.apache.org · 30 Sept 2026
Deployment: Spark documents standalone, Hadoop YARN, and Kubernetes deployment options.spark.apache.org · 30 Sept 2026
Operating systems: Spark runs on Windows and UNIX-like systems, including Linux and macOS, where a supported Java version is available.spark.apache.org · 30 Sept 2026
Client connectivity: Spark Connect separates client applications from Spark clusters and supports remote connectivity.spark.apache.org · 30 Sept 2026
Security: Security features such as authentication are not enabled by default, and the documentation says deployments are not secure by default.spark.apache.org · 30 Sept 2026
Encryption: Spark supports TLS encryption for RPC connections and encryption of temporary data written to local disks.spark.apache.org · 30 Sept 2026
Support: The project directs users to mailing lists and community resources for help.spark.apache.org · 30 Sept 2026
Release and runtime requirements: The 4.2.0 documentation lists Java 17, 21, or 25, Scala 2.13, Python 3.10+, and R 4.0+ (deprecated).spark.apache.org · 30 Sept 2026
License: The project website states Apache Spark is licensed under the Apache License, Version 2.0.spark.apache.org · 30 Sept 2026